FIM
Def: Variance of Informant. FIM reflects Curvature of probability distribution since informant indicates the sensitivity to changes in parameter values.
Fisher (Information) Matrix
Def: Expectation for Outer Product of score function. If score function (Informant) is vector, Fisher matrix acts likeCovariance Matrix (not identical but equal when average of score function is 0 mathematically).
Empirical Fisher Matrix (EFIM)
Hessian Matrix
When the loss function is negative log-likelihood (NLL) or cross entropy, and optimization has converged using likelihood, the Hessian Matrix of loss function approximates the Fisher Information Matrix.
