Seonglae ChoKronecker-Factored Approximate Curvature
We factorize the Fisher Information Matrix to get the inverse matrix based on Kronecker product.
Kronecker-Factored Eigenbasis (KFE)
Kronecker factorization
By factoring matrices into Kronecker products, we can significantly reduce their dimensions, which helps manage the dimensional growth inherent in these operations.
Eigenvalue Correction
K-FAC uses the Kronecker product of two bases as approximate eigenvector bases for the full Hessian, while EK-FAC corrects eigenvalues by calculating the variance of components obtained by projecting actual pseudo-gradients onto these bases. This is called Eigenvalue Correction and enables hundreds of times faster IHPV computation compared to traditional iterative linear system solvers.