Seonglae Cho- A product like Id⊗𝑊 (with identity on the left) represents multiplying each position in our context by a matrix.
- A product like 𝐴⊗Id (with identity on the right) represents multiplying across positions.
- A product like 𝐴⊗𝑊 multiplies the vector at each position by 𝑊 and across positions with 𝐴. It doesn't matter which order you do this in.
- The products obey the mixed-product property (𝐴⊗𝐵)⋅(𝐶⊗𝐷)=(𝐴𝐶)⊗(𝐵𝐷).
Kronecker factorization
By factoring matrices into Kronecker products, we can significantly reduce their dimensions, which helps manage the dimensional growth inherent in these operations.
Implementation by Einstein Notation
np.einsum('ij,kl->ikjl', A, B).reshape(A.shape[0] * B.shape[0], A.shape[1] * B.shape[1])