PLS Maximizes Covariance
PLS is a method for finding the directions of the maximum shared covariance between two paired views of the data. PCA can be seen as a special case of PLS where the two paired datasets are the same. It driven by within modalities variance in the data (if one view has much higher variance it can dominate the PLS solution).
Find linear combinations in two datasets X and Y to identify maximally correlated (latent) dimensions
The optimization problem can be written in its Lagrange form as the function:
Setting the derivative of the Lagrange Function with respect to and to yields to the following equations:
By substituting the second two conditions into the first two conditions, we find that
We can substitute the second into the first condition (and the first into the second) to obtain:
This is the same as finding the Eigendecomposition of the matrices and . The eigenvectors are and , respectively.
PLS can be made equivalent to CCA by whitening the data matrices prior to the computation of the covariance matrix, as this ensures that and are identity matrices.
PLS can also be expressed as a SVD of .
A common algorithm for solving PLS is the "Nonlinear Iterative Partial Least Squares" (NIPALS) algorithm.