Seonglae ChoPartial Least Squares Regression
Performs "dimension reduction + regression simultaneously"
In other words, the key point is that B is not a simple least-squares solution, but rather a linear transformation based on covariance maximization.
A regression method that simultaneously optimizes the covariance structure between input features and output targets in high-dimensional data with severe Multicollinearity.
X is colinear and high-dimensional, so we cannot directly use as OLS. Therefore, we find the direction with the highest covariance, remove the explained part, and iteratively find the next direction in the residual.
Optimization
- The first latent direction explains part of : where is the remaining residual.
- The next component tries to explain the residual :
- This process repeats iteratively until becomes sufficiently small.
K
Determines how many latent axes to use
