Partial least squares (PLS) works by extracting one factor at a time. Let be the centered and scaled matrix of predictors, and let
be the centered and scaled matrix of response values. The PLS method starts with a linear combination
of the predictors, where
is called a score vector and
is its associated weight vector. The PLS method predicts both
and
by regression on
:
The vectors and
are called the X- and Y-loadings, respectively.
The specific linear combination is the one that has maximum covariance
with some response linear combination
. Another characterization is that the X-weight,
, and the Y-weight,
, are proportional to the first left and right singular vectors, respectively, of the covariance matrix
or, equivalently, the first eigenvectors of
and
, respectively.
This accounts for how the first PLS factor is extracted. The second factor is extracted in the same way by replacing and
with the X- and Y-residuals from the first factor:
These residuals are also called the deflated and
blocks. The process of extracting a score vector and deflating the data matrices is repeated for as many extracted factors
as are wanted.