The CANCORR Procedure

Formulas

Assume without loss of generality that the two sets of variables, $\mb{X}$ with p variables and $\mb{Y}$ with q variables, have means of zero. Let n be the number of observations, and let m be $n-1$.

Note that the scales of eigenvectors and canonical coefficients are arbitrary. PROC CANCORR follows the usual procedure of rescaling the canonical coefficients so that each canonical variable has a variance of one.

There are several different sets of formulas that can be used to compute the canonical correlations, $\rho _ i$, $i = 1, \ldots , \min (p,q)$, and unscaled canonical coefficients:

  1. Let $\mb{S}_{XX} = \mb{X}^{\prime }\mb{X}/m$ be the covariance matrix of $\mb{X}$, $\mb{S}_{YY} = \mb{Y}^{\prime }\mb{Y}/m$ be the covariance matrix of $\mb{Y}$, and $\mb{S}_{XY} = \mb{X}^{\prime }\mb{Y}/m$ be the covariance matrix between $\mb{X}$ and $\mb{Y}$. Then the eigenvalues of $\mb{S}_{YY}^{-1}\mb{S}_{XY}^{\prime }\mb{S}_{XX}^{-1}\mb{S}_{XY}$ are the squared canonical correlations, and the right eigenvectors are raw canonical coefficients for the $\mb{Y}$ variables. The eigenvalues of $\mb{S}_{XX}^{-1}\mb{S}_{XY}\mb{S}_{YY}^{-1}\mb{S}_{XY}^{\prime }$ are the squared canonical correlations, and the right eigenvectors are raw canonical coefficients for the $\mb{X}$ variables.

  2. Let $\mb{T} = \mb{Y}^{\prime }\mb{Y}$ and $\mb{H} = \mb{Y}^{\prime }\mb{X}(\mb{X}^{\prime }\mb{X})^{-1}\mb{X}^{\prime }\mb{Y}$. The eigenvalues $\xi _ i$ of $\mb{T}^{-1}\mb{H}$ are the squared canonical correlations, $\rho _ i^2$, and the right eigenvectors are raw canonical coefficients for the $\mb{Y}$ variables. Interchange $\mb{X}$ and $\mb{Y}$ in the preceding formulas, and the eigenvalues remain the same, but the right eigenvectors are raw canonical coefficients for the $\mb{X}$ variables.

  3. Let $\mb{E} = \mb{T}-\mb{H}$. The eigenvalues of $\mb{E}^{-1}\mb{H}$ are $\lambda _ i = \rho _ i^2 / (1-\rho _ i^2)$. The right eigenvectors of $\mb{E}^{-1}\mb{H}$ are the same as the right eigenvectors of $\mb{T}^{-1}\mb{H}$.

  4. Canonical correlation can be viewed as a principal component analysis of the predicted values of one set of variables from a regression on the other set of variables, in the metric of the error covariance matrix. For example, regress the $\mb{Y}$ variables on the $\mb{X}$ variables. Call the predicted values $\mb{P} = \mb{X}(\mb{X}^{\prime }\mb{X})^{-1}\mb{X}^{\prime }\mb{Y}$ and the residuals $\mb{R} = \mb{Y} - \mb{P} = (\mb{I} - \mb{X}(\mb{X}^{\prime }\mb{X})^{-1}\mb{X}^{\prime })\mb{Y}$. The error covariance matrix is $\mb{R}^{\prime }\mb{R}/m$. Choose a transformation $\mb{Q}$ that converts the error covariance matrix to an identity—that is, $(\mb{RQ})^{\prime }(\mb{RQ}) = \mb{Q}^{\prime }\mb{R}^{\prime }\mb{RQ} = m\mb{I}$. Apply the same transformation to the predicted values to yield, say, $\mb{Z} = \mb{PQ}$. Now do a principal component analysis on the covariance matrix of $\mb{Z}$, and you get the eigenvalues of $\mb{E}^{-1}\mb{H}$. Repeat with $\mb{X}$ and $\mb{Y}$ variables interchanged, and you get the same eigenvalues.

    To show this relationship between canonical correlation and principal components, note that $\mb{P}^{\prime }\mb{P} = \mb{H} $, $\mb{R}^{\prime }\mb{R} = \mb{E} $, and $\mb{Q}\mb{Q}^{\prime } = m\mb{E}^{-1}$. Let the covariance matrix of $\mb{Z}$ be $\mb{G}$. Then $\mb{G} = \mb{Z}^{\prime }\mb{Z}/m = (\mb{PQ})^{\prime }\mb{PQ}/m = \mb{Q}^{\prime }\mb{P}^{\prime }\mb{PQ}/m = \mb{Q}^{\prime }\mb{H}\mb{Q}/m $. Let $\mb{u}$ be an eigenvector of $\mb{G}$ and $\kappa $ be the corresponding eigenvalue. Then by definition, $\mb{G}\mb{u} = \kappa \mb{u}$; hence $\mb{Q}^{\prime }\mb{H}\mb{Q}\mb{u}/m = \kappa \mb{u}$. Premultiplying both sides by $\mb{Q}$ yields $\mb{Q}\mb{Q}^{\prime }\mb{H}\mb{Q}\mb{u}/m = \kappa \mb{Q}\mb{u}$ and thus $\mb{E}^{-1}\mb{H}\mb{Q}\mb{u} = \kappa \mb{Q}\mb{u}$. Hence $\mb{Q}\mb{u}$ is an eigenvector of $\mb{E}^{-1}\mb{H}$ and $\kappa $ is also an eigenvalue of $\mb{E}^{-1}\mb{H}$.

  5. If the covariance matrices are replaced by correlation matrices, the preceding formulas yield standardized canonical coefficients instead of raw canonical coefficients.

The formulas for multivariate test statistics are shown in the section Multivariate Tests in Chapter 4: Introduction to Regression Procedures. Formulas for linear regression are provided in other sections of that chapter.