CV and GCV
The smoothers discussed here have a single smoothing parameter. In choosing the smoothing parameter, cross validation can be used. Cross validation works by leaving points
out one at a time, estimating the squared residual for smooth function at
based on the remaining
data points, and choosing the smoother to minimize the sum of those squared residuals. This mimics the use of training and test samples for prediction. The cross validation function is defined as
where
indicates the fit at
, computed by leaving out the
th data point. The quantity
is sometimes called the prediction sum of squares, or
(Allen 1974).
All of the smoothers fit by the GAM procedure can be formulated as a linear combination of the sample responses
for some matrix
, which depends on
. (The matrix
depends on
and the sample data as well, but this dependence is suppressed in the preceding equation.) Let
be the
th diagonal element of
. Then the
function can be expressed as
In most cases, it is very time-consuming to compute the quantity
individually. To solve this computational problem, Wahba (1990) has proposed the generalized cross validation function (
) that can be used to solve a wide variety of problems involving selection of a parameter to minimize the prediction risk.
The
function is defined as
The
formula simply replaces the
with
. Therefore, it can be viewed as a weighted version of
. In most of the cases of interest,
is closely related to
but much easier to compute. Specify the METHOD=GCV option in the MODEL statement in order to use the
function to choose the smoothing parameters.
Degrees of Freedom
The estimated GAM model can be expressed as
Because the weights are calculated based on previous iteration during the local scoring iteration, the matrices
might depend on
for non-Gaussian data. However, for the final iteration, the
matrix for the spline smoothers has the same role as the projection matrix in linear regression; therefore, nonparametric degrees of freedom (DF) for the
th spline smoother can be defined as
For loess smoothers
is not symmetric and so is not a projection matrix. In this case PROC GAM uses
The GAM procedure gives you the option of specifying the degrees of freedom for each individual smoothing component. If you choose a particular value for the degrees of freedom, then during every local scoring iteration the procedure will search for a corresponding smoothing parameter lambda that yields the specified value or comes as close as possible. The final estimate for the smoother during this local scoring iteration will be based on this lambda. Note that for univariate spline and loess components, an additional degree of freedom is used by default to account for the linear portion of the model, so the value displayed in the “Fit Summary” and “Analysis of Deviance” tables will be one less than the value you specify.