The HPREG Procedure

This example shows how you can use the SCREEN option in the SELECTION statement to greatly speed up model selection from a large number of regressors. In order to demonstrate the efficacy of model selection with screening, this example uses simulated data in which the response y depends systematically on a relatively small subset of a much larger set of regressors, which is described in Table 14.8.

The labels In and Out, which are part of the variable names, make it easy to identify whether the selected model succeeds or fails in capturing the true underlying model. The regressors that are labeled xWeakIn1 and xWeakIn2 are predictive, but their influence is substantially smaller than the influence of the other regressors in the true model.

The following DATA step generates the data:

  %let nObs      = 50000;
  %let nContIn   = 25;
  %let nContOut  = 500;
  %let nClassIn  = 5;
  %let nClassOut = 500;
  %let maxLevs   = 5;
  %let noiseScale= 1;  

   data ex4Data;
     array xIn{&nContIn}; 
     array xOut{&nContOut}; 
     array cIn{&nClassIn}; 
     array cOut{&nClassOut};

     drop i j sign nLevs xBeta;
 
     do i=1 to &nObs;  
        sign  = -1;
        xBeta = 0;
        do j=1 to dim(xIn); 
           xIn{j} = ranuni(1);
           xBeta  = xBeta + j*sign*xIn{j};
           sign   = -sign; 
        end; 
        do j=1 to dim(xOut); 
           xOut{j} = ranuni(1); 
        end;

        xWeakIn1 = ranuni(1);
        xWeakin2 = ranuni(1);

        xBeta  = xBeta + 0.1*xWeakIn1+ 0.1*xWeakIn2;

        do j=1 to dim(cIn); 
           nLevs  = 2 + mod(j,&maxlevs-1);
           cIn{j} = 1+int(ranuni(1)*nLevs);
           xBeta  = xBeta + j*sign*(cIn{j}-nLevs/2); 
           sign   = -sign;  
        end; 

        do j=1 to dim(cOut);
           nLevs  = 2 + mod(j,&maxlevs-1); 
           cOut{j} = 1+int(ranuni(1)*nLevs); 
        end;

        y = xBeta + &noiseScale*rannor(1);
       
        output;  
    end; 
  run;  

When you have insufficient prior knowledge of what effects need to be included in a parsimonious predictive model, a reasonable starting point is to use model selection to build a such a model. In such cases, you might want to consider a large number of possible model effects, even though you know that a successful model that generalizes well for predicting unseen data depends on a relatively small number of effects. In such cases, you can dramatically reduce the computational task by including screening in the model selection process. The following statements show how you do this:

 proc hpreg data=ex4Data;
     class c: ;
     model y = x: c: ;
     selection method=forward screen(details=all)=100 20;
     performance details;
 run;

The ordered pair of integers that is specified in the SCREEN option in the SELECTION statement requests that screening be used to reduce the set of regressors to 100 regressors at the first screening stage and to 20 regressors at the second screening stage. This information is reflected in the "Screening Information" table shown in Output 14.4.1.

The "Number Of Observations" table in Output 14.4.2 confirms that the data contain 50,000 observations and the "Dimensions" table shows that the selection is from 1,033 effects that have a total of 2,295 parameters.

Because you specified the DETAILS=ALL suboption of the SCREEN option, you obtain the "Screening" table in Output 14.4.3, which shows how the screened subset of 100 effects is obtained at the first screening stage. For display purposes, some ranks in this table have been suppressed.

The "Screened Effects" table shown in Output 14.4.4 lists the effects from which a model is selected at the first screening stage.

You see that the magnitude of the pairwise correlations of effects xIn1, xWeakIn1, and xWeakIn2 with response are too small for those effects to be included as candidates for selection at the first screening stage.

The first stage continues with forward selection from the screened effects that are shown in Output 14.4.4. The effects in the selected model at this stage are shown in Output 14.4.5.

You see that the selected model at this stage includes only effects that are systematically related to the response. If you had requested that only a single-stage screening method be used by specifying the SINGLESTAGE suboption of the SCREEN option, then the selected model at this stage would have been the final selected model. However, multistage screening is used in this example. The second stage repeats the steps of the first stage except that the modeled response is the residuals from the selected model at the first stage.

Output 14.4.6 shows the screening details at the second stage. You see that 20 effects are chosen by screening at this stage as specified. Because the selected effects from the first stage are orthogonal to the residuals at the first stage, none of these effects are in the screened subset. Furthermore, you see that although the effects xIn1, xWeakIn1, and xWeakIn2 are weakly correlated with y, they are the most strongly correlated effects with the residuals from the first stage.

Output 14.4.7 shows the selected effects at the second screening stage. You see that the selected effects are precisely the remaining effects that are systematically predictive of y but that were not in the screened subset at the first screening stage.

In the third and final screening stage, model selection is performed from the union of the screened effects from the first stage (which are shown in Output 14.4.4) and the selected effects from the second stage (which are shown in Output 14.4.7). The selected effects from this final stage are shown in Output 14.4.8.

You see that the final selected model contains all the true underlying model effects and just one noise effect (xOut6). Because you specified the DETAILS option in the PERFORMANCE statement, the "Timing" table shown in Output 14.4.9 is displayed.

You see that even though the selected model was obtained by selecting from thousands of effects, screening enabled the entire modeling task to be completed in about 10 seconds. You can perform the same model selection without screening as shown in the following statements:

 proc hpreg data=ex4Data;
     class c: ;
     model y = x: c: ;
     selection method=forward;
     performance details;
 run;

In this case, the model that is selected without screening is identical to model that is obtained with screening. However, there is no guarantee that you will get identical selected models. Output 14.4.10 shows the "Timing" table for the model selection without screening.

You see that the model selection without screening took about 83 seconds, which is substantially slower than the approximately 10 seconds it took when screening was included in the selection process.