Example 34.2 Outliers

This example involves data artificially generated to contain two clusters and several severe outliers. A preliminary analysis specifies 20 clusters and outputs an OUTSEED= data set to be used for a diagnostic plot. The exact number of initial clusters is not important; similar results could be obtained with 10 or 50 initial clusters. Examination of the plot suggests that clusters with more than five (again, the exact number is not important) observations can yield good seeds for the main analysis. A DATA step deletes clusters with five or fewer observations, and the remaining cluster means provide seeds for the next PROC FASTCLUS analysis.

Two clusters are requested; the LEAST= option specifies the mean absolute deviation criterion (LEAST=1). Values of the LEAST= option less than 2 reduce the effect of outliers on cluster centers.

The next analysis also requests two clusters; the STRICT= option is specified to prevent outliers from distorting the results. The STRICT= value is chosen to be close to the _GAP_ and _RADIUS_ values of the larger clusters in the diagnostic plot; the exact value is not critical.

A final PROC FASTCLUS run assigns the outliers to clusters.

The following SAS statements implement these steps, and the results are displayed in Output 34.2.3 through Output 34.2.8. First, an artificial data set is created with two clusters and some outliers. Then PROC FASTCLUS is run with many clusters to produce an OUTSEED= data set. A diagnostic plot using the variables _GAP_ and _RADIUS_ is then produced using the SGSCATTER procedure. The results from these steps are shown in Output 34.2.1 and Output 34.2.2.

   data x;
   title 'Using PROC FASTCLUS to Analyze Data with Outliers';
      drop n;
      do n=1 to 100;
         x=rannor(12345)+2;
         y=rannor(12345);
         output;
      end;
      do n=1 to 100;
         x=rannor(12345)-2;
         y=rannor(12345);
         output;
      end;
      do n=1 to 10;
         x=10*rannor(12345);
         y=10*rannor(12345);
         output;
      end;
   run;
   
   title2 'Preliminary PROC FASTCLUS Analysis with 20 Clusters';
   proc fastclus data=x outseed=mean1 maxc=20 maxiter=0 summary;
      var x y;
   run;
   
   proc sgscatter data=mean1 ;
      compare y=(_gap_ _radius_) x=_freq_ ;
   run;