Example 29.2 Crude Birth and Death Rates

This example uses the SAS data set Poverty created in the section Getting Started: CLUSTER Procedure. The data, from Rouncefield (1995), are birth rates, death rates, and infant death rates for 97 countries. Six cluster analyses are performed with eight methods. Scatter plots showing cluster membership at selected levels are produced instead of tree diagrams.

Each cluster analysis is performed by a macro called ANALYZE. The macro takes two arguments. The first, &METHOD, specifies the value of the METHOD= option to be used in the PROC CLUSTER statement. The second, &NCL, must be specified as a list of integers, separated by blanks, indicating the number of clusters desired in each scatter plot. For example, the first invocation of ANALYZE specifies the AVERAGE method and requests plots of 3 and 8 clusters. When two-stage density linkage is used, the K= and R= options are specified as part of the first argument.

The ANALYZE macro first invokes the CLUSTER procedure with METHOD=&METHOD, where &METHOD represents the value of the first argument to ANALYZE. This part of the macro produces the PROC CLUSTER output shown.

The %DO loop processes &NCL, the list of numbers of clusters to plot. The macro variable &K is a counter that indexes the numbers within &NCL. The %SCAN function picks out the &Kth number in &NCL, which is then assigned to the macro variable &N. When &K exceeds the number of numbers in &NCL, %SCAN returns a null string. Thus, the %DO loop executes while &N is not equal to a null string. In the %WHILE condition, a null string is indicated by the absence of any nonblank characters between the comparison operator (NE) and the right parenthesis that terminates the condition.

Within the %DO loop, the TREE procedure creates an output data set containing &N clusters. The SGPLOT procedure then produces a scatter plot in which each observation is identified by the number of the cluster to which it belongs. The TITLE2 statement uses double quotes so that &N and &METHOD can be used within the title. At the end of the loop, &K is incremented by 1, and the next number is extracted from &NCL by %SCAN.

title 'Cluster Analysis of Birth and Death Rates';
ods graphics on;

%macro analyze(method,ncl);
   proc cluster data=poverty outtree=tree method=&method print=15 ccc pseudo;
      var birth death;
      title2;
   run;

   %let k=1;
   %let n=%scan(&ncl,&k);
   %do %while(&n NE);

      proc tree data=tree noprint out=out ncl=&n;
         copy birth death;
      run;

      proc sgplot;
         scatter y=death x=birth / group=cluster;
         title2 "Plot of &n Clusters from METHOD=&METHOD";
      run;

      %let k=%eval(&k+1);
      %let n=%scan(&ncl,&k);
   %end;
%mend;

The following statement produces Output 29.2.1, Output 29.2.3, and Output 29.2.4:

%analyze(average, 3 8)

For average linkage, the CCC has peaks at 3, 8, 10, and 12 clusters, but the 3-cluster peak is lower than the 8-cluster peak. The pseudo statistic has peaks at 3, 8, and 12 clusters. The pseudo statistic drops sharply at 3 clusters, continues to fall at 4 clusters, and has a particularly low value at 12 clusters. However, there are not enough data to seriously consider as many as 12 clusters. Scatter plots are given for 3 and 8 clusters. The results are shown in Output 29.2.1 through Output 29.2.4. In Output 29.2.4, the eighth cluster consists of the two outlying observations, Mexico and Korea.

Output 29.2.2 Criteria for the Number of Clusters: METHOD=AVERAGE

Output 29.2.3 Plot of Three Clusters: METHOD=AVERAGE

Output 29.2.4 Plot of Eight Clusters: METHOD=AVERAGE

The following statement produces Output 29.2.5 and Output 29.2.7:

%analyze(complete, 3)

Complete linkage shows CCC peaks at 3, 8 and 12 clusters. The pseudo statistic peaks at 3 and 12 clusters. The pseudo statistic indicates 3 clusters.

The scatter plot for 3 clusters is shown.

Output 29.2.6 Criteria for the Number of Clusters: METHOD=COMPLETE

Output 29.2.7 Plot of Clusters for METHOD=COMPLETE

The following statement produces Output 29.2.8 and Output 29.2.10:

%analyze(single, 7 10)

The CCC and pseudo statistics are not appropriate for use with single linkage because of the method’s tendency to chop off tails of distributions. The pseudo statistic can be used by looking for large values and taking the number of clusters to be one greater than the level at which the large pseudo value is displayed. For these data, there are large values at levels 6 and 9, suggesting 7 or 10 clusters.

The scatter plots for 7 and 10 clusters are shown.

Output 29.2.9 Criteria for the Number of Clusters: METHOD=SINGLE

Output 29.2.10 Plot of Clusters for METHOD=SINGLE

The following statements produce Output 29.2.11 through Output 29.2.14, :

%analyze(two k=10, 3)

%analyze(two k=18, 2)

For th-nearest-neighbor density linkage, the number of modes as a function of is as follows (not all of these analyses are shown):

Thus, there is strong evidence of 3 modes and an indication of the possibility of 2 modes. Uniform-kernel density linkage gives similar results. For K=10 (10th-nearest-neighbor density linkage), the scatter plot for 3 clusters is shown; and for K=18, the scatter plot for 2 clusters is shown.

Output 29.2.13 Plot of Clusters for METHOD=TWOSTAGE K=10

Output 29.2.14 Plot of Clusters for METHOD=TWOSTAGE K=18

In summary, most of the clustering methods indicate 3 or 8 clusters. Most methods agree at the 3-cluster level, but at the other levels, there is considerable disagreement about the composition of the clusters. The presence of numerous ties also complicates the analysis; see Example 29.4.