Example 23.2 Alternative Multiple Comparison Procedures

The following is a continuation of the first example in the section One-Way Layout with Means Comparisons. You are studying the effect of bacteria on the nitrogen content of red clover plants, and the analysis of variance shows a highly significant effect. The following statements create the data set and compute the analysis of variance as well as Tukey’s multiple comparisons test for pairwise differences between bacteria strains; the results are shown in Figure 23.1, Figure 23.2, and Figure 23.3

title1 'Nitrogen Content of Red Clover Plants';
data Clover;
   input Strain $ Nitrogen @@;
   datalines;
3DOK1  19.4 3DOK1  32.6 3DOK1  27.0 3DOK1  32.1 3DOK1  33.0
3DOK5  17.7 3DOK5  24.8 3DOK5  27.9 3DOK5  25.2 3DOK5  24.3
3DOK4  17.0 3DOK4  19.4 3DOK4   9.1 3DOK4  11.9 3DOK4  15.8
3DOK7  20.7 3DOK7  21.0 3DOK7  20.5 3DOK7  18.8 3DOK7  18.6
3DOK13 14.3 3DOK13 14.4 3DOK13 11.8 3DOK13 11.6 3DOK13 14.2
COMPOS 17.3 COMPOS 19.4 COMPOS 19.1 COMPOS 16.9 COMPOS 20.8
;

proc anova data=Clover;
   class Strain;
   model Nitrogen = Strain;
   means Strain / tukey;
run;

The interactivity of PROC ANOVA enables you to submit further MEANS statements without re-running the entire analysis. For example, the following command requests means of the Strain levels with Duncan’s multiple range test and the Waller-Duncan -ratio test.

   means Strain / duncan waller;
run;

Results of the Waller-Duncan -ratio test are shown in Output 23.2.1.

Output 23.2.1 Waller-Duncan -ratio Test

Nitrogen Content of Red Clover Plants

The ANOVA Procedure

 

Waller-Duncan K-ratio t Test for Nitrogen

Kratio	100
Error Degrees of Freedom	24
Error Mean Square	11.78867
F Value	14.37
Critical Value of t	1.91873
Minimum Significant Difference	4.1665

Means with the same letter are not significantly different.
Waller Grouping		Mean	N	Strain
	A	28.820	5	3DOK1
	B	23.980	5	3DOK5
	B
C	B	19.920	5	3DOK7
C
C	D	18.700	5	COMPOS
	D
E	D	14.640	5	3DOK4
E
E		13.260	5	3DOK13

The Waller-Duncan -ratio test is a multiple range test. Unlike Tukey’s test, this test does not operate on the principle of controlling Type I error. Instead, it compares the Type I and Type II error rates based on Bayesian principles (Steel and Torrie; 1980).

The Waller Grouping column in Output 23.2.1 shows which means are significantly different. From this test, you can conclude the following:

• The mean nitrogen content for strain 3DOK1 is higher than the means for all other strains.

• The mean nitrogen content for strain 3DOK5 is higher than the means for COMPOS, 3DOK4, and 3DOK13.

• The mean nitrogen content for strain 3DOK7 is higher than the means for 3DOK4 and 3DOK13.

• The mean nitrogen content for strain COMPOS is higher than the mean for 3DOK13.

• Differences between all other means are not significant based on this sample size.

Output 23.2.2 shows the results of Duncan’s multiple range test. Duncan’s test is a result-guided test that compares the treatment means while controlling the comparison-wise error rate. You should use this test for planned comparisons only (Steel and Torrie; 1980). The results and conclusions for this example are the same as for the Waller-Duncan -ratio test. This is not always the case.

Tukey and Least Significant Difference (LSD) tests are requested with the following MEANS statement. The CLDIFF option requests confidence intervals for both tests.

   means Strain/ lsd tukey cldiff ;
run;

The LSD tests for this example are shown in Output 23.2.3, and they give the same results as the previous two multiple comparison tests. Again, this is not always the case.

If you only perform the LSD tests when the overall model test is significant, then this is called Fisher’s protected LSD test. Note that the LSD tests should be used for planned comparisons.

The TUKEY tests shown in Output 23.2.4 find fewer significant differences than the other three tests. This is not unexpected, as the TUKEY test controls the Type I experimentwise error rate. For a complete discussion of multiple comparison methods, see the section Multiple Comparisons in Chapter 39, The GLM Procedure.