Problem Note 50516: Multiple comparison (ADJUST=) results in the SurvDiff table might be incorrect for stratified tests
 |  |  |  |  |  |  |
When you specify a multiple-comparison adjustment with the ADJUST= option and stratified tests with the GROUP= option in the STRATA statement in PROC LIFETEST, the results in the SurvDiff table might be incorrect. Depending on how the TEST= option is specified, the logrank test results or the Wilcoxon test results might be substituted incorrectly. The problem occurs if both of these conditions are met:
- the ADJUST= option is used with the GROUP= option in the STRATA statement
- the logrank test and the Wilcoxon test are not requested
There are no errors or warnings to indicate a problem.
To work around the problem, include the logrank and Wilcoxon tests in one of these two ways:
- explicitly: TEST=(LOGRANK WILCOXON <optional additional tests>)
implicitly: TEST=ALL
Using TEST=ALL includes the logrank and Wilcoxon tests.
EXAMPLE
Suppose you specify the following STRATA statement in your PROC LIFETEST code.
strata clinic / group=dose_grp test=(Peto) adjust=BON;
In this case, PROC LIFETEST incorrectly uses the results of the overall logrank test to compute the multiple comparisons for the PETO test in the SurvDiff table. The HomTests table is correct. To work around the problem, specify TEST=ALL, or include the logrank and Wilcoxon tests explicitly.
Workaround 1
strata clinic / group=dose_grp test=ALL adjust=BON;
Workaround 2
strata clinic / group=dose_grp test=(logrank Wilcoxon Peto) adjust=BON;
Operating System and Release Information
| Product Family | Product | System | SAS Release | |
| Reported | Fixed* | |||
| SAS System | SAS/STAT | z/OS | 9.2 TS1M0 | 9.4 TS1M1 |
| Microsoft® Windows® for 64-Bit Itanium-based Systems | 9.2 TS1M0 | |||
| Microsoft Windows Server 2003 Datacenter 64-bit Edition | 9.2 TS1M0 | |||
| Microsoft Windows Server 2003 Enterprise 64-bit Edition | 9.2 TS1M0 | |||
| Microsoft Windows XP 64-bit Edition | 9.2 TS1M0 | |||
| Microsoft® Windows® for x64 | 9.2 TS1M0 | 9.4 TS1M1 | ||
| Microsoft Windows Server 2003 Datacenter Edition | 9.2 TS1M0 | |||
| Microsoft Windows Server 2003 Enterprise Edition | 9.2 TS1M0 | |||
| Microsoft Windows Server 2003 Standard Edition | 9.2 TS1M0 | |||
| Microsoft Windows XP Professional | 9.2 TS1M0 | |||
| Windows Vista | 9.2 TS1M0 | |||
| Windows Vista for x64 | 9.2 TS1M0 | |||
| 64-bit Enabled AIX | 9.2 TS1M0 | 9.4 TS1M1 | ||
| 64-bit Enabled HP-UX | 9.2 TS1M0 | 9.4 TS1M1 | ||
| 64-bit Enabled Solaris | 9.2 TS1M0 | 9.4 TS1M1 | ||
| HP-UX IPF | 9.2 TS1M0 | 9.4 TS1M1 | ||
| Linux | 9.2 TS1M0 | 9.4 TS1M1 | ||
| Linux for x64 | 9.2 TS1M0 | 9.4 TS1M1 | ||
| OpenVMS on HP Integrity | 9.2 TS1M0 | 9.4 TS1M1 | ||
| Solaris for x64 | 9.2 TS1M0 | 9.4 TS1M1 | ||
The code below has three PROC LIFETEST steps that each illustrate a stratified-tests scenario with a multiple comparisons adjustment. ADJUST=BON is used for illustration. The first PROC LIFETEST step requests TEST=(logrank). The stratified test results are correct. The second PROC LIFETEST step requests TEST=(Peto), and illustrates that the log-rank tests results are incorrectly populated to the SurvDiff table results for the Peto test. (The HomTests table results for the Peto test are correct.) The third PROC LIFETEST step illustrates the general TEST=ALL workaround, and in this case, all of the test results in the HomTests and SurvDiff tables are correct. The third step also illustrates post-processing the SurvDiff table to show only the Peto test results.
options ls=120 ps=42 nodate number pageno=1 center;
ods select all;
ods trace off;
ods listing;
ods graphics off;
title1;
run;
/********/
/* data */
/********/
data methadone;
input PatientID $ Clinic Status Time Prison Dose @@;
if dose lt 50 then dose_grp='Dose1_Under50mg';
else if (dose ge 50 and dose le 70) then dose_grp='Dose2_50to70mg';
else if dose gt 70 then dose_grp='Dose3_Over70mg';
datalines;
1 1 1 428 0 50 2 1 1 275 1 55
3 1 1 262 0 55 4 1 1 183 0 30
5 1 1 259 1 65 6 1 1 714 0 55
7 1 1 438 1 65 8 1 0 796 1 60
9 1 1 892 0 50 10 1 1 393 1 65
11 1 0 161 1 80 12 1 1 836 1 60
13 1 1 523 0 55 14 1 1 612 0 70
15 1 1 212 1 60 16 1 1 399 1 60
17 1 1 771 1 75 18 1 1 514 1 80
19 1 1 512 0 80 21 1 1 624 1 80
22 1 1 209 1 60 23 1 1 341 1 60
24 1 1 299 0 55 25 1 0 826 0 80
26 1 1 262 1 65 27 1 0 566 1 45
28 1 1 368 1 55 30 1 1 302 1 50
31 1 0 602 0 60 32 1 1 652 0 80
33 1 1 293 0 65 34 1 0 564 0 60
36 1 1 394 1 55 37 1 1 755 1 65
38 1 1 591 0 55 39 1 0 787 0 80
40 1 1 739 0 60 41 1 1 550 1 60
42 1 1 837 0 60 43 1 1 612 0 65
44 1 0 581 0 70 45 1 1 523 0 60
46 1 1 504 1 60 48 1 1 785 1 80
49 1 1 774 1 65 50 1 1 560 0 65
51 1 1 160 0 35 52 1 1 482 0 30
53 1 1 518 0 65 54 1 1 683 0 50
55 1 1 147 0 65 57 1 1 563 1 70
58 1 1 646 1 60 59 1 1 899 0 60
60 1 1 857 0 60 61 1 1 180 1 70
62 1 1 452 0 60 63 1 1 760 0 60
64 1 1 496 0 65 65 1 1 258 1 40
66 1 1 181 1 60 67 1 1 386 0 60
68 1 0 439 0 80 69 1 0 563 0 75
70 1 1 337 0 65 71 1 0 613 1 60
72 1 1 192 1 80 73 1 0 405 0 80
74 1 1 667 0 50 75 1 0 905 0 80
76 1 1 247 0 70 77 1 1 821 0 80
78 1 1 821 1 75 79 1 0 517 0 45
80 1 0 346 1 60 81 1 1 294 0 65
82 1 1 244 1 60 83 1 1 95 1 60
84 1 1 376 1 55 85 1 1 212 0 40
86 1 1 96 0 70 87 1 1 532 0 80
88 1 1 522 1 70 89 1 1 679 0 35
90 1 0 408 0 50 91 1 0 840 0 80
92 1 0 148 1 65 93 1 1 168 0 65
94 1 1 489 0 80 95 1 0 541 0 80
96 1 1 205 0 50 97 1 0 475 1 75
98 1 1 237 0 45 99 1 1 517 0 70
100 1 1 749 0 70 101 1 1 150 1 80
102 1 1 465 0 65 103 2 1 708 1 60
104 2 0 713 0 50 105 2 0 146 0 50
106 2 1 450 0 55 109 2 0 555 0 80
110 2 1 460 0 50 111 2 0 53 1 60
113 2 1 122 1 60 114 2 1 35 1 40
118 2 0 532 0 70 119 2 0 684 0 65
120 2 0 769 1 70 121 2 0 591 0 70
122 2 0 769 1 40 123 2 0 609 1 100
124 2 0 932 1 80 125 2 0 932 1 80
126 2 0 587 0 110 127 2 1 26 0 40
128 2 0 72 1 40 129 2 0 641 0 70
131 2 0 367 0 70 132 2 0 633 0 70
133 2 1 661 0 40 134 2 1 232 1 70
135 2 1 13 1 60 137 2 0 563 0 70
138 2 0 969 0 80 143 2 0 1052 0 80
144 2 0 944 1 80 145 2 0 881 0 80
146 2 1 190 1 50 148 2 1 79 0 40
149 2 0 884 1 50 150 2 1 170 0 40
153 2 1 286 0 45 156 2 0 358 0 60
158 2 0 326 1 60 159 2 0 769 1 40
160 2 1 161 0 40 161 2 0 564 1 80
162 2 1 268 1 70 163 2 0 611 1 40
164 2 1 322 0 55 165 2 0 1076 1 80
166 2 0 2 1 40 168 2 0 788 0 70
169 2 0 575 0 80 170 2 1 109 1 70
171 2 0 730 1 80 172 2 0 790 0 90
173 2 0 456 1 70 175 2 1 231 1 60
176 2 1 143 1 70 177 2 0 86 1 40
178 2 0 1021 0 80 179 2 0 684 1 80
180 2 1 878 1 60 181 2 1 216 0 100
182 2 0 808 0 60 183 2 1 268 1 40
184 2 0 222 0 40 186 2 0 683 0 100
187 2 0 496 0 40 188 2 1 389 0 55
189 1 1 126 1 75 190 1 1 17 1 40
192 1 1 350 0 60 193 2 0 531 1 65
194 1 0 317 1 50 195 1 0 461 1 75
196 1 1 37 0 60 197 1 1 167 1 55
198 1 1 358 0 45 199 1 1 49 0 60
200 1 1 457 1 40 201 1 1 127 0 20
202 1 1 7 1 40 203 1 1 29 1 60
204 1 1 62 0 40 205 1 0 150 1 60
206 1 1 223 1 40 207 1 0 129 1 40
208 1 0 204 1 65 209 1 1 129 1 50
210 1 1 581 0 65 211 1 1 176 0 55
212 1 1 30 0 60 213 1 1 41 0 60
214 1 0 543 0 40 215 1 0 210 1 50
216 1 1 193 1 70 217 1 1 434 0 55
218 1 1 367 0 45 219 1 1 348 1 60
220 1 0 28 0 50 221 1 0 337 0 40
222 1 0 175 1 60 223 2 1 149 1 80
224 1 1 546 1 50 225 1 1 84 0 45
226 1 0 283 1 80 227 1 1 533 0 55
228 1 1 207 1 50 229 1 1 216 0 50
230 1 0 28 0 50 231 1 1 67 1 50
232 1 0 62 1 60 233 1 0 111 0 55
234 1 1 257 1 60 235 1 1 136 1 55
236 1 0 342 0 60 237 2 1 41 0 40
238 2 0 531 1 45 239 1 0 98 0 40
240 1 1 145 1 55 241 1 1 50 0 50
242 1 0 53 0 50 243 1 0 103 1 50
244 1 0 2 1 60 245 1 1 157 1 60
246 1 1 75 1 55 247 1 1 19 1 40
248 1 1 35 0 60 249 2 0 394 1 80
250 1 1 117 0 40 251 1 1 175 1 60
252 1 1 180 1 60 253 1 1 314 0 70
254 1 0 480 0 50 255 1 0 325 1 60
256 2 1 280 0 90 257 1 1 204 0 50
258 2 1 366 0 55 259 2 0 531 1 50
260 1 1 59 1 45 261 1 1 33 1 60
262 2 1 540 0 80 263 2 0 551 0 65
264 1 1 90 0 40 266 1 1 47 0 45
;
/************/
/* LIFETEST */
/************/
ods select HomTests(persist) SurvDiff(persist);
* stratified test with ADJUST= and TEST=logrank *;
* results are correct for requesting the logrank test *;
proc lifetest data=methadone;
time time*status(0);
strata clinic / group=dose_grp test=(logrank) adjust=BON;
title1 "stratified test and ADJUST= with logrank test";
title2 "HomTests and SurvDiff results are correct here";
run;
* stratified test with ADJUST= and TEST=Peto *;
* this illustrates the bug *;
proc lifetest data=methadone;
time time*status(0);
strata clinic / group=dose_grp test=(PETO) adjust=BON;
title1 "stratified test and ADJUST= with Peto test";
title2 "HomTests results are for the Peto test (and are correct) but";
title3 "SurvDiff results are still from the logrank test";
title4 "this illustrates the bug in the SurvDiff table";
run;
* stratified test with ADJUST= and TEST=ALL *;
* this illustrates the general work-around *;
proc lifetest data=methadone;
time time*status(0);
strata clinic / group=dose_grp test=ALL adjust=BON;
ods output SurvDiff=SD;
title1 "stratified test and ADJUST= with TEST=ALL";
title2 "this illustrates the general work-around";
title3 "the SurvDiff table results are now all correct";
run;
ods select all;
proc print data=SD(where=(TEST='Peto'));
id test;
title1 "PROC PRINT of SurvDiff table from TEST=ALL work-around";
title2 "selected results for Peto test shown";
run;
stratified test and ADJUST= with logrank test 1
HomTests and SurvDiff results are correct here
The LIFETEST Procedure
Stratified Test of Equality over Group
Pr >
Test Chi-Square DF Chi-Square
Log-Rank 32.5521 2 <.0001
Adjustment for Multiple Comparisons for the Logrank Test
Group Comparison p-Values
dose_grp dose_grp Chi-Square Raw Bonferroni
Dose1_Under50mg Dose2_50to70mg 0.2236 0.6363 1.0000
Dose1_Under50mg Dose3_Over70mg 32.4694 <.0001 <.0001
Dose2_50to70mg Dose3_Over70mg 11.7954 0.0006 0.0018
stratified test and ADJUST= with Peto test 2
HomTests results are for the Peto test (and are correct) but
SurvDiff results are still from the logrank test
this illustrates the bug in the SurvDiff table
The LIFETEST Procedure
Stratified Test of Equality over Group
Pr >
Test Chi-Square DF Chi-Square
Peto 32.3585 2 <.0001
Adjustment for Multiple Comparisons for the Peto Test
Group Comparison p-Values
dose_grp dose_grp Chi-Square Raw Bonferroni
Dose1_Under50mg Dose2_50to70mg 0.2236 0.6363 1.0000
Dose1_Under50mg Dose3_Over70mg 32.4694 <.0001 <.0001
Dose2_50to70mg Dose3_Over70mg 11.7954 0.0006 0.0018
stratified test and ADJUST= with TEST=ALL 3
this illustrates the general work-around
the SurvDiff table results are now all correct
The LIFETEST Procedure
Stratified Test of Equality over Group
Pr >
Test Chi-Square DF Chi-Square
Log-Rank 32.5521 2 <.0001
Wilcoxon 29.0559 2 <.0001
Tarone 31.6461 2 <.0001
Peto 32.3585 2 <.0001
Modified Peto 32.2222 2 <.0001
Fleming(1) 32.4501 2 <.0001
Adjustment for Multiple Comparisons for the Logrank Test
Group Comparison p-Values
dose_grp dose_grp Chi-Square Raw Bonferroni
Dose1_Under50mg Dose2_50to70mg 0.2236 0.6363 1.0000
Dose1_Under50mg Dose3_Over70mg 32.4694 <.0001 <.0001
Dose2_50to70mg Dose3_Over70mg 11.7954 0.0006 0.0018
Adjustment for Multiple Comparisons for the Wilcoxon Test
Group Comparison p-Values
dose_grp dose_grp Chi-Square Raw Bonferroni
Dose1_Under50mg Dose2_50to70mg 2.4696 0.1161 0.3482
Dose1_Under50mg Dose3_Over70mg 28.9114 <.0001 <.0001
Dose2_50to70mg Dose3_Over70mg 5.8735 0.0154 0.0461
stratified test and ADJUST= with TEST=ALL 4
this illustrates the general work-around
the SurvDiff table results are now all correct
The LIFETEST Procedure
Adjustment for Multiple Comparisons for the Tarone Test
Group Comparison p-Values
dose_grp dose_grp Chi-Square Raw Bonferroni
Dose1_Under50mg Dose2_50to70mg 1.3448 0.2462 0.7386
Dose1_Under50mg Dose3_Over70mg 31.5903 <.0001 <.0001
Dose2_50to70mg Dose3_Over70mg 8.6523 0.0033 0.0098
Adjustment for Multiple Comparisons for the Peto Test
Group Comparison p-Values
dose_grp dose_grp Chi-Square Raw Bonferroni
Dose1_Under50mg Dose2_50to70mg 2.1332 0.1441 0.4324
Dose1_Under50mg Dose3_Over70mg 32.2481 <.0001 <.0001
Dose2_50to70mg Dose3_Over70mg 8.0353 0.0046 0.0138
Adjustment for Multiple Comparisons for the ModPeto Test
Group Comparison p-Values
dose_grp dose_grp Chi-Square Raw Bonferroni
Dose1_Under50mg Dose2_50to70mg 2.1867 0.1392 0.4176
Dose1_Under50mg Dose3_Over70mg 32.1044 <.0001 <.0001
Dose2_50to70mg Dose3_Over70mg 7.8942 0.0050 0.0149
Adjustment for Multiple Comparisons for the Fleming Test
Group Comparison p-Values
dose_grp dose_grp Chi-Square Raw Bonferroni
Dose1_Under50mg Dose2_50to70mg 2.1074 0.1466 0.4398
Dose1_Under50mg Dose3_Over70mg 32.3431 <.0001 <.0001
Dose2_50to70mg Dose3_Over70mg 8.1180 0.0044 0.0131
PROC PRINT of SurvDiff table from TEST=ALL work-around 5
selected results for Peto test shown
Test Group1 Group2 ChiSq Raw Bonferroni
Peto Dose1_Under50mg Dose2_50to70mg 2.1332 0.1441 0.4324
Peto Dose1_Under50mg Dose3_Over70mg 32.2481 <.0001 <.0001
Peto Dose2_50to70mg Dose3_Over70mg 8.0353 0.0046 0.0138
A fix for this issue for SAS/STAT 12.3 is available at:
https://tshf.sas.com/techsup/download/hotfix/HF2/L13.html#50516
Results of the ADJUST= option in the STRATA statement might be incorrect for stratified tests, which are invoked with the GROUP= option in the STRATA statement. Depending on how the TEST= option is specified, the log-rank test results might be substituted incorrectly. A general workaround exists.
| Type: | Problem Note |
| Priority: | alert |
| Topic: | Analytics ==> Survival Analysis SAS Reference ==> Procedures ==> LIFETEST |
| Date Modified: | 2013-07-29 13:11:37 |
| Date Created: | 2013-07-26 07:30:22 |