The SEQDESIGN Procedure

Example 89.3 Creating Two-Sided Pocock and O’Brien-Fleming Designs

This example requests two 4-stage group sequential designs for normally distributed statistics with equally spaced information levels at all stages. One design uses Pocock’s method and the other uses the O’Brien-Fleming method. The following statements invoke the SEQDESIGN procedure and request these two designs:

proc seqdesign altref=0.4
               pss
               stopprob
               errspend
               ;
   TwoSidedPocock:        design nstages=4 method=poc;
   TwoSidedOBrienFleming: design nstages=4 method=obf;
   samplesize model=twosamplemean(stddev=0.8 weight=2);
run;

By default (or equivalently if you specify ALT=TWOSIDED and STOP=REJECT in the DESIGN statement), each design has a null hypothesis $H_{0}$ with a two-sided alternative with early stopping to reject $H_{0}$.

The "Design Information" table in Output 89.3.1 displays design specifications and derived statistics for the Pocock’s design. With the specified ALTREF= option, the maximum information $I_ X= 77.6984$ is also derived.

Output 89.3.1: Pocock Design Information

The SEQDESIGN Procedure
Design: TwoSidedPocock

Design Information
Statistic Distribution Normal
Boundary Scale Standardized Z
Alternative Hypothesis Two-Sided
Early Stop Reject Null
Method Pocock
Boundary Key Both
Alternative Reference 0.4
Number of Stages 4
Alpha 0.05
Beta 0.1
Power 0.9
Max Information (Percent of Fixed Sample) 118.3143
Max Information 77.69844
Null Ref ASN (Percent of Fixed Sample) 115.6074
Alt Ref ASN (Percent of Fixed Sample) 69.74805



With the corresponding fixed-sample information

\[  I_{0} = \frac{ ( \Phi ^{-1}(1-\alpha /2) + \Phi ^{-1}(1-\beta ) )^{2} }{ {0.4}^{2} } = \frac{ (1.96 + 1.28155)^{2} }{ 0.16 } = 65.6728  \]

the fixed-sample information ratio is $77.6984 / 65.6728 = 1.1831$.

For a two-sided design with early stopping to reject the null hypothesis, lower and upper $\alpha $ boundaries are created. The "Method Information" table in Output 89.3.2 displays the $\alpha $ and $\beta $ errors, alternative references, and derived drift parameters, which are the standardized alternative references at the final stage.

Output 89.3.2: Method Information

Method Information
Boundary Method Alpha Beta Unified Family Alternative
Reference
Drift
Rho Tau C
Upper Alpha Pocock 0.02500 0.10000 0 0 2.36129 0.4 3.525869
Lower Alpha Pocock 0.02500 0.10000 0 0 2.36129 -0.4 -3.52587



With the METHOD=POC option, the Pocock method is used for each boundary. The Pocock method is one of the unified family methods, and the table also displays its corresponding parameters $\rho =0$ as a unified family method and the derived parameters $C=2.3613$ for the boundary values.

With the PSS option, the "Power and Expected Sample Sizes" table in Output 89.3.3 displays powers and expected sample sizes under various hypothetical references $\theta = c_{i} \theta _{1}$, where $\theta _{1}$ is the alternative reference and $c_{i}$ are values specified in the CREF= option. By default, $c_{i}= 0, 0.5, 1.0, 1.5$.

Output 89.3.3: Power and Expected Sample Size Information

Powers and Expected Sample Sizes
Reference = CRef * (Alt Reference)
CRef Power Sample Size
Percent
Fixed-Sample
0.0000 0.02500 115.6074
0.5000 0.34252 104.0615
1.0000 0.90000 69.7480
1.5000 0.99869 43.6600



Note that at $c_{i}=0$, the null reference $\theta =0$, and the power 0.025 corresponds to the one-sided Type I error probability 0.025. At $c_{i}=1$, $\theta =\theta _{1}$, the power 0.9 is the power of the design. The expected sample sizes are displayed in a percentage scale to its corresponding fixed-sample size design. With the specified SAMPLESIZE statement, the expected sample sizes for the specified model in the SAMPLESIZE statement are also displayed.

With the STOPPROB option, the "Expected Cumulative Stopping Probabilities" table in Output 89.3.4 displays the expected cumulative stopping stage and cumulative stopping probability to reject the null hypothesis $H_{0}$ at each stage under various hypothetical references $\theta = c_{i} \theta _{1}$, where $\theta _{1}$ is the alternative reference and $c_{i}$ are values specified in the CREF= option. By default, $c_{i}= 0, 0.5, 1.0, 1.5$.

Output 89.3.4: Stopping Probabilities

Expected Cumulative Stopping Probabilities
Reference = CRef * (Alt Reference)
CRef Expected
Stopping Stage
Source Stopping Probabilities
Stage_1 Stage_2 Stage_3 Stage_4
0.0000 3.908 Reject Null 0.01821 0.03155 0.04176 0.05000
0.5000 3.518 Reject Null 0.07005 0.15939 0.25242 0.34327
1.0000 2.358 Reject Null 0.27482 0.58074 0.78638 0.90002
1.5000 1.476 Reject Null 0.61145 0.92348 0.98900 0.99869



Note that at $c_{i}=0$, the cumulative stopping probability to reject $H_{0}$ at the final stage is the overall Type I error probability 0.05. At $c_{i}=1$, the alternative hypothesis $H_{1}: \theta =\theta _{1}$, the cumulative stopping probability to reject $H_{0}$ includes both the probability in the lower rejection region and the probability in the upper rejection region. This stopping probability to reject $H_{0}$ at the final stage, 0.90002, is slightly greater than the power $1-\beta = 0.90$, which corresponds to the cumulative stopping probability in the upper rejection region only. See the section Type I and Type II Errors for a detailed description of the Type II error probability $\beta $.

The "Boundary Information" table in Output 89.3.5 displays the information level, alternative references, and boundary values at each stage. By default (or equivalently if you specify BOUNDARYSCALE=STDZ), the standardized Z scale is used to display the alternative references and boundary values. The resulting standardized alternative reference at stage k is given by $\theta _1 \sqrt {I_ k}$, where $\theta _1$ is the alternative reference and $I_ k$ is the information level at stage k, $k= 1, 2, 3, 4$.

Output 89.3.5: Boundary Information

Boundary Information (Standardized Z Scale)
Null Reference = 0
_Stage_   Alternative Boundary Values
Information Level Reference Lower Upper
Proportion Actual N Lower Upper Alpha Alpha
1 0.2500 19.42461 55.94288 -1.76293 1.76293 -2.36129 2.36129
2 0.5000 38.84922 111.8858 -2.49317 2.49317 -2.36129 2.36129
3 0.7500 58.27383 167.8286 -3.05349 3.05349 -2.36129 2.36129
4 1.0000 77.69844 223.7715 -3.52587 3.52587 -2.36129 2.36129



By default (or equivalently if you specify INFO=EQUAL in the DESIGN statement), equally spaced information levels are used. With the SAMPLESIZE statement, the required sample size N is also displayed under the heading "Information Level." With the Pocock method, the standardized Z boundary values are identical at all stages for each $\alpha $ boundary.

At each interim stage, the hypothesis of $H_{0}: \theta = 0$ is rejected if the standardized normal test statistic $z \leq -2.36129$, the lower $\alpha $ boundary, or $z \geq 2.36129$, the upper $\alpha $ boundary. Otherwise, the trial continues to the next stage. At the final stage, stage 4, the trial stops and the hypothesis is rejected if the test statistic $|z_{4}| \geq 2.36129$. Otherwise, the hypothesis is accepted.

The "Error Spending Information" in Output 89.3.6 displays cumulative error spending at each stage for each boundary. It shows that more $\alpha $ errors are used in early stages than in later stages.

Output 89.3.6: Error Spending Information

Error Spending Information
_Stage_ Information
Level
Cumulative Error Spending
Lower Upper
Proportion Alpha Beta Beta Alpha
1 0.2500 0.00911 0.00002 0.00002 0.00911
2 0.5000 0.01577 0.00002 0.00002 0.01577
3 0.7500 0.02088 0.00002 0.00002 0.02088
4 1.0000 0.02500 0.10000 0.10000 0.02500



The "Sample Size Summary" table in Output 89.3.7 displays the specified parameters for the sample size computation of the two-sample test for mean difference.

Output 89.3.7: Sample Size Summary

Sample Size Summary
Test Two-Sample Means
Mean Difference 0.4
Standard Deviation 0.8
Max Sample Size 223.7715
Expected Sample Size (Null Ref) 218.652
Expected Sample Size (Alt Ref) 131.9167
Weight (Group A) 2
Weight (Group B) 1



The "Sample Sizes (N)" table in Output 89.3.8 displays the derived sample sizes at each stage, in both fractional and integer numbers. With the WEIGHT=2 option, the allocation ratio is 2 for the first group and 1 for the second group. See the section Test for the Difference between Two Normal Means for the derivation of these sample sizes. With the fixed-sample information ratio 1.1831, the derived sample sizes in fractional numbers are derived by multiplying 1.1831 by the corresponding sample sizes in the fixed-sample design.

Output 89.3.8: Sample Sizes

Sample Sizes (N)
Two-Sample Z Test for Mean Difference
_Stage_ Fractional N Ceiling N
N N(Grp 1) N(Grp 2) Information N N(Grp 1) N(Grp 2) Information
1 55.94 37.30 18.65 19.4246 57 38 19 19.7917
2 111.89 74.59 37.30 38.8492 113 75 38 39.4082
3 167.83 111.89 55.94 58.2738 168 112 56 58.3333
4 223.77 149.18 74.59 77.6984 225 150 75 78.1250



These fractional sample sizes are rounded up to integers under the heading "Ceiling N." When the resulting integer sample sizes are used, the corresponding information levels are slightly larger than the levels from the design. This can increase the power slightly if a trial uses these integer sample sizes.

Note that compared with other designs, a Pocock design can stop the trial early with a larger p-value. However, this might not be persuasive enough to make a new treatment widely accepted (Pocock and White, 1999).

The "Design Information" table in Output 89.3.9 displays design specifications and the derived statistics for the O’Brien-Fleming design. With the specified ALTREF= option, the maximum information $I_{X}= 67.1268$ is derived.

Output 89.3.9: O’Brien-Fleming Design Information

The SEQDESIGN Procedure
Design: TwoSidedOBrienFleming

Design Information
Statistic Distribution Normal
Boundary Scale Standardized Z
Alternative Hypothesis Two-Sided
Early Stop Reject Null
Method O'Brien-Fleming
Boundary Key Both
Alternative Reference 0.4
Number of Stages 4
Alpha 0.05
Beta 0.1
Power 0.9
Max Information (Percent of Fixed Sample) 102.2163
Max Information 67.12682
Null Ref ASN (Percent of Fixed Sample) 101.5728
Alt Ref ASN (Percent of Fixed Sample) 76.7397



With the corresponding fixed-sample information

\[  I_{0} = \frac{ ( \Phi ^{-1}(1-\alpha /2) + \Phi ^{-1}(1-\beta ) )^{2} }{ {0.4}^{2} } = \frac{ (1.96 + 1.28155)^{2} }{ 0.16 } = 65.6728  \]

the fixed-sample information ratio is $67.1268 / 65.6728 = 1.022$. That is, the maximum information for the O’Brien-Fleming design is only 2.2% more than for the corresponding fixed-sample design.

The "Method Information" table in Output 89.3.10 displays the Type I $\alpha $ level and Type II $\beta $ level. It also displays the derived drift parameter $\theta _{1} \sqrt {I_{X}}$, which is the standardized alternative reference at the final stage.

Output 89.3.10: Method Information

Method Information
Boundary Method Alpha Beta Unified Family Alternative
Reference
Drift
Rho Tau C
Upper Alpha O'Brien-Fleming 0.02500 0.10000 0.5 0 2.02429 0.4 3.277238
Lower Alpha O'Brien-Fleming 0.02500 0.10000 0.5 0 2.02429 -0.4 -3.27724



With the METHOD=OBF option, the O’Brien-Fleming method is used for each boundary. The O’Brien-Fleming method is one of the unified family methods, and the table also displays its corresponding parameters $\rho =0.5$ as a unified family method and the derived parameter $C=2.0243$ for the boundary values.

With the PSS option, the "Power and Expected Sample Sizes" table in Output 89.3.11 displays powers and expected sample sizes under various hypothetical references $\theta = c_{i} \theta _{1}$, where $\theta _{1}$ is the alternative reference and $c_{i}$ are values specified in the CREF= option.

Output 89.3.11: Power and Expected Sample Size Information

Powers and Expected Sample Sizes
Reference = CRef * (Alt Reference)
CRef Power Sample Size
Percent
Fixed-Sample
0.0000 0.02500 101.5728
0.5000 0.36495 96.3684
1.0000 0.90000 76.7397
1.5000 0.99821 57.2590



Compared with the corresponding Pocock design, the O’Brien-Fleming design has a smaller maximum sample size, and smaller expected sample sizes under hypothetical references $\theta =0$ and $\theta =0.5 \,  \theta _{1}$, but larger expected sample sizes under hypothetical references $\theta =\theta _{1}$ and $\theta =1.5 \,  \theta _{1}$.

With the STOPPROB option, the "Expected Cumulative Stopping Probabilities" table in Output 89.3.12 displays the expected stopping stage and cumulative stopping probability to reject the null hypothesis at each stage under various hypothetical references $\theta = c_{i} \theta _{1}$, where $\theta _{1}$ is the alternative reference and $c_{i}$ are values specified in the CREF= option.

Output 89.3.12: Stopping Probabilities

Expected Cumulative Stopping Probabilities
Reference = CRef * (Alt Reference)
CRef Expected
Stopping Stage
Source Stopping Probabilities
Stage_1 Stage_2 Stage_3 Stage_4
0.0000 3.975 Reject Null 0.00005 0.00422 0.02091 0.05000
0.5000 3.771 Reject Null 0.00062 0.04430 0.18392 0.36515
1.0000 3.003 Reject Null 0.00798 0.29296 0.69603 0.90000
1.5000 2.241 Reject Null 0.05584 0.73031 0.97315 0.99821



Compared with the corresponding Pocock design, the O’Brien-Fleming design has smaller stopping probabilities in early stages under each hypothetical reference.

The "Boundary Information" table in Output 89.3.13 displays the boundary values for the design that uses the O’Brien-Fleming method. Compared with the Pocock method, the standardized statistics $\alpha $ boundary values derived from the O’Brien-Fleming method in absolute values are larger in early stages and smaller in later stages. This makes the O’Brien-Fleming design less likely to reject the null hypothesis in early stages than the Pocock design. With the derived parameter $C=2.0243$ for the $\alpha $ boundary, the $\alpha $ boundaries at stage j are computed as $C \sqrt {4/j}$, $j=1, \ldots , 4$.

Output 89.3.13: Boundary Information

Boundary Information (Standardized Z Scale)
Null Reference = 0
_Stage_   Alternative Boundary Values
Information Level Reference Lower Upper
Proportion Actual N Lower Upper Alpha Alpha
1 0.2500 16.7817 48.33131 -1.63862 1.63862 -4.04859 4.04859
2 0.5000 33.56341 96.66262 -2.31736 2.31736 -2.86278 2.86278
3 0.7500 50.34511 144.9939 -2.83817 2.83817 -2.33745 2.33745
4 1.0000 67.12682 193.3252 -3.27724 3.27724 -2.02429 2.02429



The "Error Spending Information" in Output 89.3.14 displays cumulative error spending at each stage for each boundary. With smaller $\alpha $ spending in early stages for the O’Brien-Fleming method, it also indicates that the O’Brien-Fleming design is less likely to reject the null hypothesis in early stages than the Pocock design.

Output 89.3.14: Error Spending Information

Error Spending Information
_Stage_ Information
Level
Cumulative Error Spending
Lower Upper
Proportion Alpha Beta Beta Alpha
1 0.2500 0.00003 0.00000 0.00000 0.00003
2 0.5000 0.00211 0.00000 0.00000 0.00211
3 0.7500 0.01046 0.00000 0.00000 0.01046
4 1.0000 0.02500 0.10000 0.10000 0.02500



The "Sample Size Summary" table in Output 89.3.15 displays the specified parameters for the sample size computation of the two-sample test for mean difference.

Output 89.3.15: Sample Size Summary

Sample Size Summary
Test Two-Sample Means
Mean Difference 0.4
Standard Deviation 0.8
Max Sample Size 193.3252
Expected Sample Size (Null Ref) 192.1081
Expected Sample Size (Alt Ref) 145.1404
Weight (Group A) 2
Weight (Group B) 1



The "Sample Sizes (N)" table in Output 89.3.16 displays the derived sample sizes at each stage, in both fractional and integer numbers. With the fixed-sample information ratio 1.0222, the required sample sizes in fractional numbers are derived by multiplying 1.0222 by the corresponding sample sizes in the fixed-sample design.

Output 89.3.16: Derived Sample Sizes

Sample Sizes (N)
Two-Sample Z Test for Mean Difference
_Stage_ Fractional N Ceiling N
N N(Grp 1) N(Grp 2) Information N N(Grp 1) N(Grp 2) Information
1 48.33 32.22 16.11 16.7817 50 33 17 17.5313
2 96.66 64.44 32.22 33.5634 98 65 33 34.1996
3 144.99 96.66 48.33 50.3451 146 97 49 50.8669
4 193.33 128.88 64.44 67.1268 194 129 65 67.5338