The SEQTEST Procedure

Example 84.9 Conducting a Trial with a Nonbinding Acceptance Boundary

This example demonstrates a trail with a nonbinding acceptance boundary. The design is similar to the design in Example 84.1, but has a nonbinding beta boundary.

A clinic is studying the effect of vitamin C supplements in treating flu symptoms. The study consists of patients in the clinic who have exhibited the first sign of flu symptoms within the last 24 hours. These patients are randomly assigned to either the control group (which receives placebo pills) or the treatment group (which receives large doses of vitamin C supplements). At the end of a five-day period, the flu symptoms of each patient are recorded.

Suppose that you know from past experience that flu symptoms disappear in five days for $60\% $ of patients who experience flu symptoms. The clinic would like to detect a $75\% $ symptom disappearance with a high probability. A test that compares the proportions directly specifies the null hypothesis $H_{0}: \theta = p_{t} - p_{c}= 0$ with a one-sided alternative $H_{1}: \theta > 0$ and a power of 0.90 at $H_{1}: \theta = 0.15$, where $p_{t}$ and $p_{c}$ are the proportions of symptom disappearance in the treatment group and control group, respectively.

The following statements invoke the SEQDESIGN procedure and request a four-stage group sequential design by using an O’Brien-Fleming method for normally distributed data.

ods graphics on;
proc seqdesign altref=0.15 errspend
               ;
   NonbindingDesign: design nstages=4
                     method=obf
                     alt=upper
                     stop=both(betaboundary=nonbinding)
                     alpha=0.025
                     ;
   samplesize model=twosamplefreq(nullprop=0.6 test=prop);
ods output Boundary=Bnd_Count;
run;
ods graphics off;

The ALT=UPPER option specifies a one-sided alternative hypothesis, and the STOP=BOTH option specifies early stopping to reject or accept the null hypothesis $H_{0}$, The BETABOUNDARY=NONBINDING suboption requests nonbinding beta boundary to have the flexibility to continue the trial when a test statistic falls in the acceptance region at interim stages (Zhu, Ni, and Yao, 2011, pp. 132–133). The ODS OUTPUT statement with the BOUNDARY=BND_COUNT option creates an output data set named BND_COUNT which contains the resulting boundary information for the subsequent sequential tests.

The Design Information table in Output 84.9.1 displays design specifications. With the specified alternative hypothesis $H_{1}: \theta = 0.15$, the maximum information is derived to achieve a power of 0.90 at $H_{1}$. The derived fixed-sample information ratio 1.107 is the maximum information needed for a group sequential design relative to its corresponding fixed-sample design. Compare to the default BETABOUNDARY=BINDING option, the BETABOUNDARY=NONBINDING option requires a larger sample size for the design.

Output 84.9.1: O’Brien-Fleming Design Information

The SEQDESIGN Procedure
Design: NonbindingDesign

Design Information
Statistic Distribution Normal
Boundary Scale Standardized Z
Alternative Hypothesis Upper
Early Stop Accept(Nonbinding)/Reject Null
Method O'Brien-Fleming
Boundary Key Both
Alternative Reference 0.15
Number of Stages 4
Alpha (Binding Beta Boundary) 0.02228
Alpha (Nonbinding Beta Boundary) 0.025
Beta 0.1
Power 0.9
Max Information (Percent of Fixed Sample) 110.7138
Max Information 517.0296
Null Ref ASN (Percent of Fixed Sample) 62.29796
Alt Ref ASN (Percent of Fixed Sample) 78.5392


The Boundary Information table in Output 84.9.2 displays the information level, alternative reference, and boundary values at each stage. With the default BOUNDARYSCALE=STDZ option, the SEQDESIGN procedure displays the output boundaries with the standardized normal Z scale.

Output 84.9.2: O’Brien-Fleming Boundary Information

Boundary Information (Standardized Z Scale)
Nonbinding Beta Boundary, Null Reference = 0
_Stage_   Alternative Boundary Values
Information Level Reference Upper
Proportion Actual N Upper Beta Alpha
1 0.2500 129.2574 110.5151 1.70537 -1.06752 4.04859
2 0.5000 258.5148 221.0302 2.41176 0.45103 2.86279
3 0.7500 387.7722 331.5452 2.95379 1.35286 2.33746
4 1.0000 517.0296 442.0603 3.41074 2.02430 2.02430


With ODS Graphics enabled, a detailed boundary plot with the rejection and acceptance regions is displayed, as shown in Output 84.1.3. The horizontal axis indicates the information levels for the design. The stages are indicated by vertical lines with accompanying stage numbers. If the test statistic at a stage is in a rejection region, the trial stops and the hypothesis is rejected. If the test statistic is in an acceptance region, then the trial either stops to accept the hypothesis or continues to the next stage. If the statistic is not in a rejection or an acceptance region, the trial continues to the next stage.

Output 84.9.3: O’Brien-Fleming Boundary Plot

O’Brien-Fleming Boundary Plot


The boundary plot also displays the information level and critical value for the corresponding fixed-sample design. The solid and dashed lines at the fixed-sample information level correspond to the rejection and acceptance lines, respectively.

The Error Spending Information (Nonbinding Beta Boundary) in Output 84.9.4 displays cumulative error spending at each stage for each boundary. With a nonbinding beta boundary, the $\alpha $ spending at each stage is computed by using the $\alpha $ boundary only.

Output 84.9.4: Error Spending Information

Error Spending Information (Nonbinding
Beta Boundary)
_Stage_ Information
Level
Cumulative Error Spending
Upper
Proportion Beta Alpha
1 0.2500 0.00278 0.00003
2 0.5000 0.02603 0.00211
3 0.7500 0.06343 0.01046
4 1.0000 0.10000 0.02500


With the SAMPLESIZE statement, the maximum information is used to derive the required sample size for the study. The Sample Size Summary table in Output 84.9.5 displays parameters for the sample size computation.

Output 84.9.5: Required Sample Size Summary

Sample Size Summary
Test Two-Sample Proportions
Null Proportion 0.6
Proportion (Group A) 0.75
Test Statistic Z for Proportion
Reference Proportions Alt Ref
Max Sample Size 442.0603
Expected Sample Size (Null Ref) 248.7446
Expected Sample Size (Alt Ref) 313.5929


With the derived maximum information and the specified MODEL= option in the SAMPLESIZE statement, the total sample size in each group for testing the difference between two proportions under the alternative hypothesis is

\[  N_{1} = N_{2} =\left( p_{1c} \,  (1-p_{1c}) + p_{1t} \,  (1-p_{1t}) \right) \,  I_{X}  \]

where $p_{1c}=0.6$ and $p_{1t}=p_{1c}+\theta _{1}=0.75$. By default (or equivalently if you specify REF=PROP in the MODEL=TWOSAMPLEFREQ option), the required sample sizes are computed under the alternative hypothesis. See the section Test for the Difference between Two Binomial Proportions in the chapter The SEQDESIGN Procedure for a description of these parameters.

The Sample Sizes (N) table in Output 84.9.6 displays the required sample sizes at each stage, in both fractional and integer numbers. The derived sample sizes under the heading Fractional N which correspond to the design are not integers. These sample sizes are rounded up to integers under the heading Ceiling N. In practice, integer sample sizes are used, and the information levels increase slightly. Thus, 56, 111, 166, and 222 patients are needed in each group for the four stages, respectively.

Output 84.9.6: Required Sample Sizes

Sample Sizes (N)
Two-Sample Z Test for Proportion Difference
_Stage_ Fractional N Ceiling N
N N(Grp 1) N(Grp 2) Information N N(Grp 1) N(Grp 2) Information
1 110.52 55.26 55.26 129.3 112 56 56 131.0
2 221.03 110.52 110.52 258.5 222 111 111 259.6
3 331.55 165.77 165.77 387.8 332 166 166 388.3
4 442.06 221.03 221.03 517.0 444 222 222 519.3


Suppose the trial follows the study plan, and 56 patients are available in each group at stage 1. The data set count_1 contains these 108 patients. Output 84.9.7 lists the first 10 observations of the data set.

Output 84.9.7: Clinical Trial Data

First 10 Obs in the Trial Data

Obs Trt Resp
1 0 0
2 1 1
3 0 1
4 1 1
5 0 1
6 1 0
7 0 0
8 1 1
9 0 1
10 1 0


The Trt variable is a grouping variable with value 0 for a patient in the placebo control group and value 1 for a patient in the treatment group who is given vitamin C supplements. The Resp variable is an indicator variable with value 1 for a patient without flu symptoms after five days and value 0 for a patient with flu symptoms after five days.

The following statements use the GENMOD procedure to estimate the treatment effect at stage 1:

proc genmod data=count_1;
   model Resp= Trt;
ods output ParameterEstimates=Parms_Count1;
run;

Output 84.9.8 displays the treatment effect at stage 1.

Output 84.9.8: Stage 1 Treatment Difference

The GENMOD Procedure

Analysis Of Maximum Likelihood Parameter Estimates
Parameter DF Estimate Standard Error Wald 95% Confidence Limits Wald Chi-Square Pr > ChiSq
Intercept 1 0.6786 0.0619 0.5572 0.7999 120.07 <.0001
Trt 1 0.0179 0.0876 -0.1538 0.1895 0.04 0.8384
Scale 1 0.4634 0.0310 0.4065 0.5283    

Note: The scale parameter was estimated by maximum likelihood.



The test statistic is $\hat{\theta }_{1}= \hat{p}_{t}-\hat{p}_{c}= 0.0179$, and its associated standard error is

\[  \sqrt {\mr {Var}( \hat{\theta }_{1})}= \sqrt { \frac{\hat{p}_{c}(1-\hat{p}_{c})}{56} + \frac{\hat{p}_{t}(1-\hat{p}_{t})}{56} } = 0.0876  \]

The following statements create and display (in Output 84.9.9) the data set that contains the parameter estimate at stage 1, $\hat{\theta }_{1}= 0.0179$, and its associated standard error $\sqrt {\mr {Var}( \hat{\theta }_{1})}= 0.0876$ which are used in the SEQTEST procedure:

data Parms_Count1;
   set Parms_Count1;
   if Parameter='Trt';
   _Scale_='MLE';
   _Stage_= 1;
   keep _Scale_ _Stage_ Parameter Estimate StdErr;
run;

proc print data=Parms_Count1;
   title 'Statistics Computed at Stage 1';
run;

Output 84.9.9: Statistics Computed at Stage 1

Statistics Computed at Stage 1

Obs Parameter Estimate StdErr _Scale_ _Stage_
1 Trt 0.0179 0.0876 MLE 1


The initial required sample sizes are derived with the proportions $p_{c}=0.6$ and $p_{t}=0.75$. If the observed proportions are different from these assumed values, or if the number of available patients is different from the study plan in one of the stages, then the information level that corresponds to the test statistic is estimated from

\[  I_{k} = \frac{1}{\mr {Var}( \hat{\theta }_{k})}  \]

The following statements invoke the SEQTEST procedure and test for early stopping at stage 1:

ods graphics on;
proc seqtest Boundary=Bnd_Count
             Parms(Testvar=Trt)=Parms_Count1
             betaboundary=nonbinding
             infoadj=none
             errspendmin=0.001
             errspend
             ;
ods output Test=Test_Count1;
run;
ods graphics off;

The BOUNDARY= option specifies the input data set that provides the boundary information for the trial at stage 1, which was generated in the SEQDESIGN procedure. The PARMS=PARMS_COUNT1 option specifies the input data set PARMS_COUNT1 that contains the test statistic and its associated standard error at stage 1, and the TESTVAR=TRT option identifies the test variable TRT in the data set.

The BETABOUNDARY=NONBINDING option requests nonbinding beta boundary to have the flexibility to continue the trial when the test statistic falls in the acceptance region at stage 1. The INFOADJ=NONE option maintains the information levels at future interim stages (2 and 3) as provided in the BOUNDARY= data set. The default BOUNDARYSCALE=STDZ option displays the output boundaries in terms of the standardized normal Z scale.

The O’Brien-Fleming design is conservative in early stages and might not be desirable in a clinical trial. The ERRSPENDMIN=0.001 option specifies the minimum error spending at each stage to be 0.001, and it might increase the corresponding nominal p-value in early stages for the trial. The default BOUNDARYSCALE=STDZ option uses the standardized normal Z scale to display test statistics in the boundary table and boundary plots.

The ODS OUTPUT statement with the TEST=TEST_COUNT1 option creates an output data set named TEST_COUNT1 which contains the updated boundary information for the test at stage 1. The data set also provides the boundary information that is needed for the group sequential test at the next stage.

The Design Information table in Output 84.9.10 displays design specifications. The derived statistics, such as the overall $\alpha $ and $\beta $ levels, are derived from the specified maximum information and boundary values in the BOUNDARY= data set. Note that with a minor change in the information level at stage 1, the power also changes slightly from the design provided in the BOUNDARY= data set.

Output 84.9.10: Design Information

The SEQTEST Procedure

Design Information
BOUNDARY Data Set WORK.BND_COUNT
Data Set WORK.PARMS_COUNT1
Statistic Distribution Normal
Boundary Scale Standardized Z
Alternative Hypothesis Upper
Early Stop Accept(Nonbinding)/Reject Null
Number of Stages 4
Alpha (Binding Beta Boundary) 0.02237
Alpha (Nonbinding Beta Boundary) 0.025
Beta 0.10142
Power 0.89858
Max Information (Percent of Fixed Sample) 111.2654
Max Information 517.029617
Null Ref ASN (Percent of Fixed Sample) 62.27694
Alt Ref ASN (Percent of Fixed Sample) 76.31328


With the ERRSPEND option, the Error Spending Information table in Output 84.9.11 displays cumulative error spending at each stage for each boundary. By default (or equivalently if you specify BOUNDARYKEY=ALPHA), the Type I error level with nonbinding beta boundary $\alpha =0.025$ is maintained. Furthermore, with the ERRSPENDMIN=0.001 option, the $\alpha $ spending at each stage is greater than or equal to 0.001.

Output 84.9.11: Error Spending Information

Error Spending Information (Nonbinding
Beta Boundary)
_Stage_   Cumulative Error Spending
Information Level Upper
Proportion Actual Beta Alpha
1 0.2522 130.3756 0.00302 0.00100
2 0.5000 258.5148 0.02640 0.00299
3 0.7500 387.7722 0.06433 0.01101
4 1.0000 517.0296 0.10142 0.02500


The Test Information table in Output 84.9.12 displays the boundary values for the design, test statistic, and resulting action at each stage. With the default BOUNDARYSCALE=STDZ option, the standardized normal Z scale is used for the test statistic and boundary values. The table shows that the test statistic 0.2039 is between the upper $\alpha $ and $\beta $ boundaries, so the trial continues to the next stage.

Output 84.9.12: Sequential Test

Test Information (Standardized Z Scale)
Nonbinding Beta Boundary, Null Reference = 0
_Stage_   Alternative Boundary Values Test
Information Level Reference Upper Trt
Proportion Actual Upper Beta Alpha Estimate Action
1 0.2522 130.3756 1.71273 -1.03271 3.09023 0.20390 Continue
2 0.5000 258.5148 2.41176 0.45564 2.84016 .  
3 0.7500 387.7722 2.95379 1.36069 2.34211 .  
4 1.0000 517.0296 3.41074 2.03599 2.03599 .  


The information level at stage 1 is derived from the standard error,

\[  I_{1} = \frac{1}{\mr {Var}( \hat{\theta }_{1})} = \frac{1}{(\mr {s.e.}(\hat{\theta }_{1}))^{2}} = 130.376  \]

By default (or equivalently if you specify PLOTS=TEST), the Test Plot graph displays boundary values of the design and the test statistic at stage 1, as shown in Output 84.9.13. It also shows that the observed statistic is in the continuation region.

Output 84.9.13: Sequential Test Plot

Sequential Test Plot


The observed information level at stage 1, $I_{1}= 130.376$, is slightly larger than the target information level at the design. If an observed information level in the study is substantially different from its target level in the design, then the sample sizes should be adjusted in the subsequent stages to achieve the target information levels.

Suppose the trial continues to the next stage, and 111 patients are available in each group at stage 2. The data set COUNT_2 contains these 222 patients.

The following statements use the GENMOD procedure to estimate the treatment effect at stage 2:

proc genmod data=Count_2;
   model Resp= Trt;
ods output ParameterEstimates=Parms_Count2;
run;

The following statements create the parameter estimate at stage 2, $\hat{\theta }_{2}= \hat{p}_{2}-\hat{p}_{1}= 0.027$, and its associated standard error $\sqrt {\mr {Var}( \hat{\theta }_{2})}= 0.0621$ into a test data set:

data Parms_Count2;
   set Parms_Count2;
   if Parameter='Trt';
   _Scale_='MLE';
   _Stage_= 2;
   keep _Scale_ _Stage_ Parameter Estimate StdErr;
run;

proc print data=Parms_Count2;
   title 'Statistics Computed at Stage 2';
run;

Output 84.9.14 displays the test statistics at stage 2.

Output 84.9.14: Statistics Computed at Stage 2

Statistics Computed at Stage 2

Obs Parameter Estimate StdErr _Scale_ _Stage_
1 Trt 0.0270 0.0621 MLE 2


The following statements invoke the SEQTEST procedure and test for early stopping at stage 2:

ods graphics on;
proc seqtest Boundary=Test_Count1
             Parms(Testvar=Trt)=Parms_Count2
             betaboundary=nonbinding
             infoadj=none
             ;
ods output Test=Test_Count2;
run;
ods graphics off;

The BOUNDARY= option specifies the input data set that provides the boundary information for the trial at stage 2, which was generated by the SEQTEST procedure at the previous stage. The PARMS= option specifies the input data set that contains the test statistic and its associated standard error at stage 2, and the TESTVAR= option identifies the test variable in the data set.

The ODS OUTPUT statement with the TEST=TEST_COUNT2 option creates an output data set named TEST_COUNT2 which contains the updated boundary information for the test at stage 2. The data set also provides the boundary information that is needed for the group sequential test at the next stage.

The Test Information table in Output 84.9.15 displays the boundary values, test statistic, and resulting action at each stage. The table shows that the test statistic 0.43522 is less than the corresponding upper beta boundary value, so the trial can be stopped to accept the hypothesis. With the BETABOUNDARY=NONBINDING option, The clinic has the option to continue the trial while still maintaining the Type I error level.

Output 84.9.15: Sequential Test

The SEQTEST Procedure

Test Information (Standardized Z Scale)
Nonbinding Beta Boundary, Null Reference = 0
_Stage_   Alternative Boundary Values Test
Information Level Reference Upper Trt
Proportion Actual Upper Beta Alpha Estimate Action
1 0.2522 130.3756 1.71273 -1.03271 3.09023 0.20390 Continue
2 0.5015 259.3157 2.41549 0.46349 2.83297 0.43522 Accept Null (Nonbinding)
3 0.7500 387.7722 2.95379 1.36018 2.34294 .  
4 1.0000 517.0296 3.41074 2.03610 2.03610 .  


With ODS Graphics enabled, the Test Plot is displayed, as shown in Output 84.9.16. The plot displays boundary values of the design and the test statistics at the first two stages. As expected, the test statistic at stage 2 is in the Upper Acceptance Region below the upper beta boundary.

Output 84.9.16: Sequential Test Plot

Sequential Test Plot


Suppose the clinic decides to continue the trial, and at the next stage, 108 patients are available in each group at stage 3. The data set COUNT_3 contains these 332 patients.

The following statements use the GENMOD procedure to estimate the treatment effect at stage 3:

proc genmod data=Count_3;
   model Resp= Trt;
ods output ParameterEstimates=Parms_Count3;
run;

The following statements create the parameter estimate at stage 2, $\hat{\theta }_{2}= \hat{p}_{2}-\hat{p}_{1}= 0.1205$, and its associated standard error $\sqrt {\mr {Var}( \hat{\theta }_{2})}= 0.0507$ into a test data set:

data Parms_Count3;
   set Parms_Count3;
   if Parameter='Trt';
   _Scale_='MLE';
   _Stage_= 3;
   keep _Scale_ _Stage_ Parameter Estimate StdErr;
run;

proc print data=Parms_Count3;
   title 'Statistics Computed at Stage 3';
run;

Output 84.9.17 displays the test statistics at stage 3.

Output 84.9.17: Statistics Computed at Stage 3

Statistics Computed at Stage 3

Obs Parameter Estimate StdErr _Scale_ _Stage_
1 Trt 0.1205 0.0507 MLE 3


The following statements invoke the SEQTEST procedure and test for early stopping at stage 3:

ods graphics on;
proc seqtest Boundary=Test_Count2
             Parms(Testvar=Trt)=Parms_Count3
             betaboundary=nonbinding
             ;
ods output Test=Test_Count3;
run;
ods graphics off;

The Test Information table in Output 84.9.18 displays the boundary values, test statistic, and resulting action at each stage. The table shows that the test statistic 2.37437 is greater than the corresponding upper alpha boundary value, so the trial stops to reject the hypothesis.

Output 84.9.18: Sequential Test

The SEQTEST Procedure

Test Information (Standardized Z Scale)
Nonbinding Beta Boundary, Null Reference = 0
_Stage_   Alternative Boundary Values Test
Information Level Reference Upper Trt
Proportion Actual Upper Beta Alpha Estimate Action
1 0.2522 130.3756 1.71273 -1.03271 3.09023 0.20390 Continue
2 0.5015 259.3157 2.41549 0.46349 2.83297 0.43522 Accept Null (Nonbinding)
3 0.7512 388.3763 2.95609 1.36434 2.34036 2.37437 Reject Null
4 1.0000 517.0296 3.41074 2.03658 2.03658 .  


With ODS Graphics enabled, the Test Plot is displayed, as shown in Output 84.9.19. The plot displays boundary values of the design and the test statistics at the first two stages. As expected, the test statistic at stage 3 is in the Upper Rejection Region above the upper alpha boundary.

Output 84.9.19: Sequential Test Plot

Sequential Test Plot


After a trial is stopped, the Parameter Estimates table in Output 84.9.20 displays the stopping stage and the maximum likelihood estimate of the parameter. It also displays the p-value, median estimate, and confidence limits for the parameter that correspond to the observed statistic by using the specified sample space ordering.

Output 84.9.20: Parameter Estimates

Parameter Estimates
Stagewise Ordering
Parameter Stopping
Stage
MLE p-Value for
H0:Parm=0
Median
Estimate
Lower 95% CL
Trt 3 0.120482 0.0103 0.119371 0.03494


The MLE statistic at the stopping stage is the maximum likelihood estimate of the parameter and is biased. The computation of p-value, unbiased median estimate, and confidence limits depends on the ordering of the sample space $(k,z)$, where k is the stage number and z is the observed standardized Z statistic. By default (or equivalently if you specify ORDER=STAGEWISE), the stagewise ordering that uses counterclockwise ordering around the continuation region is used to compute the p-value, unbiased median estimate, and confidence limits. As expected, the p-value is less than 0.025, and the confidence interval does not contain the null reference zero.

See the section Available Sample Space Orderings in a Sequential Test for a detailed description of the stagewise ordering.