The SEQDESIGN Procedure

Example 89.2 Creating a One-Sided O’Brien-Fleming Design

This example demonstrates a group sequential design for a clinical study. A clinic is conducting a study on the effect of vitamin C supplements in treating flu symptoms. The study groups consist of patients in the clinic with their first sign of flu symptoms within the last 24 hours. These individuals are randomly assigned to either the control group, which receives the placebo pills, or the treatment group, which receives large doses of vitamin C supplements. At the end of a five-day period, the flu symptoms of each individual are recorded.

Suppose that from past experience, 60% of individuals experiencing flu symptoms have the symptoms disappeared within five days. The clinic wants to detect a 75% symptoms disappearance with a high probability in the trial. A test that compares the proportions directly is to specify a null hypothesis $H_{0}: \theta = p_{a} - p_{b}= 0$ with a Type I error probability level $\alpha =0.025$ , where $p_{a}$ and $p_{b}$ are the proportions of symptoms’ disappearance in the treatment group and control group, respectively. A one-sided alternative $H_{1}: \theta > 0$ is also specified with a power of $1-\beta = 0.80$ at $H_{1}: \theta = 0.15$ .

For a one-sided fixed-sample design, the critical value for the standardized Z test statistic is given by $C_{\alpha }= \Phi ^{-1} (1-\alpha )= 1.96$ . That is, at the end of study, if the test statistic $z \geq C_{\alpha }$ , then the null hypothesis is rejected and the efficacy of vitamin C supplements is declared. Otherwise, the null hypothesis is not rejected and the effect of vitamin C supplements is not significant.

To achieve a $1-\beta = 0.80$ power at $H_{1}: \theta = 0.15$ for a fixed-sample design, the information required is given by

$I_{0} = \frac{ ( \Phi ^{-1}(1-\alpha ) + \Phi ^{-1}(1-\beta ) )^{2} }{ {0.15}^{2} } = \frac{ (1.96 + 0.8416)^{2} }{ 0.0225 } = 348.84$

With an equal sample size on the treatment and control groups, $N_{a}= N_{b}$ , the sample size required for each group under $H_{1}$ is computed from the information $I_{0}$ :

$N_{a} = N_{b} = \left( p_{1a} \, (1-p_{1a}) + p_{1b} \, (1-p_{1b}) \right) \, I_{0}$

where $p_{1a}= 0.75$ and $p_{1b}= 0.60$ are proportions in the treatment and control groups under $H_{1}$ . That is,

$N_{a} = N_{b} = \left( 0.75 \times 0.25 + 0.6 \times 0.4 \right) \times 348.84 = 149.13$

Thus, 150 individuals are required for each group in the fixed-sample study. See the section Test for the Difference between Two Binomial Proportions for a detailed derivation of these required sample sizes.

Instead of a fixed-sample design for the trial, a group sequential design is used to stop the trial early for ethical concerns of possible harm or an unexpected strong efficacy outcome of the new drug. It can also save time and resources in the process. The following statements invoke the SEQDESIGN procedure and request a four-stage group sequential design that uses an O’Brien-Fleming method for normally distributed statistics. The design uses a one-sided alternative hypothesis $H_{1}$ with early stopping to reject or accept $H_{0}$ .

ods graphics on;
proc seqdesign altref=0.15
               ;
   OneSidedOBrienFleming: design nstages=4
                          method=obf
                          alt=upper   stop=both
                          alpha=0.025 beta=0.20
                          ;
   samplesize model(ceiladjdesign=include)
              =twosamplefreq(nullprop=0.6 test=prop);
ods output AdjustedBoundary=Bnd_Prop;
run;
ods graphics off;

At each interim stage in a sequential design, a hypothesis can be rejected, accepted, or continued to the next time point. The STOP=BOTH option specifies early stopping to reject or accept the null hypothesis. The CEILADJDESIGN=INCLUDE option creates an additional design that corresponds to the integer-valued sample sizes at the stages. The option adds this ceiling-adjusted design information to the design information table and displays the ceiling-adjusted design boundary information in a separate boundary information table.

The ADJUSTEDBOUNDARY=BND_PROP option in the ODS OUTPUT statement creates an output data set that contains the resulting ceiling-adjusted boundary information. After the actual data from the clinical trial are collected and analyzed at each stage by a procedure such as PROC GENMOD, you can use the SEQTEST procedure to test the resulting statistics at stage 1 by using the boundary information that is stored in the BOUND_PROP data set.

The "Design Information," "Method Information," and "Boundary Information" tables are displayed by default. The "Design Information" table in Output 89.2.1 displays design specifications and derived statistics such as power and maximum information. If you specify an alternative reference (for example, ALTREF=0.15), the maximum information $I_{X}$ is derived.

Output 89.2.1: Design Information

The SEQDESIGN Procedure

Design: OneSidedOBrienFleming

Design Information
Statistic Distribution	Normal
Boundary Scale	Standardized Z
Alternative Hypothesis	Upper
Early Stop	Accept/Reject Null
Method	O'Brien-Fleming
Boundary Key	Both
Alternative Reference	0.15
Number of Stages	4
Alpha	0.025
Beta	0.2
Power	0.8
Max Information (Percent of Fixed Sample)	111.5566
Max Information	389.1522
Null Ref ASN (Percent of Fixed Sample)	55.96565
Alt Ref ASN (Percent of Fixed Sample)	79.72258
Adj Design Alpha	0.025
Adj Design Beta	0.19835
Adj Design Power	0.80165
Adj Design Max Information (Percent of Fixed Sample)	111.5115
Adj Design Max Information	390.6433
Adj Design Null Ref ASN (Percent of Fixed Sample)	56.04399
Adj Design Alt Ref ASN (Percent of Fixed Sample)	79.62143

The Max Information (Percent Fixed-Sample) is the ratio in percentage between the maximum information for the group sequential design and the information required for a corresponding fixed-sample design:

$100 \times \frac{I_{X}}{I_{0}} = 100 \times \frac{389.15}{348.84} = 111.56$

That is, if the group sequential trial does not stop at any interim stages, the information needed is 11.56% more than is needed for the corresponding fixed-sample design. For a two-sample test for binomial proportions, the information is proportional to the sample size. Thus, 11.56% more observations are needed for the group sequential trial.

The Null Ref ASN (Percent Fixed-Sample) is the ratio in percentage between the expected sample size required under the null hypothesis for the group sequential design and the sample size required for the corresponding fixed-sample design. With a ratio of 56%, the expected sample size for the group sequential trial under the null hypothesis is 56% of the sample size in the corresponding fixed-sample design.

Similarly, the Alt Ref ASN (Percent Fixed-Sample) is the ratio in percentage between the expected sample size required under the alternative hypothesis for the group sequential design and the sample size required for the corresponding fixed-sample design. With a ratio of 79.7%, the expected sample size for the group sequential trial under the alternative hypothesis is 79.7% of the sample size in the corresponding fixed-sample design.

The CEILADJDESIGN=INCLUDE option creates an adjusted design that has ceiling sample sizes at the stages. The "Design Information" table also includes design information for this adjusted design. Only the Type I error level is maintained for the adjusted design when BOUNDARYKEY=BOTH (the default) in the DESIGN statement.

For a one-sided design that has an upper alternative and early stopping to reject or accept the null hypothesis, upper $\alpha$ and $\beta$ boundaries are created. The "Method Information" table in Output 89.2.2 displays the Type I error probability $\alpha$ , the Type II error probability $\beta$ , and the derived drift parameter. The drift parameter is the standardized reference improvement between the alternative and null hypotheses at the final stage. It is also the standardized alternative reference at the final stage if the null reference is zero.

Output 89.2.2: Method Information

Method Information
Boundary	Method	Alpha	Beta	Unified Family			Alternative Reference	Drift
Boundary	Method	Alpha	Beta	Rho	Tau	C	Alternative Reference	Drift
Upper Alpha	O'Brien-Fleming	0.02500	.	0.5	0	1.94947	0.15	2.959041
Upper Beta	O'Brien-Fleming	.	0.20000	0.5	0	1.00957	0.15	2.959041

With the METHOD=OBF option, the O’Brien-Fleming method is used for each boundary. The O’Brien-Fleming method is one of the unified family methods, and the "Method Information" table displays the corresponding parameter $\rho$ in the unified family method. The table also displays the critical values $C_{\alpha }=1.9495$ for the $\alpha$ boundary and $C_{\beta }=1.0096$ for the $\beta$ boundary. These critical values are used to create the boundary values.

The "Boundary Information" table in Output 89.2.3 displays information level, alternative reference, and boundary values at each stage. By default (or equivalently if you specify BOUNDARYSCALE=STDZ), the alternative references and boundary values are displayed with the standardized Z statistic scale. The resulting standardized alternative reference at stage k is given by $\theta _1 \sqrt {I_ k}$ , where $\theta _1$ is the alternative reference and $I_ k$ is the information level at stage k, $k= 1, 2, 3, 4$ .

Output 89.2.3: Boundary Information

Boundary Information (Standardized Z Scale) Null Reference = 0
_Stage_				Alternative	Boundary Values
	Information Level			Reference	Upper
	Proportion	Actual	N	Upper	Beta	Alpha
1	0.2500	97.28805	83.18128	1.47952	-0.53963	3.89893
2	0.5000	194.5761	166.3626	2.09236	0.66460	2.75696
3	0.7500	291.8641	249.5438	2.56260	1.39685	2.25105
4	1.0000	389.1522	332.7251	2.95904	1.94947	1.94947

By default (or equivalently if you specify INFO=EQUAL), equally spaced information levels are used. An information proportion is the proportion of maximum information available at each stage. With the derived maximum information, the actual information level at each stage is also displayed. With the SAMPLESIZE statement, the required sample size N is also displayed under the heading "Information Level."

At each interim stage, if the standardized Z test statistic is larger than or equal to the corresponding upper $\alpha$ boundary, then the hypothesis $H_{0}: \theta = 0$ is rejected. If the test statistic is less than the corresponding upper $\beta$ boundary, then the trial is stopped and the hypothesis $H_{0}$ is accepted. Otherwise, the process continues to the next stage. At the final stage, stage 4, the trial stops and the hypothesis $H_{0}$ is rejected if the standardized Z statistic $Z_{4} \geq 1.94947$ . Otherwise, the trial is accepted.

With ODS Graphics enabled, a detailed boundary plot with the rejection and acceptance regions is displayed, as shown in Output 89.2.4.

Output 89.2.4: Boundary Plot

The horizontal axis indicates the information levels for the design. The stages are indicated by vertical lines with accompanying stage numbers. If at any stage a test statistic is in a rejection region, the trial stops and the hypothesis is rejected. If a test statistic is in an acceptance region, then the trial also stops and the hypothesis is accepted. If the statistic is not in a rejection region or an acceptance region, the trial continues to the next stage. The boundary plot also displays the information level and the critical value for the corresponding fixed-sample design.

The SEQDESIGN procedure derives the drift parameter $\theta _{1} \sqrt {I_{X}}$ , where $\theta _{1}$ is the alternative reference and $I_{X}$ is the maximum information. If either $\theta _{1}$ or $I_{X}$ is specified, the other can be derived. With the SAMPLESIZE statement, the maximum information is used to compute the required sample size for the study.

The "Sample Size Summary" table in Output 89.2.5 displays parameters for the sample size computation. With the MODEL=TWOSAMPLEFREQ( NULLPROP=0.6 TEST=PROP) option in the SAMPLESIZE statement, the total sample size in each group for testing the difference between two proportions is computed. By default (or equivalently if you specify REF=PROP in the MODEL=TWOSAMPLEFREQ option), the required sample sizes are computed under the alternative hypothesis. That is,

$N_{a} = N_{b} =\left( \, p_{1a} \, (1-p_{1a}) + p_{1b} \, (1-p_{1b}) \, \right) \, I_{X}$

where $p_{1b}=0.60$ and $p_{1a}=p_{1b}+\theta _{1}=0.75$ are the proportions in the control and treatment groups, respectively, under the alternative hypothesis $H_{1}$ . See the section Test for the Difference between Two Binomial Proportions for a detailed description of these parameters.

Output 89.2.5: Sample Size Summary

Sample Size Summary
Test	Two-Sample Proportions
Null Proportion	0.6
Proportion (Group A)	0.75
Test Statistic	Z for Proportion
Reference Proportions	Alt Ref
Max Sample Size	332.7251
Expected Sample Size (Null Ref)	166.9213
Expected Sample Size (Alt Ref)	237.7779

The "Sample Sizes (N)" table in Output 89.2.6 displays the required sample sizes at each stage, in both fractional and integer numbers. The derived fractional sample sizes are under the heading "Fractional N." These sample sizes are rounded up to integers under the heading "Ceiling N." In practice, integer sample sizes are used, and the resulting information levels increase slightly. Thus, 42, 84, 125, and 167 individuals are needed in each group for the four stages, respectively.

Output 89.2.6: Derived Sample Sizes

Sample Sizes (N) Two-Sample Z Test for Proportion Difference
_Stage_	Fractional N				Ceiling N
_Stage_	N	N(Grp 1)	N(Grp 2)	Information	N	N(Grp 1)	N(Grp 2)	Information
1	83.18	41.59	41.59	97.2880	84	42	42	98.2456
2	166.36	83.18	83.18	194.6	168	84	84	196.5
3	249.54	124.77	124.77	291.9	250	125	125	292.4
4	332.73	166.36	166.36	389.2	334	167	167	390.6

When CEILADJDESIGN=INCLUDE in the SAMPLESIZE statement, the "Ceiling-Adjusted Design Boundary Information" table in Output 89.2.7 displays boundary information, similar to Output 89.2.3 but with ceiling sample sizes at the stages.

Output 89.2.7: Adjusted O’Brien-Fleming Boundary Information

Ceiling-Adjusted Design Boundary Information (Standardized Z Scale) Null Reference = 0
_Stage_				Alternative	Boundary Values
	Information Level			Reference	Upper
	Proportion	Actual	N	Upper	Beta	Alpha
1	0.2515	98.24561	84	1.48678	-0.53660	3.88835
2	0.5030	196.4912	168	2.10263	0.67188	2.74948
3	0.7485	292.3977	250	2.56495	1.39208	2.25390
4	1.0000	390.6433	334	2.96470	1.94999	1.94999

Because the sample sizes have integer values, the information levels at the stages are not equally spaced in this example, but the design is still an O’Brien-Fleming design.

The CEILADJDESIGN=INCLUDE option displays the adjusted-design information in the "Design Information" table and the adjusted boundary information in the "Ceiling-Adjusted Design Boundary Information" table. You can use the SEQTEST procedure to create other tables, such as error spending tables. For example, when you specify ADJUSTEDBOUNDARY=BND_PROP in the ODS OUTPUT statement, the following statements create the error-spending information table for the adjusted design:

proc seqtest Boundary=Bnd_Prop
             errspend
             ;
run;