Prospective or retrospective power computations for GLM analyses

NOTE: Beginning in SAS 9.1, use PROC GLMPOWER in SAS/STAT Software to perform prospective power analyses.

PURPOSE:

Calculates the following power-related measures for retrospective and prospective analyses (see the DETAILS section below for definitions):

• Effect size
• Power for an effect test
• Adjusted power and confidence limits
• Least significant number
• Power of least significant number

REQUIREMENTS:

Only Version 6 or later of Base SAS software is required to run the %POWER macro. However, SAS/STAT software (Version 6 or later) is required to fit the desired GLM model.

USAGE:

Follow the instructions in the Downloads tab of this sample to save the %POWER macro definition. Replace the text within quotes in the following statement with the location of the %POWER macro definition file on your system. In your SAS program or in the SAS editor window, specify this statement to define the %POWER macro and make it available for use:

   %inc "<location of your file containing the POWER macro>";

Following this statement, you may call the %POWER macro. See the Results tab for an example.

Before using the %POWER macro, you must first run PROC GLM to fit the desired model and use the OUTSTAT= option to create the input data set for the %POWER macro.

These parameters are required:

DATA=

Name of the data set created by the OUTSTAT= option in the previous run of PROC GLM.

OUT=

Name of a new data set into which %POWER will store all calculations. In the output data set, values of .N indicate statistics that were not calculated and values of .U indicate that the macro was unable to calculate the statistic.

EFFECT=

Single effect from the MODEL statement in the previous run of PROC GLM. Power calculations are performed for the specified effect.

CALCS=POWER | ADJPOW | POWCI | LSN

Specifies one of the following statistic to be computed:

• POWER to request that power be computed and displayed
• ADJPOW to request that adjusted power be computed and displayed
• POWCI to request that a confidence interval for adjusted power be computed and displayed
• LSN to request that the least significant number be computed and displayed.

These optional parameters may also be specified:

SS=SS1 | SS2 | SS3 | SS4

Sums of squares to be used in the power calculations. The default is ss3. These sums of squares must have been computed by the previous run of PROC GLM.

ALPHA=

List of significance levels separated by spaces. The default value of 0.05.

N=

List of sample sizes separated by spaces. The default value is the observed sample size. Any values you specify are used in addition to the default.

SIGMA=

List of standard deviations separated by spaces. The default value is the observed standard deviation. Any values you specify are used in addition to the default.

DELTA=

List of effect sizes separated by spaces. The default value is the observed effect size. Any values you specify are used in addition to the default.

DETAILS:

Following are definitions of terms that are relevant to using the %POWER macro:

Prospective power analysis

Used in the planning phase of a designed experiment to determine how large the sample size must be to detect an effect of a given size (such as the minimum difference between treatment effects that is of practical value).

Retrospective power analysis

Used after the analysis of an experiment to determine the power of the conducted test.

Power

Is the probability that a false null hypothesis will be rejected. Ideally you would design your experiment to be as powerful as possible at detecting hypotheses of interest. Values of power range from 0 to 1, where values near 0 are low power and values near 1 are high power. Power is a function of the sample size (N), the effect size (delta), the root mean square error (sigma), and the significance level (alpha). The power tells you how likely your experiment is to detect a given difference, delta, at a given significance level, alpha. Power has the following characteristics:

• If the true value of the parameter is the hypothesized value, the power should be alpha. You do not want to reject the null hypothesis when it is true.
• If the true value of the parameters is not the hypothesized value, you want the power to be as large as possible.
• Power increases with the sample size.
• Power increases as variance decreases.
• Power increases as the true parameter gets farther from the hypothesized value.

Adjusted Power

Is for retrospective power analyses. The adjusted power is smaller than the power as it removes the bias associated with the noncentrality parameter. The noncentrality paramater is biased for any value other than zero. Because power is a function of population quantities that are not known, the usual practice is to substitute sample estimates in power calculations. If you regard these sample estimates as random, you can adjust them to have a more proper expectation. You can also construct a confidence interval for this adjusted power, though it is often very wide. The adjusted power and confidence interval can only be computed for your observed effect size, delta.

Least Significant Number (LSN)

Is the number of observations needed to reduce the variance of the estimates enough to achieve a significant result with the given values of alpha, sigma, and delta. If you need more data to achieve significance, the LSN helps tell you how many more. The LSN has the following characteristics:

• If the LSN is less than the actual sample size N, then the effect is significant. This means that you have more data than you need to detect the significance at the given alpha level.
• If the LSN is greater than the actual sample size N, the effect is not significant. In this case, if you believe that more data will show the same variance and structural results as the current sample, the LSN suggests how much data you would need to achieve significance.
• If the LSN is equal to N, then the p-value is equal to the significance level, alpha. The test is on the border of significance.
• Power calculated when N=LSN is always greater than or equal to 0.5.
• Power when N=LSN represents the power associated with using the sample size recommended by the LSN.

Noncentrality parameter, lambda

Lambda = Nδ²/sigma², where N is the total sample size, δ is the effect size (delta), and sigma² is the mean square error. Note that the noncentrality parameter is zero when the null hypothesis is true; that is, when the effect size is zero.

Effect Size, delta

Is estimated from the data as sqrt[SS(Hypothesis)/N]. The effect size can be thought of as the minimum difference in means that you want to detect divided by the total sample size.

LIMITATIONS:

The %POWER macro is appropriate for fixed-effect linear models fit by PROC GLM only. It is not appropriate for GLM models using the RANDOM, TEST, REPEATED, or MANOVA statements.

The %POWER macro can not be given a power value as input and report the required sample size.

No error checking is done. Be careful to correctly specify the macro parameters.

---

These sample files and code examples are provided by SAS Institute Inc. "as is" without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. Recipients acknowledge and agree that SAS Institute shall not be liable for any damages whatsoever arising out of their use of this material. In addition, SAS Institute will provide no support for the materials contained herein.

---

These sample files and code examples are provided by SAS Institute Inc. "as is" without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. Recipients acknowledge and agree that SAS Institute shall not be liable for any damages whatsoever arising out of their use of this material. In addition, SAS Institute will provide no support for the materials contained herein.

EXAMPLE 1: Retrospective Power Analysis

The data come from an experiment on leprosy (Snedecor & Cochran, 1967). The pre- and post-treatment leprosy scores, X and Y respectively, are recorded for ten patients on each of three drugs. The total sample size (N) is 30.

The following statements examine the power for testing the DRUG effect on the post-treatment response (Y) in the presence of the covariate X. The power analysis is requested at the four combinations of sample size (actual (30) or 60) and significance level (0.01 or 0.05).

        data drugs;
          input drug $ x y @@;
          datalines;
        A 11 6 A 8 0 A 5 2 A 14 8 A 19 11
        A 6 4 A 10 13 A 6 1 A 11 8 A 3 0
        B 6 0 B 6 2 B 7 3 B 8 1 B 18 18
        B 8 4 B 19 14 B 8 9 B 5 1 B 15 9
        C 16 13 C 13 10 C 11 18 C 9 5 C 21 23
        C 16 12 C 12 5 C 12 16 C 7 1 C 12 20
        ;

        proc print data=drugs;
          by drug;
          id drug;
          title 'DRUGS Data for Retrospective Power Calculations Macro';
          run;
        title;

        proc glm data=drugs outstat=drugout noprint;
          class drug;
          model y=drug x;
          run;

        proc print data=drugout;
          title 'OUTSTAT data set from GLM on DRUGS Data';
          run;
        title;

        /* Define the POWER macro */
        %inc "<location of your file containing the POWER macro>";
        
        %power (data=drugout,
                 out=drugpow,
              effect=drug,
               calcs=power adjpow powci lsn,
                  ss=ss3,
               alpha=0.01 0.05,
                   n=60);

RESULTS:

Following are the results from the %POWER macro. See the DETAILS section for descriptions of the displayed statistics. Note that value N indicates statistics that were not calculated and U indicates that the macro was unable to calculate the statistic.

Of the four combinations of significance level and sample size in this example, the greatest power for detecting a DRUG effect occurs at the larger significance level (0.05) and the larger sample size (60). Notice that the Least Significant Number (LSN) is not recomputed for sample sizes other than the observed size (30) since it is not a function of sample size. This suggests a sample size of 70 in order to detect a the DRUG effect with significance assuming the same variance and effect size. However, notice that the power at the suggested sample size is still less than 0.6 which may cause you to want a sample size larger than 70.

                           Power Calculation for effect DRUG
                                  Type 3 Sums of Squares

             Type I              Root Mean
              Error    Sample      Square     Effect    Power of    Adjusted
              Rate      Size       Error       Size       Test        Power

              0.01       30         4.01      1.5116     0.17573     0.06645
              0.01       60         4.01      1.5116     0.47671     0.30237
              0.05       30         4.01      1.5116     0.39681     0.19951
              0.05       60         4.01      1.5116     0.72205     0.54998

              Confidence     Confidence       Least        Power
              Interval:      Interval:     Significant      when
             Lower Limit    Upper Limit       Number       N=LSN

                 0.01         0.97761           70        0.57050
                 0.01         0.98257            N         N
                 0.05         0.98207           46        0.59037
                 0.05         0.98477            N         N

EXAMPLE 2: Prospective Power Analysis Using the %POWER Macro and PROC GLMPOWER

In a prospective power analysis you investigate the power for detecting specified group differences given one or more settings of the:

• overall sample size,
• estimate of variance, and
• significance level.

You can do prospective power analyses with the %POWER macro. Beginning in SAS 9, you can also use PROC GLMPOWER for prospective power analysis. Both are illustrated below.

You first create a data set of proposed means and sample sizes. For example, suppose you want to compare three groups. The table below displays the means you expect the groups to have and the number of observations you can collect from each.

       Data for Prospective Power Analysis

          Group        Mean       Count
            1           40          5
            2           45         10
            3           35         10

Create a data set containing this information as follows:

  data prospect;
    input group mean count;
    cards;
  1 40  5
  2 45 10
  3 35 10
  ;

To get the OUTSTAT= data set required by the %POWER macro, submit the following statements:

  proc glm data=prospect outstat=prosout;
    class group;
    freq count;
    model mean=group;
    run;

Below are the results from PROC GLM:

                                    The GLM Procedure

Dependent Variable: mean

Frequency: count

                                           Sum of
   Source                      DF         Squares     Mean Square    F Value    Pr > F

   Model                        2     500.0000000     250.0000000      Infty    <.0001

   Error                       22       0.0000000       0.0000000

   Corrected Total             24     500.0000000


                    R-Square     Coeff Var      Root MSE     mean Mean

                    1.000000             0             0      40.00000


   Source                      DF       Type I SS     Mean Square    F Value    Pr > F

   group                        2     500.0000000     250.0000000      Infty    <.0001


   Source                      DF     Type III SS     Mean Square    F Value    Pr > F

   group                        2     500.0000000     250.0000000      Infty    <.0001

Note that the Error Sum of Squares and the Mean Square Error are zero since the data input to PROC GLM contain no variation within the groups. Suppose in planning this study that you expect the standard deviation (sigma) to be in the range of 4 to 8. For the above sample sizes and means, the effect size (delta) is sqrt(SS(Group)/N) = sqrt(500/25) = 4.47. If the means were 35, 40, 40 with the same sample sizes, delta would be sqrt(150/25) = 2.45. You decide to look at two delta values, 2 and 5, which cover this range. In addition, you may want to see how much the significance level of the test affects the power. You select 0.01 and 0.05 as possible alpha levels.

You can now enter the above information into the %POWER macro and obtain the power and least significant number for each combination of alpha, sigma, and delta. Note that the delta value from the original sample size and means scenario (4.47) is used in addition to the specified delta values 2 and 5 resulting in 12 combinations. Because population values are used to compute an unbiased noncentrality parameter in a prospective power analysis, the adjusted power and confidence limits are not requested.

%power(data=prosout,
       out=powout,
       effect=group,
       calcs=power lsn,
       alpha=.01 .05,
       sigma=4.0 8.0,
       delta=2.0 5.0)

Below are the results from the %POWER macro:

                            Power Calculation for effect GROUP
                                  Type 3 Sums of Squares

      Type I              Root Mean                             Least        Power
       Error    Sample      Square     Effect    Power of    Significant      when
       Rate      Size       Error       Size       Test         Number       N=LSN

       0.01       25          4        4.4721     0.98180         13        0.61352
       0.01       25          4        2.0000     0.27786         42        0.57327
       0.01       25          4        5.0000     0.99618         12        0.66168
       0.01       25          8        4.4721     0.36851         35        0.58151
       0.01       25          8        2.0000     0.05112        153        0.57058
       0.01       25          8        5.0000     0.47960         29        0.58188
       0.05       25          4        4.4721     0.99808          9        0.63152
       0.05       25          4        2.0000     0.54066         28        0.59915
       0.05       25          4        5.0000     0.99976          8        0.62879
       0.05       25          8        4.4721     0.64218         23        0.59718
       0.05       25          8        2.0000     0.16588         99        0.58380
       0.05       25          8        5.0000     0.74515         19        0.59421

In the first scenario (alpha=.01, sigma=4, delta=4.47), the power for detecting group differences is .98. In fact, with a total sample size as low as 13 (the LSN), you would have power=.61 for detecting group differences. If you selected equal group sample sizes of 5, for a total sample size of 15, you would get a slightly higher power.

You can see from the table that power is high when sigma is small and the effect size is large. Power is related to the effect size (separation of means), standard deviation (sigma), significance level (alpha), and sample size. In general, there are four ways to increase power: increase effect size, decrease residual variance, increase sample size, or increase alpha.

The prospective power analysis using the proposed means can also be done using PROC GLMPOWER as follows:

proc glmpower;
   class group;
   model mean=group;
   power alpha=.01 .05
         stddev=4 8
         ntotal=25
         power=.;
   weight count;
   run;

Following are the results from PROC GLMPOWER. Note that the power values match those from the POWER macro for the proposed means.

                                  The GLMPOWER Procedure

                                 Fixed Scenario Elements

                         Dependent Variable                  mean
                         Source                             group
                         Weight Variable                    count
                         Total Sample Size                     25
                         Test Degrees of Freedom                2
                         Error Degrees of Freedom              22


                                      Computed Power

                                                 Std
                               Index    Alpha    Dev    Power

                                   1     0.01      4    0.982
                                   2     0.01      8    0.369
                                   3     0.05      4    0.998
                                   4     0.05      8    0.642

Right-click on the link below and select Save to save the %POWER macro definition to a file. It is recommended that you name the file power.sas.

Download and save power.sas

Operating System and Release Information