Sample 25011: Prospective or retrospective power computations for GLM analyses
 |  |  |  |  |  |
Prospective or retrospective power computations for GLM analyses
| Contents: | Purpose / Requirements / Usage / Details / Limitations |
NOTE: Beginning in SAS 9.1, use PROC GLMPOWER in SAS/STAT Software to perform prospective power analyses.
- PURPOSE:
- Calculates the following power-related measures for retrospective and prospective analyses (see the DETAILS section below for definitions):
- Effect size
- Power for an effect test
- Adjusted power and confidence limits
- Least significant number
- Power of least significant number
- REQUIREMENTS:
- Only Version 6 or later of Base SAS software is required to run the %POWER macro. However, SAS/STAT software (Version 6 or later) is required to fit the desired GLM model.
- USAGE:
- Follow the instructions in the Downloads tab of this sample to save the %POWER macro definition. Replace the text within quotes in the following statement with the location of the %POWER macro definition file on your system. In your SAS program or in the SAS editor window, specify this statement to define the %POWER macro and make it available for use:
%inc "<location of your file containing the POWER macro>";
Following this statement, you may call the %POWER macro. See the Results tab for an example.
Before using the %POWER macro, you must first run PROC GLM to fit the desired model and use the OUTSTAT= option to create the input data set for the %POWER macro.
These parameters are required:
- DATA=
- Name of the data set created by the OUTSTAT= option in the previous run of PROC GLM.
- OUT=
- Name of a new data set into which %POWER will store all calculations. In the output data set, values of .N indicate statistics that were not calculated and values of .U indicate that the macro was unable to calculate the statistic.
- EFFECT=
- Single effect from the MODEL statement in the previous run of PROC GLM. Power calculations are performed for the specified effect.
- CALCS=POWER | ADJPOW | POWCI | LSN
- Specifies one of the following statistic to be computed:
- POWER to request that power be computed and displayed
- ADJPOW to request that adjusted power be computed and displayed
- POWCI to request that a confidence interval for adjusted power be computed and displayed
- LSN to request that the least significant number be computed and displayed.
These optional parameters may also be specified:
- SS=SS1 | SS2 | SS3 | SS4
- Sums of squares to be used in the power calculations. The default is ss3. These sums of squares must have been computed by the previous run of PROC GLM.
- ALPHA=
- List of significance levels separated by spaces. The default value of 0.05.
- N=
- List of sample sizes separated by spaces. The default value is the observed sample size. Any values you specify are used in addition to the default.
- SIGMA=
- List of standard deviations separated by spaces. The default value is the observed standard deviation. Any values you specify are used in addition to the default.
- DELTA=
- List of effect sizes separated by spaces. The default value is the observed effect size. Any values you specify are used in addition to the default.
- DETAILS:
- Following are definitions of terms that are relevant to using the %POWER macro:
- Prospective power analysis
- Used in the planning phase of a designed experiment to determine how large the sample size must be to detect an effect of a given size (such as the minimum difference between treatment effects that is of practical value).
- Retrospective power analysis
- Used after the analysis of an experiment to determine the power of the conducted test.
- Power
- Is the probability that a false null hypothesis will be rejected. Ideally you would design your experiment to be as powerful as possible at detecting hypotheses of interest. Values of power range from 0 to 1, where values near 0 are low power and values near 1 are high power. Power is a function of the sample size (N), the effect size (delta), the root mean square error (sigma), and the significance level (alpha). The power tells you how likely your experiment is to detect a given difference, delta, at a given significance level, alpha. Power has the following characteristics:
- If the true value of the parameter is the hypothesized value, the power should be alpha. You do not want to reject the null hypothesis when it is true.
- If the true value of the parameters is not the hypothesized value, you want the power to be as large as possible.
- Power increases with the sample size.
- Power increases as variance decreases.
- Power increases as the true parameter gets farther from the hypothesized value.
- Adjusted Power
- Is for retrospective power analyses. The adjusted power is smaller than the power as it removes the bias associated with the noncentrality parameter. The noncentrality paramater is biased for any value other than zero. Because power is a function of population quantities that are not known, the usual practice is to substitute sample estimates in power calculations. If you regard these sample estimates as random, you can adjust them to have a more proper expectation. You can also construct a confidence interval for this adjusted power, though it is often very wide. The adjusted power and confidence interval can only be computed for your observed effect size, delta.
- Least Significant Number (LSN)
- Is the number of observations needed to reduce the variance of the estimates enough to achieve a significant result with the given values of alpha, sigma, and delta. If you need more data to achieve significance, the LSN helps tell you how many more. The LSN has the following characteristics:
- If the LSN is less than the actual sample size N, then the effect is significant. This means that you have more data than you need to detect the significance at the given alpha level.
- If the LSN is greater than the actual sample size N, the effect is not significant. In this case, if you believe that more data will show the same variance and structural results as the current sample, the LSN suggests how much data you would need to achieve significance.
- If the LSN is equal to N, then the p-value is equal to the significance level, alpha. The test is on the border of significance.
- Power calculated when N=LSN is always greater than or equal to 0.5.
- Power when N=LSN represents the power associated with using the sample size recommended by the LSN.
- Noncentrality parameter, lambda
- Lambda = Nδ2/sigma2, where N is the total sample size, δ is the effect size (delta), and sigma2 is the mean square error. Note that the noncentrality parameter is zero when the null hypothesis is true; that is, when the effect size is zero.
- Effect Size, delta
- Is estimated from the data as sqrt[SS(Hypothesis)/N]. The effect size can be thought of as the minimum difference in means that you want to detect divided by the total sample size.
- LIMITATIONS:
- The %POWER macro is appropriate for fixed-effect linear models fit by PROC GLM only. It is not appropriate for GLM models using the RANDOM, TEST, REPEATED, or MANOVA statements.
The %POWER macro can not be given a power value as input and report the required sample size.
No error checking is done. Be careful to correctly specify the macro parameters.
These sample files and code examples are provided by SAS Institute Inc. "as is" without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. Recipients acknowledge and agree that SAS Institute shall not be liable for any damages whatsoever arising out of their use of this material. In addition, SAS Institute will provide no support for the materials contained herein.
These sample files and code examples are provided by SAS Institute Inc. "as is" without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. Recipients acknowledge and agree that SAS Institute shall not be liable for any damages whatsoever arising out of their use of this material. In addition, SAS Institute will provide no support for the materials contained herein.
- EXAMPLE 1: Retrospective Power Analysis
-
The data come from an experiment on leprosy (Snedecor & Cochran, 1967).
The pre- and post-treatment leprosy scores, X and Y respectively, are
recorded for ten patients on each of three drugs. The total sample
size (N) is 30.
The following statements examine the power for testing the DRUG effect on the post-treatment response (Y) in the presence of the covariate X. The power analysis is requested at the four combinations of sample size (actual (30) or 60) and significance level (0.01 or 0.05).
data drugs; input drug $ x y @@; datalines; A 11 6 A 8 0 A 5 2 A 14 8 A 19 11 A 6 4 A 10 13 A 6 1 A 11 8 A 3 0 B 6 0 B 6 2 B 7 3 B 8 1 B 18 18 B 8 4 B 19 14 B 8 9 B 5 1 B 15 9 C 16 13 C 13 10 C 11 18 C 9 5 C 21 23 C 16 12 C 12 5 C 12 16 C 7 1 C 12 20 ; proc print data=drugs; by drug; id drug; title 'DRUGS Data for Retrospective Power Calculations Macro'; run; title; proc glm data=drugs outstat=drugout noprint; class drug; model y=drug x; run; proc print data=drugout; title 'OUTSTAT data set from GLM on DRUGS Data'; run; title; /* Define the POWER macro */ %inc "<location of your file containing the POWER macro>"; %power (data=drugout, out=drugpow, effect=drug, calcs=power adjpow powci lsn, ss=ss3, alpha=0.01 0.05, n=60); - RESULTS:
-
Following are the results from the %POWER macro. See the DETAILS section for descriptions of the
displayed statistics. Note that value N indicates statistics that
were not calculated and U indicates that the macro was unable
to calculate the statistic.
Of the four combinations of significance level and sample size in this example, the greatest power for detecting a DRUG effect occurs at the larger significance level (0.05) and the larger sample size (60). Notice that the Least Significant Number (LSN) is not recomputed for sample sizes other than the observed size (30) since it is not a function of sample size. This suggests a sample size of 70 in order to detect a the DRUG effect with significance assuming the same variance and effect size. However, notice that the power at the suggested sample size is still less than 0.6 which may cause you to want a sample size larger than 70.
Power Calculation for effect DRUG Type 3 Sums of Squares Type I Root Mean Error Sample Square Effect Power of Adjusted Rate Size Error Size Test Power 0.01 30 4.01 1.5116 0.17573 0.06645 0.01 60 4.01 1.5116 0.47671 0.30237 0.05 30 4.01 1.5116 0.39681 0.19951 0.05 60 4.01 1.5116 0.72205 0.54998 Confidence Confidence Least Power Interval: Interval: Significant when Lower Limit Upper Limit Number N=LSN 0.01 0.97761 70 0.57050 0.01 0.98257 N N 0.05 0.98207 46 0.59037 0.05 0.98477 N N - EXAMPLE 2: Prospective Power Analysis Using the %POWER Macro and PROC GLMPOWER
-
In a prospective power analysis you investigate the power for detecting
specified group differences given one or more settings of the:
- overall sample size,
- estimate of variance, and
- significance level.
You can do prospective power analyses with the %POWER macro. Beginning in SAS 9, you can also use PROC GLMPOWER for prospective power analysis. Both are illustrated below.
You first create a data set of proposed means and sample sizes. For example, suppose you want to compare three groups. The table below displays the means you expect the groups to have and the number of observations you can collect from each.
Data for Prospective Power Analysis Group Mean Count 1 40 5 2 45 10 3 35 10Create a data set containing this information as follows:
data prospect; input group mean count; cards; 1 40 5 2 45 10 3 35 10 ;To get the OUTSTAT= data set required by the %POWER macro, submit the following statements:
proc glm data=prospect outstat=prosout; class group; freq count; model mean=group; run;Below are the results from PROC GLM:
The GLM Procedure Dependent Variable: mean Frequency: count Sum of Source DF Squares Mean Square F Value Pr > F Model 2 500.0000000 250.0000000 Infty <.0001 Error 22 0.0000000 0.0000000 Corrected Total 24 500.0000000 R-Square Coeff Var Root MSE mean Mean 1.000000 0 0 40.00000 Source DF Type I SS Mean Square F Value Pr > F group 2 500.0000000 250.0000000 Infty <.0001 Source DF Type III SS Mean Square F Value Pr > F group 2 500.0000000 250.0000000 Infty <.0001Note that the Error Sum of Squares and the Mean Square Error are zero since the data input to PROC GLM contain no variation within the groups. Suppose in planning this study that you expect the standard deviation (sigma) to be in the range of 4 to 8. For the above sample sizes and means, the effect size (delta) is sqrt(SS(Group)/N) = sqrt(500/25) = 4.47. If the means were 35, 40, 40 with the same sample sizes, delta would be sqrt(150/25) = 2.45. You decide to look at two delta values, 2 and 5, which cover this range. In addition, you may want to see how much the significance level of the test affects the power. You select 0.01 and 0.05 as possible alpha levels.
You can now enter the above information into the %POWER macro and obtain the power and least significant number for each combination of alpha, sigma, and delta. Note that the delta value from the original sample size and means scenario (4.47) is used in addition to the specified delta values 2 and 5 resulting in 12 combinations. Because population values are used to compute an unbiased noncentrality parameter in a prospective power analysis, the adjusted power and confidence limits are not requested.
%power(data=prosout, out=powout, effect=group, calcs=power lsn, alpha=.01 .05, sigma=4.0 8.0, delta=2.0 5.0)Below are the results from the %POWER macro:
Power Calculation for effect GROUP Type 3 Sums of Squares Type I Root Mean Least Power Error Sample Square Effect Power of Significant when Rate Size Error Size Test Number N=LSN 0.01 25 4 4.4721 0.98180 13 0.61352 0.01 25 4 2.0000 0.27786 42 0.57327 0.01 25 4 5.0000 0.99618 12 0.66168 0.01 25 8 4.4721 0.36851 35 0.58151 0.01 25 8 2.0000 0.05112 153 0.57058 0.01 25 8 5.0000 0.47960 29 0.58188 0.05 25 4 4.4721 0.99808 9 0.63152 0.05 25 4 2.0000 0.54066 28 0.59915 0.05 25 4 5.0000 0.99976 8 0.62879 0.05 25 8 4.4721 0.64218 23 0.59718 0.05 25 8 2.0000 0.16588 99 0.58380 0.05 25 8 5.0000 0.74515 19 0.59421In the first scenario (alpha=.01, sigma=4, delta=4.47), the power for detecting group differences is .98. In fact, with a total sample size as low as 13 (the LSN), you would have power=.61 for detecting group differences. If you selected equal group sample sizes of 5, for a total sample size of 15, you would get a slightly higher power.
You can see from the table that power is high when sigma is small and the effect size is large. Power is related to the effect size (separation of means), standard deviation (sigma), significance level (alpha), and sample size. In general, there are four ways to increase power: increase effect size, decrease residual variance, increase sample size, or increase alpha.
The prospective power analysis using the proposed means can also be done using PROC GLMPOWER as follows:
proc glmpower; class group; model mean=group; power alpha=.01 .05 stddev=4 8 ntotal=25 power=.; weight count; run;Following are the results from PROC GLMPOWER. Note that the power values match those from the POWER macro for the proposed means.
The GLMPOWER Procedure Fixed Scenario Elements Dependent Variable mean Source group Weight Variable count Total Sample Size 25 Test Degrees of Freedom 2 Error Degrees of Freedom 22 Computed Power Std Index Alpha Dev Power 1 0.01 4 0.982 2 0.01 8 0.369 3 0.05 4 0.998 4 0.05 8 0.642
Right-click on the link below and select Save to save
the %POWER macro definition
to a file. It is recommended that you name the file
power.sas.
| Type: | Sample |
| Topic: | SAS Reference ==> Procedures ==> GLMPOWER SAS Reference ==> Procedures ==> GLM Analytics ==> Power and Sample Size Analytics ==> Analysis of Variance |
| Date Modified: | 2017-07-12 11:34:02 |
| Date Created: | 2005-01-13 15:03:33 |
Operating System and Release Information
| Product Family | Product | Host | SAS Release | |
| Starting | Ending | |||
| SAS System | SAS/STAT | z/OS | ||
| z/OS 64-bit | ||||
| OpenVMS VAX | ||||
| Microsoft® Windows® for 64-Bit Itanium-based Systems | ||||
| Microsoft Windows Server 2003 Datacenter 64-bit Edition | ||||
| Microsoft Windows Server 2003 Enterprise 64-bit Edition | ||||
| Microsoft Windows XP 64-bit Edition | ||||
| Microsoft® Windows® for x64 | ||||
| OS/2 | ||||
| Microsoft Windows 8 Enterprise 32-bit | ||||
| Microsoft Windows 8 Enterprise x64 | ||||
| Microsoft Windows 8 Pro 32-bit | ||||
| Microsoft Windows 8 Pro x64 | ||||
| Microsoft Windows 8.1 Enterprise 32-bit | ||||
| Microsoft Windows 8.1 Enterprise x64 | ||||
| Microsoft Windows 8.1 Pro 32-bit | ||||
| Microsoft Windows 8.1 Pro x64 | ||||
| Microsoft Windows 10 | ||||
| Microsoft Windows 95/98 | ||||
| Microsoft Windows 2000 Advanced Server | ||||
| Microsoft Windows 2000 Datacenter Server | ||||
| Microsoft Windows 2000 Server | ||||
| Microsoft Windows 2000 Professional | ||||
| Microsoft Windows NT Workstation | ||||
| Microsoft Windows Server 2003 Datacenter Edition | ||||
| Microsoft Windows Server 2003 Enterprise Edition | ||||
| Microsoft Windows Server 2003 Standard Edition | ||||
| Microsoft Windows Server 2003 for x64 | ||||
| Microsoft Windows Server 2008 | ||||
| Microsoft Windows Server 2008 R2 | ||||
| Microsoft Windows Server 2008 for x64 | ||||
| Microsoft Windows Server 2012 Datacenter | ||||
| Microsoft Windows Server 2012 R2 Datacenter | ||||
| Microsoft Windows Server 2012 R2 Std | ||||
| Microsoft Windows Server 2012 Std | ||||
| Microsoft Windows XP Professional | ||||
| Windows 7 Enterprise 32 bit | ||||
| Windows 7 Enterprise x64 | ||||
| Windows 7 Home Premium 32 bit | ||||
| Windows 7 Home Premium x64 | ||||
| Windows 7 Professional 32 bit | ||||
| Windows 7 Professional x64 | ||||
| Windows 7 Ultimate 32 bit | ||||
| Windows 7 Ultimate x64 | ||||
| Windows Millennium Edition (Me) | ||||
| Windows Vista | ||||
| Windows Vista for x64 | ||||
| 64-bit Enabled AIX | ||||
| 64-bit Enabled HP-UX | ||||
| 64-bit Enabled Solaris | ||||
| ABI+ for Intel Architecture | ||||
| AIX | ||||
| HP-UX | ||||
| HP-UX IPF | ||||
| IRIX | ||||
| Linux | ||||
| Linux for x64 | ||||
| Linux on Itanium | ||||
| OpenVMS Alpha | ||||
| OpenVMS on HP Integrity | ||||
| Solaris | ||||
| Solaris for x64 | ||||
| Tru64 UNIX | ||||