Sample Size and Power Calculations

Introduction

Figure 12.1: Sample Size Menu

When you are planning a study or experiment, you often need to know how many units to sample to obtain a certain power, or you may want to know the power you would obtain with a specific sample size. The power of a hypothesis test is the probability of rejecting the null hypothesis when the alternative hypothesis is true. With an inadequate sample size, you may not reach valid conclusions with your work; with an excessive sample size, you may waste valuable resources. Thus, performing sample size and power computations is often quite important.

The power and sample size calculations depend on the planned data analysis strategy. That is, if the primary hypothesis test is a two-sample t-test, then the power calculations must be based on that test. Otherwise, if the sample size calculations and data analyses are not aligned, the results may not be correct.

Determining sample size requirements ahead of the experiment is a prospective exercise. Then, you proceed to select the appropriate number of sampling units and perform data collection and analysis. However, power and sample size calculations are also useful retrospectively. For a given analysis, you may want to calculate what level of power you achieved or what sample size would have been needed for a given power.

Power and sample size calculations are a function of the specific alternative hypothesis of interest, in addition to other parameters. That is, the power results will vary depending on which value of the alternative hypothesis you specify, so sometimes it is useful to do these analyses for a range of values to see how sensitive the power analysis is to changes in the alternative hypothesis value. Often, you produce plots of power versus sample size, called power curves, to see how sample size and power affect each other.

The Sample Size tasks provide prospective sample size and power calculations for several types of analyses: t-tests, confidence intervals, and tests of equivalence. Each of these calculations is available for one-sample, paired-sample, and two-sample study designs. Power and sample size calculations are also available for the one-way ANOVA design. Multiple parameter values can be input, and results and power curves are produced for each combination of values. Note that retrospective power computations are also available in a number of the statistical tasks in the Analyst Application such as the Hypothesis Test, Regression, and ANOVA tasks.

Top of Page