Introduction to Power and Sample Size Analysis: Standard Hypothesis Tests

Standard Hypothesis Tests

In statistical hypothesis testing, you typically express the belief that some effect exists in a population by specifying an alternative hypothesis $\text{[math]}$ . You state a null hypothesis $\text{[math]}$ as the assertion that the effect does not exist and attempt to gather evidence to reject $\text{[math]}$ in favor of $\text{[math]}$ . Evidence is gathered in the form of sample data, and a statistical test is used to assess $\text{[math]}$ . If $\text{[math]}$ is rejected but there really is no effect, this is called a Type I error. The probability of a Type I error is usually designated "alpha" or $\text{[math]}$ , and statistical tests are designed to ensure that $\text{[math]}$ is suitably small (for example, less than 0.05).

If there is an effect in the population but $\text{[math]}$ is not rejected in the statistical test, then a Type II error has been committed. The probability of a Type II error is usually designated "beta" or $\text{[math]}$ . The probability $\text{[math]}$ of avoiding a Type II error (that is, correctly rejecting $\text{[math]}$ and achieving statistical significance) is called the power of the test.

Most, but not all, of the power analyses in the GLMPOWER and POWER procedures are based on such standard hypothesis tests.