The SURVEYFREQ Procedure

Confidence Limits for Proportions

If you specify the CL option in the TABLES statement, PROC SURVEYFREQ computes confidence limits for the proportions in the frequency and crosstabulation tables.

By default, PROC SURVEYFREQ computes Wald ("linear") confidence limits if you do not specify an alternative confidence limit type with the CL(TYPE=) option. In addition to Wald confidence limits, the following types of design-based confidence limits are available for proportions: modified Clopper-Pearson (exact), modified Wilson (score), and logit confidence limits.

PROC SURVEYFREQ also provides the CL(PSMALL) option, which uses the alternative confidence limit type for extreme (small or large) proportions and uses the Wald confidence limits for all other proportions (not extreme). For the default PSMALL= value of 0.25, the procedure computes Wald confidence limits for proportions between 0.25 and 0.75 and computes the alternative confidence limit type for proportions that are outside of this range. For more information, see Curtin et al. (2006).

For information about design-based confidence limits for proportions (including comparisons of their performance), see Korn and Graubard (1999), Korn and Graubard (1998), Curtin et al. (2006), and Sukasih and Jang (2005). For more information about binomial confidence limits, see Brown, Cai, and DasGupta (2001) and Agresti and Coull (1998), in addition to the references cited in the following sections.

For each table request, PROC SURVEYFREQ produces a nondisplayed ODS table, "Table Summary," which contains the number of observations, strata, and clusters that are included in the analysis of the requested table. When you request confidence limits, the "Table Summary" data set also contains the degrees of freedom df and the value of $t_{\mi{df}, \alpha /2}$ that is used to compute the confidence limits. For more information about this output data set, see Example 109.3.

Wald Confidence Limits

PROC SURVEYFREQ computes standard Wald ("linear") confidence limits for proportions by default. These confidence limits use the variance estimates that are based on the sample design. For the proportion in table cell (r, c), the Wald confidence limits are computed as

\[ \widehat{P}_{rc} \pm \left( t_{\mi{df}, \alpha /2} \times \mr{StdErr}(\widehat{P}_{rc}) \right) \]

where $\widehat{P}_{rc}$ is the estimate of the proportion in table cell (r, c), $\mr{StdErr}(\widehat{P}_{rc})$ is the standard error of the estimate, and $t_{\mi{df}, \alpha /2}$ is the $100(1-\alpha /2)$th percentile of the t distribution with df degrees of freedom. (For more information, see the section Degrees of Freedom.) The confidence level $\alpha $ is determined by the value of the ALPHA= option; by default, ALPHA=0.05, which produces 95% confidence limits.

The confidence limits for row proportions and column proportions are computed similarly to the confidence limits for table cell proportions.

Modified Confidence Limits

PROC SURVEYFREQ uses the modification described in Korn and Graubard (1998) to compute design-based Clopper-Pearson (exact) and Wilson (score) confidence limits. This modification substitutes the degrees-of-freedom adjusted effective sample size for the original sample size in the confidence limit computations.

The effective sample size $n_ e$ is computed as

\[ n_ e ~ = ~ n ~ / ~ \mr{Deff} \]

where n is the original sample size (unweighted frequency) that corresponds to the total domain of the proportion estimate, and $\mr{Deff}$ is the design effect.

If the proportion is computed for a table cell of a two-way table, then the domain is the two-way table, and the sample size n is the frequency of the two-way table. If the proportion is a row proportion, which is based on a two-way table row, then the domain is the row, and the sample size n is the frequency of the row.

The design effect for an estimate is the ratio of the actual variance (estimated based on the sample design) to the variance of a simple random sample with the same number of observations. For more information, see the section Design Effect.

If you do not specify the CL(ADJUST=NO) option, the procedure applies a degrees-of-freedom adjustment to the effective sample size to compute the modified sample size. If you specify CL(ADJUST=NO), the procedure does not apply the adjustment and uses the effective sample size $n_ e$ in the confidence limit computations.

The modified sample size $n_ e^*$ is computed by applying a degrees-of-freedom adjustment to the effective sample size $n_ e$ as

\[ n_ e^* ~ = ~ n_ e ~ \left( \frac{ t_{\mi{(n-1)},\alpha /2} }{ t_{\mi{df}, \alpha /2} } \right)^2 \]

where df is the degrees of freedom and $t_{\mi{df}, \alpha /2}$ is the $100(1-\alpha /2)$th percentile of the t distribution with df degrees of freedom. The degrees of freedom computation depends on the sample design and the variance estimation method. For more information, see the section Degrees of Freedom. The confidence level $\alpha $ is determined by the value of the ALPHA= option; by default, ALPHA=0.05, which produces 95% confidence limits.

The design effect is usually greater than 1 for complex survey designs, and in that case the effective sample size is less than the actual sample size. If the adjusted effective sample size $n_ e^*$ is greater than the actual sample size n, the procedure truncates the value of $n_ e^*$ to n, as recommended by Korn and Graubard (1998). If you specify the CL(TRUNCATE=NO) option, the procedure does not truncate the value of $n_ e^*$.

Modified Clopper-Pearson Confidence Limits

Clopper-Pearson (exact) confidence limits for the binomial proportion are constructed by inverting the equal-tailed test based on the binomial distribution. This method is attributed to Clopper and Pearson (1934). For a derivation of the F distribution expression for the confidence limits, see Leemis and Trivedi (1996).

PROC SURVEYFREQ computes modified Clopper-Pearson confidence limits according to the approach of Korn and Graubard (1998). The degrees-of-freedom adjusted effective sample size $n_ e^*$ is substituted for the sample size in the Clopper-Pearson computation, and the adjusted effective sample size times the proportion estimate $n_ e^* \hat{p}$ is substituted for the number of positive responses. (Or if you specify the CL(ADJUST=NO) option, the procedure uses the unadjusted effective sample size $n_ e$ instead of $n_ e^*$.)

The modified Clopper-Pearson confidence limits for a proportion ($P_{\mi{L}}$ and $P_{\mi{U}}$) are computed as

\begin{eqnarray*} P_{\mi{L}} & = & \left( 1 + \frac{n_ e^* - \hat{p} n_ e^* + 1}{\hat{p} n_ e^* ~ F(~ \alpha /2, ~ 2 \hat{p} n_ e^*,~ 2(n_ e^* - \hat{p} n_ e^* + 1) ~ )} \right)^{-1} \\[0.1in] P_{\mi{U}} & = & \left( 1 + \frac{n_ e^* - \hat{p} n_ e^*}{(\hat{p} n_ e^* + 1) ~ F(~ 1-\alpha /2, ~ 2(\hat{p} n_ e^* + 1), ~ 2(n_ e^* - \hat{p} n_ e^*) ~ )} \right)^{-1} \end{eqnarray*}

where $F(\alpha /2, b, c)$ is the $(\alpha /2)$th percentile of the F distribution with b and c degrees of freedom, $n_ e^*$ is the adjusted effective sample size, and $\hat{p}$ is the proportion estimate.

Modified Wilson Confidence Limits

Wilson confidence limits for the binomial proportion are also known as score confidence limits and are attributed to Wilson (1927). The confidence limits are based on inverting the normal test that uses the null proportion in the variance (the score test). For more information, see Newcombe (1998) and Korn and Graubard (1999).

PROC SURVEYFREQ computes modified Wilson confidence limits by substituting the degrees-of-freedom adjusted effective sample size $n_ e^*$ for the original sample size in the standard Wilson computation. (Or if you specify the CL(ADJUST=NO) option, the procedure substitutes the unadjusted effective sample size $n_ e$.)

The modified Wilson confidence limits for a proportion are computed as

\[ \left( \hat{p} + (\kappa )^2 / 2 n_ e^* \right) ~ \pm ~ \left( \kappa \sqrt {\left( \hat{p} (1-\hat{p}) + (\kappa )^2 \right) / 4 n_ e^*} ~ / ~ \left( 1 + (\kappa )^2 / n_ e^* \right) \right) \]

where $n_ e^*$ is the adjusted effective sample size and $\hat{p}$ is the estimate of the proportion. With the degrees-of-freedom adjusted effective sample size $n_ e^*$, the computation uses $\kappa = z_{\alpha /2}$. With the unadjusted effective sample size, which you request with the ADJUST=NO option, the computation uses $\kappa = t_{\mi{df},\alpha /2}$. For more information, see Curtin et al. (2006).

Logit Confidence Limits

If you specify the CL(TYPE=LOGIT) option, PROC SURVEYFREQ computes logit confidence limits for proportions. For more information, see Agresti (2013) and Korn and Graubard (1998).

Logit confidence limits for proportions are based on the logit transformation $Y = \log ( \hat{p} / (1 - \hat{p}) )$. The logit confidence limits $P_{\mi{L}}$ and $P_{\mi{U}}$ are computed as

\begin{eqnarray*} P_{\mi{L}} & = & \exp ( Y_{\mi{L}} ) ~ / ~ ( ~ 1 + \exp ( Y_{\mi{L}} ) ~ ) \\[0.1in] P_{\mi{U}} & = & \exp ( Y_{\mi{U}} ) ~ / ~ ( ~ 1 + \exp ( Y_{\mi{U}} ) ~ ) \end{eqnarray*}

where

\[ (~ Y_{\mi{L}},~ Y_{\mi{U}} ~ ) ~ = ~ \log ( \hat{p}/(1-\hat{p}) ) ~ \pm ~ \bigl ( ~ t_{\mi{df}, \alpha /2} ~ \times ~ \mr{StdErr}(\hat{p}) ~ / ~ ( \hat{p} (1-\hat{p}) ) ~ \bigr ) \]

where $\hat{p}$ is the estimate of the proportion, $\mr{StdErr}(\hat{p})$ is the standard error of the estimate, and $t_{\mi{df}, \alpha /2}$ is the $100(1-\alpha /2)$th percentile of the t distribution with df degrees of freedom. (For more information, see the section Degrees of Freedom.) The confidence level $\alpha $ is determined by the value of the ALPHA= option; by default, ALPHA=0.05, which produces 95% confidence limits.