The FREQ Procedure

PROC FREQ computes McNemar’s test (McNemar 1947) for tables when you specify the AGREE option. This test is appropriate when you are analyzing data from matched pairs of subjects with a dichotomous (yes-no) response. By default, the null hypothesis for McNemar’s test is marginal homogeneity, which can be expressed as ; this is equivalent to a discordant proportion ratio () of 1. The corresponding test statistic is computed as

Under the null hypothesis, has an asymptotic chi-square distribution with 1 degree of freedom.

Optionally, you can specify the null ratio of discordant proportions () by using the AGREE(MNULLRATIO=) option. When the null ratio is r, McNemar’s test is computed as

where , , and D is the number of discordant pairs, (). Under the null hypothesis, has an asymptotic chi-square distribution with 1 degree of freedom.

PROC FREQ also computes an exact p-value for McNemar’s test when you specify the MCNEM option in the EXACT statement.

The null hypothesis for Bowker’s symmetry test (Bowker 1948) is symmetric table-cell proportions, which can be expressed as for all off-diagonal pairs of table cells. For tables, Bowker’s test is identical to McNemar’s test; therefore, PROC FREQ provides Bowker’s test only for square tables that are larger than .

Bowker’s symmetry test is computed as

For large samples, has an asymptotic chi-square distribution with degrees of freedom under the null hypothesis of symmetry.

By default, the number of degrees of freedom for this test () is the number of off-diagonal table-cell comparisons, where R is the dimension of the square, two-way table. You can specify the number of degrees of freedom in the AGREE(DFSYM=) option. Alternatively, you can specify the AGREE(DFSYM=ADJUST) option, which reduces the degrees of freedom by the number of off-diagonal table-cell pairs that have a total frequency of 0. For more information, see Hoenig, Morgan, and Brown (1995).

The simple kappa coefficient (Cohen 1960) is a measure of interrater agreement. PROC FREQ computes the simple kappa coefficient as

where and . The component is the proportion of observed agreement, and the component represents the proportion of chance-expected agreement.

If the two response variables are viewed as two independent ratings of the n subjects, the kappa coefficient is +1 when there is complete agreement of the raters. When the observed agreement exceeds the chance-expected agreement, the kappa coefficient is positive, and its magnitude reflects the strength of agreement. When the observed agreement is less than the chance-expected agreement, the kappa coefficient is negative. The minimum value of kappa is between –1 and 0, depending on the marginal proportions of the table.

Given the marginal proportions of the two-way table, the maximum possible value of the kappa coefficient is computed as

where

When you specify the AGREE(KAPPADETAILS) option, PROC FREQ displays the "Kappa Details" table, which includes the observed agreement, chance-expected agreement, and maximum kappa. For tables, this table also includes the prevalence index and the bias index. The prevalence index is the absolute difference between the agreement proportions, . The bias index is the absolute difference between the disagreement proportions, . For more information, see Sim and Wright (2005) and Byrt, Bishop, and Carlin (1993).

PROC FREQ computes the asymptotic variance of the simple kappa coefficient as

where

For more information, see Fleiss, Cohen, and Everitt (1969).

Confidence limits for the simple kappa coefficient are computed as

where is the th percentile of the standard normal distribution. The value of is determined by the ALPHA= option; by default ALPHA=0.05, which produces 95% confidence limits.

PROC FREQ provides an asymptotic test for the simple kappa coefficient. By default, the null hypothesis value of kappa is 0; alternatively, you can specify a nonzero null value of kappa (by using the AGREE(NULLKAPPA=) option in the TABLES statement). When the null value of kappa is nonzero, PROC FREQ computes the test statistic as

where is the null value that you specify and is the variance of the kappa coefficient.

When the null value of kappa is 0, PROC FREQ computes the test statistic as

where is the variance of the kappa coefficient under the null hypothesis (that kappa is 0) and is computed as

This test statistic has an asymptotic standard normal distribution under the null hypothesis. For more information, see Fleiss, Levin, and Paik (2003).

PROC FREQ also provides an exact test for the simple kappa coefficient. You can request the exact test by specifying the KAPPA or AGREE option in the EXACT statement. For more information, see the section Exact Statistics.

The weighted kappa coefficient is a generalization of the simple kappa coefficient that uses weights to quantify the relative differences between categories. For tables, the weighted kappa coefficient is equivalent to the simple kappa coefficient; therefore, PROC FREQ displays the weighted kappa coefficient only for tables larger than . PROC FREQ computes the kappa weights from the column scores, by using either Cicchetti-Allison weights or Fleiss-Cohen weights, both of which are described in the section Kappa Weights. The kappa weights are constructed so that for all , for all i, and . The weighted kappa coefficient is computed as

where

The component is the proportion of observed (weighted) agreement, and the component represents the proportion of chance-expected (weighted) agreement. When you specify the AGREE(WTKAPDETAILS) option, PROC FREQ displays these components in the "Weighted Kappa Details" table.

PROC FREQ computes the asymptotic variance of the weighted kappa coefficient as

where

For more information, see Fleiss, Cohen, and Everitt (1969).

Confidence limits for the weighted kappa coefficient are computed as

where is the th percentile of the standard normal distribution. The value of is determined by the ALPHA= option; by default ALPHA=0.05, which produces 95% confidence limits.

PROC FREQ provides an asymptotic test for the weighted kappa coefficient. By default, the null hypothesis value of weighted kappa is 0; alternatively, you can specify a nonzero null value of weighted kappa (by using the AGREE(NULLWTKAPPA=) option in the TABLES statement). When the null value of weighted kappa is nonzero, PROC FREQ computes the test statistic as

where is the null value that you specify and is the variance of the weighted kappa coefficient.

When the null value of weighted kappa is 0, PROC FREQ computes the test statistic as

where is the variance of the weighted kappa coefficient under the null hypothesis (that weighted kappa is 0) and is computed as

This test statistic has an asymptotic standard normal distribution under the null hypothesis. For more information, see Fleiss, Levin, and Paik (2003).

PROC FREQ also provides an exact test for the weighted kappa coefficient. You can request the exact test by specifying the KAPPA or AGREE option in the EXACT statement. For more information, see the section Exact Statistics.

When you specify the AGREE(PABAK) option, PROC FREQ provides the prevalence-adjusted bias-adjusted kappa coefficient (PABAK) (Byrt, Bishop, and Carlin 1993). This coefficient is computed as

where and R is the dimension of the square, two-way table. The component is the proportion of observed agreement, and the component represents the chance-expected agreement. When the table is , . For more information, see Sim and Wright (2005), Xie (2013), and Holley and Guilford (1964).

PROC FREQ computes the variance of the prevalence-adjusted bias-adjusted kappa as

Confidence limits are computed as

where is the th percentile of the standard normal distribution. The value of is determined by the ALPHA= option; by default ALPHA=0.05, which produces 95% confidence limits.

When you specify the AGREE(AC1) option, PROC FREQ provides Gwet’s first-order agreement coefficient, AC1 (Gwet 2008). This coefficient is computed as

where , , and The component is the proportion of observed agreement, and the component represents the proportion of chance-expected agreement. For more information, see Xie (2013) and Blood and Spratt (2007).

PROC FREQ computes the variance of AC1 as

where

Confidence limits for AC1 are computed as

where is the th percentile of the standard normal distribution. The value of is determined by the ALPHA= option; by default ALPHA=0.05, which produces 95% confidence limits.

When there are multiple strata, PROC FREQ combines the stratum-level estimates of kappa into an overall estimate of the supposed common value of kappa. Assume there are q strata, indexed by , and let denote the variance of . The estimate of the overall kappa coefficient is computed as

For more information, see Fleiss, Levin, and Paik (2003).

PROC FREQ computes an estimate of the overall weighted kappa in the same way.

When there are multiple strata, the following chi-square statistic tests whether the stratum-level values of kappa are equal:

Under the null hypothesis of equal kappas for the q strata, has an asymptotic chi-square distribution with q–1 degrees of freedom. See Fleiss, Levin, and Paik (2003) for more information. PROC FREQ computes a test for equal weighted kappa coefficients in the same way.

Cochran’s Q is computed for multiway tables when each variable has two levels, that is, for tables. Cochran’s Q statistic is used to test the homogeneity of the one-dimensional margins. Let m denote the number of variables and N denote the total number of subjects. Cochran’s Q statistic is computed as

where is the number of positive responses for variable j, T is the total number of positive responses over all variables, and is the number of positive responses for subject k. Under the null hypothesis, Cochran’s Q has an asymptotic chi-square distribution with m–1 degrees of freedom. For more information, see Cochran (1950). When there are only two binary response variables (m=2), Cochran’s Q simplifies to McNemar’s test. When there are more than two response categories, you can test for marginal homogeneity by using the repeated measures capabilities of the CATMOD procedure.

The AGREE statistics are defined only for square tables, where the number of rows equals the number of columns; if a table is not square, PROC FREQ does not compute AGREE statistics for the table. In the kappa statistic framework, where two independent raters assign ratings to each of n subjects, suppose one of the raters does not use all possible r rating levels. If the corresponding table contains r rows but only r–1 columns, the table is not square and PROC FREQ does not compute AGREE statistics. To create a square table in this situation, you can use the ZEROS option in the WEIGHT statement, which includes zero-weight observations in the analysis. You can include zero-weight observations in the input data set to represent any rating levels that are not used by a rater, so that the input data set has at least one observation for each possible rater and rating combination. When you use this input data set and specify the ZEROS option, the analysis includes all rating levels (even when all levels are not actually assigned by both raters). The resulting table (of rater 1 by rater 2) is a square table, and AGREE statistics can be computed.

For more information, see the description of the ZEROS option in the WEIGHT statement. By default, PROC FREQ does not process observations that have weights of 0 because these observations do not contribute to the total frequency count, and because many of the tests and measures of association are undefined for tables that contain zero-weight rows or columns. However, kappa statistics are defined for tables that contain zero-weight rows or columns, and the ZEROS option enables you to input zero-weight observations and construct the tables needed to compute kappa statistics.