Usage Note 22883: No statistics computed by the AGREE option
 |  |  |  |
In order to correctly compute agreement statistics, the table must be square and row labels must match corresponding column labels. The table will not be square if, for example, one rater never uses one of the categories of a rating scale but the other rater uses all of the categories. This implies that a row or column of the table contains all zero counts. Even if the data contains observations for such rows or columns, the default behavior of PROC FREQ is to delete all-zero rows or columns before computations are done. The result is a nonsquare table and this message in the SAS log:
NOTE: AGREE statistics are computed only for tables where the number of rows equals the
number of columns. To include zero-weight observations in the analysis, use the
ZERO option in the WEIGHT statement.
Note that if both raters do not use one category but the unused categories are not the same one (such as if one rater never uses the first category and the second rater never uses the second category), then the table after deleting zero rows and columns will be square and agreement statistics will be computed. However, these statistics are not correct because the rows and columns do not match. You should always check the table to verify that the rows and columns match before using the statistics.
To compute agreement statistics in these cases, the all-zero rows and/or columns must be retained in the table. With raw data in which each observation represents one subject being rated, an all-zero row or column for a category results when that category's value does not appear in the row or column variable in any observation. You can retain these missing categories and make the table square by adding at least one observation for each missing category. You will also need to add a variable to the data with value zero for these new observations and value one for the original observations. You will use this variable in the WEIGHT statement and specify the ZEROS option in that statement.
Alternatively, given the cell counts of the table to be analyzed, you can write a DATA step that creates a data set with all possible cells of the table. The observed cell counts, including any zero counts, are stored in a variable to be used in the WEIGHT statement.
Both cases – starting with either raw data or with aggregated, cell count data – are shown below.
Example - Starting with raw data
This example illustrates the process of making the table square when one rater does not use one category of a four-point rating scale. The following DATA step creates some simulated raw data in which two raters rate 50 subjects on a scale from 1 to 4. The RANTBL function generates a value (1, 2, 3, or 4) with the given probabilities. In this example, the raters' ratings are completely independent of each other. Note that Rater2 has zero probability of rating a subject as category 3 while Rater1 has equal probabilities of rating a subject in any category.
data a;
do Subject=1 to 50;
Rater1=rantbl(4198374,.25,.25,.25,.25);
Rater2=rantbl(4198374,.50,.25, 0 ,.25);
output;
end;
run;
These statements attempt the analysis, but since there are no 3 ratings from Rater2, this results in a nonsquare table which prevents PROC FREQ from computing agreement statistics.
proc freq data=a;
tables Rater1*Rater2 / agree nocol norow nopercent;
run;
|
The FREQ Procedure
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
You can make the table square by adding an observation with zero weight in each missing category – in this case, category 3 for Rater 2. It is sufficient to create an observation in only one cell of the category rather than in all cells. The following DATA steps create a single, zero-weight observation in category 3 for Rater 2 and adds it to the original data. It doesn't matter which rating is used for Rater 1 in this added observation. All observations in the original data set are given a weight of one.
data b;
Rater1=1; Rater2=3; Wt=0; output;
run;
data c;
set a b;
if Wt=. then Wt=1;
run;
The analysis is done using the WEIGHT statement and the ZEROS option, which tells PROC FREQ to allow zero-weighted cells to remain in the table.
proc freq data=c;
weight Wt / zeros;
tables Rater1*Rater2 / agree nocol norow nopercent;
run;
Note that the table is now square and that the agreement statistics are computed.
|
The FREQ Procedure
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Example - Starting with aggregated, cell count data
Given the above cell counts, the following DATA step creates a data set with one observation for each cell of the table and a count for each cell. The cells for missing category 3 in Rater 2 are included with zero count in each cell.
data a;
do Rater1=1 to 4;
do Rater2=1 to 4;
input Wt @@;
output;
end; end;
datalines;
10 4 0 3
5 3 0 3
2 6 0 3
6 3 0 2
;
These statements produce the same results as above.
proc freq data=a;
weight Wt / zeros;
tables Rater1*Rater2 / agree nocol norow nopercent;
run;
Operating System and Release Information
| Product Family | Product | System | SAS Release | |
| Reported | Fixed* | |||
| SAS System | SAS/STAT | All | n/a | |
| Type: | Usage Note |
| Priority: | low |
| Topic: | SAS Reference ==> Procedures ==> FREQ Analytics ==> Longitudinal Analysis Analytics ==> Nonparametric Analysis Analytics ==> Categorical Data Analysis Analytics ==> Descriptive Statistics |
| Date Modified: | 2019-08-09 13:36:12 |
| Date Created: | 2002-12-16 10:56:40 |