"Rank Correlation of Observed Responses and Predicted Probabilities" in the Details section of the PROC LOGISTIC documentation describes the binning of predicted probabilities and how pairs of observations are determined to be concordant, discordant, or tied. From these, the association statistics — Somers' D (Gini coefficient), gamma, tau-a, and c (the concordance index and area under the ROC curve) — can be computed using the formulas shown in the documentation.

Note that binning the predicted probabilities is more efficient and reduces execution time for large data sets, but produces a rougher approximation to these statistics. Binning can be turned off by specifying the BINWIDTH=0 option in the MODEL statement, or by specifying any of the following:

• the OUTROC= option in the MODEL or SCORE statement
• one or more ROC statements
• the PLOTS=ROC (or PLOTS=ALL) option in the PROC LOGISTIC statement when ODS Graphics is on

A more accurate approximation of the association statistics, such as the area under the ROC curve (c statistic), is obtained by using any of the above to turn off binning.

Note that, beginning in SAS 9.4 TS1M3, no binning is done if the response is binary and there are fewer than 5,000,000 observations in the input data set.

The following example uses the described method to optionally bin the predicted probabilities and compute the association statistics. The code shown applies only to binary response models.

Example

These statements produce an example data set for which the association statistics will be computed.

      data a; 
        input Seq Score Outcome;
        datalines;
      1   1525    0
      2   1641    0
      3   1706    0
      4   1722    0
      5   1738    1
      6   1758    1
      7   1770    0
      8   1775    0
      9   1798    0
      10  1839    1
      11  1842    0
      12  1848    0
      13  1848    0
      14  1856    0
      15  1864    0
      16  1864    0
      17  1879    0
      18  1895    0
      19  1909    0
      20  1917    0
      21  1944    0
      22  1975    0
      23  2002    0
      ;

These statements fit a binary logistic model to the OUTCOME variable. The EVENT="1" response variable option ensures that the probability of OUTCOME=1 is modeled. Since the BINWIDTH= option is not specified, the default bin width of 1/500 = 0.002 is used in computing the association statistics. The predicted probabilities computed by the PREDPROBS=INDIVIDUAL option are not binned and are saved in data set OUT.

      proc logistic data=a;
        model outcome(event="1") = score;
        output out=out predprobs=individual;
        run;

The resulting "Association of Predicted Probabilities and Observed Responses" table from the model fit is shown below.

By specifying the BINWIDTH=0 option (or any of the other options or statements mentioned above), binning is turned off.

      proc logistic data=a;
        model outcome(event="1") = score / binwidth=0;
        run;

Following is the table of statistics when the predicted probabilities are not binned.

The following statements define the macro CONCDISC which applies the binning method described in the LOGISTIC documentation. It creates the data set _PAIRS which contains an observation for each possible pair of event and nonevent observations and indicates whether each pair is concordant, discordant, or tied. The macro requires the data set of predicted probabilities, the name of the response variable, and the values of the event and nonevent levels of the response. If the BINWIDTH= option was not specified in the PROC LOGISTIC step, then it can be omitted when calling the macro. The macro will then use the same default bin width. Otherwise, specify the same value in the BINWIDTH= macro option as was specified in the PROC LOGISTIC step.

      %macro concdisc(data=, event=, nonevent=, response=, binwidth=0.002);
      %global n;
      /* bin the predicted probabilities and give all in the bin the lowest value */
      data _r&data;
        set &data;
        %if &binwidth ne 0 %then pbin=floor((1/&binwidth)*ip_&event);
        %else                    pbin=ip_&event;
        ;
        run;
      proc sql;
        reset noprint;
        /* number of observations in original data set into macro variable N */
        select count(*) into :N from _r&data;
        /* create data set of nonevents */
        create table _nonevents as select pbin as pbin0 from _r&data where &response=&nonevent;
        /* create data set of events */
        create table _events as select pbin as pbin1 from _r&data where &response=&event;
        /* create data set of all event-nonevent pairs and determine concordance */
        create table _pairs as select *, (pbin1>pbin0) as conc,
                                         (pbin1=pbin0) as tie,
                                         (pbin0>pbin1) as disc
                     from _nonevents, _events;
      %mend;

This statement calls the CONCDISC macro using the default bin width of 0.002.

      %concdisc(data=out, event=1, nonevent=0, response=outcome)

The following statements compute the proportions and counts of concordant, discordant, and tied observations.

      proc summary data=_pairs;
        var conc disc tie;
        output out=cd mean=propconc propdisc proptie sum=nc nd n=t;
        run;

Finally, these statements use the formulas shown in the documentation to compute and display the association statistics.

      data rankcorrs;
        set cd;
        Concordant=100*propconc; 
        Discordant=100*propdisc;
        Tied=100*proptie;
        Pairs=t;
        SomersD=(nc-nd)/t;
        Gamma=(nc-nd)/(nc+nd);
        Tau_a=(nc-nd)/(0.5*&N*(&N-1));
        c=(nc + 0.5*(t-nc-nd))/t;
        run;
      
      proc print noobs;
        var Concordant Discordant Tied Pairs SomersD Gamma Tau_a c;
        format Concordant Discordant Tied Pairs 5.1 SomersD Gamma Tau_a c 6.3;
        run;

Note that the recomputed association statistics match those produced by PROC LOGISTIC when the default binning was used.

The association statistics resulting from not binning the predicted probabilities can be obtained by using the above code with the BINWIDTH=0 option in the CONCDISC macro.

      %concdisc(data=out, event=1, nonevent=0, response=outcome, binwidth=0)

Operating System and Release Information