Working with the Assessment Window
About the Assessment Window
For a decision tree, the Assessment window plots lift, ROC, and misclassification rates. Use the Assessment window
to determine how well the model fits the data.
Lift is the ratio of the percent of captured responses within each percentile bin to the average percent of responses for the model. Similarly, cumulative lift is calculated by using all of the data up to and including the current percentile
bin.
A receiver operating characteristic (ROC) chart displays the ability of a model to avoid false positive and false negative classifications. A false positive classification means that an observation has been identified as an event when it is actually a nonevent (also referred to
as a Type I error). A false negative classification means that an observation has
been identified as a nonevent when it is actually an event (also referred to as a
Type II error).
The misclassification plot displays how many observations were correctly and incorrectly classified for each value of the response variable. When the response variable is not binary, the logistic regression model considers all levels that are not events as equal. A significant number of
misclassifications could indicate that your model does not fit the data.
Using the Assessment Window
Lift
The default lift chart displays the cumulative lift of the model. To view the noncumulative
lift, click the 
icon on the vertical axis, and select Lift.
icon on the vertical axis, and select Lift.
For comparison, the lift chart plots a best model based on complete knowledge of the
input data.
ROC
The specificity of a model is the true negative rate. To derive the false positive rate, subtract the specificity from 1. The false positive rate, labeled 1
– Specificity, is the X axis of the ROC chart.
The sensitivity of a model is the true positive rate. This is the Y axis of the ROC chart. Therefore,
the ROC chart plots how the true positive rate changes as the false positive rate
changes.
A good ROC chart has a very steep initial slope and levels off quickly. That is, for
each misclassification of an observation, significantly more observations are correctly
classified. For a perfect model, one with no false positives and no false negatives, the ROC chart would start at (0,0), continue vertically to (0,1), and then horizontally
to (1,1). In this instance, the model would correctly classify every observation before
a single misclassification could occur.
The ROC chart includes two lines to help you interpret the ROC chart. The first line
is a baseline model that has a slope of 1. This line mimics a model that correctly classifies observations
at the same rate it incorrectly classifies them. An ideal ROC chart maximizes the
distance between the baseline model and the ROC chart. A model that classifies more
observations incorrectly than correctly would fall below the baseline model. The second
line is a vertical line at the false positive rate where the difference between the
Kolmogorov-Smirnov values for the ROC chart and
baseline models is maximized.
Misclassification
The misclassification plot displays how many observations were correctly and incorrectly
classified. A significant number of misclassifications might indicate that the model
does not fit the data.
When the ratio of events to non-events in your data is relatively large, the misclassification
plot might show a large number of true positives and false positives. In this case,
your model predicts most observations as events and is correct more often than not.
Copyright © SAS Institute Inc. All rights reserved.