Format a Monitoring Report

About Monitoring Reports

After you execute a performance task from the SAS Model Manager window or run the %MM_RunReports() macro in production mode, as a batch job, SAS Model Manager stores the output data sets in the default version Resources folder. You can use the New Reports window to format the performance monitoring results in PDF, HTML, RTF, or Excel output formats, or you can view the performance monitoring results by selecting the default version Performance node.
When you create monitoring reports using the New Reports window, the report creates the following charts:
Assessment charts
Assessment charts summarize the utility that one can expect by using the respective models, as compared to using only baseline information. Assessment charts can present a model's lift at a given point in time or the sequential lift performance of a model's lift over time. A monitoring report creates the following assessment charts:
  • Lift
  • Cumulative Lift
  • Percent Response
  • Cumulative Percent Response
  • Captured Response
  • Cumulative Captured Response
  • Actual vs. Predicted for prediction models
  • Actual vs. Residual for prediction models
  • Population Stability Trend for prediction models
Assessment charts are created for the Monitoring Report.
Lift Trend chart
A Lift Trend chart displays the cumulative lift of the champion model, over time.
Gini - ROC chart
Sensitivity is the proportion of true positive events and specificity is the proportion of true negative events. The Gini - ROC chart plots Sensitivity on the Y axis and 1 - Specificity on the X axis.
Gini - Trend Chart
When the Gini - ROC chart is created, the Gini index for each ROC curve is also created. The Gini index represents the area under the ROC curve and is a benchmark statistic that can be used to summarize the predictive accuracy of a model. The Gini - Trend chart plots a model's Gini index scores over time, and these are used to monitor model degradation over time.
KS Chart
The KS chart uses the Kolmogorov-Smirnov statistic to measure the maximum vertical separation, or deviation between the cumulative distributions of events and non-events.
KS Trend Chart
When you create a Kolmogorov-Smirnov report, the underlying KS statistic and the corresponding probability cutoff are read from a summary data set in the Resources folder. The KS Trend chart uses a summary data set that plots the KS Statistic over time. The KS Trend chart is used to monitor model degradation over time.
Actual vs. Predicted
You use the Actual vs. Predicted plot to see how predicted values match actual values.
Actual vs. Residual
You use the Actual vs. Residual plot to determine how good the model is at predicting values by examining errors and error trending, and comparing them to the actual values.
Population Stability Trend
The Population Stability Trend chart measures the shift of the scoring output variable distribution over time. Scoring output that is based on a development sample is used as the baseline distribution. The deviation index is used to indicate the shift for a given point in time.
Before you create a Monitoring Report or a Champion and Challenger Performance Report, you must ensure that certain project and model properties are set. For more information, see Verify Project and Model Property Settings.
These are the tasks that you perform for monitoring reports:

See Also

Create Reports by Defining a Performance Task
Create Reports Using Batch Programs

Create a Monitoring Report

To create a monitoring report:
  1. Expand the version folder Version Folder Icon.
  2. Right-click the Reports node and select Reportsthen selectNew Report. The New Report window appears.
  3. Select Monitoring Report from the Type list box.
    New Report Window Monitoring Report
  4. In the Format list box, select the type of output that you want to create. The default is PDF. Other options are HTML, EXCEL, and RTF.
  5. In the Style list box, select a style for the output. The default is SAS default. Other options are Seaside, Meadow, and Harvest.
  6. In the Report Properties table, complete the Name and Description properties if you do not want to use the default values. The default value for the Name property uses the form Monitoring_DdateTtimeThe name can contain letters, spaces, the underscore ( _ ), the hyphen ( - ), and the period ( . ).
  7. Click OK. A message box confirms that the report was created successfully.

See Also

View Reports
Performance Report Output Files
Copyright © SAS Institute Inc. All rights reserved.