Overview of SAS Programs to Monitor Model Performance

A SAS program that creates performance monitoring reports consists of three conceptual sections:
  • The first section defines the report specifications that identify the project, the types of reports that you want to create, alert and warning conditions, and the date and time to run the batch jobs.
  • The second section extracts the champion model from a publishing channel. Any batch job that creates performance monitoring reports must extract models from a publishing channel. The champion model must have been published to the channel from the project folder.
  • The third section defines the operating environment and the performance data set. This section calls a SAS macro that creates the reports.
You define the report specifications by writing four DATA steps:
  • mm_jobs.project defines the project specifications.
  • mm_jobs.emailaddr defines the e-mail address where you send job, alert, and warning notifications.
  • mm_jobs.reportdef defines which type of reports you want to create, and the alert and warning conditions for those reports.
  • mm_jobs.jobtime defines the date and time to run the batch jobs.
After the report specification data sets have been created, you extract the champion model from the publishing channel to the local computer using the %MM_GetModels() macro. You set macro variables to define the operating environment, specify the performance data set, and run the %MM_RunReports() macro to create the reports.
You view the reports by selecting the version Performance node in the SAS Model Manager window.
SAS Model Manager provides the following performance monitoring macros:
  • %MM_GetModels() extracts models from a publishing channel.
  • %MM_UpdateCharacteristicTable creates a Characteristic report.
  • %MM_UpdateStabilityTable creates a Stability report.
  • %MM_UpdateAssessmentTable creates model monitoring reports.
  • %MM_RunReports() sets the operating environment and runs the macros to create the reports.
Note: The macros are in the modelmgr.sas7bcat file. The location of this file for Windows is \sasintalldir\SASFoundation\9.3\mmcommon\sashelp. The default value for sasinstalldir in Windows is C:\Program Files\SASHome. The location of this file for UNIX is /sasinstalldir/SASFoundation/9.3/sashelp. The default value for sasinstalldir in UNIX is /usr/local/SASHome.
SAS provides example SAS programs in the sashelp.modelmgr catalog that you can modify for your environment.