Your project plan might
include a schedule to monitor the champion model performance, or your
plan might require that you monitor the performance at any time. For
each time period that you monitor the champion model, you take a snapshot
of the data for that time period and use that data as the performance
data source for creating the monitoring reports.
You can create the monitoring
reports from the Project Tree in the
SAS Model Manager window by creating and executing a performance task, or you can
submit batch programs to create the reports. Both methods require
the same information. If you run batch programs to create the reports,
you can find example programs in the sashelp.modelmgr.source catalog.
These reports filenames are
reportexamplex, where
x is a number from 1 to 4.
The following table
lists the tasks that are required to create performance reports:
|
|
Reports Created by Using
the Define Performance Task Wizard
|
Reports Created Using
SAS Programs Run in Batch
|
Create a folder structure
for report files
|
The folder structure
is inherent in the Project Tree. No action is necessary.
|
Create a folder structure
on a local computer.
|
|
|
The performance data
is a SAS data set that is snapshot of model output. It must be registered
in SAS Management Console.
|
The performance data
is used to assess model prediction accuracy. It includes all of the
required scoring input variables as well as one or more actual target
variables. You can store performance data sets anywhere as long as
they can be accessed by the SAS session that runs the batch program.
The data set does not need to be registered with SAS Management Console.
|
Ensure access to the
champion model
|
This process is performed
by the Define Performance Task wizard. No
action is necessary.
|
Run the %MM_GetModels()
macro to extract the champion model in a channel to the local computer.
|
Map model and project
output variables.
|
Map the model and project
output variables using the Project Tree.
|
Map the model and project
output variables using the Project Tree.
|
Define report specifications
|
The report specification
are derived from project data and input that you specify in the Define Performance Task wizard. The wizard generates
the SAS code to create the performance reports.
|
Write the following
DATA steps:
|
Specify the report execution
operational environment
|
The operational environment
is known to SAS Model Manager. No action is necessary.
|
Define the required
macro variables that are used by the %MM_RunReports() macro.
|
|
|
Execute the code that
was generated by the Define Performance Task wizard. The data sets that underlie the monitoring reports are stored
in the Resources folder.
|
Create a DATA step that
points to the performance data sets and execute the %MM_RunReports()
macro. The data sets that underlie the monitoring reports are stored
in the Resources folder when the reports
are created in production mode. In Test mode, the monitoring reports
data sets reside in the location specified in the mm_jobs.project
data set.
|
|
|
Select the version Performance folder to view the reports.
|
Select the version Performance folder to view the reports.
|