Your project plan might
include a schedule to monitor the champion model performance, or your
plan might require that you monitor the performance at any time. For
each time period that you monitor the champion model, you take a snapshot
of the data for that time period and use that data as the performance
data source for creating the monitoring reports.
You can create the monitoring
reports from the Project Tree in the
SAS Model Manager window
by creating and executing a performance task, or you can submit batch
programs to create the reports. Both methods require the same information.
If you run batch programs to create the reports, you can find example
programs in the sashelp.modelmgr.source catalog. These reports filenames
are
reportexamplex,
where
x is a number from 1
to 4.
The following table
lists the tasks that are required to create performance reports:
|
|
Reports Created by Using
the Define Performance Task Wizard
|
Reports Created Using
SAS Programs Run in Batch
|
Create a folder structure
for report files
|
The folder structure
is inherent in the Project Tree. No action is necessary.
|
Create a folder structure
on a local computer.
|
|
|
The performance data
is a SAS data set that is snapshot of model output. It must be registered
in SAS Management Console.
|
The performance data
is used to assess model prediction accuracy. It includes all of the
required scoring input variables as well as one or more actual target
variables. You can store performance data sets anywhere as long as
they can be accessed by the SAS session that runs the batch program.
The data set does not need to be registered with SAS Management Console.
|
Ensure access to the
champion or challenger model
|
This process is performed
by the Define Performance Task wizard. No
action is necessary.
|
Run the %MM_GetModels()
macro to extract the champion model in a channel to the local computer.
|
Map model and project
output variables.
|
Map the model and project
output variables using the Project Tree.
|
Map the model and project
output variables using the Project Tree.
|
Define report specifications
|
The report specification
are derived from project data and input that you specify in the Define
Performance Task wizard. The wizard generates the SAS
code to create the performance reports.
|
Write the following
DATA steps:
|
Specify the report execution
operational environment
|
The operational environment
is known to SAS Model Manager. No action is necessary.
|
Define the required
macro variables that are used by the %MM_RunReports() macro.
|
|
|
Execute the code from
the PerformanceMonitor node that was generated
by the Define Performance Task wizard . The
data sets that underlie the monitoring reports are stored in the Resources folder.
|
Create a DATA step that
points to the performance data sets and execute the %MM_RunReports()
macro. The data sets that underlie the monitoring reports are stored
in the Resources folder when the reports
are created in production mode. In Test mode, the monitoring reports
data sets reside in the location specified in the mm_jobs.project
data set.
|
|
|
Select the default version Performance folder
to view the reports.
|
Select the fault version Performance folder
to view the reports.
|