Prerequisites for Running the Define Performance Task Wizard :: SAS(R) Model Manager 12.3: User's Guide

Overview of Prerequisites

Before you run the Define Performance Task wizard, the environment must be set appropriately as follows:

Ensure that the champion model is set or the challenger model is flagged.
Ensure that the champion or challenger model is within a project that is associated with a classification model function and has a binary target, or a prediction model function and has an interval target.
Ensure that the champion or challenger model contains a score.sas file. If the performance data set contains the predicted values, the score.sas file can be empty. For more information, see Monitoring Performance of a Model without Score Code.
Ensure that the performance data sets for the time period that you want to monitor are registered in SAS Management Console or that a libref has been defined for the SAS library where the performance data sets are saved.
Ensure that the appropriate project and model properties are set.
Ensure that the model output variables are mapped to the project output variables.

After the environment is set, you can run the Define Performance Task wizard.

Ensure That the Champion Model Is Set or That the Challenger Model Is Flagged

The Define Performance Task wizard generates report code for the champion model in the default version.

You can determine the default version and the champion model by looking for the Champion Model Icon

icon next to the default version name and the champion model name.

If the champion model is not set, right-click the champion model name and select Set as Champion. The Champion Model Icon

icon appears next to the champion model name and the version for the champion model.

You can determine the challenger model by looking for the Challenger Model Icon

icon next to the challenger model name.

If the challenger model is not set, right-click the challenger model name and select Flag as Challenger. The Challenger Model Icon

icon appears next to the challenger model.

Ensure That the Champion Model Function and Class Target Level Are Valid

Performance monitoring is valid only for a project that is associated with a classification model function and has a binary target, or for a prediction model function that has an interval target. You should define only performance tasks for classification and prediction models. The champion model must have a function type of classification and must contain a binary target, or a function type of prediction and must contain an interval target.

From the Projects category view, select the champion model name and verify that the Function property in the specific properties section is set to Classification or Prediction. For models that are created using SAS Enterprise Miner, select the targetvar.xml file in the model folder and verify that the LEVEL attribute is set to BINARY for a classification model or to INTERVAL for a prediction model.

Ensure That the Performance Data Source Is Available

The Define Performance Task wizard requires that the performance data either be registered in the SAS Metadata Repository using SAS Management Console, or that a libref is defined from the Edit Start-up Code window to access the performance data from a SAS library.

If your performance table is not available for selection, do one of the following actions:

Contact your administrator to add the table to the Data Library Manager using SAS Management Console. For more information, see the SAS Model Manager Administrator's Guide.
Define a libref to access the performance data in a SAS library. For more information about defining a libref for the performance data, see Using Tables from a Local or Network Drive.

Ensure That Project and Model Properties Are Set

Several properties must be defined in order to generate the model performance reports. Verify that the appropriate project and model properties are set. Here is a list of properties.

Classification Project Properties

Training Target Variable
Target Event Value
Class Target Level
Output Event Probability Variable

Prediction Project Properties

Training Target Variable
Class Target Level
Output Prediction Variable

Model Properties

Score Code Type

Map Model and Project Output Variables

In order to create the model performance reports, the model output variable must be mapped to the project output variable if the corresponding project variable and the model variable have different names. To map these output variables, follow these steps:

Select the model from the Models node.
In the right pane, click the Model Mapping tab and click Edit.
For each project output variable, select a variable from the Model Variables list box.

Prerequisites for Running the Define Performance Task Wizard