Prerequisites for Editing a Performance Definition

Overview of Prerequisites

Before you edit a Performance Definition, the environment must be set appropriately as follows:
  • Ensure that the champion model is set or the challenger model is flagged.
  • Ensure that the champion or challenger model is within a project that is associated with a classification model function and has a binary target, or that is associated with a prediction model function and has an interval target.
  • Ensure that the champion or challenger model contains a score.sas file. If the performance data set contains the predicted values, the score.sas file can be empty. For more information, Monitoring Performance of a Model without Score Code.
  • Ensure that the performance data sets for the time period that you want to monitor are registered in SAS Management Console or that a libref has been defined for the SAS library where the performance data sets are saved.
  • Ensure that the appropriate project and model properties are set.
After the environment is set, you can run the Edit Performance Definition wizard.

Ensure That Champion and Challenger Models Are Set

The Edit Performance Definition wizard generates report code for the champion model in the champion version.
You can determine the champion version and the champion model by looking for the Set champion icon next to the champion version name and the champion model name on the Versions page. The status is also indicated in the Role column on the Models page.
If the champion model is not set, select a model and click Set championor right-click the champion model name and select Set as Champion. The Set champion icon appears next to the champion model name and the version for the champion model.
You can determine the challenger model by looking at the Role column on the Models page. View the number of challengers on the Details view of the Versions page.
If the challenger model is not set, click flag challenger or right-click the challenger model name and select Flag as Challenger.

Ensure That the Champion Model Function and Class Target Level Are Valid

Performance monitoring is valid only for a project that is associated with a classification model function and has a binary target, or for a prediction model function that has an interval target. You should define only performance definitions for classification and prediction models. The champion model must either have a function type of classification and must contain a binary target, or have a function type of prediction and must contain an interval target.
From the Projects category view, select the champion model name and verify that the Model function property in the specific properties section is set to Classification or Prediction. For models that are created using SAS Enterprise Miner, verify that Class target level is set to BINARY for a classification model or to INTERVAL for a prediction model.

Ensure That the Performance Data Source Is Available

The Edit Performance Definition wizard requires that the performance data be registered in the SAS Metadata Repository. You can register the data sets in the Data category view or you can add tables to an existing library that have already been registered using SAS Management Console.
If your performance table is not available for selection, contact your administrator to add the table to the Data Library Manager using SAS Management Console. For more information, see the SAS Decision Manager: Administrator's Guide.

Ensure That Project and Model Properties Are Set

Several properties must be defined in order to generate the model performance reports. Verify that the appropriate project and model properties are set. Here is a list of properties.
Classification Project Properties
  • Training target variable
  • Target event value
  • Class target level
  • Output event probability variable
Prediction Project Properties
  • Training target variable
  • Class target level
  • Output prediction variable
Model Properties
  • Score code type

Map Model and Project Output Variables

In order to create the model performance reports, you must map the model output variable to the project output variable if the corresponding project variable and the model variable have different names.
To map the model variables to the project variables:
  1. Select and open a model.
  2. Select Model Propertiesthen selectOutput Mapping.
  3. Click the box in the Value column beside the variable in the Property column to display a list of project variables.
  4. Select a model output variable.
  5. Repeat steps 3 and 4 for each model variable that requires mapping.
  6. Click Save.
Last updated: February 22, 2017