Champion Models :: SAS(R) Model Manager 3.1: User's Guide

About Champion Models

To identify the champion model, you can evaluate the structure, performance, and resiliency of candidate models. You select the champion model from the models in a version. When a champion model is ready for production scoring, you select the version that contains this champion model as the default version for a project. Then the champion model in the default version becomes the default champion model for the project. When you export the project champion model, SAS Model Manager deploys the default champion model.

These are the tasks that you perform to use champion models:

Requirements for a Champion Model

Before you identify a model as the champion, perform the following tasks:

Create a version for your project, and register at least one model.
Verify that the model is active. If the model expiration date has passed, then you cannot set the model as a champion model.

Note: An authorized user can reset the expiration date to a later date so that you can set the champion model.
Complete the required life cycle milestones that precede the milestone task of setting the champion model under a version.

You might use the following criteria to identify a champion model:

model comparison reports that validate and assess the candidate models
business decision rules, such as using a decision tree model because of difficulty interpreting results from a neural network model even when the neural network model outperforms the decision tree model
regulatory requirements, such as when the champion model should exclude certain specific attributes (age or race)

You can register a challenger model in SAS Model Manager specifically for the purpose of comparison with the champion model. For example, your champion model for a production environment might omit restricted attributes during operational scoring because of regulatory requirements. You can use a challenger model that includes the restricted attributes in the development environment to evaluate its prediction power against the prediction of the champion model. Then you can determine the amount of predictive power that is lost due to the regulatory requirements.

Set a Champion Model

To set a champion model, follow these steps:

Expand the Models folder under the version folder .
Right-click the model that you want to use as the champion model and select Set Champion Model from the pop-up menu. A dialog box appears.
Click Yes to confirm.
Verify that the icon appears beside the champion model.
Select the version folder to examine its properties. The value for Date Modified is today's date. The value for the Champion Model ID is the champion model's UUID.

Note: To document the reasons or assumptions for your selection of the champion model, use the version Notes tab. SAS Model Manager automatically annotates the History tab.
Update the specific properties for the appropriate milestone task in the Life Cycle node. Specify that Status for selecting a champion model is set to Completed.

Clear a Champion Model

To clear a champion model, follow these steps:

Expand the Models folder under the version folder .
Right-click the champion model and select Clear Champion Model from the pop-up menu. A message box appears.
Click Yes to confirm. If the champion model is located in the default version, then SAS Model Manager also clears the default version.

Note: If the version is frozen, then you cannot clear the champion model unless you are a SAS Model Manager administrator.
Select the version folder to examine its properties. Verify that the value for Date Modified property is today's date. The value for the Champion Model ID property has been cleared.
Update the specific properties for the appropriate milestone task in the Life Cycle node. Change Status for selecting a champion model to a value that is not completed, such as Started.