Champion Models

About Champion Models

The champion model is the best predictive model that is chosen from a pool of candidate models. Before you identify the champion model, you can evaluate the structure, performance, and resilience of candidate models. You select the champion model from the models in a version. When a champion model is ready for production scoring, you set the model as the champion model. A check mark Set Champion check mark appears next to the model and version that are associated with the champion model. The version that contains the champion model is considered the default version for the project. There can be only one champion model for a project.
You can publish the champion model to a database, the SAS Metadata Repository, and a SAS channel.
These are the tasks that you perform to use champion models:

Requirements for a Champion Model

Before you identify a model as the champion, perform the following tasks:
  • Create a version for your project, and register at least one model.
  • Verify that the model is active. If the model expiration date has passed, you cannot set the model as a champion model.
    Note: An authorized user can reset the expiration date to a later date so that it is possible to set the champion model. To reset the expiration date, click the Properties tab for the model.
  • Complete the required life cycle milestones that precede the milestone task of setting the champion model under a version.
You might use the following criteria to identify a champion model:
  • model comparison reports that validate and assess the candidate models
  • business decision rules. For example, you might use a decision tree model because of difficulty interpreting results from a neural network model even when the neural network model outperforms the decision tree model
  • regulatory requirements, such as when the champion model should exclude certain specific attributes (age or race)
You can flag and publish a challenger model in SAS Model Manager specifically for the purpose of comparison with the champion model. For example, your champion model for a production environment might omit restricted attributes during operational scoring because of regulatory requirements. You can use a challenger model that includes the restricted attributes in the development environment to evaluate its prediction power against the prediction of the champion model. Then you can determine the amount of predictive power that is lost because of the regulatory requirements.

See Also

Flag a Challenger Model

Set a Champion Model

To set a champion model, follow these steps:
  1. Expand the Models folder under the version folder Version Icon.
  2. Right-click the model that you want to use as the champion model and select Set as Champion from the pop-up menu. A dialog box appears. Click Yes to confirm.
  3. If there are model input variables that are not defined as project input variables, you are prompted to add the input variables. Click Yes to confirm. The model input variables are copied to the project input variables. If project output variables are not defined, then the Select Project Output Variables window appears for you to select the output variables. After you select the output variables, click OK.
  4. If the project output variables have not been mapped to the model output variables, the Set Model Output Mapping window appears. For each project variable, click the Model Variables field and select the model output variable. Click OK.
  5. If you are replacing the champion model, follow these steps:
    1. If there are no challenger models flagged, select Yes to replace the champion model. If a challenger model is flagged, go to the next step.
    2. To keep the current challenger models, select Keep models flagged as challengers. If you do not select Keep models flagged as challengers, challenger models are cleared.
    3. Click Yes to confirm replacing the current champion model.
    4. If the project output variable has not been mapped to the model output variable, the Set Model Output Mapping window appears. For each project variable, click the Model Variables field and select the model output variable. Click OK.
  6. Verify that the Champion Model check mark icon appears beside the champion model. and the version
  7. Select the version folder to examine its properties. The value for Date Modified is today's date. The value for the Champion Model Name is the name of the champion model.
    Note: SAS Model Manager automatically annotates the History tab. To document the reasons for your selection of the champion model, use the version Notes tab.
  8. Update the specific properties for the appropriate milestone task in the Life Cycle node. Specify that Status for selecting a champion model is set to Completed.

Clear a Champion Model

To clear a champion model, follow these steps:
  1. Expand the Models folder under the version folder Version Icon.
  2. Right-click the champion model and select Set as Champion from the pop-up menu. A message box appears.
    Clear Champion Model on the Menu
  3. Click Yes to confirm. The Set as Champion check mark is cleared. SAS Model Manager also clears all challenger models and clears the version check mark.
    Note: If the version is frozen, you cannot clear the champion model unless a SAS Model Manager administrator unfreezes the version first.
  4. Select the version folder to examine its properties. Verify that the value for Date Modified property is today's date. The model that you just cleared as the champion model remains as the value for the Champion Model Name property until another champion model is set.
  5. Update the specific properties for the appropriate milestone task in the Life Cycle node. Change Status for selecting a champion model to a value that is not completed, such as Started.
Copyright © SAS Institute Inc. All rights reserved.