Create Model Comparison Reports

In this exercise, you create several model comparison reports that are used in the selection and approval of a champion model. After you create the reports, you view the reports in the Reports folder. The reports enable you to evaluate candidate models in a version or across versions by assessing the structure, performance, and resilience of your models.

Create a Model Profile Report

The Model Profile report creates three tables to display the profile data that is associated with the model input variables, output variables, and target variables. To create this report, follow these steps:

Expand the 2012 version in the Delinquency project and right-click the Reports folder. Then select ReportsNew Report. The New Report window appears.
In the New Report window, use the specified values for these fields and click OK:

Type

select Model Profile Report.

Format

select PDF. PDF is the default value, and it might already be the value for Format.

Style

select Seaside. SAS default is the default style for the SAS format that is selected. For example, the default style for the HTML format is HTMLBLUE.

Select Models

select the box for Model 1.

Report Properties

replace the default report name with the report name profile_model1 in the Name field.

Here is the New Report window at this point in the process. Click OK when you are finished.
When the information dialog box confirms that the report was created successfully, click Close.

Create a Delta Report

The Delta report compares the profile data for two models and notes the differences. To create this report, follow these steps:

Expand the 2012 version in the Delinquency project and right-click the Reports folder. Then select ReportsNew Report. The New Report window appears.
In the New Report window, use the specified values for these fields and click OK:

Type

select Delta Report.

Format

select HTML.

Style

select SAS default. SAS default is the default style for the SAS format that is selected. For example, the default style for the HTML format is HTMLBLUE.

Select Models

select the boxes for Model 1 and Model 2.

Report Properties

replace the default report name with the report name delta_mod1mod2 in the Name field.

Here is the New Report window at this point in the process. Click OK when you are finished.
When the information dialog box confirms that the report was created successfully, click Close.

Create a Dynamic Lift Report

The Dynamic Lift report provides visual summaries of the performance of one or more models for predicting a binary outcome variable performance. To create this report, follow these steps:

Expand the 2012 version in the Delinquency project and right-click the Reports folder. Then select ReportsNew Report. The New Report window appears.
In the New Report window, specify the following options and click OK:

Type

select Dynamic Lift Report.

Format

select PDF.

Style

select Seaside. SAS default is the default style for the SAS format that is selected.

Select Models

select the boxes for Model 1 and Model 3.

Report Properties

replace the default report name with the report name lift_mod1mod3 in the Name field.

Here is the New Report Wizard at this point in the process. Click OK when you are finished.
When the information dialog box confirms that the report was created successfully, click Close.

View a Model Comparison Report

To view a model comparison report, follow these steps:

Expand the version folder 2012 and the Reports folder.
Right-click the report name and select View Report.

Note: If user credentials are required, then specify a user ID and password that have permission to access the SAS Content Server.
Use the PDF viewer to distribute or print a copy of the report.
Close the PDF Viewer.

For a detailed description of the model comparison reports, see the SAS Model Manager: User's Guide.

Update the Life Cycle (Optional)

To update the Development milestone, follow these steps:

In the Delinquency project, expand 2012Life CycleDevelopment.
Select the Create comparison reports task. Select the Status box and select Completed.
Select Create comparison reports. The Completed Date and Completed By fields have been updated with today's date and your user ID.
Click the Life Cycle node to examine its properties. The value for Date Modified is today's date. The Development property displays a bar chart that shows the percentage of completed tasks for this milestone.

Update the Workflow Process (Optional)

To complete the activities in the associated workflow process, follow these steps:

Select ToolsMy Workflow Inbox or click from the SAS Model Manager main window to view the workflow process activities in your workflow inbox. Workflow Console is launched in a Web browser, and displays the Activities category view.

Note: The list displays only the activities for which you are the actual owner or are assigned as a potential owner, and that have the state of Started.
From the Activities category view, select an activity name, and click .

Note: You can select an activity name and click to release an activity that you had previously claimed.
(Optional) Enter a property value or change an existing property value in the Properties pane.
(Optional) Add a comment to the activity using the Comments pane.
Select a status value to complete the activity. The workflow process continues to the next activity.
Repeat steps 2 through 5 for the activities that you completed during this tutorial.

Note: A workflow can be configured to display the activities that are associated with a milestone or task on the Workflow Milestones tab and in the Workflow Milestones report for a version in the SAS Model Manager client application. From the Workflow Milestones tab you can view the status of milestones or tasks that are associated with activities in the workflow.

Note: For more information, see Tutorial 10: Using Workflow Console.