Create Model Comparison Reports

In this exercise, you create several model comparison reports that are used in the selection and approval of a champion model. After you create the reports, you view the reports in the Reports folder. The reports enable you to evaluate candidate models in a version or across versions by assessing the structure, performance, and resilience of your models.

Create a Model Profile Report

The Model Profile report creates three tables to display the profile data that is associated with the model input variables, output variables, and target variables. To create this report:

Expand the 2013 version in the Delinquency project and right-click the Reports folder. Then select ReportsNew Report. The New Report window appears.
In the New Report window, use the specified values for these fields and click OK:

Type

select Model Profile Report.

Format

select PDF. PDF is the default value, and it might already be the value for Format.

Style

select Seaside. SAS default is the default style for the SAS format that is selected. For example, the default style for the HTML format is HTMLBLUE.

Select Models

select the radio button for Model 1.

Report Properties

replace the default report name with the report name profile_model1 in the Name field.

Here is the New Report window at this point in the process. Click OK when you are finished.
When the information dialog box confirms that the report was created successfully, click Close.

Create a Delta Report

The Delta report compares the profile data for two models and notes the differences. To create this report:

Expand the 2013 version in the Delinquency project and right-click the Reports folder. Then select ReportsNew Report. The New Report window appears.
In the New Report window, use the specified values for these fields and click OK:

Type

select Delta Report.

Format

select HTML.

Style

select SAS default. SAS default is the default style for the SAS format that is selected. For example, the default style for the HTML format is HTMLBLUE.

Select Models

select the boxes for Model 1 and Model 2.

Report Properties

replace the default report name with the report name delta_mod1mod2 in the Name field.

Here is the New Report window at this point in the process. Click OK when you are finished.
When the information dialog box confirms that the report was created successfully, click Close.

Create a Dynamic Lift Report

The Dynamic Lift report provides visual summaries of the performance of one or more models for predicting a binary outcome variable performance. To create this report:

Expand the 2013 version in the Delinquency project and right-click the Reports folder. Then select ReportsNew Report. The New Report window appears.
In the New Report window, specify the following options and click OK:

Type

select Dynamic Lift Report.

Format

select PDF.

Style

select Seaside. SAS default is the default style for the SAS format that is selected.

Select Models

select the boxes for Model 1 and Model 3.

Report Properties

replace the default report name with the report name lift_mod1mod3 in the Name field.

Here is the New Report Wizard at this point in the process. Click OK when you are finished.
When the information dialog box confirms that the report was created successfully, click Close.

View a Model Comparison Report

To view a model comparison report:

Expand the version folder 2013 and the Reports folder.
Right-click the report name and select View Report.

Note: If user credentials are required, then specify a user ID and password that have permission to access the SAS Content Server.
Use the PDF viewer to distribute or print a copy of the report.
Close the PDF Viewer.

For a detailed description of the model comparison reports, see the SAS Model Manager: User's Guide.

Update the Life Cycle (Optional)

To update the Development milestone:

In the Delinquency project, expand 2013Life CycleDevelopment.
Select the Create comparison reports task. Select the Status box and select Completed.
Select Create comparison reports. The Completed Date and Completed By fields have been updated with today's date and your user ID.
Click the Life Cycle node to examine its properties. The value for Date Modified is today's date. The Development property displays a bar chart that shows the percentage of completed tasks for this milestone.

Update the Workflow Process (Optional)

To complete the activities in the associated workflow:

Select ToolsMy Workflow Inbox or click from the SAS Model Manager main window to view the workflow activities in your workflow inbox. Workflow Console is launched in a web browser, and displays the Activities category view.

Note: The list displays only the activities for which you are the actual owner or are assigned as a potential owner, and that have the state of Started.
From the Activities category view, select an activity name, and click .

Note: You can select an activity name and click to release an activity that you had previously claimed.
(Optional) Enter a property value or change an existing property value in the Properties pane.
(Optional) Double-click an activity to view the activity details. From the activity details view, you can modify activity properties or perform the model management tasks that are associated with the activity.

Note: If you did not claim the activity from the Activities category view, you can select the Claim check box in the activity status bar.
(Optional) Add a comment to the activity using the Comments pane.
Select a status value to complete the activity. The workflow continues to the next activity.
Repeat steps 2 through 6 for the activities that you completed during this tutorial.

Note: A workflow can be configured to display the activities that are associated with a milestone or task on the Workflow Milestones tab and in the Workflow Milestones report for a version in the SAS Model Manager client application. From the Workflow Milestones tab, you can view the status of milestones or tasks that are associated with activities in the workflow.

Note: For more information, see Tutorial 11: Using Workflow Console.