Compare Your Models

You can compare the performance of these two models using the model comparison visualization. In the explorer, model comparison requires that the response variable, level, and group by variable are identical in the compared models. The effects used in each model can differ.

To compare the models in this example, complete the following steps:

Click to create a new model comparison.
In the Model Comparison window, Response should already contain the variable Emission of Total Hydrocarbons (g/mi). Select the variable Vehicle Type for Group By. If you used a group by variable in only one of the models that you created, you cannot compare these two models.
Click between the Available models area and the Selected models area to add both visualizations to the model comparison.
Click OK.
Review the results windows for the model comparison. The default segment chosen is CAR. This is displayed in the summary bar at the top of the visualization.

Because Emission of Total Hydrocarbons (g/mi) is an interval target, the model comparison visualization displays an Assessment plot. This plot compares either the observed average value or predicted average value between each model at each percentile. Notice that these models are relatively similar in regard to both the observed or predicted average.

In the Fit Statistic window, the default displayed value is the average square error (ASE). Hold your mouse pointer over the bar for each visualization to see that the ASE value for the linear regression is better.

The fit statistic Observed Average displays the observed average value at the specified percentile. After selecting Observed Average, you can change the displayed percentile with the slider on the Properties tab.

This information is also in the details table. In addition to ASE, you can view the sum of squared errors (SSE). Notice that the SSE for these two models favors the linear regression.
In the summary bar, click the word CAR, and select TRUCK to compare the results for the truck segment. Explore the Assessment and Fit Statistic windows to notice the differences and similarities between the two models.
Repeat your exploration for the BOTH segment.
Save the exploration.

Note: Model comparisons do not persist between sessions. If you sign out of SAS Visual Analytics and want to return to this model comparison, then you must re-create it.