When you designate a holdout, out-of-sample data, and/or reconciliation in your project, multiple MAPE values are displayed in SAS Forecast Studio.

Holdout MAPE

The holdout MAPE is the selection MAPE. The holdout MAPE is calculated by evaluating the candidate model-specifications using the holdout data. This MAPE value determines the champion model-specification. If a holdout is used, then the holdout MAPE is displayed in these locations:

• model selection list (which is near the top of the Modeling View tab).
• OUTSTATSELECT data set.

The OUTSTATSELECT data set contains the statistics that were used to select the model specification for the current model. This data set contains the statistics-of-fit that are displayed in the model selection list.

After the model specification is selected, the parameter estimates are re-estimated using the entire range of the time series (excluding the out-of-sample data). This refitted model specification is the champion model.

Model MAPE (or MAPE)

The model MAPE is the MAPE value that is calculated using the champion model. The model MAPE is displayed in these locations:

• if there is no reconciliation and no out-of-sample, then the model MAPE is displayed in the banner that is at the top of the view area.
• if there is reconciliation or out-of-sample data, then the model MAPE is displayed when you click the Sigma icon. The Sigma icon is in the banner at the top of the view area.
• OUTSTAT data set.

If there is no holdout, then these statements are true:

• the model MAPE is used for model selection.
• the model MAPE is displayed in the model selection list.
• the model MAPE is displayed in the OUTSTATSELECT data set.

Reconciled MAPE

If you specified reconciliation, then the reconciled MAPE is displayed in these locations:

• banner at the top of the view area.
• RECSTAT data set.

Note: SAS Forecast Studio does not create the RECSTAT data set for the reconciliation level because the reconciled forecasts at this level are the statistical forecasts. The OUTSTAT data set contains both the reconciled forecasts and the statistical forecasts since they are identical.

Out-of-Sample MAPE

You can use an out-of-sample region along with the holdout sample to determine how the model performs with future forecasts. The out-of-sample MAPE is calculated by evaluating the champion model using the out-of-sample data. The out-of-sample MAPE values are displayed in these locations:

• Sigma icon.
• OUTSTAT data set. The out-of-sample model MAPE and in-sample model MAPE are both displayed in the OUTSTAT data set.
• RECSTAT data set. The out-of-sample reconciliation MAPE and in-sample reconciliation MAPE are both displayed in the RECSTAT data set.

Example

You might want to designate a holdout, an out-of-sample region, and a reconciliation in your SAS Forecast Studio project. If you designate all three, then in the Modeling View, click the Sigma icon (which is in the banner at the top of the view area). A Statistics of Fit window is displayed in the Modeling View.

Notes

The view area consists of the following items: Forecasting View, Modeling View, Series View, and Scenario Analysis View.

MAPE values might be displayed in additional locations.

The holdout MAPE cannot be replicated using the forecasts in the data table or the OUTFOR data set. The forecast values for the holdout MAPE are generated dynamically. These forecasts are are multi-step forecasts in the holdout region and are computed internally.

The output data sets are documented in SAS Forecast Studio User's Guide. See Appendix: Output Data Sets in the Project Directory.

SAS Forecast Studio User's Guide has more information about model selection. See the section "How SAS Forecast Server Works" in the chapter "About SAS Forecast Studio".

SAS Forecast Studio User's Guide has examples about specifying a holdout or an out-of-sample region. See the sections "Using a Holdout Sample" and "Using an Out-of-Sample Region" in the "Working with Models" chapter.

Operating System and Release Information

Product Family	Product	System	SAS Release
			Reported	Fixed*
SAS System	SAS Forecast Studio	Microsoft® Windows® for x64
		Microsoft Windows 8 Enterprise 32-bit
		Microsoft Windows 8 Enterprise x64
		Microsoft Windows 8 Pro 32-bit
		Microsoft Windows 8 Pro x64
		Microsoft Windows 8.1 Enterprise 32-bit
		Microsoft Windows 8.1 Enterprise x64
		Microsoft Windows 8.1 Pro 32-bit
		Microsoft Windows 8.1 Pro x64
		Microsoft Windows 10
		Microsoft Windows 95/98
		Microsoft Windows 2000 Advanced Server
		Microsoft Windows 2000 Datacenter Server
		Microsoft Windows 2000 Server
		Microsoft Windows 2000 Professional
		Microsoft Windows NT Workstation
		Microsoft Windows Server 2003 Datacenter Edition
		Microsoft Windows Server 2003 Enterprise Edition
		Microsoft Windows Server 2003 Standard Edition
		Microsoft Windows Server 2003 for x64
		Microsoft Windows Server 2008
		Microsoft Windows Server 2008 R2
		Microsoft Windows Server 2008 for x64
		Microsoft Windows Server 2012 Datacenter
		Microsoft Windows Server 2012 R2 Datacenter
		Microsoft Windows Server 2012 R2 Std
		Microsoft Windows Server 2012 Std
		Microsoft Windows XP Professional
		Windows 7 Enterprise 32 bit
		Windows 7 Enterprise x64
		Windows 7 Home Premium 32 bit
		Windows 7 Home Premium x64
		Windows 7 Professional 32 bit
		Windows 7 Professional x64
		Windows 7 Ultimate 32 bit
		Windows 7 Ultimate x64
		Windows Millennium Edition (Me)
		Windows Vista
		Windows Vista for x64

* For software releases that are not yet generally available, the Fixed Release is the software release in which the problem is planned to be fixed.

---

Explanation of the different MAPE values and where to find them in SAS Forecast Studio and in output data sets

Type:	Usage Note
Priority:

Date Modified:	2016-04-13 08:00:46
Date Created:	2016-04-06 08:46:13