Here are the requirements
for a performance table:
-
the input variables that you want
reported in a Characteristic report
-
if you have score code:
-
all input variables that are used
by the champion model or challenger models
-
all output variables that are used
by the champion model or challenger models
-
if you have no score code:
-
the actual value of the dependent
variable and the predicted score variable
-
all output variables that you want
reported in a Stability Report
You create a performance
table by taking a sampling of data from an operational data mart.
Make sure that your sampling of data includes the target or response
variables. The data that you sample must be prepared by using your
extract, transform, and load business processes. When this step is
complete, you can then use that data to create your performance table.
As part of the planning
phase, you can determine how often you want to sample operational
data to monitor the champion model performance. Ensure that the operational
data that you sample and prepare represents the period that you want
to monitor. For example, to monitor a model that determines whether
a home equity loan could be bad, you might want to monitor the model
every six months. To do this, you would have two performance tables
a year. The first table might represent the data from January through
June, and the second table might represent the data from July through
December.
Here is another example.
You might want to monitor the performance of a champion model that
predicts the delinquency of credit card holders. In this case, you
might want to monitor the champion model more frequently, possibly
monthly. You would need to prepare a performance table for each month
in order to monitor this champion model.
In addition to planning
how often you sample the operational data, you can also plan how much
data to sample and how to sample the data. Examples in this section
show you two methods of sampling data and naming the performance tables.
You can examine the sampling methods to determine which might be best
for your organization.