How to Backload Raw Data

You might want to process one or more sets of raw data into the SAS IT Resource Management system. Circumstances that might create the need to backload raw data range from populating an IT data mart with historical data to populating staged tables that were added to an adapter with the Add Domain Category wizard. When backloading raw data, you might want to include it in the aggregation tables and the reports that are created. The following diagram shows how you can accomplish this.

Diagram of the Backloading Process

To backload data into your IT data mart, perform the following steps.

Run the staging job with a set of the raw data.

If duplicate-data checking is enabled for this staging transformation, make sure that the data that you are trying to stage is not deleted because it appears to be duplicate data. To accomplish this, do the following:
1. Right-click the staging transformation that processes the new domain category and select Properties.
2. On the Properties dialog box, select the Staging Parameters tab.
3. On the Duplicate Checking page, ensure that the Enable duplicate checking field is set to Yes. This setting enables you to specify the parameters that govern the duplicate-data checking process.
4. Note the current setting of the Duplicate checking option. You will need to reset this option to its original value after you have completed the backloading tasks.
  
  Change the value of the Duplicate checking option to FORCE. This setting means that the transformation continues processing and accepts duplicate data if it is encountered.
  
  Duplicate Checking Options with FORCE Specified
5. Click OK to return to the process flow diagram.
Add the data to the IT data mart. To do so, perform these steps until all the sets of new data have been processed.
1. Run the staging job with a set of the raw data that you are backloading into your IT data mart.
2. Using the output of the staging transformation as the source table, run the corresponding Aggregation transformation.
3. [Optional] Run the corresponding Information Map and Performance Report transformations, if they exist. Review the reports to make sure they are what you want. (These transformations can be run for each set of raw data that is processed, or once for all the data that has been processed.)
After you have staged and aggregated all the data that you are backloading into your IT data mart, run the Information Map and Performance Report transformations.

Note: The new data that was processed might have caused changes to the structure of the aggregation tables and the information maps for those tables. Running the Information Map transformation at this time ensures that any changes to the data structure are incorporated into the information map that is used for reporting.
If you changed the setting of your Duplicate checking option in Step 1, be sure to restore the setting to its original value after you finish the backloading tasks.

Note: Duplicate-data checking macros are designed to prevent the same data from being processed into the IT data mart more than once. For information about the duplicate-data checking, see Duplicate-Data Checking Overview in the Duplicate-Data Checking Appendix.