Examples - 9.2

Example 1: Processing Data in a DATETIME Range for Which a DATETIME and IDVAR Was Already Processed

This example describes how to process data into a table that was not used earlier, or if the data is in a table that was accidentally deleted.

To process data in a datetime range for which a particular DATETIME and IDVAR variable was already processed, specify the FORCE=YES parameter on the %RMDUPCHK macro. This parameter specifies that the data is to be accepted, even though it appears to be duplicate data.

Note: The IDVAR= parameter identifies the SAS variable that is used by MXG to denote the origin of each IBM SMF record. This value must be set to SYSTEM for the IBM SMF adapter, because that is the name of the variable that is used by MXG.

Example 2: Processing Data for Two IBM SMF Files

This example describes how to process data for two separate IBM SMF files. In this example, a site has two separate IBM SMF contiguous data sets for the same IDVAR variable. It attempts to stage them into a supplied SAS IT Resource Management table. The first IBM SMF input file covers a datetime stamp range from 9:00 a.m. to 10:00 a.m., the second from 10:00 am to 11:00 a.m.

In addition, the following conditions apply to this example:

The DUPMODE=DISCARD duplicate data parameter is set for both jobs.
The records that describe the last IBM SMF interval are split across the two SMF data sets.
Both of these IBM SMF files are processed for the same IT data mart.
A 15-minute IBM SMF interval is being used—for example, an interval that starts at 9:45 a.m. and ends at 9:59:59 a.m. The RMF 70-79 records that describe all the statistics in this same interval are partially written on the first data set. When the SYS1.MAN1 data set is full, the RMF 70-79 records that start at 9:59:59 a.m are written in the SYS1.MAN2 data set.
Each time an IBM SMF file is read, the data is processed into SAS IT Resource Management and aggregated.

There are two methods that you can use to process this data:

Load all IBM SMF data in a single job, concatenating the two IBM SMF data sets. This is the preferred method, because no data is lost.
Load IBM SMF data in two unique JCL steps or batch jobs. Each job should read one IBM SMF file at a time.

Note: In each method, the first IBM SMF data set is loaded into the IT data mart. The duplicate-data-checking macros mark the data as correctly loaded from 9:45:00 a.m. to 9:59:59 a.m. However, because the second method processes the SMF data files individually, those records that match the DATETIME and IDVAR parameters that range from 9:45:00 a.m. to 9:59:59 a.m. are considered to be duplicates. Therefore, they are discarded.

Example 3: Using Macro Variables to Subset Data for HP Reporter, MS SCOM, and VMware Adapters

The staging transformations for the HP Reporter, MS SCOM, and VMware adapters do not provide a parameter for duplicate-data checking. Duplicate-data checking is always enabled for these adapters. SAS IT Resource Management provides two macro variables that enable you to subset data for the HP Reporter, MS SCOM, and VMware adapters: ITRM_LoadFromDate and ITRM_LoadToDate. These macro variables enable you to override the default action of subsetting the incoming data based on the duplicate-data control data sets.

The ITRM_LoadFromDate and ITRM_LoadToDate macro variables can be used in the following situations:

to backload data into tables that are added to a staging job after it has already run once against a given set of data

Note: Only the data for HP Reporter and VMware adapters can be backloaded. MS SCOM data cannot be backloaded.
to specify a datetime range that will be used during staging to extract only the data from the input database whose datetime stamps fall within the range specified

Note: When the ITRM_LoadFromDate and ITRM_LoadToDate macro variables are set, the duplicate data checking code is still executed. SAS IT Resource Management will discard any data that is detected as duplicates.

The following code sets the ITRM_LoadFromDate and ITRM_LoadToDate macro variables to valid start and end datetime values. These values will be used to subset the data from the database instead of the ranges in the duplicate data control data sets. This code should be added to the generated code or to the deployed job code for the staging job.

%let ITRM_loadFromDate=14FEB2010:00:00:00; 
%let ITRM_loadToDate=15FEB2010:23:59:00;