What Is Produced by the %RMPDB2DM Macro

Overview

Executing the %RMPDB2DM macro produces the following three categories of items:
  • metadata
  • physical storage
  • status report
These items are explained in more detail in the following sections.

Metadata Objects Produced by %RMPDB2DM

About the Metadata Objects Produced by %RMPDB2DM

Every table, column, computation, and data transformation, as well as the organizational structure of these items, is described by metadata in the SAS Metadata Server. The %RMPDB2DM macro defines all the metadata to support the migrated PDB in its new form as an IT data mart.
Specific metadata items that are created include the following objects:
  • the IT data mart.
  • formulas used by the various staging tables for all of the adapters in the IT data mart.
  • subfolders for each adapter. Within each folder, the following objects are created:
    • folders for Aggregation and Staging
    • SAS libraries
    • staged tables
    • simple aggregation tables
    • summarized aggregation tables
    • jobs
Each of these items is described in more detail in the following sections.

IT Data Mart

Each invocation of %RMPDB2DM creates a new IT data mart. The IT data mart is a logical metadata container of related SAS IT Resource Management tables, jobs, and other objects.
The Administrative folder is created with a library for administrative purposes.
An IT Data Mart Created by %RMPDB2DM
IT Data Mart Created by %RMPDB2DM

Formulas

In SAS IT Resource Management 2.7, the term formula was used to describe a column in a table that was calculated by means of a user-written expression. In IT Resource Management 3.2, this functionality is now accomplished by means of computed columns. However, SAS IT Resource Management 3.2 does use an object that is called a formula. In SAS IT Resource Management 3.2, a formula is a reusable expression that is associated with one or more computed columns in order to allow a single definition to be shared and replicated.
In SAS IT Resource Management 3.2, Aggregation transformations perform no transformation of the aging and datetime stamp variables. Therefore, the values that are needed for summarization must be computed as columns that are part of the table that is input to the Aggregation transformation.
SAS IT Resource Management 2.7 used a different approach. All levels of a table used the column DATETIME in the class list. In SAS IT Resource Management 2.7, the DATETIME that was read into DETAIL was transformed in some way to be the DATETIME at the other levels.
Migration does this computation by means of formulas that are used on the staged tables. These formulas are used to create the following staged columns: DAYDATE, WEEKDATE, MONTHDATE, and YEARDATE. The source code that is used to define the WEEKDATE formula respects the setting of the START OF WEEK parameter that was used in the PDB.
In addition to the formulas that are used for aging, the staged column SHIFT is created by a formula that %RMPDB2DM creates. It is based on the SHIFT definition that is used with the SAS IT Resource Management 2.7 PDB. This formula also honors the HOLIDAY SHIFT that is used by the PDB.
In addition, a new staged column is created named HOLIDAY. This column has values of Y or N, depending on whether the date of the observation is for a holiday. This formula is based on the active list of holidays that are read from the SITELIB that is used during migration.
Finally, the staged columns for DATE, HOUR, and TIME are created, using the respective formulas that manipulate the DATETIME column.
All formulas created for migration of a PDB are stored in the PDB Migration Formulas folder of the IT data mart that was created by running the %RMPDB2DM macro.
Formulas and Folder Created by %RMPDB2DM
Formulas and Folder

Objects in the IT Data Mart That Are Created for Each Adapter

In each IT data mart, the following sets of objects are created for each migrated SAS IT Resource Management 2.7 collector:
  • staged tables
  • simple aggregation tables
  • summarized aggregation tables
  • the libraries for these tables
  • jobs that load these tables
  • Aggregation and Staging folders to provide organization of these objects
Folders Created for Each Adapter
Folders Created for Each Adapter
Contents of the Staging Folder
All objects that are related to staging data for a given adapter are in the Staging folder for that adapter. This includes the following objects:
  • a SAS library to hold the data for all staged tables for this staging transformation instance
  • the staged tables needed for this adapter, as determined by the PDB
  • a job to stage the data for this adapter
Staging Folder Contents
Staging Folder Contents
Staged tables are populated by user-written staging code or staging transformations that SAS IT Resource Management supplies. These tables hold the metrics that are presented by the data source adapter. If the adapter being migrated is supported by SAS IT Resource Management 3.2, then the staging job for the adapter has a staging transformation for the specific adapter as the starting point of the job. Otherwise, the first element of the job is a user-written staging transformation. In either case, the outputs of the first transformation of the staging job are the staged tables for the adapter.
Staging Job
Staging Job
The %RMPDB2DM macro determines the columns that are needed for each staging table by determining the columns that are in existence in the SAS IT Resource Management 2.7 PDB. Staged tables can also include computed column definitions. For information about these definitions, see Formulas.
Additional name_RATE columns are defined to the staged tables of SAS IT Resource Management 2.7 tables of type INTERVAL, for columns that had a variable interpretation type of COUNT, TIME, or TIMETICKS. For more information, see Data Handling Differences between SAS IT Resource Management 2.7 and 3.2.
Contents of the Aggregation Folder
All objects that are related to the aggregation of data for a given adapter are in the Aggregation folder for that adapter. For each staged table in the Staging folder, there is a corresponding subfolder in the Aggregation folder. The following objects are in the subfolder for each staged table:
  • a SAS library to hold the data for all aggregation tables that are produced from the staged table
  • aggregation tables that corresponding to the DETAIL, DAY, WEEK, MONTH, and YEAR tables in SAS IT Resource Management 2.7
  • a job to populate the aggregation tables
Aggregation Folder Contents
Aggregation Folder
An aggregation job is created for each table that is staged by the staging job. The aggregation job starts with the staged table as the input to an Aggregation transformation. This transformation functions similarly to the SAS IT Resource Management 2.7 %CxPROCES and %CPREDUCE macros. As shown in the following display, the outputs of the Aggregation transformation are the simple and summarized aggregation tables.
Aggregation Job
Aggregation Job
Simple Aggregation Tables
Simple aggregation tables are analogous to the SAS IT Resource Management 2.7 DETAIL tables. The following table shows how the functions available in SAS IT Resource Management 2.7 can be accomplished in SAS IT Resource Management 3.2.
Comparison of DETAIL Level Tables and Simple Aggregation Tables
Tasks
SAS IT Resource Management 2.7
SAS IT Resource Management 3.2
How to purge existing data from a table before loading new data into that table
An age limit of 0 for the DETAIL level was a special setting that caused %CPREDUCE to purge the DETAIL table after it was summarized.
Specify the following settings in the Specify purging and aging criteria page of the Summarized Aggregation :
  • Ensure that the check box for Purge output table before loading new data is selected.
  • Ensure that the check box for Perform aging for output table is not selected.
Configuring these specifications causes the existing data in the simple aggregation How to load table to be purged before loading new data when the aggregation transformation is executed.
How to load data that is older than the specified age limit
Data could be loaded into the DETAIL level even if it should be aged out based on the age limit settings. The purpose of this capability was to allow post-processing (such as user-written chargeback routines) to read the DETAIL data.
Select the option Allow incoming data even if it precedes the age limit to enable this same functionality for simple aggregation tables that have been migrated.
How tables are named
The table name was the same at each level of the PDB.
The simple aggregation table that is created by migration is named <table-name>_DETAIL (for example, SARDEV_DETAIL).
Summarized Aggregation Tables
Summarized aggregation tables are analogous to the DAY, WEEK, MONTH, and YEAR tables in SAS IT Resource Management 2.7. The following table shows how the functions available in SAS IT Resource Management 2.7 can be accomplished in SAS IT Resource Management 3.2.
Comparison of DAY, WEEK, MONTH, and YEAR Level Tables and Summarized Aggregation Tables
Tasks
SAS IT Resource Management 2.7
SAS IT Resource Management 3.2
How to deactivate summarization
An age limit of 0 for a summary level table would deactivate summarization for that level.
To deactivate summarization, do not create a summary aggregation table.
How aging columns are named
The aging column for all summary levels was always named DATETIME.
A separate aging column (that is named appropriate to the summarization) is used to support the aging. These aging columns are created as computed columns in the staged tables. The names of these columns are DAYDATE, WEEKDATE, MONTHDATE, and YEARDATE.
The WEEKDATE column honors the SAS IT Resource Management 2.7 START OF WEEK setting from the PDB.
How tables are named
The table name was the same at each level of the PDB.
The summarized aggregation table created by migration is named <table-name>_<level> (for example, SARDEV_WEEK).

Physical Storage

About Physical Storage

%RMPDB2DM creates metadata, and it also creates the physical SAS libraries and tables that are needed to store the data. These libraries are allocated in accordance with the parameters that are specified to the macro. In addition, the simple and summarized aggregation tables are created and the data is loaded into these tables from the PDB.
The storage location that is specified must be accessible to the SAS session where the macro is executed. This location can be on a network drive or other device.
One SAS library is created for each staging or Aggregation transformation. Thus, all staged tables for a given adapter are stored in a single SAS library, and all of the aggregation tables for a given staged table are stored in another single SAS library.

Special Considerations for z/OS Physical Storage

For z/OS systems, the IT data mart can be created in the traditional z/OS file system or in the UNIX file system that is managed by UNIX Systems Services (zFS or HFS). One advantage of using a UNIX file system is that you do not need to specify additional allocation information.

Status Report

When the execution of %RMPDB2DM is finished, a status report is written to the standard SAS output area. The following two figures show an example of this report.
Status Report Produced by %RMPDB2DM (First Part)
Status Report - Part 1 of 2
Status Report Produced by %RMPDB2DM (Second Part)
Status Report - Part 2 of 2