Jobs That Process IT Data

Overview of Jobs That Prepare, Stage, Aggregate, Generate Information Maps, and Report for IT Data

To prepare and process IT resource data, SAS IT Resource Management requires jobs that perform the following tasks:

Prepare the raw data for staging
Stage the raw data
Aggregate the data
Generate information maps of the data
Generate reports

For information about how to generate the job that prepares the raw data for staging, see Data Sources Supported by SAS IT Resource Management Adapters. The remaining jobs (staging, aggregating, generating information maps, and reporting) can be generated in two ways:

automatically, by using the Adapter Setup wizard
manually, by using the SAS IT Resource Management transformations

Note: After the jobs are created, either automatically by using the Adapter Setup wizard or manually by using the SAS IT Resource Management transformations, only the metadata definitions of the tables exist. You must actually execute the jobs in order to load data into the physical tables.

CAUTION:

Make sure that all tables that are associated with the job are closed before you run the job.

If any one table cannot be accessed with Write access, the job will fail with a nonzero return code.

Processing IT Data Using the Adapter Setup Wizard

Generating Jobs with the Adapter Setup Wizard

You can automatically generate the metadata for the jobs that process IT resource data by invoking the Adapter Setup wizard.

To invoke the Adapter Setup wizard, from the SAS IT Resource Management menu bar select File

New

IT Resource Management

Adapter Setup. The Adapter Setup wizard appears and prompts you to specify information about the adapters that you want to work with. For any adapter, you can specify one of the following configuration levels that governs the types and number of jobs that the wizard generates.

Staging Job Only creates only the staging job for the selected adapter.
Minimal creates the staging job and only a small number of aggregations for the metrics that are used to generate a minimal set of reports for the selected adapter.
Typical creates the staging job, all output from the Minimal level, and any additional metrics, and aggregations. It also creates report definitions that the SAS IT Resource Management solution has predetermined to be of interest for the selected adapter and reports. This option is the default setting.
Full creates the staging job, all output from the Typical level, and any additional metrics, aggregations. It also creates report definitions that make up a comprehensive set of jobs that are supported by the Adapter Setup wizard for the selected adapter.

When you click Finish, the Adapter Setup wizard creates the required jobs and all the metadata that is necessary for those jobs to execute. Among other information that it requests, the Adapter Setup wizard requires you to specify the IT data mart where it stores (by default) the jobs and the associated metadata for the adapter that you are working with. These jobs and the associated metadata are stored in the <adapter><number> folder that you are working with.

The Adapter Setup wizard creates the following jobs:

one staging job that contains only the part of the ETL flow that relates to the staging transformation and the target staged tables.
zero or more aggregation jobs that each contain one Aggregation transformation with target summary tables. The number of aggregation jobs that are created varies based on the domain categories (and their respective subcategories), time periods, and reporting selections that you made when specifying the adapter.
zero or more information map jobs that each contain one Information Map transformation per aggregation table. The number of information map jobs that are created varies based on the domain categories (and their respective subcategories), time periods, and reporting selections that you made when specifying the adapter.

Note: In SAS IT Resource Management 3.1 and SAS IT Resource Management 3.1.1, the Adapter Setup wizard created jobs that created or updated aggregation tables and also generated information maps based on those tables. If the job that modified aggregation tables was run, the corresponding Information Map transformations were part of that job. Therefore, the information maps were automatically regenerated. Starting with SAS IT Resource Management 3.2, jobs that create or update aggregation tables do not include transformations that generate information maps. Therefore, if any changes are made to the tables that are input to those Information Map transformations that alter the structure of the table (for example, by adding or deleting columns), then the user must rerun the corresponding information map jobs so that the information maps are regenerated.
zero or more reporting jobs that each contain one or more Gallery transformations per information map. The number of reporting jobs that are created varies based on the domain categories (and their respective subcategories), time periods, and reporting selections that you made when specifying the adapter.

The default name of each job that is created by the Adapter Setup wizard contains the name of the adapter and the domain category that you specified with the wizard.

Note: The jobs that the Adapter Setup wizard creates depend on one another to process the raw data for reporting. For example, the aggregation jobs include aggregations that rely on the staged tables that are generated when the staging job is run. As a result, each job should be run only after the jobs that it depends on have been run. The following list shows the types of jobs that the Adapter Setup wizard creates. Each job type in the list is dependent on the job type that precedes it.

staging job
aggregation job
information map job
reporting job

For information about how to work with the Adapter Setup wizard, see Adapter Setup Wizard.

Where Does the Adapter Setup Wizard Store the Objects That It Generates?

The Adapter Setup wizard stores the objects that it generates in predetermined locations within the IT data mart that you select. The wizard generates two sets of subfolders for the objects that it creates:

the Staging subfolder, which contains the staging job, staged tables, and the library for those tables
the <Domain Category> subfolder within the Domain Categories folder, which contains the aggregation, information map, and reporting jobs, their associated tables and maps, and the library that is associated with those objects

However, jobs, tables, and libraries can be stored in any location to which you have Write access. To change where an object is stored, right-click the object (in the process flow diagram or in the tree where it is currently stored) and select Properties. On the General tab of the Properties dialog box, click the Browse button that is next to the Location field. The Select a Location dialog box appears. Navigate to the location that you want to use and click OK.

The default location for the staging job, the staged tables, and the associated library is the Staging subfolder that is located within the adapter folder of the IT data mart that you selected.

The following display shows the metadata that is created in the Staging subfolder for the third instance of the DT Performance Sentry adapter within the IT data mart. The Staging subfolder contains the DT Perf Sentry Staging job, the NTLGDSK and NTSYSTEM_ITRM staged tables, and the library for those tables—NTSMF Staging 6066.

Default Location of Objects Created by the Adapter Setup Wizard for Staging the Raw Data

Default location of objects created by the Adapter Setup wizard for staging raw data

The default location for the remaining jobs, their associated tables, and library is in the Domain Categories subfolder that is located within the adapter folder of the IT data mart that you selected.

The following display shows the metadata that is created in the Domain Categories subfolder for the third instance of the DT Performance Sentry adapter within the IT data mart. The Domain Categories subfolder contains two subfolders of domain objects: Server Disk and Server System. Each domain subfolder contains five aggregated tables, the library associated with those tables, the aggregation job, the information map job, and the two reporting jobs.

Default Location of Objects Created by the Adapter Setup Wizard for Aggregating, Creating Information Maps, and Reporting on the Staged Data

Note: Information maps are not generated until the information map job runs. When they are generated, they are stored by default in the appropriate

Domain
                        Categories

subfolder, although you can store them in any location to which you have Write access.

Processing IT Data without the Adapter Setup Wizard

You can manually generate the metadata for the tasks that process IT resource data instead of invoking the Adapter Setup wizard. To do so, you must first open a job. Then, within that job, you can specify the staging, aggregating, information map, and reporting (gallery) transformations that are needed. You can also modify existing jobs by adding and deleting transformations, tables, and information maps.

To generate the metadata for a transformation, perform the following steps:

Open a job. If you want to work with an existing job, double-click on that job to open it on the Diagram tab of the Job Editor window of SAS IT Resource Management.

If you need to create a job, you can invoke the New Object wizard from the File menu of SAS IT Resource Management. For information about how to perform this task, see Create a Job.

The new job opens on the Diagram tab of the Job Editor window.
Navigate to the ITRM Transformations folder of the Transformations tree of SAS IT Resource Management.

Note: A job can consist of a single transformation, or it can include multiple transformations such as staging transformations, Aggregation transformations, Information Map transformations, and Gallery transformations.
Drop the transformation that you want to work with onto the job in the Diagram tab of the Job Editor window.
Locate and follow the instructions for working with transformations that perform the following tasks:
- Stage the raw data
  
  For information about how to stage the raw data, see Staging the Data.
- Aggregate the data
  
  For information about how to aggregate the data, see Aggregating the Data.
- Generate information maps of the data
  
  For information about how to generate information maps, see .
- Report on the IT intelligence and analysis that SAS IT Resource Management generates
  
  For information about how to generate, manage, and view reports, see the SAS IT Resource Management: Reporting Guide.

Note: In order to create (or update) the tables that are specified by the transformation, the job that contains the transformation must be executed. Jobs can be run immediately from SAS IT Resource Management, or they can be scheduled to execute as part of the operational procedures of an organization. (Due to the size of the aggregation jobs, it is preferable to run them in batch mode.)