Jobs That Process IT Data

Overview of Jobs That Prepare, Stage, Aggregate, Generate Information Maps, Analyze Exceptions, and Report for IT Data
Processing IT Data Using the Adapter Setup or Add Domain Category Wizard
Generating Jobs with the Adapter Setup Wizard or Add Domain Category Wizard
Where Does the Adapter Setup Wizard Store the Objects That It Generates?
Processing IT Data without the Adapter Setup Wizard

Overview of Jobs That Prepare, Stage, Aggregate, Generate Information Maps, Analyze Exceptions, and Report for IT Data

To prepare and process IT resource data, SAS IT Resource Management requires jobs that perform the following tasks:
  • Prepare the raw data for staging
  • Stage the raw data
  • Aggregate the data
  • Detect and report on exceptional conditions in the incoming data
  • Generate information maps of the data
  • Generate performance reports
For information about how to generate the job that prepares the raw data for staging, see Data Sources Supported by SAS IT Resource Management Adapters. The remaining jobs (staging, aggregating, detecting exceptions, generating information maps, and reporting) can be generated in two ways:
  • automatically, by using the Adapter Setup or Add Domain Category wizard
  • manually, by using the SAS IT Resource Management transformations
Note: After the jobs are created, either by using the Adapter Setup or Add Domain Category or by using the SAS IT Resource Management transformations, only the metadata definitions of the tables exist. You must actually execute the jobs in order to load data into the physical tables.
CAUTION:
Make sure that all tables that are associated with the job are closed before you run the job.
If any one table cannot be accessed with Write access, the job fails with a nonzero return code.

Processing IT Data Using the Adapter Setup or Add Domain Category Wizard

Generating Jobs with the Adapter Setup Wizard or Add Domain Category Wizard

You can automatically generate the metadata for the jobs that process IT resource data by invoking the Adapter Setup or Add Domain Category wizard.
To invoke the Adapter Setup wizard, from the menu bar of the SAS IT Resource Management client select Filethen selectNewthen selectIT Resource Managementthen selectAdapter Setup.
Note: The Add Domain Category wizard performs in a manner that is similar to (but not completely the same as) the Adapter Setup wizard. For information about this wizard and how to invoke it, see About the Add Domain Category Wizard.
CAUTION:
Concurrent accessing of the Adapter Setup or Add Domain Category wizard by different users can cause the resulting jobs to fail if they are also executed concurrently.
If the users select the same adapter, identically numbered folders in the IT data mart are created. Concurrent execution of the jobs created by the wizard cause one user’s jobs to fail.
The Adapter Setup wizard appears and prompts you to specify information about the adapters that you want to work with. For any adapter, you can specify one of the following configuration levels that govern the types and number of jobs that the wizard generates.
  • Staging Job Only creates a staging job only for the adapter. If you select this option, no aggregation jobs, information map jobs, exception jobs, or reporting jobs are created.
  • Minimal creates a staging job and additional jobs that include metrics, aggregations, report definitions, and exception definitions for a small number of reports. This level is for reporting on a small scale. You might use this level to create jobs that are related to low-priority, non-critical machines.
  • Typical creates the staging job and a set of jobs that include recommended metrics, aggregations, report definitions, and exception definitions that satisfy a typical set of reports for an adapter. (This configuration level includes the output that is part of the Minimal setting.)
    This configuration level is the default option.
  • Full creates a staging job and a comprehensive set of jobs that include most metrics, aggregations, report definitions, and exception definitions that SAS IT Resource Management supports. This level includes all of the jobs that are part of the Typical level. In addition, it includes many of the other metrics that are recommended for an adapter. You might use this level when you want the full benefits of the domain intelligence that is available with the SAS IT Resource Management solution as well as additional information from the full set of metrics that are available from the adapter data sources.
    Note: Increasing the configuration level results in more complex transformations and jobs. If you select Full when you need only the Typical level, the extra complexity causes the Adapter Setup wizard to be slower to complete than is necessary for your needs. Also, the nightly ETL jobs take longer to process.
When you click Finish, the Adapter Setup wizard creates the required jobs and all the metadata that is necessary for those jobs to execute. The Adapter Setup wizard requires you to specify the IT data mart where it stores (by default) the jobs and the associated metadata for the adapter that you are working with. These jobs and the associated metadata are stored in the <adapter><number> folder that you are working with.
The Adapter Setup wizard creates the following jobs:
  • one staging job that contains only the part of the ETL flow that relates to the staging transformation and the target staged tables.
  • zero or more aggregation jobs that each contain one Aggregation transformation with target summary tables. The number of aggregation jobs that are created varies based on the domain categories (and their respective subcategories), time periods, and reporting selections that you made when specifying the adapter.
  • zero or more information map jobs that each contain one Information Map transformation per aggregation table. The number of information map jobs that are created varies based on the domain categories (and their respective subcategories), time periods, and reporting selections that you made when specifying the adapter.
  • zero or more reporting jobs that each contain one or more Performance Report transformations per information map. The number of reporting jobs that are created varies based on the domain categories (and their respective subcategories), time periods, and reporting selections that you made when specifying the adapter.
  • zero or more exception jobs that each contain one Exception transformation with target exception tables. The number of exception jobs that are created varies based on the exception definitions that are available for the adapter and domain categories (and the respective subcategories of their staged tables).
The default name of each job that is created by the Adapter Setup wizard contains the name of the adapter and the domain category that you specified with the wizard.
Note: The jobs that the Adapter Setup wizard creates depend on one another to process the raw data for reporting. For example, the aggregation jobs include aggregations that rely on the staged tables that are generated when the staging job is run. As a result, each job should be run only after the jobs that it depends on have been run. The following list shows the types of jobs that the Adapter Setup wizard creates. Each job type in the list is dependent on the job type that precedes it.
  1. staging job
  2. exception job
  3. aggregation job
  4. information map job
  5. reporting job
For information about how to work with the Adapter Setup wizard, see Adapter Setup Wizard.

Where Does the Adapter Setup Wizard Store the Objects That It Generates?

The Adapter Setup wizard and the Add Domain Category wizard store the objects that it generates in predetermined locations within the IT data mart that you select. The wizards generate two sets of subfolders for the objects that it creates:
  • the Staging subfolder, which contains the staging job, staged tables, the library for those tables, and the Exception subfolder if exception definitions exist for the selected adapter and domain category
    Note: The Exception subfolder contains the exception job and the exception tables and the library for those tables.
  • the <Domain Category> subfolder within the Domain Categories folder, which contains the aggregation, information map, and reporting jobs, their associated tables and maps, and the library that is associated with those objects
However, jobs, tables, and libraries can be stored in any location to which you have Write access. To change where an object is stored, right-click the object (in the process flow diagram or in the tree where it is currently stored) and select Properties. On the General tab of the Properties dialog box, click the Browse button that is next to the Location field. The Select a Location dialog box appears. Navigate to the location that you want to use and click OK.
The default location for the staging job, the staged tables, and the associated library is the Staging subfolder that is located within the adapter folder of the IT data mart that you selected.
The following display shows the metadata that is created in the Staging subfolder for the second instance of the SAR adapter within the IT data mart. The Staging subfolder contains the SAR Staging job, the SAR and SARDEV ITRM staged tables, and the library for those tables—SAR Staging 2763. In addition, the display shows the contents of the Exception subfolder. That subfolder contains the SAR2 System Exceptions job, the Exceptions_System target table, and the Exception 2902 library that holds the table.
Default Location of Objects Created by the Adapter Setup Wizard for Staging the Raw Data
Default location of objects created by the Adapter Setup wizard for staging raw data
The default location for the remaining jobs, their associated tables, and library is in the Domain Categories subfolder that is located within the adapter folder of the IT data mart that you selected.
The following display shows the metadata that is created in the Domain Categories subfolder for the second instance of the SAR adapter within the IT data mart. The Domain Categories subfolder contains two subfolders of domain objects: Disk and System. Each domain subfolder contains three aggregated tables, the library associated with those tables, the aggregation job, the information map job, and the reporting job.
Default Location of Objects Created by the Adapter Setup and Add Domain Category Wizards for Aggregating, Creating Information Maps, and Reporting on the Staged Data
Default location of objects created by the Adapter Setup Add Domain Category wizards for aggregating, creating information maps, and reporting on the staged data
Note: Information maps are not generated until the information map job runs. When they are generated, they are stored by default in the appropriate Domain Categories subfolder, although you can store them in any location to which you have Write access.

Processing IT Data without the Adapter Setup Wizard

You can manually generate the metadata for the tasks that process IT resource data instead of invoking the Adapter Setup wizard. To do so, you must first open a job. Then, within that job, you can specify the staging, aggregating, information map, exception, and reporting (Performance Report) transformations that are needed. You can also modify existing jobs by adding and deleting transformations, tables, and information maps.
To generate the metadata for a transformation, perform the following steps:
  1. Open a job. If you want to work with an existing job, double-click on that job to open it on the Diagram tab of the Job Editor window of SAS IT Resource Management.
    If you need to create a job, you can invoke the New Object wizard from the File menu of SAS IT Resource Management. For information about how to perform this task, see Create a Job.
    The new job opens on the Diagram tab of the Job Editor window.
  2. Navigate to the SAS IT Resource Management folder of the Transformations tree of SAS IT Resource Management.
    Note: A job can consist of a single transformation, or it can include multiple transformations such as staging transformations, Aggregation transformations, Information Map transformations, Exception transformations, and Performance Report transformations.
  3. Drop the transformation that you want to work with onto the job in the Diagram tab of the Job Editor window.
  4. Locate and follow the instructions for working with transformations that perform the following tasks:
    • Stage the raw data
      For information about how to stage the raw data, see Staging the Data.
    • [Optional] Process exception definitions
      For information about how to work with the exception analysis process, see Exception Analysis Processing.
    • Aggregate the data
      For information about how to aggregate the data, see Aggregating the Data.
    • Generate information maps of the data
    • Report on the IT intelligence and analysis that SAS IT Resource Management generates
      For information about how to generate, manage, and view reports, see the SAS IT Resource Management 3.7: Reporting Guide.
Note: In order to create (or update) the physical tables that are specified by the transformation, the job that contains the transformation must be executed. Jobs can be run immediately from SAS IT Resource Management, or they can be scheduled to execute as part of the operational procedures of an organization. (Due to the size of the aggregation jobs, it is preferable to run them in batch mode.)
Copyright © SAS Institute Inc. All rights reserved.