%RMRUNETL

%RMRUNETL Overview

This is the order that the ETL jobs are run by %RMRUNETL:
  1. the staging job
  2. the aggregation jobs
  3. information map jobs
  4. the report jobs
Note: The aggregation, information map, and report jobs are run in parallel using Multi-Processing (MP) Connect.
If the staging job fails, the aggregation, information map, and report jobs are not run. After each job completes, the SAS log is examined by the program. The completion status of the job (Success, Warning, or Error) is sent by e-mail to the specified recipient. If the log displays warnings or errors, the e-mail includes the text of the error or warning along with the corresponding line number in the log file.

%RMRUNETL Syntax

%RMRUNETL(
<ADAPTER=adapter-name>
,<AGGJOB1=path-to-aggregation-job1>,...,<AGGJOB20=path-to-aggregation-job20>
,<AGGLOG1=path-to-aggregation-log1>,...,<AGGLOG20=path-to-aggregation-log20>
,<EMAILFROM=sending-email-address>
,<EMAILTO=email.address@company.com >
,<INFOMAPJOB1=path-to-information-map-job1>,...,<INFOMAPJOB20=path-to-information-map-job20>
,<INFOMAPLLOG1=path-to-information-map-log1>,...,<INFOMAPLOG20=path-to-information-map-log20>
,<REPORTJOB1=path-to-report-job1>,...,<REPORTJOB20=path-to-report-job20>
,<REPORTLOG1=path-to-report-log1>,...,<REPORTLOG20=location-of-report-log20>
,<RUNDATE=date-that-this-macro-is-run>
,<STAGINGJOB=path-to-staging-job>
,<STAGINGLOG=path-to-staging-log>
);

%RMRUNETL Required Arguments

There are no required parameters for this macro. All parameters are optional. However, you must specify at least one job in order to run this macro.

%RMRUNETL Options

  • ADAPTER=adapter-name
    This value is used in the subject of the e-mail messages that are sent out and in the name of the audit table. This option can be left blank.
  • AGGJOB1=path-to-aggregation-job1...AGGJOB20=path-to-aggregation-job20
    specifies the paths to a maximum of 20 aggregation jobs. These jobs will be run in parallel.
    Note: The path must include the name of the file that contains the job.
  • AGGLOG1=path-to-aggregation-log1...AGGLOG20=path-to-aggregation-log20
    specifies the paths to a maximum of 20 aggregation logs.
    Note: The path must include the name of the file that contains the log.
  • EMAILFROM=email.address@company.com
    specifies the e-mail address from which the e-mail is to be sent. If blank, this value is set to the address that is specified in EMAILTO.
  • EMAILTO=email.address@company.com
    specifies the e-mail address to which the status e-mail is to be sent. If this value is blank, a warning is written in the log that e-mail is not going to be sent.
  • INFOMAPJOB1=path-to-information-map-job1...INFOMAPJOB20=path-to-information-map-job20
    specifies the paths to a maximum of 20 information map jobs. These jobs will be run in parallel.
    Note: The path must include the name of the file that contains the job.
  • INFOMAPLOG1=path-to-information-map-log1...INFOMAPLOG20=path-to-information-map-log20
    specifies the paths to a maximum of 20 information map logs.
    Note: The path must include the name of the file that contains the log.
  • REPORTJOB1=path-to-report-job1...REPORTJOB20=path-to-report-job20
    specifies the paths to a maximum of 20 report jobs. These jobs will be run in parallel.
    Note: The path must include the name of the file that contains the job.
  • REPORTLOG1=path-to-report-log1...REPORTLOG20=path-to-report-log20
    specifies the paths to a maximum of 20 report logs.
    Note: The path must include the name of the file that contains the log.
  • RUNDATE=date-that-this-macro-is-run
    specifies the date that is used in e-mail messages and in log filenames. If this value is blank, it is set to today's date.
  • STAGINGJOB=path-to-staging-job
    specifies the path to the staging job that you want to run.
    Note: The path must include the name of the file that contains the job.
  • STAGINGLOG=path-to-staging-log
    specifies the path to the staging log that is produced by the staging job. If the STAGINGJOB option is specified, then the STAGINGLOG must also be specified.
    Note: The path must include the name of the file that contains the job.

%RMRUNETL Notes

This macro can be run in batch mode, or interactively from the SAS Display Manager. It can run Windows, UNIX, or z/OS jobs that are stored in a PDS.
If an option for a job is specified, the corresponding log option must also be specified. For example, if there is a path specified for the AggJob2 option, the AggLog2 option cannot be left blank.

%RMRUNETL Examples

Example 1: Running a Staging Job, Two Aggregation Jobs, and a Report Job

This example shows the code that runs a staging job, two aggregation jobs, and a report job. E-mail is to be sent to and from the same e-mail address. The RUNDATE option defaults to today's date.
%rmrunetl(adapter=SNMP
   ,aggJob1=C:\jobs\_1_aggJob.sas
   ,aggJob2=C:\jobs\_2_aggJob.sas
   ,aggLog1= C:\jobs\logs\_1_aggJob.log
   ,aggLog2= C:\jobs\logs\_2_aggJob.log
   ,emailto=email@address.com
   ,reportJob1= C:\jobs\_1_reportJob.sas
   ,reportLog1= C:\jobs\_1_reportJob.log
   ,stagingJob=C:\jobs\staging.sas
   ,stagingLog= C:\jobs\logs\staging.log
);

Example 2: Running an MS SCOM Job on Windows

MS_SCOM_Overall.bat: This code shows the Windows bat job that runs the MS_SCOM_Overall.sas program.
echo off

set ADAPTER=MSSCOM
set DATETIME=%date:~10,4%%date:~4,2%%date:~7,2%_%time:~0,2%%time:~3,3%
set BASE=C:\Data\Jobs\%ADAPTER%
set LOGDIR=%BASE%\logs
set JOBDIR=C:\SAS\ITRM33\Lev1\SASITRM\SASEnvironment\SASCode\Jobsset SRC=MS_SCOM_Overall
set SAS=C:\SAS\ITRM33\Lev1\SASITRM\sas.bat

%SAS% -sysin %BASE%\%SRC%.sas -log %LOGDIR%\%SRC%_%DATETIME%.log
MS_SCOM_Overall.sas: This code shows the SAS program that runs all the daily jobs (staging, aggregation, and reporting) for an MS SCOM data mart.
%let adapter = %sysget(ADAPTER);
%let datetime = %sysget(DATETIME);
%let JOBhome = %sysget(JOBDIR);
%let LOGhome = %sysget(LOGDIR);

%rmrunetl(adapter = &ADAPTER
 ,stagingJob = &JOBhome\MS_SCOM_Staging.sas
 ,stagingLog = &LOGhome\ MS_SCOM_Staging_&datetime..log
 ,aggJob1 = &JOBhome\InfoStore_Aggregation.sas
 ,aggJob2 = &JOBhome\InfoStoreMailbox_Aggregation.sas
 ,aggJob3 = &JOBhome\Disk_Aggregation.sas
 ,aggJob4 = &LOGhome\System_Aggregation.sas
 ,aggLog1 = &LOGhome\InfoStore_Aggregation_&datetime..log
 ,aggLog2 = &LOGhome\InfoStoreMailbox_Aggregation_&datetime..log
 ,aggLog3 = &LOGhome\Disk_Aggregation_&datetime..log
 ,aggLog4 = &LOGhome\System_Aggregation_&datetime..log
 ,reportJob1 = &JOBhome\InfoStore_Daily_Reporting.sas
 ,reportJob2 = &JOBhome\InfoStoreMailbox_Daily_Reporting.sas
 ,reportJob3 = &JOBhome\Disk_Daily_Reporting.sas
 ,reportJob4 = &JOBhome\System_Daily_Reporting.sas
 ,reportLog1 = &LOGhome\InfoStore_Daily_Reporting_&datetime..log
 ,reportLog2 = &LOGhome\InfoStoreMailbox_Daily_Reporting_&datetime..log
 ,reportLog3 = &LOGhome\Disk_Daily_Reporting_&datetime..log
 ,reportLog4 = &LOGhome\System_Daily_Reporting_&datetime..log
 ,email = email.address@company.com
 );

Example 3: HP_Perf_Agent_Overall:

This code shows the UNIX script (bash, Bourne and related shells) that runs the HP_Perf_Agent_Overall.sas program.
export ADAPTER=HPPerfAgent 
export BASE=/var/opt/sas/prog/prod/$ADAPTERexport LOGDIR=$BASE/logs
export JOBDIR=/opt/sas/ITRM/itrm33srv/Lev1/SASITRM/SASEnvironment/SASCode/Jobs
export SRC=HP_Perf_Agent_Overall
export DATETIME=$(date +"%Y%m%d_%H%M")ADAPTER=HPPerfAgent

SAS=/opt/sas/ITRM/SAS/SASFoundation/9.3/sas

$SAS –sysin $BASE/$SRC.sas -log $LOGDIR/"$SRC"_"$DATETIME".log -noterminal -rsasuser &
HP_Perf_Agent_Overall.sas: This code shows the SAS program that runs all the daily jobs (staging, aggregation, and reporting) for an HP Perf Agent data mart.
%let adapter = %sysget(ADAPTER);
%let datetime = %sysget(DATETIME);
%let JOBhome = %sysget(JOBDIR);
%let LOGhome = %sysget(LOGDIR); 

%rmrunetl(adapter = &adapter
 ,stagingJob = &JOBhome/HP_Perf_Agent_Staging.sas
 ,stagingLog = &LOGhome/HP_Perf_Agent_Staging_&datetime..log
 ,aggJob1 = &JOBhome/DiskUnix_Aggregation.sas
 ,aggJob2 = &JOBhome/SystemUnix_Aggregation.sas
 ,aggLog1 = &LOGhome/DiskUnix_Aggregation_&datetime..log
 ,aggLog2 = &LOGhome/SystemUnix_Aggregation_&datetime..log
 ,reportJob1 = &JOBhome/DiskUnix_Daily_Reporting.sas
 ,reportJob2 = &JOBhome/SystemUnix_Daily_Reporting.sas
 ,reportLog1 = &LOGhome/DiskUnix_Daily_Reporting_&datetime..log
 ,reportLog2 = &LOGhome/SystemUnix_Daily_Reporting_&datetime..log
 ,emailto = email.address@company.com
 );