Configure Job Statistics Reporting for SAS Data Integration Studio :: SAS(R) 9.3 Intelligence Platform: System Administration Guide, Second Edition

Summary of Steps for Configuring the Job Statistics Reporting Package

The job statistics reporting package is delivered with the SAS Data Integration Studio client application. Before you can use the package, complete the following steps to install it on your server tier and configure it for your environment:

Create directories on the batch server host machine.
Copy the reporting components to the batch server host machine.
Customize the properties file, job logs parsing program, and batch script for your environment.
Add the Java Runtime Environment executable to the PATH environment variable.
Import the BI reporting objects from DIStudioJobStats.spk. This step is necessary only if you will be using SAS Web Report Studio to generate the job statistics reports.
Import the stored processes and customize them for your environment. This step is optional if you will be using SAS Web Report Studio to generate the job statistics reports.
Enable ARM logging and run-time statistics collection for deployed SAS Data Integration Studio jobs.
Apply the required permissions.

Details are provided in the following topics.

Step 1: Create Directories on the Batch Server Host Machine

First, you must create directories on the SAS DATA step batch server host machine. Follow these steps on the host machine that runs the jobs for which you want to report statistics:

Create a directory to store the reporting package’s programs, scripts, and configuration information. For example:
- On Windows: C:\SASDIStudioJobReports
- On UNIX: /usr/local/SASDIStudioJobReports
Create a directory to be used as the library location for the job statistics data sets. For example:
- On Windows: C:\SASDIStudioJobStats
- On UNIX: /usr/local/SASDIStudioJobStats
Give Write access to this directory for the user who will build the OLAP cubes for job statistics reporting. (This step is required only if you will be using SAS Web Report Studio to generate reports).
Create a directory to contain the source code for stored processes that produce job statistics reports. (This step is optional if you will be using SAS Web Report Studio to generate reports.) Here are examples:
- On Windows: C:\SASDIStudioJobStoredProcesses
- On UNIX: /usr/local/SASDIStudioJobStoredProcesses
Give Write access to this directory for the user who will import the stored process objects. (See Step 6: Import Stored Processes and Customize Them for Your Environment.)
Note: In the subsequent configuration steps, the directories are referred to using the example names.

Step 2: Copy Reporting Components to the Batch Server Host Machine

On the machine where SAS Data Integration Studio client software is installed, navigate to the following directory:

SAS-installation-directory/SASDataIntegrationStudio/4.3/jobreports

Copy the contents of the jobreports directory from the client machine to the SASDIStudioJobReports directory on the batch server machine. The resulting directory should have the following structure:

Step 3: Customize the Properties File, Job Logs Parsing Program, and Batch Script for Your Environment

On the batch server machine, follow these steps to customize the properties file, job logs parsing program, and batch scripts for your environment:

In the path SASDIStudioJobReports/batch, locate the job statistics properties file for your operating system environment (for example, DIStudioJobStats.properties.Windows). Copy this file to DIStudioJobStats.Properties.

Then open DIStudioJobStats.Properties in a text editor and make the following changes:
1. Find the line that contains DIStudioJobStats.logdir=. Update this property to point to the location of the logs for your batch server. For example:
  
  DIStudioJobStats.logdir=C\:\\SAS\\Config\\Lev1\\SASApp\\BatchServer\\Logs
  
  Note that in Windows environments, each colon (:) and backslash (\) must be preceded by an additional backslash character.
  
  Note: For best results, the batch server logs should be stored in their own directory rather than in a directory that is shared by other server logs. To determine the configured location, open the batch server’s logconfig.xml file (located in SAS-configuration-directory/Lev1/SASApp/BatchServer). In this file, check the FileNamePattern parameter of the RollingFileAppender configuration. If a unique location is not specified in this parameter, specify a location such as SAS-configuration-directory/Lev1/SASApp/BatchServer/Logs.
2. Find the line that contains DIStudioJobStats.user=. This property should specify the user ID for the SAS General Server account (sometimes referred to as the SAS Spawned Server account) that is defined in your operating system.
  
  Typically, the user ID for this user is sassrv. If a different user is specified, update this property with the appropriate user ID.
3. Find the line that contains DIStudioJobStats.password=. Update this property with the password of the user that is specified in DIStudioJobStats.user=. The password should be encrypted using SAS proprietary 32-bit encryption. To obtain the encrypted password, use PROC PWENCODE. Specify the encrypted password as shown in the following example:
  
  DIStudioJobStats.password={sas002}0123456789
4. Find the line that contains DIStudioJobStats.librefs=. Update this property with the directory location that you created for the job statistics data sets. For example:
  
  DIStudioJobStats.librefs=C\:\\SASDIStudioJobStats
5. Find the line that contains DIStudioJobStats.sqlurl. Update this property with the host name and port for the SAS Workspace Server. For example:
  
  DIStudioJobStats.sqlurl=jdbc:sasiom://HOST01.example.com:8591
6. Find the line that contains DIStudioJobStats.overwriteTables=. Make sure that this property is set to true.
  
  Note: Later, after you run the log parsing program for the first time, this setting should be changed to false.
  
  Save the file.
In SASDIStudioJobReports/batch, locate the file joblogsFlow.sas. Open this file in the SAS Program Editor or a text editor, and find the LIBNAME statement. Update the LIBNAME statement to specify the directory location for the job statistics data sets. This should be the same path that you specified in step 1.c. For example:
```
libname jobstats 'C:\SASDIStudioJobStats';
```
Save the file.
In the path SASDIStudioJobReports/batch, locate the file joblogsFlow.bat or joblogsFlow.sh. Open this file in a text editor and make the following changes:
- Edit the path to the batch server command to specify the correct path for your environment. For example:
  C:\SAS\Config\Lev1\SASApp\BatchServer\sasbatchjob
- Edit the value for the -log parameter to specify an appropriate path and filename for the parsing program’s log file. For example:
  -log "C:\SASDIStudioJobReports\batch\logs\joblogsFlow.log"
- Edit the value for the -sysin parameter to specify the location of the joblogsFlow.sas program. For example:
  -sysin "C:\SASDIStudioJobReports\batch\joblogsFlow.sas"
Save the file.

Step 4: Add the Java Runtime Environment Executable to the PATH Environment Variable.

The SAS Data Integration Studio job statistics reporting package requires version 1.5 or higher of the Java Runtime Environment (JRE). This software should have been installed on your batch server machine when you installed your SAS software.

Update the PATH environment variable on your machine to include the path to the JRE executable.

Step 5: Import BI Reporting Objects from DIStudioJobStats.spk

The package file DIStudioJobStats.spk contains BI objects (including information maps, cubes, SAS Web Report Studio reports, table definitions, and a library) that are used to produce SAS Data Integration Studio job statistics reports.

Note: Import this package only if you will be using SAS Web Report Studio to generate the job statistics reports. Otherwise, follow the instructions in Step 6: Import Stored Processes and Customize Them for Your Environment.

To import DIStudioJobStats.spk, follow these steps:

Log on to SAS Management Console on a machine that has access to the directories on the DATA step batch server host machine.
On the Folders tab, right-click the root folder (SASFolders) or another folder where you want to install the package. Then select Import SAS Package.
On the first Import SAS Package wizard page, click Browse and navigate to SASDIStudioJobReports/wrs in the file system of the batch server machine. Select the file DIStudioJobStats.spk to import.

If you are importing to the root folder (SASFolders), deselect the Preserve source path information for objects in the package check box. Click Next.
On the second wizard page, click Next to accept the defaults. If a warning is displayed about directly applied access control entries, click Yes to proceed.
The third wizard page displays information about required metadata connections for the imported objects. Click Next.
The next several wizard pages prompt for metadata connections such the SAS Application Server and OLAP schema. Specify values that are appropriate for your environment.
When you are prompted for report objects, click Next to accept the defaults.
The next wizard page prompts for a directory path, as shown here:

In both rows of the Target column, browse to the path for your job statistics data library. Specify the same path that you specified in DIStudioJobStats.Properties and joblogsFlow.sas (for example, C:\SASDIStudioJobStats).
The next wizard page provides a summary of the objects to be imported. Click Next to begin the import.

The import creates a metadata folder called DIStudioJobReports. The folder contains tables, reports, jobs, maps, cubes, and a library.

Step 6: Import Stored Processes and Customize Them for Your Environment

Import DIStudioJobStatsStoredProcess.spk

The package file DIStudioJobStatsStoredProcess.spk contains stored processes that are used to produce SAS Data Integration Studio job statistics reports.

Note: The stored processes are provided for sites that do not have SAS Web Report Studio. If you will be using SAS Web Report Studio to produce the reports, this step is optional.

To import the file DIStudioJobStatsStoredProcess.spk, follow these steps:

Make sure that you have Write permission on the directory that was created for the stored process source code (for example, /SASDIStudioJobStoredProcesses).
On the Folders tab of SAS Management Console, right-click the DIStudioJobStats folder, which was created in Step 5: Import BI Reporting Objects from DIStudioJobStats.spk. Then select Import SAS Package.
On the first Import SAS Package wizard page, click Browse and navigate to SASDIStudioJobReports/storedprocess in the file system of the batch server machine. Select the file DIStudioJobStatsStoredProcess.spk to import. Click Next.
On the second wizard page, click Next to accept the defaults.
The third wizard page displays information about required metadata connections for the imported objects. Click Next.
The fourth wizard page prompts for a metadata connection to a SAS Application Server. Specify the value that is appropriate for your environment, and click Next.
The fifth wizard page prompts for a directory path for source code repositories. Specify the directory as follows:
1. On the Target source code repositories tab, click Add.
2. In the Add Source Code Repository dialog box, specify the source code directory that you created in Step 1: Create Directories on the Batch Server Host Machine (for example, C:\SASDIStudioJobStoredProcesses). Click OK.
The directory should appear in the Target column on the wizard page. Click Next.
The sixth wizard page provides a summary of the objects to be imported. Click Next to begin the import, and then click Finish on the final wizard page.

The DIStudioJobStats folder should now contain three stored processes. In addition, the source code directory in the file system should contain three SAS programs.

Customize the Stored Process Source Code for Your Environment

After you import DIStudioJobStatsStoredProcess.spk, you must customize the stored process source code for your environment, as follows:

In the file system of the batch server host machine, navigate to the directory that contains the source code for the stored processes (for example, C:\SASDIStudioJobStoredProcesses).
If you are using a UNIX machine, make sure that you have Edit permission (chmod 664) for the three stored process programs in the directory (disjobmesgsreport.sas, disjobparammesgs.sas, and disjobperformancereport.sas).
Open each of the three programs that are stored in the directory, and locate the following lines in the programs:
- In disjobmesgsreport.sas:
  %isExist(libpath=d:\jobstats)
  %preProcessData(libpath=d:\jobstats)
- In disjobparammesgs.sas:
  %prepareData(libpath=d:\jobstats,jobname=,jobnumber=)
- In disjobperformancereport.sas:
  %isExist(libpath=d:\jobstats)
In each of these lines, edit the value for LIBPATH to specify the path for your job statistics data library. Specify the same path that you specified in DIStudioJobStats.Properties and joblogsFlow.sas (for example, C:\SASDIStudioJobStats).

Step 7: Enable ARM Logging and Run-Time Statistics Collection for Deployed SAS Data Integration Studio Jobs

To enable the collection of data for job statistics reporting for deployed SAS Data Integration Studio jobs, follow these steps:

Set up ARM logging and run-time statistics collection by following the instructions in About Collecting ARM Log Information for SAS Data Integration Studio Jobs.
To enable the collection of statistics for deployed jobs, select ToolsOptions in SAS Data Integration Studio. Then deselect the Override and disable performance statistics when a job is deployed check box, and click OK.

Step 8: Apply Required Permissions

To enable the creation and viewing of SAS Data Integration Studio job statistics reports, you must apply the following permissions to files, directories, metadata folders, and objects:

Directory, File, Metadata Folder, or Object	User	Required Permission
Directory that stores programs, scripts, and configuration information for the reporting package For example: `/SASDIStudioJobReports`	User who is specified in the DIStudioJobStats.user property of DIStudioJobStats.Properties, and who runs joblogsFlow.bat or joblogsFlow.sh For example: sassrv	Write
Library location for job statistics data sets For example:`/SASDIStudioJobStats`	User who is specified in the DIStudioJobStats.user property of DIStudioJobStats.Properties, and who runs joblogsFlow.bat or joblogsFlow.sh (for example, sassrv) User who builds cubes1	Write
DIStudioJobStats.sh and joblogsFlow.sh	User who is specified in the DIStudioJobStats.user property of DIStudioJobStats.Properties, and who runs joblogsFlow.bat or joblogsFlow.sh For example: sassrv	Execute (chmod +x )
OLAP schema object For example: SASApp - OLAP Schema	User who builds cubes1	Read
Metadata folder that contains the DI Job Step Messages cube and the DI Job Step Stats cube For example: `DIStudioJobStats`	User who builds cubes1	ReadMetadata
DI Job Step Messages cube, DI Job Step Stats cube, DI Job Step Messages cube job, and DI Job Step Stats cube job	User who builds cubes1	WriteMetadata
Metadata folder that contains information maps for the reporting package For example: `DIStudioJobStats`	User who accesses the SAS Web Report Studio reports1	Read and ReadMetadata
1Required only if you will be using SAS Web Report Studio to generate reports.

Configuration of the SAS Data Integration Studio job statistics reporting package is now complete. The following topics provide instructions for the ongoing production of job statistics reports.