Using SAS Logs

The SAS logs from your process flows are an excellent resource to help you understand what is happening as the flows execute. For example, when you look at the run times in the log, compare the real-time values to the CPU time (user CPU plus system CPU). For Read operations, the real time and CPU time should be close. For Write operations, however, the real time can substantially exceed the CPU time, especially in environments that are optimized for Read operations. If the real time and the CPU time are not close, and they should be close in your environment, investigate what is causing the difference.

If you suspect a hardware issue, see the document "A Practical Approach to Solving Performance Problems with the SAS System," which is available from the "Scalability and Performance Papers" page at Scalability and Performance Papers.

If you determine that your hardware is properly configured, then review the SAS code. Transformations generate SAS code. Understanding what this code is doing is very important to ensure that you do not duplicate tasks, especially SORTs, which are resource-intensive. The goal is to configure the hardware so that there are no bottlenecks, and to avoid needless I/O in the process flows.

If you need to examine additional performance statistics, you can right-click in an open job and click Collect Runtime Statistics in the pop-up menu. After you run the job, you can review the statistics that are generated in the run on the Statistics tab of the Details pane. You can display the statistics in the form of a table, a line graph, or a bar chart.

Another way to analyze performance is to turn on the following SAS options so that detailed information about the SAS tasks is captured in the SAS log:

FULLSTIMER 
MSGLEVEL=I (this option prints additional notes pertaining to index, merge 
   processing, sort utilities, and CEDA usage, along with the standard notes, 
   warnings, and error messages) 
SOURCE, SOURCE2 
MPRINT 
NOTES

To interpret the output from the FULLSTIMER option, see the document "A Practical Approach to Solving Performance Problems with the SAS System," which is available from the "Scalability and Performance Papers" page at Scalability and Performance Papers.

In addition, the following SAS statements also send useful information to the SAS log:

PROC OPTIONS OPTION=UTILLOC; run; 
PROC OPTIONS GROUP=MEMORY; run; 
PROC OPTIONS GROUP=PERFORMANCE; run; 
LIBNAME _ALL_ LIST;

The PROC OPTIONS statement sends SAS options and their current settings to the SAS log. There are hundreds of SAS options. If you prefer to see which value has been set to the SAS MEMORY option, you can issue the PROC OPTIONS statement with the GROUP=MEMORY parameter. The same is true if you want to see only the SAS options that pertain to performance.

The LIBNAME _ALL_ LIST statement sends information (such as physical path location and the engine that is being used) to the SAS log about each libref that is currently assigned to the SAS session. This data is helpful for understanding where all the work occurs during the process flow. For details about setting SAS invocation options for SAS Data Integration Studio, see Set SAS Invocation Options on Jobs.

The Process Designer window in SAS Data Integration Studio has a Log tab that displays the SAS log for the job in the window. Perform the following steps to display or hide the Log tab:

The SAS log for a job provides critical information about what happened when a job was executed. However, large jobs can create large logs, which can slow down SAS Data Integration Studio. In order to avoid this problem, you can redirect the SAS log to a permanent file. Then, you can turn off the Log tab in the Job Editor window.

When you install SAS Data Integration Studio, the Configuration Wizard enables you to set up permanent SAS log files for each job that is executed. The SAS log filenames contain the name of the job that creates the log, plus a timestamp of when the job is executed.

Alternatively, you can add the following code to the Precode and Postcode tab in the properties window for a job:

proc printto log=...path_to_log_file NEW; run;

For details about adding pre-process code to a SAS Data Integration Studio job, see Specifying Options for Jobs. This code causes the log to be redirected to the specified file. Be sure to use the appropriate host-specific syntax of the host where your job is running when you specify this log file, and make sure that you have Write access to the location where the log is written.