Collecting SAP/R3 Statistic data with IT Service Vision

Collecting SAP R/3 Performance data with IT Service Vision

Overview

An SAP R/3 system writes its performance data into a file called a statistic file or stat file. CCMS (Computing Center Management System) gets its data out of this file, presummarizing it as it does so. IT Service Vision reads the statistic file directly to get more detailed data than CCMS.

Up until release 3.1h of SAP R/3 it was possible to read the stat file directly, but since release 4.0 the stat file is compressed and can only be read via SAP R/3 RFCs (Remote Function Calls) that decompress it while being read. These function calls also allow the data to be read directly from within the R/3 system without having to depend on an external copy of the stat file.

The IT Service Vision solution for reading SAP R/3 performance data consists of an ABAP program on each R/3 application server and a C program on the IT Service Vision server. The ABAP program wakes up every hour, reads the data and sends it to a port or socket to which the C program is attached. It also records the timestamp in a control dataset and uses that timestamp as the start time for subsequent extractions. The C program receives the data and stores it in an appropriate location on the IT Service Vision server ready to be processed overnight. The combination of the ABAP and C programs acts like a daemon process. It registers itself in the R/3 system on the application server by using a rfc-destination and a unique id.

The original support for reading the stat file is still available, but you will need to consider transitioning to the new RFC method, especially if Version 4 of R/3 is to be installed. Because the format of the extracted data files is different, you need to select different parameter values on the %CPPROCES macro to differentiate. To process the external copy of the stat file, continue to use TOOLNM=STATBIN. To process the newly extracted format, code TOOLNM=STATRFC. The two file formats are incompatible and cannot be mixed. That is, you cannot process RFC-generated data with TOOLNM=STATBIN and you cannot process a stat file with TOOLNM=STATRFC. But you can continue to use any or all SAP R/3 IT Service Vision tables using either method.

Processing the Version 3 stat file

This is the information which was originally included in the Online Help which details the naming convention necessary for processing stat files. It has been copied here for your convenience.

When processing SAP R/3 stat files, there are some macro variables that are required and some that are optional. The one that is always required is SAPVER. This is used to specify the version of SAP R/3 that produced the stat file (it is not detectable from within the stat file). Different versions of R/3 have different formatted stat files, so this variable must be specified. For example, for version 3.1h you would code:

%let SAPVER=3.1h;

One of the optional macro variables is SAPPLAT. It must be used if you are processing data that has been generated (or your SAP R/3 system runs) on a Windows platform. If this is not the case, do not specify it. Code it as follows:

%let SAPPLAT=WIN;

If you want to process multiple stat files (or a directory of raw data files) then your file names must conform to the naming conventions specified below. You also do not need to specify the other optional macro variables. In fact, if you do specify them, you will receive an error message and processing will stop.

To process multiple stat files on UNIX or a PC, you must:

store all of the files in one directory or folder. This folder must not contain other non-SAP related data files.

specify the RAWDATA= parameter with a value that includes the complete directory path to the folder where the files are located. For example, you might specify RAWDATA= /dept/cpd/ (on UNIX) or RAWDATA= c:\dept\cpd\ (on a PC).

use the following naming convention for all of the raw data files in the directory that is specified by the RAWDATA= parameter:
system1.host.sysnr.sap. 
In this example:

system1 is the name of the SAP System to which the Application Server (identified by host) is connected

host represents the host name of the SAP Application Server from which the file was generated

sysnr represents the R/3 system number

sap represents the extension to the filename. This could be .txt, .data, or any other file extension.

The folder name and its preceding, high-level qualifiers can be named as you wish. For example, some customers choose to timestamp their directory names.

To process multiple data files on MVS:

the value of the RAWDATA= parameter must identify a PDS containing STAT files related to only one SAP system, such as payroll.sapdata,

the PDS should contain a separate member for each SAP Application Server and the member names must represent the application server host name,

specify the SAP System Name by using the SAPSYSNM macro variable,

also specify the SAP System Number by using the SAPSYSNR macro variable,

execute the %CPPROCES macro once for each SAP System, such that the RAWDATA= parameter identifies a different PDS. Re-specify the macro variable assignments for each execution as appropriate.

SAP STAT files must be uploaded in binary form, to MVS data sets of fixed format (i.e., no blocking).

If you are processing only one raw data file:

regardless of the platform, you should specify that file as the value for the RAWDATA= parameter.
you must also specify the SAPHOST, SAPSYSNM, and SAPSYSNR macro variables, as shown in the following example, in order to specify the hostname, system name and system number of the machine running the SAP R/3 application server from which the data came.

%let SAPHOST=sapsrv01;
%let SAPSYSNM=sapsys99;
%let SAPSYSNR=01;

If, within one SAS session, you call the %CPPROCES macro to process one raw data file and then call the %CPPROCES macro again to process multiple raw data files:

you will need to code the following (after the first call and before the second call):

%let SAPHOST=;
%let SAPSYSNM=;
%let SAPSYSNR=;
This is because the SAPHOST, SAPSYSNM, and SAPSYSNR macro variables are global macro variables, and thus remain set until changed, so your values from the first execution might inadvertently be set for the second.

To process a single stat file, use the %CPPROCES as follows:

%let SAPVER=3.0c;
%let SAPSYSNM=FINANCE; 
%let SAPSYSNR=01; 
%let SAPHOST=sapsrv01;
%CPPROCES( sapr3s, 
           collectr=sapr3, 
           toolnm=statbin, 
           rawdata=/usr/tmp/statfile.sap );

To process multiple stat files, use the %CPPROCES as follows:

%let SAPVER=3.0c; 
%CPPROCES( sapr3s, 
           collectr=sapr3, 
           toolnm=statbin, 
           rawdata=/usr/tmp/statfiles/ );

Components of the SAP R/3 RFC interface

The three main components of the interface are:

ABAP program: An ABAP report to be started in batch mode to get the performance data from the R/3 application server.
sasrfcex: A C program which calls RFC functions to collect and store the data in conjunction with the ABAP program. For UNIX platforms, the name "sasrfcex" is prefixed with an abbreviation of the operating system variant, "aix_", "hpux_" or "sol_". Please select the appropriate executable for your UNIX platform.
Rfcstrt.sh: A shell-script to start the sasrfcex program with the necessary parameters.

The RFC program runs on the machine which will receive the stat file data. Normally, this will be the IT Service Vision server.
The RFC program registers itself with the RFC gateway with a unique ID, then waits for the ABAP program to start sending the data.
The batch variant of the ABAP program runs on each instance of each R/3 application server. It contains appropriate parameters for the period of the extract and the destination of the data. Each run creates a uniquely-named extract file on the machine running the RFC program.

Installation

Installation and registration of these features requires SAP R/3 administrator privileges.

On the SAP R/3 system, set up an RFC Destination using transaction code SM59. Create a new entry under the TCP/IP section and name it as you wish. Select a connection type of "T" for "TCP" and enter the appropriate user and password information. On the next screen, select the "Registration" option and, in the "Program" field that subsequently displays, enter systemname.sasrexec, where systemname is the name of the SAP R/3 system on which you are running. Only one RFC destination definition should be required for each SAP system - the multiple batch variants defined below should share this RFC destination and the program associated with it.

For each application server or instance, create an ABAP program using transaction code SE38. Name the ABAP program ZSASITSV and use the code of the same name found in the "misc" or "sasmisc" subdirectory in the IT Service Vision product image. You can either point the transaction screen to the ABAP code or use cut & paste to enter the code directly.

Create a batch variant of the above ABAP program and enter the parameters for it as follows:
- G_DEST
  
  This is required. It should be the name of the RFC Destination (not the program associated with the destination) created in the previous step. The destination is case sensitive - the destination name may have been upper-cased by the SAP R/3 system.
- G_SDAT, G_STIM, G_EDAT, G_ETIM
These are optional, since the last hour’s data will be extracted by default. If you create an online variant (for testing purposes) they are required.
- G_PATH
This is required. It points to the output directory on the IT Service Vision server where the RFC daemon will be running. Because the performance data contains no information about the SAP R/3 server name, sub-system number (gateway service) or system name, the directory name for the data from each application server must be uniquely named following this convention:
- G_LAST_S
This is required. It is used to point to the location of a control dataset that stores the timestamp of the last data extraction. The file is created (if it doesn't exist) and is written to by the SAP R/3 system (not the IT Service Vision server). The file should be called sapsave.dat and the path should again conform to follow the file-naming convention above. For example:
```
/tmp/system-name_server-name_system-number/sapsave.dat         if SAP R/3 is running on unix
```
```
c:\temp\system-name_server-name_system-number\sapsave.dat      if SAP R/3 is running on a PC
```
Do not schedule this batch variant yet. Once everything has been set up and tested, you will then want to schedule it to run every hour.

Create a tree of directories on the IT Service Vision server machine to receive the performance data. It is suggested that the parent directory have a name such as "sapdata". Other, higher level qualifiers can be added as necessary, but underneath subdirectories must be created according to the naming convention described above in the details for the G_PATH parameter description. It will be the "sapdata" directory with which you will subsequently associate the RAWDATA parameter of the %CPPROCES macro.

Create a directory on the IT Service Vision server machine and copy into it the sasrfcex RFC program from the "misc" or "sasmisc" subdirectory in the IT Service Vision product image. On UNIX platforms, there are multiple copies of the sasrfcex executable present in that directory, all prefixed with a 3-character abbreviation for the operating system variant - "aix_" for AIX, "alx_" for Compaq Tru64, "hpux_" for HP-UX and "sol_" for Solaris. Ensure the correct executable is copied. On PC platforms, there is only one copy. Be sure it is stored in a location from where it can be executed by the rfcstrt.sh or rfcex.bat script (detailed in the next step). It will need to be either in the PATH list or executed directly by specifying its exact location.

Copy the script called rfcstrt.sh (if IT Service Vision is running on UNIX) or rfcex.bat (on PC) from the "misc" or "sasmisc" subdirectory in the IT Service Vision product image. Store it in a convenient location form where it can be executed to call the sasrfcex program. Then, tailor the script. The path to the sasrfcex program should be reviewed and modified as necessary. Then, modify the execution parameters, as follows:

the -a parameter should be set to the name of the program associated with the RFC Destination created above,

the -g parameter should be set to the name of the SAP Gateway and

the -x parameter should be set to the name of the Gateway service.

Note that the -a parameter refers to the Program associated with the RFC Destination, not the destination's name itself.

The SAP Gateway is normally the same as the name of the SAP R/3 Application Server and must appear in the local etc/hosts file. Its name and definition can be checked by scanning for its entry in the TCP services file on the R/3 Server.

The Gateway service should already be defined in the SAP R/3 Application Server's TCP services file - it must also be defined in the IT Service Vision server's TCP services file.

For testing, there is another parameter, -t, that produces a trace of the RFC program's activity in the directory where the shell script resides. Simply adding the -t parameter creates a file called dev_rfc (on UNIX) or dev_rfc.trc (on PC) in the directory in which the script was started. This trace file is appended to if it already exists. Be sure to clear or delete it before rerunning the script.

Example for the rfcstrt.sh script on Unix

#!/bin/sh
##########################################################################
RFC=/opt/rfcsdk
export RFC
OLDPATH=$PATH:
PATH=$RFC:$RFC/bin:/dsk/md/tmeset/d18/sasdata/bin:$PATH
export PATH
sasrfcex -a wktest01.sasrexec -g wktest01 -x sapgw20
PATH=$OLDPATH
export PATH

Example for the rfcex.bat batchfile on Windows NT

@set RFC=c:\programme\sappc\sapgui\rfcsdk
@set OLDPATH=%PATH%
@set PATH=%RFC%;%RFC%\bin;%PATH%
sasrfcex.exe -a wktest01.sasrexec -g 194.55.88.19 -x sapgw20
@set PATH=%OLDPATH%

On the IT Service Vision server, start the rfcstrt.sh or rfcstrt.bat script. The script will load the RFC program which will sit and wait for incoming data.

Then, on the SAP R/3 system, execute the batch variant of the ZSASITSV program. Data should be sent to the RFC and stored in the appropriately-named directories on the IT Service Vision server. You should see messages in the log window indicating success or failure.

If it succeeded, you should see files prefixed pfnorm in the location specified in the G_PATH parameter. You may also see files prefixed pfbtc - these can be safely ignored and should not take up much disk space, if any.

If it failed, a return code should appear in the SAP log. The most common failures are codes of:

1

system failure - this could be where the name of the destination was not found or the program/script file was not running.

2

communications failure - this could be an incorrect gateway or gateway service specification or that the SAP R/3 server was not found on the network.

To correct a failure, stop the script execution, change any parameters necessary and restart the script. Then, on the SAP R/3 server, re-execute the batch variant and re-check the log. More information can be found by browsing the dev_rfc file on the IT Service Vision server, even if the script is still executing.

Once you have successfully transferred data, you should consider scheduling a job to execute the ABAP code using transaction code SM36. Name the job to suit your needs and specify the job class and target host (the name of the SAP R/3 server to run on). Create a step and specify the ZSASITSV program and the name you chose for the batch variant. Then, schedule the job to start at the top of the next hour and select "periodic" to specify that you want it to run hourly thereafter. Be sure to save the job.

Use transaction code SM37 to check that the job scheduled correctly. Once data starts accumulating, you can start IT Service Vision and run your process job to read all the raw data files. This process can be scheduled to run on a regular basis. After each execution, you will want to consider moving the data files out of their current location to avoid them being read in subsequent executions. Refer to the IT Service Vision Macro Reference for details on specifying the %CPPROCES macro - there is a parameter (DUPMODE=) which prevents duplicate data from being inadvertently passed into the PDB. See also below for details on using %CPPROCES.

PC and UNIX naming convention

To re-iterate, the data must be stored in directories according to the following naming convention:

/preceeding-folders/system-name_server-name_system-number/

The process macro will then parse the location for these values and store them in the data. For example:

/sapdata/c11_wktest01_20/   on a unix system

c:\sapdata\c11_wktest01_20\   on a PC system

where "c11" is the R/3 system name, "wktest01" is the SAP server name and "20" is the SAP system number (also known as the gateway service number). The files within these directories will be named automatically dependent on the data file being fetched and the timestamp of the fetch.

MVS naming convention

If the data is to be processed on an OS/390 or MVS system, the data should be uploaded there in binary form and stored in either a sequential file or a PDS. If a sequential file is chosen then the following naming convention must be used:

leading-qualifiers.c11.wktest01.N20.PFNORMnn

If a PDS is chosen, then the following naming convention must be used: If a PDS is chosen, then the following naming convention must be used:

leading-qualifiers.c11.wktest01.N20(PFNORMnn)

In both cases, "c11" is the R/3 system name, "wktest01" is the SAP server name and "20" is the SAP system number (also known as the gateway service number). Notice that the system number must be prefixed with an N since MVS dataset qualifiers cannot start with a number. Also notice that the last qualifier or PDS member name can be suffixed with any number to distinguish it from other similar files or members.

How to get the data into the IT Service Vision PDB

The SAP R/3 data must be stored in a location whose name follows the naming conventions listed above. Failure to do so will cause the process macro to not find your data.

If you are currently processing stat file data and are implementing this new RFC support, you must change the external name of your SAPR3S table in your IT Service Vision data dictionary. The external name is used to match the R/3 table names with those of IT Service Vision's. Failure to do so will result in a failure of %CPPROCES. Use the following code to achieve the change:

%cpcat;
cards4;
update table name=sapr3s 
extname=PFNORM;
stop;
;;;;
%cpcat(cat=work.ddutl.sapr3.source);
%cpddutl(entrynam=work.ddutl.sapr3.source);

In IT Service Vision 2.3, there is a new QuickStart Wizard for SAP R/3 data which can be used to easily load data and produce web galleries. This support also includes five new pre-summarized interval tables which provide better support for handling large data volumes. They are SAPHST, SAPSMT, SAPTSK, SAPSYS anbd SAPTRN. For more details, use the online GUI to explore the supplied data dictionary or just run the QuickStart Wizard.

There are three possibilities when processing data from SAP R/3 using the new RFC method:.

Process just one data file: IT Service Vision will ascertain the system name, server name (host name) and system number from the file's parent directory.
Process one directory of data files: IT Service Vision will use the directory name to ascertain the system name, server name (host name) and system number.
Process the parent directory that contains subdirectories of data files: IT Service Vision will open each subdirectory and use its name to ascertain the system name, server name (host name) and system number for each of the files within it.

To specify any of these data locations, either:

Code a FILENAME statement for RAWDATA pointing to the location, or
Specify the RAWDATA= parameter on the %CPPROCES macro.

For PC and UNIX, RAWDATA should refer to either the single file; single directory or parent directory. For MVS, RAWDATA should refer to either the single file or PDS name.

Use the %CPPROCES macro to process the data into the IT Service Vision PDB. Preceed the macro with a macro variable setting that specifies the version of SAP R/3 that is installed on the systems from where the data was extracted. Optionally, add another macro variable setting if (and only if) the data was generated on a Windows platform:

%let sapver = 4.0a;   /* Version of SAP R/3   */
%let sapplat = WIN;   /* Specify only if data was indeed */
                      /* generated on a Windows platform */
%cpproces(rawdata="sapdata/",
          collector=SAPR3,
          toolnm=STATRFC,
          dupmode=DISCARD);

For optimum performance, remember to clean out these files and/or directories for subsequent process executions.