SNMP — Accessing Raw Data :: SAS(R) IT Resource Management 3.7: Administrator's Guide

Sources of SNMP Data

Simple Network Management Protocol (SNMP) data can be collected from the following sources:

HP Network Node Manager (HP NNM)
Round-Robin Database

Reporting on the data available from the SNMP adapter can be enhanced by mapping devices to their device type. For information about how to enable this functionality, see Working with the ITMS_SNMP_DeviceType Lookup Table.

The SNMP adapter is designed with a data model that contains MIB2 data and can be extended to other MIBs. For information about how to do so, see ForWorking with Management Information Base (MIB) Definitions for SNMP Staged Tables.

In order to read the raw data from either HP NNM or RRDtool, it is necessary to create Perl scripts as part of the staging process. This means that Perl must be installed on the machine that the SAS program runs on, that is, the SAS Workspace Server, the SAS batch server, or interactive SAS.

Note: If you run the staging job without having the Perl script installed, then the staging transformation cannot execute successfully to extract the data. You might find the following text about the missing Perl script in the SAS log file:

Stderr output: 'perl' is not recognized as an internal or external command, operable program or batch file. NOTE: 0 records were read from the infile RAWDATA.

The Perl scripts that are used by the staging code are generated as part of the code generation of the staging job. This code can be viewed and modified as necessary in the Code screen of the job. To run these scripts in batch mode, set the XCMD option for the Batch server. To do so, change set USERMODS_OPTIONS= to

set
                     USERMODS_OPTIONS=XCMD

.

By default, on the Workspace Server, this option is turned off. For instructions to turn on the XCMD option, see How to Turn On the XCMD Option.

Extracting Data By Using HP NNM Tools

For versions of HP NNM that are earlier than HP NNMi 8.x, there are tools that produce a repository of SNMP data that can be processed the SAS IT Resource Management solution.

The two commands to produce the SNMP data are:

snmpCollect, which gathers SNMP data into binary files
snmpColDump, which reads the binary files and writes the data into ASCII files that can be processed by the SNMP staging transformation

The snmpCollect executable uses a file called snmpCol.conf to determine the metrics to collect and the devices from which to gather data.

You can create the snmpCol.conf file by using either of the following methods.

Use the sample UNIX script (snmpmkcf) that SAS IT Resource Management provides. The sample script reads the snmphost file and the snmpvars file and creates the snmpCol.conf file. The sample script and the snmphost and snmpvars files are in a location that is based on the operating environment.

Windows Specifics: SASHome\SASFoundation\9.4\itmsmvadata\sasmisc

UNIX Specifics: SASHOME/SASFoundation/9.4/misc

z/OS Specifics: &prefix.ITRM.CPMISC
- The snmphost file contains a list of the device names for which SNMP data is being gathered.
  
  Note: The job can run successfully even if, in the Perl script, the alias of the host name contains a hyphen (-) or an underscore (_).
- The snmpvars file contains a list of the SNMP metrics that are to be gathered from each device. By default, the values in this list are the metrics that are in the recommended staging tables.
Note: This method creates the snmpCol.conf file more quickly than using the NNM graphical user interface.
Use the HP NNM graphical user interface to create snmpCol.conf. The user interface prompts you to select the metrics that you want to gather and the devices about which you want to gather data. Using your responses, it then creates the snmpCol.conf file.

When the snmpCol.conf file is created, you can run the snmpCollect command to gather data. You can run the collect as often as you choose. The command collects a set of files in the location that you specify. For each metric and interface, two files are generated:

metricName.interface-number, that contains the raw data
metricName.interface-number!, that contains the information about the metrics that are collected

For example, the ifOctets.3 and ifOctets.3! files are generated for the metric that is called ifOctets and for interface 3. The SNMP staging code runs the snmpColDump command against these files to get the raw data that is to be read and loaded into staged tables. In order to read the data, the snmpColDump executable requires both the file that contains the binary data and file that contains the information about the metrics that are collected.

Tip

The binary data files continue to accumulate until they are deleted. For best results, these data files (whose names do not end with "!") should be moved to another location before staging. The snmpCollect command then creates a new set of binary files into which the data is to be collected. The files that contain the information about the metrics (whose names end with "!") should be copied (not moved) to the other location as well. In this way, the snmpCollect command can continue to use these metrics as it collects the raw data. The staging code can then read the files from the location to which they were moved or copied.

Extracting Data By Using the Round-Robin Database Tool (RRDTOOL)

The SNMP adapter can read data from a round-robin database that was created with RRDtool.

Note: The round-robin database tool, RRDtool, is a freeware package that is available for download from Tobias Oetiker. This software is widely used, often in conjunction with Tobias Oetiker's MRTG package, which is a real-time monitor of data.

The SNMP adapter reads only those round-robin databases that contain one round-robin database for each combination of the device, the network interface to which the device is connected, and the metric. The data can be gathered at any interval, and any metrics and devices can be gathered.

In order to produce the correct set of round-robin databases, SAS IT Resource Management provides a sample Perl script (snmpuprd.pl) for data collection and storage in the round-robin database. The section at the top of the script enables you to specify modifications that control how the data is collected. The Perl script uses the snmphost and snmpvars files to determine which metrics to gather from your devices. (The Perl script and the snmphost and snmpvars files are in a location that is based on the operating environment.

Windows Specifics: SASHome\SASFoundation\9.4\itmsmvadata\sasmisc

UNIX Specifics: SASHOME/SASFoundation/9.4/misc

z/OS Specifics: &prefix.ITRM.CPMISC

The Perl script also uses the snmpwalk command to collect the snmp data. When the script is correctly set up, you can use your operating system's scheduling facility to run the script at the appropriate interval.

Note: In some instances, the data might be collected by a tool other than the snmpuprd.pl script that is provided with SAS IT Resource Management. In that case, it might be necessary to modify the staging code to handle the format differences. This code can be modified by one of the following methods:

Edit the code from the staging job’s Code window. To do so, change the Code Generation Mode option at the top of the screen to All user written.
Deploy the job. The deployed code can be modified from an external editor. The code to be modified begins with this comment line:
```
/*---- Create script to read data from an RRD  ----*/
```

Note: You can backload data from the rrdtool. To do so, specify the Number of days to load option in the Staging Parameters property tab of the SNMP staging transformation. (This option is available only if the chosen access command is RRDTool.) For information about backloading, see How to Backload Raw Data.

Notes about the SNMPWALK Command

The snmpwalk command is an executable that is available as part of the Net-SNMP package. This package is available as freeware that you can download from the Sourceforge.net website. The snmpwalk command provides an easy way to read SNMP data from specified devices.

The SNMP adapter uses snmpwalk in two places:

Using RRDtool: If you choose to store your raw data using RRDtool, the script that is provided with SAS IT Resource Management uses snmpwalk to gather the data to put in the round-robin database.
Staging the data: The SNMP staging transformation provides the ability to collect the non-numeric data at staging time instead of at the normal regular intervals that the rest of the SNMP data is collected. This can be useful for character data that does not change frequently. If you want to collect character data and are using RRDtool to gather your raw data, then you must collect it at staging time, because RRDtool does not store character data. If you are using HP NNM, you can set it up to gather the character data at regular intervals along with the rest of the data. Alternatively, you can choose to collect it at staging time.

If you want to use the snmpwalk method, go to the Staging Parameters tab of the Properties dialog box for the SNMP transformation. On the Use snmpwalk to gather character data option, select Yes. Then fill in values for the following options:
- snmpwalk executable: Enter the path and the .exe file that contains the snmpwalk command.
- HostFile for snmpwalk: Enter the path and file that contains a list of the devices to poll for character data.
- Community value for snmpwalk: Check with your SNMP administrator to ascertain the correct value for your site. If you choose to use snmpwalk, then a script is generated and executes when the staging code runs. This script executes the snmpwalk command in order to gather the SNMP data for specified character metrics and adds that data directly to the staged tables.
  
  The typical value for this option is public.

Working with the SSH Host Command

The SSH host command is an executable object that is available as part of the functionality of the Ganglia, RRDtool, and SNMP adapters. This command specifies the RSH or SSH version of the command and the name of the host for running the rrdtool command. For more information about the SSH Host command, see Notes about the SSH Host Command.

Working with the ITMS_SNMP_DeviceType Lookup Table

To provide better reporting, the SNMP adapter includes a SAS data table that enables you to indicate the type of device for each device for which you are gathering data. This table is located by default in the SASHELP library and contains no data. To use this table to define your device types, perform the following steps from the SAS IT Resource Management client:

Locate, or create, the IT data mart that you want to use for staging SNMP data.
Locate the Administrative folder. Right-click the library within that folder. (The library is called Adminnnnn, where nnnn is a random number that was generated when the IT data mart was created.)

Select View Libname.
Select the entire LIBNAME statement in the displayed box. Then right-click the LIBNAME statement and select Copy.
Launch an interactive session of SAS. Use the Paste function to paste the LIBNAME statement into an Editor window.
Submit the LIBNAME statement.
In the Explorer window, locate the SAS data table called Itms_snmp_devicetype by drilling down into the SASHELP library.
Use the Copy and Paste functions to copy this table to the Adminnnnn library.
Now you can edit the table that you copied to Adminnnnn library. To do so, use VIEWTABLE to add and update rows. (To access VIEWTABLE, double-click the table in the Explorer window.)
The value in the Device column should be the name of the device, and deviceType is the type of device.
- The name of the device should match the device for which SNMP is gathering data. Make sure that the device names use the same case.
- DeviceType can be any character string that helps identify the device. For example, "Router," "Switch," or "Server" would be reasonable choices for device Type.
You can also update this table programmatically as long as the table has the device and deviceType columns.

After this table is populated, when you run the staging code, the deviceType column in all the SNMP staging tables will have the correct value from the lookup table. If a device is not in the deviceType table, then deviceType is set to "Unknown."

If your devices are named using a convention (for example, the device type is always the first part of the name, like router_2123), you can create the deviceType value programmatically. To do so, change the expression for the deviceType column in the staged table metadata. By default, the expression for the deviceType column is: put(device, ?? $deviceType.)

You can change this to a different expression. An example is:

upcase(scan(device,
                     1, "_"))

Or, you can use standard SAS DATA step code to create an expression. An example is:

if (device eq: "R") then 
		deviceType = "Router";
	else if (device eq: "W") then 
		deviceType = "Switch";
	else if (device eq: "S") then 
		deviceType = "Server";
	else
		deviceType = "Unknown";

Working with Management Information Base (MIB) Definitions for SNMP Staged Tables

The SAS macro %rmmssnmp enables the SNMP adapter to read Management Information Base (MIB) definitions and create the metadata for SAS IT Resource Management staged tables. This macro enables you to generate staged data from any MIBs that you have. The SAS macro is available in the SAS autocall macro library so that you can use it from any SAS session.

To use the macro, you must first create a job with an SNMP transformation in SAS IT Resource Management. To do so, perform the following steps:

In the SAS IT Resource Management client, navigate to the IT Data Marts tree.
Navigate to an existing IT data mart, or create a new one. (To create a new IT data mart, from the menu bar, select NewIT Resource ManagementIT Data Mart. The IT Data Mart wizard opens and prompts you to enter information to specify your new IT data mart.)
Create a folder in the IT data mart for your new job and tables.
In the new folder, create a new job. (To create a new job, select NewJob. Then enter the information to specify your new job.)
In the process flow diagram, drag the SNMP transformation from the SAS IT Resource Management folder of the Transformations tree and drop it onto the job.
Save and close the job.
Select the job in the tree and in the Basic Properties window, scroll to the Metadata ID property, and make a note of the value. This is the job ID that you need to run the macro in the next set of steps.

After the job is created, you can run the %rmmssnmp macro. To do so, perform the following steps:

Invoke an interactive SAS session.
Specify the parameters for the %rmmssnmp macro. For information, see Introduction to the Macros in SAS IT Resource Management.
Submit the macro call to read the MIB definitions. The macro can generate a report of the staged tables that would be created from this definition, or it can create the staged table metadata.