IT Service Vision 2.2.1 Collector Updates

Contents:

Updates to the Enhanced NTSMF Support

NTSMF Support Description

Support for Cabletron Spectrum 5.0

IT Service Vision 2.2 Updates

Updates to the Enhanced NTSMF Support

%cpcat;
cards4;
maintain table name=_all_
  report
  scope=newvars
  update;
;;;;
%cpcat(cat=work.temp.temp.source);
%cpddutl(entrynam=work.temp.temp.source);

%cpcat;
cards4;
    SET TABLE NAME=NTPRCS;
    UPDATE VARIABLE NAME=IDPRCS
      KEPT=YES
      LENGTH=8;
 /*--------------------------------------------------------------*
  | By, Class, ID, and Index variable lists for table: NTPRCS
  *--------------------------------------------------------------*/
  UPDATE TABLE NAME= NTPRCS
    DETAIL =( BYVARS= 'DOMAIN MACHINE PROCESS IDPRCS DATETIME HOUR SHIFT' );
;;;;
%cpcat(cat=work.temp.temp.source);
%cpddutl(entrynam=work.temp.temp.source);

NTSMF Support Description

• Prerequistites
• NTSMF Data Requirements
• Creating a PDB and adding tables
• Processing and Reducing data into the PDB

Prerequisites

• IT Service Vision Release 2.2 Server is installed with Maintenance Pack 1 applied or higher.
• Demand Technology's NTSMF must be installed and collecting data. It is recommended that you have installed at least version 2.1.9, although earlier releases are supported (see NTSMF Data Requirements below).

NTSMF Data Requirements

NTSMF data requirements :-

• Discovery records must be collected in NTSMF smf log files.
• If your version of SeNTry Administration allows you to select which Discovery record types are collected then you must keep at least the type 5, however keeping the record types 6 and 7 will not add any significant overhead in terms of space or performance.
• Write Discovery records for ALL  objects.
• Compressed and uncompressed records can be processed by IT Service Vision Release 2.2.
• IT Service Vision requires that the delimiter used in NTSMF  is a comma.
• Suppress writing _Total records to the data file.
• NTSMF filters should NOT be used.

Creating a PDB and adding tables

  There are three approaches you can use when starting out with NTSMF data.

• NTSMF QuickStart - this is the recommended approach. Start the IT Service Vision interface and follow the instruction given by the QuickStart Wizard.
  
• Interactively - Start up IT Service Vision Server and use the PDB Wizard to create a new PDB and populate it with the Supplied NTSMF tables the you need.
  .
• Batch - The following example code allocates a new PDB and adds 2 supplied NTSMF tables.

%cpstart(pdb=pdb-name,root=root-location,access=write,mode=batch,_rc=cpstrc);

%put 'CPSTART Return Code is ' &cpstrc;

%cpcat;
cards4;
add table name=ntcache;
add table name=ntlgdsk;
;;;;
%cpcat(cat=work.cpddutl.add.source);
%cpddutl(entrynam=work.cpddutl.add.source);

For MVS use the following %cpstart.

%cpstart(pdb=pdb-name,
root=root-location,
disp=new,
mode=batch
_rc=cpstrc);

Once the tables have been added, dictionary characteristics (age limits, variables kept status) can be modified either using the interactive interface or the %CPDDUTL macro.

Processing and Reducing Data into the PDB

The NTSMF data can be processed on any platform, MVS, UNIX or Windows NT Server, irrespective of which platform it originated. Once that data is moved to the appropriate platform the processing is identical.

Enhanced NTSMF support is different from the original support that was provided in IT Service Vision Release 2.1 in that it dynamically builds the necessary code to process NTSMF smf files. It uses the NTSMF Discovery records at the head of each file to provide the information on the layout of the NTSMF data records. Different versions of the same software can produce different format NTSMF records, counters are inserted, deleted and appended and the only way to ensure the data is processed correctly is to use the Discovery record. Worse case, each machine on which NTSMF is logging could have different NTSMF record layouts due to different versions of the same software.

If multiple NTSMF smf files are to be passed to IT Service Vision and they all have different layouts, then staging code is built for each separate file. This ensures data integrity, however it also increases the processing time of the data. Fortunately, this is not typically the case, normally organizations have some control over the software and software versions that are installed on the monitored machines and as a result it is possible to group the NTSMF smf files according to common Discovery records. If several NTSMF smf files have the same Discovery record layouts then these can be processed using the same staging code.

To improve processing performance by reducing the number of times the staging code is built IT Service Vision performs an initial scan of all the NTSMF smf files and groups them according to common Discovery record layouts. Once completed each group of files is processed as a single file and the staging code is built only once per group (See note below for information on large volumes of data)..

The following should be considered when processing NTSMF smf files :-

• Wildcards - On NT it is possible to use wildcards (*.smf) in the RAWDATA parameter, however this is NOT recommended.
  
  If %CPPROCES detects a wildcards is being used it treats the files that match the wildcard as a single file and the Discovery records of the first file encountered are used to construct the staging code for all the subsequent files. This is fine as long as all the NTSMF smf files have the same format discovery records. If the NTSMF smf files are different, then data could be processed incorrectly.
  
  
• Concatenation of NTSMF smf files is NOT recommended.
  
  Do not concatenate NTSMF smf files into one large file. As for wildcards, %CPPROCES uses the first set of discovery records encountered to construct the staging code for the complete concatenated file. The result could be that data is processed incorrectly.
  
  
• Maintain NTSMF smf files as separate files.
  
  
• Processing multiple NTSMF files - Place all NTSMF smf files in one directory and point to the directory in the RAWDATA parameter.
  
  On MVS place each NTSMF smf file in it's own PDS member in one PDS which is specified in the RAWDATA parameter (do not concatenate PDS).  Ensure that only NTSMF smf files are in this directory, other files and subdirectories will cause failures. By pointing to the directory %CPPROCES will automatically group the NTSMF smf files by matching Discovery records and will process them accordingly.

Transferring the Data

For the Enhanced NTSMF support that is provided in IT Service Vision Release 2.2 or higher it is recommended that each NTSMF smf file is maintained as a separate smf file, that is, we do not recommend that they are concatenated together.

Transfer the file to the appropriate platform so that it retains it's text format.

Unix and PC: Place all the NTSMF smf files in a single directory which will be pointed to by the %cpproces macro. No other files or sub-directories should be in this directory.

MVSPlace each smf file in its own PDS member with the following DCB attributes DSORG=PO,RECFM=VB,LRECL=32756,BLKSIZE=32760. By specifying the PDS name in the %cpproces macro, each member will be picked up and processed.

Input Filtering

If you decide to use Input Filtering, we recommend that you do the following before running your first %CPPROCES :-

1. Bring up your NTSMF PDB interactively.
   
2. Copy pgmlib.ntsmf.cpdupchk.source to admin.ntsmf.cpdupchk.source.
   To do this, submit the following code from your program editor.

   proc catalog cat=pgmlib.ntsmf;
   copy out=admin.ntsmf;
   select cpdupchk.source;
   quit;

3. Review and update if necessary the following parameters for the %CPDUPCHK macro invocation contained in admin.ntsmf.cpdupchk.source. To do this type note admin.ntsmf.cpdupchk.source on the command line (or from the command box) make the necessary updates and SAVE and END out of the notepad window.

• INT=interval   represents the maximum interval allowed between the timestamps on any two consecutive data records from the same system. If the interval between the timestamp values exceeds the value of this parameter a new time range is created. Default is 00:18 - 18 minutes.
• SYSTEMS-number of systems  represents an estimate of the maximum number of systems for which the data file will contain data. Default is 50.
• RANGES=number of ranges  represents the maximum number of interval ranges that can occur during this execution of %CxPROCES. A new range is created when the difference between the datetime stamps of two consecutive records exceeds the value of the INT= parameter.  This break id referred to as a gap in the data. Default is 10.
• KEEP=number of weeks  represents the maximum number of weeks for which you want to retain control data. Control data is aged out or removed when the last datetime value in a range exceeds the value of this parameter. Default is 52.

If you do not do the above, the first run of %cpproces with Input Filtering active will copy the default %CPDUPCHK invocation into the admin library automatically, and you will receive the following warning message recommending that you review the %CPDUPCHK parameter values.

WARNING: *** WARNING *** WARNING *** WARNING *** WARNING *** WARNING *** WARNING
WARNING: DO NOT OVERLOOK THIS IMPORTANT WARNING - IT WILL NOT APPEAR AGAIN.
WARNING: A sample invocation of the %CPDUPCHK macro has been copied to
your ADMIN library. You should review its contents before the
next execution. To do so, start IT Service Vision with this
PDB in update mode and type "NOTE ADMIN.NTSMF.CPDUPCHK.SOURCE"
on the SAS command line. The only parameter values you need to
review and probably change are the RANGES=, SYSTEMS=, and KEEP=
settings. Review the comments therein for guidance and the
documentation on input filtering for more details.
WARNING: *** WARNING *** WARNING *** WARNING *** WARNING *** WARNING *** WARNING

If you require more information on Input Filtering please refer to the How To/Macro section from the online help for IT Service Vision.

Processing and Reducing the Data

The following process example should be run a %CPSTART. For the purpose of this example I have included input filtering in this process run. The rawdata parameter on the %CPPROCES macro should point to a directory (on Unix and PC) which contains the NTSMF logs to be processed and a PDS (on MVS).

%put 'CPPROCES return code is ' &cpprc;

%put 'CPREDUCE return code is ' &cprrc;

In the SAS log you can expect to see the following :-

+-----------------------------------------------------------------------------------------+
| IT Service Vision input data duplication check report                            |
| =================================================== |
|                                                                                                                          |
| NOTE: All input records for new machine nightingale will be added. |
|                                                                                                                          |
+-----------------------------------------------------------------------------------------+

The above message will only appear when Input Filtering is active. The message shown will depend on whether the input data is considered duplicate.

==========================================================
The following objects were not kept to be processed but
existed in the input file.

Object Name = FTP Server
Object Name = MSExchangeWEB
Object Name = Memory
Object Name = Paging File
Object Name = PhysicalDisk
Object Name = Process
Object Name = Server
Object Name = Server Work Queues
Object Name = System
Object Name = WINS Server

A record is not processed for the following reasons :-

1 - The ITSV table for this object was not specified
in the PROCESS macro.
2 - The ITSV table for this object is marked KEPT=N
in the PDB.
3 - The object is a new object for which a table
definition needs to be built (see GENERATE SOURCE).
==========================================================

This report is always produced when processing NTSMF data. It reports the objects that were found in the rawdata that were not processed. If an object appears in this report for which you want to collect the data for, then you should add the appropriate table to the PDB. If you do not want to keep the data for an object, you can update your collection process to no longer keep the history data. If an object appears for which there is no supplied table, then one can be constructed using the GENERATE SOURCE function of the %CPDDUTL macro and an INTYPE= parameter of NTSMF.

Occasionally, you may see messages appear in the SAS log similar to the ones below. These messages are produced when new NTSMF counters appear in an NTSMF object that are not defined in the IT Service Vision data dictionary.

WARNING: Process discovery record contains unrecognized metric label: ID Logon .
WARNING: Process discovery record contains unrecognized metric label: ID User .

To stop these warning messages appearing you need to add these metrics to the table definition. Cut and paste the NTSMF discovery record for the object in question into a text file and then pass that to Generate Source using the INTYPE=NTSMF to generate a table and variable definitions. You can then use the definitions of the new counters as a starting point for adding them to your PDB.

Messages similar to the following may appear in the SAS log during a %CPPROCES and are caused by variables (in this case ACTVSSN and INCTSSN) being marked as KEPT=YES in the table being processed, however, data for that variable not existing in the raw NTSMF data. In this example, the reason that there is no data for these varaibles in the NTSMF data was becaused the NT machine that was recording this information had an older service pack applied that did not produce values for these counters. This is not a problem and to stop these warnings from appearing just mark the appropriate variables as KEPT=NO.

WARNING: Variable name Active Session in table NTSSTM is not in data set COLLECT
NOTE: This may cause missing values to be generated for ACTVSSN
WARNING: Variable name Inactive Session in table NTSSTM is not in data set COLLECT
NOTE: This may cause missing values to be generated for INCTSSN

Support for Cabletron Spectrum 5.0