Add-in Code Generators and the Process Library

Process Library Technical Reference

The Process Library is a collection of registered routines that extract, transform, and load data into warehouse tables. As you use the Process Editor to define a step in the data flow for a particular table, you have the option of selecting a pre-defined routine from the Process Library, rather than defining your own process for that step.

The Process Library is made up of a registered set of Process Catalogs. A Process Catalog is a SAS catalog that has a specific set of entries and performs a specific process step. The MAIN entry in a Process Catalog is a reference to the routine that actually performs the step. This routine can be a SOURCE entry or an SCL entry (add-in code generator). For details about the difference between these two kinds of routines, see Add-In Code Generator Technical Reference. For a description of the Process Catalog format, see Process Catalog Format.

Invoking the Process Library

The Process Library window can be invoked using the Process Library pull-down Tools menu selection in the Process Editor. If a process step is currently selected, the purpose of calling the Process Library is to define the process information for that particular step. If no step is selected, the Process Library window is called to browse the Process Library. Example Process Library Window, Unexpanded shows a Process Library window.

Example Process Library Window, Unexpanded

[Example Process Library Window, Unexpanded]

Navigating the Process Library

The Process Library contains a configurable hierarchy of Process Groups and Process Catalogs. A Process Group is a set of related Process Catalogs. In Process Library Window With A Process Selected, a Process Group (Data Transformations) is expanded, and a Process Catalog (Straight Map Process) is selected. Note the items in the figure are only examples.

Process Library Window With A Process Selected

[Process Library Window With A Process Selected]

The tabbed pages on the right of the display show the metadata about the selected Process Group or Process Catalog. If a Process Group is selected, only the General tab is enabled. If a Process Catalog is selected, all tabs are enabled.

General Tab

The General tab displays descriptive information about the selected object. The tab contains fields for the name (up to 40 characters), a description (up to 200 characters), and, if the selected entry is a process catalog, the catalog name (up to 8 characters).

Note: A Process Catalog naming convention has been adopted to differentiate between Process Catalogs that you define and those that have been supplied by SAS. [cautionend]

The first letter of the catalog name is restricted to these alphabetic ranges:

User-defined catalogs:: start with a letter between A and R.
Catalogs supplied by SAS Institute:: start with a letter between S and Z.

Library Tab

The Library tab displays library information that is retrieved for the registry for the currently selected Process Catalog. It contains field entries for Libref, Engine, Path, and Options.

Help Tab

The Help tab displays any note information that is contained in the Process Catalog. You can print this information by positioning the cursor in the white space of the tab, clicking the right mouse button, and selecting Print.

Attributes Tab

The Attributes tab displays any extended attribute information that is contained in the Process Catalog. In this context, extended attributes are additional items of metadata that the Process Catalog uses to generate the appropriate code for the selected process step. The author of the Process Catalog decides whether to include extended attributes. To print this information, position the cursor in the white space of the tab, click the right mouse button, and select Print.

Selecting a Process Catalog

When the desired Process Catalog is located, it can be selected by clicking the name of the Process Catalog in the left viewer. If the user desires to select this Process Catalog as the process information for the selected step in the Process Editor, the user can then click OK.

When OK is selected, the selected Process Catalog information will be interrogated for correctness.

The library information will be validated by assigning the libref using the supplied library information. If an error occurs while assigning the libref, an appropriate error message will be displayed to the user.
The metadata for this Data Warehouse Environment will be searched for a Library definition that matches the library information supplied. To determine a match, the libref, engine, path and options information must match exactly, including case, those of an existing library definition.

If no matching definition is found, the user will be prompted to determine whether a new metadata definition should be created.

If the user decides to create a new definition, the user will be prompted to enter a name for this new library definition. The default name is the phrase "Process Library -" followed by the supplied libref.
If the Process Catalog contains an ATTRS.SLIST entry, the attributes will be verified. For any attributes for which an error is found, the user will be prompted to ignore this attribute and continue or to stop the selection.
If the user is replacing process information that is currently marked as being generated by SAS/Warehouse Administrator, the user will be prompted to confirm the replacement of this process information.

If any errors are discovered, the user selects No or Cancel to any confirmation dialogs, and the termination of the frame is halted. To end the frame, the user must then fix the errors, change selections to the confirmation dialogs, or select Cancel.

If a Process Group is selected at the time OK is selected, the process information for the step will not be modified. If Cancel is selected, the process information for the step will not be modified.

If no step in the Process Editor is selected at the invocation of the Process Library frame, no process information will be modified upon exit of the frame, whether or not OK or Cancel is selected.

Selection Results

After selecting a Process Catalog and returning to the Process Editor, the process information for the active step, if any, will have been modified to include a reference to the Process Catalog Main entry. To see this, select Properties on a process step, or Edit Load Step on a table. Source Code Tab After Update by a Process Catalog shows the Source Code tab for a process that has been updated by a Process Catalog.

Source Code Tab After Update by a Process Catalog

[Source Code Tab After Update by a Process Catalog]

Any attributes that were registered for the Process Catalog have been added to the metadata as Extended Attributes. These can be seen by selecting Extensions from the File pull-down menu option.

Process Library Registry

The display of groups and catalogs are controlled using the WAPRCS data set. If the WAPRCS data set exists in the _SASWA library, this data set will be used. If it is not found in this library, or this library is not assigned, the WAPRCS data set will be used from the SASHELP library. The WAPRCS data set contains one row for every Process group or Process Catalog. The data set contains the following columns:

WAPRCS Data Set Columns
Col Name	Col Type	Col Length	Col Label	Description
Parent	C	40	Parent Group	Name of the group that contains this group or catalog.
Name	C	40	Name of Catalog or Group	The name of the Process Catalog or Group.
Desc	C	200	Description of Catalog or Group	A brief description of the catalog or group.
Catalog	C	8	Name of SAS catalog	The name of the SAS catalog. This field should be left blank for a Process Group.
Active	N	8	1=Row active 0=Row inactive	Whether this row in the data set is active. Only those rows that are active will be used to populate the display.
Libref	C	8	Libref where this catalog resides	The SAS Libref that contains the catalog for this Process Catalog. This field should be left blank for a Process Group.
Engine	C	8	Libname engine of library	The SAS Libname engine used to access this library.
Path	C	200	Libname Path of Library	The host-specific path of the catalog. This corresponds to the path designation on the SAS Libname statement.
Options	C	200	Libname options of Library	The SAS Libname statement options.

Notes

All names that exist in the PARENT column must match a NAME column in another row. If the PARENT column has a blank value, this entry is the initial level of the hierarchy.
A row in the data set can depict either a Process Group or a Process Catalog.

If the NAME of this row exists in the data set as a PARENT, then it is a Process Group.

If the row is just for a Group, the CATALOG and LIBREF column values should be left blank.

If the row is a Catalog, at least the CATALOG and LIBREF column values must have values.
When entering Library information, the following rules apply:

For a Process Catalog, the LIBREF column must contain a non-blank value.

If any of the other Library information fields are non-blank (for example, PATH, ENGINE, OPTIONS), this information is used to automatically assign the library when this catalog is selected. If errors are discovered when the library is assigned, the user will be given an appropriate message. This type of definition corresponds to the Let the SAS System assign the library option in SAS/Warehouse Administrator. When a new metadata definition is created for this library, this option will be selected.
If only the LIBREF column has a non-blank value (for example, PATH, ENGINE, OPTIONS are blank), this represents a library that is to be assigned by the user. This libref must be assigned by the user at the time the Process Library User Interface is invoked. This type of definition corresponds to the The user will pre-assign the library option in SAS/Warehouse Administrator. When a new metadata definition is created for this library, this option will be selected.
When entering the same library information for several catalogs in the registry data set, it is important to match all of the library information (LIBREF, ENGINE, PATH, OPTIONS) exactly. This includes spacing and case. If the information is not entered in this way, the metadata might be polluted with several library definitions for a single set of library information.

Example Contents for WAPRCS Data Set

The following table lists the contents of an example WAPRCS data set. A Process Library with this data set would look similar to the library shown in the following table:

Example WAPRCS Contents
Parent	Name	Desc	Active	Catalog	Libref	Path	Engine Options
	Process Library	Main Process Library Group	1
Process Library	Data Loaders		1
Process Library	Data Transformations		1
Data Transformations	Straight Map		1	Datastep	Transfrm	C:\MyPrograms\loaders
Data Transformations	Range Check		1	Range	Transfrm	C:\MyPrograms\loaders
Data Transformations	Value De-duplication		1	Dedup	Dataddup	C:\MyPrograms\transforms	BASE Access=Readonly
Data Transformations	Transpose Table		1	Transpos	DataTran		Server=server.shr6

Process Catalog Format

A Process Catalog is a SAS catalog with a defined format. A Process Catalog can contain only a single process. A Process Catalog naming convention has been adopted to differentiate between Process Catalogs that you define and those that have been supplied by SAS.

User-defined catalogs:: start with a letter between A and R.
Catalogs supplied by SAS Institute:: start with a between letter S and Z.

A Process Catalog contains these standard entries:

Main.xxxx	(Required) is the main entry point of the catalog. This can either be a SOURCE or SCL type. If it is a SOURCE entry, it will be assumed that the source code should be included as is. If it is an SCL type, the SCL entry will be called during the code generation process. For details, see Add-In Code Generator Technical Reference. The entry name seen on the Source Code tab will be "catalog.MAIN.xxxx".
HELP.SOURCE	(Optional) is a source entry that contains Help information. This Help file will contain information explaining the functionality, any usage notes, and other pertinent information.
ATTRS.SLIST	(Optional) contains the information about the parameters for the process catalog. This entry is an SLIST with a sublist for each parameter (attribute). Each sublist has the following named items: Name - The name of the parameter (attribute). This name can be up to 40 characters. This item is required for each attribute. Desc - The description of the parameter (attribute). This can be up to 200 characters in length. This item is optional. Value - Any initial, default value for the parameter. This named item is optional and if not passed, no default value will be set. See ATTRS.SLIST Example.
BLDATTRS.SCL	(Optional) is recommended that the SCL code that creates the ATTRS.SLIST entry be included in the Process Catalog as BLDATTRS.SCL. See BLDATTRS.SCL Example.

ATTRS.SLIST Example

SLIST(
( NAME='FALLBACK' {T}
DESC='Teradata FALLBACK Option' {T}
VALUE='NO' {P}
)[5] {L}
( NAME='ERRLIMIT' {T}
DESC='Teradata ERRLIMIT Option' {T}
)[7] {L}
( NAME='SESSIONS' {T}
)[9] {L}
)[3]

In this example, there are three extended attributes for this Process Catalog.

For the first attribute, FALLBACK, a description has been given, as well as a default value, NO.
For the second attribute, ERRLIMT, a description has been given, but no default value has been specified.
For the third attribute, SESSIONS, no description or default value is specified.

Note: It is recommended that a description (item name DESC) and default value (item name VALUE) be included in all attributes.

This will give the user of the Process Library more information to determine what the attribute is used for and, if a default value is supplied, to lessen the amount of manual effort associated with using a process. [cautionend]

BLDATTRS.SCL Example

/************************************************************** 
* 
* Create the ATTRS.SLIST entry for this Process Catalog 
* 
***************************************************************/ 
init: 
/* 
* Determine the name of the Process Catalog and build the 
* ATTRS.SLIST entry into this catalog. 
*/ 
myname=screenname(); 
mylib=scan(myname,1,'.'); 
mycat=scan(myname,2,'.'); 
attrname=mylib||'.'||mycat||'.ATTRS.SLIST'; 
/* 
* Create the Master List 
*/ 
l_attrs=makelist(); 
/* 
* Create a sublist for each attribute 
*/ 
/* 
* Feedback attribute 
*/ 
l_attr=makelist(); 
l_attrs=insertl(l_attrs,l_attr,-1); 
l_attr=insertc(l_attr,'FALLBACK',-1,'NAME'); 
l_attr=insertc(l_attr,'Teradata FALLBACK Option',-1,'DESC'); 
l_attr=insertc(l_attr,'NO',-1,'VALUE'); 
/* 
* ERRLIMIT attribute 
*/ 
l_attr=makelist(); 
l_attrs=insertl(l_attrs,l_attr,-1); 
l_attr=insertc(l_attr,'ERRLIMIT',-1,'NAME'); 
l_attr=insertc(l_attr,'Teradata ERRLIMIT Option',-1,'DESC'); 
/* 
* SESSIONS attribute 
*/ 
l_attr=makelist(); 
l_attrs=insertl(l_attrs,l_attr,-1); 
l_attr=insertc(l_attr,'SESSIONS',-1,'NAME'); 
/* 
* Save the ATTRS.SLIST entry 
*/ 
list_rc=savelist('CATALOG',attrname,l_attrs,0,'Attribute List'); 
if list_rc ne 0 then do; 
msg='ERROR: Saving of '||attrname||' failed with rc='|| 
trim(left(putn(list_rc,'8.'))); 
put msg; 
end; /* if */ 
return;

Top of Page