Acknowledgements
SMP Background
Problems with SAS/C and SMP Maintenance
SMP Distributable SAS/C Library
Multi-FMID Load Modules
Special thanks to Mr. Ron Higgin of Boole & Babbage Inc. for his assistance in developing this paper.
System Modification Program (SMP) and its successor product, System Modification Program/Extended (SMP/E), were designed to maintain accurate functional and maintenance information for all elements that comprise an MVS software product.
Description of SMP
Products distributed in SMP format consist of SMP identifiable elements. Each element is recorded in an SMP dataset. For the original (non-program-product) version of SMP this dataset was called the Control Data Set or CDS. Today, for SMP/E, it is called the Consolidated Software Inventory or CSI.
Each product is presented to and installed by SMP in the form of one or more function packages. Each function package is assigned a unique seven character identifier by the company that distributes the product. This identifier is referred to as an function management identifier or FMID. The complete function package represented by the FMID is referred to as a FUNCTION SYSMOD (system modification).
At a minimum, each FUNCTION SYSMOD usually consists of one or more module (MOD) elements and a JCLIN set instructing SMP how to combine the MODs to form load modules (LMODs) as well as the library (actually a DDname) in which to place the LMODs. A FUNCTION SYSMOD may contain other kinds of elements but these are not discussed in this report because they are irrelevant to the SAS/C SMP support methodology.
It is relatively easy for SMP to maintain accurate records for non-code elements like macros, ISPF panels, and the like. On the other hand, object module elements represent a much more complex problem to SMP.
Information Maintained by SMP
SMP maintains the following information for each MOD element under its control:
Limitations of SMP
It is important to note here that a MOD element is owned by one, and only one, FMID, that being the FMID assigned to the FUNCTION SYSMOD that introduced, or defined, it to SMP. Conversely, LMODs are usually defined to SMP with JCLIN contained within a FUNCTION SYSMOD and are not assigned an owner. That is, LMOD entries in the CSI are not associated with any particular FMID. This is a subtle point, the importance of which will be discussed later.
The current SMP design mandates that SMP be informed of the exact contents of each and every LMOD under its control. In other words, all MOD elements that are contained within an LMOD element must be individually identified to SMP. The method used to do this is a link edit job, including key JCL DD statements. The job is read by the SMP JCLIN function, which parses out the information needed to build a new or update an existing LMOD entry.
It is important to note that SMP presumes that the linkage editor NCAL parameter has been specified, regardless of whether or not the PARM field on the linkage editor EXEC statement processed by the JCLIN function actually contains the NCAL specification. In other words, SMP does not support the linkage editor autocall functionality.
For years this SMP design limitation did not cause any problems. However, with the use of high-level languages and MVS callable services, the basic SMP design philosophy of having to be informed of the exact contents of an LMOD breaks down.
The use of a high-level language, and of SAS/C in particular, leads to a number of problems if the resulting software is to be installed and maintained by SMP. In brief, the problems are:
static and extern variables is not compatible with the SMP process for applying object module maintenance. extname option.) This is also not compatible with the SMP process for applying object module
maintenance. These problems and their solutions are discussed individually in this programmer's report.
Linkage Editor Autocall Traditional high-level languages are comprised of three basic components:
To simplify use of the language by programmers not familiar with a particular operating system's implementation, language implementors for IBM systems have made extensive use of external references and the linkage editor autocall function. In this scheme the compiler, based on programmer coding and compiler implementation, generates strong external references for a variety of resident library routines. It is worth noting that the resident library routines usually contain strong external references to other resident library routines.
The linkage editor accepts as input object modules produced by the compiler. These objects contain machine instructions, static data and constants representing the programmer's high level code. They also contain information like external references that are used by the linkage editor and, for the most part, only at link time.
The linkage editor collects the external reference and external symbol definition information contained in all object modules read from its primary input (SYSLIN) data set. It then matches the reference and definition information to determine which, if any, external symbol references remain unresolved. These unresolved symbols usually indicate one or more required object or load modules have been omitted from the linkage editor's primary or secondary input data sets. (An INCLUDE control statement can be used to specify a secondary input data set.)
The linkage editor, when invoked with autocall enabled, attempts to locate missing external symbol definitions by searching any object or load module libraries defined by the autocall file (SYSLIB). It does this by repeatedly searching the autocall libraries for a member whose name exactly matches a missing external symbol name. If such a member is found it is read and processed in a similar manner to the primary input. That is, external symbol reference and definition information from the autocalled module is combined with that already obtained from the modules contained in the primary input and previously autocalled modules. This process continues until all strong external symbol references have been resolved; that is, until a matching external symbol definition is located for each strong external symbol reference contained in the modules explicitly or implicitly included in the link edit
The autocall function makes high level languages simpler both to use and to implement. The programmer need only include in the linkage editor input, either directly or indirectly with INCLUDE control statements, the application modules, in object or load format, that from the programmer's perspective constitute a complete and functional application load module. From the compiler developer's point of view a relatively complex library structure can be developed, as the resident library modules required for successful execution of an application load module are automatically included into that load module when it is link edited. Furthermore, the compiler, with a suitably designed resident library such as that provided by SAS/C, can force a relatively large number of resident library modules to be automatically included by generating a small number of strong external references.
The Linkage Editor Autocall Problem
Clearly the autocall process is very useful, if not absolutely required, to both high level language programmers and compiler or library developers. Unfortunately SMP does not support this process. In fact, from the above description of the autocall process, it is apparent that this procedure is incompatible with SMP. SMP needs to have information about each module contained within a target library load module and yet the linkage editor autocall process automatically includes modules, in load or object format, without regard to the library in which they reside, their service level, and so on. In short, if SMP were to invoke the linkage editor without the NCAL option, thus implying autocall, then it would be unable to determine the exact composition of the target load module. This would violate current SMP design requirements; therefore, the autocall function of the linkage editor is not supported.
The Solution
SAS/C provides a vendor enablement function to solve the autocall problem. This solution is implemented in the form of a
SMP distributable version of the SAS/C resident and all-resident libraries and the COOL SMPJCLIN option. See SMP Distributable SAS/C Library,
later in this report.
SMP Object Maintenance Application
When a PTF is applied with SMP, and the PTF supplies a new version of an object module, the object module
must be replaced in each load module of which it is a component. SMP performs this maintenance by calling the
linkage editor to include replacement object modules followed by the previous version of the load module.
For instance, replacing modules B and C in the load module X1 results in a linkage editor run with SYSLIN input
similar to the following: INCLUDE SMPWRK3(B)
INCLUDE SMPWRK3(C)
INCLUDE SYSLMOD(XI)
NAME X1(R)
The new version of X1 contains new versions of B and C, while the other component modules (A, D, and E) are unchanged.
The SMP Object Maintenance Application Problem
This method of load module regeneration does not work properly when object modules created using the SAS/C rent or rentext option are included, if the C source code includes initializations of extern or static variables. (Initializations of static variables are only a problem when rent is used.) To see why this is so, we must review the implementation of reentrant static and extern initializations. When rent is used, the SAS/C compiler generates a CSECT named @EXTERN#, defining the static and external initializations required for the compilation. When the object modules are combined by COOL, COOL reads all the @EXTERN# CSECTs and produces a new, consolidated @EXTERN# CSECT containing the information from all the compiler-generated CSECTs. This consolidated CSECT is read by the library at run-time to control the initialization process.
Suppose, in the example above, that B was a SAS/C module that was compiled with rent and that initializes extern variables. The object module B will therefore include a compiler-generated @EXTERN# CSECT. The @EXTERN# CSECTs in C and X1 are seen by the linkage-editor as duplicates, and discarded. The net effect is that in the new module B's extern and static variables are initialized but not those of A, C, D or E. This is clearly unacceptable.
The Solution
To accommodate SMP update methods, SAS/C now provides an alternate mechanism for reentrant initialization of static and extern data. The compiler option extvec can be used to cause the compiler to generate, in place of @EXTERN#, a CSECT with a unique name of the following form for each compilation:
sname@.
Additionally, COOL provides the SMPXIVEC option. When SMPXIVEC is specified to COOL, COOL builds a vector named @EXTVEC# that references the sname@. CSECT's generated with the extvec compiler option. Note that the vendor must assign an element identification to @EXTVEC#, distinct from all other elements of the vendor's product.
The SMP maintenance process now behaves correctly. even if maintenance changes static or extern initializations. A replacement of the module B in the example replaces the B@. CSECT, but leaves @EXTVEC# unchanged. Since @EXTVEC# references B@., all of module B's initializations are still performed. (Note that when extvec is specified the compiler generates an sname@. CSECT even for a compilation that performs no initializations. This means it is possible to add initializations to such a module without replacing @EXTVEC#.
If a new object module is added to a load module, the @EXTVEC# CSECT must also be replaced in order to add the sname@. pointer for the new module.
Extended Names
The IBM mainframe object deck format and linkage editor implementation restrict external symbols to 8 characters, and to the use of alphabetic characters in upper-case only. Due to the prevalence of C programs written for other implementations that have no such limit, SAS/C has implemented an extended names facility. This facility provides optional compile- and linktime processing to replace extended names (names containing upper case letters or more than 8 characters) with generated names that meet linkage editor requirements.
The Extended Names Problem
When the SAS/C compiler extname option is used, the compiler and COOL perform additional processing that can interfere with the SMP maintenance process.
When the extname option is used and extended names (names containing upper case or more than eight characters) are used, the compiler assigns artificial names like a @@523217 in place of the extended names. These names are not guaranteed to differ from the names produced by other compilations. Additionally, changes to a source module may cause the generated names to change as well.
When code containing extended names is processed by COOL, COOL resolves any clashes and assigns a unique artificial name to each extended name. When there are no clashes, an extended name keeps the artificial name assigned by the compiler.
Thus, when one module compiled with extname is changed, the names of external symbols or references in other modules may also change, due to clashes or changes in the names generated by the changed module. This is unacceptable for SMP maintenance, as it means that any PTF would have to reship all object modules which were linked with a changed module.
The Extended Names Problem
To allow SMP maintenance to be possible on modules compiled with extname, stability of the artificial names is required. The COOL extended names exit facility allows the user to implement stable names.
When the ENEXIT COOL option is specified, COOL loads a user load module named CLKEXIT, and calls it each time an extended name must be assigned an artificial name. The exit can choose the name to be assigned. It is anticipated that the exit will maintain a database of extended names and their artificial equivalents. With such an exit in place, COOL can be forced to always assign the same extended symbol to the same extended name.
This will then guarantee that a single changed module can be replaced in the SMP maintenance process without threatening the integrity of the resulting load module.
The ENEXIT COOL option is described in more detail in the SAS/C Compiler and Library User's Guide.
The SAS/C library is now available in an SMP distributable form. It is important to note that this library is not provided in SMP format. Vendors are responsible for building one or two SMP format SAS/C library FUNCTION SYSMODs according to the rules described in this document.
The modules of the SAS/C resident and all-resident libraries have been assigned unique SMP element identifications by SAS Institute. The libraries on the SAS/C SMP distribution tape have information about the distribution libraries (DLIBs) and element names encoded into their members. When the COOL program is run using these libraries for autocall, and the SMPJCLIN option is specified. COOL generates, as an output file associated with the DDname SYSJCLIN, a list of link-age editor INCLUDE statements for library elements included by autocall. This output can be used to build an SMPJCLIN file that defines to SMP the structure of a software product including its use of SAS/C library elements.
Packaging SAS/C Libraries for SMP
Part of the process of preparing a SAS/C compiled software product for SMP distribution is the packaging of the
relevant SAS/C libraries in SMP format. The SAS/C SMP support distribution tape contains all the necessary files
to enable you to do this. Present on the tape are:
The distribution or load library names selected for the SAS/C resident library are:
| ASAROBM | base library |
| ASAROSM | standard library |
| ASAROMM | minimal library |
Additionally, the following names are used for the SAS/C allresident library support:
| ASAROAM | all-resident library |
| ASAROTM | all-resident modules shared with the transient library. |
| ASAROMM | base library |
These DLIBs, when packaged in SMP format, must be defined to SMP as fully copied DLIBs whose target library is SSAROMOD. (See Multi-FMID Load Modules, later in this report, for a discussion of this point.) Vendors must write a tool or tools to convert the SAS/C libraries to SMP format and to generate other SMP-required files. Output of the tool or tools must include:
++ FUNCTION (ASARvrl) FILES(2)
++ VER (Z038) .
++ JCLIN RELFILE(1)
++ MOD (BBCRAA00) RELFILE(2) DISTLIB(ASAROMM) LEPARM (RENT, REFR) .
.
.
.
++ FUNCTION (ASATvrl) FILES(2) /* if all-res support needed */.
++ VER (Z038) .
++ JCLIN RELFILE(1) .
++ MOD (BBCTAA01) RELFILE(2) DISTLIB(ASAROAM) LEPARM (RENT, REFR) .
.
.
.
A separate ++MOD statement is required for EACH SAS/C library module. The DISTLIB specification for each module can be determined from the SMP module cross-reference file.
If both library FUNCTIONs are required, they must be defined in the same MCS file.
//STEP0001 EXEC PGM=IEBCOPY //ASAROMM DD DSN=SYS1.ASAROMM
//ASAROSM DD DSN=SYS1.ASAROSM
//ASAROBM DD DSN=SYS1.ASAROBM
//SSAROMOD DD DSN=SYS1.SSAROMOD
//SYSIN DD *
COPY INDD=ASAROMM,OUTDD=SSAROMOD
COPY INDD=ASAROSM,OUTDD=SSAROMOD
COPY INDD=ASAROBM,OUTDD=SSAROMOD
/*
If the all-resident function is also required, a similar PDS should be created containing the following JCLIN statements.
//STEP0001 EXEC PGM=IEBCOPY
//ASAROAM DD DSN=SYS1.ASAROAM
//ASAROTM DD DSN=SYS1.ASAROTM
//SYSIN DD *
COPY INDD=ASAROAM,OUTDD=SSAROMOD
COPY INDD=ASAROTM,OUTDD=SSAROMOD
These outputs from the vendor utility are then combined to form a complete SMP installable package. The final SMP installable SAS/C library package will consist of a standard labeled tape constructed as shown in Table 1. Table 1 SMP Installable Tape Files
| File | Filename | File Contents |
| 1 | DSNAME - SMPMCS | pre-generated MCS statements as described above. |
| 2 | DSNAME - ASARvrl.F1 | an IEBCOPY unloaded PDS consisting of the single JCLIN member for the resident libraries as described above. |
| 3 | DSNAME - ASARvrl.F2 | an IEBCOPY unloaded PDS containing the SAS/C SMP distributable resident library modules using element names as member names. |
| 4 | DSNAME - ASATvrl.F1 | an IEBCOPY unloaded PDS consisting of the single JCLIN member for the all-resident libraries as described above. |
| 5 | DSNAME - ASATvrl.F2 | an IEBCOPY unloaded PDS containing the SAS/C SMP distributable all-resident library modules using element names as member names. |
Note that files 4 and 5 should be omitted if the all-resident libraries are not used.
The SAS/C Cross-Reference File Format
The SAS/C SMP cross-reference file is a text file containing one line for each SMP element identification defined for the current release. Each line contains the following information, separated by blanks:
library-name element-name DLIB-name member-name reentrency
The file can be read with the C f scanf function. The library-name entry defines the
object file in which the element is to be found. It will be either SPEOBJ, BASEOBJ, STDOBJ, ARESOBJ, or TRNOBJ.
Similarly, the DLIB-name entry will be either ASAROMM, ASAROBM, ASAROSM, ASAROAM, or ASAROTM.
The reentrency entry will be the letter R if the element is reentrant, or N if it is not. For example, the following
line defines the element BBCRAB10 which SMP will store in the DLIB ASAROMM.
SPEOBJ BBCRAB10 ASAROMM L$UMAIN R
The object code for this module resides in the SAS/C SPEOBJ data set, with member name LSUMAIN, and is reentrant.
Note that some library members are stored in more than one data set. For instance, LSUOSIO is stored in both SPEOBJ and STDOBJ. In such cases where the code is identical, the same element identification is used and one of the two object libraries is selected arbitrarily as the home of the element. For instance, LSUOSIO is considered to reside in SPEOBJ. Also note that some members of ARESOBJ do not have entries in the cross-reference. Such members comprise several elements, for which object code generally resides in the TRNOBJ data set. In this case, the INCLUDE statements generated by the COOL SMPJCLIN option include individual elements from TRNOBJ rather than using the consolidated ARESOBJ members.
As already discussed, SMP maintains precise function (FMID) ownership and maintenance level information for each MOD element under its control. Furthermore, as has already been pointed out, SMP does not maintain owner FMID information for LMODs. This section discusses the importance of this fact as it applies to high level languages and callable system services in general and SAS/C in particular.
Companies that develop products implemented in a high level language like C cause some of the MOD elements that comprise the product to contain external references to resident library modules provided by the high level language. These references must be resolved when the LMODs containing these MOD elements are link edited.
Forgetting for a moment the autocall process and presuming that a means (JCLIN) is provided to identify the required resident library or MODs to SMP, the resulting executable load module will contain MOD elements owned by at least two FMIDs: the product FMID and the language-provider resident library FMID. Such LMODs are referred to as multi-FMID load modules because they are comprised of MOD elements owned by more than one FMID.
The original SMP design did not anticipate multi-FMID load modules and thus, while not explicitly prohibited, they introduce enormous complexity into the SMP process. This complexity can, and often does, cause SMP to incompletely or incorrectly build LMODs which in turn leads to product failures. The majority of these problems are detected when a product is first installed; however, this is not always the case. Such problems can also be introduced during the (SMP) application of service (PTFs).
The specific conditions under which SMP fails to operate as expected where multi-FMID load modules are involved are:
A unique but common occurence of this happens when two or more related FUNCTION SYSMODs (elements of which are combined to form one or more multi-FMID load modules) are installed in separate SMP/E APPLY operations. This is the normal case when a SAS/C dependent product is installed on a system that already has the SAS/C resident library FMID installed.
In this case the product supplied PTF that introduces the change cannot ship the required resident library MOD as part of the PTF because the MOD is owned by a different FMID. When SMP/E attempts to install the PTF, the link edit fails because SMP/E assumes that all required modules have been previously included in the target load module or have been shipped as elements of the same PTF. SMP/E therefore does not generate INCLUDE statements for the required resident library MOD. Note that this is true even if the PTF uses JCLIN to declare that the MOD is required, that is, included in the target LMOD.
Clearly both of these problems must be addressed by product distributors until such time, if ever, SMP solves the problem. Fortunately there are workarounds for both problems.
Fully Copied DLIBs
The solution to both of the above problems lies in a seldom used SMP/E entity known as the fully copied DLIB, hereafter referred to as an FCD.
SMP/E recognizes an FCD whenever it processes a JCLIN step that invokes the IEBCOPY utility and the COPY step does not contain any IEBCOPY SELECT or EXCLUDE control statements. This type of IEBCOPY operation tells SMP/E that there is a one-to-one correspondence between the distribution library (DLIB) modules (MODs) and the target library (SYSLIB) load modules (LMODs). This is very useful information to SMP/E because it says that including a load module from the target library in a link edit produces the same result as including the corresponding module of the same name from the related DLIB. Note that in some cases it produces a better result because, when PTFs have been applied but not accepted, the target library could be at a higher function or service level than the DLIB copy of the module since target libraries are updated at APPLY time whereas DLIBs are updated at ACCEPT time.
By introducing an FCD to contain the SAS/C resident library modules involved in cross-FMID link-edit operations, SMP/E is in effect provided with an alternative source from which to fetch the required cross FMID modules at link-edit time. Unfortunately FCDs introduce a new problem that must be addressed. To understand this new problem it is necessary to look at how SMP/E uses FCDs during the PTF application process.
Reentrant DLIB Modules
When SMP/E recognizes an FCD from JCLIN, it builds a special CSI entry called a DLIB. The DLIB entry records the fact that the DLIB is fully copied and specifies the related target library, SYSLIB. This information is used during subsequent PTF APPLY operations to locate the SYSLIB if and when it is needed for link edit input.
During the application of a FUNCTION or PTF SYSMOD SMP/E selects each module identified by ++MOD MCS statement and determines which load modules must be built or updated to include the new or updated module. SMP/E does this by examining the CST MOD entry for the module to determine the names of all load modules that must be link edited. For each LMOD in the list SMP/E examines the corresponding CSI LMOD entry to determine its associated target library. Armed with this information SMP/E issues a BLDL against the target library to determine if the load module exists in the library. If the LMOD exists, SMP/E will prepare to update the load module; otherwise, it prepares to build a new LMOD. The basic difference between an update and build operation is that in the build case SMP/E generates an INCLUDE statement for each MOD known to SMP to be within the load module, whereas in the update case INCLUDE statements are only generated for modules changed by the SYSMOD. In the update case, SMP also generates an INCLUDE SYSLMOD control statement to include the previous version of the LMOD, while in a build situation this control statement cannot be used, since no copy of the LMOD exists.
If SMP/E determines that a load module build operation is required then it scans the CSI to locate the MOD entries for all modules required to build the load module. Having selected a MOD entry, SMP/E examines the currently selected SYSMOD to determine whether or not the module has been included in the selected PTF. If not, SMP/E examines the MOD's DISTLIB subentry to determine its related DLIB. If the module is not associated with any active PTF, that is, a PTF in APPLY status, then SMP/E determines whether or not the associated DLIB has been or can be allocated. If so, SMP/E generates the following linkage editor control statement:
INCLUDE dlib(mod)
However, if the DLIB is not available, that is, has not and cannot be allocated, or if there is an active PTF associated with the module, then SMP/E attempts to locate a CSI DLIB entry for the DLIB indicated in the CSI MOD entry. If a DLIB entry is found then SMP/E generates the following linkage editor control statement for the target library (TLIB) indicated in the DLIB subentry:
INCLUDE tlib(mod)
If a CSI DLIB entry is not found, the PTF application is failed.
In the case where a module included in a SYSMOD being applied is defined to exist in both an FCD and one or more composite load modules, that is load modules consisting of more than one object module, SMP/E must do at least two link edits: one for the FCD load module, containing only the PTF provided module, and one for the target composite load module containing the PTF provided module.
MultiFMID Load Module Problem
SMP/E cannot determine the link edit attributes of the FCD resident load modules. This is because FCD load modules are defined with IEBCOPY JCLIN steps that do not provide load module attributes. In the absence of this information SMP/E link edits the FCD module to the related target library using default load module attributes. Since the default attribute list usually does not include RENT, REUS, or REFR this results in an FCD that contains non-reentrant load modules. This is a time bomb armed to explode the first time SMP/E selects the FCD module for inclusion in a related composite load module that must be marked reentrant in order to properly execute. Since the included FCD module is non-reentrant, the composite load module containing it will not be marked reentrant even if SMP specified the RENT parameter when it invoked the linkage editor.
The Solution
The solution is to provide SMP/E with the load module attribute information for all FCD resident SAS/C modules if and when they are replaced by a PTF SYSMOD. This is done with the LEPARM option of the ++MOD MCS statement. When provided, SMP/E will link edit the associated module to the FCD using the MCS specified attributes. To allow this information to be specified correctly for library modules, the SAS/C cross-reference file included in the SMP distribution indicates for each library module whether or not it is reentrant.
Product Installation and PTF Construction Rules
Adhering to the vendor product installation and PTF SYSMOD construction rules described in this section will solve, or at least bypass, the multi-FMID load module problem and at the same time avoid the non-reentrant FCD module problem.
The following rules apply to the installation of all vendor products containing one or more modules compiled with the SAS/C compiler.
The following rules apply to all SAS/C resident library module replacements introduced through PTF SYSMODs:
In addition to the above stated rules, a SAS/C library or vendor supplied FUNCTION or PTF SYSMOD that introduces a new module to an existing load module, where the new module is owned by a different FMID than the product module being updated in that load module, must provide the following:
Note that this last point applies to FUNCTION SYSMODs that replace prior versions or releases of the SAS/C library or vendor products.