Process Results Reporting

Reports 1 and 2 have multiple sections or panels. Each section can be optionally generated. Several sections are common to each report, including a report summary, a listing of key process inputs and outputs as defined in the SASReferences data set, a summary of validation metrics, and a general process messaging panel.
A sample driver program is provided to define the SAS Clinical Standards Toolkit environment and to call the primary task framework macro (%cstutil_createreport). This excerpt from the driver program header provides a brief overview:
cst_report.sas

Sample driver program to perform a primary Toolkit action, in this case, 
reporting of process results. This code performs any needed set-up and data
management tasks, followed by one or more calls to the %cstutil_createreport()
macro to generate report output.

Two options for invoking this routine are addressed in these scenarios:
  (1) This code is run as a natural continuation of a CST process, within
       the same SAS session, with all required files available.  The working
       assumption is that the SASReferences data set (referenced by the
       _cstSASRefs macro) exists and contains information on all input files
       required for reporting.
  (2) This code is being run in another SAS session with no CST setup
       established, but the user has a CST results data set and therefore can
       derive the location of the SASReferences file that can provide the full
       CST setup needed to run the reports.
       
Assumptions:
  To generate all panels for both types of reports, the following metadata
  is expected:
        - the SASReferences file must exist, and must be identified in the
          call to cstutil_processsetup if it is not work.sasreferences.
        - a Results data set.
        - a (validation-specific) Metrics data set.
        - the (validation-specific) run-time Control data set itemizing the
          validation checks requested.
        - access to the (validation-specific) check messages data set.
The reporting as implemented in the SAS Clinical Standards Toolkit attempts to address these two scenarios described in the driver module header above:
  1. Some SAS Clinical Standards Toolkit task (such as validation against a reference standard) has been completed. The Results data set has been created. And, in the same SAS session (or batch job stream), you want to generate one or both reports. In this scenario, the reporting process uses the SASReferences data set defined by the global macro variable _cstSASRefs that was used by the previous process. The Results data set to be summarized in the report is the data set that was previously created and perhaps persisted to a location other than the SAS Work library. (Whether the data set was persisted was specified in the SASReferences data set.) Other files required by the report are identified in Metadata Sources for Reporting.
    Tip
    Best Practice Recommendation: The cleanup macro, %cstutil_cleanupcstsession, should not be called between primary tasks in a SAS Clinical Standards Toolkit SAS session (such as between validation and reporting). This keeps required files, macro variables, autocall paths, and so on, available for the reporting code.
  2. The Results data set that was created in some prior SAS Clinical Standards Toolkit session is available. You want to generate one or both reports. The SAS Clinical Standards Toolkit processes add informational records to the Results data set, documenting the process itself. For example, a SAS Clinical Standards Toolkit CDISC SDTM validation process writes records to the Results data set that contains this sample message text:
    Message
    PROCESS STANDARD: CDISC-SDTM
    PROCESS STANDARDVERSION: 3.1.1
    PROCESS DRIVER: SDTM_VALIDATE
    PROCESS DATE: 2010-01-25T11:56:17
    PROCESS TYPE: VALIDATION
    PROCESS SASREFERENCES: 
          !sasroot/../SASClinicalStandardsToolkitSDTM311/
          9.1.3/sample/cdisc-sdtm-
          3.1.1/SASDemo/control/sasreferences.sas7bdat
    
    From this information, a reporting process can attempt to find and open the referenced SASReferences data set to derive information for some or all of the report sections.
    CAUTION:
    There are obvious limits to how useful any SAS Clinical Standards Toolkit Results data set can be in rebuilding a session for reporting purposes.
    For example, if the SASReferences data set was built in the Work library in a previous session, then it will not be available and the report process fails. Similarly, if the SASReferences data set references library and file paths using a macro variable prefix (for example, &_cstGRoot or &studyRootPath), and those macro variables are not set or point to a different root path than the original process, then the report process might fail or yield unpredictable results. In the example above, the referenced SASReferences data set points to a !sasroot folder hierarchy that was used for a SAS Clinical Standards Toolkit 1.2 process. This folder hierarchy no longer exists in the SAS Clinical Standards Toolkit 1.5, so the results data set would not be found. This scenario or technique is most appropriate for sites that adopt a consistent means of building and populating SASReferences data sets.
Metadata Sources for Reporting
Data or Metadata Source
Scenario 1: Continuation of an Active SAS Session
Scenario 2: Using a Results Data Set from a Previous SAS Session
SASReferences
&_cstSASRefs used by the prior task that generated the Results data set.
The Results data set record containing the message PROCESS SASREFERENCES attempts to use the referenced file. &_cstSASRefs is set to this file.
Results
Precedence:
  1. The data set referenced in &_cstSASRefs with type=results and subtype is either results or validationresults.
  2. The data set referenced by &_cstResultsDS.
As provided in the cst_report.sas driver program _cstRptResultsDS macro variable.
Metrics
Precedence:
  1. The data set referenced in &_cstSASRefs with type=results and subtype is either metrics or validationmetrics.
  2. The data set referenced by &_cstMetricsDS.
The data set referenced in &_cstSASRefs with type=results and subtype is either metrics or validationmetrics.
Validation_Control
The data set referenced in &_cstSASRefs with type=control and subtype=validation.
The data set referenced in &_cstSASRefs with type=control and subtype=validation.
Messages
&_cstMessages used by the prior task.
&_cstMessages built by a call to %cstutil_allocatesasreferences.
Note: Beginning in the SAS Clinical Standards Toolkit 1.3, you are able to define report output locations in the SASReferences data set. These locations can be defined with type=report in SASReferences. They can be further specified in the framework Standardlookup data set. For more information, see Framework.
This code is excerpted from the cst_report.sas driver module and performs the setup tasks that are specific to reporting:
* Initialize macro variables used for this task *; 
%let _cstRptControl=; 
%let _cstRptLib=; 
%let _cstRptMetricsDS=; 
%let _cstRptOutputFile=&studyOutputPath/results/cstreport.pdf; 
%let _cstRptResultsDS=; 
%let _cstSetupSrc=SASREFERENCES; 
%let _cstStandard=CDISC-SDTM; 
%let _cstStandardVersion=3.1.2; 

%cstutil_processsetup(_cstSASReferencesLocation=&studyrootpath/control); 
%cstutil_reportsetup(_cstRptType=Results);
In this piece of code:
  • The report output is specified in the _cstRptOutputFile variable and is in &studyOutputPath/results/cstreport.pdf. The studyOutputPath variable was previously defined to point to a folder with Write permissions.
  • The _cstSetupSrc=SASREFERENCES statement tells the process that a SASReferences data set is available and should be used to complete setup tasks.
  • The call to the %cstutil_processsetup macro provides the location of the SASReferences data set using the previously defined &studyRootPath variable.
  • The call to the %cstutil_reportsetup macro completes the setup steps that are required to generate report 1, itemizing results data set records by validation check.
An alternative setup to support Scenario 2, as described, would include these code excerpts:
%let _cstSetupSrc=RESULTS; 
%cstutil_processsetup(); 
%let _cstRptResultsDS=work.validation_results; 
%cstutil_reportsetup(_cstRptType=Results);
In this piece of code:
  • The _cstSetupSrc=RESULTS statement tells the process that a SAS Clinical Standards Toolkit process results data set should be used as the initial metadata source to complete the setup tasks.
  • The call to the %cstutil_processsetup macro without parameters, and with _cstSetupSrc=RESULTS, defers the remaining setup steps to the %cstutil_reportsetup macro.
  • The call to the %cstutil_reportsetup macro completes the setup steps required to generate report 1, itemizing work.validation_results records.
As the final step, the reporting driver program makes one or more calls to the utility reporting macro. At a minimum (using default parameter values), a simple macro call to create report 2 might include this code:
%cstutil_createreport(_cstsasreferencesdset=&_cstSASRefs,_cstreportbydomain=Y, 
_cstreportoutput=&studyrootpath/results/cstchecktablereport.pdf);
This table describes all supported parameters in the sample %cstutil_createreport macro.
Supported Parameters for the %cstutil_createreport Macro
Parameter
Description
_cstsasreferencesdset
The libref.dataset of SASReferences data set used for a specific process. This parameter is optional. If it is specified, then _cstresultsdset and _cstmetricsdset parameters are ignored. Either _cstsasreferencesdset or _cstresultsdset must be provided.
_cstresultsdset
The libref.dataset of the SAS Clinical Standards Toolkit process Results data set. This parameter is optional. Either _cstsasreferencesdset or _cstresultsdset must be provided. This parameter is ignored if _cstsasreferencesdset is specified.
_cstmetricsdset
The libref.dataset of the SAS Clinical Standards Toolkit process Metrics data set. This parameter is optional. This parameter is ignored if _cstsasreferencesdset is specified.
_cstreporterrorsonly
If N (default), then this parameter reports all records in the Results data set, including information and non-error results. If Y, then this parameter reports only records in error (where the Results data set field results.resultflag=1).
_cstreportobs
If null (default), then this parameter reports all records in error (where results.resultflag=1) in the Results data set. Otherwise, set this parameter to any integer value > 0, signifying the number of records to print per checkid (where results.checkid is non-null). If _cstreportobs > 0 excludes any records, then a footnote is printed, noting that not all records were printed.
_cstreportbytable
If N (default), then this parameter does not report results by table (that is, run report 1). If Y, then this parameter reports results by table (that is, run report 2).
_csttablechecksdset
Report 2 parameter. A data set that provides a list of tables for each check. Using this parameter assumes that this data set has been built before running this report. For more information, see Supplemental Validation Check Metadata: CDISC SDTM Domains by Check. This parameter is optional. If this parameter is not used, then the data set is created.
_csttablecheckscode
Report 2 parameter. The code module (macro) to build _csttablechecksdset if it does not exist, or is not passed as a parameter. This parameter is required only if _cstreportbytable=Y and _csttablechecksdset is not provided.
_cstkeeptablechecklist
Report 2 parameter. The value is Y or N (default). If running report 2, then keep the derived list of tables (_csttablechecklist) to reuse in subsequent report requests. Building this file takes a while.
_csttablesubset
Report 2 parameter. This parameter is optional. It produces a report based on a specific table, indicated by libref.data set. If the value is blank or the keyword _ALL_ is specified, then all tables are included in the report. This parameter is ignored if _cstreportbytable=N.
_cstreportoutput
The path and filename where report output is to be written. File types HTML, RTF, and PDF are supported. This parameter is required.
_cstsummaryReport
The value is Y (default) or N. If set to Y, then generate the report summary panel.
_cstioReport
The value is Y (default) or N. If set to Y, then generate the process inputs and outputs panel.
_cstmetricsReport
The value is Y (default) or N. If set to Y, then generate the process metrics panel. This parameter should be set to N for any non-validation reports and cases where metrics are not generated.
_cstgeneralResultsReport
The value is Y (default) or N. If set to Y, then generate the general process reporting panel.
_cstcheckIdResultsReport
The value is Y (default) or N. If set to Y, then generate the process results panel.
A more complete example of the %cstutil_createreport reporting macro includes this macro call:
%cstutil_createreport(
   _cstsasreferencesdset=&_cstSASRefs,
   _cstresultsdset=&_cstRptResultsDS,
   _cstmetricsdset=&_cstRptMetricsDS,
   _cstreportbytable=N,
   _cstreporterrorsonly=Y,
   _cstreportobs=50,
   _cstreportoutput=%nrbquote(&_cstRptOutputFile),
   _cstsummaryReport=Y,
   _cstioReport=Y,
   _cstmetricsReport=Y,
   _cstgeneralResultsReport=Y,
   _cstcheckIdResultsReport=Y);
Interpretation of this request (based on the parameter descriptions in Supported Parameters for the %cstutil_createreport Macro) produces a (validation) results listing that contains all five report panels and includes only the first 50 errors that are reported for each validation check.
These displays show report content. The displays apply to report 1 (by checkid) unless otherwise indicated.
Example of Report Summary
Example report summary
Example of Process Inputs and Outputs
Example process inputs and outputs from a report
Example of Process Metrics (Report 1)
Example process metrics from a report.
Example of Process Metrics by Domain (Report 2)
Example process metrics by domain from a report
Example of General Process Reporting
Example general process reporting
Example of Validation Results by CheckID (Report 1)
Example validation results by CheckID for a report
Example of Validation Results by Domain (Report 2)
Example Validation Results by Domain for a Report