Process Results Reporting

Reports 1 and 2 have multiple sections or panels. Each section can be generated, if you choose to do so. Here are the sections that are common to each report:

a report summary
a listing of key process inputs and outputs as defined in the SASReferences data set
a summary of validation metrics
a general process messaging panel

A sample driver program is provided to define the SAS Clinical Standards Toolkit environment and to call the primary task framework macro (%CSTUTIL_CREATEREPORT). This excerpt from the driver program header provides a brief overview:

cst_report.sas

Sample driver program to perform a primary Toolkit action, in this case, 
reporting of process results. This code performs any needed set-up and data
management tasks, followed by one or more calls to the %cstutil_createreport()
macro to generate report output.

Two options for invoking this routine are addressed in these scenarios:
  (1) This code is run as a natural continuation of a CST process, within
       the same SAS session, with all required files available.  The working
       assumption is that the SASReferences data set (referenced by the
       _cstSASRefs macro) exists and contains information on all input files
       required for reporting.
  (2) This code is being run in another SAS session with no CST setup
       established, but the user has a CST results data set and therefore can
       derive the location of the SASReferences file that can provide the full
       CST setup needed to run the reports.
       
Assumptions:
  To generate all panels for both types of reports, the following metadata
  is expected:
        - the SASReferences file must exist, and must be identified in the
          call to cstutil_processsetup if it is not work.sasreferences.
        - a Results data set.
        - a (validation-specific) Metrics data set.
        - the (validation-specific) run-time Control data set itemizing the
          validation checks requested.
        - access to the (validation-specific) check messages data set.

The reporting as implemented in the SAS Clinical Standards Toolkit attempts to address these two scenarios described in the driver program header above:

Some SAS Clinical Standards Toolkit task (such as validation against a reference standard) has been completed. The Results data set has been created. And, in the same SAS session (or batch job stream), you want to generate one or both reports. In this scenario, the reporting process uses the SASReferences data set defined by the global macro variable _cstSASRefs that was used by the previous process. The Results data set to be summarized in the report is the data set that was previously created and perhaps persisted to a location other than the SAS Work library. (Whether the data set was persisted was specified in the SASReferences data set.) Other files required by the report are identified in Metadata Sources for Reporting.

Tip
Best Practice Recommendation: Do not call the cleanup macro %CSTUTIL_CLEANUPCSTSESSION between primary tasks in a SAS Clinical Standards Toolkit SAS session (such as between validation and reporting). This keeps required files, macro variables, autocall paths, and so on, available for the reporting code.
The Results data set that was created in some prior SAS Clinical Standards Toolkit session is available. You want to generate one or both reports. The SAS Clinical Standards Toolkit processes add informational records to the Results data set, documenting the process itself. For example, a SAS Clinical Standards Toolkit CDISC SDTM validation process writes records to the Results data set that contains this sample message text:
```
Message
PROCESS STANDARD: CDISC-SDTM
PROCESS STANDARDVERSION: 3.1.3
PROCESS DRIVER: SDTM_VALIDATE
PROCESS DATE: 2012-10-01T09:57:14
PROCESS TYPE: VALIDATION
PROCESS SASREFERENCES: 
      &_cstSRoot./cdisc-sdtm-3.1.3-1.7/sascstdemodata/control/
      sasreferences.sas7bdat
```
From this information, a reporting process can attempt to find and open the referenced SASReferences data set to derive information for some or all of the report sections.

CAUTION:

There are obvious limits to how useful any SAS Clinical Standards Toolkit Results data set can be in rebuilding a session for reporting purposes.

For example, if the SASReferences data set was built in the Work library in a previous session, then it is not available and the report process fails. Similarly, if the SASReferences data set references library and file paths using a macro variable prefix (for example, &_cstGRoot or &studyRootPath), and those macro variables are not set or point to a different root path than the original process, then the report process might fail or yield unpredictable results. In the example above, the referenced SASReferences data set points to the sample library folder hierarchy that was used for a SAS Clinical Standards Toolkit 1.5 process. This folder hierarchy still exists in the SAS Clinical Standards Toolkit 1.7, so the results data set would more likely be found. This scenario or technique is most appropriate for sites that adopt a consistent means of building and populating SASReferences data sets.

Metadata Sources for Reporting
Data or Metadata Source	Scenario 1: Continuation of an Active SAS Session	Scenario 2: Using a Results Data Set from a Previous SAS Session
SASReferences	&_cstSASRefs used by the prior task that generated the Results data set.	The Results data set record containing the message PROCESS SASREFERENCES attempts to use the referenced file. &_cstSASRefs is set to this file.
Results	Precedence: The data set referenced in &_cstSASRefs with type=results and subtype is either results or validationresults. The data set referenced by &_cstResultsDS.	As provided in the cst_report.sas driver program _cstRptResultsDS macro variable.
Metrics	Precedence: The data set referenced in &_cstSASRefs with type=results and subtype is either metrics or validationmetrics. The data set referenced by &_cstMetricsDS.	The data set referenced in &_cstSASRefs with type=results and subtype is either metrics or validationmetrics.
Validation_Control	The data set referenced in &_cstSASRefs with type=control and subtype=validation.	The data set referenced in &_cstSASRefs with type=control and subtype=validation.
Messages	&_cstMessages used by the prior task.	&_cstMessages built by a call to %CSTUTIL_ALLOCATESASREFERENCES.

Note: In the SAS Clinical Standards Toolkit, you are able to define report output locations in the SASReferences data set. These locations can be defined with type=report in SASReferences. They can be further specified in the framework Standardlookup data set. For more information, see Framework.

This code is excerpted from the cst_report.sas driver program and performs the setup tasks that are specific to reporting:

* Initialize macro variables used for this task *; 
%let _cstRptControl=; 
%let _cstRptLib=; 
%let _cstRptMetricsDS=; 
%let _cstRptOutputFile=&studyOutputPath/results/cstreport.pdf; 
%let _cstRptResultsDS=; 
%let _cstSetupSrc=SASREFERENCES; 
%let _cstStandard=CDISC-SDTM; 
%let _cstStandardVersion=3.1.2; 

%cstutil_processsetup(_cstSASReferencesLocation=&studyrootpath/control); 
%cstutil_reportsetup(_cstRptType=Results);

In this piece of code:

The report output is specified in the _cstRptOutputFile variable and is in &studyOutputPath/results/cstreport.pdf. The studyOutputPath variable was previously defined to point to a folder with Write permissions.
The _cstSetupSrc=SASREFERENCES statement tells the process that a SASReferences data set is available and should be used to complete setup tasks.
The call to the %CSTUTIL_PROCESSSETUP macro provides the location of the SASReferences data set using the previously defined &studyRootPath variable.
The call to the %CSTUTIL_REPORTSETUP macro completes the setup steps that are required to generate report 1, itemizing results data set records by validation check.

An alternative setup to support Scenario 2, as described, would include these code excerpts:

%let _cstSetupSrc=RESULTS; 
%cstutil_processsetup(); 
%let _cstRptResultsDS=work.validation_results; 
%cstutil_reportsetup(_cstRptType=Results);

In this piece of code:

The _cstSetupSrc=RESULTS statement tells the process that a SAS Clinical Standards Toolkit process results data set should be used as the initial metadata source to complete the setup tasks.
The call to the %CSTUTIL_PROCESSSETUP macro without parameters, and with _cstSetupSrc=RESULTS, defers the remaining setup steps to the cstutil_reportsetup macro.
The call to the %CSTUTIL_REPORTSETUP macro completes the setup steps required to generate report 1, itemizing work.validation_results records.

As the final step, the reporting driver program makes one or more calls to the utility reporting macro. At a minimum (using default parameter values), a macro call to create report 2 might include this code:

%cstutil_createreport(_cstsasreferencesdset=&_cstSASRefs,_cstreportbydomain=Y, 
_cstreportoutput=&studyrootpath/results/cstchecktablereport.pdf);

Note: For more information about the %CSTUTIL_CREATEREPORT macro, see the SAS Clinical Standards Toolkit: Macro API Documentation.

A more complete example of the %CSTUTIL_CREATEREPORT reporting macro includes this macro call:

%cstutil_createreport(
   _cstsasreferencesdset=&_cstSASRefs,
   _cstresultsdset=&_cstRptResultsDS,
   _cstmetricsdset=&_cstRptMetricsDS,
   _cstreportbytable=N,
   _cstreporterrorsonly=Y,
   _cstreportobs=50,
   _cstreportoutput=%nrbquote(&_cstRptOutputFile),
   _cstsummaryReport=Y,
   _cstioReport=Y,
   _cstmetricsReport=Y,
   _cstgeneralResultsReport=Y,
   _cstcheckIdResultsReport=Y);

Interpretation of this request produces a (validation) results listing that contains all five report panels and includes only the first 50 errors that are reported for each validation check.

The following displays show report content. The displays apply to report 1 (by checkid) unless otherwise indicated.

Example of Report Summary

Example of Process Inputs and Outputs

Example process inputs and outputs from a report

Example of Process Metrics (Report 1)

Example of Process Metrics by Domain (Report 2)

Example process metrics by domain from a report

Example of General Process Reporting

Example of Validation Results by CheckID (Report 1)

Example validation results by CheckID for a report

Example of Validation Results by Domain (Report 2)

Example Validation Results by Domain for a Report