Working with SAS Data Remediation

Overview

Working with Issues

Portlets

Overview

Data Remediation Summary Portlet

Data Remediation Portlet

SAS Data Remediation Tab

Quick Search

Advanced Search

Managing Issues

Deep Linking to Issues in Data Remediation

Reports for SAS Visual Analytics

Overview

Components

Sub-Reports

Database Connections to the Data Remediation Database

Overview

The Data Remediation tab enables you to take action necessary to resolve data issues. An issue is a problem caused by a conflict with user-defined business rules that has been found in data or a process. An item is a record that is associated with an issue. A package is a collection of items and issues. The package can originate from any system your enterprise uses to store and process data.

Your system administrator defines applications in SAS Data Remediation that send data remediation issues to the data remediation environment. Applications can be used to correct data issues. Subject areas categorize the data in such a way that groups of users are given access only to the necessary domain. For example, in SAS MDM, the subject areas are the different entity types that are defined in SAS MDM.

Working with Issues

SAS Data Remediation is designed to manage a practical number of issues assigned to it by an application. A practical number of issues means that they can be reasonably managed by the data stewards assigned to the task. For example, it would be unreasonable for SAS Data Remediation to track 50,000 issues a week because, without a large number of data stewards, it would not be possible to review and correct all of these issues. Although the SAS Data Remediation database might be able to handle very large numbers of issues, those issues that can be displayed at a given time in the web application and allow it to remain responsive to user control is much smaller.

SAS Data Remediation has a setting that enables you to control the maximum number of issues that are retrieved at any given time in the SAS Data Remediation application views. For more information about this setting, see Specifying SAS Data Remediation Preferences.

If you set the maximum number of issues to be viewed to a value that is less than the total number of issues in an item or package, you need to use a combination of sorting and filtering in the appropriate view to see all data in the remediation database that might be relevant. For example, a default view for items might show only the first 500 items. You can use a filter to limit that list to only the items with issues that you own and that have been open for longer than one week. After refreshing the view, you see updated data with your filter applied. The maximum number limit is applied after the filter is applied. Users can work through the entire set of remediation issues, while ensuring that the web application continues to work optimally, by applying selective filters to data and refreshing the view periodically to retrieve the latest issues.

Portlets

Overview

On the SAS Data Management Console, the Data Remediation portlet displays an overview of issues and a sampling of data and enables you to open individual issues. The Data Remediation Summary portlet summarizes data, but does not allow you to take action on any of the individual items.

SAS Data Management Console

Note: In the following descriptions, Actions menu items are accessed by clicking Actions Icon

. Some functions available from the Actions menu are also available from icons next to the Actions menu. Clicking the icon performs the function.

You can click Data Remediation or Data Remediation Summary in the portlets, or Data Remediation under APPS to open SAS Data Remediation on a new tab, which displays the table of issues:

SAS Data Remediation Tab

Data Remediation Summary Portlet

The top right area of the portlet contains an Actions menu with the following options:

Only show my issues

displays only the issues assigned to you.

Show all issues

displays all issues.

Refresh

refreshes the list of items in the portlet list.

Note: The portlet does not refresh automatically. You must refresh it.

Close the Portlet

closes the portlet on the SAS Data Management Console.

Data Remediation Portlet

The top right area of the portlet contains an Actions menu with the following options:

Display

enables you to check the number of items to display in the portlet. You can choose from 5, 10, 20, or 40 items.

Refresh

refreshes the list of items in the portlet list.

Note: The portlet does not refresh automatically. You must refresh it.

Close the Portlet

closes the portlet on the SAS Data Management Console.

When you click an item in the portlet list, the item opens on a new tab.

SAS Data Remediation Tab

Quick Search

The SAS Data Remediation tab displays a complete list of issues. You can perform either a quick search or an advanced search to narrow this list to the issues that you want to view.

To perform a quick search:

Enter the search term into the search field in the toolbar
Do one of the following:
- Click to begin the search.
- Click to clear the search data.

Note: To view the complete issues list again, click Clear Search Icon

to clear the search characters, and then click Begin Search Icon

to restore the list.

Advanced Search

To perform an advanced search:

Click to display search criteria:

Advanced Search Options
You can do any of the following:
- Click to expand all the search categories.
- Click to collapse all the search categories.
- Click to restore the search criteria to their default settings.
- Click again to close search options.
After specifying the search criteria, click to initiate the search.

Note:

To view the complete issues list again, click to restore the search criteria to their default settings, and then click to restore the list.

Managing Issues

The toolbar contains a drop-down list that enables you to group the entries in the table in a number of ways. For example, you might choose to group the entries by Issue Type, Item, or Package, among others. The default is ungrouped, which displays the complete list of issues.

Note: To group issues by exactly the same item or package, use the group by options for Item ID and Package ID. If you choose to group by Item or Package, issues might appear in the grouping that share the same label for the item or package, but might actually refer to different underlying data.

The toolbar also contains an Actions menu with the following options:

Open

opens the issue selected in the issues table on a new tab. Selecting multiple issues opens multiple new tabs. The actions available on the new tab are dependent on the application in which the issue is found.

Delete

deletes the selected package when the table is grouped by Package ID. Deleting a package is the only way to delete items and their associated issues.

Details Pane

displays or hides details of selected rows in the issues table in a separate pane beneath the table. Changes made in the Details pane are saved immediately. The Details pane does not have a save button.

Select Columns

enables you to select which columns appear in the table view.

Refresh

refreshes information about the tab.

About SAS Data Remediation

displays application version information.

If you have chosen to group items such that folders are displayed in the table entries, the following additional options are enabled:

Expand Selected

expands the folder selected in the table.

Expand All

expands all the folders in the table.

Collapse Selected

collapses the folder selected in the table.

Collapse All

collapses all the folders in the table.

The Details pane, if open, displays details about the selected item:

Details Pane

The Details pane includes a toolbar at the bottom of the pane. When an individual issue is opened on a new tab, the toolbar is displayed at the top of the pane.

The toolbar contains the following options:

Claim

assigns the issue to you as Assignee and sets the issue to a status of In-Progress. After you have claimed the issue, this icon changes to Release, which enables you to release the issue.

Release

sets the issue to a status of Open, so others can claim the issue, but leaves the current Assignee as is.

closes the issue when it is resolved. An issue with a closed status cannot be changed. Information about the closed status can be entered in the Note field.

Reject

rejects the issue when it is invalid. Information about the invalid issue can be entered in the Note field.

Put in Workflow

creates a new workflow instance for the selected issue. Available only if you have associated one or more task templates with the issue type of the selected issue.

Note: Put in Workflow opens a dialog box where you can associate a SAS Workflow Studio workflow template with the issue.

If a task has already been created for the issue, the options available in the toolbar depend on the SAS Workflow Studio workflow template associated with the issue.

The Details pane includes two, and possibly three, tabs:

Issue Details

On the Issue Details tab, you can view or edit specific information about the issue. Changes made on the Issue Details tab are saved immediately. The Details pane does not have a save button.

The Issue Details tab displays the following fields:

Status

the current state of the issue, such as Open, In Progress, Closed, or Rejected. If a task was created for the issue, then the workflow supplies the status, depending on the current step in the workflow.

Open is the initial status of a data remediation issue. This status can also be selected if the assigned owner wants to reassign the issue to a new owner.
In Progress indicates that ownership has been accepted and the issue is being actively addressed. Users other than the assigned owner cannot work on the issue.

Note: When you select a status of In Progress, you automatically become the Assignee for the issue.
In Workflow indicates that a data remediation issue is being actively addressed in the workflow. Users other than the participants in the workflow cannot work on the issue.

Note: After the workflow is completed, the Assignee is automatically set to the initiator of the workflow and the status is changed to Open. If the initiator is the current user, the status of the issue is set to In Progress.
Closed indicates that a data remediation issue is resolved. An issue cannot be changed from a Closed status. Enter resolution information in the Note field before closing the issue.
Rejected indicates an invalid data remediation issue that requires no additional action. Enter rejection information in the Note field before closing the issue.

Importance

the priority of the issue represented by star icons. The scale ranges from very low (1 star) through low, medium, and high, up to critical (5 stars).

Assignee

is the user name of the person assigned to the issue. Click Assignee

to select from an available list of assignees. The list of users is derived from those who have been assigned at least one data remediation role in SAS Management Console.

Note: When you select a status of In Progress, you automatically become the Assignee for the issue.

Due Date

is the date by which an issue should be resolved. Click Due Date

to select a date from the calendar. This option is not connected to alerts. To set up alerts, you must create a task for the issue that includes defined notifications.

Note

is a field used to record notes about the issue. Click Note

to enter your comments in the Note field.

Issue

is the label for the type of issue found in the data.

Item

is the record that triggered the remediation process. Clicking the link next to the Item name opens a dialog box displaying details of the item in an item

issue hierarchy table:

Item Details

Package

is the package in which the issue is located. Clicking the link next to the Package name opens a dialog box displaying details of the package. The area above the package

item

issue hierarchy table displays any package level notes and is editable.

Package Details

Subject Area

is the subject area assigned to the issue.

Application

is the application where the issue is found.

On the Issue Data tab, you can view data about the issue. The Issue Data tab displays the following fields:

Issue Data

Link

displays an Open in Application hyperlink that opens a new browser tab in the specified URL location if a properly formatted URL was submitted with the remediation issue.

Field 1

is a user-defined field. It contains a data value from the external application that originally flagged the item as an issue. The label for this field can be changed programmatically.

Field 2

is a user-defined field. It contains a data value from the external application that originally flagged the item as an issue. The label for this field can be changed programmatically.

Field 3

is a user-defined field. It contains a data value from the external application that originally flagged the item as an issue. The label for this field can be changed programmatically.

For example, the three user-defined fields might appear as follows:

Issue Data

If a task has already been created for the issue, the Details pane includes a third tab, Task Data:

Task Data

The following data object values are always provided by the workflow engine in Task Data:

Process Invoker

the user who sent the task to SAS Task Manager.

Process Title

the name used in SAS Data Remediation for the selected task.

The display of other values on the Task Data are controlled by the design of your workflow template. These values are data objects defined in the workflow template itself. For details, see the SAS Task Manager: User’s Guide.

Deep Linking to Issues in Data Remediation

Deep linking refers to constructing a URL hyperlink that, once initiated, can take a user directly to an issue in SAS Data Remediation rather than requiring the user to search for it. For example, you can construct a URL link in an e-mail and send to a data steward. When the data steward clicks the URL link, it opens the SAS Data Remediation application and loads the issue encoded in the URL.

From within the appropriate environment, create a URL using the following syntax:

http://<your_host>/SASDataManagement/#issue=<issue_ID>&module=REMEDIATION

For example, you might create the following:

http://localhost:8080/SASDataManagement/#issue=4312&module=REMEDIATION

Note: Creating a remediation issue in SAS MDM submits a deep link along with the issue, which is accessible through the SAS Data Remediation web service API. For more information, see the SAS MDM: User’s Guide.

Reports for SAS Visual Analytics

Overview

A single report with sub-reports is available for use with SAS Visual Analytics. You must execute a scheduled load process in the SAS Visual Analytics environment to generate the data needed for the SAS Data Remediation charts and tables in the report. Data for the report can be pushed from the SAS Data Remediation database to SAS Visual Analytics using a SAS MDM tool or other means documented in SAS Visual Analytics: User's Guide. For details about pushing data to reports, see SAS MDM: User's Guide.

Note:

If you attempt to load more data or allow more users than SAS guidelines specify, you are likely to see performance degradation. See SAS documentation regarding SAS Visual Analytics server sizing and in-memory table size limitations.
If you plan to make customizations to this report to meet the requirements of your project, make a copy of this initial report and add your customizations to the new version. Doing so protects customizations from being lost if report template enhancements become available in future releases.

Components

Several components are required to make the report properly viewable to users:

RemediationReport

The RemediationReport is the SAS Visual Analytics report file. It is installed for you in your SAS metadata environment in this location:

..\Products\SAS
                                    Data Remediation\Reports

data table view

The RemediationReport requires a data table view from the SAS Data Remediation database to be available to load into memory of the SAS Visual Analytics environment. The metadata for this view can be found at this location:

..\Products\SAS
                                    Data Remediation\Data Sources\Data Remediation LASR\sas_rem_issues_vw

stored procedure

A SAS stored procedure is required to enable drill-down functionality from within the report. The stored procedure must be set up to run on the appropriate SAS Stored Process Server. The stored procedure can be found at this location: ..\Products\SAS Data Remediation\URL Link

Verify that the base server URL defined in this stored process is correct. In the following code, the http address must be set to the SAS mid-tierserver used to host SAS Data Remediation:

%let server_string = http://sasmdm.demo.sas.com;

The RemediationReport sub-report named issue details contains an interaction that invokes the stored procedure described above. Edit the host and port of the stored procedure defined in the interaction to point to the SAS mid-tier server user to host the SAS Stored Process server. For more information, see stored procedure.

With these pieces in place, you can use standard SAS scheduling capabilities to load new data remediation data routinely into the SAS Visual Analytics environment.

Sub-Reports

The RemediationReport contains the following sub-reports on different tabs:

issue summary

displays the number of records added to remediation (by assignee, by type, by subject area, by application, by status, by importance). Data can be filtered using range-slider and drop-down controls at the top of the report. Progressing through the charts takes you to a filtered Issue Details tab.

issue details

displays issue in remediation in tabular form. Double-clicking an issue opens the selected issue in the SAS Data Remediation application. This action is enabled by the URL Link stored procedure.

issue activity

displays issue status by date and issue activity by type. Data can be filtered by subject area and importance using the drop-down lists. The date section in the top panel can be used to progress to smaller date increments.

package metrics

displays issues by package. Work flow details are displayed if available. Data can be filtered by date range slider or package type.

response time snapshot

displays issue response metrics by assignee and by issue type.

Database Connections to the Data Remediation Database

The SAS Visual Analytics reports for SAS Data Remediation rely on a database connection to the Data Remediation mid-tier Postgres data store. A connection to this database is made using SAS/ACCESS to Postgres.

Last updated: March 24, 2017