Profile Data :: SAS(R) Data Loader 2.2 for Hadoop: User's Guide

Introduction

Use the Profile Data directive to generate profile reports for one or more tables. You can select a subset of the columns that you want to include in the profile report.

Table Name Length Requirement

Hive tables have a maximum table name length of 132 characters. Many of the SAS Data Loader directives can create tables with names that exceed the SAS table name length limit of 32 characters. The tables that you submit for profiling in the Profile Data directive must conform to the 32-character name length limit. Table names that exceed 32 characters generate error messages.

Configure Profile Jobs

Follow these steps to view and edit the properties for profile jobs:

Note: These are advanced settings that are not normally changed. In normal operation, the default values are sufficient.

On the SAS Data Loader page, click and select Configuration. The Configuration dialog box is displayed:

In the Configuration dialog box, click Profiles.
For Stop processing a column..., select the check box to stop processing a column if the number of unique values is greater than or equal to the number that you enter in the field.
For Maximum number of frequency..., select the check box and in the field enter the maximum number of frequency distribution values to save. If there are more frequency distribution values than this number, the less-frequent values are combined into an Other frequency distribution.
For Number of outlier values to save, select the check box and in the field enter the maximum number of outlier values to save.
Click OK to close the window and save your changes.

Create a Profile

To create a profile:

On the SAS Data Loader page, click the Profile Data directive. The Source Table page is displayed:
Click a data source to display its tables:
Select the table or tables for the profile report.

Tip
To view sample data from a table, select the table, and then click in the Source Table header to display the SAS Table Viewer.

Tip
If a profile already exists for a table, PROFILED appears beneath the table name. You can view the existing profile by selecting the table and clicking View Profile.

Tip
Profile results are also available in the Filter Transformation. If a table has been profiled, an ellipsis button appears next to the filter value selection. Click that button to view profile results while building your filters.

Click Next. The Columns page is displayed:
The Columns page displays the total number of columns that are to be processed in the profile report. If you selected more than one table for your report, the tables are listed by name. Click next to the tables to display the columns that are included in the profile report.
The column names in the Selected columns pane appear in the report. Select an individual column name and click or to move the column name between the Available columns pane and the Selected columns pane until the correct list of names appears in the Selected columns pane. Click or to move all column names at once.

When the column selection is complete, click Next. The Report page is displayed:
In the Report page, enter a name for the profile report in the Report name field. Click next to the Report location field to change the storage location of the profile report.

After specifying a name and location, click Next. The Result page is displayed:
Click Create Profile Report. After successfully creating the profile report, a screen similar to the following is displayed:

The following actions are available:

View Profile Report

enables you to view the Profile Report. See Saved Profile Reports for more information about the profile report.

Log

displays the SAS log that is generated during the creation of the profile.

Code

displays the SAS code that generates the profile.