What's New in SAS Enterprise Miner 5.3

Overview

SAS Enterprise Miner 5.3 marks a major upgrade with the following new features and enhancements:

improved usability and functionality
enhanced administration and configuration
enhanced user interface
new options for converting SAS Enterprise Miner projects
new nodes
enhanced nodes
experimental nodes
enhanced PMML capabilities
enhanced capabilities for creating extension tools
enhanced documentation

Improved Usability

SAS Enterprise Miner 5.3 features an improved user interface that incorporates common design principles established for SAS software. These enhancements include the following:

The diagram workspace for process flow diagrams now contains navigation tools.
Node and shortcut toolbars have been reorganized.
Library and data explorer interfaces have been expanded.
Interactive graphics contain controls for greater ease of use and cleaner, more technical renderings.
New multivariate graphs have been introduced.
Variables tables include a WHERE clause to easily find features in wide data sets.
Property sheets have been reorganized to support training, scoring, and reporting tasks. Subsets of properties have been grouped into tearoff sheets.
Each node supports user-defined entry notes for better version control and a more complete documentation trail of the process flow diagram analysis.
Secure Sockets Layers (SSLs) can be used to secure authentication.

Improved Functionality

New data preparation tools make it easier to manipulate data and to construct a data sample for mining. For data exploration, SAS Enterprise Miner 5.3 now supports the following:

Hierarchical market baskets have been introduced that make it easier to model actual product categories.
Multivariate graphical data exploration tools save your interactive selections.
The new Variable Clustering node can be used for dimension-reduction techniques.
There are more interactive user controls in features.
Feature creation has been enhanced with new Interactive Binning and interactive Rules Builder nodes, as well as new options in the Transformation node.
Three new nodes that provide core predictive modeling techniques are introduced: Gradient Boosting, Support Vector Machines, and Partial Least Squares.
The new Import Model node makes it easier to import models that were previously produced using SAS/STAT code.
In the new Cutoff node, you can examine posterior probability distributions by using different cutoff values that you specify.
The new Reporter node uses SAS ODS to produce reports for printing and editing that span the entire analysis.

Enhanced Administration and Configuration

The following changes and enhancements improve the installation, administration, and configuration of SAS Enterprise Miner 5.3:

Analytics Platform 1.4 — The SAS Analytics Platform, which is required by SAS Enterprise Miner 5.3, is included with a SAS Enterprise Miner 5.3 license and is installed automatically with SAS Enterprise Miner 5.3.
SSL Encryption — User logins and related network traffic between the SAS Enterprise Miner 5.3 client and the SAS Analytics Platform middle tier can now be encrypted using standard Secure Sockets Layers (SSLs). Enabling SSL encryption requires additional configuration through the SAS Analytics Platform.
SAS Enterprise Miner Web Viewer — The SAS Enterprise Miner Web View has been redesigned to work within the SAS Analytics Platform session instead of requiring an external Web server. You can access the Web View functionality through the Analytics Platform Configuration Web page by clicking the Enterprise Miner link. A SAS Metadata Server user name and password are required. The Web View page presents a streamlined view of the Mining Models that are registered in the SAS Metadata Server. The SAS Model Manager product provides full functionality, including project management, version control, code review, score testing, champion selection, score deployment, and model monitoring reports.
SAS Version — SAS Enterprise Miner 5.3 runs on SAS 9.1.3 SP4 and automatically includes the correct version of Base SAS.
Enterprise Miner 5.2 Migration — SAS Enterprise Miner 5.2 projects can be used in SAS Enterprise Miner 5.3 without any migration action.
Enterprise Miner 4.3 Migration — SAS Enterprise Miner 4.3 projects can be converted to SAS Enterprise Miner 5.3 projects. The project conversion preserves process flow diagram structures, many of the node properties, and many of the tool results such as the SAS log and output listings, scoring code, and results tables that are needed to produce gains charts. You can run converted SAS Enterprise Miner 4.3 diagrams in SAS Enterprise Miner 5.3 and compare the output. This satisfies the need to archive and retrieve SAS Enterprise Miner 4.3 results from within SAS Enterprise Miner 5.3.
EMMETA Library — The EMMETA library is a new library that is created at startup by SAS Enterprise Miner 5.3. The local location is named EMLMETA and the global location is named EMGMETA. You can assign the location in your project or server startup code. Use the EMMETA library to store tables that define functions used by SAS Enterprise Miner nodes. For example, you might use EMMETA to store custom variable transformations to be called by the Transform node, or variable metadata for new data or new variables that were generated by a SAS Code node.

The SAS Enterprise Miner 5.3 nodes toolbar and the SAS Enterprise Miner 5.3 shortcut toolbar have both been relocated and their contents have been reorganized for consistency. Additional information is available in the SAS Enterprise Miner Reference Help section of the SAS Enterprise Miner 5.3 User Interface.

Diagram Workspace

The SAS Enterprise Miner 5.3 diagram workspace provides improved navigation and greater ease of use when working with large process flow diagrams.

Properties Panel

The SAS Enterprise Miner 5.3 Properties Panel has been reorganized.

The Basic and Advanced views for the SAS Enterprise Miner node property displays have been removed. Groups of related advanced properties have been relocated to subgroups in the Properties Panel.
SAS Enterprise Miner 5.3 now groups node properties by Train, Score, and Report functionalities. Changing a Train property causes the Train, Score, and Report actions to run the next time you run the node. Changing a Score property causes the Score and Report actions to run the next time you run the node. Changing a Report property causes only the Report action to run the next time you run the node.
Dynamic property selections simplify tool configuration by disabling irrelevant properties based on the selected values of related properties. For example, until the Regression property for Selection Method is set to Stepwise, the Regression properties for Stepwise Selection Options remains dimmed and unavailable.
Sets of related properties have been combined into subgroups that are displayed in a separate pop-up panel, which greatly simplifies the user interface.

Library Wizard

A new SAS Enterprise Miner Library Wizard enables you to select directories in the scope of the SAS session and create simple SAS library definitions. The Library Wizard is especially useful if you are not accustomed to using SAS LIBNAME statements to allocate SAS librefs. For more complex library definitions, use the SAS Enterprise Miner project startup code or the SAS Metadata LIBNAME Engine that is accessed through SAS Management Console. The Library Wizard has a Browse Directories action that browses directories on the SAS server, and not the SAS client.

SAS Library Explorer

The revised SAS Library Explorer displays data sets and catalogs, including some catalog contents such as SOURCE entries. You can select a data set in the SAS Library Explorer and initiate SAS Enterprise Miner data source creation. The Refresh button in the SAS Library Explorer updates the list of libraries, data sets, and catalogs.

Target Profile Panel

SAS Enterprise Miner Target Profile panels have been revised in the Data Source Wizard, the Data Source object, and the Decisions node. You can now select inverse proportions for decision weights in the Target Profile panels. Inversely proportioned decision weights are useful when working with very unbalanced data.

Variables Editor

The SAS Enterprise Miner Variables Editor has been revised to include a configurable WHERE clause filter, which is helpful when navigating a long list of variables.

Graph Persistence

You can create new graphs in the Results window as before, but the graphs are saved and re-created when the Results window is closed and reopened. This includes the new Graph Explore node that is the migration path for the Insight and Distribution Explorer nodes from SAS Enterprise Miner 4.3.

Conversion of SAS Enterprise Miner Projects

New Conversion Macro

SAS Enterprise Miner 5.3 contains a SAS macro that enables you to convert SAS Enterprise Miner 4.3 projects to SAS Enterprise Miner 5.3 projects. The conversion macro functions only on the Windows version of SAS.

The conversion macro runs in the SAS language. It can be used to convert multiple SAS Enterprise Miner 4.3 projects to multiple SAS Enterprise Miner 5.3 projects, or to convert and consolidate multiple SAS Enterprise Miner 4.3 projects into a single SAS Enterprise Miner 5.3 project.

Converting diagrams from SAS Enterprise Miner 4.3 to SAS Enterprise Miner 5.3 requires some changes due to differences in the versions.

Basic Diagram Structure

The basic process flow diagram structure is retained during project conversion, including the following:

tool properties
logs and output listings
selected output tables such as LIFT
plots and tabular listings that were created in the SAS Enterprise Miner 4.3 Results windows.

Retaining the basic diagram and results structures enables you to evenly compare SAS Enterprise Miner 4.3 and SAS Enterprise Miner 5.3 results.

Subdiagrams

SAS Enterprise Miner 4.3 subdiagrams are expanded to the top level. Enter and Exit nodes appear and function in diagrams that were converted, but Enter and Exit nodes cannot be added to new SAS Enterprise Miner 5.3 process flow diagrams.

Replacement Node

When converting a SAS Enterprise Miner 4.3 project whose process flow diagrams contain the Replacement node, the Replacement node can be expanded to a combination of an Impute node and a Replacement node. The combination of Impute and Replacement nodes gives you better control over the processes.

Ensemble Node

When converting a SAS Enterprise Miner 4.3 project whose process flow diagrams contain stratified, bagging, or boosting models that use the Ensemble node, the Ensemble node is replaced by an End Groups node.

Insight and Distribution Explorer Nodes

When converting a SAS Enterprise Miner 4.3 project whose process flow diagrams contain Insight or Distribution Explorer nodes, the nodes are replaced in the SAS Enterprise Miner 5.3 process flow diagram with Graph Explore nodes.

Link Analysis Node

SAS Enterprise Miner 5.3 supports conversion of SAS Enterprise Miner 4.3 projects whose process flow diagrams contain the Link Analysis node. Link Analysis nodes from SAS Enterprise Miner 4.3 process flow diagrams appear and function in diagrams that were converted for use with SAS Enterprise Miner 5.3, but the Link Analysis node cannot be added to a new SAS Enterprise Miner 5.3 diagram. If you want to analyze transaction data, refer to the SAS Enterprise Miner 5.3 Association, Path Analysis, and Market Basket nodes. If you want to analyze training data, continue to use the Link Analysis node from SAS Enterprise Miner 4.3.

New Nodes in SAS Enterprise Miner 5.3

SAS Enterprise Miner 5.3 contains the following new nodes that add functionality to the environment:

Append Node

The Append node has been added to the SAS Enterprise Miner 5.3 Sample tab of the node toolbar. The Append node belongs to the Sample category of the SAS SEMMA (Sample, Explore, Modify, Model, Assess) data mining process. You use the Append node to append data sets that are exported by two or more paths in a single SAS Enterprise Miner process flow diagram.

The Append node can append data according to the data role, such as joining training data to training data, transaction data to transaction data, score data to score data, and so on. The Append node can also append data from other roles that was previously partitioned in train, test, and validate roles to one large training data set.

Graph Explore Node

The Graph Explore node belongs to the Explore category of the SAS SEMMA (Sample, Explore, Modify, Model, Assess) data mining process. The Graph Explore node is an advanced visualization tool that enables you to sample data and to interactively explore the data graphically in the Results window. The interactive Results window plots can be saved and used to uncover patterns and trends in the data as well as reveal extreme values in the database. Graph Explore plots are fully interactive. You can rotate a chart to different angles and move it anywhere on the screen to obtain different perspectives on the data.

Variable Clustering Node

The Variable Clustering node belongs to the Explore category of the SAS SEMMA (Sample, Explore, Modify, Model, Assess) data mining process. The Variable Clustering node is a useful tool for data reduction, such as choosing the best variables or cluster components for analysis. Variable clustering can be used to decrease variable redundancy and help to reveal the underlying structure of the input variables in a data set. Large numbers of variables can complicate the task of determining the relationships that might exist between the independent variables and the target variable in a model. Models that are built with correlated variables can destabilize parameter estimates, confound variable interpretation, and increase the computing time that is required to run the model. Variable clustering can reduce the number of variables that are required to build reliable predictive or segmentation models.

Interactive Binning Node

The Interactive Binning node belongs to the Modify category of the SAS SEMMA (Sample, Explore, Modify, Model, Assess) data mining process. The Interactive Binning node is an interactive grouping tool that you use to model nonlinear functions of multiple modes of continuous distributions. The interactive tool computes initial bins by quintiles, then you can interactively split and combine the initial bins.

Rules Builder Node

The Rules Builder node belongs to the Modify category of the SAS SEMMA (Sample, Explore, Modify, Model, Assess) data mining process. You use the Rules Builder node to create ad hoc sets of rules for your data that result in user-definable outcomes.

Gradient Boosting Node

The Gradient Boosting node belongs to the Model category of the SAS SEMMA (Sample, Explore, Modify, Model, Assess) data mining process. The Gradient Boosting node uses tree boosting to create a series of decision trees that together form a single predictive model. A tree in the series is fit to the residual of the prediction from the earlier trees in the series. The residual is defined in terms of the derivative of a loss function. Boosting is available for binary, nominal, and interval targets. The Gradient Boosting node is not included with SAS Enterprise Miner for the Desktop.

Model Import Node

The Model Import node belongs to the Model category of the SAS SEMMA (Sample, Explore, Modify, Model, Assess) data mining process. You use the Model Import node to import models to the SAS Enterprise Miner environment that were not created by SAS Enterprise Miner. For example, models that were created using SAS PROC LOGISTIC can now be run, assessed, and modified in SAS Enterprise Miner 5.3.

Partial Least Squares Node

The Partial Least Squares node belongs to the Model category of the SAS SEMMA (Sample, Explore, Modify, Model, Assess) data mining process. You use the Partial Least Squares node to model continuous and binary targets based on SAS/STAT PROC PLS. The Partial Least Squares node produces DATA step score code and standard predictive model assessment results.

Cutoff Node

The Cutoff node belongs to the Assess category of the SAS SEMMA (Sample, Explore, Modify, Model, Assess) data mining process. You use the Cutoff node as a probability cutoff tool that creates a frequency table of probability distributions and graphs to assist you in determining the best cutoff point or points for decision-making models with binary target variables. Probability distributions are aggregated to the 1% level and are not smoothed between bins. Cumulative and marginal distributions are plotted. You specify the cutoff value by using a report property.

Start Groups Node

The Start Groups node belongs to the Utility group. You use the Start Groups node to initiate a group processing segment in the process flow diagram. The Start Groups node performs the following types of group processing:

Stratified group processing that repeats processes for values of a class variable, such as GENDER=M and GENDER=F.
Bagging, or bootstrap aggregation, using repeated resampling.
Boosting, or boosted bootstrap aggregation, using repeated resampling with residual-based weights.
Index processing, which repeats processes a fixed number of times. Index processing is typically used with a Sampling node or with your code for a sample selection.

End Groups Node

The End Groups node belongs to the Utility group. You use the End Groups node to terminate a group processing segment in the process flow diagram. If the group processing function is stratified, bagging, or boosting, the End Groups node will function as a model node and present the final aggregated model. Nodes that follow the End Groups node continue data mining processes normally, after the group processing stage is completed.

Reporter Node

The Reporter node belongs to the Utility group. The Reporter node uses SAS ODS capability to create a single document for the given analysis in *.pdf or *.rtf format. The document includes important SAS Enterprise Miner results such as variable selection, model diagnostic tables, and model results plots. The document can be viewed and saved directly, and will be included in SAS Enterprise Miner report package files.

Enhanced Nodes in SAS Enterprise Miner 5.3

Several nodes in SAS Enterprise Miner 5.3 contain significant enhancements in functionality or have been reorganized into new SAS Enterprise Miner tool groups.

Sample Tools

The following changes have been made to the Sample tools in SAS Enterprise Miner 5.3:

Filter Node — The Filter node has been moved from the Modify tool group to the Sample tool group because the Filter node enables you to select rows of data by defining some user criteria.
Input Data Node — The Input Data node adds a new dialog box for selecting a new table. You can use the new dialog box to change the input data table, but retain the metadata for variable roles and levels. The Input Data node no longer requires a predefined data source in the Project Tree folder. You can use an imported diagram, without creating a data source, by simply selecting a data table.
Merge Node — The Merge node has been moved from the Utility tool group to the Sample tool group because the Merge node augments and subsets data for model building.
Sample Node — The Sample node now permits you to disable sample summaries of class and interval variables in order to speed processing when working with very large data sets. The summaries are typically used to compare the sampled data with the input data, but in many cases large data sets from a stable population do not need comparison.
Data Partition Node — The Data Partition node now provides you with a choice between partitioning data sets into data views or into data tables to conserve disk space. Like the Sample node, the Data Partition node also enables you to disable sample summaries of class and interval variables to speed processing when working with very large data sets.

Explore Tools

The following changes have been made to the Explore tools in SAS Enterprise Miner 5.3:

Cluster Node — The Cluster node now provides you with the option to assign user-defined labels to clusters.
SOM/Kohonen Node — The SOM/Kohonen node has been enhanced so that you have more control over row and column dimensions.
Variable Selection Node — The Variable Selection node has been enhanced so that you can map levels to output variables. The node also features a new interactive selection of final terms, and provides information about how class levels are collapsed in the Results window. A new option enables you to exclude classes of variables.

Modify Tools

The following changes have been made to the Modify tools in SAS Enterprise Miner 5.3:

Principal Components Node — The Principal Components node enables you to specify a suffix for the Principal Components output variables.
Replacement Node — The Replacement node now provides interactive interval selection for truncating and replacing distributions.
Transform Node — The Transform node features a new code editor for entering user-defined terms in transformations. User-entered code must be pure DATA step code for transformations. The user-defined transformation code cannot contain ARRAY, DATA, MERGE, OUTPUT, POINT, RETAIN, RUN, SET, or UPDATE statements. The interactive code editor is useful for automating routine transformations that are applied to multiple analyses. Transform definitions are stored in the EMMETA.TRANSFORM table.

The Transform node supports metadata-based transformations to automate routine transformations that are applied to multiple analyses.

Model Tools

The following changes have been made to the Model node tools in SAS Enterprise Miner 5.3:

AutoNeural Node — The AutoNeural node now provides a final training step to refine the selected model. You can select a target distribution (or error) function that permits modeling distributions such as Poisson, exponential, and sinusoidal.
Decision Tree Node — The Decision Tree node now features cross-validation options. The interactive tree desktop application gives you the ability to add, copy, and paste tree descendents, which is useful when using the tree to define customer segmentation schemes.
DMine Regression Node — The DMine Regression node now produces Group, AOV16, and Group Interaction tables. The DMine Regression node also creates PMML, including embedded transformation, binning, and grouping functions.
Neural Network Node — The Neural Network node now provides the weight decay parameter in the node properties panel, and the network parameters have been reorganized.
Regression Node — The Regression node has been enhanced to enable you to assign the variable role to Rejected for successor nodes.

Assess Tools

The following enhancements have been made to the existing Assess node tools in SAS Enterprise Miner 5.3:

Model Comparison Node — The Model Comparison node features enhancements such as transposed fit statistics for easier model comparisons and a greater choice of model selection statistics, including depth of lift. The Model Comparison node also enables you to create your own labels for models, as well as the ability to manually select a champion model for export.

Utility Tools

The following enhancements have been made to the existing Utility node tools in SAS Enterprise Miner 5.3:

SAS Code Node — A new interactive code editor replaces the individual editors for Train and Score code. The new editor supports Train, Score, and Report code views. You can edit and submit code interactively while viewing the log and output listings. You can also run the diagram path up to and including the SAS Code node and view the Results window without closing the programming interface. These changes greatly enhance the ability to write SAS code in the SAS Enterprise Miner environment.

Credit Scoring Tools

The following enhancements have been made to the add-on Credit Scoring node tools of SAS Enterprise Miner 5.3:

Interactive Grouping Node — The Interactive Grouping node is able to handle mixed-type data. Special missing values in numeric columns are mapped to unique bins of the output variable instead of a separate variable. This addresses many types of credit bureau data that contains both numeric values and special codes.

A new interactive window displays "fine grain" bins that give you more control over split points for continuous data. An initial high-resolution scan of the data is made to create the "fine grain" bins before a second pass is made to create the "coarse grain" candidate bins. You can interactively use the fine grain split points when constructing the final bins.

Weight of Evidence (WOE) values can now be manually adjusted, and you can interactively override variable Input/Reject roles when performing interactive training.
Scorecard Node — The Scorecard node has been enhanced so that individual terms can be ordered for model selection algorithms such as the stepwise selection algorithm. Link function selections of LOGIT, CLOGLOG, and PROBIT are available. Model selection options for STAY, STOP, and Term Ordering have been added. The Scorecard node also features Adverse Characteristic Scores for use with individual credit analysis.

Experimental Nodes in SAS Enterprise Miner 5.3

The following experimental nodes are introduced in SAS Enterprise Miner 5.3:

Market Basket Node

The Market Basket node belongs to the Explore category of the SAS SEMMA (Sample, Explore, Modify, Model, Assess) data mining process. You use the experimental Market Basket node to perform association rule mining over transaction data in conjunction with item taxonomy. This node is useful in retail marketing scenarios that involve tens of thousands of distinct items, where the items are grouped into subcategories, categories, departments, and so on, called item taxonomy. The Market Basket node uses the taxonomy data and generates rules at multiple levels in the taxonomy. The experimental Market Basket node is not included with SAS Enterprise Miner for the Desktop.

Support Vector Machine Node

The Support Vector Machine node belongs to the Model category of the SAS SEMMA (Sample, Explore, Modify, Model, Assess) data mining process. You use the experimental Support Vector Machine node for classification and regression modeling. The Support Vector Machine node uses hyperplanes to separate points that are mapped on a higher dimensional space. The data points that make up the hyperplane are called support vectors. The Support Vector Machine node is not included with SAS Enterprise Miner for the Desktop.

Enhanced PMML Capabilities

The following PMML enhancements are introduced in SAS Enterprise Miner 5.3:

PMML 3.1 — SAS Enterprise Miner 5.3 PMML is updated to full compatibility with DMG Version 3.1.
DMine Regression Node — The DMine Regression node now produces PMML, including optional binning and grouping transformations that are part of the model.

Enhanced Capabilities for Creating Extension Tools

In SAS Enterprise Miner 5.3, the capabilities available for SAS programmers to write custom SAS Enterprise Miner extension tools have been significantly enhanced.

SAS Enterprise Miner 5.2 and SAS Enterprise Miner 5.3 Extension Tools — SAS Enterprise Miner 5.2 extension tools should function in the SAS Enterprise Miner 5.3 environment. Both Basic and Advanced level properties are mapped to the Train property group.
New XML Header — SAS Enterprise Miner 5.3 extension tools can use a new XML header:

     <Component type="AF"
     serverclass="EM6" prefix="MYMODEL"
     name="MyModel" group="MODEL" icon="MyModel.gif"
     displayname="MyModel"
     description="This tool implements My Model"
     resource=""
   />

Using the serverclass="EM6" declaration enables new features in your SAS code.
EM_ACTION Macro — The EM_ACTION macro variable is a significant enhancement when used with the serverclass="EM6" declaration. The EM_ACTION macro variable can be used to separate your Train, Score, and Report code.
Table Display and Edit — SAS programmers can use simple table display and edit controls from their property sheets. This greatly improves interactive capabilities. Programmers can define which columns are displayed to the user and which columns can be edited through the parameters that are available in the tool properties XML. You can also define SAS code that runs before and after the table is edited.
%EM_REPORT Macro — New plot types and options have been added to the EM_REPORT macro. Plots and data views that were created using the EM_REPORT macro are added to documents that are created by the Reporter node.

Documentation Enhancements

The Getting Started with SAS Enterprise Miner 5.3 book is now available from SAS OnlineDoc 9.1.3 for the Web. This guide introduces you to the core functionality of SAS Enterprise Miner and shows you how to perform basic data mining tasks. You will learn how to use the graphical user interface (GUI) tools to create and manage process flow diagrams and projects, and to export mining results for reporting and integration with other SAS software. The data mining tasks you will learn include sampling, exploring, modifying, modeling, and assessing data in order to create and refine predictive models. Getting Started with SAS Enterprise Miner 5.3 provides step-by-step examples that create a complete process flow diagram, including graphic results. This title is also available online and in hard-copy format.

Nodes Toolbar and Shortcut Toolbar