DataFlux Data Management Studio 2.5: User Guide

Introduction to Business Rules

• How Rules and Tasks Are Used
• Overview of the Business Rule Manager
• Example: Using Rules to Monitor Finances
• Three Types of Rules
• Overview of DataFlux Expressions
• Overview of Standard Metrics

• Default Business Rules

How Rules and Tasks Are Used

In Data Management Studio, a business rule is a reusable object that specifies a DataFlux expression. The expression evaluates the values in one or more table fields. Events can be triggered based on the results that are returned by the rule. A business rule can be used in one or more profiles or data jobs.

A task specifies one or more rules and one or more events that can be triggered based on the results that are returned by a rule. For example, you can create a business rule that tracks the profit per employee against assets over time. Then, you link the business rule to a task. When you add the task to a monitoring job and run the job, you trigger input to the Monitor tab in the Information riser bar. For details about this scenario, see Example: Using Rules to Monitor Finances.

Some DataFlux Data Management Studio features, such as Using DataFlux Master Data Management Foundations, install a set of business rules and tasks that are used to monitor the data that is associated with that feature. If you examine the properties of these installed rules, you can get an idea of how rules, tasks, dimensions, etc. are defined and used.

Overview of the Business Rule Manager

You can use the Business Rule Manager to manage business rules, tasks, and related objects. You can also use this dialog to manage custom metrics (user-defined DataFlux expressions) that can be selected in one or more business rules or profiles.

If you select Tools > Business Rule Manager > repository_name from the main menu, the Business Rules Manager displays.

The tree on the left of the Business Rule Manager displays folders for tasks, rules, and other objects that are associated with rules in the current repository (Monitor Repository). The panel on the right displays information about the items that are selected in the tree. The rules that are displayed in the previous figure are some of the rules that are installed with the DataFlux Master Data Management Foundations feature.

The tree contains the following kinds of objects:

Tasks - A task specifies one or more rules and one or more events that can be triggered based on the results that are returned by a rule.

Rules - A rule is reusable object that specifies a DataFlux expression that filters table rows. For example, a rule might check for rows in which the value of Profits per Employee is less than Assets. Events can be triggered based on the results that are returned by the rule. A business rule can be used in one or more profiles or data jobs.

Rule Sets - A rule set is a group of related rules.

Fields - A field in a business rule is an alias for the kind of field that is monitored by the rule.

Field Sets - A field set is a group of related fields.

Custom Metrics - A custom metric is a reusable object that specifies a DataFlux expression that performs an aggregate calculation. For example, a custom metric might calculate the average value of a sales field in all records from a particular state. Use a custom metric when there is no standard metric for that operation. A custom metric can be used in one or more business rules, profiles, or data jobs.

Sources - A source is the name of a table that contains a field that is monitored by a rule. In the Business Rules Manager, sources are used for documentation purposes. Each source identifies a table where a rule is applied.

Dashboards - A dashboard is used to group a set of dimensions and their rules into a subject area. For example, you might create one dashboard for monitoring sales data, and another for monitoring product data. The dashboards that you create are in addition to the system-defined dashboards that appear on the Monitor tab in the Information riser bar. Custom dashboards can be selected in the Monitor Viewer, as described in Using the Monitor Viewer.

Dimensions - A dimension is used to group similar rules together. For example, you might have a dimension called Accuracy that groups a set of rules that check the accuracy of data. You might have another dimension called Completeness that groups a set of rules that check the completeness of data. Dimensions can be selected in the Monitor Viewer, as described in Using the Monitor Viewer.

Threshold Definitions - A threshold is used to classify the status of an exception score. For example, you could have three thresholds, Low, Medium, and High, that would add meaning to certain ranges of values.

Example: Using Rules to Monitor Finances

Suppose that you want to monitor various kinds of financial measures, and you would like to view them in a dashboard that is available in the Monitor Viewer in DataFlux Data Management Studio. You could begin by performing the following tasks:

• Create a dashboard called Fiscal. This custom dashboard would display a set of dimensions and their rules that are related to financial measurements. For more information, see Creating Dashboards, Dimensions, and Thresholds.
• Create a dimension called Profit. This dimension would contain a set of rules that measure profit. For more information, see Creating Dashboards, Dimensions, and Thresholds.
• Create a rule called ProfitEmp_LessThan_Assets. This rule would identify all records in a table where the profits per employee are less than the assets. For more information, see Creating a Rule that Compares Two Fields.
• Create a task called Fiscal_Profit_Log_to_Repos. This task would log exceptions to a DataFlux repository, where they could serve as input to the Monitor pane in the Information riser bar. This task would be associated with the ProfitEmp_LessThan_Assets rule, and this rule would be linked to the Profit dimension. For more information, see Creating a Rule that Compares Two Fields.
• Create a data monitoring job called Monitor Profit Per Employee. This job would apply the Fiscal_Profit_Log_to_Repos task to a data source. The exceptions to the rule would be logged to a DataFlux repository. For more information, see Creating a Data Monitoring Job.
• View the output of the job in the Monitor Viewer. For more information, see Using the Monitor Viewer.

Three Types of Rules

In the Business Rule Manager, if you right-click the Rules folder and select New Rule, the Rule properties dialog displays.

This Rule properties dialog enables you to create three types of rules:

Row - A row rule checks every row in a table to see whether one or more fields meet certain criteria. For example, a row rule might check every row in a table for a null value in a certain field. The rule would return the number of rows where the field contained a null value. The check in a row rule is a DataFlux expression. A row rule is the only type of rule that can be used in a profile. For an example of a row rule, see Creating a Rule that Compares Two Fields.

Set - A set rule checks all values in one or more fields to see if the values meet certain criteria. For example, a set rule might check to see if the sum of all values in a field is greater than 1000. The check in a set rule can be a standard metric or a custom metric (a user-defined DataFlux expression). A set rule cannot be used in a profile, but it can be associated with a task, and the task can be called from a Data Monitoring node in a data job.

Group - A group rule checks a group of related values in a field. To specify a group of related values, click Group By and select one of the generic fields that have been defined in the Business Rules Manager. For example, if you were to select a generic field named State, the rule would be applied to groups of values from the same state. The check in a group rule can be a standard metric or a custom metric. A group rule cannot be used in a profile, but it can be associated with a task, and the task can be called from a Data Monitoring node in a data job.

Overview of DataFlux Expressions

You can specify a DataFlux expression as the check in all three types of business rules. The Expression Builder that is available from the Rule properties dialog enables you to build the expression for a rule without typing any code.

The Expression Builder dialog (shown above) provides a number of predefined expressions that enable you to perform operations such as the following:

• Compare an entire field to a value, to another field, to a list of values, or to a field in a data source.
• Determine if a field is inside or outside a range of values.
• Determine how a field differs in amount or percentage from another field
• Determine if a field matches a pattern or regular expression.
• Compare a portion of a field to a value.
• Determine if a field begins with, ends with, or contains a value.
• Compare part of a date field to a value.
• Get information about the contents of a field (is null, is numeric. etc.).
• Check the case of a field.
• Compare the length of a field to a value.

If the predefined expressions do not meet your needs, you can edit the predefined expressions or create your own DataFlux expressions. For more information about DataFlux expressions, see the DataFlux Expression Language Reference Guide.

In addition to the Expression Builder dialog, you can use the Expression tab in the Rule properties dialog to build expressions. The Functions pane on the Expression tab lists a number of DataFlux functions that you can use to build expressions, as shown in the next display.

Overview of Standard Metrics

You can specify a standard metric, such as MEAN, MODE, or MINVAL, as the check in a set rule or a group rule. The set rule or group rule can be associated with a task, and this task can be called from a Data Monitoring node in a data job.

When you specify a rule as a set rule or a group rule, you can use the Fields pane on the Expression tab to select standard metrics for one or more fields. For example, the next display shows some fields that have been defined for rules in the current repository: GIVEN_NAME2Name, GOVERNMENT_ID, ID, etc.

In the previous display, the folders for the ID field have been expanded to show the subfolders for two kinds of standard metrics: Standard and Historical. The metrics in each folder are identical, but they are used in different ways. Metrics in the Standard folder evaluate the current run of a Data Monitoring node. Metrics in the Historical folder evaluate the previous run of a Data Monitoring node. These two kinds of metrics could be used to compare the current value and the previous (historical) value of a field.

Default Business Rules

A default set of business rules is shipped with DataFlux Data Management Studio. These rules are located in the Rules folder that you can access by selecting Business Rules Manager > <Repository Name> in the Tools menu in the main window. Note that these rules are marked with a DF_ prefix. You can use these rules as models when you create your own rules, but you should not modify them in any way. The rules are used in running DataFlux Data Management Studio functions, such as DataFlux Master Data Foundations.