DataFlux Data Management Studio 2.7: User Guide

N-Gram Analysis Group

The N-Gram Analysis Group holds a number of N-Gram Scheme Nodes. See the Language Guess Definitions special constraints.

Used in:

• Language Guess Definitions

Properties

N-Gram window size: The number of characters in an N-Gram.

Include word boundaries in n-gram lookup - Check this box to treat the beginning and end of a line as special characters.

Output

Unlike other groups, the N-Gram Analysis Group's output is not the output of the last node in the group. This is because the nodes in the group do not have individual outputs. Instead, all of the schemes are combined to produce a single output.

Results

Table with four columns:

• 1st column - each of the N-Grams in the input
• 2nd column - number of occurrences in the input
• 3rd column - index of first occurrence in the input
• 4th column - number of occurrences in the training data

Raw score

The score obtained from N-Gram analysis (between 0 and 1000).

Bias setting

The bias setting previously chosen.

Bias factor

The numerical weight that the bias setting translates to.

Bias score

The N-Gram score after applying the bias.