The
Text
Filter node can be used to reduce the total number of
parsed terms or documents that are analyzed. Therefore, you can eliminate
extraneous information so that only the most valuable and relevant
information is considered. For example, the
Text Filter node
can be used to remove unwanted terms and to keep only documents that
discuss a particular issue. This reduced data set can be orders of
magnitude smaller than the one that represents the original collection,
which might contain hundreds of thousands of documents and hundreds
of thousands of distinct terms.
For more information
about the
Text Filter node, see the SAS Text
Miner Help.
The rest of this chapter
presents an example of how you can use the
Text Filter node.