In the Text Import node of SAS^® Text Miner, the maximum number of documents that might be converted into text documents in the destination directory folder is 99,999. If you have more than 99,999 documents in the "Import File Directory", then a portion of the documents is not correctly processed. The destination directory folder might not contain all of the first 99,999 converted documents.

There are not warning or error messages.

The following steps provide a work-around:

1. Move the documents in the "Import File Directory" to multiple folders, say k folders, so that each folder contains approximately 1/k of the total number of documents. Make sure that each folder contains fewer than 99,999 documents.

2. Create k Text Import nodes that import documents from their corresponding folders. Run each Text Import node.

3. After all of the k Text Import nodes are completed, create a SAS Code node and connect it to each of those k Text Import nodes. Insert the following SAS code in the Code Editor. The code concatenates all of the exported data sets that are created by those k Text Import nodes into a new SAS data set. The new SAS data set is saved in a system folder that you define:

Submit the code and check the Log to make sure that there are no errors. Create a new data source for the "mylib.combined" data set.

Operating System and Release Information