Further Notes

SAS/IML matrices are kept in RAM. Memory is automatically allocated when needed. If you are interested in how memory is used and allocated in PROC IML, the following sections explain the details.

There are two logical areas of memory: the symbol space and the workspace. The symbol space contains memory that is associated with the symbol table and compiled statements. The workspace contains all values in SAS/IML matrices. The workspace itself is divided into one or more blocks of memory.

At the start of a PROC IML session, the symbol space and the first workspace block are allocated automatically. If matrix operations consume the available workspace, more memory is allocated. The SYMSIZE= and WORKSIZE= options in the PROC IML statement give you control over the size of symbol space and the size of each extent of workspace. If you do not specify these options, PROC IML uses host-dependent defaults. For example, you can begin an IML session and set the SYMSIZE= and WORKSIZE= options with the statement

   proc iml symsize=n1 worksize=n2;

where n1 and n2 are specified in kilobytes.

If the symbol space memory becomes full, more memory is allocated automatically. The symbol space is stable memory and is not compressible like workspace memory. Symbol space is recycled whenever possible. For example, temporary symbols can be deleted after they are used in evaluating an expression. The symbol space formerly used by these temporary variables is added to a list of free symbol-table nodes. When allocating temporary variables to evaluate another expression, PROC IML looks for symbol-table nodes in this list first before consuming unused symbol space.

Workspace memory is compressible. As matrices are assigned, the workspace memory fills up. As you free matrices (or as PROC IML frees temporary intermediate results), holes appear in the memory blocks. When a memory block does not have room to store a new matrix, compression reclaims the holes that have appeared in the memory. If compression does not reclaim enough memory for the current allocation, IML allocates a new block of memory. This procedure results in the existence of a list of extents, each of which contains a mixture of active memory and holes of unused memory. There is always a current extent, the one in which the last allocation was made.

For a new allocation, the search for free space begins in the current extent and proceeds around the extent list until finding enough memory or returning to the current extent. If the search returns to the current extent, IML begins a second transversal of the extent list, compressing each extent until either finding sufficient memory or returning to the current extent. If the second search returns to the current extent, IML opens a new extent and makes it the current one.

If the SAS System cannot provide enough memory to open a new extent with the full extent size, IML repeatedly reduces its request by 2K. In this case, the successfully opened extent is smaller than the standard size.

If a single allocation is larger than the standard extent size, IML requests an allocation large enough to hold the matrix.

The WORKSIZE= and SYMSIZE= options offer tools for tuning memory usage. For data-intensive applications that involve a few large matrices, use a high WORKSIZE= value and a low SYMSIZE= value. For symbol-intensive applications that involve many matrices, perhaps through the use of many IML modules, use a high SYMSIZE= value.

You can use the SHOW SPACE command to display the current status of IML memory usage. This command also lists the total number of compressions done on all extents.

Setting the DETAILS option in the RESET command prints messages in the output file when IML compresses an extent, opens a new extent, allocates a large object, or acquires more symbol space. These messages can be useful because these actions normally occur without the user’s knowledge. The information can be used to tune WORKSIZE= and SYMSIZE= values for an application. However, the default WORKSIZE= and SYMSIZE= values should be appropriate in most applications.

Do not specify a very large value in the WORKSIZE= and SYMSIZE= options unless absolutely necessary. Many of the native functions and all of the DATA step functions used are dynamically loaded at execution time. If you use a large amount of the memory for symbol space and workspace, there might not be enough remaining to load these functions, resulting in the error message

Should you run into this problem, issue a SHOW SPACE command to examine current usage. You might be able to adjust the SYMSIZE= or WORKSIZE= values.

The amount of memory your system can provide depends on the capacity of your computer and on the products installed. The following techniques for efficient memory use are recommended when memory is at a premium: