Most sites use the SPD Engine to manage very large amounts of data, which can have
thousands of variables and some
of them indexed. At these sites, separate storage paths are usually defined for the
various component types. In addition, using disk striping and
RAID (Redundant Array of Independent Disks) can be very efficient. For more information,
see “SPD Engine Disk
I/O Setup” in Scalability and Performance at
http://support.sas.com/rnd/scalability/spde/setup.html.
The metadata component files for all data sets in a library must reside in the primary
path.
In addition, specifying
separate paths for the data component files and index component files
provides performance gains. You specify separate paths because the
Read load is distributed across disk drives. Separating the data and
index component files helps prevent disk contention and increases
the level of
parallelism that can be achieved, especially in complex WHERE evaluations. The following example
code specifies a primary path for the metadata component files. The code uses the
DATAPATH= option and the
INDEXPATH= option to specify additional, separate paths
for the data and index component files:
libname all_users spde '/disk1/metadata'
datapath= ('/disk2/userdata' '/disk3/userdata')
indexpath= ('/disk4/userindexes' '/disk5/userindexes');
The
metadata component files are stored on disk1, which is the primary
path. The data component files are on disk2 and disk3, and the index
component files are on disk4 and disk5. For all path specifications,
you must specify the complete pathname.
CAUTION:
The primary
path must be unique for each library.
If two librefs are created with the same primary path but with differences in the
other paths, data can be lost. You cannot use NFS in
any path other than the primary path.
Note: If you are planning to store
data in locally mounted drives and to access the data from a remote
computer, use the remote pathname when you specify the LIBNAME. If /data01
and /data02
are
locally mounted drives on the localA computer, use the pathnames /nfs/localA/data01
and /nfs/localA/data02
in
the LIBNAME statement.