When troubleshooting performance problems, be aware that SAS processes put different demands on file systems and I/O than traditional databases or simple query processes. For that reason, you need to be able to determine the throughput rates of any file system that SAS uses. This paper documents a utility that system administrators and IT teams can use to help make that determination.

The sasiotest.exe utility for Microsoft Windows platforms can be used to measure the I/O behavior of the system under defined loads. The utility is easy to use and can be used to launch individual or multiple concurrent I/O tests to flood the file system and determine its raw performance.

The sasiotest.exe utility creates and writes files to the file system being tested, then reads them back, to test both Write and Read performance of the file system. The primary output from this tool is the megabytes written per second (Write MB/sec) and megabytes read per second (Read MB/sec). These metrics help you analyze and tune I/O for file systems that support SAS.

You can compare those results to specific recommendations given in any SAS Enterprise Excellence Center System Sizing, or general per-file system recommendations in SAS papers referenced below.

There are several SAS papers available that can help you understand how SAS performs I/O as well as minimum I/O recommendations for SAS file systems. These papers outline the recommended I/O metrics for file systems that support SAS deployments, and they identify guidelines that can help with provisioning and tuning I/O characteristics for optimal SAS performance. You can access these papers via SAS Note 42197: "A list of papers useful for troubleshooting system performance problems":

• Best Practices for Configuring Your IO Subsystem for SAS9 Applications
• Frequently Asked Questions Regarding Storage Configurations
• Guidelines for Preparing your Computer Systems for SAS^®
• How to Maintain Happy SAS^®9 Users

Also, SAS Note 42197 contains papers related to the setup and tuning of specific vendor operating systems and storage arrays.

Before beginning I/O throughput tests, it is important to consider the following concepts:

• SAS performs its Write and Read operations via the host system file cache. File sizes used for the Write and Read tests should be larger than your host system file cache. (It is safe to make them greater than the RAM in your host system.) For example, if you run your test with a 1GB file size, and your system has 5GB of cache, then the tests exercise limited disk I/O on the Read test. The file that is written remains in the system file cache, and the subsequent Read operation reads it from cache instead of from disk. If the Read tests are satisfied via cache instead of coming from disks, the Read performance metrics are misleading, and you are not able to generate consistent, repeatable results.
• An invocation of a single instance test of either of these tools (one Write/Read test) shows how a single SAS job might fare at that point in time on that file system. If your system is already very busy with other users’ workloads, this might be adequate.
• A multiple instance invocation of the tests show better what a concurrent SAS working load might experience, and is recommended for systems experiencing no other current jobs, or a very light workload.
• It is wise to conduct several of these test sets at various times of the day, and over several busy days to get a comprehensive profile of I/O characteristics on systems that run a variety of workloads.
• The time frame in which you run your tests can make a difference in the results. It is preferable to place a targeted test during the same time of day and file system(s) as you are currently experiencing under-performing SAS jobs.
• Run the tests against the file system you want to target. Direct the results output to a different file system than the one under test, by executing the sasiotest.exe program in a different file system, and/or directing the results to a different file system using the pipe command, if running in batch.
• The test output shows throughput results in MB/sec. Note that a minimum of 100MB/sec per core is recommended for SAS file systems.
• If you are testing your file system throughput to solve an existing performance problem, see the following papers on performance problem resolution using SAS logs in combination with host monitors:
• Solving SAS^® Performance Problems: Employing Host-Based Tools
• A Practical Approach to Solving Performance Problems with SAS^®
• SAS^® Performance Monitoring - A Deeper Discussion

The sasiotest.exe Utility

The sasiotest.exe utility for Microsoft Windows is included as part of your SAS software delivery, starting in SAS^® 9.2. This utility executes on Windows 64, X64, and 64-bit platforms. The utility simulates SAS Read and Write activity, although it is a stand-alone program (which means you can run this program on systems where SAS software is not yet installed). For each run, the program writes an output file that captures elapsed real time, and I/O rate, expressed as megabytes per second (MB/sec). The program is delivered in your SAS software depot and is installed in SASHOME at this general address: SASHOME\SASFoundation\9.x\sasiotest.exe.

The utility uses the Windows API Writefile(), Readfile() and Closefile(). These are the same API calls SAS makes. The utility also supports the Readfilescatter() and Writefilescatter() APIs for Scatter-Gather I/O Testing outside the file cache. Starting with SAS^® 9.4, the utility also supports reading random pages from existing files. Since the utility is independent of the SAS engine, it can be copied and used as a stand-alone executable on any system with or without SAS software present. The sasiotest.exe utility from a SAS 9.4 installation can be copied to and used on a machine where earlier releases of SAS are deployed.

Command Syntax

The syntax of the command is shown below. The command can be run interactively from a Command Prompt window, or submitted as a batch job. One benefit of submitting as a batch job is that you can run several I/O tests concurrently to simulate an increased system load.

   SASIOTEST filename [-options] 

Where:

filename (required) specifies the filename to read or write, and MUST be the first argument
[-options] is one or more of the following (defaults are in parentheses):

• -w or –r
  When using (-w), SASIOTEST invokes a Write operation to create and write a specified file. When using –r, SASIOTEST reads from the specified file.
• -seq or -ran
  When using –seq, SASIOTEST reads sequentially from the specified file. With –ran, SASIOTEST reads randomly from the specified file. If not specified, the test defaults to -seq.
• -filesize [N]
  Specifies the total size of file to be read or created. For example, substitute [N] with 800k, 512K, 5MB, 40MB, 2GB, 10GB, and so on. If –filesize is not specified, the default is 1 MB.
• -pagesize [N]
  Specifies the size of the page for I/O operation (for example, 8k, 64k, 128k). See the note on -pagesize below. If –pagesize is not specified, the default is 8k.
• -sgio
  Specifies the use of Scatter Gather (SGIO) Read and/or SGIO Write APIs for I/O operations.
  Following are options specific to -sgio:
• -numpages [N]
  specifies the number of pages to read or write for each SGIO operation. The numpages option is required if the SGIO option is used.
• -unbuffered
  causes the pages written to bypass the file cache.
• -writethru
  causes the pages to write through any intermediate cache and go directly to disk.

Syntax Notes:

1. sasiotest.exe displays the elapsed real time, and I/O rates in MB/sec.
2. When using the -sgio option, -pagesize must be a multiple of 4K on x86 systems, and a multiple of 8K on x64 and IPF systems.

Command Example and Sample Output

The sasiotest.exe command can be launched from a Command Prompt window, and the options are defined in-line per the syntax above. A sample command to run sasiotest.exe follows:

c:\Program Files\SASHome\SASFoundation\9.4\sasiotest.exe c:\saswork\testfile1.dat -w -filesize 129G -pagesize 128K 

In the example above, sasiotest.exe is launched from its original directory in SASHOME. It creates a test file named testfile1.dat in the C:\ file system (which is the file system being tested). It is doing a Write test (-w), on a 129-gigabyte file (-filesize 129G) with a pagesize of 128K (-pagesize 128K). The command opens a Microsoft Command window, executes the test, and writes its results to the command window. The file created remains in the location that it was written to, it is deleted. It is still needed for the Read test. The command for the Read test is as follows:

c:\Program Files\SASHome\SASFoundation\9.4\sasiotest.exe c:\saswork\testfile1.dat -r -filesize 129G -pagesize 128K

This test reads back the file created above in the Write test. It is important to use the same options specified on the Write test for optimal test performance.

Note that in the examples above we are running the program from its original location in SASHOME. This is not required. It can be copied to and run from any location. You can pipe the results to a file located anywhere, or you can simply view them via the Microsoft Command window.

Multiple concurrent test instances can be launched via simple BAT files to create a large multiple concurrent workload test set. Note that each concurrent test must specify unique filenames for the test file in the Write test, and for the results files used to hold the results, if you choose to use a “pipe” command to store them on disk. In the example, below, WRITE.BAT is the “container” script, which calls each of the concurrent 5 runs that we chose to execute (that is, WRUN1.BAT – WRUN5.BAT), and specifies the unique file names used to write to, as well as the unique file names used to pipe the results to for persistent keeping. In the WRITE.BAT container script, using the “start” command allows the next command to be processed immediately so that the tests execute simultaneously rather than sequentially. Each of the concurrent runs should use the same script parameters for -pagesize and -filesize.

There is something important to note about the –pagesize value. Your underlying storage system typically is set up with RAID striping, which is constructed with a physical strip depth. This defines the size of each individual write to that physical stripe set. This might be the individual I/O Write/Read size presented back to the host server, or it might be overridden by a LUN setting or logical volume manager. Consult with your storage array administrator to see what the actual physical transfer size is to your array, and match it with the –pagesize value. These values typically fall in the range values of 64KB, 128KB, 256KB, or 1MB.

Note: The syntax of the sasiotest.exe command expresses these values as K and M, not as KB and MB.

In the following batch script example, WRITE.BAT starts batch scripts wrun1.bat – wrun5.bat, which are located in the C:\ file system. It starts them simultaneously. Each of the runs, wrun1, 2, 3, 4, and 5.bat, all write to unique file names. If you were to run this sample script more than once, you would be overwriting the file names and results files. Making this a production run script that could be run repeatedly would require an allowance for not hardcoding the same file names as we did in this simple example. The highlighted number 1 in the example below would be rendered as 1 in wrun1.bat, 2 in wrun2.bat, and so on, to render unique file names.

WRITE.BAT  
c:
start “wrun1" run1.bat
start “wrun2" run2.bat
start “wrun3" run3.bat
start “wrun4" run4.bat
start “wrun5" run5.bat
EXIT
WRUN1.BAT – WRUN5.bat (each test is a single write instance creating a unique file name)
c:
cd C:\Program Files\SASHome\SASFoundation\9.4
sasiotest.exe c:\saswork\testfile1.dat -w -filesize 129M -pagesize 128K > c:\run1results.txt
exit

The “read” batch scripts can be written to automatically delete the file after the Read test. An example is below:

RRUN1.BAT – RRUN5.bat (each test is a single Read instance creating a unique file name)
    c:
    cd C:\Program Files\SASHome\SASFoundation\9.4
    sasiotest.exe d:\testfile1.dat -r -filesize 50M -pagesize 128K > c:\run1results.txt
    del c:\saswork\testfile1.dat
    exit

The output of the tests run via a concurrent-run batch script is displayed to the individual Microsoft Command windows launched for each run above when run interactively, or written to a designated file (run1results.txt – run5 results.txt in the sample above) when piped to a file in a batch run (as in >c:\run1results.txt above). The general format of the output is as follows:

Write Example:  (sasiotest.exe c:\saswork\testfile1.dat -w -filesize 129G –pagesize  128K)
SAS I/O Test Utility: v1.0
Sequentially writing 138512695296 bytes to file: c:\saswork\testfile1.dat with a 131072 pagesize.
Write Throughput: 544.231743 MB/Sec.

Read Example:  (sasiotest.exe c:\saswork\testfile1.dat -r -filesize 129G –pagesize  128K)
SAS I/O Test Utility: v1.0
Sequentially reading 138512695296 bytes from file: c:\saswork\testfile1.dat with a 131072 pagesize.
Read Throughput: 326.574128 MB/Sec.

In the simple example above, a test file, testfile1.dat, is created on file system c:\saswork, with a -filesize of 129 GB using a –pagesize of 128K. The file is written in the first test (-w), then read back in the second test (-r). The file remains in the tested directory until manually deleted.

The output in the Read and Write examples above suggest that the I/O performance when using a -pagesize of 128K is good for SAS performance on the underlying array, which supports an I/O transfer size of 128K. The Write throughput is 544 MB/sec, and the Read 326 MB/sec. This was expected because this is a very large array with many spindles to aggregate disk performance. It is typical that Reads are faster than Writes for most file systems, but this can be storage-array-dependent in terms of array setup and architecture, as in this case.

By specifying options on the command line, you can vary the type of test (Read or Write), the type of I/O that is performed by the test, the size of the I/O operations (-pagesize), and the total size of the file to be created (or read). By default, if you do not enter any of the optional parameters on the sasiotest.exe command, the utility performs a Read test for a 1MB file, using sequential I/O and an 8K page size. By entering -pagesize and -filesize parameters in the test invocation, you can override those values to use larger page sizes (64K is recommended as a default if you do not know your array I/O transfer size) and larger file sizes (larger than your RAM is recommended).

Note: If your host system has a lot of RAM (>64 GB), the file sizes created for the test to be larger than RAM can benefit from a larger –pagesize than the I/O transfer size of your storage array. For example, the test above was run again with a higher transfer size than the array’s 128 KB. Because the file was so large, we experimented with a –pagesize value of 1M to see whether it would help. This increased the Write rate to 820 MB/sec and the Read rate to 772 MB/sec. Why did this happen? Because the file was so large, and the array had a lot of Write cache enabled, the larger transfer size was very efficiently de-staged to 128K Writes to physical disk very efficiently by the array. With very large files, a larger transfer size can sometimes be advantageous. If not all of your files are so large though (for general, mixed SAS workloads), we recommend defaulting to your actual array transfer size, or 64K if you do not know what that is.

What Should You Expect to See?

For nominally good SAS I/O performance, you should see job steps in your SAS logs with combined CPU + User CPU time within 10% of the Real Time. If this is not the case, there is generally some benefit to tuning or better provisioning I/O from your SAS file systems.

A good performance target for a typical system is data throughput from the file system to match at least 75 MB/sec per processor core for small to average systems, and upward of 100 for larger, more heavily used systems. A minimum throughput rate of 100 MB/sec throughput should be provided for any SAS file system.

Work with your Storage Administrator if you believe you are not getting the performance that you need. Share with them the various operating system, I/O, and storage array tuning papers that can be found in SAS Note 42197.

Operating System and Release Information

Product Family	Product	System	SAS Release
			Reported	Fixed*
SAS System	Base SAS	Microsoft® Windows® for 64-Bit Itanium-based Systems
		Microsoft Windows Server 2003 Datacenter 64-bit Edition
		Microsoft Windows Server 2003 Enterprise 64-bit Edition
		Microsoft® Windows® for x64
		Microsoft Windows Server 2003 for x64
		Microsoft Windows Server 2008 for x64

* For software releases that are not yet generally available, the Fixed Release is the software release in which the problem is planned to be fixed.

---

Type:	Usage Note
Priority:

Date Modified:	2014-02-05 19:50:29
Date Created:	2013-11-20 13:27:47