Problem Note 59909: Using user-defined formats in a WHERE clause in SAS® Scalable Performance Data Engine slows performance in some applications
 |  |  |  |  |  |
Using a user-defined format in a WHERE clause might prevent the SPD Engine from performing engine-level optimization.
For example, in a reported case, the SUMMARY procedure was pointing to an SPD Engine data set and using a WHERE clause with user-defined formats. Pointing to a data set with PROC SUMMARY takes less than a second to run, whereas pointing to an SPD Engine data set takes more than 40 minutes to run.
The following warning occurred in the log:
WARNING: A function, format, or informat has prevented SPD Engine from processing the where clause. The SAS supervisor will process it instead, and no parallel where-clause processing will occur.
To see an example of code that can cause the issue, click the Full Code tab.
Click the Hot Fix tab in this note to access the hot fix for this issue.
Operating System and Release Information
| Product Family | Product | System | SAS Release | |
| Reported | Fixed* | |||
| SAS System | Base SAS | z/OS | 9.4 TS1M2 | 9.4 TS1M4 |
| z/OS 64-bit | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Microsoft® Windows® for x64 | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Microsoft Windows 8 Enterprise 32-bit | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Microsoft Windows 8 Enterprise x64 | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Microsoft Windows 8 Pro 32-bit | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Microsoft Windows 8 Pro x64 | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Microsoft Windows 8.1 Enterprise 32-bit | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Microsoft Windows 8.1 Enterprise x64 | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Microsoft Windows 8.1 Pro 32-bit | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Microsoft Windows 8.1 Pro x64 | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Microsoft Windows 10 | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Microsoft Windows Server 2008 | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Microsoft Windows Server 2008 R2 | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Microsoft Windows Server 2008 for x64 | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Microsoft Windows Server 2012 Datacenter | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Microsoft Windows Server 2012 R2 Datacenter | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Microsoft Windows Server 2012 R2 Std | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Microsoft Windows Server 2012 Std | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Windows 7 Enterprise 32 bit | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Windows 7 Enterprise x64 | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Windows 7 Home Premium 32 bit | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Windows 7 Home Premium x64 | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Windows 7 Professional 32 bit | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Windows 7 Professional x64 | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Windows 7 Ultimate 32 bit | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Windows 7 Ultimate x64 | 9.4 TS1M2 | 9.4 TS1M4 | ||
| 64-bit Enabled AIX | 9.4 TS1M2 | 9.4 TS1M4 | ||
| 64-bit Enabled Solaris | 9.4 TS1M2 | 9.4 TS1M4 | ||
| HP-UX IPF | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Linux for x64 | 9.4 TS1M2 | 9.4 TS1M4 | ||
| Solaris for x64 | 9.4 TS1M2 | 9.4 TS1M4 | ||
The following PROC SUMMARY shows how the problem manifests.
LIBNAME myspde SPDE 'c:\tmp\SPDE';
proc summary data=myspde.test missing;
where
(((put(MONYY,newformat.) = "Value13")
or ( trim(left(put(MONYY, ? MONYY.))) = "Value13"))
OR
((put(YYQTR,newformat.) = "Value5")
or ( trim(left(put(YYQTR, ? YYQ6.))) = "Value5"))
OR
((put(MONYY,newformat.) = "Value2")
or ( trim(left(put(MONYY, ? MONYY.))) = "Value2"))) ;
format MONYY ;
format YYQTR ;
format HOSPOPTID ;
class MONYY / ASCENDING order=UNFORMATTED ;
class YYQTR / ASCENDING order=UNFORMATTED ;
class HOSPOPTID / ASCENDING order=UNFORMATTED ;
types ()
MONYY
YYQTR
MONYY* YYQTR
HOSPOPTID
MONYY*YYQTR*HOSPOPTID
;
format COUNTCOVERS_W best16.;
var
COUNTCOVERS_W
;
output out=WORK.summaryOutput(drop=_freq_)
SUM(COUNTCOVERS_W)=M0003755
;
run;A fix for this issue for Base SAS 9.4_M4 is available at:
http://ftp.sas.com/techsup/download/hotfix/HF2/A3Z.html#59909| Type: | Problem Note |
| Priority: | medium |
| Date Modified: | 2017-02-15 12:06:09 |
| Date Created: | 2017-02-06 17:13:18 |