Problem Note 60755: SAS® Real-Time Decision Manager experiences intermittent high response times when you use DS2 and SAS® Federation Server 4.1
 |  |  |  |  |
SAS Real-Time Decision Manager enables you to use custom DS2 code in order to add functionality to campaigns. The DS2 code is executed on an underlying SAS Federation Server that is installed and configured as part of SAS Real-Time Decision Manager. When you use DS2 code heavily in your campaigns, you might see intermittent periods in which response time is unacceptably high. This problem is more likely to occur if the custom code makes many SQL calls. Restarting SAS Real-Time Decision Manager returns response time to normal, but does not prevent eventual recurrence of the problem.
There are no easily detectable errors or warnings that are specific to this problem. Memory usage does not become unusually high, nor does CPU usage. In fact, there are no obvious resource constraints. In a Java thread dump for the SAS Federation Server, like the one shown below, you can see that an unexpectedly large number of threads (sometimes > 200) are in a state of waiting:
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:503)
at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1104)
- locked <0x00000004cccd0000> (a org.apache.commons.pool.impl.GenericObjectPool$Latch)
at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
The Decision Services log might also contain messages that the connections to the SAS Federation Server are killed and restarted more often than expected because of timeouts in DS2 activity execution.
The underlying memory management for DS2 objects can require too much time as the number of DS2 objects grows. Making multiple unparameterized SQLSTMT calls can cause particular strain.
You can alleviate this problem by adding more SAS Federation Servers to your architecture. However, modifying SAS Federation Server configuration or adding resources to existing SAS Federation Servers does not resolve the issue.
Click the Hot Fix tab in this note to access the hot fix for this issue.
Post Hot Fix Instructions
In addition to installing the hot fix, you must also enable pool caching by setting the SKM_POOL_CACHE=1 environment variable:
- Stop SAS Federation Server.
- Set the SKM_POOL_CACHE=1 environment variable.
Note: For Linux, you use: export SKM_POOL_CACHE=1 - Restart SAS Federation Server
Operating System and Release Information
| Product Family | Product | System | Product Release | SAS Release | ||
| Reported | Fixed* | Reported | Fixed* | |||
| SAS System | SAS Federation Server | Linux for x64 | 4.1 | 4.1_M1 | 9.4 TS1M3 | 9.4 TS1M5 |
| SAS System | SAS Real-Time Decision Manager | 64-bit Enabled Solaris | 5.4 | 6.1 | 9.2 TS2M3 | 9.4 TS1M5 |
| 64-bit Enabled AIX | 5.4 | 6.1 | 9.2 TS2M3 | 9.4 TS1M5 | ||
| Windows 7 Ultimate x64 | 5.4 | 6.1 | 9.2 TS2M3 | 9.4 TS1M5 | ||
| Windows 7 Ultimate 32 bit | 5.4 | 6.1 | 9.2 TS2M3 | 9.4 TS1M5 | ||
| Windows 7 Professional x64 | 5.4 | 6.1 | 9.2 TS2M3 | 9.4 TS1M5 | ||
| Windows 7 Professional 32 bit | 5.4 | 6.1 | 9.2 TS2M3 | 9.4 TS1M5 | ||
| Windows 7 Home Premium x64 | 5.4 | 6.1 | 9.2 TS2M3 | 9.4 TS1M5 | ||
| Windows 7 Home Premium 32 bit | 5.4 | 6.1 | 9.2 TS2M3 | 9.4 TS1M5 | ||
| Windows 7 Enterprise x64 | 5.4 | 6.1 | 9.2 TS2M3 | 9.4 TS1M5 | ||
| Windows 7 Enterprise 32 bit | 5.4 | 6.1 | 9.2 TS2M3 | 9.4 TS1M5 | ||
| Microsoft Windows Server 2008 for x64 | 5.4 | 6.1 | 9.2 TS2M3 | 9.4 TS1M5 | ||
| Microsoft Windows Server 2008 R2 | 5.4 | 6.1 | 9.2 TS2M3 | 9.4 TS1M5 | ||
| Microsoft Windows Server 2008 | 5.4 | 6.1 | 9.2 TS2M3 | 9.4 TS1M5 | ||
| Microsoft® Windows® for x64 | 5.4 | 6.1 | 9.2 TS2M3 | 9.4 TS1M5 | ||
| Linux for x64 | 5.4 | 6.1 | 9.2 TS2M3 | 9.4 TS1M5 | ||
| Solaris for x64 | 5.4 | 6.1 | 9.2 TS2M3 | 9.4 TS1M5 | ||