Problem Note 57776: Queries run against a large Hive table might be slow
With SAS/ACCESS® Interface to Hadoop, a standard part of each query includes a SELECT * FROM TABLE statement to retrieve table metadata. This query is generally very fast and has no significant overhead. However, if the hive table is large and the hive property hive.fetch.task.conversion.threshold setting is exceeded, it can cause the launch of a large MapReduce job, which would be quite costly.
To circumvent this issue:
- Add this to the SAS LIBNAME statement: properties="hive.fetch.task.conversion.threshold=-1;hive.fetch.task.conversion=more"
- If you prefer, set the hive.fetch.task.conversion property to minimal. Setting it prevents the statement from picking up a value of NONE, which would result in a MapReduce being run for all queries.
Operating System and Release Information
| SAS System | SAS/ACCESS Interface to Hadoop | Microsoft® Windows® for x64 | 9.4 TS1M2 | 9.4 TS1M4 |
| Microsoft Windows 8 Enterprise 32-bit | 9.4 TS1M2 | 9.4 TS1M4 |
| Microsoft Windows 8 Enterprise x64 | 9.4 TS1M2 | 9.4 TS1M4 |
| Microsoft Windows 8 Pro 32-bit | 9.4 TS1M2 | 9.4 TS1M4 |
| Microsoft Windows 8 Pro x64 | 9.4 TS1M2 | 9.4 TS1M4 |
| Microsoft Windows 8.1 Enterprise 32-bit | 9.4 TS1M2 | 9.4 TS1M4 |
| Microsoft Windows 8.1 Enterprise x64 | 9.4 TS1M2 | 9.4 TS1M4 |
| Microsoft Windows 8.1 Pro | 9.4 TS1M2 | 9.4 TS1M4 |
| Microsoft Windows 8.1 Pro 32-bit | 9.4 TS1M2 | 9.4 TS1M4 |
| Microsoft Windows 10 | 9.4 TS1M2 | 9.4 TS1M4 |
| Microsoft Windows Server 2008 | 9.4 TS1M2 | 9.4 TS1M4 |
| Microsoft Windows Server 2008 R2 | 9.4 TS1M2 | 9.4 TS1M4 |
| Microsoft Windows Server 2008 for x64 | 9.4 TS1M2 | 9.4 TS1M4 |
| Microsoft Windows Server 2012 Datacenter | 9.4 TS1M2 | 9.4 TS1M4 |
| Microsoft Windows Server 2012 R2 Datacenter | 9.4 TS1M2 | 9.4 TS1M4 |
| Microsoft Windows Server 2012 R2 Std | 9.4 TS1M2 | 9.4 TS1M4 |
| Microsoft Windows Server 2012 Std | 9.4 TS1M2 | 9.4 TS1M4 |
| Windows 7 Enterprise 32 bit | 9.4 TS1M2 | 9.4 TS1M4 |
| Windows 7 Enterprise x64 | 9.4 TS1M2 | 9.4 TS1M4 |
| Windows 7 Home Premium 32 bit | 9.4 TS1M2 | 9.4 TS1M4 |
| Windows 7 Home Premium x64 | 9.4 TS1M2 | 9.4 TS1M4 |
| Windows 7 Professional 32 bit | 9.4 TS1M2 | 9.4 TS1M4 |
| Windows 7 Professional x64 | 9.4 TS1M2 | 9.4 TS1M4 |
| Windows 7 Ultimate 32 bit | 9.4 TS1M2 | 9.4 TS1M4 |
| Windows 7 Ultimate x64 | 9.4 TS1M2 | 9.4 TS1M4 |
| 64-bit Enabled AIX | 9.4 TS1M2 | 9.4 TS1M4 |
| 64-bit Enabled Solaris | 9.4 TS1M2 | 9.4 TS1M4 |
| HP-UX IPF | 9.4 TS1M2 | 9.4 TS1M4 |
| Linux for x64 | 9.4 TS1M2 | 9.4 TS1M4 |
| Solaris for x64 | 9.4 TS1M2 | 9.4 TS1M4 |
*
For software releases that are not yet generally available, the Fixed
Release is the software release in which the problem is planned to be
fixed.
| Type: | Problem Note |
| Priority: | medium |
| Date Modified: | 2016-03-25 11:58:24 |
| Date Created: | 2016-03-01 16:25:18 |
