Usage Note 60129: DS2 and HPDS2 programs fail on Hortonworks using Tez
 |  |  |  |
If the Hive.execution.engine property is set to Tez, DS2 and HPDS2 programs that reference multiple tables might fail. For example, DS2 code that references a table that was created by the UNION ALL operator fails with the following message stack:
ERROR: java.io.IOException: Not a file:hdfs://sasserver.demo.sas.com:8020/apps/hive/warehouse/kcs_union_all/1
ERROR: at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:322)
[output truncated]
ERROR: at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:322)
[output truncated]
The issue occurs due to changes with Tez on Hortonworks.
You can circumvent the issue by setting the hive.merge.tezfiles=true property in the Properties= option in the LIBNAME statement. You can also set the property globally in the hive-site.xml file pointed to by the SAS_HADOOP_CONFIG_PATH environment variable by setting the following:
<property>
<name>hive.merge.tezfiles</name>
<value>true</value>
</property>
<name>hive.merge.tezfiles</name>
<value>true</value>
</property>
This problem is fixed in SAS® 9.4 TS1M4.
Operating System and Release Information
| Product Family | Product | System | SAS Release | |
| Reported | Fixed* | |||
| SAS System | SAS/ACCESS Interface to Hadoop | Windows 7 Home Premium x64 | 9.4 TS1M3 | 9.4 TS1M4 |
| Windows 7 Home Premium 32 bit | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Windows 7 Enterprise x64 | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Windows 7 Enterprise 32 bit | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Microsoft Windows Server 2012 Std | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Microsoft Windows Server 2012 R2 Std | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Microsoft Windows Server 2012 R2 Datacenter | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Microsoft Windows Server 2012 Datacenter | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Microsoft Windows Server 2008 for x64 | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Microsoft Windows Server 2008 R2 | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Microsoft Windows Server 2008 | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Microsoft Windows 10 | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Microsoft Windows 8.1 Pro x64 | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Microsoft Windows 8.1 Pro 32-bit | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Microsoft Windows 8.1 Enterprise x64 | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Microsoft Windows 8.1 Enterprise 32-bit | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Microsoft Windows 8 Pro x64 | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Microsoft Windows 8 Pro 32-bit | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Microsoft Windows 8 Enterprise x64 | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Microsoft Windows 8 Enterprise 32-bit | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Microsoft® Windows® for x64 | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Windows 7 Professional 32 bit | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Windows 7 Professional x64 | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Windows 7 Ultimate 32 bit | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Windows 7 Ultimate x64 | 9.4 TS1M3 | 9.4 TS1M4 | ||
| 64-bit Enabled AIX | 9.4 TS1M3 | 9.4 TS1M4 | ||
| 64-bit Enabled Solaris | 9.4 TS1M3 | 9.4 TS1M4 | ||
| HP-UX IPF | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Linux for x64 | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Solaris for x64 | 9.4 TS1M3 | 9.4 TS1M4 | ||
| Type: | Usage Note |
| Priority: |
| Date Modified: | 2017-04-04 10:18:18 |
| Date Created: | 2017-03-14 17:14:18 |