Problem Note 60410: Using the HDFS_TEMPDIR= option in a LIBNAME statement to connect to Hadoop might produce an error
Using the Hadoop HDFS_TEMPDIR= LIBNAME option in a Hadoop environment secured by Kerberos might produce a misleading error.
When you connect to a Hadoop cluster that is secured by Kerberos, the HDFS_TEMPDIR= location is checked before actually assigning the LIBNAME statement. If a valid Kerberos ticket does not exist or the ticket has expired, the LIBNAME connection fail with an error message similar to the following in the SAS® log.
libname hd hadoop server=server-name hdfs_tempdir='/sastmp';
ERROR: HDFS_TEMPDIR directory /sastmp does not exist or is inaccessible.
ERROR: Error in the LIBNAME statement.
Even though the location specified in the HDFS_TEMPDIR option might physically exist, the error occurs because the LIBNAME statement fails. As a result, the HDFS_TEMPDIR location cannot be accessed.
The above error message does not indicate which error might have occurred so that information is not documented in the SAS log. Removing the HDFS_TEMPDIR= option from the LIBNAME statement allows the underlying connection error to be written to the SAS log. The following "GSS initiate failed" error indicates that a valid Kerberos ticket does not exist:
libname hd hadoop server=server-name;
ERROR: java.sql.SQLException: Could not open client transport with JDBC Uri:
jdbc:hive2://drama:10000/default;principal=hive/_HOST@TSDREALM01.COM:
GSS initiate failed
ERROR: Error trying to establish connection.
ERROR: Error in the LIBNAME statement.
You should assign the Hadoop LIBNAME statement without the HDFS_TEMPDIR= option to verify whether Java Database Connectivity (JDBC) is successful. When the LIBNAME statement is successfully assigned and the HDFS_TEMPDIR= option is necessary to redirect temporary files, add the option to the LIBNAME statement.
A fix for this issue is planned for a future release.
Operating System and Release Information
SAS System | SAS/ACCESS Interface to Hadoop | Microsoft® Windows® for x64 | 9.4_M1 | | 9.4 TS1M1 | |
Microsoft Windows 10 | 9.4_M1 | | 9.4 TS1M1 | |
Microsoft Windows Server 2008 | 9.4_M1 | | 9.4 TS1M1 | |
Microsoft Windows Server 2012 Datacenter | 9.4_M1 | | 9.4 TS1M1 | |
Microsoft Windows Server 2012 Std | 9.4_M1 | | 9.4 TS1M1 | |
64-bit Enabled AIX | 9.4_M1 | | 9.4 TS1M1 | |
64-bit Enabled Solaris | 9.4_M1 | | 9.4 TS1M1 | |
HP-UX IPF | 9.4_M1 | | 9.4 TS1M1 | |
Linux for x64 | 9.4_M1 | | 9.4 TS1M1 | |
Solaris for x64 | 9.4_M1 | | 9.4 TS1M1 | |
*
For software releases that are not yet generally available, the Fixed
Release is the software release in which the problem is planned to be
fixed.
Type: | Problem Note |
Priority: | high |
Date Modified: | 2018-07-26 08:24:06 |
Date Created: | 2017-05-04 13:59:08 |