SUPPORT / SAMPLES & SAS NOTES
 

Support

Problem Note 60410: Using the HDFS_TEMPDIR= option in a LIBNAME statement to connect to Hadoop might produce an error

DetailsHotfixAboutRate It

Using the Hadoop HDFS_TEMPDIR= LIBNAME option in a Hadoop environment secured by Kerberos might produce a misleading error. 

When you connect to a Hadoop cluster that is secured by Kerberos, the HDFS_TEMPDIR= location is checked before actually assigning the LIBNAME statement. If a valid Kerberos ticket does not exist or the ticket has expired, the LIBNAME connection fail with an error message similar to the following in the SAS® log.   

libname hd hadoop server=server-name hdfs_tempdir='/sastmp';
ERROR: HDFS_TEMPDIR directory /sastmp does not exist or is inaccessible.
ERROR: Error in the LIBNAME statement.

Even though the location specified in the HDFS_TEMPDIR option might physically exist, the error occurs because the LIBNAME statement fails. As a result, the HDFS_TEMPDIR location cannot be accessed.  

The above error message does not indicate which error might have occurred so that information is not documented in the SAS log. Removing the HDFS_TEMPDIR= option from the LIBNAME statement allows the underlying connection error to be written to the SAS log. The following "GSS initiate failed" error indicates that a valid Kerberos ticket does not exist:

libname hd hadoop server=server-name;
ERROR: java.sql.SQLException: Could not open client transport with JDBC Uri:
       jdbc:hive2://drama:10000/default;principal=hive/_HOST@TSDREALM01.COM:
       GSS initiate failed
ERROR: Error trying to establish connection.
ERROR: Error in the LIBNAME statement.

You should assign the Hadoop LIBNAME statement without the HDFS_TEMPDIR= option to verify whether Java Database Connectivity (JDBC) is successful. When the LIBNAME statement is successfully assigned and the HDFS_TEMPDIR= option is necessary to redirect temporary files, add the option to the LIBNAME statement.  

A fix for this issue is planned for a future release.



Operating System and Release Information

Product FamilyProductSystemProduct ReleaseSAS Release
ReportedFixed*ReportedFixed*
SAS SystemSAS/ACCESS Interface to HadoopMicrosoft® Windows® for x649.4_M19.4 TS1M1
Microsoft Windows 109.4_M19.4 TS1M1
Microsoft Windows Server 20089.4_M19.4 TS1M1
Microsoft Windows Server 2012 Datacenter9.4_M19.4 TS1M1
Microsoft Windows Server 2012 Std9.4_M19.4 TS1M1
64-bit Enabled AIX9.4_M19.4 TS1M1
64-bit Enabled Solaris9.4_M19.4 TS1M1
HP-UX IPF9.4_M19.4 TS1M1
Linux for x649.4_M19.4 TS1M1
Solaris for x649.4_M19.4 TS1M1
* For software releases that are not yet generally available, the Fixed Release is the software release in which the problem is planned to be fixed.