Problem Note 63913: Errors occur with the HADOOP procedure or the FILENAME HADOOP statement when you try to access Amazon S3 data
Various errors can appear in your SAS® log when you use PROC HADOOP and the FILENAME HADOOP statement to access S3 data paths. Examples follow here.
You might use the SAS_HADOOP_RESTFUL=1 variable in PROC HADOOP, as shown here:
option set = sas_hadoop_restful=1;
proc hadoop verbose;
hdfs ls='s3a://sas-xxx/temp';
After submitting this procedure, an error that is similar to the following might occur:
ERROR: An exception has been encountered.
Please contact technical support and provide them with the following traceback
information:
The SAS task name is [HADOOP]
ERROR: Read Access Violation HADOOP
Exception occurred at (081FE2D0)
Task Traceback
Address Frame (DBGHELP API Version 4.0 rev 5)
00000000081FE2D0 000000000FE6E630 sashadoo:tkvercn1+0x2D280
00000000081E003C 000000000FE6E6C0 sashadoo:tkvercn1+0xEFEC
00000000081D9B7D 000000000FE6F410 sashadoo:tkvercn1+0x8B2D
00000000081D1721 000000000FE6F710 sashadoo:tkvercn1+0x6D1
00000000090EB1E7 000000000FE6F718 sasxshel:tkvercn1+0x7A197
00000000081FA7B0 000000000FE6F9C0 sashadoo:tkvercn1+0x29760
00000000081D23CC 000000000FE6FB60 sashadoo:tkvercn1+0x137C
0000000006026225 000000000FE6FB68 sashost:Main+0x26F35
0000000006030954 000000000FE6FF50 sashost:Main+0x31664
00007FFACD791FE4 000000000FE6FF58 KERNEL32:BaseThreadInitThunk+0x14
00007FFACF0FCB31 000000000FE6FF88 ntdll:RtlUserThreadStart+0x21
You might submit a FILENAME HADOOP statement like the following:
filename see2 hadoop 's3a://sas-XXX/temp' dir debug;
However, if you add in the SAS_HADOOP_RESTFUL=1 variable, the following errors can occur:
ERROR: The file was not found: s3a://sas-xxx/temp/
ERROR: java.lang.IllegalArgumentException: Wrong FS: s3a://sas-xxx/temp,
expected: hdfs://ip-xxx.sas.internal:8020
ERROR: at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:669)
These errors can occur when the S3 path uses unusual or unexpected path specifications.
Click the Hot Fix tab in this note for a link to instructions about accessing and applying the software update.
Operating System and Release Information
SAS System | Base SAS | Microsoft® Windows® for x64 | 9.4_M6 | | 9.4 TS1M6 | |
Microsoft Windows 8 Enterprise 32-bit | 9.4_M6 | | 9.4 TS1M6 | |
Microsoft Windows 8 Enterprise x64 | 9.4_M6 | | 9.4 TS1M6 | |
Microsoft Windows 8 Pro 32-bit | 9.4_M6 | | 9.4 TS1M6 | |
Microsoft Windows 8 Pro x64 | 9.4_M6 | | 9.4 TS1M6 | |
Microsoft Windows 8.1 Enterprise 32-bit | 9.4_M6 | | 9.4 TS1M6 | |
Microsoft Windows 8.1 Enterprise x64 | 9.4_M6 | | 9.4 TS1M6 | |
Microsoft Windows 8.1 Pro 32-bit | 9.4_M6 | | 9.4 TS1M6 | |
Microsoft Windows 8.1 Pro x64 | 9.4_M6 | | 9.4 TS1M6 | |
Microsoft Windows 10 | 9.4_M6 | | 9.4 TS1M6 | |
Microsoft Windows Server 2012 Datacenter | 9.4_M6 | | 9.4 TS1M6 | |
Microsoft Windows Server 2012 R2 Datacenter | 9.4_M6 | | 9.4 TS1M6 | |
Microsoft Windows Server 2012 R2 Std | 9.4_M6 | | 9.4 TS1M6 | |
Microsoft Windows Server 2012 Std | 9.4_M6 | | 9.4 TS1M6 | |
Microsoft Windows Server 2016 | 9.4_M6 | | 9.4 TS1M6 | |
Microsoft Windows Server 2019 | 9.4_M6 | | 9.4 TS1M6 | |
Windows 7 Enterprise 32 bit | 9.4_M6 | | 9.4 TS1M6 | |
Windows 7 Enterprise x64 | 9.4_M6 | | 9.4 TS1M6 | |
Windows 7 Home Premium 32 bit | 9.4_M6 | | 9.4 TS1M6 | |
Windows 7 Home Premium x64 | 9.4_M6 | | 9.4 TS1M6 | |
Windows 7 Professional 32 bit | 9.4_M6 | | 9.4 TS1M6 | |
Windows 7 Professional x64 | 9.4_M6 | | 9.4 TS1M6 | |
Windows 7 Ultimate 32 bit | 9.4_M6 | | 9.4 TS1M6 | |
Windows 7 Ultimate x64 | 9.4_M6 | | 9.4 TS1M6 | |
64-bit Enabled AIX | 9.4_M6 | | 9.4 TS1M6 | |
64-bit Enabled Solaris | 9.4_M6 | | 9.4 TS1M6 | |
HP-UX IPF | 9.4_M6 | | 9.4 TS1M6 | |
Linux for x64 | 9.4_M6 | | 9.4 TS1M6 | |
Solaris for x64 | 9.4_M6 | | 9.4 TS1M6 | |
*
For software releases that are not yet generally available, the Fixed
Release is the software release in which the problem is planned to be
fixed.
Type: | Problem Note |
Priority: | medium |
Date Modified: | 2019-04-16 11:34:57 |
Date Created: | 2019-03-26 12:52:34 |