Problem Note 63200: Importing a CSV file with UTF-16LE encoding and TERMSTR=CRLF can result in data loss
 |  |  |  |  |
In certain, rare cases, data loss can occur after you attempt to convert a CSV file with UTF-16LE encoding into a SAS data set. This issue is observed when either you use the Import Wizard in SAS® Foundation or you submit a DATA step with an INFILE statement that contains a TERMSTR=CRLF argument.
The issue can occur because the 0x000d 0x000a record-terminator sequence in the CSV file is seen as part of the record. If that happens, there is a problem reading the rest of the file.
If you encounter this issue, there are two workarounds:
- Use the Import Wizard in SAS® Enterprise Guide® to read the CSV file. It does not rely on the same underlying I/O routine that causes this issue.
- If the CSV file is a Microsoft Excel file, resave the file with UTF-8 encoding. The problem is limited to cases in which the length of the record-terminator sequence is greater than one byte. With UTF‑8 encoding, both the carriage return (CR) and line feed (LF) values require only a single byte to encode.
Click the Hot Fix tab in this note to access the hot fix for this issue.
Operating System and Release Information
| Product Family | Product | System | SAS Release | |
| Reported | Fixed* | |||
| SAS System | N/A | z/OS | 9.4 TS1M3 | |
| z/OS 64-bit | 9.4 TS1M3 | |||
| Android Operating System | 9.4 TS1M3 | |||
| Apple Mobile Operating System | 9.4 TS1M3 | |||
| Macintosh | 9.4 TS1M3 | |||
| Microsoft Windows 10 | 9.4 TS1M3 | |||
| Microsoft Windows 8 Enterprise 32-bit | 9.4 TS1M3 | |||
| Microsoft Windows 8 Enterprise x64 | 9.4 TS1M3 | |||
| Microsoft Windows 8 Pro 32-bit | 9.4 TS1M3 | |||
| Microsoft Windows 8 Pro x64 | 9.4 TS1M3 | |||
| Microsoft Windows Server 2008 R2 | 9.4 TS1M3 | |||
| Microsoft Windows Server 2012 R2 Datacenter | 9.4 TS1M3 | |||
| Microsoft Windows Server 2012 R2 Std | 9.4 TS1M3 | |||
| Microsoft® Windows® for 64-Bit Itanium-based Systems | 9.4 TS1M3 | |||
| Microsoft® Windows® for x64 | 9.4 TS1M3 | |||
| Microsoft Windows 8.1 Enterprise 32-bit | 9.4 TS1M3 | |||
| Microsoft Windows 8.1 Enterprise x64 | 9.4 TS1M3 | |||
| Microsoft Windows 8.1 Pro 32-bit | 9.4 TS1M3 | |||
| Microsoft Windows 8.1 Pro x64 | 9.4 TS1M3 | |||
| Microsoft Windows Server 2008 | 9.4 TS1M3 | |||
| Microsoft Windows Server 2008 for x64 | 9.4 TS1M3 | |||
| Microsoft Windows Server 2012 Datacenter | 9.4 TS1M3 | |||
| Microsoft Windows Server 2012 Std | 9.4 TS1M3 | |||
| Windows 7 Enterprise 32 bit | 9.4 TS1M3 | |||
| Windows 7 Enterprise x64 | 9.4 TS1M3 | |||
| Windows 7 Home Premium 32 bit | 9.4 TS1M3 | |||
| Windows 7 Home Premium x64 | 9.4 TS1M3 | |||
| Windows 7 Professional 32 bit | 9.4 TS1M3 | |||
| Windows 7 Professional x64 | 9.4 TS1M3 | |||
| Windows 7 Ultimate 32 bit | 9.4 TS1M3 | |||
| Windows 7 Ultimate x64 | 9.4 TS1M3 | |||
| 64-bit Enabled AIX | 9.4 TS1M3 | |||
| 64-bit Enabled HP-UX | 9.4 TS1M3 | |||
| 64-bit Enabled Solaris | 9.4 TS1M3 | |||
| HP-UX | 9.4 TS1M3 | |||
| HP-UX IPF | 9.4 TS1M3 | |||
| Linux | 9.4 TS1M3 | |||
| Linux for x64 | 9.4 TS1M3 | |||
| Solaris for x64 | 9.4 TS1M3 | |||
A fix for this issue for Base SAS 9.4_M6 is available at:
https://tshf.sas.com/techsup/download/hotfix/HF2/D9T.html#63200A fix for this issue for Base SAS 9.4_M3 is available at:
https://tshf.sas.com/techsup/download/hotfix/HF2/V01.html#63200
In rare instances, records are missing after a CSV file is read and the file encoding is UTF-16LE.
| Type: | Problem Note |
| Priority: | high |
| Date Modified: | 2019-02-07 12:34:24 |
| Date Created: | 2018-11-08 19:15:05 |