Usage Note 7018: Exercise caution using ENCODING= option with mixed character and binary data
The ENCODING= option enables you to dynamically change the
character-set encoding for processing external data. This allows you
to read or write data in a different encoding than your operating
environment's session encoding.
Use caution when an external file contains a mix of character and
binary data (such as packed decimal or integer binary) that has been
created on a non-native system, as the ENCODING= option trancodes the
entire file. The character data trancodes correctly but the binary
data is corrupted.
For example, consider a file created on an IBM mainframe which has
both character data and packed decimal data in EBCDIC. A Windows
system is used to read the file. The ENCODING= option is specified to
change the character-set encoding to match the file's encoding, which
is different than the session encoding. The ENCODING= option
trancodes the entire file. The character data is trancoded and read
in correctly. The packed decimal data is trancoded and is corrupted.
This situation can also cause the following error message(s):
FATAL: Wide character FILE/INFILE/FILENAME options have been
specified. Wide character support is not currently enabled.
OR
NOTE: Invalid data for ... in line ...
The correct way to process this type of file is to read the file
twice, once with the ENCODING= option to read the character data, and
then a second time without the ENCODING= option to read the packed
decimal data.
Alternatively, you can leave out the ENCODING= option on the FILENAME statement and transcode each character variable with the KCVT function. For example: CVAR = KCVT(CVAR,'UTF-8');
Or, if the character data is in Unicode format, you can use an appropriate Unicode informat for reading the data.
If problems still occur, another thing to check is how the file was transferred from one operating system to another. It is possible the file was converted from EBCDIC to ASCII or vice versa which may have corrupted the data during the transfer. Use a binary mode to transfer the file so that the data remains in the native form of the operating system it was created on.
Operating System and Release Information
| SAS System | Base SAS | Windows Millennium Edition (Me) | 8.2 TS2M0 | |
| Microsoft Windows 95/98 | 8.2 TS2M0 | |
| Microsoft Windows NT Workstation | 8.2 TS2M0 | |
| Microsoft Windows 2000 Server | 8.2 TS2M0 | |
| Microsoft Windows 2000 Professional | 8.2 TS2M0 | |
| Microsoft Windows 2000 Datacenter Server | 8.2 TS2M0 | |
| Microsoft Windows 2000 Advanced Server | 8.2 TS2M0 | |
| Solaris | 8.2 TS2M0 | |
| OpenVMS VAX | 8.2 TS2M0 | |
| 64-bit Enabled Solaris | 8.2 TS2M0 | |
| z/OS | 8.2 TS2M0 | |
| IRIX | 8.2 TS2M0 | |
| OS/2 | 8.2 TS2M0 | |
| Linux | 8.2 TS2M0 | |
| 64-bit Enabled HP-UX | 8.2 TS2M0 | |
| HP-UX | 8.2 TS2M0 | |
| ABI+ for Intel Architecture | 8.2 TS2M0 | |
| CMS | 8.2 TS2M0 | |
| OpenVMS Alpha | 8.2 TS2M0 | |
| 64-bit Enabled AIX | 8.2 TS2M0 | |
| Tru64 UNIX | 8.2 TS2M0 | |
| AIX | 8.2 TS2M0 | |
| Microsoft Windows XP Professional | 8.2 TS2M0 | |
*
For software releases that are not yet generally available, the Fixed
Release is the software release in which the problem is planned to be
fixed.
| Type: | Usage Note |
| Priority: | |
| Topic: | SAS Reference ==> Procedures ==> OPTIONS SAS Reference ==> System Options ==> ENCODING Common Programming Tasks ==> Reading and Writing External Data ==> File Options Common Programming Tasks ==> Reading and Writing External Data
|
| Date Modified: | 2009-04-28 12:38:37 |
| Date Created: | 2002-02-28 13:11:53 |