The possibility exists that the SAS® Metadata Server journal file might become corrupted during the import process of an *.spk file. The problem might occur even with relatively small import packages in the size range of ~30 MB range.
The *.spk files are created and used by the Import/Export SAS Package promotion tools, which are available in SAS® Management Console as well as from the command-line as batch scripts. For details about these tools, see the Introduction to the Promotion Tools documentation.
The import process fails and returns the following messages to the SAS Metadata Server log file:
WARN [00000004] :sas - *** FOMSjnlfileRead: Asked to skip -951887848 (0xc7435818) uncompressed bytes of compressed data, but -951887848 (0xc7435818) bytes of uncompressed data were compressed and written at file position 72854909 (0x457ad7d) . Will skip what was written ***
ERROR [00000004] :sas - Journal file check discovered that start reader (0xffffffffcb9b05a5) < header reserve (0x400).
ERROR [00000004] :sas - The metadata journal is corrupted, please contact Technical Support for help.
ERROR [00000004] :sas - CRITICAL ERROR encountered updating journal file header.
ERROR [00000004] :sas - Crash recovery may not be possible.
ERROR [00000004] :sas - OMS Journal commit task is now terminating as a result of previously logged errors.
ERROR [00000004] :sas - The error that caused the failure is reproduced on the following line.
ERROR [00000004] :sas - The metadata journal is corrupted, please contact Technical Support for help.
ERROR [00000004] :sas - You should determine the cause, fix it, then stop and restart the server as soon as possible.
ERROR [00000004] :sas - When the server is restarted it should recover all journalled update transactions.
ERROR [00000004] :sas - An ALERT EMAIL was sent to 'sas-administrator@mycompany.com'.
ERROR [00112147] 1623:a12345 - Journal file check discovered that start reader (0xffffffffcb9b05a5) < header reserve (0x400).
ERROR [00112147] 1623:a12345 - The metadata journal is corrupted, please contact Technical Support for help.
ERROR [00112147] 1623:a12345 - CRITICAL ERROR encountered writing to journal file.
ERROR [00112147] 1623:a12345 - Crash recovery may not be possible.
ERROR [00112147] 1623:a12345 - Workunit commit that can not be written to disk repositories.
ERROR [00112147] 1623:a12345 - The error which caused the commit to fail is: 807fed35
ERROR [00112147] 1623:a12345 - The metadata journal is corrupted, please contact Technical Support for help.
ERROR [00112147] 1623:a12345 - Emergency situation requires we discard all in memory data which may have been affected.
ERROR [00112147] 1623:a12345 - *** FOMSworkunitCommit: Close and unload container #1 A36GEPS0.event ***
When you try to restart the SAS Metadata Server, it fails to start and the following messages are written to the log file:
INFO [00000008] :sas - Initializing journal file: Journal/2014-11-12T01_00_00+01_00_Journal.dat
INFO [00000008] :sas - Using the TKR compression algorithm for the journal file.
WARN [00000008] :sas - There were 3343166288 bytes of pending updates in the journal file.
INFO [00000008] :sas - Initial journal file statistics:
INFO [00000008] :sas - Journal file statistics:
INFO [00000008] :sas - queue length: 0
INFO [00000008] :sas - space available: 4611686015084220592
INFO [00000008] :sas - data available: 3343166288
INFO [00000008] :sas - data available maximum: 3343166288
INFO [00000008] :sas - data written: 3.18142 GB
INFO [00000008] :sas - maximum file entry size: 69761292
INFO [00000008] :sas - decompressed data available: -951335160
INFO [00000008] :sas - decompressed data available maximum: 1807655248
INFO [00000008] :sas - decompressed data written: 0.848199 GB
INFO [00000008] :sas - maximum decompressed entry size: 1807261040
INFO [00000008] :sas - current journal entry number: 11840
INFO [00000008] :sas - journal entry GUID: 336A77D9-B727-BE45-B3B5-AB6E6BBC016F
INFO [00000008] :sas - journal entry counter: 11841
INFO [00000008] :sas - WARNING: Insufficient memory available for journal file buffer.
INFO [00000008] :sas - -951887848 bytes are needed.
INFO [00000008] :sas - Journal Reload complete. 0 entries with entry numbers from 0 to 0.
INFO [00000008] :sas - The journal could not be fully reloaded because of errors noted above.
INFO [00000008] :sas - Correct any conditions that you can and attempt to start the server again.
ERROR [00000008] :sas - Journal reload has failed.
INFO [00000008] :sas - Crash recovery reload has failed.
ERROR [00000008] :sas - Insufficient memory
ERROR [00000008] :sas - An error occurred while attempting to Recover the Journal File after I/O failures.
ERROR [00000008] :sas - ALERT EMAIL includes the previous message.
ERROR [00000008] :sas - An ALERT EMAIL was sent to 'sas-administrator@mycompany.com'.
WARN [00000008] :sas - /sas/wky/mva-v940/tkmeta/src/tkomsBackupmf.c: Close was called for manifest file Backups/MetadataServerBackupHistory.xml but it was not open.
WARN [00000008] :sas - /sas/wky/mva-v940/tkmeta/src/tkomsBackupmf.c: Close was called for manifest file Backups/MetadataServerBackupConfiguration.xml but it was not open.
The SAS Metadata Server Import log shows messages similar to these:
INFO - ******** Importing Metadata ********
INFO - Including access controls during import.
INFO - Metadata imported successfully.
INFO - ******** Importing Properties ********
ERROR - Error while applying substitution properties to <object name>.
ERROR - Unexpected substitution exception
ERROR - The metadata journal is corrupted, please contact Technical Support for help.
ERROR - The metadata journal is corrupted, please contact Technical Support for help.
In most cases, UNIX systems are more susceptible to the problem than Microsoft Windows systems. This difference is a result of how each operating system stores threaded-kernel characters (TKChars). Under Windows, 2 bytes are used, whereas under UNIX, 4 bytes are used.
To circumvent the problem, perform the import process with smaller *.spk files or select a subset of objects to import from the larger *.spk files.
Hot Fix Details
Click the Hot Fix tab in this note to access the hot fix for this issue.
For the first and second maintenance releases of SAS 9.4 (TS1M1 and TS1M2), the hot fix detects that the import process might result in a corrupted journal file. It is important to note that the hot fix does not allow the large import to succeed. Even with the hot fix applied, it is necessary to retry the import process again using a smaller *.spk file or with a smaller subset of objects to be imported.
The import process ends prematurely and writes the following messages to the log file. This action prevents corruption of the journal file and enables normal operation of the metadata server:
DEBUG [00002197] 2:sasadm@saspw - ^^^^^^^^^^^^^^^ Method ReturnList [Size=23338K] ^^^^^^^^^^^^^^^
DEBUG [00002197] 2:sasadm@saspw - ====IP:090======== Terminate =================
DEBUG [00002197] 2:sasadm@saspw - ====IP:190======== Terminate RI =================
DEBUG [00002197] 2:sasadm@saspw - ====IP:191======== Terminate CM =================
DEBUG [00002197] 2:sasadm@saspw - ====IP:099======== End Import =================
ERROR [00002197] 2:sasadm@saspw - CRITICAL ERROR journal entry of size 3310194480 (0xc54d8f30) will exceed 2GB limit for journal compression.
INFO [00002197] 2:sasadm@saspw - ImportMetadata return code=807fed34....
ERROR [00002197] 2:sasadm@saspw - The transaction could not be written to the journal file.
DEBUG [00002197] 2:sasadm@saspw - 2:SASADM@SASPW: PutCookie: Task CONTEXT 0000081B returning cookie number 15.
Operating System and Release Information
| SAS System | SAS Metadata Server | Solaris for x64 | 9.4_M1 | 9.4_M3 | 9.4 TS1M1 | 9.4 TS1M3 |
| Linux for x64 | 9.4_M1 | 9.4_M3 | 9.4 TS1M1 | 9.4 TS1M3 |
| HP-UX IPF | 9.4_M1 | 9.4_M3 | 9.4 TS1M1 | 9.4 TS1M3 |
| 64-bit Enabled Solaris | 9.4_M1 | 9.4_M3 | 9.4 TS1M1 | 9.4 TS1M3 |
| 64-bit Enabled AIX | 9.4_M1 | 9.4_M3 | 9.4 TS1M1 | 9.4 TS1M3 |
*
For software releases that are not yet generally available, the Fixed
Release is the software release in which the problem is planned to be
fixed.
