Recovering the SAS Metadata Server

About Metadata Server Recovery

You can use SAS Management Console to recover the SAS Metadata Server. The following options are available:
  • You can recover just the metadata repositories and the repository manager, or you can choose to recover the configuration files as well.
  • You can recover from a backup that is listed in the backup history panel, or you can recover from backup files that are stored in an alternate network-accessible location.
  • You can use the roll-forward recovery to apply metadata updates that are stored in the journal file.
The recovery facility provides safeguards to ensure the integrity of the backup files from which you are recovering. The recovery operation checks to make sure that the backup directory contains all of the correct files and that the files have the correct name and file sizes. In addition, each backup file contains a universal unique identifier that is used to make sure that you are recovering files for the correct metadata server. If any problems exist, the recovery is not started and a warning message is displayed.
During recovery operations, the metadata server is automatically paused to a RECOVERY state. This state is similar to an OFFLINE state but more restrictive. External systems such as Windows Services Manager report the server as paused. After the recovery, the metadata server automatically takes a new backup. If the recovery was successful, the server is automatically returned to the state that it was in before the recovery process.
Note: In SAS 9.3, recoveries must be run from SAS Management Console. They cannot be run from a script or a command.

About Roll-Forward Recovery

When recovering the metadata server, you can choose to use roll-forward recovery to update the recovered metadata repository files with transactions from the journal file. Roll-forward recovery uses the journal file that is located in the same directory as the backup files. The updates that are applied include additions, modifications, and deletions of metadata, as well as additions and deletions of metadata repositories. You can choose to apply all transactions that are in the journal file, or you can apply all transactions through a specified date and time.
Roll-forward recovery can be used only if the JOURNALTYPE option of the metadata server configuration file (omaconfig.xml) was set to ROLL_FORWARD at the time that the metadata server updates were made. (The ROLL_FORWARD option is enabled automatically by the SAS Deployment Wizard.)
CAUTION:
If you recover the metadata server without selecting the roll-forward option, then any update or delete transactions that occurred after the backup was taken will be lost. In addition, any repositories that were after the backup was taken will be deleted.

Executing a Metadata Server Recovery

To recover the SAS metadata server from a backup, either with or without roll-forward recovery, follow these steps:
  1. Log on to SAS Management Console as an unrestricted user, a user who is in the SAS Administrators group, or a user who is a member of the Metadata Server: Operation role. On the Plug-ins tab, navigate to Environment Managementthen selectMetadata Managerthen selectMetadata Utilitiesthen selectServer Backup.
  2. As a best practice, it is recommended that you pause the server to the ADMINISTRATION state.
    If you start the recovery while the server in the ADMINISTRATION state, the server changes the state to RECOVERY, performs the recovery, and then returns the server to the ADMINISTRATION state. You can then verify system readiness before clients reconnect. When readiness has been verified, you can manually resume the server.
    If you start the recovery while the server is ONLINE, the recovery process pauses the server to the RECOVERY state and then returns it to the ONLINE state after a successful recovery.
  3. Use one of the following methods to start the recovery process:
    • If you want to recover using backup files that are in the configured backup location, follow these steps:
      1. On the backup history list in the right panel, find the backup from which you want to recover. To determine its status, either double-click the backup entry or right-click and select Properties. If the files are usable for a recovery, the Verification field of the Properties dialog box displays this message: All files in this backup are currently correct.
        You can also review the detailed information about the backup on the General and Repositories tabs.
      2. When you are sure that you have identified the correct backup, right-click the backup entry in the backup history and select Recover from This Backup.
    • If you want to recover from a different location (for example, the location that contains your system backups), right-click the Server Backup node (located on the Plug-ins tab under Metadata Utilities), and select Recover from Alternate Location. In the Recover from Alternate Location dialog box, enter the complete pathname for the directory that contains the backup. You can specify any network location that is accessible from the metadata server. If the backups are remote to the metadata server, then specify a UNC path, as in the following example:
      \\D2345\C$\SASMetadataServer\2011-01-14T01_00_01-05_00
      Note: If your metadata server is installed on a Windows host, do not specify a mapped drive.
  4. After you select Recover from This Backup (or after you enter a pathname in the Recover from Alternate Location dialog box and click OK), the server accesses the backup directory and attempts to validate the backup files. It checks to make sure that all of the backup files are present and that they have the correct filenames, file sizes, and universal unique identifier.
    Note: If the validation is not successful, one of the following messages is displayed:
    • A Recover cannot be run because this Backup is offline and not available. If this message appears, you will need to recover from a different set of backup files.
    • This Backup does not match what was backed up. Are you sure you want to recover from this Backup? If this message appears, select Details to determine which files in the backup directory are incorrect.
      If the configuration file backups are incorrect but the repository backups are correct, you can click OK to resume the recovery. However, make sure that you have not selected the option to Also recover server configuration files.
      If the repository backups are incorrect, click Cancel and select a different backup from which to recover.
    If the validation is successful (or if you resumed the recovery after receiving an error message), the Recover from This Backup dialog box appears:
    Recover from This Backup dialog box
  5. In the first text field of the Recover from This Backup dialog box, enter a comment that will appear in the backup history panel with this recovery. For example, you might want to state the reason that the server is being recovered.
  6. In the second text field, enter a comment that will be displayed to users while the server is paused during the recovery.
  7. If appropriate, select the check boxes for the following options:
    Also recover server configuration files
    This option does the following:
    • deletes all files from the server’s configuration directory (for example, SAS-configuration-directory/Lev1/SASMeta/MetadataServer )
    • from the CongurationFiles subdirectory in the backup location, copies all files (with the exception of MetadataServerBackupConfiguration.xml, MetadataServerBackupHistory.xml, and MetadataServerRecoveryManifest.xml) to the server’s configuration directory
    CAUTION:
    If you select “Also recover server configuration files,” you will lose any additions or modifications to files in the MetadataServer directory that have been made since the backup was taken.
    As a best practice, you should select this option only if your configuration files have become corrupted.
    Reorganize recovered repositories
    This option reorganizes the metadata repositories when they are backed up following the recovery. The reorganization reclaims any unused disk space that is left from previously deleted metadata objects. See About the Reorganize Repositories Backup Option.
    Roll forward transactions
    In addition to restoring the repository files from the backup, this option recovers transactions that were recorded in the metadata server journal file after the backup was created. If you select this option, the following additional options become available:
    All transactions in journal file
    This option recovers all transactions that are recorded in the journal file. The time period that is covered by the journal file is displayed on the right side of the dialog box in server local time. (If you want to display the time period in Greenwich Mean Time (GMT), select Show time values in GMT.)
    Transactions through date/time
    This option recovers all transactions through a date and time that you specify. This option is useful if you want to recover the server to the state that it was in just before a specific problematic transaction. You can use the metadata server log to determine the exact time that the transaction occurred. See Use the Log Tab in the Server Manager.
    Enter the roll-forward ending date and time in the format yyyy-mm-ddThh:mm:ss. You can either edit the default values that are displayed in the field, or you can cut and paste the date and time from the appropriate event in the metadata server log. Most log events use server local time.
    The dialog box enables you to specify the time in either server local time or GMT, depending on which of the following radio buttons is selected:
    • Show time values in server local time displays the journal file’s default starting and ending time in server local time. You can override the ending date and time. Be sure to enter the time in server local time if this radio button is selected.
    • Show time values in GMT displays the journal file’s default starting and ending times in Greenwich Mean Time. You can override the ending date and time. Be sure to enter the time in GMT if this radio button is selected.
    CAUTION:
    The preceding radio buttons do not convert your entry to server local time or to GMT. If you select or deselect one of these radio buttons after entering a date and time, your entry remains unchanged.
  8. Before starting the recovery operation, the server pauses itself to a RECOVERY state. This state is similar to an OFFLINE state but more restrictive.
    Note: For example, the RECOVERY state prevents clients from obtaining a list of registered repositories. When the server is in the RECOVERY state, external systems such as Windows Services Manager report the server as paused.
  9. When the recovery is completed, a backup of the newly recovered server is taken automatically, regardless of whether the recovery completed successfully.
    When the post-recovery backup is completed, all previous backups are retained (that is, no backups are deleted based on the retention policy).
  10. If the recovery completed without errors, the server is returned to the state that it was in prior to the recovery process.
    If a recovery fails, the server is placed in the ADMINISTRATION state. The administrator can then access SAS Management Console and try to resolve the errors or recover the server from a different backup.

Troubleshooting Metadata Server Recovery Jobs

If a recovery does not finish successfully, an e-mail with the subject line Metadata Server Recovery Failed is sent to the designated recipients for metadata server alerts. The e-mail specifies the server’s host name, the path to the server’s configuration directory, and the recovery name. For details about the failure, refer to the metadata server log and search for the recovery name. The log is typically located in the Logs subdirectory of the server’s configuration directory.
When a recovery fails, an “x” icon (Invalid backup icon: White “x” in red circle) also appears in the Status column of the backup and recovery history panel in SAS Management Console. To determine why the recovery failed, right-click the recovery and select Properties. On the Properties dialog box, the Error Summary field provides high-level information to help resolve the problem. If more details are available, then the Full Report button is activated. Click this button to obtain a text report that you can browse, copy, and paste.
The ConfigurationFiles subdirectory of each backup contains files called MetadataServerBackupHistory.xml, MetadataServerBackupConfiguration.xml and MetadataServerRecoveryManifest.xml. These files might be useful to SAS Technical Support if you require troubleshooting assistance.
Note: In a Windows environment, the recovery operation cannot update a READONLY file. If you have manually set any of the metadata server files to READONLY, recovery errors will be reported. In this situation, the recovery might still be valid if no changes have been made to the READONLY file since the backup was taken.