Advanced Topics for the IMS DATA Step Interface |
Building Synchronization Points |
There is always a risk of abnormal termination in any program. If an update program ends before processing is completed, you can complete processing by restarting the program, but you do not want to repeat updates that have already been made. The synchronization point feature of DL/I helps to prevent duplicate updating in a restarted program.
If an online access region program or control region abends, the DL/I control region restores databases up to the last synchronization point. In a batch subsystem, a batch back-out utility must be executed to back out updates made since the last synchronization point. After backing out updates, any updates made by the program before the last synchronization point are intact and any made after the last synchronization point are not. When an update program is restarted after an abend, processing must resume at the synchronization point or duplicate updating might occur.
When building synchronization points into an online access region program, keep these things in mind:
If the program updates a large number of database records between synchronization points, the DL/I control region enqueue tables can overflow and cause the online DL/I system to abend.
The DL/I control region dynamic log can also overflow, which can cause the online access region or the whole online system to abend, depending on the online system used.
On the other hand, if synchronization points are too frequent, they can tie up the master console and prevent other IMS messages from being sent.
Your database administration staff can help you determine how frequently synchronization points should be executed.
Example 1: Updating a Database |
This sample program updates the ACCTDBD database with data from wire transactions in the WIRETRN database. (See Defining SAS/ACCESS Descriptor Files for complete database information about the WIRETRN database.) The program takes checkpoints and thereby releases database resources at regular intervals. Because the program is set up with checkpoints, it is appropriate for shared update access.
As you study this example, you will notice that the WIRETRAN segments are deleted from the WIRETRN database as soon as the ACCTDBD segments are successfully updated. There are no synchronization points between the ACCTDBD segment updates and the WIRETRAN deletions. Therefore, if an abend occurs and changes are backed out to the last synchronization point, you know that any WIRETRAN segments remaining in the database have not been processed. There is no danger of duplicating updates, and the program is inherently restartable. No special recovery logic is required for restarts.
The numbered comments following this program correspond to the numbered statements in the program:
data _null_; length ssa1 $ 43 ssa2 $ 32 ssa3 $ 9; retain blanks ' ' wirenum 0 chkpnum 0; 1 infile acctsam dli ssa=(ssa1,ssa2,ssa3) call=func pcb=pcbindex status=st segment=seg; /* get hold next WIRETRAN segment from WIRETRN database */ func = 'GHN '; ssa1 = ' '; ssa2 = ' '; ssa3 = ' '; 2 pcbindex = 5; 3 input @1 wiressn $char11. @12 wireacct $char12. @24 accttype $char1. @25 wiredate mmddyy8. @33 wiretime time8. @41 wireammt pd5.2 @46 wiredesc $char40.; if st ¬= ' ' then if st = 'GB' then do; _error_ = 0; go to reptotal; end; else link abendit; 4 if wirenum/5 = chkpnum then link chkp; 5 amount = abs(wireammt); /* insert debit or credit segment into ACCTDBD database */ 6 if accttype = 'C' then do; ssa2 = 'CHCKACCT (ACNUMBER= '|| wireacct ||')'; if wireammt > 0 then ssa3 = 'CHCKCRDT'; else ssa3 = 'CHCKDEBT'; end; else 7 if accttype = 'S' then do; ssa2 = 'SAVEACCT (ACNUMBER= ' || wireacct || ')'; if wireammt > 0 then ssa3 = 'SAVECRDT'; else ssa3 = 'SAVEDEBT'; end; 8 else do; file log; put / '***** Invalid ' accttype= 'for ' wiressn= wireacct= '*****'; return; end; 9 ssa1 = 'CUSTOMER (SSNUMBER= ' || wiressn || ')'; func = 'ISRT'; pcbindex = 4; file acctsam dli; 10 put @1 amount pd5.2 @6 wiredate mmddyy6. @14 wiretime time8. @22 wiredesc $char40. @62 blanks $char19.; 11 if st ¬= ' ' then if st = 'GE' then do; _error_ = 0; file log; if seg = 'CUSTOMER' then if accttype = 'C' then put / '***** No CHCKACCT segment with ' wiressn= wireacct= '*****'; else put / '***** No SAVEACCT segment with ' wiressn= wireacct= '*****'; else put / '***** No CUSTOMER segment with ' wiressn= '*****'; return; end; else link abendit; /* get hold checking or savings segment from ACCTDBD database */ 12 ssa3 = ' '; func = 'GHU'; input @1 acnumber $char12. @13 balance pd5.2 @18 stmtdate mmddyy6. @26 stmt_bal pd5.2; 13 if st ¬= ' ' then link abendit; /* replace checking or savings segment into ACCTDBD database */ balance = balance + wireammt; ssa1 = ' '; ssa2 = ' '; func = 'REPL'; put @1 acnumber $char12. @13 balance pd5.2 @18 stmtdate mmddyy6. @26 stmt_bal pd5.2; if st ¬= ' ' then link abendit; /* delete WIRETRAN segment from WIRETRN database */ 14 func = 'DLET'; ssa1 = ' '; pcbindex = 5; put @1 wiressn $char11. @12 wireacct $char12. @24 accttype $char1. @25 wiredate mmddyy8. @33 wiretime time8. @41 wireammt pd5.2 @46 wiredesc $char40.; 15 if st ¬= ' ' then link abendit; 16 wirenum +1; return; 17 reptotal: file log; put // 'Number of Wire Transactions Posted =' wirenum 5. / ' Number of CHKP Calls Issued =' chkpnum 5.; stop; 18 chkp: chkpnum +1; func = 'CHKP'; pcbindex = 1; file acctsam dli; put @1 'SAS' @4 chkpnum z5.; if st ¬= ' ' then link abendit; func = 'GHU '; ssa1 = 'WIRETRAN (SSNACCT = ' || wiressn || wireacct || ')'; pcbindex = 5; input; if st ¬= ' ' then link abendit; return; 19 abendit: file log; put _all_; abort; run;
Example 2: Incorrectly Updating a Database without Recovery Logic |
Unless a program is structured so that it can be restarted without duplicating updates, special recovery logic should be included. The previous example shows a data program designed so that it can be restarted if necessary. The following example is not designed to be restarted and does not include special recovery logic. We include it as an example of the kind of program that should not be used for updating in a shared environment because it could result in erroneous data.
This program updates the ACCTDBD database with wire transactions that are stored in a sequential file rather than in the WIRETRN database. The program is similar to Example 1: Updating a Database, but it is not designed to be restarted. Example program 3 illustrates the modifications to this program to add recovery logic.
The numbered comments following this sample program correspond to the numbered statements in the example:
filename tranin '<your.sas.tranin>' disp=shr; data _null_; length ssa1 $31 ssa2 $32 ssa3 $9; retain blanks ' ' wirenum 0 chkpnum 0; /* get data from TRANIN flatfile */ 1 infile tranin eof=reptotal; input @1 cust_ssn $char11. @12 acct_num $char12. @24 accttype $char1. @25 wiredate mmddyy8. @33 wiretime time8. @41 wireammt pd5.2 @46 wiredesc $char40.; if _error_ then link abendit; 2 if wirenum/5 = chkpnum then link chkp; 3 amount = abs(wireammt); 4 if accttype = 'C' then do; ssa2 = 'CHCKACCT (ACNUMBER =' || acct_num || ')'; if wireammt < 0 then ssa3 = 'CHCKCRDT'; else ssa3 = 'CHCKDEBT'; end; else if accttype = 'S' then do; ssa2 = 'SAVEACCT (ACNUMBER =' || acct_num || ')'; if wireammt < 0 then ssa3 = 'SAVECRDT'; else ssa3 = 'SAVEDEBT'; end; else do; file log; put / '***** Invalid ' accttype= 'for ' cust_ssn= acct_num= '*****'; return; end; /* insert debit or credit segment into ACCTDBD database */ 5 infile acctsam dli ssa=(ssa1,ssa2,ssa3) call=func pcb=pcbindex status=st segment=seg; ssa1 = 'CUSTOMER(SSNUMBER =' || CUST_SSN || ')'; func = 'ISRT'; pcbindex = 4; file acctsam dli; put @1 amount pd5.2 @6 wiredate mmddyy6. @14 wiretime time8. @22 wiredesc $char40. @62 blanks $char19.; 6 if st ¬= ' ' then if st = 'GE' then do; _error_ = 0; file log; if seg = 'CUSTOMER' then if accttype = 'C' then put / '***** No CHCKACCT segment with ' cust_ssn= acct_num= ' *****'; else put / '***** No SAVEACCT segment with ' cust_ssn= acct_num= ' *****'; else put / '***** No CUSTOMER segment with ' cust_ssn= '*****'; return; end; else link abendit; /* get hold checking or savings segment from ACCTDBD database */ ssa3 = ' '; 7 func = 'GHU'; input @1 acnumber $char12. @13 balance pd5.2 @18 stmtdate mmddyy6. @26 stmt_bal pd5.2; if st ¬= ' ' then link abendit; balance = balance + wireammt; /* replace checking or savings segment into ACCTDBD database */ ssa1 = ' '; ssa2 = ' '; func = 'REPL'; 8 put @1 acnumber $char12. @13 balance pd5.2 @18 stmtdate mmddyy6. @26 stmt_bal pd5.2; if st ¬= ' ' then link abendit; 9 if wireammt > 0 then debtnum +1; else crdtnum +1; 10 wirenum +1; return; reptotal: file log; put // 'Number of debit transactions posted =' debtnum 8. / 'Number of credit transactions posted =' crdtnum 8.; stop; 11 chkp: chkpnum +1; func = 'CHKP'; pcbindex = 1; file acctsam dli; put @1 'SAS' @4 chkpnum z5.; if st ¬= ' ' then link abendit; return; abendit: file log; put _all_; abort; run; filename tranin clear;
The standard INFILE statement specifies the external sequential file containing the data to update ACCTDBD. The fileref is TRANIN. When the end-of-file condition is set, the program branches to the REPTOTAL subroutine to print a summary report. The standard INPUT statement reads a record from TRANIN. If any error occurs, the program links to the ABENDIT subroutine. | |
As in the previous example, this program issues CHKP calls after every fifth update. If the value of WIRENUM divided by five is equal to the value of CHKPNUM, the program links to a section that issues the CHKP call. | |
The DATA step sets up for the REPL call that will update balance information in the CHCKACCT and SAVEACCT segments of the ACCTDBD database. The absolute value of WIREAMMT is saved. | |
Depending on the value of ACCTTYPE, SSAs are built for the CHCKACCT and either the CHCKDEBT or CHCKCRDT segments, or for the SAVEACCT and either the SAVEDEBT or SAVECRDT segments. | |
The DL/I INFILE statement specifies the ACCTSAM PSB. An ISRT call for the ACCTDBD database is formatted and issued. Depending on the account type and transaction type, a new CHCKCRDT, CHCKDEBT, SAVECRDT, or SAVEDEBT segment is inserted. | |
This section checks status codes and prints explanatory messages on the SAS log if the status code is GE (segment not found). For other non-blank status codes, the program links to the ABENDIT subroutine. | |
If the ISRT call is successful, a GHU call is issued to retrieve the parent of the added segment. The status code is checked after the call and, if it is not successful, the program links to the ABENDIT routine. | |
If the GHU call is successful, the account balance is updated by a REPL call. The status code is checked after the call and, if it is not successful, the program links to the ABENDIT routine. | |
Accumulator variables count the number of debits and credits posted by the program. These values are used to print a summary report. | |
The WIRENUM variable is incremented. It is used to determine whether or not a CHKP call is needed (see item 2). | |
This section is like the one in Example 1: Updating a Database, but no GHU call is issued to re-establish database position because there is no database position to maintain. (This is because the wire transactions are not coming from an IMS database on which the program can reposition.) |
Example 3: Correctly Updating a Database with Recovery Logic |
This example is a modified version of Example 2: Incorrectly Updating a Database without Recovery Logic. The modifications consist of the recovery logic added to enable the program to be restarted. The same sequential file is used to update the ACCTDBD database.
The numbered comments following this program describe the statements added to enable a restart:
filename tranin '<your.sas.tranin>' disp=shr; 1 filename restart '<your.sas.restart>' disp=shr; data _null_; length ssa1 $31 ssa2 $32 ssa3 $9 chkpnum 5; retain wireskip wirenum 0 chkpnum 0 first 1 debtnum crdtnum errnum 0 blanks ' '; infile restart eof=process; input @1 chkpid 5. @6 chkptime datetime13. @19 chkdebt 8. @27 chkcrdt 8. @35 chkerr 8.; wireskip = chkdebt + chkcrdt + chkerr; file log; put 'Restarting from checkpoint ' chkpid 'taken at ' chkptime datetime13. ' to bypass ' wireskip 'trans already processed'; do while(wireread < wireskip); infile tranin; input @1 cust_ssn $char11. @12 acct_num $char12. @24 accttype $char1. @25 wiredate mmddyy8. @33 wiretime time8. @41 wireammt pd5.2 @46 wiredesc $char40.; wireread + 1; end; debtnum = chkdebt; crdtnum = chkcrdt; wirenum = debtnum + crdtnum; errnum = chkerr; 2 process: infile tranin eof=reptotal; input @1 cust_ssn $char11. @12 acct_num $char12. @24 accttype $char1. @25 wiredate mmddyy8. @33 wiretime time8. @41 wireammt pd5.2 @46 wiredesc $char40.; if _error_ then link abendit; if wirenum/5 = chkpnum or first =1 then do; link chkp; first =0; end; amount = abs(wireammt); if accttype = 'C' then do; ssa2 = 'CHCKACCT (ACNUMBER= ' || acct_num || ')'; if wireammt < 0 then ssa3 = 'CHCKCRDT'; else ssa3 = 'CHCKDEBT'; end; else if accttype = 'S' then do; ssa2 = 'SAVEACCT (ACNUMBER= ' || acct_num || ')'; if wireammt < 0 then ssa3 = 'SAVECRDT'; else ssa3 = 'SAVEDEBT'; end; 3 else do; file log; put / '***** Invalid ' accttype= 'for ' cust_ssn= acct_num= '*****'; go to outerr; end; infile acctsam dli ssa=(ssa1,ssa2,ssa3) call=func pcb=pcbindex status=st segment=seg; ssa1 = 'CUSTOMER(SSNUMBER= ' || cust_ssn || ')'; func = 'ISRT'; pcbindex = 4; file acctsam dli; put @1 amount pd5.2 @6 wiredate mmddyy6. @14 wiretime time8. @22 wiredesc $char40. @62 blanks $char19.; if st ¬= ' ' then if st = 'GE' then do; _error_ = 0; file log; if seg = 'CUSTOMER' then if accttype = 'C' then put / '***** No CHCKACCT segment with ' cust_ssn= acct_num= '*****'; else put / '***** No SAVEACCT segment with ' cust_ssn= acct_num= '*****'; else put / '***** No CUSTOMER segment with ' cust_ssn= '*****'; go to outerr; end; else link abendit; ssa3 = ' '; func = 'GHU '; input @1 acnumber $char12. @13 balance pd5.2 @18 stmtdate mmddyy6. @26 stmt_bal pd5.2; if st ¬= ' ' then link abendit; balance = balance + wireammt; ssa1 = ' '; ssa2 = ' '; func = 'REPL'; put @1 acnumber $char12. @13 balance pd5.2 @18 stmtdate mmddyy6. @26 stmt_bal pd5.2; if st ¬= ' ' then link abendit; if wireammt > 0 then debtnum = debtnum +1; else crdtnum = crdtnum +1; wirenum = wirenum +1; return; reptotal: file log; put // 'Number of debit transactions posted =' debtnum 8. / 'Number of credit transactions posted =' crdtnum 8.; stop; 4 chkp: chkpnum +1; chkptime = datetime(); file log; put @1 'Next checkpoint will be' @25 chkpnum @30 chkptime datetime13. @43 debtnum @51 crdtnum @59 errnum; func = 'CHKP'; pcbindex = 1; file acctsam dli; put @1 'SAS' @4 chkpnum z5.; if st ¬= ' ' then link abendit; return; outerr: errnum = errnum +1; return; abendit: file log; put _all_; abort; run; filename tranin clear; filename restart clear;
This group of statements initiates the restart, if a restart is necessary. The standard INFILE statement points to a file with fileref RESTART. The RESTART file has one record, a "control card" with data that will determine where processing should resume in the sequential input file. The data in the RESTART file is taken from the last checkpoint message written on the SAS log by the program that ended before completing processing. The message includes the number and time of the last checkpoint, and the values of the accumulator variables counting the number of debit transactions posted (CHCKDEBT), credit transactions posted (CHCKCRDT), and the number of bad records in the TRANIN file (CHKERR). The RESTART DD statement can be dummied out to execute the program normally (not as a restart). If RESTART is dummied out in the control language, end-of-file occurs immediately, and the program skips to the PROCESS subroutine (see item 6), as indicated by the EOF= option. The WIRESKIP variable is the sum of CHCKDEBT, CHCKCRDT, and CHKERR; that is, WIRESKIP represents the number of records in TRANIN that were processed by the program before the last checkpoint. A message is written to the SAS log that shows the checkpoint from which processing resumes. To position itself at the correct TRANIN record, the program reads the number of records indicated by the WIRESKIP variable. In other words, the program re-reads all records that were read in the first execution of the program, up to the last checkpoint. The values of DEBTNUM, CRDTNUM, WIRENUM, and ERRNUM are reset so that the final report shows the correct number of transactions. Otherwise, the report would show only the number of transactions processed in the restarted execution. | |
These statements are the same as the statements in Example 2: Incorrectly Updating a Database without Recovery Logic except that they are labeled "PROCESS." If the program is not being restarted, end-of-file for the INFILE RESTART occurs immediately, and the program branches to this subroutine. | |
If the value of ACCTTYPE is anything but C or S , the TRANIN record is a bad record. The program prints a message on the SAS log and branches to the OUTERR subroutine, which increments the ERRNUM accumulator variable. | |
The CHKP call is issued by this group of statements. This group is like that in Example 2: Incorrectly Updating a Database without Recovery Logic except that a message about the checkpoint is also printed on the SAS log. This message provides the necessary information for a restart. Note that the message is written to the SAS log before the CHKP call is actually issued, so it is possible that a system failure could occur between the time the message is written and the time the call is issued. Therefore, if a restart is necessary, you should verify that the last checkpoint referenced in the SAS log is the same as the last checkpoint in the DL/I log. This can be done by comparing checkpoint IDs. |
Copyright © 2007 by SAS Institute Inc., Cary, NC, USA. All rights reserved.