MUPIP JOURNAL command analyzes, extracts from, reports on, and recovers journal files. The format for the MUPIP JOURNAL command is:
MUPIP J[OURNAL] -qualifier[...] file-selection-argument
file-selection-argument
is a comma-separated list of journal files.
-qualifier [...]
is a combination of Action, Direction, Time, Sequence Number, Control, and Selection qualifiers that perform various MUPIP JOURNAL operations. To create any MUPIP JOURNAL command, select an appropriate combination of qualifiers by moving horizontally from the Action column extending to the Selection column:
Action |
Direction |
Time (optional) |
Sequence Number (optional) |
Control (optional) |
Selection (optional) |
---|---|---|---|---|---|
One or more |
Only one |
One or more |
Only one |
One or more |
One or more |
|
|
|
|
|
|
Also ensure that you adhere to the following rules:
-AFTER is incompatible with -RECOVER or -ROLLBACK; that is -AFTER requires -FORWARD, and only applies to action qualifiers: -EXTRACT, -SHOW, and -VERIFY.
-APPLY_AFTER_IMAGE is compatible only with -RECOVER, or -ROLLBACK.
-BACKWARD is incompatible with -FORWARD, -AFTER, -CHECKTN, -NOCHAIN, and -REDIRECT.
-[NO]BROKENTRANS is compatible only with -RECOVER, -ROLLBACK, or -EXTRACT.
-CHAIN is only compatible with -FORWARD.
-CHECKTN is incompatible with -BACKWARD.
-DETAIL is compatible only with -EXTRACT.
-FETCHRESYNC is only compatible with the -ROLLBACK action in the -FORWARD direction and is incompatible with RESYNC.
-FORWARD is incompatible with -BACKWARD, -FETCHRESYNC, -LOOKBACK_LIMIT, -ONLINE and -SINCE.
-FULL is compatible only with -EXTRACT, -SHOW, or -VERIFY.
-[NO]LOSTTRANS is compatible only with -RECOVER, -ROLLBACK, or -EXTRACT.
-REDIRECT is compatible only with -BACKWARD and -RECOVER.
-RESYNC is only compatible with the -ROLLBACK action and incompatible with FETCHRESYNC.
-ROLLBACK is incompatible with -RECOVER, -CHAIN, -CHECKTN, -REDIRECT, time qualifiers of -SHOW except -BEFORE.
-SINCE is incompatible with -FORWARD.
-TRANSACTION is compatible only with -EXTRACT and -SHOW.
-USER is compatible only with -EXTRACT and -SHOW.
file list must not be asterisk (*) for -REDIRECT.
file list must be asterisk (*) for -BACKWARD -ROLLBACK; -ROLLBACK -FORWARD accepts a list of journal file names.
Journal selection qualifiers are incompatible with -RECOVER, -ROLLBACK, and -VERIFY.
If -BEFORE (time-based) and -FETCHRESYNC/-RESYNC (sequence-number-based) are specified in the same MUPIP JOURNAL -ROLLBACK command, the qualifier that corresponds to an earlier database state or point in time prevails. For example, -BEFORE prevails when the update corresponding to the sequence number obtained through the -FETCHRESYNC command happened at a later time relative to the -BEFORE qualifier and vice versa.
-FETCHRESYNC, -ONLINE, and -RSYNC_STRM qualifiers are not compatible with -ROLLBACK -FORWARD.
For example, MUPIP JOURNAL -EXTRACT=gtm.mjf -FORWARD -DETAIL is a valid command which performs forward processing to extract detailed the journal records to gtm.mjf. However, MUPIP JOURNAL -EXTRACT -REDIRECT=gtm.dat=test/gtm.dat -FORWARD is an invalid command because -REDIRECT is not compatible with -EXTRACT.
MUPIP JOURNAL requires an inactive journal file that is available for exclusive (standalone) use.
Press <CTRL-C> to stop JOURNAL processing. A JOURNAL command that terminates abnormally by operator action or error produces an incomplete result. In this case, the resulting database may be corrupt. If you stop a JOURNAL operation by mistake, reissue the command to produce the proper result for -RECOVER (or -ROLLBACK) -BACKWARD. For -RECOVER -FORWARD, restore the database from backup and reissue the command.
This section describes the journaling action qualifiers.
Transfers information from journal files into files formatted for processing by M routines. It reports the journal time stamps using the $H format, as controlled by the time zone setting from the OS and the process environment for the process running the EXTRACT.
-EXTRACT takes <file-name> or -stdout as an optional argument. <file-name> may also be the name of a currently open named pipe (FIFO) set up to receive the extracted records.
<file-name> specifies the name of the output file. -stdout specifies that -EXTRACT write to standard output (stdout) instead of writing to a file.
With no arguments, MUPIP JOURNAL derives the output file specification of the extract file using the name of the first journal file that is processed in the forward processing phase and a file type of .mjf. Note that, if multiple journal names are specified in the command line the first journal specified might be different from the first journal processed in the forward phase. When -EXTRACT is specified with -RECOVER (or -ROLLBACK), the -JOURNAL command extracts all the journal records processed during a -RECOVER -FORWARD command or the forward phase of (-RECOVER or -ROLLBACK) -BACKWARD command.
-EXTRACT applies to forward processing of the journal file; if the combined state of the journal file and the Journal Time qualifiers does not cause forward processing, -EXTRACT does not create an output file.
When used independent of -RECOVER (or -ROLLBACK), -EXTRACT option can produce a result even though the database file does not exist, although it does try to access the database if it is available.
If a database having custom collation is inaccessible or the replication instance is frozen with a critical section required for the access held by another process and the environment variable gtm_extract_nocol is defined and evaluates to a non-zero integer or any case-independent string or leading substrings of "TRUE" or "YES", MUPIP JOURNAL -EXTRACT issues the DBCOLLREQ warning and proceeds with the extract using the default collation. If gtm_extract_nocol is not set or evaluates to a value other than a positive integer or any case-independent string or leading substrings of "FALSE" or "NO", MUPIP JOURNAL -EXTRACT exits with the SETEXTRENV error if it encounters such a situation. Note that if default collation is used for a database with custom collation, the subscripts reported by MUPIP JOURNAL -EXTRACT are those stored in the database, which may differ from those read and written by application programs.
Note that, a broken transaction, if found, is extracted to a broken transaction file (refer to “Journal Control Qualifiers” for details), and all future complete transactions are considered as lost transactions, and are extracted to a lost transaction file (refer to “Journal Control Qualifiers” for details).
To avoid broken transaction or lost transaction processing and instead extract all journal records into one file, use the control qualifier -FENCES=NONE. FIS strongly recommended against using -FENCES=NONE if -RECOVER/-ROLLBACK is also specified.
PARA[LLEL][=n] specifies the number of parallel threads (for backward processing) and parallel processes (for forward processing). Parallel threads typically increase the speed of MUPIP JOURNAL RECOVER/ROLLBACK operations.
Omitting the qualifier or specifying a value of one (1) defaults to a single process with no threads. Omitting the value or specifying a value of zero (0) specifies one thread or process per region.
A value greater than one (1) specifies the maximum number of concurrent threads or processes MUPIP should use, although it never uses more than one per region. If the number of regions exceeds the specified value, MUPIP allocates one thread or processes in an order determined by timestamps in the journal records.
The environment variable gtm_mupjnl_parallel provides a value when the MUPIP JOURNAL command has no explicit -PARALLEL qualifier; when defined with no value gtm_mupjnl_parallel acts like -PARALLEL with no value. When the -PARALLEL qualifier (or the gtm_mupjnl_parallel environment variable) specifies the use of parallel processes in the forward phase of a MUPIP JOURNAL command, MUPIP may create temporary shared memory segments and/or extract files (corresponding to -extract or -losttrans or -brokentrans qualifiers) and clean these up at the end of the command; however an abnormal termination such as a kill -9 might cause these to be orphaned. Journal extract files (created by specifying one of -extract or -brokentrans or -losttrans to a MUPIP JOURNAL command) contain journal records sorted in the exact order their corresponding updates happened in time.
Instructs MUPIP JOURNAL to initiate database recovery. -RECOVER initiates the central JOURNAL operation for non-replicated database. From the list of JOURNAL action qualifiers, select RECOVER alone or with any other action qualifiers except -ROLLBACK.
-RECOVER -FORWARD with time qualifiers initiates forward recovery. Forward recovery ignores the current journaling state of the target database file. It disables journaling of the target database file, (if currently ENABLE and ON), while playing forward the database updates. However, it restores the journaling state of the database at the end of a successful recovery (if necessary), except when journaling is ENABLE'd and ON before the recovery. In the latter case, the journaling state at the end of a successful recovery, is switched to ENABLE and OFF. No journaling is performed for the logical updates to the database for JOURNAL -RECOVER -FORWARD. If the target database's current transaction number is less than first transaction number to be processed in the specified journal file for that region, -RECOVER attempts to include previous generation journal file(s) in its processing, unless the -NOCHAIN qualifier is specified. Following the successive previous links of journal files -RECOVER tries to include previous generations of journal files until the transaction number when the journal file was created is less than, or equal to that of the target database. -RECOVER issues one or more informational messages when it includes previous generation journal files. If target database's current transaction number is not equal to the first transaction number of the earliest journal file to be processed for a region, -RECOVER exits with an error. If multiple journal files for a single region are specified with -RECOVER -FORWARD, it behaves as if -NOCHAIN was specified. If the journal files are not a complete set (for example mumps1.mjl and mumps3.mjl were specified, with mumps2.mjl missing from the command line), MUPIP JOURNAL produces an error because the journal files specified are discontinuous in terms of database transaction numbers. On the other hand, specifying just mumps3.mjl automatically includes mumps2.mjl and mumps1.mjl in the recovery.
-RECOVER -BACKWARD with time qualifiers initiates backward recovery. For backward recovery, the target database file should be the same as when GT.M wrote the last complete transaction to the journal. Because the database may be in an indeterminate state due to a failure, exact checks for this match are not possible. If the target database has journaling DISABLE'd (or ENABLE, OFF), -RECOVER -BACKWARD exits with an error message.
If the target database has journaling ENABLE, ON, but the journal file name in database file header does not match the latest generation journal file name specified for that region, -RECOVER exits with an error.
During forward processing phase of JOURNAL -RECOVER -BACKWARD, MUPIP journals the logical updates to the database. It also creates before images. It is always required to have journaling ENABLE'd and ON for -RECOVER -BACKWARD or -ROLLBACK.
If a transaction is found with incomplete fence, it is considered broken. During forward phase of recovery, if a complete transaction (fenced or unfenced) is found after a broken transaction. -RECOVER increments the error count. If -ERRORLIMIT is reached, the complete transaction goes to lost transaction file, otherwise, it is applied to the database.
All broken and lost transactions are made available as the result of the -RECOVERY. They are written as journal extract format in two different text files. They are the broken transaction file and the lost transaction file. Refer to the sections on BROKENTRANS and LOSTTRANS in “Journal Control Qualifiers”. MUPIP JOURNAL does not allow any two of -EXTRACT, -LOSTTRANS or -BROKENTRANS to specify the same file name unless they are special files (-stdout or /dev/null).
When performing JOURNAL -RECOVER with fences (FENCES="PROCESS" or FENCES="ALWAYS"), it is essential for the command to include all the journal files corresponding to the complete set of database files that make up the logical database. If the specified set of journals is incomplete, the recovery reports all transactions that included any missing region as broken. Typically, this means that the results of the recovery are unsatisfactory or even unusable.
MUPIP JOURNAL -RECOVER requires exclusive access to database files before recovery can occur. It keeps the exclusive access to the database files, which means that the database files become inaccessible during the time of recovery.
If time qualifiers are not specified, -BACKWARD -RECOVER/-ROLLBACK performs optimal recovery. An optimal recovery checks whether the datatabase is in a wholesome state and attempts to perform an automatic recovery if there is a crash. If needed, optimal recovery goes back to include some previous generation files in order to get a consistent starting point and then comes forward as far as the available journal record allow it to while preserving consistent application state. At the end, the journaling state of the database stays ENABLE, ON. Note that the gtm script performs an optimal recovery on every run.
When a database file is rolled back by -RECOVER -BACKWARD, the corresponding journal file is also rolled back so that the two are synchronized. -RECOVER -BACKWARD then creates a new journal file. If no forward play of journal records is neccessary, the newly created journal file stays empty and the database points to the new journal file. The values for journal allocation and extension in the new journal file, are copied over from the database. The autoswitchlimit value in the new journal file is the maximum of the autoswitchlimit values of all journal files from the latest generation journal file until the turnaround point journal file generation (turnaround point is the point in the journal file where backward processing stops and forward processing begins). The journal allocation/extension values in the new journal file are picked up from the earliest generation of the set of those journal files sharing the maximum autoswitchlimit value.
GT.M adds a prefix rolled_bak_ to the journal file whose entire contents are eliminated (rolled back) by -RECOVER -BACKWARD. GT.M does not use these files after a successful recovery therefore you might want to consider moving or deleting them. You should never use rolled_bak* files for any future database recovery. If there is a need to process rolled_bak* files, you should extract the journal records from rolled_back* files and process them using a M program.
Note | |
---|---|
Using -RECOVER on a replicated database initiates database recovery but turns replication OFF. Under most circumstances, there is no need to perform a -RECOVER operation on replicated regions. |
-ROLLBACK -FORWARD "*" command does what a -RECOVER -FORWARD "*" would do except that the ROLLBACK also updates sequence number related fields in the database file header and ensures update serialization across regions. -RECOVER can leave one database region ahead of another region. -RECOVER cannot ensure database Consistency across regions whereas -ROLLBACK can.
When used without time qualifiers, -ROLLBACK -FORWARD "*" applies update records in journal files to backed up copies of database files to bring them to the same state that -ROLLBACK -BACKWARD "*" would bring crashed database files. Note that, in the context of -RECOVER and -ROLLBACK, the "*" indicates the use of all the appropriate journal files in all the replicated regions and the quotes prevent inappropriate expansion by the OS shell.
Databases recovered with -ROLLBACK can be used in replicated instances.
|
-ROLLBACK initiates the central JOURNAL operation for a replicated database. MUPIP JOURNAL commands may specify -ROLLBACK with other action qualifiers but not with -RECOVER. With -BACKWARD, if you do not specify -BEFORE or -FETCHRESYNC, the database rolls back to the last consistent state. With -BACKWARD, the command allows only an asterisk (*) argument for the journal file selection, that is, -ROLLBACK selects journal files by itself.
If a transaction is found with incomplete fence, it is considered incomplete or broken.
During the forward phase of rollback, if a complete transaction (fenced or unfenced) is found after a broken transaction, it is considered as a lost transaction. During forward phase of rollback, MUPIP journals the logical updates to the database. All broken and lost transactions are made available as a result of the rollback. These are written as journal extract format in two different text files.
When a database file is rolled back by -ROLLBACK, the corresponding journal file is also rolled back so that the two are synchronized. -ROLLBACK then creates a new journal file. If no forward play of journal records is necessary, the newly created journal file is empty and the database points to the new journal file. The journal allocation/extension/autoswitchlimit values in the new journal file is set in the way as described for -RECOVER -BACKWARD in the previous section under -RECOVER.
A prefix rolled_bak_ is added to the journal file, whose entire contents are eliminated by a -ROLLBACK. These files are not used by GT.M after the MUPIP JOURNAL -RECOVER, and can be moved/deleted as needed.
For -ROLLBACK the target database file should be the same as when GT.M wrote the last complete transaction to the journal.
If the -FETCHRESYNC or -RESYNC qualifiers are not specified, MUPIP does an optimal rollback (that is, check whether the database is in a wholesome state and attempt to automatically recover a database if there is a crash).
-ROLLBACK -BACKWARD exits with an error message if a database does not have both journaling and replication either enabled or disabled.
Note | |
---|---|
If ROLLBACK (either -NOONLINE or -ONLINE) terminates abnormally (say because of a kill -9), it leaves the database in a potentially inconsistent state indicated by the FILE corrupt field in the database file header. When a ROLLBACK terminates leaving this field set, all other processes receive DBFLCORRP errors any time they attempt to interact with the database. You can clear this condition as following in descending order of risk:
However, the MUPIP and DSE actions do not ensure that the database has consistent state; check for database integrity with MUPIP INTEG. |
-NOO[NLINE]
Specifies that ROLLBACK requires exclusive access to the database and the replication instance file, which means the database and the replication instance files are inaccessible during a -ROLLBACK -NOONLINE.
-ROLLBACK -FORWARD does not support the -[NO]O[NLINE] qualifier.
-ON[LINE]
Specifies that ROLLBACK can run without requiring exclusive access to the database and the replication instance file.
Any utility/command attempted while MUPIP JOURNAL -ONLINE -ROLLBACK operates waits for ROLLBACK to complete; the $gtm_db_startup_max_wait environment variable configures the wait period. For more information on $gtm_db_startup_max_wait, refer to “Environment Variables”.
Note | |
---|---|
Because MUPIP ROLLBACK -ONLINE can take a database backwards in state space, please make sure that you understand what it is that you intend it to do when you invoke it. MUPIP ROLLBACK -ONLINE issues an ORLBKROLLED message to the system log indicating the logical state of the database changed. |
By default, MUPIP JOURNAL -ROLLBACK -BACKWARD is -NOONLINE.
GT.M increments ISV $ZONLNRLBK every time a process detects a concurrent MUPIP JOURNAL -ONLINE -ROLLBACK.
The logical state of the database after the completion of MUPIP JOURNAL -ONLINE -ROLLBACK matches the logical state of the database at the start of MUPIP JOURNAL -ONLINE -ROLLBACK, that is, the ROLLBACK only removes any incompletely committed TP transactions or non-TP mini-transactions; any concurrent transaction (TP or Non-TP) incurs a restart.
If MUPIP JOURNAL -ONLINE -ROLLBACK changes the logical state of the database, the behavior is as follows:
For the duration of the rollback, replication is turned OFF on all regions and turned back ON at the end of the rollback.
-ONLINE -ROLLBACK increments ISV $ZONLNRLBK
In a TP transaction including trigger code within a transaction, -ONLINE -ROLLBACK restarts the transaction.
In a non-TP mini-transaction, including within an implicit transaction caused by a trigger, -ONLINE -ROLLBACK produces a DBROLLEDBACK error, which, in turn, invokes the error trap if $ETRAP or $ZTRAP are in effect.
Specifies which information for the JOURNAL command to display about a journal file.
Use -FORWARD with -SHOW together (but without -RECOVER ) to process the entire journal file. Specify -SHOW with -RECOVER (or -ROLLBACK) to consider all the journal files/records processed during a -RECOVER -FORWARD command or forward phase of a -RECOVER (or -ROLLBACK) -BACKWARD command. Without -RECOVER (or -ROLLBACK), -SHOW does not require database access.
The show-option-list includes (these are not case-sensitive):
AL[L]
Displays all the available type of information about the journal file. ALL is the default if you omits the show-option-list.
AC[TIVE_PROCESSES]
Displays all processes active at the end of the period specified implicitly or explicitly by the JOURNAL command time qualifiers.
B[ROKEN_TRANSACTIONS]
Display all processes that had incomplete fenced transactions at the end of the period covered by the JOURNAL command.
H[EADER]
Displays the journal file header information. If the MUPIP JOURNAL command includes only the -SHOW=HEADER action qualifier, GT.M processes only the journal file header (not the contents) even if you specify -BACKWARD or -FORWARD with it. The size of a journal file header is 64K.
HEADER displays almost all the fields in the journal file header. The NODE field is printed up to a maximum of the first 12 characters. The following is an example of SHOW=HEADER output:
------------------------------------------------------------------------------- SHOW output for journal file /home/jdoe/.fis-gtm/V6.3-002_x86/g/gtm.mjl ------------------------------------------------------------------------------- Journal file name /home/jdoe/.fis-gtm/V6.3-002_x86/g/gtm.mjl Journal file label GDSJNL23 Database file name /home/jdoe/.fis-gtm/V6.3-002_x86/g/gtm.dat Prev journal file name /home/jdoe/.fis-gtm/V6.3-002_x86/g/gtm.mjl_2012310190106 Next journal file name Before-image journal ENABLED Journal file header size 65536 [0x00010000] Virtual file size 2048 [0x00000800] blocks Journal file checksum seed 2272485152 [0x87735F20] Crash FALSE Recover interrupted FALSE Journal file encrypted FALSE Journal file hash 00000000000000000000000000000000000 Blocks to Upgrade Adjustment 0 [0x00000000] End of Data 65960 [0x000101A8] Prev Recovery End of Data 0 [0x00000000] Endian Format LITTLE Journal Creation Time 2012/11/06 17:30:33 Time of last update 2012/11/06 17:30:33 Begin Transaction 1 [0x0000000000000001] End Transaction 1 [0x0000000000000001] Align size 2097152 [0x00200000] bytes Epoch Interval 300 Replication State CLOSED Jnlfile SwitchLimit 8386560 [0x007FF800] blocks Jnlfile Allocation 2048 [0x00000800] blocks Jnlfile Extension 2048 [0x00000800] blocks Maximum Journal Record Length 1048672 [0x00100060] Turn Around Point Offset 0 [0x00000000] Turn Around Point Time 0 Start Region Sequence Number 1 [0x0000000000000001] End Region Sequence Number 1 [0x0000000000000001] Process That Created the Journal File: PID NODE USER TERM JPV_TIME ------------------------------------------------------------ 0000006706 jdoe-laptop jdoe 0 2012/11/06 17:30:33 Process That First Opened the Journal File: PID NODE USER TERM JPV_TIME ------------------------------------------------------------ 0000006706 jdoe-laptop jdoe 0 2012/11/06 17:30:33
P[ROCESSES]
Displays all processes active during the period specified implicitly or explicitly by the JOURNAL command time qualifiers.
S[TATISTICS]
Displays a count of all journal record types processed during the period specified implicitly or explicitly by the JOURNAL command time qualifiers.The following is an example of SHOW=STATISTICS output:
------------------------------------------------------------------------------- SHOW output for journal file /home/jdoe/.fis-gtm/V6.3-002_x86/g/gtm.mjl ------------------------------------------------------------------------------- Record type Count ---------------------- *BAD* 0 PINI 2 PFIN 2 ZTCOM 0 KILL 1333533 FKILL 0 GKILL 0 SET 0 FSET 0 GSET 0 PBLK 4339 EPOCH 2 EOF 1 TKILL 0 UKILL 0 TSET 0 USET 0 TCOM 0 ALIGN 49 NULL 0 ZKILL 0 FZKIL 0 GZKIL 0 TZKIL 0 UZKIL 0 INCTN 4314 AIMG 0 TZTWO 0 UZTWO 0 TZTRI 0 UZTRI 0 TRUNC 0 %GTM-S-JNLSUCCESS, Show successful %GTM-S-JNLSUCCESS, Verify successful %GTM-I-MUJNLSTAT, End processing at Tue Nov 6 17:42:21 2012
The following example displays the cryptographic hash of the symmetric key stored in the journal file header (the output is one long line).
$ mupip journal -show -backward mumps.mjl 2>&1 | grep hash Journal file hash F226703EC502E975784 8EEC733E1C3CABE5AC146C60F922D0E7D7CB5E 2A37ABA005CE98D908B219249A0464F5BB622B72F5FDA 0FDF04C8ECE52A4261975B89A2
Verifies journal files for integrity. This qualifier cannot have a value. -VERIFY scans journal files and checks if they have legal form, if not, it terminates without affecting the database files.
-NOVERIFY is the default for -RECOVER -FORWARD and -ROLLBACK -FORWARD. -VERIFY is the default for RECOVER -FORWARD -NOCHECKTN. -VERIFY is also the default for all other MUPIP JOURNAL commands (including -RECOVER -BACKWARD and -ROLLBACK -BACKWARD).
-VERIFY when specified along with -FORWARD verifies the entire journal file For -NOVERIFY -FORWARD, only the tail of a journal file is verified for cross region integrity. In both cases, if -RECOVER is also specified, the forward play of journal records is done in a separate pass only after the verification pass is complete and error-free.
-VERIFY along with -BACKWARD verifies all journal records from the end of the journal file till the turn around point. When -VERIFY -BACKWARD is specified along with -RECOVER or -ROLLBACK, backward processing involves two passes, the first pass to do the verification until the turn around point, and the second pass to apply before image (PBLK) records.
When -NOVERIFY -BACKWARD is specified along with -RECOVER or -ROLLBACK, PBLKs are applied to the database in the same pass as the verification. This speeds up processing. But the disadvantage of this approach is that in the event of verification terminating in the middle of backward processing, there is no protection of cross-region integrity. FIS recommends the use of -VERIFY (the default) when -BACKWARD is used with -RECOVER or -ROLLBACK. For -FORWARD, unless there is reason to suspect that the journal files have sustained structural damage, FIS suggests the use of -NOVERIFY (the default).
When used independent of -RECOVER (or -ROLLBACK), -[NO]VERIFY option does not need database access. In this case the default is -VERIFY.
The following two qualifiers control the journal processing direction:
-BACKWARD
Specifies that MUPIP JOURNAL processing should proceed from the end of the journal file. If the actions include -RECOVER, JOURNAL -BACKWARD restores before-images from the end-of the file back to an explicitly or implicitly specified point (the turn around point), before it reverses and processes database updates in the forward direction (the forward phase).
Note | |
---|---|
-BACKWARD is incompatible with -FORWARD. |
-FO[RWARD]
Specifies that MUPIP JOURNAL processing for the specified action qualifier should proceed from the beginning of the given journal file. When processing a -RECOVER action qualifier, in certain cases, MUPIP JOURNAL may need to go before the first record of the specified journal file, that is, it can start from a previous generation journal file(refer to “-RECover ” for details).
If multiple journal files are specified in the command line, -FORWARD sorts the journal files within each region based on creation time and processes them starting from the earliest journal file. Unless the -NOCHECKTN qualifier is specified, -FORWARD performs checks on journal files corresponding to each region to ensure they are contiguous, both in terms of time span, as well as, transaction number span. -FORWARD errors out if it detects a discontinuity.
Note | |
---|---|
-FORWARD is incompatible with -BACKWARD and -ROLLBACK. |
Journal qualifiers specifying time accept arguments in absolute or delta time format. Enclose time arguments in quotation marks (" ") . Include a back-slash (\) delimiter before both, the beginning and ending quotation marks to escape it from being processed by the UNIX shell.
Absolute format is day-mon-yyyy hh:mm:ss
, where day
denotes the date of the month, mon
indicates the abbreviated 3-letter month name (for example, Jan, Feb,..) and the year yyyy
and hour hh
are separated by a space. Absolute time may indicate today's date with "-- " before the hours.
Delta format is day hh:mm:ss, indicating the number of days, hours, minutes, and seconds; where the day and the hours (hh) are separated by a space. If delta time is less than a day, it must start with zero (0) followed by a space.
Delta time is always relative to the maximum time of the last record in all journal files specified by arguments to the MUPIP JOURNAL command.
Note | |
---|---|
All time qualifiers except -BEFORE are incompatible with -ROLLBACK. |
The following section describes the time qualifiers in more detail:
-A[FTER]=time
Specifies the starting time stamp in the journal file after which any -FORWARD action should start processing. This time qualifier is compatible only with -EXTRACT,-SHOW, or -VERIFY.
If -AFTER= provides a time following the last time recorded in the journal file or following any -BEFORE= time, JOURNAL processing produces no result and MUPIP displays a warning message. If -AFTER provides a time preceding the first time recorded in the journal file specified in the command line, and, previous generation journal file(s) exists for that journal file, then previous generation journal file(s) are not included for the processing. You must specify previous generation journal files explicitly in the command line in order for them to be considered.
Using -BEFORE with -AFTER restricts processing to a particular period of time in the journal file.
-BE[FORE]=time
Specifies the ending time for any action -FORWARD or -BACKWARD. The ending time is inclusive, that is, -BEFORE includes the records that match the specified ending time in the journal records. If -BEFORE= specifies a time preceding the first time recorded in the journal file, or preceding any -AFTER= or -SINCE= time, JOURNAL processing produces no result, and MUPIP displays a warning message.
If -BEFORE= time exceeds the last time recorded in journal files, JOURNAL processing effectively ignores the qualifier and terminates at the end of the journal file. By default, JOURNAL processing terminates at the end of the journal file.
When used with -ROLLBACK or -RECOVER, -BEFORE specifies the time at which MUPIP stops applying updates to the database in its forward processing phase (i.e., no journal records with update times after the -BEFORE time are applied to the database).
When both -FETCHRESYNC/-RESYNC and -BEFORE are used with -ROLLBACK -BACKWARD, the qualifier corresponding to an earlier database state or point in time prevails. For example, -BEFORE prevails when the update corresponding to the sequence number obtained through the -FETCHRESYNC command happened at a later time relative -BEFORE and vice versa.
-[NO]LOO[KBACK_LIMIT][=lookback-option-list]
Specifies how far JOURNAL -RECOVER -BACKWARD processes past the turnaround point (the explicit or implicit point in journal file up to which -RECOVER proceeds backward before it reverses and processes database in forward direction), while attempting to resolve open transaction fences. This option is applicable only for transactions fenced with ZTSTART and ZTCOMMIT. For transaction fenced with TSTART and TCOMMIT, -RECOVER always resolves open transaction fences.
-LOOKBACK_LIMIT=options, include time and transaction counts. -NOLOOKBACK_LIMIT specifies that JOURNAL -BACKWARD can process all the way to the beginning of the journal file, if necessary, to resolve open transaction fences. -LOOKBACK_LIMIT= is incompatible with -FORWARD.
When -FENCES=NONE, JOURNAL processing ignores -LOOKBACK_LIMIT.
The -LOOKBACK_LIMIT options are:
TIME=
time
This limits LOOKBACK by a specified amount of delta or absolute journal time.
OPERATIONS=
integer
This limits LOOKBACK to the specified number of database transactions.
The TIME LOOKBACK option name and its value must be enclosed in quotes ("").
For example:
-lookback=\"time=0 00:00:30\"
When -LOOKBACK_LIMIT= specifies both options, they must be separated by a comma (,), for example:
-lookback=\"time=0 00:00:30,operations=35\"
When -LOOKBACK_LIMIT= specifies both options, the first limit reached terminates the LOOKBACK.
By default, MUPIP JOURNAL uses -LOOKBACK_LIMIT=\"TIME=0 00:05\" providing five minutes of journal time prior to -SINCE= to resolve open fences. A -LOOKBACK_LIMIT that specifies a limit much before the beginning of the earliest journal file acts as if -NOLOOKBACK_LIMIT was specified.
-SI[NCE]=time
The -SINCE time qualifier applies to MUPIP JOURNAL -BACKWARD. The -SINCE qualifier specifies how far back in time MUPIP JOURNAL should at least process (from the end of the journal file), before starting the forward processing. The actual turn-around point for -RECOVER and -ROLLBACK in each database region is an epoch in the journal files before or at the -SINCE time, but not after it.
The time specified references time stamps in the journal files. If there are open fenced transactions when JOURNAL -BACKWARD locates the -SINCE= time, it continues processing backward to resolve them, unless the command also specifies -FENCES=NONE. If -SINCE= time exceeds the last time recorded in the journal files or, follows any -BEFORE=time, JOURNAL processing effectively ignores the qualifier, and displays a warning message.
By default, -SINCE= time is 0 00:00:00 which denotes the time at the end of the journal file (the time when the last journal record was updated).
The following qualifier is compatible only with -EXTRACT.
-S[EQNO]=<sequence_number_list>
Specifies a list of sequence numbers to include or exclude in the journal extract. <sequence_number_list> is a comma separated list of sequence number(s) in decimal form. When a sequence number has a (~) prefix, -SEQNO excludes it from the journal extract. For replicated regions, EXTRACT -SEQNO uses replication sequence numbers, which may select records from multiple regions. For unreplicated regions, EXTRACT uses journal sequence numbers, but specifying sequence number selection with more than one regions produces a JNLEXTRCTSEQNO error. When the sequence number list contains a sequence number involved in a TP transaction, EXTRACT reports it in a broken transaction file when the result does not contain all regions, which is commonly the case without replication, and may be the case with replication when not all regions are available to the utility.
Example:
$mupip journal -extract -seqno="~1,2,3,4,~5" -forward -broken=trans.broken -lost=trans.lost "*"
This example produces a journal extract containing journal sequence numbers 2,3,and 4. 1 and 5 are not part of the journal extract as they have the (~) prefix.
The following qualifiers are compatible only with -ROLLBACK.
-FET[CHRESYNC]
For more information on -FETCHRESYNC, refer to “Rolling Back a Replicated Database ”.
-RES[YNC]=<journal sequence number>
For more information on -RESYNC, refer to “Rolling Back a Replicated Database ”.
The following qualifiers control journal processing:
-[NO]AP[PLY_AFTER_IMAGE]
Specifies that after image records (AIMG) be applied to the database as part of forward processing of -RECOVERY or -ROLLBACK. AIMG are "snapshots" of the database updates captured by GT.M immediately after the change caused by a DSE update. By default, during forward phase of backward recovery or rollback, AIMG records are applied to the database.
By default, -RECOVER -FORWARD does not apply AIMG record into the database. -APPLY_AFTER_IMAGE is compatible with -RECOVER, or -ROLLBACK action qualifiers only.
-[NO]BR[OKENTRANS]=<extract file>
-[NO]BROKENTRANS is an optional qualifier for -ROLLBACK, -RECOVER and -EXTRACT. NOBROKENTRANS suppresses the generation of a broken transaction file. Otherwise, if the command does not specify a file name and MUPIP finds any broken transactions, MUPIP JOURNAL creates a broken transaction file using the name of the current journal file being processed with a .broken extension.
Note that, if selection qualifiers are specified, the broken transaction determination (and therefore lost transaction determination as well) is done based on the journal file that is filtered by the selection qualifiers. This means that a transaction's journal records may be considered complete or broken or lost, depending on the nature of the selection qualifiers. Using -FENCES=NONE along with the selection qualifiers will result in every journal record to be considered complete and hence prevent broken or lost transaction processing.
-[NO]CHA[IN]
-CHAIN allows JOURNAL processing to include previous generations of journal files with -FORWARD. If JOURNAL -RECOVER needs to process previous generation journal file(s) and -NOCHAIN is specified, MUPIP JOURNAL exits with an error.
-CHAIN is the default.
-[NO]CHE[CKTN]
-CHECKTN specifies that JOURNAL -FORWARD must verify for each region that the begining transaction number of the earliest journal file to be processed for that region is same as the current transaction in the database file and that the ending transaction number of every journal file is equal to the begining transaction number of the next generation journal file for a given region. By default, -FORWARD uses -CHECKTN.
-NOCHECKTN forces forward recovery by overriding inbuilt mechanisms for checking transaction integrity. MUPIP performs -VERIFY when -NOCHECKTN is specified. Use -NOCHECKTN with caution because it may lead to integrity issues in the recovered database and journal files.
ROLLBACK -FORWARD accepts only -CHECKTN, which is the default, but does not accept -NOCHECKTN.
-CHECKTN is incompatible with -BACKWARD.
-[CO]RRUPTDB
Extracts journal records into a single file even if the database is corrupt or missing. Always specify a journal file name when you are using -CORRUPTDB. -CORRUPTDB does not recognize the wildcard character "*" for journal file name and is incompatible with -FENCES, -LOST, and -BROKEN qualifiers.
-[NO]ER[ROR_LIMIT][=integer]
Specifies the number of errors that MUPIP JOURNAL processing accepts. When the number of errors exceeds the -ERROR_LIMIT, the -INTERACTIVE qualifier determines whether JOURNAL processing halts or defers to the operator. -NOERROR_LIMIT prevents MUPIP JOURNAL from stopping because of errors. Journal processing continues until it reaches the end of the journal file, regardless of the number of errors.
Note that, -NOERROR_LIMIT is not the same as -ERROR_LIMIT=0.
By default, MUPIP JOURNAL uses -ERROR_LIMIT=0, causing the first error to initiate the appropriate error action. In case of a crash there could be some incomplete journal records at the end of a journal file. MUPIP JOURNAL does not consider these as errors. In addition, fenced transactions that are broken are not considered as errors.
During the forward phase of recovery, if journal processing finds a broken transaction, all the logical records processed afterwards are considered suspect. If a complete transaction is found after any broken transactions, MUPIP JOURNAL -RECOVER increments the error count and, if it is less than the error limit, it is applied to the database. Otherwise, it is treated as a lost transaction and extracted. If a complete transaction is found after any broken transactions, MUPIP JOURNAL -ROLLBACK treats it as a lost transaction and extracts it irrespective of the error limit.
If MUPIP JOURNAL needs to increment error count during its processing, a warning message is issued for every error encountered except in the following cases when the error count is incremented but no warning message is displayed:
When a complete transaction is found after a broken transaction
When -EXTRACT -FULL encounters errors
If MUPIP JOURNAL completes successfully with a non-zero value of error count, the return status is not a success, but a warning.
-FE[NCES][=fence-option]
Specifies how JOURNAL processes fenced transactions. Fenced transactions are logical transactions made up of database updates preceded by a TSTART command followed by a TCOMMIT command. All updates between a TSTART and a TCOMMIT are designed to occur together so that after journal recovery the database contains either all the updates corresponding to a fenced transaction, or none of them.
The argument values for -FENCES option for MUPIP -RECOVER/-ROLLBACK are not case-sensitive.
The fence options are:
NONE
This causes MUPIP JOURNAL -RECOVER to apply all individual updates as if transaction fences did not exist. Note that, this means journal processing treats a SET/KILL within a TP transaction as if it was an unfenced SET/KILL. -FENCES=NONE is not permitted for MUPIP JOURNAL -ROLLBACK.
ALWAYS
This causes MUPIP JOURNAL -RECOVER to treat any unfenced or improperly fenced updates as broken transactions. FENCES=ALWAYS is not permitted for MUPIP JOURNAL -ROLLBACK.
PROCESS
This causes MUPIP JOURNAL to accept unfenced database updates, and also to observe fences when they appear, generating broken transaction files in the case of a TSTART with no corresponding TCOMMIT. It also generates broken transactions if a multi-region transaction with TSTART and TCOMMIT expects N regions to participate, but the number of TSTART/TCOMMIT pairs found is less than N. -ROLLBACK accepts -FENCES=PROCESS, which is the default.
By default, MUPIP JOURNAL uses -FENCES=PROCESS.
-FU[LL]
-FULL when used with -EXTRACT, specifies that all journal records be extracted. A journal file's contents can be rolled back in case of backward recovery or rollback(refer to “-RECover ” or “-ROLLBACK [{-ON[LINE]|-NOO[NLINE]}] ” for more details) in order to keep the database and journal in sync. This is achieved not by truncating the contents of the journal file but instead setting a field in the journal file header, which shows up as "Prev Recovery End of Data" in a MUPIP JOURNAL -SHOW=HEADER output, to indicate the end of the journal file before rolling back and setting another field in the file header to indicate the new end of the journal file (this field shows up as "End of Data" in a MUPIP JOURNAL -SHOW=HEADER output). Once a journal file's contents are rolled back, all future MUPIP JOURNAL commands (including -EXTRACT) operate on the rolled back journal file only. But if -FULL is specified along with -EXTRACT, MUPIP extracts the entire journal file contents (including those records that were rolled back). This qualifier is to be used only as a diagnostic tool and not in normal operation.
-FULL qualifier is compatible with -EXTRACT only.
-[NO]IN[TERACTIVE]
Specifies whether, for each error over the -ERROR_LIMIT, JOURNAL processing prompts the invoking operator for a response to control continuation of processing. If the operator responds that processing should not continue, the MUPIP JOURNAL command terminates.
-NOINTERACTIVE terminates the journal processing as soon as the MUPIP JOURNAL command generates the number of errors specified in -ERROR_LIMIT.
This qualifier applies when the MUPIP command is entered from a terminal. The default is -INTERACTIVE.
-[NO]LOST[TRANS]=<extract file>
-[NO]LOSTTRANS is an optional qualifier for -RECOVER, -ROLLBACK and -EXTRACT. NOLOSTTANS suppresses the generation of a broken transaction file. Otherwise, if the command does not specify a file name and MUPIP finds any lost transactions, MUPIP JOURNAL creates a lost transaction file using the name of the current journal file being processed with a .lost extension.
Journal processing treats any complete transactions after a broken transaction as a lost transaction, and writes such transactions into the lost transaction file. -RECOVER might consider it as good transaction and apply it to the database, if -ERROR_LIMIT qualifier allows it to do so.
Note that, if selection qualifiers are specified, journal processing does the broken transaction determination (and therefore lost transaction determination as well) based on the journal file that is filtered by the selection qualifiers. This means that a transaction's journal records may be considered complete or broken or lost, depending on the nature of the selection qualifiers. Using -FENCES=NONE along with the selection qualifiers results in every journal record being considered complete and hence preventing broken or lost transaction processing.
In the case of a replicated database, lost transaction can have an additional cause. If failover occurs (that is, the originating Source Server, A, fails and the replicating Source Server, B, assumes the originating instance's role), some transactions committed to A's database may not be reflected in B's database. Before the former originating instance becomes the new replicating instance, these transactions must be rolled back. These transactions are known as "lost transactions". Note that these are complete transactions and different from a broken transaction. MUPIP JOURNAL -ROLLBACK stores extracted lost transactions in the extract-file specified by this qualifier. The starting point for the search for lost transactions is the journal sequence number obtained from the originating Source Server in the -FETCHRESYNC operation.
-RED[IRECT]=file-pair-list
Replays the journal file to a database different than the one for which it was created. Use -REDIRECT to create or maintain databases for training or testing.
This qualifier applies to -RECOVER action and -FORWARD direction qualifier only. JOURNAL rejects -REDIRECT unless it appears with -RECOVER.
The file-pair-list consists of one or more pairs of file-names enclosed in parentheses () and separated by commas (,). The pairs are separated by an equal sign in the form:
old-file-name=new-file-name
where the old file-name identifies the original database file and the new file-specification file-name identifies the target of the -RECOVER. The old-file-specification can always be determined using -SHOW.
By default, JOURNAL directs -RECOVER to the database file from which the journal was made. -REDIRECT is not compatible with -ROLLBACK.
Example:
$ mupip journal -recover -forward -redirect="bgdbb.dat=test.dat" bgdbb.mjl
This JOURNAL command does a forward recovery that -REDIRECTs the updates in bgdbb.mjl from bgdbb.dat to test.dat.
-VERB[OSE]
Prints verbose output in the course of processing. It is not negatable and it is set to OFF by default.
Journal Selection Qualifiers are compatible with -EXTRACT and -SHOW operations only. This is because most applications are not constructed to safely remove a subset of transactions based on criteria that is exterior to the application design. To exclude transactions from a recovery based on some selection criteria, the methodology is to -EXTRACT the records, and then reapply them through application logic rather than by journal recovery. This approach permits the application logic to appropriately handle any interactions between the removed and the retained transactions. Note that, selection qualifiers might result in only a subset of a fenced transaction's journal records to be extracted (for example, a TSTART record may not be extracted because the first update in that transaction was filtered out by a selection qualifier, while the corresponding TCOMMIT record may get extracted). This can cause a fenced transaction to seem broken when actually it is not.
The following qualifiers control the selection criteria for journal processing.
Except for -TRANSACTION, all qualifiers allow for specifying a comma (,) seperated list of values.
-G[LOBAL]=global-list
Specifies globals for MUPIP JOURNAL to include or exclude from processing. You might find this qualifier useful for extracting and analyzing specific data.
The global-list contains one or more global-names (without subscripts) preceded by a caret symbol (^). To include more than one global use one of the following syntaxes.
$ mupip journal -forward -extract -global="^A*,^C" mumps.mjl
or
$ mupip journal -forward -extract -global="(^A*,^C)" mumps.mjl
The names may include the asterisk (*) wildcard. That is, -GLOBAL="^A*" selects all global variables with names starting with A. The entire list or each name may optionally be preceded by a tilda sign (~), requiring JOURNAL to exclude database updates to the specified global(s). When the global-list with a MUPIP JOURNAL -GLOBAL does not start with a tilda sign (~), JOURNAL processes only the explicitly named globals. By default, JOURNAL processes all globals.
To specify subscripts, using -GLOBAL="^A(1)" results in all keys under the ^A(1) tree to be included, that is, it is equivalent to using -GLOBAL="^A(1,*)". An asterisk (*) or a percent (%) anywhere in the global specification is permitted. Percent (%) matches any character, and asterisk (*) matches any string (possibly zero length too). The asterisk (*) or percent (%) specification can be used for -USER qualifier too.
Example:
To extract all ^GBL* except for ^GBLTMP:
$ mupip journal -extract -global="^GBL*,~^GBLTMP" -forward mumps.mjl
To extract all ^GBL except for ^GBL(1,"TMP"):
$ mupip journal -extract -global=\"^GBL,~^GBL\(1,\"\"TMP\"\"\)\" -forward mumps.mjl
The backslash (\) delimiter characters are required in UNIX to pass MUPIP the double quotes (") of the string subscript.
An INVGLOBALQUAL error is issued along with the error offset in the command line, whenever a parse error of the global qualifier string is encountered.
-GV[PATFILE]=/path/to/pattern-file
Specifies the location of a pattern file containing a list of patterns for all types of SET journal records that MUPIP JOURNAL -EXTRACT should include or exclude from processing. Use this qualifier to restrict the output of a journal extract by global nodes value (that is, by all types of SET records).
The following details the syntax of the pattern file, and examples of how MUPIP JOURNAL -EXTRACT responds:
When a pattern entry starts with a tilda sign (~), -GVPATFILE excludes the matching global node values from the JOURNAL EXTRACT file. For example, ~(not this value) excludes all globals that exactly match " not this value".
When the pattern does not start a tilda sign (~) or contain an asterisk (*), MUPIP JOURNAL -EXTRACT reports only those global node values that exactly match the pattern. For example: " match this value".
When a pattern contains an asterisk (*), MUPIP JOURNAL -EXTRACT expands it and tries to match multiple characters.
Example : "*a*b*" matches values like "ab", "..ab", "ab.. ", "a..b", "aaabbabb", and so on but does not match values like "ba", "aaa", "bbb", and so on.
When a pattern contains a percentage (%), MUPIP JOURNAL -EXTRACT matches it for one character. Example : "a%b%" matches values like "a1b1" but does not match values like "ab", "aabbc", and so on.
A pattern can be enclosed within parentheses "()" for readability.
When you use any of the following characters in the pattern, you can escape them by preceding the character with "\". Example : "a\**b" matches values like "a*..b" but not "a..b".:
"(" and "~" at the beginning.
")" at the end.
"\", "*", and "%" occurring anywhere within the pattern.
In UTF-8 mode, the contents of the pattern file can include Unicode characters.
If a pattern file does not exist, MUPIP JOURNAL -EXTRACT produces the FILEOPENFAIL error and returns a non-zero exit code to the shell.
You can specify multi-line entries in a pattern file.With multiple lines, MUPIP JOURNAL EXTRACT produces those SET records that match any one of the pattern lines with the exception of exclusion patterns (those starting ~) which take precedence over other non-exclusion patterns.
Here are a few examples of the pattern file, and how MUPIP JOURNAL -EXTRACT matches the pattern file values:
> cat matchA_notAA.txt ~(*AA*) *A* > $gtm_dist/mupip journal -extract -gvpatfile=matchA_notAA.txt -forward "*" Extracts global values that contain a single "A", but not "AA". > cat ending22.txt *notmatching* *22 > $gtm_dist/mupip journal -extract -gvpatfile=ending22.txt -forward "*" Extracts global values ending with "22", even when there are no globals containing "notmatching". > cat startswithsplchars.txt \** \~* > $gtm_dist/mupip journal -extract -gvpatfile=matchA_notAA.txt -forward "*" Extracts global values that start with a "*" or a "~".
-ID=pid-list
Specifies that JOURNAL processing include or exclude database updates generated by one or more processes, identified by process identification numbers (PIDs). The entire list or each PID may optionally be preceded by a tilda sign (~), requiring JOURNAL to exclude database updates initiated by the specified PID. You may use this qualifier for trouble shooting or analyzing data.
By default, JOURNAL processes database updates regardless of the PID that initiated it.
-T[RANSACTION]=transaction-type
Specifies transaction-types for JOURNAL to include or exclude from processing. For example, you may use this qualifier to report only on KILL operations to locate possible causes for missing data.
The transaction-types are SET and KILL and can be negated. These types correspond to the M commands of the same names. When the transaction-type with a JOURNAL -TRANSACTION is not negated, JOURNAL processes only transactions of the type named (for example, -TRANSACTION=KILL), whereas if it is negated, JOURNAL does not process transactions of the type named (for exmaple, -TRANSACTION=NOKILL).
By default, JOURNAL processes transactions, regardless of its type.
-U[SER]=user-list
Specifies that MUPIP JOURNAL processing include or exclude database updates generated by one or more users. You can use this qualifier to audit the actions of a particular user. The user-list contains names of one or more users. Indicate multiple users by separating the names with commas (,). The names may include the wildcard asterisk (*). The entire list or each name may optionally be preceded by a minus sign (-) tilda sign (~), requiring JOURNAL to exclude database updates initiated by the specified user(s). When the user-list with a JOURNAL -USER does not start with a tilda sign (~), JOURNAL processes only those database updates, which are generated by explicitly named users. The asterisk (*) or percent (%) specification can be used for -USER qualifier. Percent (%) matches any character, and asterisk (*) matches any string (possibly zero length too).
By default, JOURNAL processes database updates regardless of the user who initiated them.