MUPIP JOURNAL

MUPIP JOURNAL analyzes, extracts from, reports on, and recovers journal files. The following sections describe the MUPIP JOURNAL command in reference format.

The format for the JOURNAL command is:

MUPIP J[OURNAL] /qualifier[...] <file selection argument>[,...]

where <file selection argument> is described in the following section.

For a summary table of MUPIP commands and qualifiers, including MUPIP JOURNAL, refer to the "MUPIP" chapter.

The MUPIP JOURNAL command must contain one or more of the following action qualifiers:

/EX[TRACT][=file-specification]

/REC[OVER]

/RO[LLBACK]

/SH[OW][=show-option-list]

/[NO]V[ERIFY]

The MUPIP JOURNAL command must contain one (and only one) of the following direction qualifiers:

/BA[CKWARD]

/FO[RWARD]

The MUPIP JOURNAL command may optionally contain one or more of the following time qualifiers:

/A[FTER]=time

/BE[FORE]=time

/[NO]LOO[KBACK_LIMIT][=lookback-option-list]

/SI[NCE]=time

The MUPIP JOURNAL may optionally contain one of the following sequence number qualifiers:

/FET[CHRESYNC]=<port number>

/RES[YNC]=<jnl sequence number>

The MUPIP JOURNAL command may optionally contain one or more of the following control qualifiers:

/[NO]AP[PLY_AFTER_IMAGES]

/BR[OKENTRANS]=<extract file name>

/[NO]CHA[IN]

/[NO]CHE[CKTN]

/[NO]ER[ROR_LIMIT][=integer]

/FE[NCES]=fence-option

/FU[LL]

/[NO]IN[TERACTIVE]

/LOST[TRANS]=<extract file name>

/RED[IRECT]=file-pair-list

/VERB[OSE]

The MUPIP JOURNAL command may optionally contain one or more of the following selection qualifiers:

/G[LOBAL]=global-list

/ID=pid-list

/P[ROCESS]=process-name-list

/T[RANSACTION]=transaction-type

/U[SER]=user-list

MUPIP JOURNAL manipulates an inactive journal file that is available for exclusive (standalone) use. Journal files can be transcribed to tape. However, they are always produced on disk and they must be restored to disk for processing by MUPIP JOURNAL.

JOURNAL processing can be stopped by pressing <CTRL-C> or <CTRL-Y>. A JOURNAL command terminated abnormally by operator action or error produces an incomplete result. In this case, the resulting database may be corrupt. If a JOURNAL operation is stopped by mistake, reissuing the command produces the proper result for /RECOVER (or /ROLLBACK) /BACKWARD. For /RECOVER /FORWARD, restore the database from backup and reissue the command.

Journal File Selection Argument

Use this argument to indicate a comma-separated list of journal files to be used in the RECOVER (or ROLLBACK) operation. When you specify asterisk (*) for this argument, GT.M uses the journal files of the database files listed in the Global Directory that VMS logical variable gtm$gbldir points to. Currently, the only value accepted for this argument with /ROLLBACK qualifier is asterisk (*). This prevents incomplete input from being supplied to the /ROLLBACK operation. GT.M reports an error if any other value is supplied for /ROLLBACK.

The value asterisk (*) can be specified for both /FORWARD and /BACKWARD direction qualifiers. However, if a database does not have journaling enabled, MUPIP terminates with a journal file name not found error. /RECOVER accepts a list of journal files.

MUPIP JOURNAL /RECOVER /BACKWARD does not accept multiple generation journal file names. Note that, asterisk (*) cannot be specified with /REDIRECT control qualifier.

FIS strongly recommends the use of asterisk (*) argument for the journal file selection argument for /RECOVER and /ROLLBACK.

Journal Action Qualifiers

This section describes the journaling action qualifiers.

/EX[TRACT][=file-specification]

Specifies that JOURNAL should transfer the contents of one or more journal files to a single output file in a format intended for processing by an M program. For a description of /EXTRACT output record formats, refer to the Journal Extract Formats section later in this chapter.

/EXTRACT takes an optional argument, which provides an output file-specification. If the output file specification is not provided, MUPIP JOURNAL derives the output file specification of the extract file using the name of the first journal file that is processed in the forward processing phase and a file type of .mjf. Note that, if multiple journal names are specified in the command line the first journal specified might be different from the first journal processed in the forward phase. When /EXTRACT is specified with /RECOVER (or /ROLLBACK), the /JOURNAL command extracts all the journal records processed during a /RECOVER /FORWARD command or the forward phase of (/RECOVER or /ROLLBACK) /BACKWARD command.

When used independent of /RECOVER (or /ROLLBACK), /EXTRACT option can produce a result even though the database file does not exist, although it does try to access the database if it is available. If accessing the database errors out for some reason, the contents of the journal file can still be extracted by temporarily renaming or moving the database file.

If globals in the database have collation enabled, their collation information is stored in the global variable tree of the database and not in the journal file. A journal extract of such globals without the presence of the corresponding database will not be meaningful as the MUPIP JOURNAL command has no way of knowing just from the journal files if the globals are collated, and hence will extract them as if they were uncollated.

Note that, a broken transaction, if found, is extracted to a broken transaction file (refer to the section on /BROKENTRANS for details), and all future complete transactions are considered as lost transactions, and are extracted to a lost transaction file (refer to the section on /LOSTTRANS for details).

To avoid broken transaction or lost transaction processing and instead extract all journal records into one file, use the control qualifier /FENCES=NONE. It is strongly recommended not to use /FENCES=NONE if /RECOVER or /ROLLBACK is also specified.

/RECOVER

Instructs the JOURNAL command to replay database updates from the specified journal file into the appropriate database. /RECOVER initiates the central JOURNAL operation for non-replicated database. JOURNAL commands may specify /RECOVER alone, or with other action qualifiers except /ROLLBACK.

If a transaction is found with incomplete fence, it is considered broken. During forward phase of recovery, if a complete transaction (fenced or unfenced) is found after a broken transaction. /RECOVER increments the error count. If /ERRORLIMIT is reached, the complete transaction goes to lost transaction file, otherwise, it is applied to the database.

All broken and lost transactions are made available as the result of the /RECOVERY. They are written as journal extract format in two different text files. They are the broken transaction file and the lost transaction file. Refer to the sections on BROKENTRANS and LOSTTRANS control qualifier.

When performing JOURNAL /RECOVER with fences (FENCES="PROCESS" or FENCES="ALWAYS"), it is essential for the command to include all the journal files corresponding to the complete set of database files that make up the logical database. If the specified set of journals is incomplete, the recovery reports all transactions that included any missing region as broken. Typically, this means that the results of the recovery are unsatisfactory or even unusable.

MUPIP JOURNAL /RECOVER requires exclusive access to database files before recovery can occur. It keeps the exclusive access to the database files, which means that the database files become inaccessible during the time of recovery.

For direction qualifier /FORWARD, if the target database's current transaction number is less than first transaction number to be processed in the specified journal file for that region, /RECOVER attempts to include previous generation journal file(s) in its processing, unless the /NOCHAIN qualifier is specified. Following the successive previous links of journal files /RECOVER tries to include previous generations of journal files until the transaction number when the journal file was created is less than, or equal to that of the target database. /RECOVER issues one or more informational messages when it includes previous generation journal files.

If target database's current transaction number is not equal to the first transaction number of the earliest journal file to be processed for a region, /RECOVER exits with an error, unless /NOCHECKTN is used. If multiple journal files for a single region are specified in a MUPIP JOURNAL /RECOVER /FORWARD command it behaves as if /NOCHAIN was specified. If the journal files are not a complete set (for example mumps1.mjl, & mumps3.mjl were specified, with mumps2.mjl missing from the command line), MUPIP JOURNAL errors out since the journal files specified are discontinuous in terms of database transaction numbers. On the other hand, specifying just mumps3.mjl automatically includes mumps2.mjl and mumps1.mjl in the recovery.

Forward recovery ignores the current journaling state of the target database file. It disables journaling of the target database file, (if currently ENABLE and ON), while playing forward the database updates. However, it restores the journaling state of the database at the end of a successful recovery (if necessary), except when journaling is ENABLE'd and ON before the recovery. In the latter case, the journaling state at the end of a successful recovery, is switched to ENABLE and OFF. No journaling is performed for the logical updates to the database for JOURNAL /RECOVER /FORWARD.

For direction qualifier /BACKWARD, the target database file should be the same as when GT.M wrote the last complete transaction to the journal. Because the database may be in an indeterminate state due to a failure, exact checks for this match are not possible. If the target database has journaling DISABLE'd (or ENABLE, OFF), /RECOVER /BACKWARD exits with an error message.

If the target database has journaling ENABLE, ON, but the journal file name in database file header does not match the latest generation journal file name specified for that region, /RECOVER exits with an error.

During forward phase of JOURNAL /RECOVER /BACKWARD MUPIP journals the logical updates to the database. It also creates before images. It is always required to have journaling ENABLE'd and ON for /RECOVER /BACKWARD or /ROLLBACK.

If time qualifiers are not specified, MUPIP JOURNAL /RECOVER (or /ROLLBACK) /BACKWARD does an optimal recovery (processing records for the minimum period that is required to assure a proper recovery). /RECOVER /BACKWARD may need to include some previous generations of journal files based on time qualifier (implicit or explicit). It issues appropriate messages whenever such inclusions occur. At the end of backward recovery, the journaling state of the database stays ENABLE, ON.

When a database file is rolled back by /RECOVER /BACKWARD, the corresponding journal file is also rolled back so that the two are synchronized. /RECOVER /BACKWARD then creates a new journal file. If no forward play of journal records is neccessary, the newly created journal file stays empty and the database points to the new journal file. The values for journal allocation and extension in the new journal file, are copied over from the database. The autoswitchlimit value in the new journal file is the maximum of the autoswitchlimit values of all journal files from the latest generation journal file until the turnaround point journal file generation (turnaround point is the point in the journal file where backward processing stops and forward processing begins). The journal allocation/extension values in the new journal file are picked up from the earliest generation of the set of those journal files sharing the maximum autoswitchlimit value.

A prefix rolled_bak_ is added to the journal file, whose entire contents are eliminated (rolled back) by a /RECOVER /BACKWARD. These files are not used by GT.M after the MUPIP JOURNAL /RECOVER, and can be moved/deleted as needed.

/ROLLBACK

/ROLLBACK initiates the central JOURNAL operation for replicated databases. MUPIP JOURNAL commands may specify /ROLLBACK alone or with other action qualifiers, but not with /RECOVER. If you do not use the /FETCHRESYNC qualifier, the database rolls back to the last consistent state. Only asterisk (*) qualifier is allowed for the journal file selection, that is, /ROLLBACK does journal file selection by itself. MUPIP JOURNAL /ROLLBACK requires exclusive access to database files before recovery can occur. It keeps an exclusive access to the database files, that is, these database files are inaccessible during the /ROLLBACK operation.

/ROLLBACK /BACKWARD exits with an error message for following conditions:

  • Any database region corresponding to a journal file processed has replication state turned OFF. Note that, a configuration where there are replicated regions and at least one non-replicated-but-journaled region is not permitted by the replication source server. The source server errors out at startup on such a configuration without having set up the journal pool. Since all GT.M updates to replicated regions need the source server to have set up a journal pool, no updates are possible until the configuration is changed to have only replicated regions or non-replicated-and-non-journaled regions.

  • Any database region corresponding to a journal file identified by the command argument has journaling state DISABLE'd or ENABLE'd and OFF.

  • Any database region corresponding to a journal file has journal state ENABLE'd and ON, but the journal file name specified in the database file header is different than one identified by the command argument.

If a transaction is found with incomplete fence, it is considered broken. For the duration of the rollback, replication is turned OFF on all regions and turned back ON at the end of the rollback.

During the forward phase of rollback, if a complete transaction (fenced or unfenced) is found after a broken transaction, it is considered as a lost transaction. During forward phase of rollback, MUPIP journals the logical updates to the database. All broken and lost transactions are made available as a result of the rollback. These are written as journal extract format in two different text files.

When a database file is rolled back by /ROLLBACK, the corresponding journal file is also rolled back so that the two are synchronized. /ROLLBACK then creates a new journal file. If no forward play of journal records is necessary, the newly created journal file is empty and the database points to the new journal file. The journal allocation/extension/autoswitchlimit values in the new journal file is set in the way as described for /RECOVER /BACKWARD in the previous section under /RECOVER.

A prefix rolled_bak_ is added to the journal file, whose entire contents are eliminated by a /ROLLBACK. These files are not used by GT.M after the MUPIP JOURNAL /RECOVER, and can be moved/deleted as needed.

For /ROLLBACK the target database file should be the same as when GT.M wrote the last complete transaction to the journal.

If the /FETCHRESYNC or /RESYNC qualifiers are not specified, MUPIP does an optimal rollback.

[Note]

/FORWARD, /CHAIN, /CHECKTN, /REDIRECT and all other time qualifiers are not allowed with /ROLLBACK.

/SH[OW]=show-option-list

Specifies what information the JOURNAL command displays about a journal file. If /FORWARD direction qualifier is used with /SHOW and /RECOVER action qualifier is not specified, the entire journal file is processed. When /SHOW is specified with /RECOVER (or /ROLLBACK), the JOURNAL command considers all the journal files/records processed during a /RECOVER /FORWARD command or forward phase of a /RECOVER (or /ROLLBACK) /BACKWARD command. When used independent of /RECOVER (or /ROLLBACK), this option does not require database access.

The show-option-list includes (these are not case-sensitive):

AL[L]

H[EADER]

P[ROCESSES]

AC[TIVE_PROCESSES]

B[ROKEN_TRANSACTIONS]

S[TATISTICS]

The following list describes the /SHOW options.

  • AL[L]

    ALL displays every available type of information about the journal file. ALL is the default if the show-option-list is omitted. For additional information, refer to the descriptions of each of the other SHOW keywords.

  • AC[TIVE_PROCESSES]

    ACTIVE_PROCESSES displays all processes active at the end of the period specified implicitly or explicitly by the JOURNAL command time qualifiers.

  • B[ROKEN_TRANSACTIONS]

    BROKEN_TRANSACTIONS displays all processes that had incomplete fenced transactions at the end of the period covered by the JOURNAL command.

  • H[EADER]

    HEADER displays the journal file header information. If the MUPIP JOURNAL command includes only the /SHOW=HEADER action qualifier, only the journal file header (not the contents) is processed, irrespective of whether /BACKWARD or /FORWARD direction qualifier is specified.

    HEADER displays almost all the fields in the journal file header. The NODE field is printed upto a maximum of the first 12 characters. This information includes:

    Before-image journal                          ENABLED
    Journal file header size                         2048 [0x00000800]
    Virtual file size                                 100 [0x00000064] blocks
    Crash                                           FALSE
    Recover interrupted                             FALSE
    End of Data                                      7568 [0x00001D90]
    Prev Recovery End of Data                           0 [0x00000000]
    Journal Creation Time         18-JUN-2003 11:17:43
    Time of last update           18-JUN-2003 11:17:48
    Begin Transaction                                   1 [0x00000001]
    End Transaction                                   101 [0x00000065]
    Align size                                      65536 [0x00010000] bytes
    Epoch Interval                                            300
    Replication State                              CLOSED
    Updates Disabled on Secondary                   FALSE
    Jnlfile SwitchLimit                           8388600 [0x007FFFF8] blocks
    Jnlfile Allocation                                100 [0x00000064] blocks
    Jnlfile Extension                                 100 [0x00000064] blocks
    Maximum Physical Record Length                           1080 [0x00000438]
    Maximum Logical Record Length                    1064 [0x00000428]
    Turn Around Point Offset                            0 [0x00000000]
    Turn Around Point Time                                      0
    Start Region Sequence Number          0000000000000001 [0x0000000000000001]
    End Region Sequence Number            0000000000000000 [0x0000000000000000]
  • P[ROCESSES]

    PROCESSES displays all processes active during the period specified implicitly or explicitly by the JOURNAL command time qualifiers.

  • S[TATISTICS]

    STATISTICS displays a count of all journal record types processed during the period specified implicitly or explicitly by the JOURNAL command time qualifiers.

/[NO]V[ERIFY]

/VERIFY verifies a journal file for integrity. This qualifier cannot have a value. /VERIFY scans the files and checks if it is in legal form, if not, it terminates without affecting the database files.

/VERIFY when specified along with /FORWARD verifies the entire journal file For /NOVERIFY /FORWARD, only the tail of a journal file is verified for cross region integrity. In both cases, if /RECOVER is also specified, the forward play of journal records is done in a separate pass only after the verification pass is complete and error-free.

/VERIFY along with /BACKWARD verifies all journal records from the end of the journal file till the turn around point. When /VERIFY /BACKWARD is specified along with /RECOVER or /ROLLBACK, backward processing involves two passes, the first pass to do the verification until the turn around point, and the second pass to apply before image (PBLK) records.

When /NOVERIFY /BACKWARD is specified along with /RECOVER or /ROLLBACK, PBLKs are applied to the database in the same pass as the verification. This speeds up processing. But the disadvantage of this approach is that in the event of verification terminating in the middle of backward processing, there is no protection of cross-region integrity. FIS recommends the use of /VERIFY with /RECOVER or /ROLLBACK.

When used independent of /RECOVER (or /ROLLBACK), /[NO]VERIFY option does not need database access. The default is /VERIFY.

Journal Direction Qualifiers

The following two qualifiers control the journal processing direction:

/BACKWARD

Specifies that MUPIP JOURNAL processing should proceed from the end of the journal file. If the actions include /RECOVER, JOURNAL /BACKWARD restores before-images from the end-of the file back to an explicitly or implicitly specified point (the turn around point), before it reverses and processes database updates in the forward direction (the forward phase).

[Note]

/BACKWARD is incompatible with /FORWARD.

/FO[RWARD]

Specifies that MUPIP JOURNAL processing for the specified action qualifier should proceed from the beginning of the given journal file. When processing a /RECOVER action qualifier, in certain cases, MUPIP JOURNAL may need to go before the first record of the specified journal file, that is, it can start from a previous generation journal file (see /RECOVER section for details).

If multiple journal files are specified in the command line, /FORWARD sorts the journal files within each region based on creation time and processes them starting from the earliest journal file. Unless the /NOCHECKTN qualifier is specified, /FORWARD performs checks on journal files corresponding to each region to ensure they are contiguous, both in terms of time span, as well as, transaction number span. /FORWARD errors out if it detects a discontinuity.

[Note]

/FORWARD is incompatible with /BACKWARD and /ROLLBACK.

Journal Time Qualifiers

While DCL requires a colon (:) between the date and time, MUPIP requires a space. Journal qualifiers specifying time accept arguments in OpenVMS absolute or delta time format. Enclose time arguments in quotation marks (" ") and include all leading delimiters.

Absolute format is day-mon-yyyy hh:mm:ss, where (day) denotes the date of the month, (mon) indicates the abbreviated 3-letter month name (for example, Jan, Feb,..) and the year (yyyy) and hour (hh) are separated by a space. Absolute time may indicate today's date with "-- " before the hours.

Delta format is day hh:mm:ss, indicating the number of days, hours, minutes, and seconds; where the day and the hours (hh) are separated by a space. If delta time is less than a day, it must start with zero (0) followed by a space.

For information on how to specify the time, refer to the OpenVMS Programming Concepts Manual.

Delta time is always relative to the maximum time of the last record in all journal files specified by arguments to the MUPIP JOURNAL command.

[Note]

All time qualifiers are incompatible with /ROLLBACK.

The following section describes the time qualifiers in more detail:

/A[FTER]=time

Specifies reference time stamps in the journal and identifies the point after which JOURNAL starts processing in the journal file(s). This time qualifier applies to /FORWARD only.

If /AFTER= provides a time following the last time recorded in the journal file or following any /BEFORE= time, JOURNAL processing produces no result and a warning message is displayed. If /AFTER provides a time preceding the first time recorded in the journal file specified in the command line, and, previous generation journal file(s) exists for that journal file, then previous generation journal file(s) are not included for the processing. You must specify previous generation journal files explicitly in the command line in order for them to be considered.

Using /BEFORE with /AFTER restricts processing to a particular period of time in the journal file.

[Note]

/AFTER= is incompatible with /BACKWARD and all action qualifiers, except /EXTRACT, /SHOW, and /VERIFY.

/BE[FORE]=time

Specifies an ending time for any action /FORWARD or /BACKWARD. The time specified references time stamps in the journal files. If /BEFORE= specifies a time preceding the first time recorded in the journal file, or preceding any /AFTER= or /SINCE= time, JOURNAL processing produces no result, and a warning message is displayed.

If /BEFORE= time exceeds the last time recorded in journal files, JOURNAL processing effectively ignores the qualifier and terminates at the end of the journal file. By default, JOURNAL processing terminates at the end of the journal file.

[Note]

/BEFORE= is compatible with all other JOURNAL qualifiers except /ROLLBACK.

/[NO]LOO[KBACK_LIMIT][=lookback-option-list]

Specifies how far JOURNAL /BACKWARD processes past the turnaround point (the explicit or implicit point in journal file up to which /RECOVER proceeds backward before it reverses and processes database in forward direction), while attempting to resolve open transaction fences. This option is applicable only for transactions fenced with ZTSTART and ZTCOMMIT. For transaction fenced with TSTART and TCOMMIT, /RECOVER always resolves open transaction fences.

/LOOKBACK_LIMIT=options, include time and transaction counts. /NOLOOKBACK_LIMIT specifies that JOURNAL /BACKWARD can process all the way to the beginning of the journal file, if necessary, to resolve open transaction fences. /LOOKBACK_LIMIT= is incompatible with /FORWARD.

When /FENCES=NONE, JOURNAL processing ignores /LOOKBACK_LIMIT.

The /LOOKBACK_LIMIT options are:

  • TIME=time

    This limits LOOKBACK by a specified amount of delta or absolute journal time.

  • OPERATIONS=integer

    This limits LOOKBACK to the specified number of database transactions.

The TIME LOOKBACK option name and its value must be enclosed in quotes ("").

For example:

/lookback=(time="0 00:00:30")

When /LOOKBACK_LIMIT= specifies both options, they must be separated by a comma (,) and enclosed in paratheses (), for example:

/lookback=(time="0 00:00:30",operations=35)

When /LOOKBACK_LIMIT= specifies both options, the first limit reached terminates the LOOKBACK.

By default, MUPIP JOURNAL uses /LOOKBACK_LIMIT=\"TIME=0 00:05\" providing five minutes of journal time prior to /SINCE= to resolve open fences. A /LOOKBACK_LIMIT that specifies a limit much before the beginning of the earliest journal file acts as if /NOLOOKBACK_LIMIT was specified.

/SI[NCE]=time

Specifies a starting time for an action qualifier with /BACKWARD, that is, /SINCE specifies how far back in time JOURNAL /BACKWARD should process (from the end of the journal file), before starting the forward processing.

The time specified references time stamps in the journal files. If there are open fenced transactions when JOURNAL /BACKWARD locates the /SINCE= time, it continues processing backward to resolve them, unless the command also specifies /FENCES=NONE. If /SINCE= time exceeds the last time recorded in the journal files or, follows any /BEFORE=time, JOURNAL processing effectively ignores the qualifier, and displays a warning message.

By default, /SINCE= time is 0 00:00:00.

[Note]

/SINCE= is incompatible with /FORWARD.

Journal Sequence Number Qualifiers

These qualifiers are compatible only with /ROLLBACK.

/FET[CHRESYNC]=<port number>

/FETCHRESYNC is used to roll back to the transaction that is identified by a reference point (a JNL_SEQNO) fetched from the primary system. This command rolls back a former primary source server to the journal sequence number at which the then secondary source server (current primary) took over. The <port number> is the communication port number that the rollback command uses while fetching the reference point from the current primary source server.

Note that the <port number> for rollback is the same number used by the receiver server, with which the primary attempts to establish communication.

The reference point sent by the primary system is the RESYNC_SEQNO that the primary source server maintains. The database/journal files are rolled back to the earlier RESYNC_SEQNO (that is, the one received from primary source server , or the one maintained locally).

/RES[YNC]=<journal sequence number>

/RESYNC= is used to roll back to the transaction identified by the journal sequence number, only when the database/journal files need to be rolled back to a specific point. If you specify a journal sequence number that is greater than the last consistent state, the database/journal files are rolled back to the last consistent state. Under normal operating conditions, this qualifier is not needed. FIS strongly recommends relying on /FETCHRESYNC, and avoiding the use of the /RESYNC qualifier.

Journal Control Qualifiers

The following qualifiers control journal processing:

/[NO]AP[PLY_AFTER_IMAGE]

/APPLY_AFTER_IMAGE specifies that after image records (AIMG) be applied to the database as part of forward processing of backward recovery or rollback. AIMG are "snapshots" of the database updates captured by GTM immediately after the change caused by a DSE update. By default, during forward phase of backward recovery or rollback, AIMG records are applied to the database.

By default, /RECOVER /FORWARD does not apply AIMG record into the database. /APPLY_AFTER_IMAGE is compatible with /RECOVER, or /ROLLBACK action qualifiers only.

/BR[OKENTRANS]=<extract file>

/BROKENTRANS= is an optional qualifier for /ROLLBACK, /RECOVER and /EXTRACT. If this is not specified and a broken transaction file creation is necessary, MUPIP JOURNAL creates one using the name of the current journal file being processed with a .broken extension.

Note that, if selection qualifiers are specified, the broken transaction determination (and therefore lost transaction determination as well) is done based on the journal file that is filtered by the selection qualifiers. This means that a transaction’s journal records may be considered complete or broken or lost, depending on the nature of the selection qualifiers. Using /FENCES=NONE along with the selection qualifiers will result in every journal record to be considered complete and hence prevent broken or lost transaction processing.

/[NO]CHA[IN]

/CHAIN specifies that it is allowed for JOURNAL processing to include previous generations of journal files for /FORWARD qualifier. If JOURNAL /RECOVER needs to process previous generation journal file(s) and /NOCHAIN is specified, MUPIP JOURNAL exits with an error.

/CHAIN is the default. This qualifier is compatible with /FORWARD direction qualifier only.

/[NO]CHE[CKTN]

/CHECKTN specifies that JOURNAL /FORWARD must verify for each region that the begin transaction number of the earliest journal file to be processed for that region is same as the current transaction in the database file and that the end transaction number of every journal file is equal to the the begin transaction number of the next generation journal file for a given region.

/CHECKTN is incompatible with the /BACKWARD qualifier and also /ROLLBACK command. By default, JOURNAL /FORWARD uses /CHECKTN.

/[NO]ER[ROR_LIMIT][=integer]

Specifies the number of errors that MUPIP JOURNAL processing accepts. When the number of errors exceeds the /ERROR_LIMIT, the /INTERACTIVE qualifier determines whether JOURNAL processing halts or defers to the operator. /NOERROR_LIMIT prevents MUPIP JOURNAL from stopping because of errors. Journal processing continues until it reaches the end of the journal file, regardless of the number of errors.

Note that, /NOERROR_LIMIT is not the same as /ERROR_LIMIT=0.

By default, MUPIP JOURNAL uses /ERROR_LIMIT=0, causing the first error to initiate the appropriate error action. In case of a crash there could be some incomplete journal records at the end of a journal file. MUPIP JOURNAL does not consider these as errors. In addition, fenced transactions that are broken are not considered as errors.

During the forward phase of recovery, if a broken transaction is found, all the logical records processed afterwards are considered suspect. If a complete transaction is found after any broken transactions, MUPIP JOURNAL /RECOVER increments the error count and, if it is less than the error limit, it is applied to the database. Otherwise, it is treated as a lost transaction and extracted. If a complete transaction is found after any broken transactions, MUPIP JOURNAL /ROLLBACK treats it as a lost transaction and extracts it irrespective of the error limit.

If MUPIP JOURNAL needs to increment error count during its processing, a warning message is issued for every error encountered except in the following cases when the error count is incremented but no warning message is displayed.

  • When a complete transaction is found after a broken transaction

  • When /EXTRACT /FULL encounters erros

If MUPIP JOURNAL completes successfully with a non-zero value of error count, the return status is not a success, but a warning.

/FE[NCES][=fence-option]

Specifies how JOURNAL processes fenced transactions. Fenced transactions are logical transactions made up of database updates preceded by a TSTART or ZTSTART command and followed, respectively, by a TCOMMIT or ZTCOMMIT command. All updates between a TSTART or ZTSTART and a TCOMMIT or ZTCOMMIT are designed to occur together so that after journal recovery the database contains either all the updates corresponding to a fenced transaction, or none of them.

The argument values for /FENCES option for MUPIP /RECOVER are not case-sensitive.

The fence options are:

  • NONE

    This causes MUPIP JOURNAL to apply all individual updates as if transaction fences did not exist. Note that, this means a SET/KILL within a TP or ZTP transaction would be played as if it was an unfenced SET/KILL. This may cause the database and new journals created (if /BACKWARD is specified), to be different from the state before the recovery took place.

  • ALWAYS

    This causes MUPIP JOURNAL to treat any unfenced or improperly fenced updates as broken transactions.

  • PROCESS

    This causes MUPIP JOURNAL to accept unfenced database updates, and also to observe fences when they appear, generating broken transaction files in the case of a TSTART with no corresponding TCOMMIT, or a ZTSTART with no corresponding ZTCOMMIT. It also generates broken transactions if a multi-region transaction with TSTART and TCOMMIT expects N regions to participate, but the number of TSTART/TCOMMIT pairs found is less than N. The same happens for multi-region ZTSTART and ZTCOMMIT.

By default, MUPIP JOURNAL uses /FENCES=PROCESS.

/FU[LL]

/FULL when used with /EXTRACT qualifier specifies that all journal records be extracted. A journal file's contents can be rolled back in case of backward recovery or rollback (refer to RECOVER or ROLLBACK sections for details) in order to keep the database and journal in sync. This is achieved not by truncating the contents of the journal file but instead setting a field in the journal file header, which shows up as "Prev Recovery End of Data" in a MUPIP JOURNAL /SHOW=HEADER output, to indicate the end of the journal file before rolling back and setting another field in the file header to indicate the new end of the journal file (this field shows up as "End of Data" in a MUPIP JOURNAL /SHOW=HEADER output). Once a journal file's contents are rolled back, all future MUPIP JOURNAL commands (including /EXTRACT) operate on the rolled back journal file only. But if /FULL is specified along with /EXTRACT, the entire journal file contents (including those records that were rolled back) are extracted. This qualifier is to be used only as a diagnostic tool and not in normal operation.

/FULL qualifier is compatible with /EXTRACT only.

/[NO]IN[TERACTIVE]

Specifies whether, for each error over the /ERROR_LIMIT, JOURNAL processing prompts the invoking operator for a response to control continuation of processing. If the operator responds that processing should not continue, the MUPIP JOURNAL command terminates.

/NOINTERACTIVE terminates the journal processing as soon as the MUPIP JOURNAL command generates the number of errors specified in /ERROR_LIMIT.

This qualifier is applicable only to interactive mode or terminal mode. The default is /INTERACTIVE.

/LOST[TRANS]=<extract file>

/LOSTTRANS is an optional qualifier for /RECOVER, /ROLLBACK and /EXTRACT. If this is not specified and a lost transaction file creation is necessary, MUPIP JOURNAL creates one using the name of the current journal file being processed with a .lost extension.

Any complete transactions after a broken transaction is considered a lost transaction. They are written into the lost transaction file. For /RECOVER it might be considered as good transaction and applied to the database, if /ERROR_LIMIT qualifier allows it to do so.

Note that, if selection qualifiers are specified, the broken transaction determination (and therefore lost transaction determination as well) is done based on the journal file that is filtered by the selection qualifiers. This means that a transaction’s journal records may be considered complete or broken or lost, depending on the nature of the selection qualifiers. Using /FENCES=NONE along with the selection qualifiers will result in every journal record to be considered complete and hence prevent broken or lost transaction processing.

In the case of a replicated database, lost transaction can have an additional cause. If failover occurs (that is, the primary source server, A, fails and the secondary source server, B, assumes the primary system's role), some transactions committed to A's database may not be reflected in B's database. Before the former primary becomes the new secondary these transactions must be rolled back. These transactions are known as "lost transactions". Note that these are complete transactions and different from a broken transaction. MUPIP JOURNAL /ROLLBACK stores extracted lost transactions in the extract-file specified by this qualifier. The starting point for the search for lost transactions is the journal sequence number obtained from the primary source server in the /FETCHRESYNC operation.

/RED[IRECT]=file-pair-list

Specifies that JOURNAL /RECOVER replay the journal file to a database different than the one for which it was created. Use this qualifier to create or maintain databases for training or testing.

This qualifier applies to /RECOVER action and /FORWARD direction qualifier only. JOURNAL rejects /REDIRECT unless it appears with /RECOVER.

The file-pair-list consists of one or more pairs of file-specification separated by commas (,). The pairs are separated by an equal sign in the form:

(old-file-specification=new-file-specification)

where the old file-specification identifies the original database file and the new file-specification file-name identifies the target of the /RECOVER. The old-file-specification can always be determined using /SHOW. When /REDIRECT specifies only one file pair, the parentheses are optional.

By default, JOURNAL directs /RECOVER to the database file from which the journal was made. /REDIRECT is not compatible with /ROLLBACK.

Example:

$ MUPIP JOURNAL /REC /FOR /RED=(BGDBB.DAT=TEST.DAT) BGDBB.MJL

This JOURNAL command does a forward recovery that /REDIRECTs the updates in BGDBB.MJL from BGDBB.DAT to TEST.DAT.

This JOURNAL command does a forward recovery that -REDIRECTs the updates in bgdbb.mjl from bgdbb.dat to test.dat.

/VERB[OSE]

/VERBOSE is an optional qualifier that makes MUPIP JOURNAL print out verbose output in the course of processing. It is not negatable and it is set to OFF by default.

Journal Selection Qualifiers

Journal Selection Qualifiers are compatible with /EXTRACT and /SHOW operations only. This is because most applications are not constructed to safely remove a subset of transactions based on criteria that is exterior to the application design. To exclude transactions from a recovery based on some selection criteria, the methodology is to /EXTRACT the records, and then reapply them through application logic rather than by journal recovery. This approach permits the application logic to appropriately handle any interactions between the removed and the retained transactions. Note that, selection qualifiers might result in only a subset of a fenced transaction's journal records to be extracted (for example, a TSTART record may not be extracted because the first update in that transaction was filtered out by a selection qualifier, while the corresponding TCOMMIT record may get extracted). This can cause a fenced transaction to seem broken when actually it is not.

The following qualifiers control the selection criteria for journal processing.

[Note]

Except for /TRANSACTION, all qualifiers allow for specifying a comma (,) seperated list of values.

/G[LOBAL]=global-list

Specifies globals for MUPIP JOURNAL to include or exclude from processing. You may find this qualifier useful for extracting and analyzing specific data.

The global-list contains one or more global-names (without subscripts) preceded by a caret symbol (^). To include more than one global use one of the following syntaxes.

$ MUPIP JOURNAL /FORW /GLOB="^A*,^C" MUMPS.MJL
or
$ MUPIP JOURNAL /FORW /GLOB="(^A*,^C)" MUMPS.MJL

The names may include the asterisk (*) wildcard. That is, /GLOBAL="^A*" selects all global variables with names starting with A. The entire list or each name may optionally be preceded by a minus sign (-) , requiring JOURNAL to exclude database updates to the specified global(s). When the global-list with a MUPIP JOURNAL /GLOBAL does not start with a minus sign (-) , JOURNAL processes only the explicitly named globals. By default, JOURNAL processes all globals.

To specify subscripts, using /GLOBAL="^A(1)" results in all keys under the ^A(1) tree to be included, that is, it is equivalent to using /GLOBAL="^A(1,*)". An asterisk (*) or a percent (%) anywhere in the global specification is permitted. Percent (%) matches any character, and asterisk (*) matches any string (possibly zero length too). The asterisk (*) or percent (%) specification can be used for /USER and /PROCESS qualifiers too.

Example:

To extract all ^GBL* except for ^GBLTMP:

MUPIP JOURNAL /EXTRACT /GLOBAL="^GBL*,-^GBLTMP" /FOR MUMPS.MJL

To extract all ^GBL except for ^GBL(1,"TMP"):

MUPIP JOURNAL /EXTRACT /GLOBAL="^GBL,-^GBL(1,""TMP"")" /FOR MUMPS.MJL

An INVGLOBALQUAL error is issued along with the error offset in the command line, whenever a parse error of the global qualifier string is encountered.

/ID=pid-list

Specifies that JOURNAL processing include or exclude database updates generated by one or more processes, identified by process identification numbers (PIDs). The entire list or each PID may optionally be preceded by a minus sign (-) , requiring JOURNAL to exclude database updates initiated by the specified PID. You may use this qualifier for trouble shooting or analyzing data.

By default, JOURNAL processes database updates regardless of the PID that initiated it.

/P[ROCESS]=process-name-list

Specifies that JOURNAL processing include or exclude database updates generated by one or more processes, identified by process names. The entire list or each process name may optionally be preceded by a minus sign (-), requiring JOURNAL to exclude database updates initiated by the specified process name. This qualifier is useful in troubleshooting or analyzing data.

By default, JOURNAL processes database updates regardless of the process by which they were initiated. The asterisk (*) or percent (%) specification can be used for /PROCESS qualifier too. Percent (%) matches any character, and asterisk (*) matches any string (possibly zero length too).

/T[RANSACTION]=transaction-type

Specifies transaction-types for JOURNAL to include or exclude from processing. For example, you may use this qualifier to report only on KILL operations to locate possible causes for missing data.

The transaction-types are SET and KILL and can be negated. These types correspond to the M commands of the same names. When the transaction-type with a JOURNAL /TRANSACTION is not negated, JOURNAL processes only transactions of the type named (for example, /TRANSACTION=KILL), whereas if it is negated, JOURNAL does not process transactions of the type named (for exmaple, /TRANSACTION=NOKILL).

By default, JOURNAL processes transactions, regardless of its type.

/U[SER]=user-list

Specifies that MUPIP JOURNAL processing include or exclude database updates generated by one or more users. You can use this qualifier to audit the actions of a particular user. The user-list contains names of one or more users. Indicate multiple users by separating the names with commas (,). The names may include the wildcard asterisk (*). The entire list or each name may optionally be preceded by a minus sign (-) tilda sign (~), requiring JOURNAL to exclude database updates initiated by the specified user(s). When the user-list with a JOURNAL /USER does not start with a minus sign (-) , JOURNAL processes only those database updates, which are generated by explicitly named users. The asterisk (*) or percent (%) specification can be used for /USER qualifier. Percent (%) matches any character, and asterisk (*) matches any string (possibly zero length too).

By default, JOURNAL processes database updates regardless of the user who initiated them.

Journal Qualifier Rules & Compatibilities

Although the description against each qualifier listed the incompatibilities it shares with other qualifiers, all these are summarised in this section. In the description below, if X is compatible only with a set of Y action qualifiers, this means whenever X is specified in the command line, at least one of the Y action qualifiers must also be specified in the same command line or else it is an error.

  • file list must be asterisk (*) for /ROLLBACK

  • file list must not be asterisk (*) for /REDIRECT

  • /FORWARD is incompatible with /BACKWARD, /ROLLBACK, /SINCE, and /LOOKBACK_LIMIT

  • /BACKWARD is incompatible with /FORWARD, /AFTER, /CHECKTN, /NOCHAIN, and /REDIRECT

  • /ROLLBACK action qualifier is incompatible with /RECOVER action qualifier

  • /AFTER is incompatible with /RECOVER, and /ROLLBACK action qualifiers

  • /FETCHRESYNC or /RESYNC are compatible only with /ROLLBACK action qualifier

  • /LOSTTRANS is compatible only with /RECOVER, /ROLLBACK, or /EXTRACT action qualifiers

  • /BROKENTRANS is compatible only with /RECOVER, /ROLLBACK, or /EXTRACT action qualifiers

  • /REDIRECT is compatible only with /RECOVER action qualifier

  • /APPLY_AFTER_IMAGE is compatible only with /RECOVER, or /ROLLBACK action qualifiers

  • /DETAIL is compatible only with /EXTRACT action qualifier

  • /FULL is compatible only with /EXTRACT, /SHOW, or /VERIFY action qualifiers

  • Journal time qualifiers are incompatible with /ROLLBACK action qualifier

  • Journal selection qualifiers are incompatible with /RECOVER, /ROLLBACK, and /VERIFY action qualifiers

Journal Extract Formats

Journal EXTRACT files always start with a label. For the current release of GT.M, the label is GDSJEX01 for a simple journal extract file. This label is necessary to identify the format of the file.

Label is followed by journal record extracts. /EXTRACT output records are constructed of fields or pieces delimited by a back slash (\). The first piece of an /EXTRACT output record contains a two-digit decimal transaction record type (for example, 01 for a process initialization record). The second piece contains the full date and time of the operation, represented in the $HOROLOG format. The third piece contains the database transaction number at the time of writing this journal record. The fourth piece contains the process ID (PID) of the process that performed the operation, represented as a decimal number. The remainder of the record depends on the record type. The fields described as "database transaction number" contain a GT.M assigned number, which uniquely identifies the transaction within the time covered by the journal file.

Records of type SET, KILL, ZKILL, TSTART, and TCOMMIT include the token_seq as part of the output. It is the sixth field in the output of the journal record extract. When replication is in use, token_seq is a journal sequence number (jsnum) that uniquely identifies each transaction (for more information on journal sequence number, see Chapter 7: “Database Replication). When replication is not in use and the transaction is a TP or ZTP transaction, token_seq is an 8-byte token that uniquely identifies the entire TP or ZTP transaction. For non-replicated, non-TP, and non-ZTP journal records, token_seq is zero (0).

Type 01 - Process Initialization Record

A Type 01 record indicates a process/image-initiated update into the current journal file for the first time. The format for a process initialization record is:

01\time\tnum\pid\nnam\unam\term\mode\logintime\image_count\pname\clntpid\clntnnam\clntunam\clntterm\clntmode\clntlogintime\clntimage_count\clntpname

where:

time

full time and date in $HOROLOG format

tnum

database transaction number

pid

process ID

nnam

node name (upto first 20 characters are extracted)

unam

user name

term

terminal name

clntpid

pid of the client (in case of a GT.CM server update), zero (0) otherwise

clntnnam

node name of the client process (in case of GT.CM)

clntunam

user name of the client process (in case of GT.CM)

clntterm

terminal name of the client process (in case of GT.CM)

mode

prints if it is interactive or batch, or other

logintime

login time in $HOROLOG format

image_count

image count

pname process

name

clntmode

mode of client process (in case of GT.CM)

clntlogintime

login time of client process (in case of GT.CM)

clntimage_count

image count of client process (in case of GT.CM)

clntpname

client process name (in case of GT.CM)

Type 02 - Process Termintation Record

A Type 02 record indicates a process/image dropped interest in the current journal file. The format for a process termination record is:

02\time\tnum\pid\clntpid

where:

time

full time and date

tnum

database transaction number

pid

process ID

clntpid

pid of client process (in case of a GT.CM server update), zero (0) otherwise

Type 03 - End of File Record

A Type 03 record indicates all GT.M images dropped interest in this journal file and the journal file was closed normally. The format for an end-of-file record is:

03\time\tnum\pid\clntpid\jsnum

where:

time

full time and date

tnum

database transaction number

pid

process ID

clntpid

pid of client process (in case of GT.CM server update), zero (0) otherwise

jsnum

journal sequence number

Type 04 - Kill Record

A Type 04 record indicates a database update caused by a KILL command. The format for a KILL record is:

04\time\tnum\pid\clntpid\token_seq\node

where:

time

full time and date

tnum

database transaction number

pid

process ID

clntpid

pid of client process (in case of GT.CM server update), zero (0) otherwise

token_seq

jsnum for replication, token if no replication and fenced, zero (0) otherwise

node

an M node reference in external format

Type 05 - Set Record

A Type 05 record indicates a database update caused by a SET command. The format for a SET record is:

05\time\tnum\pid\clntpid\token_seq\sarg

where:

time

full time and date

tnum

database transaction number

pid

process ID

clntpid

pid of client process (in case of GT.CM server update), zero (0) otherwise

token_seq

jsnum for replication, token if no replication and fenced, zero (0) otherwise

sarg

an M set argument

[Note]

An M SET argument has a node reference followed by an equal sign (=) and M data string expression.

Type 06 - ZTransaction Start Record

A Type 06 record indicates a ZTSTART command. The format for a transaction start record is:

06\time\tnum\pid\clntpid\token\jsnum

where:

time

full time and date

tnum

database transaction number

pid

process ID

clntpid

pid of client process (in case of GT.CM server update), zero (0) otherwise

token

8-byte token

jsnum

journal sequence number for replication, zero (0) otherwise

Type 07 - ZTransaction Commit Record

A Type 07 record indicates a ZTCOMMIT command. The format for a transaction commit record is:

07\time\tnum\pid\clntpid\token\jsnum\part

where:

time

full time and date

tnum

database transaction number

pid

process ID

clntpid

pid of client process (in case of GT.CM server update), zero (0) otherwise

token

8-byte token

jsnum

journal sequence number for replication, zero (0) otherwise

part

number of different regions involved in this fenced transaction

Type 08 - Transaction Start Record

A Type 08 record indicates a TSTART command. The format for a transaction start record is:

08\time\tnum\pid\clntpid\token_seq

where:

time

full time and date

tnum

database transaction number

pid

process ID

clntpid

pid of client process (in case of GT.CM server update), zero (0) otherwise

token_seq

jsnum for replication, token if no replication

Type 09 - TRANSACTION COMMIT Record

A Type 09 record indicates a TCOMMIT command. The format for a transaction commit record is:

09\time\tnum\pid\clntpid\token_seq\part\tid

where:

time

full time and date

tnum

database transaction number

pid

process ID

clntpid

pid of client process (in case of a GT.CM server update), zero (0) otherwise

token_seq

jsnum for replication, token if no replication

part

number of different regions involved in the transaction

tid

transaction ID string (BATCH, or ONLINE, and so on) specified in the corresponding TSTART command

Type 10 - ZKILL Record

A Type 10 record indicates a database update caused by a ZKILL command. The format for a ZKILL record is:

10\time\tnum\pid\clntpid\token_seq\node

where:

time

full time/date

tnum

database transaction number

pid

process ID

clntpid

pid of client process (in case of GT.CM server update), zero (0) otherwise

token_seq

jsnum for replication, token if no replication and fenced, zero (0) otherwise

node

an M node reference in external format

Example of Extract file

This section details a sample journal extract file where the only update was an M SET operation of the global variable ^x to the value 1.

GDSJEX01
01\59372,47556\1\559940242\GTMNODE\GTMUSER\TNA17:\5\59366,40662\76\GTMUSER\0\\\\0\48229,86101\0\
02\59372,47556\1\559940242\0
01\59372,47565\1\559940242\GTMNODE\GTMUSER\TNA17:\5\59366,40662\77\GTMUSER\0\\\\0\48229,86101\0\
05\59372,47565\1\559940242\0\0\^x="1"
02\59372,47566\2\559940242\0
03\59372,47566\2\559940242\0\0