GT.M V5.3-001A — 64-bits on x86_64 GNU/Linux and IBM pSeries AIX

The Linux version of GT.M can now utilize O_DIRECT support for journal files. The O_DIRECT option is made available by using the sync_io option to the MUPIP SET -JOURNAL command just as for the other platforms that support direct I/O. [Linux] (C9D10-002409)

GT.M now appropriately handles TP transactions that include multiple global directories (due to SET $ZGBLdir or extended references) having multiple regions that map to the same database file. Previous versions could, in rare cases, terminate the GT.M process abnormally with TPFAIL or SIG-11 errors (ACCVIO in VMS) and damage the database if TP transactions accessed global variables in such a database files via different global directories.

If you use the VIEW "NOISOLATION" command to turn NOISOLATION characteristics ON for a global name in one global directory and turn it OFF for the same name in a different global directory and if the two global directories map the global name to the same database file, then GT.M leaves NOISOLATION OFF if any path specifies OFF, irrespective of the order of the specifying VIEW command. (C9E04-002596)

After changing $ZGBLDIR, $VIEW() now properly reflects the current value of $ZGBLDIR even with no intervening global variable access. Previously this sequence could give an incorrect answer, or a memory access violation with the workaround being to perform an intervening global variable access. (C9H09-002900)

The GT.M database critical section lock wakeup mechanism has been enhanced to use memory semaphores. Previous versions used a socket-based scheme for wakeup. In some cases this led to processes being blocked trying to wake other processes up. This is no longer an issue in the new scheme. Since memory semaphores were introduced in 2.6 kernels, running GT.M on 2.4 kernels now requires a kernel which has had memory semaphores back-ported from 2.6. Note that contemporary releases of major Linux distributions use 2.6 kernels. [Linux] (C9H10-002918).

GT.M now avoids restarting a TP transaction in some additional cases where the global variable it is planning on updating has NOISOLATION turned ON even though it has been concurrently updated thereby improving GT.M transaction throughput rates. (C9I01-002938)

TP transactions now do not sleep between TP restarts thereby improving GT.M transaction throughput rates. Previously they used to sleep for a short time between every restart potentially resulting in inefficient use of system resources. (C9I01-002939)

Using TSTART with no prior global reference now works if you precede it with a SET $ZGBLDIR command and no intervening database references. In previous GT.M versions, such an unusual usage would terminate a GT.M process abnormally with SIG-11 (ACCVIO in VMS) errors. (C9I02-002963)

GT.M now closes journaling and continues with database updates even if:

This enhancement eliminates the need to retry writing to journal file and in doing so saves CPU time.

In previous versions (starting with V5.1-000), if a process was holding the critical section lock on the database file:

If the process was not holding the critical section lock, these previous versions used to indefinitely retry writing to the journal file without any sleep loop and consume CPU time. This misfeature was introduced in V5.1-000 as part of C9D11-002447 and fixed in V5.2-001 as part of S9H02-002643 . (C9I03-002964)

ZWRITE now preserves the representation of global variable numeric subscripts in exponential ("E") form. Previously large subscript values (>1E46) in exponential format [represented as strings] triggered a NUMOFLOW error. (S9E08-002478)

A new message, GTMSECSHRTMPPATH, has been added to help detect the situation where not all of the users of a GT.M instance have the same value for gtm_tmp. This message is issued in conjunction with the GTMSECSHRSRVF message when a client is unable to communicate with gtmsecshr, and with the GTMSECSHRSTART message when a new gtmsecshr is started and detects an existing gtmsecshr. In both cases the GTMSECSHRTMPPATH message displays the directory used for socket files that GT.M uses to communicate with gtmsecshr.

The directory for socket files used to communicate with gtmsecshr, either defaulting to /tmp or taken from the environment variable gtm_tmp, ensures all processes using the same GT.M instance can properly communicate with gtmsecshr. For proper operation, all users of an instance of GT.M must have the same value for gtm_tmp, including the case of it not being defined (defaults to /tmp).

The gtmsecshr process is started by the first client that needs its services and client is unable to communicate with the gtmsecshr process (normally, this means that the gtmsecshr process does not exist). The gtmsecshr process inherits the directory for socket files from the client that started it (via the gtm_tmp environment variable).

gtmsecshr is a long running process. Once it has been initiated, it waits an hour from the last time it has received a service request before it automatically shuts down. Consequently, it is unusual to see multiple instances of a client being unable to communicate with gtmsecshr within a short period of time. It is also unusual to see the gtmsecshr process shutting down because another one exists already. If you encounter either of these situations, please see the action section of the GTMSECSHRPATH message.

If you have poor performance, this may be the cause. Examine the syslog for these messages.

Below is an example of syslog messages showing this situation. Notice the multiple failed attempts to communicate with gtmsecshr (a few seconds apart by the same client process) and corresponding failed start attempts (because gtmsecshr was already running).

This enhancement helps you troubleshoot improperly configured GT.M installations. (S9H11-002670)

Jan 23 17:17:34 host1 GTM[18950]: %GTM-E-GTMSECSHRSRVF, Client - 18950 : Attempt to service request failed, %GTM-I-TEXT, sendto to gtmsecshr failed, %SYSTEM-E-ENO2, No such file or directory -- generated from 0x00282B3E.
Jan 23 17:17:34 host1 GTM[18950]: %GTM-I-GTMSECSHRTMPPATH, gtmsecshr path is /secshrpath/two, %GTM-I-TEXT, (from $gtm_tmp) -- generated from 0x00282BAD.
Jan 23 17:17:34 host1 GTM[18960]: %GTM-F-GTMSECSHRSTART, Server - 18960 : gtmsecshr failed to startup, %GTM-I-TEXT, server already running, %SYSTEM-E-ENO11, Resource temporarily unavailable -- generated from 0x0804D275.
Jan 23 17:17:34 host1 GTM[18960]: %GTM-I-GTMSECSHRTMPPATH, gtmsecshr path is /secshrpath/two, %GTM-I-TEXT, (from $gtm_tmp) -- generated from 0x0804D2E4.
Jan 23 17:17:37 host1 GTM[18950]: %GTM-E-GTMSECSHRSRVF, Client - 18950 : Attempt to service request failed, %GTM-I-TEXT, sendto to gtmsecshr failed, %SYSTEM-E-ENO2, No such file or directory -- generated from 0x00282B3E.
Jan 23 17:17:37 host1 GTM[18950]: %GTM-I-GTMSECSHRTMPPATH, gtmsecshr path is /secshrpath/two, %GTM-I-TEXT, (from $gtm_tmp) -- generated from 0x00282BAD.
Jan 23 17:17:37 host1 GTM[18962]: %GTM-F-GTMSECSHRSTART, Server - 18962 : gtmsecshr failed to startup, %GTM-I-TEXT, server already running, %SYSTEM-E-ENO11, Resource temporarily unavailable -- generated from 0x0804D275.
Jan 23 17:17:37 host1 GTM[18962]: %GTM-I-GTMSECSHRTMPPATH, gtmsecshr path is /secshrpath/two, %GTM-I-TEXT, (from $gtm_tmp) -- generated from 0x0804D2E4.
Jan 23 17:17:40 host1 GTM[18950]: %GTM-E-GTMSECSHRSRVF, Client - 18950 : Attempt to service request failed, %GTM-I-TEXT, sendto to gtmsecshr failed, %SYSTEM-E-ENO2, No such file or directory -- generated from 0x00282B3E.
Jan 23 17:17:40 host1 GTM[18950]: %GTM-I-GTMSECSHRTMPPATH, gtmsecshr path is /secshrpath/two, %GTM-I-TEXT, (from $gtm_tmp) -- generated from 0x00282BAD.
Jan 23 17:17:40 host1 GTM[18964]: %GTM-F-GTMSECSHRSTART, Server - 18964 : gtmsecshr failed to startup, %GTM-I-TEXT, server already running, %SYSTEM-E-ENO11, Resource temporarily unavailable -- generated from 0x0804D275.
Jan 23 17:17:40 host1 GTM[18964]: %GTM-I-GTMSECSHRTMPPATH, gtmsecshr path is /secshrpath/two, %GTM-I-TEXT, (from $gtm_tmp) -- generated from 0x0804D2E4.

GT.M now handles name indirection and call-ins in an a wholesome way. In V5.3-001, an inconsistency in handling these operations in NOUNDEF mode could cause various misfortunes including out of memory conditions and segmentation faults (SIG-11/ACCVIO). (S9I03-002674)

GT.M now suppresses STACKCRIT errors in the current frame at the time of the error. Previously, GT.M suppressed STACKCRIT errors in stack levels invoked because of an error, but this did not cover the GOTO implicit in $ETRAP processing or any explicit GOTO or ZGOTO in a $ZTRAP that placed control at the original stack level of the error. (S9I03-002676)

GT.M now signals the STACKCRIT error when a process has 16K stack space left available. This gives processes more opportunity to recover and trim the stack using QUIT, ZGOTO, or HALT. Previous versions used to display the STACKCRIT message when there was 1K stack space left. [UNIX] (S9I03-002676)

GT.M introduces a new environment variable gtm_noundef. If it is defined, and evaluates to a non-zero integer, the string "TRUE", or the string "YES" (or any case independent leading substrings thereof), then GT.M, at process startup, treats undefined variables as having an implicit value of an empty string. Otherwise, GT.M triggers an error on an attempt to use the value of an undefined variable. Previously the only way to alter this behavior of undefined variables was to use the UNDEF or NOUNDEF arguments of the VIEW command.

	To establish the behavior of undefined variables on OpenVMS, set "GTM$UNDEF_INHIBIT == 1" in GTM$DEFAULTS.M64 to treat undefined variables as having a null value or set "GTM$UNDEF_INHIBIT == 0" to trigger an error on an attempt to use the value of an undefined variable.

Running with VIEW "NOUNDEF" no longer has the disconcerting result of a test of a value implicitly creating (with a null value) previous undefined local unsubscripted variables. Now, while the variables still appear to have a null value in NOUNDEF mode, they remain undefined and will not show up in the output of ZWRITE, and functions such as $DATA() and $ORDER().

When running in VIEW "UNDEF" mode, $STACK(X,Y) where the variable Y is undefined now causes an UNDEF error to be issued by GT.M at runtime. Previous versions used to incorrectly issue a INVSTACODE error.

When running in VIEW "UNDEF" mode, JOB @LABEL+OFFSET^@ROUTINE where any of the variables LABEL, OFFSET or ROUTINE are undefined now causes an UNDEF error to be issued by GT.M at runtime. Previous versions used to incorrectly issue a JOBFAIL error. (C9B03-001664)

GT.M now handles an error while $ZTRAP is active (a recursive error) in a similar fashion to an error while $ETRAP is active -- $ZTRAP now returns to the previous stack level and does not enter the direct mode. $ZTRAP creates an implicit additional frame, and therefore GT.M clears the current $ZTRAP to avoid indefinite recursion. GT.M invokes any $ETRAP or $ZTRAP handler that it encounters during unwinding the stack. If GT.M cannot locate any valid handler, it eventually terminates the process or, if invoked from direct mode, returns to direct mode. This behavior means that production environments should always start with a mumps -run. In prior versions, if GT.M could not locate a valid handler during $ZTRAP error processing, it would enter direct mode as a last resort. Although this behavior may be appropriate for some development environments, it did not match many production requirements.

GT.M now handles an error in an EXCEPTION or ZBREAK action by triggering the current $ETRAP/$ZTRAP handler. In prior versions, GT.M would enter direct mode in those circumstances.

GT.M now handles a compilation error in the SET of a $ZSTEP action by triggering the current $ETRAP/$ZTRAP handler. In prior versions, GT.M ignored such errors. Note -- Unlike ZBREAK actions, $ETRAP, $ZTRAP, and EXCEPTION strings, $ZSTEP is compiled when it's SET rather than when it's first used. This means that the prior $ZTEP action value (which defaults to "B" - BREAK) is not changed on a defective SET. We don't expect or recommend production code use ZSTEP commands. Nonetheless, if it should happen to, we recommend ensuring $ZSTEP is successfully assigned a value other than the default; for example, $ZSTEP="" causes a ZSTEP to have no effect (other than to slow execution if ZSTEP is used). (C9C11-002181)

GT.M now reports the correct error location if it encounters errors during $ZTRAP error handling. In prior versions, in some cases, certain errors could cause GT.M to misreport the current location particularly in $ZSTATUS, but potentially in other location reports, such as $ZPOSITION. (C9D01-002232, S9I03-002676)

GT.M no longer triggers a GTMASSERT error (causing a process termination) while trying to determine the error location under certain conditions. Previous versions occasionally issued the spurious GTMASSERT error. (C9D01-002234, C9E08-002617)

GT.M processes on IBM pSeries AIX are now 64-bit. See "Migrating to 64-bit platform" and "Additional Information on AIX" for more information on installing GT.M on IBM pSeries AIX. [AIX] (C9G04-002788)

GT.M processes on the x86_64 GNU/Linux platform are now 64-bit. GT.M for x86 Linux (32-bit) remains available as a separate build and continues to run on both x86 and x86_64 platforms. On x86_64 GNU/Linux, GT.M now has the ability to generate object modules that can be included in shared libraries using the standard system ld utility. See "Migrating to 64-bit platform" and "Additional Information on Linux" for more information on installing GT.M on x86_64 GNU/Linux. [Linux] (C9H07-002880)

The v5cbsu V4 to V5 upgrade utility component now operates with TRANSACTIONID="BATCH" which reduces its run-time by not waiting for journal hardening. It is safe to do this because the v5cbsu can be safely rerun or restarted should the original run stop before it is complete. (C9H09-002899)

If an application fails to detect and act on the loss of a SOCKET $PRINCIPAL device (using $DEVICE, $ZEOF, or EXCEPTION) and attempts additional READs or WRITEs, GT.M handles this condition by sending a NOPRINCIO message to the system log and terminating the process (as if performing a HALT). (C9H10-002916)

GT.M now prevents a module on the execution stack from being ZLINK'd in a pathological case. A pattern where an attempted ZLINK of a program on the stack (which would fail, as it should), followed by a ZLINK of a program with the same name when it was not on the stack (which would succeed), was followed by a ZLINK of the same module name when it was again on the stack would not fail with an error (whereas it should). Subsequent behavior could include memory access violations. (C9H11-002926)

The "execstack" command is no longer needed to permit GT.M processes to execute dynamically generated code. [Linux] (C9H11-002927)

GT.M is now able to explicitly ZLink object modules in shared libraries. In prior versions, doing an explicit ZLink on an object module in a shared library produced a ZLink error. The new behavior is now consistent between explicit and implicit ZLinks. If you have $ZROUTINES search lists that assume that explicit ZLINK would bypass your shared libraries, you should revise those in light of what you were trying to accomplish. (C9H11-002932)

GT.M no longer generates unaligned access errors in the system log files. Previously, due to an incorrect size of a structure in 64-bit UNIX versions of GT.M, running the gtcm_gnp server could generate unaligned access messages. (C9I02-002959)

An auto-ZLINKing process now holds control of a newly generated object file until it completes loading the fresh code from it. Previous versions used to close the newly created object file at the end of the compile and then reopen it to load the code - this left a small window where another process could delete that file and cause a somewhat mystifying file-not-found error. (C9I02-002960)

GT.M now guards against signals interrupting time requests other than those for internal use. Previously limitations on reentrancy of UNIX services could occasionally cause time requests to hang if a signal resulted in another time request while one was already active. [UNIX] (C9I03-002967)

GT.M now reports if it encounters errors or a full OPCOM mailbox while trying to send operator messages; it sends this report at the first non-failing attempt. It also retries sending an operator message for the mail box full condition. [OpenVMS] (C9I03-002969)

GT.M now manages object files on some 64-bit platforms in a way that avoids a small memory leak that existed in prior versions. [Linux/IA64, Linux/x86_64, HPUX/IA64] (C9I03-002971)

The source server now replicates data correctly whether it reads from journal files or the journal pool. In GT.M V5.3-001, if the source server needed to read from journal files and if the journal files had sync_io turned on and were using alignsize values greater than 64Kb, it read more data than its internal buffers could hold leading to memory corruption and mysterious abnormal termination. [UNIX] (S9I03-002675)

The GT.M utilities except GDE (DSE, LKE, MUPIP) now require arguments and values entered directly from the shell and containing single-quote (') or double-quotes (") to have the quotes "protected" by an escape mechanism. Each one must be preceded by a back-slash (\) or the entire string must be enclosed in a set of quotes (single or double) that do not also appear in the string. Previously GT.M attempted to implicitly escape quotes within some strings when it added quotes in preprocessing strings, but that prior approach led to inconsistent behavior in certain cases. The following are examples of appropriate usage:

  lke show -lock='^global("embed=and""nospace")'
  lke show -lock="^global(\"embedded\"\" quote\")"
  lke show -lock='^a("two words",5)'
  

	This change may cause some existing scripts to fail.

The following is an example from our test system where the old usage required revision:

$MUPIP backup -o -i -t=1 DEFAULT "| gzip -c > online5pipe.inc.gz"

Now must be:

$MUPIP backup -o -i -t=1 DEFAULT "\"| gzip -c > online5pipe.inc.gz"\"

Behavior at the utility prompt is unchanged. [UNIX] (D9G12-002633)

Backward journal recovery (MUPIP JOURNAL ROLLBACK or MUPIP JOURNAL RECOVER BACKWARD) now maintains the integrity of the master bitmap when the total blocks in the database is a multiple of 512. In previous versions, these operations could, in that case coupled with other unusual conditions, result in a database with a DBMBPINCFL integrity error (Master bitmap incorrectly marks this local map full). Please note that although no database integrity error is desirable, DBMBINCFL is a "benign" error whose only effect is wasted space in the database. (C9H10-002923)

MUPIP REORG now preserves application data integrity even in case of concurrent GT.M updates. In previous versions, running MUPIP REORG concurrently with GT.M could result in data loss in certain extremely rare cases. In GT.M V5.3-000 one particular case of this issue was addressed by C9B11-001813. Additional cases are addressed by this change. (C9H12-002934)

MUPIP INTEG now correctly reports a DBINVGBL integrity error when it finds records with one global name in the Global Variable Tree (GVT) belonging to a different global name. Previous GT.M versions used to miss reporting an integrity error in case the incorrectly placed records were in the rightmost leaf block of the target GVT and the global name corresponding to the incorrectly placed records lexically sorted AFTER the global name corresponding to the target GVT. For example, if MUPIP INTEG encountered records of ^AA in the rightmost leaf block of the GVT of ^BB, it issued (and continues to issue) an integrity error but if there were records of ^CC incorrectly placed in the rightmost leaf block of the GVT of ^BB, then INTEG previously issued no error. (C9I02-002956)

None.