ZLink

The ZLINK command adds an executable GT.M routine to the current process if the current process does not contain a copy of a routine. If the current process contains a copy of a routine and the routine is not active, the ZLINK command replaces the current routine process with a "new" version. If necessary, the ZLINK command compiles the routine prior to integrating it with the process.

With VIEW "LINK":"RECURSIVE" specified or by starting the process with the environment variable gtm_link set to "RECURSIVE", the ZLINK command adds an executable routine even when a routine with the same name is active and available in the current stack. When a process links a routine with the same name as an existing routine, future calls use the new routine. Prior versions of that routine referenced by the stack remain tied to the stack until they QUIT, at which point they become inaccessible. This provides a mechanism to patch long-running processes.

[Important] Important

An active routine is displayed with $STACK() or ZSHOW "S" of the M virtual stack. By default, an attempt to replace an active routine results in a run-time error . To replace an active routine with a new version, either use VIEW "LINK":"RECURSIVE" or remove the active routine from the stack using ZGOTO or the appropriate number of QUITs and then execute the ZLINK command.

The format of the ZLINK command is:

ZL[INK][:tvexpr] [expr1[:expr2][,...]]

When the ZLINK command specifies a file, GT.M sets $ZSOURCE to that filename. By default, ZLINK and ZEDIT use $ZSOURCE for a filename when they have a missing or null argument. A subsequent ZLINK without an argument is equivalent to ZLINK $ZSOURCE. For more information on $ZSOURCE, see the appropriate section in Chapter 8: “Intrinsic Special Variables.

[Note] Note

In order to ensure compatibility with GT.M versions that do not permit the percent sign (%) in a file name, use an underscore (_) in place of the percent in the ZLINK file name for routines beginning with a percent sign.

If the expression includes an explicit directory, ZLINK searches only that directory. Otherwise, if $ZROUTINES is not null, a ZLINK command uses $ZROUTINES to locate files. If $ZROUTINES is null, ZLINK uses the current directory. For more information on $ZROUTINES, see the appropriate section in Chapter 8: “Intrinsic Special Variables.

If the filename contains an explicit file extension, ZLINK processes the file according to the extension, object (.o) or source (usually .m). If the file name does not specify a file extension, ZLINK attempts to find and match both the object and source for a routine.

The following table illustrates how ZLINK processes the three possibilities of file extension.

ZLINK Operation Summary

EXTENSION SPECIFIED

EXTENSION SOUGHT BY ZLINK

RESULT

 

.o

.m

 

.o

found

N/A

link only

 

not found

N/A

error

.m

N/A

found

compile and link

  N/A

not found

error

None

not found

found

compile and link

  found not found link only
  not found not found error
 

found .o file newer than .m and version okay

found .m file older than .o

link only

 

found .o file older than .m or version mismatch

found .m file newer than .o

compile and link

ZLINK Compilation

If ZLINK compiles a routine and the -OBJECT= qualifier does not redirect the output, it places the resulting object file in the directory indicated by the search criteria. ZLINK incorporates the new object file into the image, regardless of its directory placement.

If the command does not specify compile qualifiers (with expr2) and $ZCOMPILE is null, GT.M uses the default M command qualifiers, -ignore, -labels=lower, -nolist, and -object. For more information on $ZCOMPILE, refer to the appropriate section in Chapter 8: “Intrinsic Special Variables. For detailed descriptions of the M command qualifiers, see Chapter 3: “Development Cycle.

For information on producing object files, but not adding them to the current image, see “ZCOMpile”.

Examples of ZLINK

Example:

GTM>ZLINK "test"

If ZLINK finds test.m or test.o, it adds the routine test to the current image. If ZLINK does not find test.o, or finds that test.o is older than test.m, GT.M compiles test.m to produce a new test.o, and adds the contents of the new object file to the image. This example assumes "test" is not on the current M stack - if it is on the stack, GT.M gives an error.

Example:

GTM>zlink "test.m":"-noobject -list"

This compiles the routine "test" and produces a listing but no object file. Because the example produces no object file, it must locate an existing object file (which might be the same as any copy in the current image); if there is noexisting object file, GT.M produces an error. While this example shows the use of compilation qualifiers with ZLINK, a -noobject -list compilation might better be done with ZCOMPILE.

Example:

GTM>zlink "sockexamplemulti2"
%GTM-E-LOADRUNNING, Cannot ZLINK an active routine sockexamplemulti2
GTM>zshow "S"
sockexamplemulti2+12^sockexamplemulti2    (Direct mode)
GTM>view "LINK":"RECURSIVE"
GTM>zlink "sockexamplemulti2"
GTM>

This example demonstrates how VIEW "LINK":"RECURSIVE" command ZLINKs a routine when its prior version is already there in the active M virtual stack.

Auto-ZLINK

If a GT.M routine refers to a routine that is not linked in the process memory, GT.M automatically attempts to ZLINK that routine. An auto-ZLINK is functionally equivalent to an explicit ZLINK of a routine without a specified directory or file extension.

The following GT.M commands and functions can initiate auto-ZLINKing:

  • DO

  • GOTO

  • ZBREAK

  • ZGOTO

  • ZPRINT

  • $TEXT()

GT.M auto-ZLINKs the routine if the following conditions are met:

  • ZLINK can locate and process the routine file, as indicated in the previous ZLINK Operation Summary table

  • The name of the routine is the same as the name of the source file; the only exception is that GT.M converts a leading percent sign (%) in a file name to an underscore (_).

Auto-ZLINK setup

This section describes the procedure to setup the auto-relink functionality. GT.M loads an object file linked from an object directory (in $ZROUTINES) with a *-suffix (i.e. auto-relink-enabled) into a shared memory segment (referred to henceforth as a Rtnobj shared memory segment). At the invocation of DO, GOTO, or ZGOTO, extrinsic functions, ZBREAK, ZPRINT or $TEXT() that specify an entryref which includes a routine name (in contrast to a label without a routine name), GT.M processes (and MUPIP processes executing trigger logic) automatically relink ("auto-relink") and execute published new versions of routines.

[Note] Note
  • Label references (that is, without a routine name), whether direct or through indirection, always refer to the current routine, and do not invoke auto-relink logic.

  • Use shell quoting rules when appending asterisks to directory names in the gtmroutines environment variable - asterisks must be passed in to GT.M, and not expanded by the shell.

  • GT.M accepts but ignores asterisk suffixes to directory names on 32-bit Linux on x86 platforms, where it does not provide auto-relinking.

The ZRUPDATE command publishes of new versions of routines to subscribers. To remove routines, delete the object files and publish the names of the deleted object files. Removal requires file names to be explicitly specified, because patterns with wildcards cannot match deleted files.

If the path to a file is non-existent, the request is ignored except in the case where one desires a currently shared object file (one that was accessed before it was deleted) to no longer be shared.

For each auto-relink enabled directory which a GT.M process accesses while searching through $ZROUTINES, GT.M creates a small control file (Relinkctl) in the directory identified by $gtm_linktmpdir (defaulting to $gtm_tmp, which in turn defaults to /tmp, if unspecified). The names of these files are of the form gtm-relinkctl-<murmur> where <murmur> is a hash of the realpath() to an auto-relink directory; for example: /tmp/gtm-relinkctl-f0938d18ab001a7ef09c2bfba946f002). With each Relinkctl file, GT.M creates and associates a block of shared memory that contains associated control structures. Among the structures is a cycle number corresponding to each routine found in the routine directory; a change in the cycle number informs a process that it may need to determine whether there is a new version of a routine. Although GT.M only creates relinkctl records for routines that actually exist on disk, it may increment cycle numbers for existing relinkctl records even if they no longer exist on disk.

GT.M creates both the Relinkctl file and shared memory with permissions based on the logic described in the "IPC Permissions" column of the "Shared Resource Authorization Permissions" section in the Administration and Operations Guide, except that the object directory, rather than the database file, provides the base permissions.

The MUPIP RCTLDUMP command reports information related to relinkctl files and their associated shared memory segments.

The environment variable gtm_autorelink_keeprtn if set to 1, t[rue], or y[es] causes exiting processes to leave auto-relinked object code in the shared memory repositories, while if undefined, 0, f[alse] or n[o] causes exiting processes to purge any routines currently use by no processes. All values are case-independent. When gtm_autorelink_keeprtn is defined and TRUE:

  • Process exit is simplified, with the performance gain - faster process termination - likely to be observable only when a large number of processes exit concurrently.

  • Where routines are likely to be repeatedly used by other processes, such as in a production environment, leaving a routine in shared memory even when no longer used by existing processes, results in slightly faster linking of that routine by future processes, although the effect may not be observable except when an application frequently uses short-lived processes, such as GT.M routines invoked by web servers using a CGI interface.

FIS recommends that a directory in the $zroutines of a process be either auto-relink-enabled or auto-relink-disabled for the life of the process. Changing the auto-relink mode of the directory within a process is likely to result in counter-intuitive results..

As arguments, ZRUPDATE takes object file names, including wild-cards of the form accepted by $ZSEARCH(). If ZRUPDATE fails to find at least one file to match an argument with a wild card, it issues an INFO message (seen only if $PRINCIPAL has CENABLE). When the argument specifies an explicit name without a wild card, but there is no file in the directory or a corresponding entry in the Relinkctl, ZRUPDATE produces an error. ZRUPDATE issues most errors as FILEPARSE errors with a secondary error describing the actual issue although some errors, depending on the reason and path by which ZRUPDATE detects them, can be rather cryptic.

An explicit ZLINK or an auto-relink check the hash of an object and its replacement. If they are identical, GT.M may take no action to replace the current object, saving both memory and time.GT.M bypasses the dynamic link of a routine when it determines the requested object matches the currently linked object file under either of the following circumstances:

  • auto-relink from the same directory

  • explicit ZLINK where neither the original object nor the new object are auto-relinked.

In other cases GT.M always performs the dynamic link.

An explicit ZLINK from an auto-relink directory acts as an implicit ZRUPDATE.

Any ZBREAK in a routine disables that routine from auto-relinking by a process until all ZBREAKs are removed.

If recursive relink is not enabled, routines currently active in the M virtual machine stack are disabled from auto-relinking until they complete (or are removed from the stack by a ZGOTO).

Auto-zlink Benefits and Example

The benefits of auto-relink are as follows:

  • Auto-relink provides the convenience of automatically running the current routine under most conditions.

    When combined with VIEW "LINK":"RECURSIVE", auto-relink automatically relinks routines even when they are active and available in the current stack. While it is possible to run auto-relink without VIEW "LINK":"RECURSIVE", routines currently active in a stack do not auto-relink and, if explicitly ZLINK'd, induce a LOADRUNNING error until they complete or are removed from the stack.

  • Use of auto-relink loads routine object files into the shared memory. Therefore, the use of a given routine by multiple processes results in significant memory savings (one copy per system instead of one copy per user). This is analogous to the memory sharing from using shared object libraries, but allows dynamic updates, where shared libraries do not.

  • When combined with routines explicitly compiled with the -embed_source option or auto-compiled with $ZCOMPILE set to "-embed_source", auto-relink may improve the performance of $TEXT() and ZPRINT as they access source code from shared memory instead of the disk.

  • When $gtm_autorelink_keeprtn is defined and TRUE, applications that frequently invoke GT.M routines in short running processes (such as those over interfaces like CGI) may give better performance because it keeps routines in shared memory so that they can be reused when short running processes need them.

The use and setup of the auto-relink facility depends upon the requirements. Here is an example:

$ /usr/lib/fis-gtm/V6.2-001_x86_64/gtm
GTM>w $zroutines
/home/jdoe/.fis-gtm/V6.2-001_x86_64/o*(/home/jdoe/.fis-gtm/V6.2-001_x86_64/r /home/jdoe/.fis-gtm/r) /usr/lib/fis-gtm/V6.2-001_x86_64/plugin/o/_POSIX.so /usr/lib/fis-gtm/V6.2-001_x86_64/plugin/o(/usr/lib/fis-gtm/V6.2-001_x86_64/plugin/r) /usr/lib/fis-gtm/V6.2-001_x86_64/libgtmutil.so /usr/lib/fis-gtm/V6.2-001_x86_64

In $ZROUTINES, the *-suffix after the object directory enables the auto-relink facility. By default, the gtm/gtmprofile scripts that are available as part of GT.M distribution on sourceforge.net have auto-relink enabled.

With auto-relink enabled, GT.M loads an object file from an object directory into Rtnobj shared memory segment on an explicit ZLINK, implicit ZLINK (DO, GOTO, ZPRINT, $TEXT()), and extrinsic function invocations ($$) enabling the routines to be accessed by other concurrent/future processes.

[Note] Note

With auto-relink, GT.M creates an initial Rtnobj shared memory segment of 1 MiB (2 MiB or more if hugepages is configured) and allocates 92MiB of shared memory segment for managing the auto-relink facility. Therefore, always ensure that your system has adequate shared memory configured; if not, GT.M displays messages along the lines of:

%GTM-E-SYSCALL, Error received from system call shmget() failed

Refer to your OS documentation to configure shared memory limits (for example, on common Linux systems, the kernel.shmmax parameter in /etc/sysctl.conf).

If your routines require more MiB shared memory, set the environment variable $gtm_autorelink_shm to an integer value (in powers of two). When auto-relink needs more shared memory for storing routines, GT.M automatically allocates twice the size of $gtm_autorelink_shm MiB for auto-relink operations.

GTM>zedit "myprogram.m"
GTM>

ZEDIT puts a new file into the first source directory in $ZROUTINES, that is, in the /home/jdoe/.fis-gtm/V6.2-001_x86_64/r directory.

GTM>do ^myprogram

The first invocation of an implicit ZLINK (DO, GOTO ZGOTO, ZPRINT, $TEXT() or function/extrinsic invocation) or an explicit ZLINK "myprogram.m" or ZRUPDATE "/home/jdoe/.fis-gtm/V6.2-001_x86_64/myprogram.o" creates a Relinkctl file if one does not already exist and the associated shared memory. The relinkctl file has a name associated with the hash of the directory to provide a pointer in the form of segment ids to shared memory so that processes can locate routines.

As the gtm_linktmpdir environment variable is not set by default in the gtm/gtmprofile scripts, GT.M stores the Relinkctl file in the directory pointed to by the gtm_tmp environment variable.

GTM>zshow "A"
Object Directory : /home/jdoe/.fis-gtm/V6.2-001_x86_64/o
Relinkctl filename : /tmp/fis-gtm/V6.2-001_x86_64/gtm-relinkctl-43b26ca8384ddbf74b94d90a830c0bc9
# of routines : 1
# of attached processes : 1
Relinkctl shared memory : shmid: 375586821 shmlen: 0x5800000
Rtnobj shared memory # 1 : shmid: 375619590 shmlen: 0x200000 shmused: 0x400 shmfree: 0x1ffc00 objlen: 0x280
rec#1: rtnname: myprogram cycle: 1 objhash: 0xd81f1cdcc275e13d numvers: 1 objlen: 0x280 shmlen: 0x400

ZSHOW "A" command displays information related to relinkctl file and the routine records that it points to in the shared memory segments. The routine records appears in the order in which they were inserted into the shared memory in the context of the relinkctl file.

GTM>zedit "myprogram2.m"

ZEDIT puts a new file into the first source directory in $ZROUTINES, that is, in the /home/jdoe/.fis-gtm/V6.2-001_x86_64/r directory.

GTM> zrupdate "/home/jdoe/.fis-gtm/V6.2-001_x86_64/o/*.o"

The ZRUPDATE command increments the cycle counter of those routine records whose object hash is different than the one last loaded in the Rtnobj shared memory. In this case, it would be rec#2, that is, myprogram.o. ZRUPDATE does not recompile/relink the routines. Instead, it instructs all current and future processes that the object code is out-of-date and must be auto-relinked (if required) on the next invocation. An explicit ZLINK or an auto-relink (whichever happens first) checks the hash of an object and its replacement and initiates recompile/relink on finding that are they not identical.

GTM>zshow "A":zru

If there are hundreds of routines, a command like ZSHOW "A":zru transfers the output of ZSHOW "A" to a local variable. The following example uses that result to display the information on all routines that contain the string passed to the disprtn function:

GTM>zprint ^disprtn
disprtn(rtn)
set x="" for set x=$order(zru("A",x)) quit:x="" write:$piece(zru("A",x),":",3)[rtn zru("A",x),!
quit ""

and produces a result like the following:

GTM>w $$^disprtn("myprogram")
rec#1: rtnname: myprogram2 cycle: 1 objhash: 0x436c855d5891e7cf numvers: 1 objlen: 0x370 shmlen: 0x400
rec#2: rtnname: myprogram cycle: 1 objhash: 0xd81f1cdcc275e13d numvers: 1 objlen: 0x280 shmlen: 0x400
GTM>

ZLINK, auto-ZLINK and Routine Names

In GT.M, the name of the source file determines the name of the GT.M routine. The file name of the object file is not required to match the name of the routine. Linking the object file makes the internal routine name (derived from the source file) known to GT.M. This can lead to potential confusion, however, since both ZLINK and auto-ZLINK use the name of the object file to find the routine. When the object file name differs from the name of the routine, auto-ZLINK generates a run-time error.

[Note] Note

Auto-ZLINK and ZLINK commands without a .m or .o file extension in their argument determine the need to recompile based on whether the object file was more recently modified than the source file using time in nanoseconds, as provided by the underlying system call. Note that, although the format of the file modification timestamps provides a nanosecond granularity, many supported OSs currently update the file timestamps with an accuracy of one second.