Using Socket Devices
Chapter 9. Input/Output Processing

From an application perspective, the transport layers used by a socket device are stream-oriented, with no provisions for implicit application messages. Therefore, the following are two common protocols used to segment application messages.

The SOCKET device provides a facility for recognizing delimiters to simplify parsing messages.

TCP/IP is a stream-based protocol that guarantees that bytes arrive in the order in which they were sent. However, it does not guarantee that they will be grouped in the same packets.

If packets arrive infrequently, or at varying rates that are sometimes slow, a short interval can waste CPU cycles checking for an unlikely event. On the other hand, if the handling of packets is time critical, a long interval can introduce an undesirable latency. If packets arrive in a rapid and constant flow (an unusual situation), the interval doesn't matter as much, as there is always something in the buffer for the READ to work with. If you do not specify MOREREADTIME, SOCKET READ implements a dynamic approach of using a longer first interval of 200 ms when it finds no data, then shortening the interval to 10 ms when data starts to arrive. If you specify an interval, the SOCKET device always uses the specified interval and does not adjust dynamically. For more information on MOREREADTIME, refer to “MOREREADTIME”.

Most SOCKET READ operations terminate as a result of the first condition detected from (a) receipt of delimiters, (b) receipt of the maximum number of characters, or (c) expiration of a timeout. Note that all of these conditions are optional, and a specific READ may specify zero or more of them. This section refers to these three conditions as "defined terminating conditions". If a SOCKET READ is not subject to any of the defined terminating conditions, it terminates after it has received at least one character followed by an interval with no new characters. An error can also terminate a READ. While none of the terminating conditions is satisfied, the READ continues.

The following flowchart represents the logic of a SOCKET READ.

A SOCKET READ operation terminates if any of the following conditions are met:

* denotes Defined Terminating Conditions

A non-fixed-length read, with no timeout and no delimiters (the sixth row in the above table) requires a complex implementation of sequence of READs to ensure a predictable result. This is because the transport layer stream fragments delivered to the reader has only accidental correspondence with the operations performed by the writer. For example, the following:

Write "Message 1","Message 2" is presented to the reader as the stream "Message1Message2" but it can take from one (1) to 18 READ commands to retrieve the entire stream.

Messaging protocol should implement READ in any of the following ways:

Each device can have from zero (0) to 64 delimiters associated with it. Each delimiter can be from one (1) to 64 characters. All the delimiters declared for a device are valid for any READ from any associated socket, which means, any of the defined delimiters terminate the READ. The actual terminating delimiter is available in $KEY. A WRITE to a socket associated with a device with one or more delimiters inserts the first of the delimiters for any WRITE ! format.

The READ command may be used to obtain data from a socket. A READ operation terminates if any of the following are detected, in the order specified below:

A non-fixed-length read, with no timeout and no delimiters requires a complex implementation of sequence of READs to ensure a predictable result. This is because the transport layer stream fragments delivered to the reader has only accidental correspondence with the operations performed by the writer. For example, the following

Write "Message 1","Message 2"

is presented to the reader as the stream "Message1Message2" but it can take from one (1) to 18 READ commands to retrieve the entire stream.

The WRITE command sends data to a socket.

WRITE ! inserts the character(s) of the first I/O delimiter (if any) to the sending buffer. When the "ZFF=expr" characteristic is specified, WRITE # inserts the characters of expr. Otherwise WRITE # has no effect. WRITE ! and WRITE # always maintain $X and $Y in a fashion that emulates a terminal cursor position except when the device is OPENed with a UTF CHSET because the units for $X and $Y for terminals are in display columns while for sockets they are in codepoints/characters.

The WRITE command for SOCKET devices accepts the following controlmnemonics:

/L[ISTEN][(numexpr)]

where numexpr specifies the listen queue depth for a listening socket. The value of numexpr must be between 1 and the system-enforced maximum. By default, an OPEN or USE with LISTEN sets the listen queue size to 1. For vendor-specific information on how to change your system's maximum queue length, refer to the listen manpage.

WRITE /WAIT[(timeout[,[what][,handle]])]

where the required timeout is a numeric expression specifing how long in seconds a server waits for a connection or data to become available on one of the sockets in the current Socket Device, what may contain a string containing "READ" and/or "WRITE", and handle specifies the socket handle.

If the optional second argument only specifies "WRITE", WRITE /WAIT does not check incoming connections for listening sockets.

The optional third argument to WRITE /WAIT can be used to check only a single socket instead of all sockets in the current SOCKET device by specifying the handle name of a socket.

	Note
	In most circumstances, WRITE /WAIT(timeout[,"WRITE"]) for SOCKET devices which contain a non blocking socket returns immediately because non blocking sockets are usually ready for writing. If the current Socket Device is $PRINCIPAL and input and output are different SOCKETs, WRITE /WAIT applies to the input side of the device.

WRITE /PASS([targetpid],[timeout],handle[,handle]...)

WRITE /PASS allows a GT.M process to send DETACHed TCP or LOCAL sockets (that is, sockets in the socket pool) to another GT.M process. The receiving process must execute WRITE /ACCEPT to receive the socket.

WRITE /ACCEPT(.lvar,[sourcepid],[timeout][,[handle]]...)

WRITE /ACCEPT allows a GT.M process to receive a DETACHed TCP or LOCAL sockets (that is, sockets in the socket pool) from another GT.M process . The sending process must execute WRITE /PASS to send the socket.

Both WRITE /PASS and WRITE /ACCEPT require the current $IO to be a SOCKET device with a CONNECTed (not LISTENing) and LOCAL domain (not TCP) current socket. GT.M issues CONNSOCKREQ or LOCALSOCKREQ errors, respectively, when those conditions are not met.

SOCKET devices do not support mixing other READs and WRITEs with socket passing on the same CONNECTED LOCAL socket and produce SOCKPASSDATAMIX errors. The application may perform multiple WRITE /PASS and WRITE /ACCEPT operations in either direction on the socket before issuing a CLOSE.

Note that the receiving process must establish desired deviceparameters (e.g., DELIMITER) for a socket either by ATTACHing it to a SOCKET device that provides the characteristic for all its sockets, or by a subsequent USE that specifies the appropriate deviceparameter(s). GT.M transfers only the socket connection itself, the socket handle, and buffered socket data (if any), but no characteristics established by the sender.

WRITE /TLS(option[,[timeout][,tlsid[,[obfuscatedpassword][,cfg-file-options]]]])

SOCKET devices support encrypted connections with TLS using an encryption plugin. GT.M ships with a reference implementation of the plugin which uses OpenSSL; the reference implementation also supports TLS for GT.M replication streams. OpenSSL options are controlled by a configuration file. The WRITE /TLS command activates this feature for connected sockets.

Example:

set obspass="CD86FF2BFD1F06EE" ; maskpass output of password for private key
set cfgoptions="cert:""/path/to/certificate"";key:""/path/to/key.pem"";"
write /tls("server",,"tlsone",obspass,cfgoptions)

	Note
	Note that SOCKET device actions may produce the following errors: TLSDLLOPEN, TLSINIT, TLSCONVSOCK, TLSHANDSHAKE, TLSCONNINFO, TLSIOERROR, and TLSRENEGOTIATE.

The TLS plugin uses OpenSSL options in the configuration file specified under the tls: label as the default for all TLS connections and under the specific labels to override the defaults for corresponding connections.

GT.M buffers WRITEs to TLS enabled sockets until a subsequent USE :FLUSH, WRITE !, WRITE #, or an internal 400 millisecond timer expires.

Note

Because this functionality has a wide variety of user stories (use cases) and has substantial complexity, although the code appears robust, we are not confident that we have exercised a sufficient breadth of use cases in testing. Also we may make changes in future releases that are not entirely backwards compatible. We encourage you to use with this facility in development and testing, and to provide us with feedback. If you are an FIS customer and wish to use this in production, please contact us beforehand to discuss your use case(s).

Note

Owing to the range of OpenSSL versions in use across the breadth of platforms and versions supported by GT.M, on all platforms, but especially on non-Linux UNIX platforms, FIS recommends rebuilding the plugin for any production or production staging environments that use TLS from sources included with the GT.M binary distribution in order to use the specific version of OpenSSL installed on your systems. For more information on recompiling the reference implementation, refer to the Installing GT.M chapter of Administration and Operations Guide.

WRITE /BLOCK("OFF")

WRITE /BLOCK("OFF") enables non-blocking WRITEs for the current socket of the current SOCKET device. Sockets default to blocking WRITEs.

A socket must be enabled for non-blocking WRITEs before enabling it for TLS when using both features on the same socket.

For non-blocking sockets, GT.M retries a WRITE that blocks up to the number of times specified by the $gtm_non_blocked_write_retries environment variable with a 100 millisecond delay between each retry. The default retries value is 10 times if $gtm_non_blocked_write_retries is not defined.

If WRITE remains blocked after the specified retries, the WRITE sets $DEVICE to "1,Resource temporarily unavailable" and issues an error if IOERROR is "TRAP".

If IOERROR is not "TRAP", the application must check $DEVICE after each WRITE. An attempt to WRITE to a socket after it has been blocked is an error which sets $DEVICE to "1,Non blocking WRITE blocked - no further WRITEs allowed". Thus the only operation permitted on a blocked socket is a CLOSE.

Note

Note that multi-argument WRITEs are equivalent to a series of one argument WRITEs, and that GT.M turns unparenthesized concatenation within a write argument into multi-argument WRITEs. Format control characters such as "!" and "#" are each considered as an argument. Note that a significant delay between bytes for any reason, including blocking, especially within a multibyte character when CHSET is UTF-8, may be considered an error by the receiving end of a connection. If the application is unable to handle such a delay, it may result in an application error.

A WRITE to a non-blocking socket, which is not enabled for TLS, may terminate early on the following events:

<CTRL-C>, exceeding $ZMAXTPTIME, or $ZTIMEOUT expiring. These events result in a transfer to the interrupt vector or error handler at the next execution boundary as described in “Interrupt Handling”.

When non-blocking WRITEs are enabled for a socket, WRITE /WAIT may check if that socket would not block on WRITE in addition to READ. The optional second argument may contain a string containing "READ" and/or "WRITE".

If the second argument is omitted or specifies both "READ" and "WRITE" and the socket selected by WRITE /WAIT is ready for both READ and WRITE, $KEY contains:

READWRITE|<socket handle>|<address>.

If the second argument is omitted or contains "WRITE", WRITE /WAIT checks readiness for WRITE on non-blocking sockets, but never checks readiness to WRITE on blocking sockets, even if explicitly requested.

If the socket selected by a WRITE /WAIT implicitly or explicitly requests the state for writing would block on a READ but not block on WRITE, $KEY contains:

WRITE|<socket handle>|<address>

Note that a WRITE may still not be able to complete if it tries to write more bytes than the system is ready to accept.

If the socket selected by WRITE /WAIT which implicitly or explicitly requests the state for reading would not block on a READ but would block on a WRITE, $KEY contains:

READ|<socket handle>|<address>

$ZKEY after a prior WRITE /WAIT will contain a piece of the format "WRITE|sockethandle|ipaddress" if a non-blocking socket was considered writable, which we expect to be typical. If a socket was also readable, there will be two pieces in $ZKEY for the socket, one for WRITE and the other for READ.

An application can determine whether a socket is enabled for non-blocking WRITEs with:

$ZSOCKET(device,"BLOCKING",index)

which returns either 1 (TRUE) for blocking, or 0 (FALSE) for non-blocking.

Each socket may be in one of the following states (observable through $KEY):

A listening socket used for accepting new connections goes through these three states in one step with a single OPEN or USE. When a server does a WRITE /WAIT, a client can establish a connection which creates a new server socket. $KEY includes information about this new socket in the form of CONNECT|handle|<address> where <address> is the IP address for TCP sockets and path for LOCAL sockets.

Each socket may have one or more sockets waiting for either an incoming connection or data available to READ (observable through $ZKEY). $ZKEY contains semi- colon (";") separated list of entries detailing any waiting sockets for a current SOCKET device.

For more information on $KEY and $ZKEY, refer to “Intrinsic Special Variables”.

The following table provides a brief summary of deviceparameters for socket devices.

The sockexamplemulti32.m routine shows the use of $KEY and $ZKEY in a basic socket I/O setup. It's functionality is atypical in order to demonstrate a number of features. It launches two jobs: a server process which opens a listening socket and a client process which makes five connections to the server. The server sends a message to each connection socket. Even-numbered client sockets read the message partially but do not send a response back to the server. Odd-numbered client sockets receive the full message and respond to the server with the message "Ok.". The server reads two characters (but the client sends three) and $ZKEY shows sockets with unread characters.Please click to download the sockexamplemulti32.m program and follow instructions in the comments near the top of the program file. You can also download sockexamplemulti32.m from http://tinco.pair.com/bhaskar/gtm/doc/books/pg/UNIX_manual/sockexamplemulti32.m.

You can start a GT.M process in response to a connection request made using inetd/xinetd. The following example uses inetd/xinetd to implement a listener which responds to connections and messages just as the prior example.

In the configuration file for xinetd, define a new service called gtmserver. Set socket_type to "stream" and wait to "no" as in the following snippet:

service gtmserver 
{ 
disable = no 
type = UNLISTED 
port = 7777 
socket_type = stream 
wait = no 
user = gtmuser 
server = /path/to/startgtm 
} 

If you define the server in /etc/services, the type and port options are not needed. For more information, the xinetd.conf man page for more details.

If you are using inetd, add a line to /etc/inetd.conf with the sockettype "stream", protocol "tcp", and specify the "nowait" flag as in the example below, which assumes a gtmserver service is defined in /etc/services:

gtmserver stream tcp nowait gtmuser /path/to/startgtm 

In both of the above examples, "gtmuser" is the name of the user to own and run the gtmserver service, and "/path/to/startgtm" is the name of a script which defines some environment variables needed before invoking GT.M. Please check the man page for inetd.conf on your system as the details may be slightly different.

The minimum variables are: $gtm_dist, which specifies the directory containing the GT.M distribution, and $gtmroutines, which specifies the paths used to locate the GT.M routines. As an example:

#!/bin/bash 
cd /path/to/workarea 
export gtm_dist=/usr/local/gtm 
export gtmroutines="/var/myApp/o(/var/myApp/r) $gtm_dist" 
export gtmgbldir=/var/myApp/g/mumps.dat 
$gtm_dist/mumps -r start^server 

When start^server begins, the $PRINCIPAL device is the current device which is the incoming connection and $KEY contains "ESTABLISHED|socket_handle| remote_ip_address". In most cases, a USE command near the beginning of the routine sets various device parameters such as delimiters.

The ZSHOW "D" command reports available information on both the local and remote sides of a TCP socket including local and remove addresses and ports.

0 OPEN SOCKET TOTAL=1 CURRENT=0 
SOCKET[0]=h11135182870 DESC=0 CONNECTED ACTIVE NOTRAP 
REMOTE=10.1.2.3@53731 LOCAL=10.2.3.4@7777 
ZDELAY ZIBFSIZE=1024 ZIBFSIZE=0