Technical Bulletin - GT.M on IBM eServer zSeries z/OS

Revision History

Table of Contents

z/OS for those with knowledge of GT.M

• The total object code for any process must be less than 2GB.
• The Mapped Memory (MM) database access method works only for database files of 1GB or less and MM database files cannot expand while in use.
  

GT.M for those with knowledge of z/OS

• Unlike common database systems such as DB2, GT.M has no database daemon or address space acting as a common server.  Processes accessing the database manage the database cooperatively, and each process has the database engine logic packaged as a shared library within its address space.
• With the exception of a small helper process requiring administrative "root" access, processes run with normal POSIX user & group ids and access permissions.  GT.M implements no security layer of its own, and leverages security as implemented by the underlying operating system.  So, for z/OS, the various features of SAF security apply, as do  the various, but optional, BPX.* security profiles.  There is a straightforward GT.M security technical bulletin.
  
• The daemonless architecture means a database file can be concurrently opened and accessed by hundreds to thousands of processes, all of which are performing random access reads and writes.  Like most databases, GT.M uses journal files for recoverability and business continuity, committing records to journal files before applying updates to database files.  Like its database files, GT.M journal files are also concurrently opened and written to by hundreds to thousands of processes.  Although journal files are written to sequentially, each process performs a seek to the end of a journal file and then writes to it - attempts to tune IO can be thrown off by this behavior which looks like random access but is actually sequential.  Also, under normal operation, journal files are almost never read from - only written to.  On z/OS, effective zFS file system caching and particularly the non-volatile write cache available on DS83xx disk subsystems are central to high performance GT.M logging of journals.
• On systems that are not resource starved, I/O subsystem performance and shared memory performance are the two most important factors affecting the throughput of GT.M based applications.
• Since processes cooperate to manage the database, a process' I/O activity has little to no correlation to the workload it executes - I/O performed by one process may actually be generated from the workload of another process.  Do not impose quotas on process I/O or otherwise penalize I/O intensive processes.
  
• GT.M makes extensive use of shared memory.  A system upgrade which alters a system from a Uniform Memory Architecture to a Non Uniform Memory Architecture (NUMA) may improve throughput less than hoped for, and can even decrease throughput if a result of the upgrade is to force processes to use shared memory that has a high access penalty.
• Since all processes concurrently accessing a database must access the same POSIX IPC resources and have the same view of process ids, GT.M does not support concurrent access of a database file by multiple computing nodes in a cluster or Sysplex.  However, when multiple nodes have access to the zFS file system supporting GT.M, in the event that node fails, another node can recover the database and application and provide application continuity.
• For even better continuity of business than clusters provide, GT.M has the ability to deploy "Logical Multi-Site" (LMS) application configurations.  Unlike clusters, which provide business continuity in the face of unplanned events (such as system crashes), GT.M LMS configurations can provide business continuity even in the face of planned events such as system and application software upgrades, even many upgrades involving a schema change.  LMS has no geographical distance limitations.
  

Terminal

Platform

Required

• IBM z10 EC 2097
• IBM z10 BC 2098
• IBM z9 EC 2094
• IBM z9 BC 2096

To use encryption with GT.M requires the addition of the Crypto Express2 card and the ICSF (Integrated Cryptographic Service Facility) software stack.  Database users must also have read access to the CSFRNG (random number generate service) in the RACF(R) CSFSERV class.

• APAR PK84982
• APAR OA28222
• APAR OA29325

zFS

1. Freeze updates to the database with MUPIP, e.g.: mupip freeze -on "*"
2. Create new journal files with MUPIP, e.g.: mupip set -journal="before" -region "*"
3. Clone the database file system(s).  If you have multiple database files on different zFS file systems, clone all of them at this time in order to get a snapshot of the database consistent across all the database files that together make up the logical database.
4. Release the freeze with with MUPIP, e.g.: mupip freeze -off "*"

Environment Variables

• _BPXK_AUTOCVT="ON"

• _CEE_RUNOPTS="FILETAG(AUTOCVT,AUTOTAG) POSIX(ON) TERMTHDACT(UAONLY)"
  

• _TAG_REDIR_ERR, _TAG_REDIR_IN, and _TAG_REDIR_OUT should all be set to "txt".  The first and last ensure any file redirected STDERR or STDOUT is tagged TXTFLAG=ON and CCSID set at first write by the process (819 for GT.M.)

Optional

• Note: these popular packages are neither distributed by nor supported by FIS GT.M support.  GT.M will use them if they are appropriately installed and configured.  FIS suggests that you ensure that you have an appropriate level of support for any software that you use in production.  Instructions in GT.M documentation reflect the way that we downloaded and built these packages in our environments for GT.M development and testing and are provided solely in the hope that they will be helpful to you when you install them.

zlib 1.1.4

---------------------------------- zlib_1.1.4_zos.patch --------------------------------------------------diff -purN downloads/zlib/src.orig/configure downloads/zlib/src/configure--- downloads/zlib/src.orig/configure Tue Dec 16 14:09:57 2008+++ downloads/zlib/src/configure Mon Feb 9 14:30:49 2009@@ -116,6 +116,11 @@ else SFLAGS=${CFLAGS-"-Kconform_pic -O"} CFLAGS=${CFLAGS-"-O"} LDSHARED=${LDSHARED-"cc -G"};;+ OS/390*)+ CC=xlc+ SFLAGS=${CFLAGS-"-qascii -q64 -Wc,DLL,LP64,XPLINK,EXPORTALL -D_ALL_SOURCE_NOTHREADS"}+ CFLAGS=${CFLAGS-"-qascii -q64 -Wc,DLL,LP64,XPLINK,EXPORTALL -D_ALL_SOURCE_NOTHREADS"}+ LDSHARED=${LDSHARED-"xlc -qascii -q64 -Wl,dll,LP64,XPLINK "};; # send working options for other systems to support@gzip.org *) SFLAGS=${CFLAGS-"-O"} CFLAGS=${CFLAGS-"-O"}

Device Handling

Background

iconv -T -f IBM-1047 -t ISO8859-1 inputfile >outputfile

The z/OS chtag utility program modifes tag metadata.  For example, to tag a file as ASCII, you can use:

chtag -t -c ISO8859-1 asciifile

Unlike iconv, chtag changes only the metadata tag, leaving the file contents unchanged.

Return to Table of Contents

Automatic Conversion

SD and FIFO

1. The GT.M device $Principal internally maps to two operating system file descriptors, one for STDIN and another for STDOUT (GT.M does not at present provide the ability to write to STDERR).  Since STDIN and STDOUT can be tagged differently, the GT.M runtime system potentially enables different conversions, for READ from $Principal and WRITE to $Principal.
2. MUPIP extracts in ZWR or GO format always contain only ASCII characters, and are tagged as ASCII - ZWR uses $CHAR() representation of non-ASCII characters.  MUPIP extracts in BIN format are tagged as binary.  Extract files imported into a z/OS environment from other environments must be converted with iconv or tagged with chtag before they are loaded.

PIPE DEVICE

Devices with chset="BINARY"

External Routines

1. xlc -q64 -qascii -D_ENHANCED_ASCII_EXT=0xFFFFFFFF -W c,DLL,XPLINK,EXPORTALL,RENT,NOANSIALIAS,LANGLVL(EXTENDED),ARCH(7) -W l,DLL,XPLINK,REUS=RENT $gtm_exe/libgtmshr.x source files

1. Compile source files into object files
   xlc -c  -q64 -qascii -D_ENHANCED_ASCII_EXT=0xFFFFFFFF -W c,DLL,XPLINK,EXPORTALL,RENT,NOANSIALIAS,LANGLVL(EXTENDED),ARCH(7) -W l,DLL,XPLINK source files
2. Link object files into an executable or shared library
   xlc -q64 -W l,DLL,XPLINK,REUS=RENT -o dll name object files $gtm_exe/libgtmshr.x
   

1. -D_ALL_SOURCE_NO_THREADS
   enables POSIX.1, POSIX.1a, POSIX.2, XPG4.2, and XPG4.2 Ext, DISABLES THREADING
2. -D_XOPEN_SOURCE_EXTENDED=1
   enables POSIX.1, POSIX.2, XPG4.2, and XPG4.2 Ext., this should be redundant to -D_ALL_SOURCE_NO_THREADS, but it enables other feature test macros
3. -D_UNIX03_SOURCE
   exposes a few of the SUSv3 options, for exmaple dlclose(), dlerror(), dlopen(), dlsym()
4. -D_IEEEV1_COMPATIBILITY
   enable IEEE math compatibility
   

Object files in shared libraries

1. Compile your M sources to object files, e.g.: mumps a.m b.m c.m
2. Link the object files into one DLL, e.g.: xlc -q64 -W c,DLL,XPLINK,EXPORTFULL -W l,DLL,XPLINK -o my.dll a.o b.o c.o
3. Include the DLL in the gtmroutines search path, e.g.: export gtmroutines=". my.dll $gtm_dist"