Database File Organization with GDS

GT.M processes a GDS file using predominantly low-level system services. The GDS file consists of two parts:

Database File Header

The fields in the file header convey the following types of information:

  • Data Elements

  • Master Bitmap

File Header Data Elements

All GT.M components, except GDE, (the run-time system, DSE, LKE, MUPIP) use the data elements of the file header for accounting, control, and logging purposes.

The current state of the file header always determines the characteristics of the database. The MUPIP CREATE command initializes the values of the file header data elements from the global directory and creates a new .DAT file.

The file header data elements are listed in alphabetical order below for easier access, rather than the order in which they appear in the file header.

Data Elements

Description

Access method

The buffering strategy of the database. Access Method can have 2 values - BG or MM. The default value is BG.

Buffered Global (BG) manages the buffers (the OS/file system may also buffer "in series"); MM - the OS/file system manages all the buffering.

Block size (in bytes)

The size (in bytes) of a GDS block. Block size can have values that are multiples of 512. The default value is 1024. Block size should be a multiple of the native block size for the OS file system chosen to accommodate the maximum required record size. For additional information, see Chapter 4: “Global Directory Editor.

Blocks to Upgrade

The count of the blocks in the database that are still in prior major version format. GT.M uses this element during incremental upgrades.

Cache freeze id

The process identification number (PID) of a process which has suspended updates to the segment.

Certified for Upgrade to V5

Count of blocks "pre-certified" (with the dbcertify utility) for an incremental upgrade. GT.M uses this element during incremental upgrades.

Create in progress

Create in progress is TRUE only during the MUPIP CREATE operation. The normal value is FALSE.

Collation Version

The version of the collation sequence definition assigned to your database. DSE only reports this if an external collation algorithm is specified.

Commit Wait Spin Count

COMMITWAIT_SPIN_COUNT specifies the number of times a GT.M process waiting for control of a block to complete an update should spin before yielding the CPU when GT.M runs on SMP machines.

Current transaction

The 64-bit hexadecimal number of the most recent database transaction.

Default Collation

The collation sequence currently defined for your database. DSE only reports this if an external collation algorithm is defined.

Desired DB Format

The desired version format of database blocks. Desired DB Format can have 2 possible values- the current major version or the last prior major version. Newly created databases and converted databases have the current major version.

Endian Format

The Endian byte ordering of the platform.

Extension Count

The number of GDS blocks by which the database file extends when it becomes full. The default value is 100. In production, typically this value should reflect the amount of new space needed in a relatively long period (say a week or a month). UNIX file systems use lazy allocations so this value controls the frequency at which GT.M checks the actual available space for database expansion in order to warn when space is low.

Flush timer

Indicates the time between completion of a database update and initiation of a timed flush of modified buffers. The default value is 1 second.

Flush trigger

The total number of modified buffers that trigger an updating process to initiate a flush.

Free blocks

The number of GDS blocks in the data portion of the file that are not currently part of the indexed database (that is, not in use). MUPIP INTEG -NOONLINE (including -FAST) can rectify this value if it is incorrect.

Free space

The number of currently unused blocks in the fileheader (for use by enhancements).

Freeze match

The PID extension (image count) for Cache freeze id - VMS only

Global Buffers

The number of BG buffers for the region. It can have values that are multiples of 512 (in bytes). The default value is 1024. In a production system, this value should typically be higher.

In critical section

The process identification number (PID) of the process in the write-critical section, or zero if no process holds the critical section.

Journal Alignsize

Specifies the number of 512-byte-blocks in the alignsize of the journal file. DSE only reports this field if journaling is ENABLED (or ON).

If the ALIGNSIZE is not a perfect power of 2, GT.M rounds it up to the nearest power of 2.

The default ALIGNSIZE value is 128 blocks (=64 kilobytes) and the minimum value is 32 blocks (=16 KiloBytes). The maximum value is 4194304 (=2 GigaBytes).

A small alignsize can make for faster recover or rollback operations, but makes less efficient use of space in the journal file.

Journal Allocation

The number of blocks at which GT.M starts testing the disk space remaining to support journal file extensions. DSE only reports this field if journaling is ENABLED (or ON).

Journal AutoSwitchLimit

The number of blocks after which GT.M automatically performs an implicit online switch to a new journal file. DSE only reports this field if journaling is ENABLED (or ON).

The default value for Journal AutoSwitchLimit is 8388600 & the maximum value is 8388607 blocks (4GB-512 bytes). The minimum value for Journal AutoSwitchLimit is 2048 (number of blocks needed to have 64 ALIGNSIZE boundaries each 16 KB, which is the minimum ALIGNSIZE).

Journal Before imaging

Indicates whether or not before image journaling is allowed; DSE only reports this field if journaling is ENABLED (or ON).

Journal Before imaging can either be TRUE or FALSE.

Journal Buffer Size

The amount of memory allotted to buffer journal file output. The default value of Journal Buffer Size is 128. Larger journal buffers can improve run-time performance, but they also increase the amount of information at risk in failure. Journal Buffer size must be large enough to hold the largest transaction.

Journal Epoch Interval

The elapsed time interval between two successive EPOCHs in seconds. An EPOCH is a checkpoint, at which all updates to a database file are committed to disk. All journal files contain epoch records. DSE only reports this field if journaling is ENABLED (or ON).

The default value of Journal Epoch Interval is 30 seconds. Longer Epoch Intervals can increase run-time performance, but they can also cause longer recovery times.

Journal Extension

The number of blocks used by GT.M to determine whether sufficient space remains to support continuing journal file growth. DSE only reports this field if journaling is ENABLED (or ON).

The default value of Journal Extension is 100. In production, this value should typically be either zero (0) to disable journal extensions and rely entirely on the Journal Allocation, or it should be large. In UNIX, this value serves largely to allow you to monitor the rate of journal file growth.

UNIX file systems use lazy allocations so this value controls the frequency at which GT.M checks the actual available space for journal file expansion in order to warn when space is low.

Journal File

The name of the journal file. DSE only reports this field if journaling is ENABLED (or ON).

Journal State

Indicates whether journaling is ON, OFF, or DISABLED (not allowed).

Journal Sync IO

Indicates whether WRITE operation to a journal file commits directly to disk. The default value is FALSE.

DSE only reports this field if journaling is ENABLED (or ON).

Journal Yield Limit

The number of times a process needing to flush journal buffer contents to disk yields its timeslice and waits for additional journal buffer content to be filled-in by concurrently active processes, before initiating a less than optimal I/O operation.

The minimum Journal Yield Limit is 0, the maximum Journal Yield Limit is 8.

The default value for Journal Yield Limit is 8. On a lightly loaded system, a small value can improve run-time performance, but on actively updating systems a higher level typically provides the best performance.

KILLs in progress

The sum of the number of processes currently cleaning up after multi-block KILLs and the number of Adandoned KILLs.

[Note]

Abandoned KILLs are associated with blocks incorrectly marked busy errors.

Last Bytestream Backup

The transaction number of the last transaction backed up with the MUPIP BACKUP -BYTESTREAM command.

Last Database Backup

The transaction number of the last transaction backed up with the MUPIP BACKUP -DATABASE command. (Note -DATABASE is the default BACKUP type.)

Last Record Backup

Transaction number of last MUPIP BACKUP -RECORD or FREEZE -RECORD command.

Lock space

A hexadecimal number indicating the pages of space dedicated to LOCK information.

The default value for Lock space is 28. In production with an application that makes heavy use of LOCKs, this value should be higher.

Master Bitmap Size

The size of the Master Bitmap. The current Master Bitmap Size of V5 format database is 112 (512 byte blocks).

Maximum key size

Maximum key size acts as a check on application design assumptions. Maximum key size must be at least three (3) or four (4) less than the record size but never more than 255. You may encounter database intergerity issues if you increase the maximum key size beyond 255. GT.M triggers an error if you decrease it and then make an attempt to update nodes with existing longer keys. .

Maximum record size

Maximum record size is a value less than or equal to the block size minus reserved bytes. The default value is 256. Maximum record size acts as a check on application design assumptions. You can increase it up to the block size less the block header size and the reserved bytes. You may encounter database integrity issues if you increase it beyond that size. GT.M triggers an error if you decrease and then make an attempt to update nodes with existing longer records.

Maximum TN

The maximum number of TNs that the current database can hold. For a database in V5 format, the default value of Maximum TN is 18,446,744,073,239,789,567 or 0xFFFFFFFFE3FFFFFF.

Maximum TN Warn

The transaction number after which GT.M generate a warning and update it to a new value. The default value of Maximum TN Warn is 0xFFFFFFFF73FFFFFF.

Modified cache blocks

The current number of modified blocks in the buffer pool waiting to be written to the database.

Mutex Hard Spin Count

The number of attempts to grab the mutex lock before initiating a less CPU-intensive wait period. The default value is 128.

Mutex Sleep Spin Count

The number of timed attempts to grab the mutex lock before initiating a wait based on interprocess wake-up signals. The default value is 128.

Mutex Spin Sleep Time

The number of milliseconds to sleep during a mutex sleep attempt. The default value is 2048.

No. of writes/flush

The number of blocks to write in each flush. The default value is 7.

Null subscripts

"ALWAYS" if null subscripts are legal. "NEVER" if they are not legal and "EXISTING" if they can be accessed and updated, but not created anew.

Number of local maps

(Total blocks + 511)\512.

Online Backup NBB

Block to which online backup has progressed. DSE displays this only when an online backup is currently in progress.

Reference count

The number of GT.M processes and utilities currently accessing that segment on a given node

Note: GT.M does not rely on this field. A database segment initially has a reference count of zero. When a GT.M process or utility accesses a segment, GT.M increments the reference count. GT.M decrements the reference count upon termination.

GT.M counts DSE as a process. When examining this field with DSE, the reference count is always greater than zero. When DSE is the only process using a region, the reference count should be one.

Region Seqno

The current replication relative time stamp for a region.

Replication State

Either On or OFF. [WAS ON] OFF means that replication is still working, but a problem with journaling has caused GT.M to turn it off, so GT.M is still replicating, but will turn replication OFF if it ever has to turn to the journal because the pool has lost data needed for replication.

Reserved Bytes

Number of bytes reserved in database blocks.

Starting VBN

Virtual Block Number of the first GDS block after the GDS file header; this is block 0 of the database and always holds the first local bitmap.

Timers pending

Number of processes considering a timed flush.

Total blocks

Total number of GDS blocks, including local bitmaps.

Wait Disk

Seconds that GT.M waits for disk space to become available before it ceases trying to flush a GDS block's content to disk. During the wait, it sends eight (8) approximately evenly spaced operator log messages before finally issuing a GTM-E-WAITDSKSPACE error. For example, if Wait Disk is 80 seconds and GT.M finds no disk space to flush a GDS block, it sends a GTM-E-WAITDSKSPACE syslog message about every 10 seconds, and after the eighth message issues a WAITDSKSPACE error. This field is only used in Unix because of its reliance on lazy disk space allocation.

Zqgblmod Seqno

The replication sequence number associated with the Zqgblmod Trans(action number).

Zqgblmod Trans

Transaction number used by the $ZQGBLMOD() function in testing whether a block was modified by overlapping transactions during a replication switchover.

Average Blocks Read per 100 Records

Acts as a clue for replication update helper processes as to how aggressively they should attempt to prefetch blocks. It's an estimate of the number of database blocks that GT.M reads for every 100 update records. The default value is 200. For very large databases, you can increase the value up to 400.

Update Process Reserved Area

An approximate percentage (integer value 0 to 100) of the number of global buffers reserved for the update process. The reader helper processes leaves at least this percentage of the global buffers for the update process. It can have any integer value between 0 to 100. The default value is 50.

Pre read trigger factor

The percentage of Update Process reserved area after which the update process processes signals the reader helper processes to resume processing journal records and reading global variables into the global buffer cache. It can have any integer value between 0 to 100. The default value is 50.

Update writer trigger factor

One of the parameters used by GT.M to manage the database is the flush trigger. One of several conditions that triggers normal GT.M processes to initiate flushing dirty buffers from the database global buffer cache is when the number of dirty buffers crosses the flush trigger. GT.M processes dynamically tune this value in normal use. In an attempt to never require the update process itself to flush dirty buffers, when the number of dirty global buffers crosses the update writer trigger factor percentage of the flush trigger value, writer helper processes start flushing dirty buffers to disk. It can have any integer value between 0 to 100. The default value is 33, that is, 33%.

MLOCK Space

The file header also provides M lock information, specifically whether any resource names belonging to the database region are locked. M code commonly uses locks as flags or semaphores controlling access to global data. Generally the LOCK argument specifies the resource of the same name as the name of the global variable that requires protected access. For more information about LOCKs, see the "M LOCK Utility" chapter and the LOCK section in the "Commands" chapter in the GT.M Programmer's Guide.