Appendix C. Building Encryption Libraries
.
Prev
|
Next
Appendix C. Building Encryption Libraries
Contents
Building Encryption Libraries
Debian, Ubuntu, RedHat and Fedora
IBM AIX 7.1 (pSeries)
Contents
Search
About This Manual
Intended Audience
Purpose of the Manual
How to Use This Manual
Overview
Conventions Used in This Manual
About GT.M
Hardware/Operating System Environment
Installation
Security
Program Development Environment
Database Subsystem
GT.M Utility Programs
GDE
MUPIP
LKE
DSE
Command Qualifiers
Database Integrity
Interprocess Communication
Installing GT.M
Obtaining GT.M Distribution Media
Before you begin
Installation Procedure
Compiling the Reference Implementation Plugin
Instructions
gtminstall script
Basic Operations
GT.M Environment Setup
gtmprofile
gtmcshrc
gtmbase
gdedefaults
gtm
Environment Variables
Configuring and operating GT.M with Unicode® support (optional)
M mode and UTF-8 mode
Compiling ICU
Starting GT.M
Configuring huge pages for GT.M x86[-64] on Linux
Using huge pages
Using huge pages for shared memory
Using huge pages for GT.M process private memory
Configuring the Restriction facility
ZSYSTEM and PIPE OPEN command restriction facility
Audit Logging facility
Logging
Global Directory Editor
Global Directory
GDE Overview
Identifying the Current Global Directory
Creating a Default Global Directory
Mapping Global Variables in a Global Directory
Examining the Default Global Directory
Global Directory Abbreviations
Customizing a Global Directory
Adding a Journaling Information Section
Using GDE
Guidelines for Mapping
Example of a Basic Mapping
Global Director Editor Commands
Specifying File Names in Command Lines
Font/Capitalization Conventions Used in this Chapter
@
Add
-Name
-Segment
-Region
-Gblname
Change
Delete
Exit
Help
LOCks
LOG
Quit
Rename
SEtgd
SHow
Template
Verify
Name, Region, and Segment Qualifiers
Name Qualifiers
Region Qualifiers
Segment Qualifiers
Gblname Qualifiers
Instance Qualifier
GDE Command Summary
GDE Command Qualifier Summary
General Database Management
Introduction
Operations - Standalone and Concurrent Access
MUPIP
Commands and Qualifiers
BACKUP
-BKupdbjnl
-Bytestream
-Database
-NETtimeout
-NEWJNLFILES
-Online
-Record
-REPLace
-REPLInstance
-RETry
-SHowprogress
-Since
-Transaction
Examples for MUPIP BACKUP
CREATE
-V6
-Region
Examples for MUPIP CREATE
DOWNGRADE
-VERSION={V5|V63000A}
Examples for MUPIP DOWNGRADE
DUMPFHEAD
-FILE=file
-FLUSH
-REGION=region
Examples for MUPIP DUMPFHEAD
ENDIANCVT
-OVerride
Examples for MUPIP ENDIANCVT
EXIT
EXTEND
-Blocks
Examples for MUPIP EXTEND
EXTRACT
-FOrmat
-FReeze
-LAbel
-LOg
-Null_iv
-Region
-Select
-STdout
Examples for MUPIP EXTRACT
FREEZE
-OFf
-ON
-[NO]A[UTORELEASE
-ONLine
-OVerride
-Record
Examples for MUPIP FREEZE
FTOK
-DB
-ID
-[NO]HEADER
-ONLY
-JNLPOOL
-RECVPOOL
HASH
INTEG
-ADjacency
-BLock
-BRief
-DMAXblocksize
-FAst
-FIle
-FUll
-IMAXblocksize
-Keyranges
-MAP
-MAXkeysize
-Online
-Region
-Subscript
-Stats
-TN_reset
-TRansaction
Examples for MUPIP INTEG
INTRPT
JOURNAL
LOAD
-FOrmat
-BEgin
-End
-FIll_factor
-Onerror
-Stdin
Examples for MUPIP LOAD
RCTLDUMP
REORG
-Downgrade
-Encrypt
-Exclude
-FILE
-FILL_factor
-Index_fill_factor
-Min_level
-Keep
-NOCOalesce
-NOSPlit
-NOSWap
-Resume
-Region
-Select
-Truncate
-UPgrade
Examples for MUPIP REORG
REPLICATE
RESTORE
-Extend
Examples for MUPIP RESTORE
RUNDOWN
-File
-Override
-Region
-Relinkctl
SEMAPHORE
SET
-File
-Jnlfile
-Region
-REPlication
-ACcess_method
-ASyncIO
-DATA_reserved_bytes
-Defer_time
-DEFER_Allocate
-Epochtaper
-ENCRYPTAble
-ENCRYPTIoncomplete
-EXtension_count
-Flush_time
-FUllblkwrt
-Global_buffers
-Hard_spin_count
-INDEX_reserved_bytes
-INST_freeze_on_error
-Journal
-Key_size
-Lck_shares_db_crit
-Lock_space
-Mutex_slots
-Null_subscripts
-PROblksplit
-Qdbrundown
-PArtial_recov_bypass
-Read_only
-RECord_size
-REServed_bytes
-SLeep_spin_count
-Spin_sleep_mask
-STATS
-StatsDB_allocation
-STDnullcoll
-Trigger_flush
-Version
-WAit_disk
-WRites_per_flush
Examples for MUPIP SET
SIZE
STOP
TRIGGER
Examples for MUPIP TRIGGER
UPGRADE
Example for MUPIP UPGRADE
MUPIP Command Summary
GT.M Journaling
Introduction
Journal Files
Recovery from a Journal File
Forward Recovery
Backward Recovery
rolled_bak* files
Journal Files Access Authorization
Triggers in Journal Files
BEFORE_IMAGE Journaling
NOBEFORE_IMAGE Journaling
Choosing between BEFORE_IMAGE and NOBEFORE_IMAGE
Broken Transaction File
Lost Transaction File
Epoch
Journaling Benefits
Backup Journal Files
Select database files for Journaling
Fencing Transactions
Deciding Whether to Use Fencing
Fencing Advantages
Fencing Disadvantages
VIEW Keywords
$VIEW() Keywords
SET
SET Object Identifying Qualifiers
SET Action Qualifiers
SET -JOURNAL Options
Examples for MUPIP SET
JOURNAL
Journal Action Qualifiers
-EXtract[=<file-name>|-stdout]
-PARA[LLEL][=n]
-RECover
-ROLLBACK [{-ON[LINE]|-NOO[NLINE]}]
-SHow=show-option-list
-[NO]Verify
Journal Direction Qualifiers
Journal Time Qualifiers
Journal Sequence Number Qualifiers
Journal Control Qualifiers
Journal Selection Qualifiers
Journal Extract Formats
Database Replication
Introduction
LMS Group
Database Transaction Number
Journal Sequence Number
Stream Sequence Number
Instance Bound Global Directories
Examples
Simple Example
Ensuring Consistency with Rollback
Rollback Not Desired or Required by Application Design
Two Originating Primary Failures
Replication and Online Rollback
Limitations - SI Replication
Replication Architecture
Helper Processes
Filters
Replication Instance File
Implementing Replication and Recovery
Application Architecture
Implement a Message Delivery System
System Requirements
Root Primary Status Identification
Switchover
Instance Freeze
TLS/SSL Replication
Network Link between Systems
Choosing between BEFORE_IMAGE and NOBEFORE_IMAGE journaling
Recovery
Comparison other than Recovery
Database Repair
Procedures
Download Replication Examples
env
db_create
backup_repl
repl_setup
originating_start
replicating_start
suppl_setup
repl_status
rollback
originating_stop
replicating_stop
replicating_start_suppl_n
Setting up an A→B replication configuration with empty databases
Setting up an A→B→C replication configuration with empty databases
Setting up an A→P replication configuration with empty databases
Replicating Instance Starts from Backup of Originating Instance (A→B and A→P )
Switchover possibilities in an A→B replication configuration
Switchover possibilities in a B←A→P replication configuration
A requires rollback
A and P require rollback
Rollback not required by application design
Rollback automatically
Switchover possibilities in a B←A→P→Q replication configuration
Changing the global directory in an A→B replication configuration
Rolling Software Upgrade
Upgrade the replicating instance first (A→B)
Upgrade the originating instance first (A→B)
Shutting down an instance
Creating a new Replication Instance File
Setting up a secured TLS replication connection
Schema Change Filters
Recovering from the replication WAS_ON state
Rollback data from crashed (idle) regions
Setting up a new replicating instance of an originating instance (A→B, P→Q, or A→P)
Replacing the replication instance file of a replicating instance (A→B and P→Q)
Replacing the replication instance file of a replicating instance (A→P)
Setting up a new replicating instance from a backup of the originating instance (A→P)
Setting up an A→P configuration for the first time if P is an existing instance (having its own set of updates)
Commands and Qualifiers
Turning Replication On/Off
Creating the Replication Instance File
Displaying/Changing the attributes of Replication Instance File and Journal Pool
Starting the Source Server
Shutting down the Source Server
Activating a Passive Source Server
Deactivating an Active Source Server
Stopping the Source Filter
Stopping the Receiver Filter
Freezing an Instance
Checking Server Health
Changing the Log File
Enabling/Disabling Detailed Logging
Reporting the Current Backlog of Journal Records
Processing Lost Transactions File
Lost Transaction File format
Starting the Receiver Server
Starting the Update Process
Stopping the Receiver Server and/or the Update Process
Checking Server Health
Changing the Log File
Enabling/Disabling Detailed Logging
Reporting the Current Backlog of Journal Records
Rolling Back a Replicated Database
M Lock Utility (LKE)
Introduction
To Invoke and Exit LKE
To establish a Global Directory
LKE Commands and Qualifiers
Clear
CLNup
SHow
Exit
Help
SPawn
Summary
LKE Exercises
Exercise 1: Preventing concurrent updates using M Locks
Exercise 2: Rectifying a deadlock situation
GT.M Database Structure(GDS)
Database File Organization with GDS
Database File Header
File Header Data Elements
Local Bitmaps
Master Bitmap
Database Structure
Tree Organization
GDS Blocks
GDS Records
Using GDS records to hold spanning nodes
GDS Keys
Compression Count
Use of Keys
Characteristics of Keys
Global Variable Names
String Subscripts
Numeric Subscripts
Database Structure Editor
Operating in DSE
DSE Commands and Qualifiers
ADD
Qualifiers of ADD
Examples for ADD
ALL
Qualifiers
Examples of ALL
Buffer_flush
CHange
CHANGE -BLock Qualifiers
CHANGE -FIleheader Qualifiers
Examples for CHANGE
CAche
Qualifiers of CACHE
Examples for CACHE
CLose
CRitical
Qualifiers of CRITICAL
Examples for CRITICAL
Dump
Qualifiers of DUMP
Examples for DUMP
EValuate
Qualifiers of Evaluate
Examples for EVALUATE
EXit
Find
Qualifiers of FIND
Examples for FIND
Help
Integrit
Qualifiers of Integrit
Maps
Qualifiers for MAP
Examples
OPen
Qualifiers for OPEN
Examples for OPEN
OVerwrite
Qualifiers for OVERWRITE
Examples for Overwrite
Page
RAnge
Qualifiers of RANGE
Examples for RANGE
REMove
Qualifiers of REMOVE
REStore
Qualifiers of RESTORE
SAve
Qualifiers of SAVE
SHift
Qualifiers of SHIFT
SPawn
Examples of SPAWN
Wcinit
DSE Command Summary
Maintaining Database Integrity
Verifying Database Integrity
Regularly Scheduled Verification
Before or After Major Transfers
Immediately after Catastrophic Events
Immediately after Run-Time Database Errors
Immediately After Database Repairs
Approaches to Database Recovery
Recover from Journals
Restore from Backup
Repair with DSE
Preventive Maintenance
Determining the Cause of the Problem
MUPIP Recovery
Follow-up
Repairing the Database with DSE
Using the Proper Database File
Locating Structures with DSE
Safety in Repairs
Discarding Data
Concurrent Repairs
Terminating Processes
Recovering data from damaged binary extracts
CORRUPT Errors
LDSPANGLOINCMP Errors
Example–Repairing an error in a binary extract
Finding and Fixing Database Errors
C1–Possible Cache Control Problems
H1–Process Hangs
H3–Database Access Problems
H4–Database Cache Problems
H5–Critical Section Problems
H6–UNIX Problems
H7–Disk Hardware Problems
H8–Application Problems
Persistent LOCKs
Deadlocks
Preventing Deadlocks
I1–MUPIP INTEG Errors
Evaluating the Danger Level of a Database Problem
Requires Immediate Attention
Can Be Deferred
MUPIP INTEG Error Classification Table
I2–GT.M Version Mismatch
I3–File Header Errors
I4–File Size Errors
I5–More Database Access Problems
I6–Transient Errors
I7–Database Rundown Problem
I8–Repair-Induced Problems
K1–Bad Key
K2–Keys Misplaced
K3–Block Doubly Allocated
K4–Pointer Problems
K5–Star Key Problems
K6–Compression Count Error
K7–Key Warning
M1–Bitmap Errors
M2–Bitmap Header Problems
O1–Bad Block
O2–Record Errors
O3–Data Block Errors
O4–Salvage of Data Blocks with Lost Indices
Download salvage.m
O5–Salvage of a damaged spanning node
O6–Block Size Errors
P1–Process Damage
Q1–Restricting Database Access
R1–GT.M Run-Time Errors
R2–Structural Database Integrity Errors
Run-Time Database Restart Codes
R3–Run-time Database Cache Problems
R4–Stopped Processes
R5–No More Room in the File
R6–GTMASSERT and GTMCHECK Errors
R7–Interlocked Queue Hardware Problems
R8–Database Tree Maximum Level Exceeded
R9–Read-only Process Blocked
Database Encryption
Introduction
Overview
Disclaimer
Limitations of GT.M Database Encryption
Data Not At Rest Not Protected
Keys in the Process Address Space / Environment
Long Lived Keys
Voluminous Samples of Encrypted Data
Encryption Algorithms Neither Endorsed Nor Supported by FIS
No Key Recovery
Human Intervention Required
MM Databases
Alternatives to Database Encryption
Device IO
GT.CM
FIPS Mode
Theory of Operation
Definition of Terms
Overview
Warning
Data in Database and Journal Files
Symmetric and Asymmetric Ciphers
Key Ring on Disk
Master Key Configuration File and Encryption Keys
Memory Key Ring
Key Validation and Hashing
Database Operation
Examples of use
Key Management
Tested Reference Implementations
Special note - GNU Privacy Guard and Agents
Using the reference implementation's custom pinentry program
Installation
Administration and Operation of Encrypted Databases
Utility Programs
GDE
Format / Upgrade
-[NO]ENcryption
MUPIP
MUPIP CREATE
MUPIP JOURNAL
MUPIP LOAD
DSE
Changing the hash in the database file header
Changing the Encryption Keys
Encrypted Database Creation
Plugin Architecture & Interface
Packaging
Extensions to the GT.M External Interface
Operation
Using the Reference Implementation with Older Releases
GT.CM Client/Server
Introduction
Overview
GT.CM Server
GT.CM Client
GT.CM Server Startup and Shutdown
GT.CM Server Startup
GT.CM Server Shutdown
Types of Operations
Error Messages
Examples
GT.M's IPC Resource Usage
Examining GT.M's IPC Resources
gmtsecshr
Monitoring GT.M
Monitoring GT.M Messages
Managing core dumps
Building Encryption Libraries
Building Encryption Libraries
Debian, Ubuntu, RedHat and Fedora
IBM AIX 7.1 (pSeries)
GPG-ERROR
GCRYPT
CRYPTO (From OpenSSL)
GPGME
GNUPG
GT.M Security Philosophy
Philosophy
Normal User and Group Id Rule
Exceptions
Shared Memory Exception for BG
gtmsecshr Exception
Triggers
Recommendations
gtmsecshr commands
Shared Resource Authorization Permissions
GTMPCAT - GT.M Process/Core Analysis Tool
Overview
Usage
Interactive Mode
Packaging GT.M Applications
Setting up a Captive User Application with GT.M
Sample .profile
Invoking GT.M through a C main() program
Defensive Practices
Other
Creating a $gtmcrypt_config file
Why do we need a $gtmcrypt_config file?
OpenSSL Options
Generating demo TLS certificates
Creating a demo CA (Certification Authority)
Creating and signing demo leaf-level certificates
Example: Creating demo certificates
cert_setup
gen_ca
gen_leaf
example101.cnf
V6 to V7 DB Upgrade
Database Upgrade Matrix
Search
