Defining a FileLine Descriptor

A FileLine descriptor defines aspects of monitoring a text file. For each FileLine entry in the configuration, the Publisher master process JOBs a process that reads the monitored file line by line, executing a GT.M expression (an extrinsic function call) for each line read in order to gather relevant information. The FileLine process OPENs the monitored file with the FOLLOW deviceparameter and then READs individual lines (the GT.M equivalent of tail -f).

The syntax of a FileLine descriptor is

FileLine:{InfoDictID | InfoDictName}:[FileLineName]:[FileLineID]:{PublisherID | PublisherName}:Filename:[CheckCycle]:[Timeout]:[PieceSeparator]:[PreExpr]:[InfoExpr]:[PostExpr]

InfoDictID or InfoDictName: The ID or name of the InfoDict domain that maps FileLineID to FileLineName.
FileLineName: The name of the FileLine.
If FileLineName is specified, the configuration gets the FileLineID from the specified InfoDict domain. If the specified domain has no definition for FileLineID, the InfoHub configuration process generates the FileLineID by adding one or two to the last ID used in this InfoDict domain to obtain an odd-numbered entry. Note that the SNMP sub-agent provided with the InfoHub Reference Implementation uses FileLineID as an index in the OIDs for each FileLine.
FileLineID: The ID of the FileLine.
If FileLineID is specified, the configuration process verifies before configuration completion that the InfoDict domain maps that FileLineID to a FileLineName; if the configuration detects that there is no FileLineName or FileLineID, it produces an IHIDORNAMEREQ error.
The descriptor must specify one, or both, of FileLineName and FileLineID. When the descriptor specifies both the name and ID and there is no corresponding InfoDict entry, configuration processing implicitly creates an InfoDict entry. If the configuration file specifies the same ID for more than one FileLine associated with a Publisher, the configuration processing produces an IHDUPFILELINE error.
PublisherID or PublisherName: The ID or name of the Publisher associated with the FileLine.
Filename: The absolute path to the monitored file. If configuration processing determines the path is not absolute, it produces an IHABSPATHREQ error.
CheckCycle:The period of time in seconds at which a FileLine process checks for a newer version of Filename.
Because a file can be renamed and replaced while the InfoHub has it open (for example, under the control of an application such as logrotate), if CheckCycle is specified, FileLine process checks every CheckCycle seconds whether the name of the file it has open still matches Filename; if not, it closes the file it has open, and opens the current file Filename. A value of 0 (the default) means InfoHub should not check for a newer file. If the configuration processing finds a non-numeric cycle value, it produces an IHBADCYCLE error.
Timeout: The maximum amount of time, in seconds, that a FileLine process should wait for additional input in Filename before it shuts down.
When the FileLine process is asked to terminate, it issues a USE of the file with NOFOLLOW and proceeds with timed READs for the duration of Timeout seconds. If FileLine receives an end-of-file prior to that time, it does not wait for additional input. If not specified, Timeout defaults to one second. If the configuration processing finds a non-numeric timeout, it produces an IHBADTIMEOUT error.
PieceSeparator: A single character (which can span several bytes if the InfoHub is operating in UTF-8 mode) that specifies a delimiter used to separate subscripts and information returned by PreExpr, PostExpr, and InfoExpr.
If PieceSeparator is not specified, it defaults to a colon (":"). Since colons separate syntactic elements of the FileLine descriptor, the only way to specify a colon as the PieceSeparator is to default it. The PieceSeparator can also be the case-insensitive keyword "VAR" which indicates that the first character of the expression returned by the gleaner has to be removed and used as a delimiter by the Reporting process. If configuration processing finds that the specified PieceSeparator is not an empty-string, a single character, the keyword "VAR", or a valid $CHAR() expression, it produces an IHBADDPSEP error.
PreExpr: A GT.M extrinsic function call, which, if specified, automatically executes before FileLine starts to process Filename, and for every subsequent Filename rotation. If a FileLine gleaner needs certain values initialized only once, it must differentiate between the first and subsequent invocations of PreExpr. Like the other *Expr functions, it can return zero or more PieceSeparator-delimited values, as appropriate, for storage in the InfoHub database. If not specified, PreExpr defaults to an empty string.
InfoExpr: A GT.M extrinsic function call that returns zero to many (typically one) PieceSeparator-delimited sets of "key<PieceSeperator>value" expressions for each line of the monitored file. Note that a key should be the numeric ID of a defined InfoDictItem. The local variable %l is available to InfoExpr with the current line read from Filename. A typical InfoDictItem might correspond to a particular message or message type logged in the operator log. If not specified, InfoExpr defaults to an empty string.
PostExpr: A GT.M extrinsic function call which, if specified, automatically executes at the end of processing Filename, either when a new version of Filename is detected, or at InfoHub shutdown. If a FileLine gleaner needs certain values summarized only once, it must differentiate between the last and prior invocations of PostExpr. Like the other *Expr functions, it can return zero or more PieceSeparator-delimited values, as appropriate, for storage in the InfoHub database. If not specified, PostExpr defaults to an empty string.
If PreExpr, InfoExpr, or PostExpr return a value which is not an empty string or a string in the format below, the FileLine process terminates. In order for Subscriptions to detect an error, the Expr should return a "key<PieceSeparator>value" pair for the error. The normal return is an empty string (meaning no information stored), or one or more "key : value" pairs to store, in the form of one or more comma-delimited positive integer keys, followed by the value to be stored, separated by PieceSeparator. Multiple pairs could be (as an implementation detail) separated by pairs of PieceSeparators. Neither the key nor the value can contain a PieceSeparator.
If the Configuration File does not specify at least one of PreExpr, InfoExpr, and PostExpr, configuration processing produces an IHSOMEEXPRREQ error.

	InfoHub Internals
On starting up, a FileLine process executes its PreExpr, gathering any information provided, and logging an error if the data does not match the expected format, as described earlier. In the event that PreExpr does not return an error, the FileLine process OPENs the file using the FOLLOW deviceparameter, and sits on an untimed READ of the file. Every line read gets processed by InfoExpr, and the returned data placed into the InfoHub database. Triggers on ^InfoHubInfo provide asynchronous notification to Subscribers; this eliminates polling to provide the most efficient processing (all other actions, including commands to shutdown, are sent via asynchronous interrupt-driven messages). In response to a shutdown command, the FileLine process executes its PostExpr, which may provide summary information, as appropriate, before shutting down. GT.M V6.0-002 is the minimum compatible version because InfoHub uses the FOLLOW deviceparameter functionality which was first introduced in V6.0-002. For information on installing GT.M, refer to "Installing GT.M" chapter of the UNIX Administration and Operations Guide. Configuration stores FileLine specifications in the following node: ^InfoHubConfInfoHubID,"Publishers",PublisherID,BeginningSequenceNumber,FileLineID)= EndingSequenceNumber]:FileName:InfoDictID:FileLineName:CheckCycle:Timeout:PieceSeparator]:[PreExpr]:[InfoExpr]:[PostExpr] PreExpr, InfoExpr, and PostExpr always execute with $ZGBLDIR pointing to the global directory in the temporary directory. The FileLine process logs gathered information in the InfoHub database using an extended reference.

InfoHub Internals

On starting up, a FileLine process executes its PreExpr, gathering any information provided, and logging an error if the data does not match the expected format, as described earlier. In the event that PreExpr does not return an error, the FileLine process OPENs the file using the FOLLOW deviceparameter, and sits on an untimed READ of the file. Every line read gets processed by InfoExpr, and the returned data placed into the InfoHub database. Triggers on ^InfoHubInfo provide asynchronous notification to Subscribers; this eliminates polling to provide the most efficient processing (all other actions, including commands to shutdown, are sent via asynchronous interrupt-driven messages). In response to a shutdown command, the FileLine process executes its PostExpr, which may provide summary information, as appropriate, before shutting down.

GT.M V6.0-002 is the minimum compatible version because InfoHub uses the FOLLOW deviceparameter functionality which was first introduced in V6.0-002. For information on installing GT.M, refer to "Installing GT.M" chapter of the UNIX Administration and Operations Guide.

Configuration stores FileLine specifications in the following node:

^InfoHubConfInfoHubID,"Publishers",PublisherID,BeginningSequenceNumber,FileLineID)= EndingSequenceNumber]:FileName:InfoDictID:FileLineName:CheckCycle:Timeout:PieceSeparator]:[PreExpr]:[InfoExpr]:[PostExpr]

PreExpr, InfoExpr, and PostExpr always execute with $ZGBLDIR pointing to the global directory in the temporary directory. The FileLine process logs gathered information in the InfoHub database using an extended reference.

Example:

FileLine:GenericDict:OpLog::System:/var/log/messages:2:1:$char(30):PreExpr^LogFileGleaner:InfoExpr^LogFileGleaner:PostExpr^LogFileGleaner

FileLine:GenericDict:AuthLog::System:/var/log/auth.log:2:1:$char(30):PreExpr^LogFileGleaner:InfoExpr^LogFileGleaner:PostExpr^LogFileGleaner

This example configures two FileLine Gleaners, OpLog and AuthLog, as presented in the SimpleMonitor.conf configuration file in the ULFM Reference Implementation. The referenced PreExpr, InfoExpr, and PostExpr functions are implemented in "LogFileGleaner.m" (also available in the ULFM Reference Implementation), which provides a simple implementation of a gleaner that processes information from the system log.

Prev	Up	Next
	Home

Defining a FileLine DescriptorChapter 4. Configuring an InfoHub

Defining a FileLine Descriptor

Defining a FileLine Descriptor
Chapter 4. Configuring an InfoHub