Torrus Framework Architecture

Configuration Processing

The XML configuration is compiled into the database representation by operator's manual request.

The compiled version of configuration is not a one-to-one representation of the XML version. All aliases and templates are expanded. The backwards restoration of XML from the database is available with the snapshot utility.

Aliases are the way to represent the data in a more convenient format. An alias can point to a subtree or a leaf, and it works similarly as a symbolic link in a filesystem.

A template defines a piece of configuration which can be used in multiple places. Templates can be nested.

The configuration can consist of several XML files. They will be processed in the specified order. Each new file is treated as an additive information to the existing tree.

The XML configuration compiler validates all the mandatory parameters.

Database Architecture

All runtime data is stored in Berkeley DB database environment (http://www.sleepycat.com).

The compiled version of the configuration XML is stored in the ds_config_DSINST.db and other_config_OINST.db. These databases have similar structure, and ds_config_DSINST.db keeps all datasource-related information. DSINST and OINST stand for the productive instance number, and have values of 0 or 1. Current productive instance numbers are stored in db_config_instances.db database.

For each datasource tree, the database files are resided in /var/torrus/db/sub/>tree_name< directory.

The runtime modules access the configuration via ConfigTree objects.

Each datasource subtree or leaf is identified by a token. A token is a short alphanumeric string, generated internally. Two types of tokens are recognized: single tokens and token sets.

Single token starts with letter T. The rest is made with decimal digts.

Token set name starts with letter S. The rest is an arbitrary sequence of word characters.

The special token SS is reserved for tokensets list. Also tokenset parameters are inherited from this token's parameters.

View and monitor names must be unique, and must start with a lower case letter.

db_config_instances.db is a Hash database, with keys of form ds:<tree_name> or other:<tree_name>, and 0 or 1 as values. Also the compiler adds an entry compiling:<tree_name> during the compilation time, in order to avoid two compiler processes running at the same time on the same tree.

ds_config_DSINST.db is a Btree database, with the keys and values defined as follows:

other_config_OINST.db is a Btree database, with the keys and values defined as follows:

paramprops_DSINST.db is a Btree database for storing the datasource parameter properties, such as expandable, list parameters, searchable, etc.:

aliases_DSINST.db is a Btree database with alias paths as keys and target tokens as values. It is used for quick alias expansion.

tokensets_DSINST.db is a Hash database containing the token sets. The keys and values are as follows:

nodepcache_DSINST.db is a Btree database containing the cached node parameter values. The keys and values are as follows:

nodeid_DSINST.db is a Btree database that stores the mapping between NodeID values and tokens. Database keys are NodeID strings, and values are tokens. One NodeID corresponds to maximum one token.

config_readers.db is a Hash database which contains the informaton about active processes which read the configuration. The configuration compiler waits until all readers finish using the current configuration database. Process IDs are used as keys, and the values contain timestamps.

timestamps.db is a Hash database containing various kinds of timestamps. The timestamp name is the key, and the number of seconds since epoch is the value.

render_cache.db keeps the status information about the graphs ready to display. Last known timestamp of the configuration is compared with the actual one. When the actual timestamp differs from known, the renderer cache is cleaned up. This is a Hash database, with the following keys and values:

monitor_cache.db is a Hash database used in order to avoid the unneccessary configuration tree walk. The keys are the leaf tokens, and the values are comma-separated monitor names. At each monitor invocation, the confguration timestamp is compared against the last known, and the cache database is rebuilt if needed.

monitor_alarms.db is a Hash database that keeps alarm status information from previous runs of Monitor, with the keys and values as follows:

collector_tokens_X_Y.db is a Hash database used in order to avoid the unneccessary configuration tree walk. X is the collector instance number, and Y is the datasource configuration instance number. Keys and values are as follows:

scheduler_stats.db is a Btree database which stores the runtime statistics of Scheduler tasks. Each key is of structure <taskname>:<period>:<offset>:<variable>, and the value is a number representing the current value of the variable. Depending on variable purpose, the number is floating point or integer.

users.db is a Hash database containing user details, passwords, and group membership:

acl.db is a Hash database containing group privileges information:

serviceid_params.db is a Btree database containing properties for each Service ID (exported collector information, usually stored in an SQL database):

searchwords.db is a Btree database with DB_DUP and DB_DUPSORT flags. It contains the search strings for the given tree:

globsearchwords.db is a Btree database with DB_DUP and DB_DUPSORT flags. It contains the search strings for all trees:

snmp_failures_X.db is a Btree database containing SNMP collector failures information for a given collector instance for a tree.

collector_lock.db is a Hash database that does not contain any data. It's only used for acquiring exclusive locks for collector processes.

Modular Structure

The Torrus framework consists of several functional modules:


Author

Copyright (c) 2002-2005 Stanislav Sinyagin ssinyagin@k-open.com