Torrus: The Data Series Processing Framework

In short, Torrus is an alternative software platform to MRTG, Cricket and Cacti. In most cases it brings more flexibility and performance. We are able to poll more than 50,000 SNMP OIDs every 5 minutes from a single moderately equipped server, with simultaneous threshold monitoring of half of them.

Torrus is designed to be the universal data series processing framework. Its scalable hierarchical design, application-independent core, and highly customizable architecture make Torrus an attractive choice both for small installations and for big enterprise or carrier networks. Although most of our users deploy Torrus for SNMP monitoring, it might be useful for data series of any nature. Tobi Oetiker's RRDtool is used for data storage.

See the Torrus demo server, provided by Bluewin AG.

See also Torrus Wiki for useful information and tips.

Torrus functional overview slideshow gives some conceptional and quick explanations (Open Office presentation, PDF)

Torrus presentation at SwiNOG meeting tells where we are today (Open Office presentation, PDF)

Currently Torrus consists of the following parts:

Configuration compiler and validator. It processes the XML configuration files and saves the configuration into a database.
View renderer and the web interface. They generate HTML and the graphical representation of the datasources and provide user authentication and authorization. All generated output is controlled by the configuration parameters and templates. The users can easily create their own presentation of data series.
SNMP Collector. Thanks to David M. Town's Net::SNMP Perl module, Torrus has got an extremely efficient SNMP data poller, with ability to poll dozens of thousands of SNMP variables from a single server. Modular collector core architecture allows further extension with new collector and storage types. Any datasource can have its own polling schedule.
SNMP Device Discovery Tool. Devdiscover is a new, modular, flexible, and expandable tool for automatic generation of Torrus configuration files. New device types and MIBs are easily added as independent Perl modules. Thanks to our contributors, the number of supported devices and vendors is constantly growing.
Billing reports generator. Torrus can be configured to store the network traffic usage in an SQL database (without RRDTools' aggregation and interpolation), and produce monthly reports on traffic volume and 95th Percentile bandwidth usage.
Threshold monitor. All data, regardless of their type and nature, can be monitored according to the user-defined rules. The rules can also include the datasource-specific parameters, e.g. boundary values etc. The following types of action may be combined for event reporting: 1) Displaying the list of alarmed datasources; 2) E-mail notification; 3) SNMP trap; 4) Any custom executable. Thresholds are specified by RPN expressions.

See the Project summary and Download Area at SourceForge.Net.

Current development snapshots are available for testing.

Documentation

Installation Instructions: system requirements and installation steps
Web Interface Reference: installation details for mod_perl handler
RRFW to Torrus migration guide: step by step guide for those having an RRFW installation.
User Guide: explains what is where and how to start using the system.
Scalability Guide: advanced setup guidelines for large-scale monitoring platforms.
SNMP Discovery User Guide: Guide to SNMP discovery tools
Configuration Guide: XML configuration reference
Vendor Support List: List of vendors and MIBs supported.
Manual pages: all executables documented
Reporting Setup Guide: Traffic usage and 95th percentile reports
RPN expressions in Torrus: Torrus-specific features in RPN expressions, as compared to rrdtool graph
Styling Profile Guide: how to create your own display styles for Torrus graphs
Development documentation: internals documented, working drafts, and technical reqirements.
Plugins documentation: optional features and contributions.

Features of Torrus

Data-oriented design. Most of the existing frontends are SNMP-centric. In Torrus, the SNMP collector is not the central part, but rather one of possible data sources. It is easy to integrate some existing RRD files that are updated by some external agents.
Multiple independent datasource trees in one instance of Torrus.
User authentication and authorization. Users belong to groups, and each group has privileges to browse a certain subset of trees.
Flexible hierarchical configuration in XML format. Multuiple XML files are processed additively.
Externally generated RRD files can be automatically grouped, based on the filename regular expressions.
Flexible representation of the data. One datasource may be displayed in multiple subtrees of the hierarchy. It allows you to group them at your convenience.
Aberrant behaviour detection based on Holt-Winters prediction is fully supported (requires rrdtool version 1.1.x). See the references below for more information.
Support for international character sets (Latin1 and Unicode).
First open-source complete implementation of Cisco QoS (CISCO-CLASS-BASED-QOS-MIB) discovery and monitoring.

Up-to-date list of ongoing tasks is published in TODO document.

Product Support

Torrus is free software. It is distributed under the GNU General Public License.

Please send all your requests, comments, and bug reports to the torrus-users mailing list. Only list subscribers are allowed to post the messages there. All new release announcements are sent to this list.

The development issues of Torrus are discussed in torrus-devel mailing list. Copies of all CVS commits are posted to torrus-commit mailing list.

Archives of the old mailing lists are available for rrfw-users and rrfw-devel .

Commercial support for Torrus is provided by:

K-Open GmbH
Switzerland
Tel. +41 79 407 0224

Commercial support includes:

Software installation and customizing
Development of custom user data import utilities
Development of new Torrus features
Software maintenance contracts

For more details about commercial support, please contact us by email.

SNMP MIBs

By courtesy of Tobi Oetiker, we are using the subtree of RRDtool Enterprise ID. Torrus is now capable of sending the SNMPv2c traps, with the variables described in the corresponding MIB file.

RRDTOOL-SMI: the original file from RRDtool enterprise tree
Torrus-MIB: The MIB which describes the SNMPv2c traps sent by action_snmptrap (see Manpages for details)

References

Torrus was the new name for RRFW, the Round Robin Database Framework. The changes are summarized in the transition roadmap. Old RRFW documentation is available at RRFW home page.
Development of RRFW releases 0.0.1 to 0.0.7, as well as development and testing of Cisco class-based QoS monitoring was sponsored by Cablecom GmbH Internet Core Routing Services Team.
Cablecom is the leading country-wide cable-TV and Internet provider in Switzerland.
This project would not even start without Tobi Oetiker's RRDTool.
RRD is a very efficient and popular system to store and display time-series data.
Many concepts and ideas were taken from Cricket.
Cricket is a widely used and very efficient network monitoring system. However, its limitations were actually the reason to start this project.
Jake Brutlag. Aberrant Behavior Detection in Time Series for Network Service Monitoring.
LISA 2000 paper.
The article gives the basic theory overview of Holt-Winters forecasting algorithm, and describes its implementation in RRDTool.
Jake Brutlag. Notes on RRDTOOL implementation of Aberrant Behavior Detection.
The article describes the implementation of aberrant behavior detection for RRDTOOL.

Credits

Chris Amley. Xylan switches SNMP discovery
Scott Brooks. Atmel wireless devices SNMP discovery
Aaron Bush. Contributed APC and HP vendor templates.
Shawn Ferry <sferry at sevenspace.com> <lalartu at obscure.org>. Many contributions in various parts.
Marc Haber. Lots of bug reports and architecture ideas.
Roman Hochuli. Bug reports and new feature ideas. Vendor templates for Ascend and DOCSIS.
Ian Holsman <http://holsman.net>. Adapted the bluerobot.com HTML layout design for Torrus.
Gord Philpott. Numerous bug reports.
Christian Schnidrig. Architecture ideas and interface improvements.
Jurij Smakov. Contributed in manpages creation. Maintainer of Torrus Debian package
Jon Nistor. Bug reports and contributions for JunOS, Cisco SCE and other vendors discovery.