You are here: TWiki > Integration Web>SysLogNg (16 Dec 2008, KyleGross)

SysLogNg

Introduction

The Center for Distributed Petascale Science (CEDPS) SciDAC project is developing a set of tools to aid in Grid troubleshooting. These new tools assume the existence of one or more centralized log file repositories. We assume most sites will want a site repository, and that a subset of the site data will be forwarded to one or more central OSG and/or VO repositories.

The CEDPS project recommends the commonly used, open-source (GPL) tool called syslog-ng for this purpose. syslog-ng (v1.6) is included with all standard Linux and BSD distributions, but is typically not installed by default. However the new version of syslog-ng (v2.0) is recommended due to support for ISO timestamps, including time zone support

Eventually other CEDPS troubleshooting services will need to be deployed on the central log archive hosts. The requirements for those services will be detailed in a future readiness plan.

Description of the Service

syslog-ng can be used to collect existing log files and forward them to one or more log destinations. syslog-ng, unlike standard syslog, includes the following features that make it ideal for this purpose:

  • A powerful regular expression (regex)-based filtering mechanism to only forward logs of interest.
  • Timezone support
  • Fully qualified hostnames
  • Automatic reconnect if socket connections fail
  • Support for fifos, TCP, and UDP
  • Built-in file rotation
  • Secure data transport (via stunnel)

Duration: Indefinite, as we expect syslog-ng to be useful to the ITB itself.

Dependencies and Other Services

For encrypted log file transfers, the standard open source tool stunnel is required. Otherwise there are no dependencies.

Required Resources

A syslog-ng forwarder should be installed on all hosts running Grid services such as gridFTP, gatekeeper, WS-Gram, VOMS, or Condor-G.

Server Requirements

One central collection server per site is needed, and well as one central collection server for OSG.

A moderate amount of disk space is required for the central log file collection hosts (maybe 100 GB?) One central collection host per site should be identified. CPU and network requirements are minimal, so this does not need to be a dedicated machine.

Note that by default the syslog-ng port of the central collector is open to the world. This can be protected using tcp_wrappers or iptables. The installation instructions for the central collector should include information on optionally configuring one of these mechanisms.

Packaging

syslog-ng should be packaged with VDT with sample configuration files and installed via pacman.

Installation and Configuration

syslog-ng can be run as a standard user on a non-privileged network port, or it can be installed as a drop-in replacement for the standard system syslog daemon. For OSG we recommend it be installed as user 'nobody' or 'daemon' on a non-privileged network port.

Sample configuration files and boot scripts can be found on the CEDPS wiki

Configuration for vtb sites can be found at VTBSyslogNG.

Test Harness

To test the syslog-ng configuration, run something like this:

        /usr/bin/logger -t test -u /tmp/gridlogs.socket "this is a test message"

Where "-u" points to the listen socket of your syslog-ng server, (See http://www.cedps.net/wiki/index.php/Syslog-ng) and verify that the test message shows up on the central collection host.

Validation

syslog-ng contains a number of performance tuning knobs to help guarantee that it does not drop log messages or overwhelm the central collection host. These will need to be tested on the VTB and ITB. Fortunately syslog-ng reports anytime messages are lost, so it will be fairly easy to monitor whether or not the syslog-ng configuration is working well.

We will need to write test scripts that generate lots of log messages on many hosts to verify that the central collection host can keep up. Testing must be done to determine the scalability limits for a single receiver in terms of both number of senders and sender rate.

Contact Information

More Documentation

Major updates:
-- BrianTierney - 14 Mar 2007

Topic revision: r13 - 16 Dec 2008 - 16:16:03 - KyleGross

Hello, TWikiGuest!
Register

 
TWIKI.NET

TWiki | Report Bugs | Privacy Policy

This site is powered by the TWiki collaboration platformCopyright by the contributing authors. All material on this collaboration platform is the property of the contributing authors..