Overview

Lognag is a simple Perl script that provides simple log event notification via Nagios alerts. It works by keeping an open tail on a set of log files and comparing each new line against a set of search patterns that are defined via config files. When a message matches a pattern, additional information from the config file is used to build a Nagios command sent to the Nagios command file via SSH.

History

Regardless of how embarrassing the truth may be, Lognag is the product of about three years of struggling to create a monitoring/notification system that would work in the environment at SUNY Potsdam. We already had an enterprise notification/monitoring solution for our systems (Nagios) and were ultimately looking for a solution that would integrate with Nagios “seamlessly”.
My first shot at a solution was an over-ambitious project entitled Aggy that aimed too high and too broad, most likely because it didn’t really have an aim. The next attempt to solve this problem was with Splunk which we ran for a short while under the free license and used with moderate success. The biggest detractor of Splunk was our perception of their shift in licensing terms in the 4.x series which made it seem like they were effectively neutering the free version. We do not generally like to be held to the whim of others in our solutions here so we again decided to re-evaluate.
Lognag came to be out of the experiences we had with the previous two “attempts” to solve our needs. I had a clearer understanding of what we needed and the seamless integrate we desired with Nagios. I wrote Lognag to provide a compact solution to integrate basic log monitoring with an existing enterprise notification system. As such, Lognag has a minimal set of features. I’m trying to keep things simple, and expand functionality as needed.

TODO:

  • Finish work on ‘exclude’ config field which will allow pattern exclusion from otherwise matching messages.

Example config:

active:yes
name:MySQL Backup Error
search:mysqlbackup ERROR
severity: critical
test_message:Dec 10 04:05:01 sol mysqlbackup.pl[28605]: dumping to /var/mysqlbkup/sol-20091210040501.sql (expect hidden password)
test_message:Dec 10 04:05:04 sol mysqlbackup.pl[28605]: reaped /var/mysqlbkup/sol-20091203040501.sql
test_message:Dec 10 04:05:04 sol mysqlbackup.pl[28605]: finished mysqlbackup in 3s

  • active – Sets the config file as being one that will be considered active and loaded into memory as a search pattern to look for.
  • name – A logical identifier with which to identify this particular search pattern.
  • search – The actual search pattern to be used. By default, lognag will logically AND multiple terms together (i.e. in the example it will match a message that contains the string ‘mysqlbackup’ and the string ‘ERROR’). If you precede the search pattern with [R], then lognag will assume the rest is a regular expression.
  • severity – This is used when sending status updates nagios. If the search pattern matches, it will send an event to Nagios with the designated severity, the default is warning.
  • test_message – A message to be used in conjunction with invoking lognag with the –checkconfig switch. This will run the search pattern against the specified test messages (can be more than one per config file) and present a report of whether or not the test message matched against the search pattern.