When
syslog-ng 3.5.1 was released, the existence of the
syslog-ng Incubator project was already announced, but I did not go into detail as to why it exists, what is in there, how to build it, or how to use it. The documentation that exists on it is almost non-existent, and what does exist, is usually in the form of a commit message and some example files. But there's nothing on how the things in the Incubator can be used, and what problems they solve.With this post, I will try to ease the situation a little, and provide some insight on how I use some of the things in there, like the
Riemann destination.About the IncubatorThe idea for the
Incubator existed for a long time, the first incarnation of it was called
syslog-ng module collection, and had a different scope: it was a playing field not only for new modules, but updates to existing ones too. The idea was to try out new, possibly experimental features separately from syslog-ng. That didn't work well: it was a nightmare to maintain, and even harder to use. So it never got anywhere, as the developments made there were quickly merged into syslog-ng itself anyway.Then, two years later, I had a need for a new destination, but there was no open development branch to base it against, and forking and maintaining a whole syslog-ng branch for the purpose of a single module sounded like an overkill. So the Incubator was born, from the ashes of the failed module collection.The purpose of the Incubator is to be a place of experimental modules, and an easier way to enter the world syslog-ng development. It's something with far less requirements than syslog-ng (I do not care all that much about portability when it comes to the Incubator, especially not to horrible abominations like HP-UX or AIX), less rules, and more freedom. It's meant to be more developer friendly than syslog-ng proper.It also serves as an example of developing modules completely external to syslog-ng, and parts of it will be used in future posts of mine, to demonstrate the development of new modules.Compiling & InstallationThe first thing we need to do, is to compile whatever we need from the
Incubator, unless we happen to have pre-built binaries available (which are going to appear in your usual third-party syslog-ng repositories over the next few weeks). For this, at a minimum, one will need
syslog-ng 3.5+ and
ivykis installed, along with autotools (autoconf, automake and libtool), pkg-config, bison, and depending on which modules one needs,
riemann-c-client and
libmongo-client too. Most of these are available packaged in better distributions, but only in recent versions, so one may need to compile those too. They follow a very similar scheme, though.If everything is installed at the standard locations, getting the Incubator to compile is as simple as this:
$ git clone https://github.com/balabit/syslog-ng-incubator.git
$ cd syslog-ng-incubator
$ autoreconf -i
$ ./configure && make && sudo make install
If anything happens to be installed in a non-standard location, one will need to adjust
PKG_CONFIG_PATH to help the configure script locate the needed libraries.Once installed (the configure script will figure out where syslog-ng modules are, and modules will be put there), syslog-ng will automatically recognise the new modules. One can make sure that this is the case by running the following command after installation:
$ syslog-ng --module-registry
The PluginsNow that we're over the hard part of compiling and installing the Incubator, lets see what is inside! I will start with the easier things, and move on to the more complicated features as we progress. That is, we'll start with some
template functions, have a glance at the
trigger source, then explore the
rss destination, and finish off with the
riemann destination.
Template functionsThe Incubator gives us three new template functions, some less useful than the others, and one that's a huge, ugly hack for a problem that I ended up solving in a very different way - without the hack.These functions are the
$(or) template function, which takes any number of arguments, and returns the first one that is not empty. The main use case here is that if you have, say, similarly named fields, but some messages have one, the others another, and you want to normalize it,
$(or) is one way to do it:
$(or $ HOST $ HOSTNAME $ HOST_NAME "<unknown>")
Another function is
$(//), which does the same thing as the built-in
$(/) template function: divide its arguments. Except this one works with floating-point numbers only, while the built-in one is for integers exclusively. Using it is simple, too:
$(// $ some_number 3.4)
The last template function provided by the Incubator is
$(state), which can be used to maintain global state, that does not depend on log messages. You can set values in here, like counters, from within a template function. It is possible to count the total amount downloaded data when processing a HTTP server log, for example. But it's slow, and there are better ways to do the same thing, syslog-ng really isn't the best tool for this kind of job. If anyone happens to find a use-case for it, please let me know. As for using it, it has two modes: set (with two arguments) and get (with one):
$(state some-variable $ VALUE )
$(state some-variable)
Trigger sourceThe trigger source has many in common with the built-in mark feature: at given intervals, it sends a message. This is mostly a debugging aid, when you want to generate messages without an external tool. It only has two options:
trigger-freq() and
trigger-message(), which default to
10 and
"Trigger source is trigger happy.", respectively. It also accepts a number of common source options such as
program-override(),
host-override() and
tags().To use it, one just needs to set it up like any other source, and bind it to a destination with a log statement:
source s_trigger
trigger(
program-override("trigger")
tags("trigger-happy")
trigger-freq(5)
trigger-message("Beep.")
);
;
Without a
program-override() option, messages will be attributed to syslog-ng, which is likely not what you want, even while debugging. Internal messages are usually routed somewhere else.
RSS destinationThe RSS destination is an interesting beast. It offers an Atom feed of the last hundred messages routed to the destination. I could very well imagine this being useful in a situation where one already has monitoring set up to listen on various RSS sources - this would be just another. It also works well with most RSS feed readers. The length of the feed is not configurable at this time, and the number of options is limited to
port(),
title(),
entry-title() and
entry-description().The first one specifies which port the destination should listen on (it serves one client at a time!);
title() can be used to set the title of the feed itself, while
entry-title() and
entry-description() can be used with templates to fill in the per-message Atom entries.Once we have a suitable path we want to route to the RSS destination (such as critical error messages only), we can set it up like this:
destination d_rss
rss(
port(8192)
feed-title("Critical errors in the system")
entry-title("Error from $ PROGRAM @ $ HOST_FROM at $ ISODATE ")
entry-description("$ MESSAGE ")
);
;
Riemann destinationBeing the original motivator for the Incubator, I left this last. This module is the interface between your logs and the
Riemann monitoring system. With this, you can take all the legacy applications that are hard to monitor, but provide log files, use syslog-ng's extraordinary log processing power, and send clear and concise events over to Riemann.One can use it to monitor logins, downloads, uploads, exceptions - pretty much anything. Just extract some metric, or state, send it over to Riemann, and that will do the heavy lifting. What exactly can be done, will be worth a separate blog post, so for now, I will give just a very tiny example:
destination d_riemann
riemann(
ttl("120")
description("syslog-ng internal errors")
metric(int("$ SEQNUM "))
);
;
Hook it up to a path that collects syslog-ng internal messages, keeps only error messages, and routes it toward this destination:
log
source internal(); ;
filter level(err..emerg); ;
destination(d_riemann);
;
The destination itself has the following options:
server(): The server to connect to, defaults to localhost.port(): The port the Riemann server is listening on, defaults to 5555.type(): The type of connection: UDP or TCP. Defaults to TCP.host(): The host field of the Riemann event, defaults to $ HOST .service(): The service field of the Riemann event, defaults to $ PROGRAM state(): The state field of the Riemann event, without a default.description(): The description of the event, with no default.ttl(): The time-to-live of the event, no default.metric(): The metric to send to Riemann. This needs to be either an integer or a floating point number. Using type-hinting is advised here. Without one, the destination will try to parse the value of this option as a float.tags(): As the name implies, this adds tags to the Riemann event. By default, all the tags that are set on the message, will be forwarded.attributes(): With this option, one can set custom attributes on the Riemann event. The syntax is the same as for value-pairs(), with a few enhancements. What the difference is, is left as an exercise to the reader: the example config that comes with the Incubator has a hint.
Closing thoughtsThe Incubator contains one more tool, one which can be used to visualise logs in strange ways. But that's not strictly related to syslog-ng, isn't a module, either, so I will not describe it here right now. The above was quite a lot already, I believe.As a closing thought, I would just like to say that while the Incubator is home for experimental modules, some of them are used in production. Don't be afraid to use them, especially when packages start to arrive to your favourite GNU/Linux distribution!
So finally I did it, I managed to ride the 2-wheeler in Bangalore
roads!. Officially I've got a license to drive 2 and 4 wheelers back
in 2011 but I was never confident enough to ride the vehicle on road
and in traffic. Partially this is because I don't own a vehicle and
I've never practiced driving, and partially because of road and
traffic conditions in India I was bit afraid to do so.
So today morning my roommate and me planned to go to temple in the
early morning. We got up at 530 in the morning and we were ready to
drive to Mahalaxmi Layout at 615. Since during this time traffic is
very less I asked my roommate shall I drive and he agreed!.
So I drove the bike from Malleshwaram to Mahalaxmi Layout and it was fun. I
did had hick up in uphill driving as my mind was not co-ordinating
well with breaks and gear. I did have hickups in first gear too but I
guess this will be alright once I get hold of things.
I'm still not having enough confidence to drive during the peak hours
because people are not at all following traffic rules. They just
change the lane without any hints or just come to middle of road from
side roads without looking at other side to see if there is a vehicle
is coming from other side of the road (yes they don't have courtesy to
wait). And finally they can shout at persons who is actually following
rules. In general very less people have traffic discipline in
India. But I will adapt to this once I start driving so I'm now all
set to get my 2 wheeler soon and ride on!
For the past few years I've hosted all my websites in a "special" way:
Recently I've been thinking about getting a new laptop. I have this rule that a laptop should last me at least 3 years (ideally more) and my old laptop was bought in September 2010. So for the past few months I've been trying to work out if there's something suitable on the market that is a good replacement (last time I didn't manage to find something that ticked all the boxes, but did pretty well for the price I paid).
To start with I decided to track my laptops over time - largely because one of my concerns was about the size of a replacement, because I have a significant leaning towards subnotebooks. In the end the reason I decided to upgrade was for some extra CPU grunt; my old machine had a tendency to get pretty hot under any sort of load.
Zoe woke up pretty early today, at around 5:30am. I also officially became a
Xoogler at some point overnight. I used this transition as the opportunity
to move from my Nexus 4 to my Nexus 5, so I could have a clean start on a
phone that had never had work credentials on it.
I started that process before breakfast, and mostly had a usable phone by
the time we biked to Kindergarten. Zoe was a bit more like her usual clingy
self at drop off today, so I spent a bit longer getting her settled.
I got home and pretty much spent the rest of the day sorting out my phone. I
can say many good things about 
I eventually answered to the 
This is my monthly summary of my free software related activities. If you re among the people who 
In the discussions about the setup of the 
Handing over packages to new maintainers is hard
Init system discussion. The technical committee has the hard task of picking the default init system that will replace the traditional System V init (see 
Good morning all my hungover friends.
New Hy release - sounds like the perfect thing to do while you re waiting for your headaches to go away.
Here s a short-list of the changes (from NEWS) - enjoy!
Recently, I gave the very elegant board game 
I've