• Facebook Like
• Google Plus One
• Linkedin Share Button
• Tweet Widget

For the past few days I've been making minor changes to my static-site generator, templer (source on github). The recent changes have all had one aim, which was to allow me to rebuild my main site. Now I've finished http://www.steve.org.uk is up to date, and the source code to the website is stored in a mercurial repository. No real functional changes have been made, but I've rationolized several ad-hoc bits of the site, marked areas are depreciated/unsupported where appropriate, and removed a few things that were completely broken. I almost removed the software for Microsoft Windows, but didn't. By a strange coincidence I was recognized as the author of a windows utility back in 2004 - almost ten years ago now - on Hacker News. Guess I made the right choice. I'm going to spend a while working on my slaughter documentation in the next week or two, although "the definitive guide" is a great starting point. "Yes, this is dog" - Landscape in The Mist (1984).

Also this year I registered a booth in the name of the Debian Project at Chemnitzer Linux-Tage. Unfortunalety this is the second time in the row, that we seem to be short of manpower while preparing the booth. Actually just one person was able to commit his available time. A general offer was announced by 3 people (including me). This makes me a little sad, cause CLT is a very community driven event which is really nice organized. I always liked to chat and discuss with very interesting people from other projects and visitors. As the CLT runs a 6 track lecture program with commonly very good lectures, it is not possible to get the booth running into good shape with such a small amount of volunteers. If you feel you are interested into visiting the Chemnitzer Linux-Tage and want to make our booth a success like last years for the Debian Project, please have a look into my announcment and our coordination wiki. So please ... let come more brave people into our band wagon! Don't wait to long! Registration period for (possibly sponsored) Booth tickets (read here) ends on 4th March, many thanks!

Following up on using Debian on the WeTab with GNOME Shell I had a look at automatically adjusting the screen orientation when rotating the device. The accelerometer is handled in the asus_laptop module since kernel 3.2, so this shows up in /proc/bus/input/devices:

I: Bus=0019 Vendor=0000 Product=0000 Version=0000
N: Name="Pegatron Lucid Tablet Accelerometer"
P: Phys=pega_accel/input0
S: Sysfs=/devices/platform/asus_laptop/input/input4
U: Uniq=
H: Handlers=event4 js0 
B: PROP=0
B: EV=9
B: ABS=7

Whenever the screen orientation changes the driver emits a udev event. The helper in /lib/udev/accelerometer processes it thanks to /lib/udev/rules.d/61-accelerometer.rules, determines the orientation and adds the ID_INPUT_ACCELEROMETER_ORIENTATION property to the event which can be checked with udevadm monitor --property:

UDEV  [5778.534171] change   /devices/platform/asus_laptop/input/input4 (input)
ABS=7
ACTION=change
DEVPATH=/devices/platform/asus_laptop/input/input4
EV=9
ID_FOR_SEAT=input-platform-asus_laptop
ID_INPUT=1
ID_INPUT_ACCELEROMETER=1
ID_INPUT_ACCELEROMETER_ORIENTATION=right-up
ID_PATH=platform-asus_laptop
ID_PATH_TAG=platform-asus_laptop
MODALIAS=input:b0019v0000p0000e0000-e0,3,kra0,1,2,mlsfw
NAME="Pegatron Lucid Tablet Accelerometer"
PHYS="pega_accel/input0"
PRODUCT=19/0/0/0
PROP=0
SEQNUM=1550
SUBSYSTEM=input
TAGS=:seat:
UDEV_LOG=3
USEC_INITIALIZED=6154288

This is captured by gnome-settings-daemon (>=3.2) that adjusts the screen rotation using xrandr2 accordingly. This can be checked by running gnome-settings-daemon with --debug. The old way of using xinput events is no longer supported. Unfortunately Debian's udev has a minor bug that misdetects the input device so this patch is needed to have the screen orientation changed automatically on the WeTab. This blog is flattr enabled.

Finland. Helsinki, Lahti, streets Arriving at Helsinki airport, we filed a claim with Lufthansa as a hard shell suitcase had a splintered corner. We were surprised that so many Finns arrived from Munich with skis, more on that later. We picked up our car and started on our way towards Koli; driving with a top speed of 100 km/h and often being limited to 80 km/h or even 60 km/h is... unusual... Finnish police/authorities seem to be obsessed with enforcing those speed limits as there are a lot of speed cameras along the way. Finnish people seem to be similarly obsessed with slot machines; there is an incredible amount of them at gas stations and a constant stream of people playing them. From an outsider's perspective, it's weird that a country as strict about one form of addiction, alcohol, and working against it vigorously, by means of taxes, would allow another form of addiction, gambling, run as freely and this allow so many slot machines. Speaking of taxes on alcohol: a single 0.33 l bottle of beer is more expensive in a Finnish supermarket than 0.5 l of beer in a German restaurant. Which also explains why supermarkets tend to have a rather large section with relatively cheap alcohol free beer. Anyway, coming back to streets: Highway intersections don't have continuous on/off ramps from which you change from one highway to another; you drive off of the highway, stop at a traffic light, and then continue onto the other highway. Weird system, but given the amount of traffic we witnessed, it's probably Good Enough (tm). Stopping for a short time in Lohti simply because it's apparently famous for winter sports competitions, we arrived at Future Freetime in Koli national park after about five to six gruelling hours of net driving through somewhat bad weather and behind slow drivers. Koli Hiking up to Ukko-Koli and its sister peaks proved to be rather exhausting as we kept on breaking through the snow cover to our knees and sometimes even our hips. Once we were up there, we realized that even though you couldn't see it in between the trees, there was fog all over the plains so we couldn't see anything. Still, it was a nice hike even if somewhat short. Note to self: Even when a trail is marked locally, if OpenStreetMap does not know about it... don't walk along it. Especially not when the going's rough already. And if there's a sign suggesting you wear snow shoes... wear snow shoes. Returning to Koli Hotel and the museum next to it, we walked over to the ski slope. The highest peak within Koli,Ukko-Koli, is 347 meters high, the local ski slope starts a good way below that. This would explain why a lot of Finns came back from Munich with their skis... Afterwards, we rented a snow mobile, without guide or supervision, and drove from Loma-Koli over lake Pielien towards Purnuniemi and in a large circle down towards lake Ryyn skyl where we turned around and went back the same way. If we thought Finnish streets don't have a lot of signs we soon realized that snow mobile tracks have even less. There are at most two or three signs pointing you in the right direction, but on the plus side, there are no posted speed limits for snow mobiles, either. In somewhat related news, snow mobiles can go at least 95 km/h. At that point, the scratched and dirty visor of your rental helmet will keep slamming down, forcing you to take one hand off the handle and thus stop accelerating to maintain stability. To round off the day, we heated up the sauna built into our little wooden hut. Running outside three times to rub myself off with snow from head to toes, I almost slipped and fell while standing still. When your feet are too hot for the snowy ground, you'll start to melt your own little pools of slippery water/snow mush within seconds. File that one under "I would never have guessed unless I had experienced it myself". Generic The MarkDown source of this blog post is not even 5 kiB in size; even in a worst case scenario, pushing this to my ikiwiki instance via git will eat up less 10 kiB of mobile data. Which is good because I have 78 MiB of international data left on this plan. This is also the reason why there are no links in this blog post: I am writing everything off line and don't want to search for the correct URLs to link to. I really wish EU regulators would start to tackle data roaming now that SMS and voice calls are being forced down into somewhat sane pricing regions by regulations. PS:

-rw-r--r-- 1 richih richih 4.6K Feb 11 22:55 11-Finland-I.mdwn
[...]
Writing objects: 100% (7/7), 2.79 KiB, done.

A few weeks ago, a new version of lldpd, a 802.1AB (aka LLDP) implementation for various Unices, has been released. LLDP is an industry standard protocol designed to supplant proprietary Link-Layer protocols such as EDP or CDP. The goal of LLDP is to provide an inter-vendor compatible mechanism to deliver Link-Layer notifications to adjacent network devices. In short, LLDP allows you to know exactly on which port is a server (and reciprocally). To illustrate its use, I have made a xkcd-like strip: If you would like more information about lldpd, please have a look at its new dedicated website. This blog post is an insight of various technical changes that have affected lldpd since its latest major release one year ago. Lots of C stuff ahead!

Version & changelog UPDATED: Guillem Jover told me how he met the same goals for libbsd :

1. Save the version from git into .dist-version and use this file if it exists. This allows one to rebuild ./configure from the published tarball without losing the version. This also handles Thorsten Glaser s critic.
2. Include CHANGELOG in DISTCLEANFILES variable.

Since this is a better solution, I have adopted the appropriate line of codes from libbsd. The two following sections are partly technically outdated.

Automated version In configure.ac, I was previously using a static version number that I had to increase when releasing:

AC_INIT([lldpd], [0.5.7], [bernat@luffy.cx])

Since the information is present in the git tree, this seems a bit redundant (and easy to forget). Taking the version from the git tree is easy:

AC_INIT([lldpd],
        [m4_esyscmd_s([git describe --tags --always --match [0-9]* 2> /dev/null   date +%F])],
        [bernat@luffy.cx])

If the head of the git tree is tagged, you get the exact tag (0.7.1 for example). If it is not, you get the nearest one, the number of commits since it and part of the current hash (0.7.1-29-g2909519 for example). The drawback of this approach is that if you rebuild configure from the released tarball, you don t have the git tree and the version will be a date. Just don t do that.

Automated changelog Generating the changelog from git is a common practice. I had some difficulties to make it right. Here is my attempt (I am using automake):

dist_doc_DATA = README.md NEWS ChangeLog
.PHONY: $(distdir)/ChangeLog
dist-hook: $(distdir)/ChangeLog
$(distdir)/ChangeLog:
        $(AM_V_GEN)if test -d $(top_srcdir)/.git; then \
          prev=$$(git describe --tags --always --match [0-9]* 2> /dev/null) ; \
          for tag in $$(git tag   grep -E '^[0-9]+(\.[0-9]+) 1, $$'   sort -rn); do \
            if [ x"$$prev" = x ]; then prev=$$tag ; fi ; \
            if [ x"$$prev" = x"$$tag" ]; then continue; fi ; \
            echo "$$prev [$$(git log $$prev -1 --pretty=format:'%ai')]:" ; \
            echo "" ; \
            git log --pretty=' - [%h] %s (%an)' $$tag..$$prev ; \
            echo "" ; \
            prev=$$tag ; \
          done > $@ ; \
        else \
          touch $@ ; \
        fi
ChangeLog:
        touch $@

Changelog entries are grouped by version. Since it is a bit verbose, I still maintain a NEWS file with important changes.

Core

C99 I have recently read 21st Century C which has some good bits and also handles the ecosystem around C. I have definitively adopted designated initializers in my coding style. Being a GCC extension since a long time, this is not a major compatibility problem. Without designated initializers:

struct netlink_req req;
struct iovec iov;
struct sockaddr_nl peer;
struct msghdr rtnl_msg;
memset(&req, 0, sizeof(req));
memset(&iov, 0, sizeof(iov));
memset(&peer, 0, sizeof(peer));
memset(&rtnl_msg, 0, sizeof(rtnl_msg));
req.hdr.nlmsg_len = NLMSG_LENGTH(sizeof(struct rtgenmsg));
req.hdr.nlmsg_type = RTM_GETLINK;
req.hdr.nlmsg_flags = NLM_F_REQUEST   NLM_F_DUMP;
req.hdr.nlmsg_seq = 1;
req.hdr.nlmsg_pid = getpid();
req.gen.rtgen_family = AF_PACKET;
iov.iov_base = &req;
iov.iov_len = req.hdr.nlmsg_len;
peer.nl_family = AF_NETLINK;
rtnl_msg.msg_iov = &iov;
rtnl_msg.msg_iovlen = 1;
rtnl_msg.msg_name = &peer;
rtnl_msg.msg_namelen = sizeof(struct sockaddr_nl);

With designated initializers:

struct netlink_req req =  
    .hdr =  
        .nlmsg_len = NLMSG_LENGTH(sizeof(struct rtgenmsg)),
        .nlmsg_type = RTM_GETLINK,
        .nlmsg_flags = NLM_F_REQUEST   NLM_F_DUMP,
        .nlmsg_seq = 1,
        .nlmsg_pid = getpid()  ,
    .gen =   .rtgen_family = AF_PACKET  
 ;
struct iovec iov =  
    .iov_base = &req,
    .iov_len = req.hdr.nlmsg_len
 ;
struct sockaddr_nl peer =   .nl_family = AF_NETLINK  ;
struct msghdr rtnl_msg =  
    .msg_iov = &iov,
    .msg_iovlen = 1,
    .msg_name = &peer,
    .msg_namelen = sizeof(struct sockaddr_nl)
 ;

Logging Logging in lldpd was not extensive. Usually, when receiving a bug report, I asked the reporter to add some additional printf() calls to determine where the problem was. This was clearly suboptimal. Therefore, I have added many log_debug() calls with the ability to filter out some of them. For example, to debug interface discovery, one can run lldpd with lldpd -ddd -D interface. Moreover, I have added colors when logging to a terminal. This may seem pointless but it is now far easier to spot warning messages from debug ones.

libevent In lldpd 0.5.7, I was using my own select()-based event loop. It worked but I didn t want to grow a full-featured event loop inside lldpd. Therefore, I switched to libevent. The minimal required version of libevent is 2.0.5. A convenient way to check the changes in API is to use Upstream Tracker, a website tracking API and ABI changes for various libraries. This version of libevent is not available in many stable distributions. For example, Debian Squeeze or Ubuntu Lucid only have 1.4.13. I am also trying to keep compatibility with very old distributions, like RHEL 2, which does not have a packaged libevent at all. For some users, it may be a burden to compile additional libraries. Therefore, I have included libevent source code in lldpd source tree (as a git submodule) and I am only using it if no suitable system libevent is available. Have a look at m4/libevent.m4 and src/daemon/Makefile.am to see how this is done.

Client

Serialization lldpctl is a client querying lldpd to display discovered neighbors. The communication is done through an Unix socket. Each structure to be serialized over this socket should be described with a string. For example:

#define STRUCT_LLDPD_DOT3_MACPHY "(bbww)"
struct lldpd_dot3_macphy  
        u_int8_t                 autoneg_support;
        u_int8_t                 autoneg_enabled;
        u_int16_t                autoneg_advertised;
        u_int16_t                mau_type;
 ;

I did not want to use stuff like Protocol Buffers because I didn t want to copy the existing structures to other structures before serialization (and the other way after deserialization). However, the serializer in lldpd did not allow to handle reference to other structures, lists or circular references. I have written another one which works by annotating a structure with some macros:

struct lldpd_chassis  
    TAILQ_ENTRY(lldpd_chassis) c_entries;
    u_int16_t        c_index;
    u_int8_t         c_protocol;
    u_int8_t         c_id_subtype;
    char            *c_id;
    int              c_id_len;
    char            *c_name;
    char            *c_descr;
    u_int16_t        c_cap_available;
    u_int16_t        c_cap_enabled;
    u_int16_t        c_ttl;
    TAILQ_HEAD(, lldpd_mgmt) c_mgmt;
 ;
MARSHAL_BEGIN(lldpd_chassis)
MARSHAL_TQE  (lldpd_chassis, c_entries)
MARSHAL_FSTR (lldpd_chassis, c_id, c_id_len)
MARSHAL_STR  (lldpd_chassis, c_name)
MARSHAL_STR  (lldpd_chassis, c_descr)
MARSHAL_SUBTQ(lldpd_chassis, lldpd_mgmt, c_mgmt)
MARSHAL_END;

Only pointers need to be annotated. The remaining of the structure can be serialized with just memcpy()¹. I think there is still room for improvement. It should be possible to add annotations inside the structure and avoid some duplication. Or maybe, using a C parser? Or using the AST output from LLVM?

Library In lldpd 0.5.7, there are two possible entry points to interact with the daemon:

1. Through SNMP support. Only information available in LLDP-MIB are exported. Therefore, implementation-specific values are not available. Moreover, SNMP support is currently read-only.
2. Through lldpctl. Thanks to a contribution from Andreas Hofmeister, the output can be requested to be formatted as an XML document.

Integration of lldpd into a network stack was therefore limited to one of those two channels. As an exemple, you can have a look at how Vyatta made the integration using the second solution. To provide a more robust solution, I have added a shared library, liblldpctl, with a stable and well-defined API. lldpctl is now using it. I have followed those directions²:

• Consistent naming (all exported symbols are prefixed by lldpctl_). No pollution of the global namespace.
• Consistent return codes (on errors, all functions returning pointers are returning NULL, all functions returning integers are returning -1).
• Reentrant and thread-safe. No global variables.
• One well-documented include file.
• Reduce the use of boilerplate code. Don t segfault on NULL, accept integer input as string, provide easy iterators,
• Asynchronous API for input/output. The library delegates reading and writing by calling user-provided functions. Those functions can yield their effects. In this case, the user has to callback the library when data is available for reading or writing. It is therefore possible to integrate the library with any existing event-loop. A thin synchronous layer is provided on top of this API.
• Opaque types with accessor functions.

Accessing bits of information is done through atoms which are opaque containers of type lldpctl_atom_t. From an atom, you can extract some properties as integers, strings, buffers or other atoms. The list of ports is an atom. A port in this list is also an atom. The list of VLAN present on this port is an atom, as well as each VLAN in this list. The VLAN name is a NULL-terminated string living in the scope of an atom. Accessing a property is done by a handful of functions, like lldpctl_atom_get_str(), using a specific key. For example, here is how to display the list of VLAN assuming you have one port as an atom:

vlans = lldpctl_atom_get(port, lldpctl_k_port_vlans);
lldpctl_atom_foreach(vlans, vlan)  
    vid = lldpctl_atom_get_int(vlan,
                               lldpctl_k_vlan_id));
    name = lldpctl_atom_get_str(vlan,
                                lldpctl_k_vlan_name));
    if (vid && name)
        printf("VLAN %d: %s\n", vid, name);
 
lldpctl_atom_dec_ref(vlans);

Internally, an atom is typed and reference counted. The size of the API is greatly limited thanks to this concept. There are currently more than one hundred pieces of information that can be retrieved from lldpd. Ultimately, the library will also enable the full configuration of lldpd. Currently, many aspects can only be configured through command-line flags. The use of the library does not replace lldpctl which will still be available and be the primary client of the library.

CLI Having a configuration file was requested since a long time. I didn t want to include a parser in lldpd: I am trying to keep it small. It was already possible to configure lldpd through lldpctl. Locations, network policies and power policies were the three items that could be configured this way. So, the next step was to enable lldpctl to read a configuration file, parse it and send the result to lldpd. As a bonus, why not provide a full CLI accepting the same statements with inline help and completion?

Parsing & completion Because of completion, it is difficult to use a YACC generated parser. Instead, I define a tree where each node accepts a word. A node is defined with this function:

struct cmd_node *commands_new(
    struct cmd_node *,
    const char *,
    const char *,
    int(*validate)(struct cmd_env*, void *),
    int(*execute)(struct lldpctl_conn_t*, struct writer*,
        struct cmd_env*, void *),
    void *);

A node is defined by:

• its parent,
• an optional accepted static token,
• an help string,
• an optional validation function and
• an optional function to execute if the current token is accepted.

When walking the tree, we maintain an environment which is both a key-value store and a stack of positions in the tree. The validation function can check the environment to see if we are in the right context (we want to accept the keyword foo only once, for example). The execution function can add the current token as a value in the environment but it can also pop the current position in the tree to resume walk from a previous node. As an example, see how nodes for configuration of a coordinate-based location are registered:

/* Our root node */
struct cmd_node *configure_medloc_coord = commands_new(
    configure_medlocation,
    "coordinate", "MED location coordinate configuration",
    NULL, NULL, NULL);
/* The exit node.
   The validate function will check if we have both
   latitude and longitude. */
commands_new(configure_medloc_coord,
    NEWLINE, "Configure MED location coordinates",
    cmd_check_env, cmd_medlocation_coordinate,
    "latitude,longitude");
/* Store latitude. Once stored, we pop two positions
   to go back to the "root" node. The user can only
   enter latitude once. */
commands_new(
    commands_new(
        configure_medloc_coord,
        "latitude", "Specify latitude",
        cmd_check_no_env, NULL, "latitude"),
    NULL, "Latitude as xx.yyyyN or xx.yyyyS",
    NULL, cmd_store_env_value_and_pop2, "latitude");
/* Same thing for longitude */
commands_new(
    commands_new(
        configure_medloc_coord,
        "longitude", "Specify longitude",
        cmd_check_no_env, NULL, "longitude"),
    NULL, "Longitude as xx.yyyyE or xx.yyyyW",
    NULL, cmd_store_env_value_and_pop2, "longitude");

The definition of all commands is still a bit verbose but the system is simple enough yet powerful enough to cover all needed cases.

Readline When faced with a CLI, we usually expect some perks like completion, history handling and help. The most used library to provide such features is the GNU Readline Library. Because this is a GPL library, I have first searched an alternative. There are several of them:

• NetBSD Editline library (libedit).
• Autotool port of the NetBSD Editline library (also libedit).
• Debian version of the Editline library (libeditline).
• linenoise, a small and minimal readline library.
• Many others.

From an API point of view, the first three libraries support the GNU Readline API. They also have a common native API. Moreover, this native API also handles tokenization. Therefore, I have developed the first version of the CLI with this API³. Unfortunately, I noticed later this library is not very common in the Linux world and is not available in RHEL. Since I have used the native API, it was not possible to fallback to the GNU Readline library. So, let s switch! Thanks to the appropriate macro from the Autoconf Archive (with small modifications), the compilation and linking differences between the libraries are taken care of. Because GNU Readline library does not come with a tokenizer, I had to write one myself. The API is also badly documented and it is difficult to know which symbol is available in which version. I have limited myself to:

• readline(), addhistory(),
• rl_insert_text(),
• rl_forced_update_display(),
• rl_bind_key()
• rl_line_buffer and rl_point.

Unfortunately, the various libedit libraries have a noop for rl_bind_key(). Therefore, completion and online help is not available with them. I have noticed that most BSD come with GNU Readline library preinstalled, so it could be considered as a system library. Nonetheless, linking with libedit to avoid licensing issues is possible and help can be obtained by prefixing the command with help.

OS specific support

Netlink on Linux Previously, the list of interfaces was retrieved through getifaddrs(). lldpd is now using directly Netlink on Linux. This is not a big change since the GNU C Library already uses it to implement getifaddrs() and additional information, like VLAN, are still retrieved through ioctl() or sysfs. However, lldpd now gets notified when a change happens and update all interfaces in the next second. Like many other projects, I have written my own Netlink implementation instead of using libnl, a nice collection of libraries providing everything you need to query the kernel through Netlink, including some advanced bits. Why?

1. The latest version of libnl is still young and its availability in major distributions is scarce. It is not available in Debian Squeeze but will be available in Debian Wheezy. Like libevent, I could circumvent this problem by shipping the library with lldpd and use it when there is not system alternative. But
2. libnl is licensed under LGPL 2.1. This makes static linking difficult because the license is quite shaddy about static linking being derivative work or not. It is believed that it is authorized under the same provisions as in LGPL 3 which handles the case explicitely. This has been a problem with many projects. For example, OGRE has added an exception for static linking in version 1.6 and switched to MIT license in version 1.7.

I had a short discussion with Thomas Graf about this issue and he seems willing to add a similar exception. This may take some time, but once this is done, I will happily switch to libnl and retrieve more stuff from Netlink.

BSD support Until version 0.7, lldpd was Linux-only. The rewrite to use Netlink was the occasion to abstract interfaces and to port to other OS. The first port was for Debian GNU/kFreeBSD, then for FreeBSD, OpenBSD and NetBSD. They all share the same source code:

• getifaddrs() to get the list of interfaces,
• bpf(4) to attach to an interface to receive and send packets,
• PF_ROUTE socket to be notified when a change happens.

Each BSD has its own ioctl() to retrieve VLAN, bridging and bonding bits but they are quite similar. The code was usually adapted from ifconfig.c. The BSD ports have the same functionalities than the Linux port, except for NetBSD which lacks support for LLDP-MED inventory since I didn t find a simple way to retrieve DMI related information. They also offer greater security by filtering packets sent. Moreover, OpenBSD allows to lock the filters set on the socket:

/* Install write filter (optional) */
if (ioctl(fd, BIOCSETWF, (caddr_t)&fprog) < 0)  
    rc = errno;
    log_info("privsep", "unable to setup write BPF filter for %s",
        name);
    goto end;
 
/* Lock interface */
if (ioctl(fd, BIOCLOCK, (caddr_t)&enable) < 0)  
    rc = errno;
    log_info("privsep", "unable to lock BPF interface %s",
        name);
    goto end;
 

This is a very nice feature. lldpd is using a privileged process to open the raw socket. The socket is then transmitted to an unprivileged process. Without this feature, the unprivileged process can remove the BPF filters. I have ported the ability to lock a socket filter program to Linux. However, I still have to add a write filter.

OS X support Once FreeBSD was supported, supporting OS X seemed easy. I got sponsored by xcloud.me which provided a virtual Mac server. Making lldpd work with OS X took only two days, including a full hour to guess how to get Apple Xcode without providing a credit card. To help people installing lldpd on OS X, I have also written a lldpd formula for Homebrew which seems to be the most popular package manager for OS X.

Upstart and systemd support Many distributions propose upstart and systemd as a replacement or an alternative for the classic SysV init. Like most daemons, lldpd detaches itself from the terminal and run in the background, by forking twice, once it is ready (for lldpd, this just means we have setup the control socket). While both upstart and systemd can accommodate daemons that behave like this, it is recommended to not fork. How to advertise readiness in this case? With upstart, lldpd will send itself the SIGSTOP signal. upstart will detect this, resume lldpd with SIGCONT and assume it is ready. The code to support upstart is therefore quite simple. Instead of calling daemon(), do this:

const char *upstartjob = getenv("UPSTART_JOB");
if (!(upstartjob && !strcmp(upstartjob, "lldpd")))
    return 0;
log_debug("main", "running with upstart, don't fork but stop");
raise(SIGSTOP);

The job configuration file looks like this:

# lldpd - LLDP daemon
description "LLDP daemon"
start on net-device-up IFACE=lo
stop on runlevel [06]
expect stop
respawn
script
  . /etc/default/lldpd
  exec lldpd $DAEMON_ARGS
end script

systemd provides a socket to achieve the same goal. An application is expected to write READY=1 to the socket when it is ready. With the provided library, this is just a matter of calling sd_notify("READY=1\n"). Since sd_notify() has less than 30 lines of code, I have rewritten it to avoid an external dependency. The appropriate unit file is:

[Unit]
Description=LLDP daemon
Documentation=man:lldpd(8)
[Service]
Type=notify
NotifyAccess=main
EnvironmentFile=-/etc/default/lldpd
ExecStart=/usr/sbin/lldpd $DAEMON_ARGS
Restart=on-failure
[Install]
WantedBy=multi-user.target

OS include files Linux-specific include files were a major pain in previous versions of lldpd. The problems range from missing header files (like linux/if_bonding.h) to the use of kernel-only types. Those headers have a difficult history. They were first shipped with the C library but were rarely synced and almost always outdated. They were then extracted from kernel version with almost no change and lagged behind the kernel version used by the released distribution⁴. Today, the problem is acknowledged and is being solved by both the distributions which extract the headers from the packaged kernel and by kernel developers with a separation of kernel-only headers from user-space API headers. However, we still need to handle legacy. A good case is linux/ethtool.h:

• It can just be absent.
• It can use u8, u16 types which are kernel-only types. To work around this issue, type munging can be setup.
• It can miss some definition, like SPEED_10000. In this case, you either define the missing bits and find yourself with a long copy of the original header interleaved with #ifdef or conditionally use each symbol. The latest solution is a burden by itself but it also hinders some functionalities that can be available in the running kernel.

The easy solution to all this mess is to just include the appropriate kernel headers into the source tree of the project. Thanks to Google ripping them for its Bionic C library, we know that copying kernel headers into a program does not create a derivative work.

---

1. Therefore, the use of u_int16_t and u_int8_t types is a left-over of the previous serializer where the size of all members was important.
2. For more comprehensive guidelines, be sure to check Writing a C library.
3. Tokenization is not the only advantage of libedit native API. The API is also cleaner, does not have a global state and has a better documentation. All the implementations are also BSD licensed.
4. For example, in Debian Sarge, the Linux kernel was a 2.6.8 (2004) while the kernel headers were extracted from some pre-2.6 kernel.

Linux is a great platform for dealing with all kinds of different file systems, partition tables etc. But one of the few annoying situations when working in IT forensics are Microsoft Windows dynamic disks, AKA LDM (Logical Disk Manager). Thanks to libldm s ldmtool this is no longer true. A short demonstration from a real-life IT forensics investigation (actual IDs/data randomized for obvious reasons):

# ldmtool
ldm> scan /dev/sdc*
[
  "1bad5bbc-a4b5-42e1-8823-001014b00003"
]
ldm> show diskgroup 1bad5bbc-a4b5-42e1-8823-001014b00003
 
  "name" : "FOOBAR-Dg0",
  "guid" : "1bad5bbc-a4b5-42e1-8823-001014b00003",
  "volumes" : [
    "Volume1"
  ],
  "disks" : [
    "Disk1",
    "Disk2"
  ]
 
ldm> show volume 1bad5bbc-a4b5-42e1-8823-001014b00003 Volume1
 
  "name" : "Volume1",
  "type" : "striped",
  "size" : 3907039232,
  "chunk-size" : 128,
  "hint" : "D:",
  "partitions" : [
    "Disk1-01",
    "Disk2-01"
  ]
 
ldm> show partition 1bad5bbc-a4b5-42e1-8823-001014b00003 Disk1-01
 
  "name" : "Disk1-01",
  "start" : 1985,
  "size" : 1953519616,
  "disk" : "Disk1"
 
ldm> create all
Unable to create volume Volume1 in disk group 1bad5bbc-a4b5-42e1-8823-001014b00003: Disk Disk2 required by striped volume Volume1 is missing
[
]
ldm> scan /dev/sdd*
[
  "1bad5bbc-a4b5-42e1-8823-001014b00003"
]
ldm> create all
[
  "ldm_vol_FOOBAR-Dg0_Volume1"
]
ldm>

The just created device mapper device then can be handled as usual:

# dmsetup ls   grep ldm
ldm_vol_FOOBAR-Dg0_Volume1        (254:4)
# mount /dev/mapper/ldm_vol_FOOBAR-Dg0_Volume1 /mnt/whatever

ldmtool just hit Debian unstable (and I intend to ship the tool with the upcoming version of Grml-Forensic).

So, the price of heating my apartment has gone up significantly since last year and a lot of people have noticed the same trend. As a geek, I want not just any solutions, but geekiest solutions possible to that enter a Smart Home system. The heating in my apartment is separated from all other apartments with a separate heat meter that measures both heating water flow and the temperature difference on the incoming and outgoing pipes, so if I reduce the heating consumption I will immediately see that in lower heating bills. This particular smart home system is very simple and made in Latvia and also relatively cheap, so I decided to give this a try. The start was very simple I filled out a form on their home page and got a call the next day to verify the specifics. They already knew my building and had experience installing there, so they offered me two options: I could have two thermal sensors and a radiator controller on each of the 3 radiators in my apartment or (seeing how my apartment is relatively small and well ventilated thus reducing the efficiency of per-room control) one thermal sensor and one radiator controller installed on a new valve that they would build into the heating pipe just outside my apartment. I chose the second option as that was cheaper, simpler and involved less installation for me. Also they offered a nice discount. A plumber of my building installed the valve at some point during the day, I did not even have to be there or arrange anything special, it was all included in the main bill. Then the next day a friendly guy came by to my office and gave me three very cute boxes. The packaging was very, very Apple-like. The three devices were named mother , spy and puller basically that is a base station, a temperature sensor and a radiator regulator a wireless stepper motor. The installation was straight forward I connected the base station (something very similar to this thin in size and looks) to Ethernet and power, went to the web address specified on the sticker on the base station, completed a registration process there, entered a code from the base of the base station into the website and it just found it. After that I pressed a button on the temperature sensor (a Zippo lighter sized box with a built-in Li-Ion battery rechargeable via microUSB once a year or so) and put that on a shelf. I secured it with some double-sided tape after the cat got to it once and played some hockey while I was away. The temperature sensor immediately showed up in the Web UI and I could assign it to a room. The regulator installation was the most complex bit, but it was trivial as well all I had to do was insert some AA batteries (helpfully included in the box, should last a couple years by their estimates), then go outside to the newly installed inlet valve, put this thing on, screw the thread with my hands and then push and hold a button for 10 seconds. After that the controller came to life, connected with the base station and started spinning the valve, testing its operational range I presume. The range is very good you can see 13% signal in the screenshots that is across the whole of the apartment, trough 3 solid walls and another 5 meters out in the hallway inside another wall. As soon as that was done, the Web UI showed the controller and asked my to assign it to a room. So now I have 1 room defined with 1 temperature sensor and one controller. That is all that is needed. From the charts I have already found out that at current inside/outside temperatures my apartment cools down at around 0.3 degrees C per hour and with current radiator settings it heats up by around 1 degree C per hour. I should turn the radiators up a bit thus increasing the rate at which the system is able to heat things up. Also I should try to re-inspect all possible cold spots to see why the cooldown speed is so high. I expected it to be lower. At least now I have real data to compare things. At this point you can either fire up your web browser or a Android/iOS client on you mobile device to monitor and control the temperature of the apartment. But that was not enough for me I opted for the PRO upgrade which includes the ability to program a weekly cycle of desired temperatures. The programming is limited and thus simple to do there are three temperature modes: day, night and eco. The idea is that you have the night mode every night, then at some time during the day, you have the day mode (to wake up), then you switch to eco mode (when noone is home) and then in the evening you go back to day mode and later to night mode. There are separate sliders for each day of the week and temperature settings for each of the modes for every programmed room. Total investment so far 210 Ls (or around 3 months worth of just heating bills). The experience of other users tells of 30-50% heating bill reduction, so this should pay for itself in 2-3 years. In addition I get a more stable and predictable temperature at home and some more geeky stuff to brag about yay! These guys have their work cut out for them I have no idea how/of they do international orders, I would definately like to see more information in the web site about how my system is working (more temperature history, valve state history, wireless signal and battery history, mobile apps are a bit wonky it feels like the mobile app does not respect the temperature set over the web and tries to reassert itself when opened and it often shows out-of-date temperature data with no indication that it is out-of-date + the mobile apps are missing charts. All in all this is a great start: easy, quite cheap and stylish home heating automation. Good work! P.S. I am pretty sure they use Linux, but it is so packaged, that there is basically no way to confirm or deny that. P.P.S. Also check out this overnight chart, there is definitely some mode switch anticipation going on

In April of 2012 I bought a ColorHug colorimeter. I got a bit discouraged when the first thing I realized was that one of my monitors needed replacing, and put the the colorhug in a drawer until today. With quite a lot of help and encouragment from Pascal de Bruijn, I finally got it going. Pascal has written an informative blog post on color management. That's a good place to look for background. This is more of a "write down the commands so I don't forget" sort of blog post, but it might help somebody else trying to calibrate their monitor using argyll on the command line. I'm not running gnome, so using gnome color manager turns out to be a bit of a hassle. I run Debian Wheezy on this machine, and I'll mention the packages I used, even though I didn't install most of them today.

1. Find the masking tape, and tear off a long enough strip to hold the ColorHug on the monitor. This is probably the real reason I gave up last time; it takes about 45minutes to run the calibration, and I lack the attention span/upper-arm-strength to hold the sensor up for that long. Apparently new ColorHugs are shipping with some elastic.
2. Update the firmware on the colorhug. This is a gui-wizard kindof thing.

    % apt-get install colorhug-client
    % colorhug-flash 
   

3. Set the monitor to factory defaults. On this ASUS PA238QR, that is brightness 100, contrast 80, R=G=B=100. I adjusted the brightness down to about 70; 100 is kindof eye-burning IMHO.
4. Figure out which display is which; I have two monitors.

    % dispwin -\?
   

   Look under "-d n"
5. Do the calibration. This is really verbatim from Pascal, except I added the ENABLE_COLORHUG=true and -d 2 bits.

    % apt-get install argyll
    % ENABLE_COLORHUG=true dispcal -v -d 2 -m -q m -y l -t 6500 -g 2.2 test
    % targen -v -d 3 -G -f 128 test
    % ENABLE_COLORHUG=true dispread -v -d 2 -y l -k test.cal test
    % colprof -v -A "make" -M "model" -D "make model desc" -C   "copyright" -q m -a G test
   

6. Load the profile

    % dispwin -d 2 -I test.icc           
   

   It seems this only loads the x property _ICC_PROFILE_1 instead of _ICC_PROFILE; whether this works for a particular application seems to be not 100% guaranteed. It seems ok for darktable and gimp.

(insert here: I've been to FOSDEM, I got a nasty flu, and other $lame_excuses for the delay in sending out this report) Dear Project Members, here's the monthly DPL activity report, this time for January 2013. About the next DPL This is the last DPL report before the start of the election process for the next term: around early March, about 20 days from now, the Secretary will send out the call for nominations. I'd like to respond (also) here to inquiries I'm receiving these days: I will not run again as DPL. So you have about 20 days to mob^Wconvince other DDs to run, or decide to run yourself. Do not to wait for the vary last minute, as that makes for lousy campaigns. I'm available to give feedback about my DPL experience to prospective candidates, ... and also to join mobbing^Wconvincing actions toward potential candidates. Just contact me. Call for helps

• Last year delegation for Google Summer of Code Admins 1 has expired and the program for 2013 will likely start soon. I'm looking for volunteer admins for this year, to organize Debian activities in the program. If you're interested, please contact me.
• In January we had a couple of related discussions on -project about DFSG 10 and maintaining an authoritative list of DFSG-free licenses. The latter would be an important contribution to the Free Software "political" ecosystem. An ikiwiki-based infrastructure to maintain such a list has been created by ftp-masters but needs to be populated. At this point we need volunteers willing to review licenses already present in main and fill them in. If you're interested, please review the discussion and manifest yourself on debian-dak@lists.d.o, where coordination about this work will happen.

Assets

• The long standing issue of writing a proper (outbound) trademark policy for Debian marks has been completed. I've reviewed on -project outstanding items from the last discussion, and documented how they've been implemented in a new policy draft. Later on, I've published the updated policy draft on our website.
• Complementary to the above, Ian Jackson has summarized the state of the discussion about our (inbound) trademark policy, i.e. what to do when accepting in the Debian archive software subject to trademark. It looks like we are close to conclusion on that front too.
• I've worked with representatives of Debian France, on the shared interest in having the association become a Debian Trusted Organization (per Constitution 9.3). We're not yet ready to start the 2 weeks discussion period to accept the orga as such (see Constitution 5.1.11), but I'd like to do that soon. So I encourage all of you to find out about the association, which is run by well-known project members.

Cloud Images Work has gone on also on the front of supporting Debian installation in public "clouds". Thanks to Arnaud Patard, Jose Miguel Parrella Romero, Pierre Couzy, and Gianugo Rabellino, we now have Debian testing images for Microsoft Azure. Together with Amazon EC2, this is the second large provider supporting Debian via images maintained by Debian Developers. More providers are welcome, exactly as more hardware/CD vendors shipping Debian are always welcome. If you want to contribute support for other providers just show up on the -cloud mailing list and say so. Some documentation effort in view of Wheezy are in need of help too, in order to let our users know about "cloud" options, see #695681. DPL helpers The DPL helpers experiment goes on. We have had 2 more IRC meetings in January (see the minutes). Documentation of the "team" communication channels (mailing list, IRC, Git, etc.) is now available from the DPL wiki page. Talks I've given an invited Debian talk at Polytech'Grenoble, as part of a free software event organized for students of local universities. Slides of the talk are available. I'd like to thank Vincent Danjean for the event organization. Let's release Wheezy now!
Cheers.

---

PS the day-to-day activity log for January 2013 is available at the usual place master:/srv/leader/news/bits-from-the-DPL.txt.201301

Well, I had a very nice run at the start of this year in writing journal entries nearly every day, and then I seem to have gotten much quieter. This is not actually because less stuff is going on, but I started working on larger projects without meaningful daily progress, and I also got drawn into some very rewarding creative work that has kicked my schedule in the ass. Schedules are something that I've always struggled with. I'm fairly convinced that I have even more than the natural human tendency towards a longer than 24-hour schedule. (I lived in 28-hour days and 6-day weeks for one full quarter when I was a college student and rather liked it, but it wasn't practical enough to stay with.) This means that, without applying a lot of discipline, I tend to stay up later and later because I'm engaged in interesting things and don't want to stop, and then drift further and further out of sync with the world. When I was just out of college and working as just another member of a team that wasn't very close-knit, I just shrugged and let it happen. I used to get into work at about two or three in the afternoon and work until well past midnight. It had some lovely properties, such as getting to work most of my "day" after everyone else had left and the building was quiet. Unfortunately, it's really not compatible with a lead position that requires interaction with people and quite a few meetings and other types of coordination. Given that I can't just let my schedule go wherever, I realized a year or two ago that I do much better if I can stick to a fairly reliable and consistent schedule and go to bed at a regular time, even if I'm involved in something. It cuts off my evenings in a way I don't like, but it makes mornings much more pleasant and it makes it easier to schedule and stick with various chores like exercise. But it's remarkably hard to get back into that mode once I've gotten out of it, particularly when I get caught up in things in the evenings. This meander is mostly my way of getting these thoughts out of my head and on a screen in front of me so that I'll think about them and build up enough willpower to go back to a regular schedule. It really is worth it; I was keeping one through most of January and it had a lot to do with why January was awesome. Hopefully a more sustantive update tomorrow.

After the studies of the rebuild of the Debian archive using clang 2.9 & 3.0 and 3.1, the results of the 3.2 rebuild have been published on http://clang.debian.net. The percentage of failure is the same as clang 3.1: 12.1% of failure. That means that on 18264 packages, 2204 failed to be built with clang (it was 17710/2137 with the version 3.1). The fact that the percentage is the same can be explained by at least two reasons:
First, some upstreams packages have been fixed to be built correctly with clang.
Second, the new warnings + -Werror (example: -Wsometimes-uninitialized) introduced in clang 3.2 triggered some new build failures. To conclude, I think that the 3.2 release confirms the turning point of the 3.1. In term of support of the C and C++ standard, clang has now reached an equivalent quality to gcc (with better detection of errors and an extended set of warnings in -Wall).
Now, from the perspective of the Debian rebuilds, the decrease of number of failures will come from the upstream developers improving their codes (exemple of the error non-void function should return a value) or the Debian Developer/maintainer fixing the programming errors.

When I was testing C code coverage analysis tools using WebAuth, I really should have investigated the segfault that I started getting in the test suite before releasing 4.4.1. But since valgrind didn't report any problems, I thought it was something specific to code coverage. Nope, uninitialized variable inside the WebKDC code. Which valgrind didn't catch at all for some reason. Now fixed, and I did check for similar issues, but it's sort of disturbing that valgrind completely missed it, particularly for a variable whose address is taken and therefore which I wouldn't think would be optimized away. Since I hate to release WebAuth to only fix one thing, I also tested out-of-tree builds for the first time in a while and fixed a pile of issues around how the Perl module builds work. There's got to be a better way. Maybe sometime I'll try to come up with some patches to teach Module::Build how to handle VPATH. I also imported some additional Autoconf glue that will hopefully help Hurd builds proceed farther, although I didn't promise anything in NEWS since WebAuth has not yet successfully built on Hurd. You can get the latest release from the official WebAuth distribution site or from my WebAuth distribution pages.

For the second year in a row, I've led the team that organized video at FOSDEM. This year, most things went pretty smoothly; more so than last year. We definitely learned from some of our mistakes, and although I can't say that everything went better than it did last year, after having done a first look at things I can certainly say that overall, the result is better now than it was last year. Our setup was very similar to what it was last year. However, there were some key differences:

• Last year, we used two semi-professional cameras (with XLR audio inputs), two HD camcorders (with only mini-jack audio inputs), and one old SD-only camcorder (similarly, with only mini-jack audio input). This year, we decided to rent three XLR-capable cameras instead; while this drove the cost up, it seriously reduced the complexity for the audio setup. As a result, we had much less issues with talks that only had audio recorded.
• Due to some misunderstanding, last year the ULB's A/V department was not contacted to request access to their equipment. As a result, we had to do our own audio set-up in Janson and the K auditorium. This year, we made sure that this problem did not repeat itself; as a result, in the K auditorium and in Janson, all we had to do was connect an XLR feed to the camera, and we had audio.
• Last year, the machines doing transcoding were two rented laptops, one fairly recent spare laptop that I had lying around, and one somewhat older desktop machine. This was needlessly complex. This year, I rented less laptops but added a server with significant diskspace that we could do the flumotion recording on. This made the streaming and recording somewhat less complex to administer.
• Most critically, this year I had arranged for special green Videoteam t-shirts. This made it much easier to recognize who my trained videoteam volunteers were, and was instrumental in ensuring we didn't have the same volunteers man the same recording station all sunday afternoon, as was the case last year.

Unfortunately, that didn't mean all was well. The internal server would transcode the DV stream into WebM, which was then sent on to the Flumotion platform where it was going to be transcoded to Ogg Theora, before being streamed in both formats. Due to an issue which we could not track down (possibly something incorrect in my configuration), the transcoding into Ogg Theora did not work smoothly, and even had the unwanted effect of disrupting the WebM streaming. Since this made it difficult to debug without disrupting service, eventually we decided to give up on Ogg Theora and only provide WebM streams; I considered that to be a better solution than no streaming at all. In addition, apparently in the year since FOSDEM 2012, libav and gstreamer have changed significantly enough that the scripts that I used then had to be somewhat revisited in order to be able to transcode the existing recordings into WebM ready for video.fosdem.org. Oh well, I had to add in normalisation anyway; only it would've been nice if I didn't have to revisit it from scratch. Meh. At any rate, our streams seem to have been very popular. Some statistics:

Saturday

Sunday

Served traffic by country (over the whole weekend)

Served sessions by country (over the whole weekend)

As you can see from these graphs, watching the streams from Belgium seems to be happening fairly often; I suspect much of that is people at the conference watching a stream in a different room. Additionally, we peaked at 212 viewers, which is slightly more than the peak of 206 that we had last year. The final conclusion we can take home from these statistics is that the most popular talk on Saturday (at least according to people watching the streams) was the eudev one in the cross-distro room (with 68 viewers); on sunday, the talks in Janson were all fairly equally popular. The peak of 102 viewers was reached around 15:45. Overall, I'm quite happy with the result. Yesterday, Thomas Eug ne of NamurLUG (who used to do video for several years before I took over last year) helped me do the first review. What's left now is to do some transcoding and uploading; this will happen over the next few days and weeks.

Public service announcementWhen porting GNU/Linux applications to a new architecture, such as 64-Bit ARM, one gets familiar with the following error message:

checking build system type... x86_64-pc-linux-gnu
checking host system type... Invalid configuration  aarch64-oe-linux': machine  aarch64-oe' not recognized
configure: error: /bin/sh config.sub aarch64-oe-linux failed

This in itself is trivial to fix - run autoreconf or just copy in new versions of config.sub and config.guess. However, when bootstrapping a distribution of 12000+ packages, this becomes quickly tiresome. Thus we have a small request:

If you are an upstream of a software that uses autoconf - Please run autoreconf against autotools-dev 20120210.1 or later, and make a release of your software. Aarch64 porters will be grateful as updated software trickles down to distributions. This was the most discussed point during my FOSDEM talk "Porting applications to 64-Bit ARM".

The Debian PTS now speaks the Turtle representation format for the export of RDF meta-data about Debian source packages. Alongside HTML pages for humans, and the RDF/XML that had already been added to it this means that a new flavour of RDF is now available. The Turtle format offers the benefits of both machine-readable meta-data, and a somehow human readable textual format too. For instance, you may check the apache2 Turtle meta-data from the command-line with :
$ curl -L -s -H "Accept: text/turtle" http://packages.qa.debian.org/apache2 Here s a link to a colorized HTML preview of http://packages.qa.debian.org/a/apache2.ttl. Under the hood, the XSLT stylesheets of the PTS have been reworked to produce the Turtle format by default, and later convert them to RDF/XML. Every Debian source package then has a reference URI in the Linked Data word, in the form http://packages.qa.debian.org/PACKAGE_NAME, that redirects, through proper content-negociation (the HTTP Accept header) to the HTML, RDF/XML or Turtle documents. For apache2, these are, resp. at http://packages.qa.debian.org/a/apache2.html, http://packages.qa.debian.org/a/apache2.rdf and http://packages.qa.debian.org/a/apache2.ttl. The meta-data uses the model of the ADMS.SW ontology (1.0), and the content has also been slightly updated to make it more conformant to the ADMS.SW specifications (checks done with the ADMS.SW validator). Let s hope this makes RDF more familiar to Debian folks, and allows more Linked Data interlinking with other resources about FLOSS packages.

Following my last rant about configuration management, I've had a closer look at Salt, both for my personal use, as well as for a client who would like to deploy something that is not Puppet. Salt has some very good ideas, for instance:

• Configuration is just data in YAML format, which makes it really easy to whack up stuff and also to generate it using other means;
• There's a lot of plugability: runners, renderers, states, modules, returners, outputters, all of which can be replaced with custom implementations;
• It builds on remote execution. In fact, in the docs, they talk first about remote execution and then about configuration management. Not only does this makes a lot of sense, it also gives the admin the additional benefit of having quick, remote access to all hosts in the infrastructure. I like that as it means I don't also have to deploy a parallel-execution solution.
• Salt supports both, a push and a pull model for configuration management. In fact, the push model is part of its remote execution base and it's implemented through persistent connections from the clients to the master. This neatly solves all problems relating to firewalling and NAT, and it also means that the master always has a pretty good understanding of who's around. It feels good to me. And there is no NIH going on with respect to pulling: there is no daemon, you have to use cron. Good.

But there are also a couple of downsides to Salt:

• The default templating engine is Jinja. While that may be an excellent choice for applications in inherently insecure environments, such as the Web, they are quite painful in the context of Salt: no multi-line logic, whitespace control is a nightmare, and a completely unfamiliar set of conditionals and filters, rather than access to e.g. Python. This is only the default, and mako is an alternative, but it seems like a poor choice and that does not instill confidence.
• I don't think that the method of targeting hosts that Salt uses is scalable. I don't want to intersperse my configuration with if-then-else statements referencing hostnames (what would that mean if a hostname changed or a new host was added?). Instead, I want the configuration to be fully parametrised and assemble the parameters from a node database, ideally a hierarchical one. I have a solution for that which works with Salt, called reclass.
• I don't think much of "pillars", which are Salt way to provide random data (parameters) to nodes, again because data needs to be targeted at hosts, rather than defined for a host. This is also solved with reclass.
• Logging and error reporting in Salt are abysmal, inconsistent, and useless. This is offset a little bit by the ability to use pdb, but it's not ideal.
• Exceptions are not properly handled, the file descriptors aren't closed when daemonising, children are not reaped as they should, and the exit code will almost always be zero, even on failure.
• State requirements are not enforced where they are required, but only come into play when a higher order function is called.
• The code reinvents module loading and does that in very complex ways, while putting tight limitations on what's possible. State functions and modules cannot be namespaced, for instance, nor can they easily make use of each other. There is also no way to quickly define macros without resorting to the templating language (and thus a completely different syntax and paradigm), and the concept of having modular, self-contained states (e.g. one for sudo, one for SSH, etc.) with all their dependencies in one hierarchy is not really part of the design. Also, it's not trivial for a state definition to export an interface for other states to use.
• While we're on the topic of code and design, there's a lot of duplicate stuff in need of refactoring (e.g. saltutil._sync, and it seems that this even causes a lot of crypto work to be done over and over again. It also doesn't help that salt-call, which is used to execute commands on the clients (e.g. pull configuration), works completely independently of the daemon process running on each client (although the same code base is being used).
• And why aren't state and execution modules implemented as classes? Instead, the module is itself treated as an entity, and a home-cooked "virtual" system is used to determine which platforms support which modules. Object-oriented programming principles gives you all of that, and more.
• Fortunately, the Salt authors didn't cave in and write the entire communication layer themselves. Instead, they employed ZeroMQ, but it's still quite common for a client to become unreachable (permanently), e.g. due to an IP address change, or networking problems. What's worse is that the master does not keep track of who's listening.
• While we're on the subject of standing on the shoulders of giants, Python's msgpack apparently cannot handle Unicode. Yay.

Those are the big issues. There are many small issues two, but those won't be around for too long as the project is moving along quickly and the community is vibrant. This is surely an important point that speaks for Salt. However, the above issues seem to hint at design choices that might well turn out to stand in the way later. Following a day of frustration, I now feel the overpowering urge to write my own configuration management system, because of course I feel that I could do it better than everyone else. Does this sound familiar to you? Let's just say hypothetically that I would, then I'd want to reuse as much existing functionality as possible. For instance, I'd want the entire remote execution framework to be independent from any configuration management implemented on top. So what does this mean? What would such a remote execution framework need? Here are some thoughts:

• I'd want to keep it the way of Salt, i.e. the clients maintain persistent connections to the master, and the master regularly pings the clients, for housekeeping. And the clients know to expect such pings at regular intervals, so can reset themselves in case they don't hear from the master. Or scream.
• Of course, authentication and encryption need to be part of this. Ideally, key and roster management are already available in the tool that's being reused. I don't want to have to have a new PKI for this
• The protocol would probably be something like XMLRPC, with an extensible list of modules on the client to do the work. Data would be standardised to JSON-format.
• Asynchronous execution would be a plus. I am even tempted to say that it should be all built asynchronously, even though I don't really know a use case for this.
• The clients should be able to feed back information to the master, where data can be accumulated, allowing for cross-node configuration (cf. http://comments.gmane.org/gmane.comp.sysutils.salt.user/2098). This could be implemented using master-side polls to keep the protocol easier.

Doesn't this sound like a Unix botnet to you? ;) I could imagine whacking this up with a bit of Python, some shell glue, socat and SSH: the server would have an authorized_keys file with forced commands connecting the client to the server process via sockets. Or I could imagine using twisted for that. But I would prefer if something like this already existed. Anyone? Comments are broken on my blog, and I cannot be bothered to work on them. If you have any input, please write to me. I will (eventually) condense all feedback into a new article. NP: Mouse on Mars: Parastrophics

So, today an endurance test can be declared as finished. In early 2008, for the first time, I paid for a cellular phone (as my previous ones were all 100% subsidized by the operator in a fixed plan). I got a Nokia N95. And, although at the beginnning I was quite thrilled with my smartphone (when such things were still a novelty), it didn't take much to me to start dumbing it down to what is really useful to me: A phone with a GPS. And the GPS only because it is the only toy I want in such a form factor. Anyway, despite the operator repeatedly offering me newer and more capable models, I kept this one, and as soon as I was free of the forced 18 month rental period, switched to a not-data-enabled, pay-as-you-go plan. I don't want to be connected to the intertubes when I'm away from a usable computer! But yes, five years are over what a modern phone wants to endure. Over time, I first started getting SIM card errors whenever the phone was dropped or slightly twisted - As I'm a non-frequent phone user, I didn't care much. Charging it also became a skill of patience, as getting the Nokia micro-connector to make contact has been less and less reliable. Over one year ago, the volume control (two sensors on the side) died after a phone drop (and some time later I found the switches broken from the mainboard loose) - A nuisance, yes, but nothing too bad. I don't know how, but some time ago the volume went down when using the radio, and as I can't raise it again, my phone became radio-disabled. And today, the screen died (it gets power, but stays black). I can blind-operate the phone, but of course, it is really not meant for that. So, I expect this Saturday to go get a new phone. Between now and then, I'll be cellphone-deprived (in case you wonder why I'm not answering to your messages or whatever). I would love to get a phone with a real keyboard (as I prefer not to look at the screen when writing messages, just to check if everything came out right and fix what's needed). I understand Android phones are more likely to keep me happy as a free software geek, and I'd be delighted to use Cyanogen if it is usable and stable But my phone is *not* my smart computer and it should not attempt to be, so it's not such a big deal. I will look for something with FM radio capability, and GPS. Of course, I want something cheap. It would be great to get it at no cost, but I don't expect I'll find such a bargain. Oh, and I want something I can find at the first Telcel office I come to, am I asking for too much? :) Anyway - I'll enjoy some days of being really disconnected from any wireless bugs (that I am aware off).

Well, today was another day of sleeping, zoning, and having no brain for programming (and no willpower for that or exercise, which is even more frustrating), so you all get more musing about missions and work. Tomorrow, I will re-engage my normal schedule and turn the television off again, because seriously, two days of recovery should be enough even for a whole afternoon of meetings. Perhaps the best concept in the scrum methodology for agile project management is the concept of the product backlog and the product owner. For those who aren't familiar, a scrum project has, as one of its inputs, a prioritized list of stories that the team will implement. During each planning session (generally between once a week and once a month), the development team estimates and takes from the front of the product backlog the top-priority stories that can be completed during that period of time. The product owner is responsible for building the list of pending development stories based on business needs and keeping them sorted in priority order so that the work of the team follows the goals of the larger organization. But the product owner does not get to say how difficult the stories are; the development team estimates the stories, and the product owner may change order based on those estimates (to do simpler things earlier, for example). We've been doing scrum to some extent or another for quite some time now, and while there are a variety of things that I like about it, I think this is the best part. And I think it has very useful concepts for management in general. The product backlog is effectively a problem stream of the sort that I talked about in my last couple of posts. It's a set of problems that need to be solved, that matter to other people. And, if the team is doing scrum properly, the problems are presented in a form that describe a complete unit of desired functionality with its requirements and constraints, but without assuming any sort of implementation. The problem is the domain of the product owner; the solution is the domain of the development team. I am increasingly convinced that my quality of life at work, and that of many other people doing similar sorts of development and project work, would be drastically improved if managers considered this role the core of their jobs. (Ironically, at least in our initial experiences with scrum, it was quite rare for regular managers to be product owners; instead, individual line staff or project managers act as product owners. I think this speaks to how confused a lot of managers are about their roles in a technical development organization. This seems to be improving.) The core of the sort of work that I do is a combination of ongoing maintenance and solving new problems. Both of those can be viewed as a sequence of (possibly recurring) stories; I know this because that's largely how I do my own time management. Apart from HR and staffing, the core role of a manager is almost exactly what scrum calls "backlog grooming": communicating with everyone who has input into what their group is doing, finding out what their problems are, translating that into a list of things that the organization needs to have done, prioritizing them, breaking them into manageable chunks, ensuring they are fully specified and actionable (read: well-written stories), and communicating their completion back to other stakeholders (and then possibly iterating). This lept out at me when I started thinking about our larger strategic vision. That strategic vision is a sort of product backlog: a set of development stories (or, in this case, epics that would be broken down into a large number of smaller stories). But most strategic plans have glaring flaws if they're evaluated by the standards of scrum product backlogs. They're normally written as marketing statements and aimed at the clients or customers rather than at the staff. From the staff perspective, they're often hopelessly vague, not concrete, actionable epics. Often, they are almost entirely composed of grand, conceptual epics that any scrum team would immediately reject as too large and nebulous to even discuss. And often they're undefined but still scheduled: statements that the organization will definitely complete some underspecified thing within the next year or two. Scrum rightfully rejects any form of scheduling for unspecified stories and epics. Additional vagueness and sketchiness is tolerated the farther the story is from the current iteration, so they can be put into the future backlog, but scrum makes no guarantees about when they'll get done until they've been sized and you can apply a reasonable velocity estimate. If you want to put a date on something, you have to make it concrete. This is exactly the problem that I have with long-range strategic plans. We already know this about technology development: long-range strategic plans are feel-good guesses, half-truths, and lies that we tell other people because we're expected to do so, but which no one really believes. The chance that we can predict anything about the shape of projects and rate of delivery of those projects three years from now is laughable, and certainly none of the work that would be required to make real estimates has normally been done for those sorts of long-term strategic projects. There are a lot of scrum advocates who would go farther than I in asking for a complete revolution of how technical projects are planned. I'm not sure how realistic that is, or how well the whole scrum process would work when rolling up work across dozens, if not hundreds, of scrum-sized teams. But in the specific area of the role of managers in a development organization, I think we could learn a lot from this product backlog concept. Right now, there is a constant tension between managers who feel like they need to provide some sort of visionary leadership and guidance (the cult of Steve Jobs) and line staff who feel like managers don't understand enough about specific technology to lead or guide anything. But if you ask a technical developer whether, instead, it would be useful for managers to provide a well-structured, priority-ordered, and clean problem stream from the larger organization so that the developer can trust the problem stream (and not have to go talk to everyone themselves) but have control over the choice of implementation within the described constraints, I think you would find a great deal of enthusiasm. As a further bonus, scrum has a lot of advice about what that problem stream should look like. I think some of the story structuring is overblown (for example, I can't stand the story phrasing structures, the "AS A... I WANT TO... SO THAT..." shackles), but one thing it does correctly emphasize is the difference between problem and solution. The purpose of the story is to describe the outcome from the perspective of the person who will use the resulting technology. Describing the constraints on the solution, in terms of cost or integrations, is actively encouraged. Describing the implementation is verboten; it's up to the project team to choose the implementation that satisfies the constraints. If managers in general would pick up even that simple distinction, I think it would greatly improve the engagement, excitement, and joy that developers could take in their work. There's very little that's more frustrating than to be given an interesting problem in the guise of a half-designed solution and be told that one is not permitted to apply any of one's expertise in elegance, robustness, or durability to replacing the half-baked partial solution. Or even be allowed to know the actual motivating problem, instead of the half-digested form that the manager is willing to pass on.