When fixing or modifying a package I at times change the installed files directly. However, once the changes are okay and it is time to prepare a diff, having to keep track and manually diff'ing the installed files is time consuming.

That's when ldebdiff does its thing: it runs diff against the original files of a given package and the ones installed on a system.
For example:

$ ldebdiff acpi-fakekey
--- unpacked/etc/init.d/acpi-fakekey    2012-04-05 05:14:21.000000000 -0500
+++ /etc/init.d/acpi-fakekey    2012-04-12 12:10:55.000000000 -0500
@@ -4,8 +4,8 @@
 
 ### BEGIN INIT INFO
 # Provides:          acpi-fakekey
-# Required-Start:    $local_fs $remote_fs
-# Required-Stop:     $local_fs $remote_fs
+# Required-Start:    $remote_fs
+# Required-Stop:     $remote_fs
 # Default-Start:     2 3 4 5
 # Default-Stop:      
 # Short-Description: Start acpi_fakekey daemon

($remote_fs implies $local_fs)

To whom it may concern. I ve just uploaded new upstream version of dibbler (portable DHCPv6 client/server) to unstable.
The resolvconf patch needed some changes to apply on that version so whoever actually uses dibbler, please test it ASAP so we could address any issues. This version amongst other things fixes nasty Debian bug #659476.

• The "Save the baltic languages" week..:-)
• Latvian translations completed for level 1. Latvian won't be deactivated!
• Also progress in levels 2 and 3 for Latvian
• Lithuanian progress for level 1. Language won't be deactivated!
• Estonian was already safe..:-)

Status for D-I level 1 (core D-I files):

• 51 languages 100%: ar ast be bg bn bs ca cs da de el eo es et eu fa fi fr gu he hi id it ja kk km kn ko lo lv mr nb nl pa pl pt pt_BR ru si sk sl sr sv te th tr ug uk vi zh_CN zh_TW
• 1 languages 99%: ta
• 2 languages 97%: ga hu
• 5 languages 95%: dz is lt mk ro
• 1 language 92%: ml
• others are 90% or below

Status for D-I level 2 (packages that have localized material that may appear during default installs, such as iso-codes, tasksel, etc.):

• 31 languages 100%: ast be bg ca cs da de eo es fa fr gu he id is it ja kk km mr nl pl pt ru si sk sl sv tr uk zh_CN
• 8 languages 99%: eu fi hr lv nb th vi zh_TW
• 3 languages 98%: dz pt_BR ro
• 1 language 97%: ta
• 1 language 96%: ug
• 3 languages 94%: ar el gl
• 1 language 92%: sr
• 5 languages 92%: bn bs hu ko ne
• others are 90% or below

Status for D-I level 3 (packages that have localized material that may appear during non-default installs, such as win32-loader)

• 38 languages 100%: ast be bg bs ca cs da de el eo es fa fi fr ga gl he hr id is it ja kk km nb nl pl pt ru sk sl sr sv th tr vi zh_CN zh_TW
• 3 languages 98%: hu ug uk
• 1 language 95%: lv
• others are 90% or below

Full 100% completeness (hall of fame) for 26 languages: Asturian, Belarusian, Bulgarian, Catalan, Czech, Danish, German, Esperanto, Spanish, Persian, French, Hebrew, Indonesian, Italian, Japanese, Kazakh, Khmer, Dutch, Polish, Portuguese, Russian, Slovak, Slovenian, Swedish, Turkish, Simplified Chinese

• Latvian is now completed.
• Lithuanian is now completed.
• I got an offer for Welsh

At this very moment, it means that I would deactivate 14 languages:

• Amharic (5f17u in sublevel 2)
• Welsh (39f82u, 32f47u)
• Galician (7f10u, 6f9u)
• Croatian (16f19u, 6f9u)
• Georgian (17f47u, 10f21u)
• Kurdish (12f25u, 6f22u)
• Malayalam (5u in sublevel 2 only!)
• Nepali (16f54u, 10f21u)
• Norwegian Nynorsk (13f34u, 13f23u)
• Albanian (19f50u, 13f23u)
• Tamil (2f1u, 1f)
• Tagalog (18f54u, 24f39u)

I ve recently found myself in need of more then one sbuild chroot, each one set up slightly differently then the next. For a while, providing different behavior with unstable and sid was all well and good, but I figure, like any good computer scientist, I should solve the general problem. As a result, I ve written an sbuild wrapper to help maintain sanity. It uses a series of symbolic links to hide everything out of the way, and some bash hackery. Basically, I ve set up a /var/sbuild directory, full of a few different files. For each distribution, there is one folder, one symbolic link and up to two other files. There s the dist.d file (such as /var/sbuild/sid.d) that contains all the variants , dist symbolic link (such as /var/sbuild/sid) which points to the currently active chroot, and up to two lock files. The script will default to sid-default (that is to say, exactly, /var/sbuild/sid.d/sid-default - which is a symbolc link to the default chroot), but may take two arguments, in the order of variant, then dist. Currently, I have two chroots, one for gcc, and one for clang (set up like Sylvestre used for cland.d.n) but I may add some more with different setups. Currently the behavior is to have the second wait kindly if you kick off two sbuilds at once, and fail out a third. There are a bunch of bugs, and I m not sure that this will work very well, but it s working. The next step is to combine this with unionfs overlays to help protect accidentally trashing the chroots. You can find the script on my GitHub, pasted below.

#!/bin/bash
DIST="sid"
VARIANT="default"
SBUILD=/var/sbuild
if [ "x$1" != "x" ]; then
    VARIANT=$1
fi
if [ "x$2" != "x" ]; then
    DIST=$2
fi
if [ -e $SBUILD/$DIST.lock ]; then
    PID= cat $SBUILD/$DIST.lock 
    echo "W: We *must* wait for the finish of PID $PID"
    echo "W: Blocking execution until this file is removed."
    if [ -e $SBUILD/$DIST.wait ]; then
        echo "E: Someone else is waiting. Bombing out."
        exit 2
    fi
    echo $$ > $SBUILD/$DIST.wait
    while [ -e $SBUILD/$DIST.lock ]; do
        sleep 1
    done
    echo "I: Dropping out of lock. Purging wait."
    rm $SBUILD/$DIST.wait
fi
echo "I: Aquiring lock"
echo $$ > $SBUILD/$DIST.lock
CURRENT=$(basename  ls -xl $SBUILD/$DIST   awk ' print $11 ' )
# We set up the name like:
#   dist-variant
# such as "sid-clang" is cool.
#echo "I: We're currently $CURRENT."
if [ "x$CURRENT" == "x$DIST-$VARIANT" ]; then
    echo "I: This matches the desired setup. No action."
else
    if [ -e $SBUILD/$DIST.d/$DIST-$VARIANT ]; then
        echo "I: Re-setting $DIST symlink."
        rm $SBUILD/$DIST
        ln -s $SBUILD/$DIST.d/$DIST-$VARIANT $SBUILD/$DIST
    else
        echo "E: No such dist! Looking in $SBUILD/$DIST.d/$DIST-$VARIANT."
        exit 1
    fi
fi
echo "I: "
echo "I: Dist:    $DIST"
echo "I: Variant: $VARIANT"
echo "I: "
# echo "I: Before we go farther, we're updating."
# sbuild-update -d $DIST --upgrade
sbuild -d $DIST
echo "I: Releasing lock"
rm $SBUILD/$DIST.lock

I ve decided to start putting together packages to rebuild the bare bones install I use for most of my systems, since rebuilding everything by hand is starting to get annoying. I call it hairy candy. Because my setup relies on some chem interesting chem hacks, none of this is fit in any way for the archive. If the vim team was to catch wind of what I was doing, they d be none too happy :) While these packages work great for me, I can t stress enough how badly they can break something. Using Zack s guide, I was able to get an archive up and running, no problem. Taking his advice, I even got signing working. Simple. mini-dinstall rocks a nice alternative to a PPA, for sure. At least, for one arch :) Now, the one spin I took on Zack s work was to add a landing page a listing of what s in the archive. I did this using a bit of Python & Jinja2. The script follows.

#!/usr/bin/env python
# Copyright (c) Paul Tagliamonte, under the terms of the Expat license.
from jinja2 import Template
import sys
def parse_packages(fd):
    ret = []
    cur =  
    key = None
    for line in fd.readlines():
    if line.strip() == "":
        ret.append(cur)
        cur =  
        continue
    if line[0] == " ":
        cur[key] += "\n" + line.strip()
    else:
        key, val = line.split(":", 1)
        cur[key] = val.strip()
    return ret
pkgs = parse_packages(open(sys.argv[1], 'r'))
context =  
    "pkgs": pkgs
 
t = Template(open(sys.argv[2], 'r').read())
print t.render(**context)

Invocation looks something like:

render.py $ARCHIVE/debian/wicked/Packages $DATA/index.html

I kick this script off in the singing script, since mini-dinstall doesn t have a exit hook. Works great, thanks everyone!

i dislike having to run Windows as much as the next free software developer, but like many sysadmins, i am occasionally asked to maintain some legacy systems. A nice way to keep these systems available (while not having to physically maintain them) is to put them in a virtual sandbox using a tool like kvm. While kvm makes it relatively straightforward to install WinXP from a CD (as long as you have the proper licensing key), it is more challenging to transition a pre-existing hardware windows XP installation into a virtual instance, due to Windows only wanting to boot to ide chipsets that it remembers being installed to. In particular, booting a disk image pulled from a soon-to-be-discarded physical disk can produce a Blue Screen of Death (BSOD) with the message:

Stop error code 0x0000007B

or

(INACCESSABLE_BOOT_DEVICE)

This seems like it's roughly the equivalent (in a standard debian GNU/Linux environment) of specifying MODULES=dep in /etc/initramfs-tools/initramfs.conf, and then trying to swap out all the hardware. At first blush, Microsoft's knowledge base suggests doing an in-place upgrade or full repartition and reinstall, which are both fairly drastic measures -- you might as well just start from scratch, which is exactly what you don't want to have to do for a nursed-along legacy system which no one who originally set it up is even with the organization any more. Fortunately, a bit more digging in the Knowledge Base turned up an unsupported set of steps that appears to be the equivalent of setting MODULES=most (at least for the IDE chipsets). Running this on the old hardware before imaging the disk worked for me, though i did need to re-validate Windows XP after the reboot by typing in the long magic code again. i guess they're keying it to the hardware, which clearly changed in this instance. Such silliness to spend time working around, really, when i'd rather be spending my time working on free software. :/

After a relatively long period of flux, I've managed to settle into a routine where I've got all the time I can dedicate to open source stuff bunched together: every Tuesday night. I've started calling it "Debian Day", although Debian won't get all that time exclusively. One evening a week doesn't sound like much, but that's unfortunately a realistic amount given my other commitments. Still, having a firm idea of how much resource I have makes planning what I can and can't do much easier. It also means I need to be quite selective about what I spend my time on. This also impacts the type of interactions I can use. I can respond to email, bugs, etc., but I cannot effectively interact with folks on IRC, for example: the period between windows is too large for it to work effectively. Therefore, I can't work reliably with any team or area of Debian that relies on IRC to get things done. Tonight, then. I have recently been ripping some CDs, a task for which I use grip, despite it long since having been dropped by Debian. It still works very well, whereas Sound Juicer, which still pops up whenever I insert an audio CD, does not. I've been interested in helping out with Debian GNOME packaging for a while, and it seems the issue has been fixed upstream, but after asking around in #debian-gnome, the root cause is that the newer musicbrainz API packages have not been uploaded to Debian. (the existing package uses the old API). Two separate people appear to have been working on a new Music Brainz package, but neither filed an ITP bug. I've spent the largest chunk of my Debian time just trying to figure out what has been going on, sadly, but at least there's the likelyhood this will be fixed for wheezy. There's also the tail end of a musicbrainz-2.1 transition finishing up. Another thing I considered this evening was the future of debgtd in Debian stable. On reflection, and after sitting on the fence for several months, I've decided I don't have the time to fix it up enough to have it ready for wheezy, so with regret I've filed a removal bug. With any luck, at some point in the wheezy cycle I'll manage to put more time towards this, because I think it addresses a genuine need and could be useful to people. Finally, whenever I sit down and look at Debian stuff to finish up, I have a strong temptation to fix some issues belonging to the games team. In particular, Gustavo Panizzo has put a lot of work into vavoom packages, but sadly his requests for sponsorship are falling on deaf ears. There's also Johey Shmit's work on packaging zdoom which needs some attention to finish removing some embedded libraries. I have to keep reminding myself I left the team for a reason, to focus on something else, and finding that something else should be the priority. So all in all not the most uplifting evening: I'm starting to think I simply couldn't work effectively with the Debian GNOME team without being able to dedicate more time to use IRC effectively; I spent more time that could be necessary figuring out a bunch of other folks have potentially wasted each other's time by working on packages and not filing ITPs; and I've removed one of my own packages from the distribution altogether. It's not all doom and gloom, though: last week I packaged Simon Howard's lhasa, a library and tool for decompressing LHA files, as used by various old DOS games. (See? I just can't escape that stuff ) I also managed to 'resolve' two bup bugs, one RC. I also finished transitioning from my old PGP key to my new one.

Enjoy burp 1.3.6 in unstable!

The new version of Lintian (2.5.8) can pretty much be summed up as:

Its like 2.5.7, only with less false positives and no FTBFS.

Especially people annoyed by the hardening flags will hopefully find that 2.5.8 greatly reduces the number of false positives. I believe this is best demonstrated with an example:

$ lintian --print-version && lintian -q -C binaries amarok_2.5.0-1_i386.deb   wc -l 
2.5.7
94
[...]
$ lintian --print-version && lintian -q -C binaries amarok_2.5.0-1_i386.deb   wc -l
2.5.8
4

However, nothing comes for free. We dropped hardening-no-stackprotector from the default profile (and demoted it to an I tag). For hardening-no-fortify-functions we made a false positive -> false negative trade-off by ignoring binaries if their only unprotected function is memcpy. For more information, please refer to #673112. hardening-no-stackprotector is still available and can be used via the debian/extra-hardening profile (or via the tags argument). For the 2.5.7 behaviour of hardening-no-fortify-functions, you have to use hardening-check directly. But 2.5.7 had other changes besides the myriad of false-positives:

• Around 19 redundant tags were removed.

A consequence of this is that we no longer warn if you use things like dpkg assert-working-epoch or your postinst creates a usr/doc transition symlink .

• The last of the Lintian s (perl) modules have been put under the Lintian name space.

With all the Lintian modules under the Lintian namespace, we can install them in the standard perl @INC path. Admittedly, I am not certain we are ready to commit to the current API in these modules, which is one of the reasons why they are still installed in /usr/share/lintian. But in a couple of releases things may look differently.

• Lazy loading of data files

Lintain 2.5.7 ships over 50 data files of various kinds (usually white- or blacklists of some kind). Before 2.5.7, all data files with a few exceptions would be loaded eagerly (i.e. as soon as the check was run for the first time). A few data files had been special cased with manual laziness . In 2.5.7, Lintian::Data was updated to lazily load the data. So depending on the packages being checked, Lintian may now load fewer data files.

• Create proper data files for tables embedded within checks.

While this is not something exclusive to 2.5.7, we have separated quite a few checks from their data tables by now. My personal favorites are the table of known interpreters and (from 2.5.8) the table of known versioned interpreters.

• Support for Vendor specific data files.

I am certainly biased here. But this is probably the most awesome feature in Lintian 2.5.7. It is now possible for vendors to extend or simply shadow the core Lintian data files. Allow me to demonstrate how this can be used:

$ lintian --tags build-depends-on-obsolete-package a2ps_4.14-1.1.dsc 
E: a2ps source: build-depends-on-obsolete-package build-depends: dpatch
$ cat ~/.lintian/profiles/local/main.profile 
Profile: local/main
Extends: debian/main
$ cat  ~/.lintian/vendors/local/main/data/fields/obsolete-packages 
@include-parent
@delete dpatch
bison
$ lintian --profile local/main --tags build-depends-on-obsolete-package a2ps_4.14-1.1.dsc
E: a2ps source: build-depends-on-obsolete-package build-depends: bison
$

It is a toy example, but I believe it is a good demonstration of the feature. The full documentation of Vendor specific data files can be found in the Lintian User manual (in lintian/2.5.7 or newer). It will also be on lintian.debian.org when we find time to update lintian there.

Android support in Gammu is one of frequently requested features. I don't have neither time to motivation to do that, so unless somebody else will step up, it won't be done. Recently, I got offer from MyPhoneExplorer author to share it's applet for communicating with Android phones. It basically provides protocol similar to Sony-Ericsson phones Gammu already supports, just over TCP/IP. So all what would have to be done is glue layer for using TCP/IP and possibly some minor adjustments to Sony-Ericsson code. Both of these should be quite easy tasks and I'm willing to guide you through this process. In case anybody would be interested in this challenge, just let me know :-).

Filed under: English Gammu 1 comments Flattr this!

The past few weeks have consisted of a heady mixture of taking interesting pictures of cute people, and writing code. I spent a while getting to grips with seccomp filters, using the facilities present in recent GNU/Linux Kernels to filter system calls binaries are allowed to make. My initial test was to patch GNU Less to only allow it to open, read, and close files. The side-effect of this was that the built in shell-escape was closed, thus allowing me to test it. After that I toyed around with interfacing with spidermonkey seamonkey. In this regard I was less succesful, but I did manage to write code in C that would invoke javascript functions loaded dynamically. Similarly I could call from my (loaded) javascript code into functions defined in C. I don't have a use for a Javascript to/from C bridge, but I'm sure that time will come. Photography has been a constant distraction, I took some fun shots of the Edinburgh Marathon, and then distracted myself with a volunteer to take an abstract fishnet photograph. We went on to do some more fun shots begin careful to stay on the safe side of the NSFW limit. I think this is borderline NSFW, but we were both clear exactly what we wanted and we got it perfectly there. Next week will be quieter, but providing we don't have another mini-heatwave in Edinburgh I'll be cheerful regardless. ObQuote: One, two, Freddy's coming for you. - A Nightmare on Elm Street (original)

As we are getting closer to the World IPv6 Day, I thought that I'd see if I can upgrade my WRT54GL's OpenWRT (White Russian) installation to accept Router Adverts from the ISP to acquire an IPv6 address (rather than via a 4-to-6 tunnel broker, which seems to be the most documented case), and to propagate its IPv6 address space to my local network via my WRT45GL's radvd. Unfortunately, most of the instructions out there concern newer OpenWRT releases to which I cannot upgrade to because their software base takes up significantly more space than White Russian does. Still, I figure that someone in the community must have already succeeded at configuring their WRT54GL for this and might be able to help. Anyone? :)

Multiarch has been coming RSN for an extremely lengthy meaning of the word "soon". I remember watching Tollef give a presentation about it at DebConf4 and I'm pretty sure it's been talked about at every DebConf since then as well. Deemed the "correct" answer to the issue of running i386 binaries on x86_64 machines, or old ARM ABI programs on more modern hardware, it's always seemed to be at least another Debian release away. Not so anymore. Through foolishness I ended up buying a Brother HL3040CN when I first moved to the US. It was a cheap networked laser printer and it touted Linux support. Quality wise it's been fine. I don't use it a lot, but unlike an inkjet I don't have to worry about not using it for a month and then needing to print something in a hurry and having to clean print heads etc. Where it falls down is that I failed to check that "Linux support" involved source. No. Instead it involves an i386 binary (at least packaged as a .deb, but in a horrible fashion). Up until now I've mostly been printing from my laptop, so all the drivers are installed there. I've got some guests this week and they needed to print their boarding passes, so I decided it was time to make the house server act as a print server too. It's an AMD64 box and before now I haven't had any need to run i386 code on it, so when I installed the driver deb it failed to work. Normally I'd just install ia32-libs, but this time I decided to try multiarch. So I did:

# dpkg --add-architecture i386
# apt-get update
# apt-get install libc6:i386

and magically I was now able to run the printer driver binary. I know there's a lot more work still to be done (I need to check if I can ditch ia32-libs on my laptop which runs a few more i386 only apps), but this is pretty cool - thanks to all those involved in making it happen! Update: I tried to install all the multiarch bits required for Skype on my laptop but hit an issue with libqtgui4:i386 which ends up pulling in liblcms1:i386 which isn't yet multiarch enabled. There was already a bug, #637732 filed by vorlon, and mhy did the appropriate NMU a week ago, so it should hopefully hit testing in the next week. Thanks guys.

NetSNMP comes with its own event loop based on the select() system call. While you can build your program around it, you may prefer to use an existing event loop, either a custom one or something like libevent, libev or Twisted.

Own custom event loop Let s start with the easiest case: you have written your own event loop. This means you make a call to select(), poll(), epoll(), kqueue() or something similar. All those functions take a set of file descriptors and wait for any of them to become available in a specified time frame. Here is a typical use of select() (adapted from Quagga)

readfd = m->readfd;
writefd = m->writefd;
exceptfd = m->exceptfd;
timer_wait = thread_timer_wait (&m->timer);
num = select (FD_SETSIZE,
              &readfd, &writefd, &exceptfd,
              timer_wait);
if (num < 0)
   
    if (errno == EINTR)
      continue; /* signal received - process it */
    zlog_warn ("select() error: %s",
               safe_strerror (errno));
    return NULL;
   
if (num == 0)
   
    /* Timeout handling */
    thread_timer_process (&m->timer);
   
if (num > 0)
   
    thread_process_fd (&readfd);
    thread_process_fd (&writefd);
   

thread_process_fd (fds) function iterates on each file descriptor fd and executes the appropriate action if FD_ISSET (fds, fd) is true. Integrating NetSNMP into such an event loop is easy: NetSNMP provides the snmp_select_info() function which alters a set of file descriptors to insert its own. Here is the new code:

#if defined HAVE_SNMP
struct timeval snmp_timer_wait;
int snmpblock = 0;
int fdsetsize;
#endif
/* ... */
#if defined HAVE_SNMP
fdsetsize = FD_SETSIZE;
snmpblock = 1;
if (timer_wait)
   
    snmpblock = 0;
    memcpy(&snmp_timer_wait, timer_wait,
           sizeof(struct timeval));
   
snmp_select_info(&fdsetsize, &readfd,
                 &snmp_timer_wait, &snmpblock);
if (snmpblock == 0)
  timer_wait = &snmp_timer_wait;
#endif
num = select (FD_SETSIZE,
              &readfd, &writefd, &exceptfd,
              timer_wait);
if (num < 0)   /* ... */  
#if defined HAVE_SNMP
if (num > 0)
  snmp_read(&readfd);
else if (num == 0)
   
    snmp_timeout();
    run_alarms();
   
netsnmp_check_outstanding_agent_requests();
#endif
/* ... */

snmp_select_info() may modify the provided set of file descriptors (and therefore, its size). It may also modify the provided timer in case it needs to schedule an action before the original timeout. snmpblock is a tricky variable. While select() can be called with a timeout set to NULL, this is not the case of snmp_select_info(): you have to pass a valid pointer. snmpblock is set to 0 if the provided timeout must be considered or to 1 otherwise. snmp_select_info() will set snmpblock to 0 if and only if it alters the timeout. Here are a two examples of integration. Both are for a subagent but the code for a manager is the same:

• Quagga
• Keepalived

Third-party event loop With a third party event loop, you don t have access to the select() system call anymore. Therefore, you cannot alter it with snmp_select_info(). Instead, at each iteration of the loop, a list of SNMP related file descriptors is kept up-to-date with the help of snmp_select_info(): new ones are added and old ones are removed.

libevent Let s assume that snmp_fds is the list of current SNMP related events¹. A function levent_snmp_update() calls snmp_select_info() to update this list. Here is a partial implementation (error handling removed and some declarations omitted, look at the complete version from lldpd):

static void
levent_snmp_update()
 
    int maxfd = 0;
    int block = 1;
    FD_ZERO(&fdset);
    snmp_select_info(&maxfd, &fdset, &timeout, &block);
    /* We need to untrack any event whose FD is not in  fdset 
       anymore */
    for (snmpfd = TAILQ_FIRST(snmp_fds);
         snmpfd;
         snmpfd = snmpfd_next)  
        snmpfd_next = TAILQ_NEXT(snmpfd, next);
        if (event_get_fd(snmpfd->ev) >= maxfd  
            (!FD_ISSET(event_get_fd(snmpfd->ev), &fdset)))  
            event_free(snmpfd->ev);
            TAILQ_REMOVE(snmp_fds, snmpfd, next);
            free(snmpfd);
          else
            FD_CLR(event_get_fd(snmpfd->ev), &fdset);
     
    /* Invariant: FD in  fdset  are not in list of FD */
    for (int fd = 0; fd < maxfd; fd++)
        if (FD_ISSET(fd, &fdset))  
            snmpfd->ev = event_new(base, fd,
                EV_READ   EV_PERSIST,
                levent_snmp_read,
                NULL);
            event_add(snmpfd->ev, NULL);
            TAILQ_INSERT_TAIL(snmp_fds, snmpfd, next);
         
    /* If needed, handle timeout */
    evtimer_add(snmp_timeout, block?NULL:&timeout);
 

Then, replace the main loop (usually, a call to event_base_dispatch()) with this:

do  
    if (event_base_got_break(base)  
        event_base_got_exit(base))
        break;
    netsnmp_check_outstanding_agent_requests();
    levent_snmp_update();
  while (event_base_loop(base, EVLOOP_ONCE) == 0);

Here is how are defined the two callbacks:

static void
levent_snmp_read(evutil_socket_t fd, short what, void *arg)
 
    FD_ZERO(&fdset);
    FD_SET(fd, &fdset);
    snmp_read(&fdset);
    levent_snmp_update();
 
static void
levent_snmp_timeout(evutil_socket_t fd, short what, void *arg)
 
    snmp_timeout();
    run_alarms();
    levent_snmp_update();
 

Twisted reactor As a second example, here is how to integrate NetSNMP into Twisted reactor. Twisted is an event-driven network programming framework written in Python. It does not come with support for SNMP². Because of the language mismatch, the integration of NetSNMP in Twisted needs to be done using a C extension or the ctypes module. Twisted event loop is called a reactor. Like for libevent, events need to be registered. It is possible to register file descriptor-like objects using a class implementing a handful of methods. Here is the implementation of such a class (adapted from PyNetSNMP, a subproject of Zenoss using the ctypes module):

class SnmpReader:
    "Respond to input events"
    implements(IReadDescriptor)
    def __init__(self, fd):
        self.fd = fd
    def doRead(self):
        netsnmp.snmp_read(self.fd)
    def fileno(self):
        return self.fd

Like for libevent, we have a function to update the list of SNMP related events (stored as a mapping between file descriptors and SnmpReader instances):

class Timer(object):
    callLater = None
timer = Timer()
fdMap =  
def updateReactor():
    "Add/remove event handlers for SNMP file descriptors and timers"
    fds, t = netsnmp.snmp_select_info()
    for fd in fds:
        if fd not in fdMap:
            reader = SnmpReader(fd)
            fdMap[fd] = reader
            reactor.addReader(reader)
    current = Set(fdMap.keys())
    need = Set(fds)
    doomed = current - need
    for d in doomed:
        reactor.removeReader(fdMap[d])
        del fdMap[d]
    if timer.callLater:
        timer.callLater.cancel()
        timer.callLater = None
    if t is not None:
        timer.callLater = reactor.callLater(t, checkTimeouts)

Contrary to libevent, we cannot alter the main loop to call updateReactor() at each iteration. Therefore, it must be called after each SNMP related function. For another example, have a look at my equivalent implementation as a C extension.

Miscellaneous

Lack of asynchronicity Integrating NetSNMP into an event-based program is not risk-free. I have written a fairly comprehensive article on the lack of asynchronicity in NetSNMP AgentX protocol implementation. I urge you to read it if you want to integrate a SNMP subagent into an existing program. On the manager side, you will get similar drawbacks when using SNMPv3. To retrieve or manipulate management information using SNMPv3, it is necessary to know the identifier of the remote SNMP protocol engine. This identifier can be configured directly on each manager but it is usually discovered by querying SNMP-FRAMEWORK-MIB:: snmpEngineID, as described in RFC 5343. Unfortunately, this discovery is done synchronously by NetSNMP. There is a bug report for this issue but it seems difficult to fix. I have not been able to come up with a proper patch but I have described a workaround around snmp_sess_async_send(). There is no such problem with SNMPv2.

Limitation of file descriptor sets snmp_select_info() and snmp_read() uses the fd_set type to handle a set of file descriptors. This type should be manipulated with FD_CLR(), FD_ISSET(), FD_SET() and FD_ZERO(). Those may be defined as functions but they usually are macros. Moreover, no file descriptor greater than FD_SETSIZE (which is usually set to 1024) can be handled with the fd_set type. This means that if you have a file descriptor greater than 1024, you won t be able to use snmp_select_info() with it. Starting from NetSNMP 5.5, you can use snmp_select_info2() and snmp_read2() instead of snmp_select_info() and snmp_read(). They use the netsnmp_large_fd_set. If you want to keep compatibility with NetSNMP 5.4, here is another twist. The fd_set type is usually a fixed-size array of long integers. FD_CLR(), FD_ISSET() and FD_SET() are size-independant. The size only matters for FD_ZERO() which is not used by either snmp_select_info() or snmp_read(). You can therefore allocate a larger fd_set. A common way is to compile your program with -D__FD_SETSIZE=4096. You can still use FD_ZERO() yourself. However, this is not portable (GNU C Library only). Another way is to allocate your own array of long integers and cast it to fd_set. You ll have to redefine FD_ZERO():

typedef struct  
   long int fds_bits[4096/sizeof(long int)];
  my_fdset;
#undef FD_ZERO
#define FD_ZERO(fdset) memset((fdset), 0, sizeof(my_fdset))
 
   /* ... */
   my_fdset fdset;
   FD_ZERO(&fdset);
   /* ... */
   rc = snmp_select_info(&n, (fd_set *)&fdset, &timeout, &block);
   /* ... */
 

Threads NetSNMP provides two API:

• the traditional session API which is not thread-safe and
• the single session API.

In the above examples, I have used the traditional session API. If you are using threads, you need to ensure that all SNMP operations are done in a single thread. Otherwise, you need to adapt the examples to use the single session API. snmp_select_info() is part of the traditional session API. You should replace it with snmp_sess_select_info() and keep a list of SNMP sessions. This API is not available for the agent side of NetSNMP.

---

1. The event list is global but some code could be added to bind a different list for each base in case you use different bases. This is not needed in the case of lldpd which uses only one event base.
2. TwistedSNMP is an SNMP protocol implementation for Twisted based on PySNMP but is not maintained anymore. PySNMP comes with examples on how to integrate with Twisted.

Hi! I have been selected to work on multiarch cross-toolchains as part of the Google summer of Code. Although coding officially started last week, I have been busy with exams and other uni stuff at the time. Anyway, let's talk about the project itself, that is, multiarch cross-toolchains. What is multiarch? In order to understand what the project is about, one need to know what multiarch is. I'll quickly describe it, and I invite anyone interested to read the spec (really, it's interesting, and not that long). Basically, multiarch lets you install binary packages for multiple architectures on one system, and let package managers resolve dependencies as one would expect. This works by identifying four sets of packages:

• Packages that don't need to be aware of multi-arch. Such packages can only be installed for one architecture at a time, and can only be used to resolve dependencies of packages having the same architecture. Those packages are Multi-Arch: none or multiarch-unaware packages.
• Packages that can be installed for multiple architectures at a time, and can only be used to resolve dependencies of packages having the same architecture. Those packages are Multi-Arch: same packages. They typically are libs.
• Packages that can only be installed for one architecture at a time but can satisfy dependencies of any package regardless of its architecture. Those packages are Multi-Arch: foreign packages. They typically are tools working with architecture-independent interfaces. (for instance, gpg, zenity or tar).
• Packages that can only be installed for one architecture at a time but can satisfy any dependency annotated with :any , regardless of the architecture. Those packages are Multi-Arch: allowed packages. They are packages providing both arch-dependent and arch-independent interfaces. They are typically things like python, which can be used to interpret arch-indep files, but providing arch-dependent interfaces for C extensions.

Anyway, multiarch is an amazing thing that may be of use in various cases, such as running i386-only software on an amd64 system, running armel software on an armhf system, running i386 software on an armhf system using qemu-user-static and binfmt, cross-grading (switching, for instance, from i386 to amd64 on a running system), and, in our case, easing cross-compilation. State of cross-toolchains in Debian There aren't really cross-toolchains available in Debian (although some can be built from source packages), but they are available in emdebian's repositories. However, those packages predate multiarch, and don't make use of it. Instead, they depend on special packages converted using dpkg-cross. Such packages are merely arch: all versions of foreign packages, with files moved to arch-qualified paths to prevent conflicts with normal packages. Such packages thus have to be generated from real packages, either manually or using some sort of automation. In addition, this additional step brings several issues, like keeping in sync converted packages with real packages from the archive. With multiarch, such packages are rendered useless, and may in fact bring additional problems, since converted and multiarch packages may share the same files, and thus may not be co-installable. Switching from dpkg-cross to cross-arch deps Cross-toolchains packages currently depend on a few packages converted using dpkg-cross (ex: libc6-$ARCH-cross and libc6-dev-$ARCH-cross for some packages). They need to depend on the real package with the right arch (ex: libc6:$ARCH, libc6-dev:$ARCH). The obvious way to do so is by annotating the dependency with a specific arch qualifier (ex: Depends: libc6:armhf ). This is, however, not covered by the multiarch spec. Note that some lib*-$ARCH-cross packages will remain (for instance, libgcc1-armhf-cross). Indeed, those packages are different from the packages mentioned above in that they are of the architecture of the build system and actually part of the toolchain itself.
The above isn't actually true. Indeed, those packages are runtime library cross-built during gcc's build. Likewise, cross-toolchains may build-depend on foreign packages (ex: libc6-dev). Thus, specific arch qualifiers must be handled in the Build-Depends source package field too. Changing dpkg and apt to support that should be fairly straightforward. In fact, I already have a few patches, but I need to refresh them, maybe clean them up a little, and push them a little harder to get them accepted!

Managing the puppet manifest using a vcs is a best practice and there is a lot of material on the web. The easier way to do it, is to use git directly in the directory /etc/puppet and use a simple synchronization strategy with an external repo, either to publish your work, or simply to keep a backup somewhere. Things are a bit more complicated when you would like to co-administer the machine with multiple people. Setting up user accounts, permission and everything can be a pain in the neck. Moreover working from your desktop is always more comfortable then logging in as root on a remote system and make changes there ... The solution I've chosen to make my life a bit easier is to use gitolite, that is a simple git gateway that works using ssh public keys for authentication and does not require the creating of local users on the server machine. Gitolite is available in debian and installing it is super easy : apt-get install gitolite . If you use puppet already you might be tempted to use puppet to manage your gitolite installation. This is all good, but I don't advice you to use modules like this one http://forge.puppetlabs.com/gwmngilfen/gitolite/1.0.0 as it's going to install gitolite from source and on debian it's not necessary ... For my purposes, I didn't find necessary to manage gitolite with puppet as all the default config options where good enough for me. Once your debian package is installed, in order to initialize your repo, you just need to pass to gitolite the admin public key, that is your .ssh/id_rsa.pub key and then run this command: This will create the admin and testing repo in /var/lib/gitolite/repositories and setup few other things. At this point you are ready to test you gitolite installation by cloning the admin repo : Gitolite is engineered to use only the gitolite use to manage all your repositories. To add more repositories and users you should have a look at the documentation and then editing the file conf/gitolite.conf to add your new puppet repository. At this point, you can go two ways. If you use git to manage your puppet directory, you can just make a copy of it somewhere and then add gitolite as a remote If you didn't use git before, you can just copy the manifest in your new git repository, make a first commit and push it on the server. Every authorized users can now use git to clone your puppet repository, hack, commit, push ... One last step is to add a small post-receive hook on the server to synchronize your gitolite repository with the puppet directory in /etc : This will sync your main puppet directory and trigger changes on the nodes for the next puppetd run. First I created a small shell script in /usr/local/bin/puppet-post-receive-hook.sh : This script presuppose that your git repo in /etc/puppet has the gitolite repo as origin ... Then I added a simple hook in the gitolite git repo that calls this script using sudo : And since you are at it you should also add a pre-commit hook to check the manifest syntax. This will save you a lot of useless commits. If you use a more complicated puppet setup using environments (I'm not there yet, and I don't think my setup will evolve in that direction in the near future), you can use puppet-sync that seems a neat script to do the job. For the moment this setup works pretty well. I'm tempted to explore mcollective to trigger puppet runs on my nodes, but I'm there yet...

I installed a Lenovo Thinkpad X130e with Wheezy.

OpenSSH 6.0p1 was released a little while back; this weekend I belatedly got round to uploading packages of it to Debian unstable and Ubuntu quantal. I was a bit delayed by needing to put together an improvement to privsep sandbox selection that particularly matters in the context of distributions. One of the experts on seccomp_filter has commented favourably on it, but I haven't yet had a comment from upstream themselves, so I may need to refine this depending on what they say. (This is a good example of how it matters that software is often not built on the system that it's going to run on, and in particular that the kernel version is rather likely to be different. Where possible it's always best to detect kernel capabilities at run-time rather than at build-time.) I didn't make it very clear in the changelog, but using the new seccomp_filter sandbox currently requires UsePrivilegeSeparation sandbox in sshd_config as well as a capable kernel. I won't change the default here in advance of upstream, who still consider privsep sandboxing experimental.

new week, new report about my work on RC bugs. again, mostly gcc 4.7 related, & again, mostly using patches prepared by others.

• #667111 bcov: "bcov: ftbfs with GCC-4.7"
  add patch from Cyril Brulebois, upload to DELAYED/2
• #667308 oggvideotools: "oggvideotools: ftbfs with GCC-4.7"
  add patch from Paul Tagliamonte, upload to DELAYED/2
• #667322 pianobooster: "pianobooster: ftbfs with GCC-4.7"
  add patch to fix Windows linker flags, upload to DELAYED/2
• #667329 powertop: "powertop: ftbfs with GCC-4.7"
  add patch from Cyril Brulebois, upload to DELAYED/2
• #667361 rtorrent: "rtorrent: ftbfs with GCC-4.7"
  add patch from Cyril Brulebois, upload to DELAYED/2, later fixed in maintainer upload
• #667362 rubberband: "rubberband: ftbfs with GCC-4.7"
  add patch from Cyril Brulebois, upload to DELAYED/2
• #667364 rutilt: "rutilt: ftbfs with GCC-4.7"
  add patch from Cyril Brulebois, upload to DELAYED/2
• #667388 tagcoll2: "tagcoll2: ftbfs with GCC-4.7"
  manual close
• #667403 uisp: "uisp: ftbfs with GCC-4.7"
  add compiler flag, upload to DELAYED/2
• #667419 xdiskusage: "xdiskusage: ftbfs with GCC-4.7"
  add patch from Cyril Brulebois, upload to DELAYED/2
• #667866 src:cenon.app: "cenon.app: FTBFS with GCC-4.7"
  add patch from Yavor Doganov, upload to DELAYED/2
• #667873 src:gridlock.app: "gridlock.app: FTBFS with GCC-4.7"
  apply patch from Yavor Doganov, upload to DELAYED/2
• #667874 src:gworkspace: "gworkspace: FTBFS with GCC-4.7"
  add patch from Yavor Doganov, upload to DELAYED/2
• #672003 src:dibbler: "dibbler: FTBFS: ClntIfaceIface.cpp:34:49: error: 'unlink' was not declared in this scope"
  add patch from Paul Tagliamonte, upload to DELAYED/2
• #672025 src:mmpong: "mmpong: FTBFS: client.cpp:276:71: error: 'usleep' was not declared in this scope"
  add patch from Cyril Roelandt, upload to DELAYED/2
• #672042 src:l2tp-ipsec-vpn-daemon: "l2tp-ipsec-vpn-daemon: FTBFS: src/VpnClientConnection.cpp:167:60: error: '::chown' has not been declared"
  add patch from Paul Tagliamonte, upload to DELAYED/2
• #672089 src:icewm: "icewm: FTBFS: Xlib.h:1694:1: error: 'deprecated' was not declared in this scope"
  add patch from Ubuntu / Andreas Moog, upload to DELAYED/2, then included in maintainer upload
• #673286 gem: "gem: FTBFS with gcc-4.7"
  send debdiff to the BTS, later NMUd by a fellow DD
• #674301 src:libplack-middleware-reverseproxy-perl: "libplack-middleware-reverseproxy-perl: FTBFS: tests failed"
  upload new upstream release (pkg-perl)
• #674304 src:clusterssh: "clusterssh: FTBFS: tests failed"
  add missing build dependency
• #674395 src:libdbd-mysql-perl: "libdbd-mysql-perl: FTBFS: ld: cannot find -lssl"
  manual close (pkg-perl)