Version 0.1.0 of RInside, my C++ wrapper classes which facilitate embedding R into your own C++ application, has been released and is now an official CRAN package. This release improves on the build process and should work on any sane Unix-alike operating system. A few more examples were added or extended. Details and the full ChangeLog are on my RInside page, and there is now a RInside page on CRAN as well.

I m currently working on an updated version of python-debianbts a Python interface to Debian s Bugtracker. The goal is to equip the Bugreport class with all available attributes delivered by the SOAP interface. The problem is, that for some attributes it is not quite clear what data they provide and in which datatype they are wrapped. For example the mergedwith attribute should be a list of bugnumbers, but it seems to be a single Integer when merged with one bug and an empty String when the bug is not merged at all. Some attributes have an ambiguous name and it s hard to guess what they mean, for example there is an id and a bug_nr and both seem to contain the same information. There is a git branch for this task and a wiki page collecting all available information. If you have some experience with SOAP, our BTS and some time to kill, your help would be appreciated.

this weekend while giving a bit of TLC to the php packages, i thought i'd finally get around to tackling #490023 and similar bugs, which meant learning about how to use the triggers feature provided by dpkg. unfortunately documentation for this feature was in fairly short supply, and after extensive searching (and by extensive searching i mean typing "dpkg triggers howto" into a google search) i had a couple short manpages (dpkg-trigger(1) and deb-triggers(5)) and an overly verbose and possibly out of date /usr/share/doc/dpkg/triggers.txt.gz. none of these documents really gave a clear "big-picture" for how to get going either. so, as is often the case, i got distracted from my intended task and ended up on a little side-project putting together a nice howto/tutorial for how to integrate dpkg triggers into a package. note that there may be misunderstandings or inaccuracies in the document, as it's based on an afternoon's worth of hacking and q&a directed at #debian-dpkg (thanks to Guillem and Rapha l for fielding those questions). so if i'm off on anything, please feel free to let me know! to get started:

git clone http://git.debian.org/git/users/seanius/dpkg-triggers-example.git

and for those who are too lazy even for that, the README follows: The debian package triggertest This package is a dead-simple "triggers in a nutshell". It shows a few different ways that triggers can be used in practice, to hopefully cover most of the general use cases for triggers. The purpose of triggers Triggers are used to ensure that during the standard package-management process certain operations always run, but not more than necessary. For example:

• during an upgrade many packages might provide updated texinfo(5) documentation, and triggers are used to ensure that update-info-dir(8) is run without having to be run several unnecessary times from the maintainer scripts of package in question.
• if a system has both apache2 and PHP's apache2 engine installed, apache2 will need to be restarted whenever new php extensions are installed or removed. Without triggers it would be very disruptive (and inefficient) to explicitly stop and start apache by each PHP extension.
• update-initramfs(8) should be run once and only once after kernels are updated or certain other packages are installed/upgraded that would want to have a new initramfs generated.
• other expensive operations that are desirable to be aggregated and consolidated during installs/upgrades, such as update-texmf(8) or scrollkeeper-update.

How triggers work in a nutshell Trigger-using packages can be classified in two behavioral categories:

• Consumers: packages which declare triggers and thus can be "triggered"
• Producers: packages which activate triggers (explicitly or implicitly)

When a consumer is triggered, its postinst script is run with the arguments:

postinst triggered "<trigger1> ... <triggerN>"

i.e. $2 contains an iterable list of activated triggers. Note that if a consumer is going to be normally configured (i.e. it is also being updated), then no triggering may occur and thus the standard control flow of the maintainer scripts should still take care to handle this. A "trigger" is declared for the consumer by shipping a file in DEBIAN/triggers (in the case of debhelper based packages, ./debian/consumer-package.triggers will JDTRT). Useful RTFM: deb-triggers(5). Example:

interest /path/to/a/directory
interest my-second-trigger

This declares that the package in question is interested in two triggers. The first trigger has a leading path separator in its name, which instructs dpkg that any filesystem modifications underneath this directory by other packages should cause the respective consumer(s) to be triggered. The second name is completely arbitrary (and also global, so you should probably pick something reasonably specific and identifiable), and will only cause a trigger to be activated when a producer

1. ships its own triggers file with a corresponding "activate my-second-trigger", or
2. invokes the trigger explicitly in a maintainer script by calling "dpkg-trigger my-second-trigger".

Another perhaps useful RTFM: dpkg-trigger(1). Examples contained in this source package This source package produces a single "consumer" package and a number of different "producer" packages. The consumer is triggered in various ways, such as:

• Filesystem updates to a given directory from any other package
• "activate foo" from a producer's triggers file
• Activation from a producer's explicit call to dpkg-triggers

In this case the triggers correspond to an "update-foo" and "update-bar" command, which do nothing other than echo to the console that they are being run. In practice they'd correspond to updating some kind of database or restarting a daemon. More information For more information about triggers, see /usr/share/doc/dpkg/triggers.txt.gz, which will make a lot more sense having a working example like this package as reference. Especially useful is "Transition hints for existing packages", which can be used if you need to gracefully handle migrating in support for triggers across other packages not directly under your control. Corrections, suggestions, etc Are appreciated and should be sent to the author, below. -- Sean Finney seanius@debian.org Sat, 19 Sep 2009 15:57:48 +0200

One of my professional duties in my past ten years was monitoring systems. Even my diploma thesis was dedicated to distributed monitoring (altough my professor sucked badly). Apart from a few custom-programmed scripts to analyze special situations (e.g. proxy clusters) I used tools that fellow administrators will find familiar: Nagios and Cacti. And another less famous text-configuration-based monitoring tool called Cricket. That worked well somehow but Cricket was hard to learn for my coworkers and Cacti seems unreliable and fundamentally broken in terms of SNMP checking. Besides why do I have to set up availability checking in Nagios and set up checking of the same parameters in another software to draw graphs? Just recently I came across an open-source software I hadn t heard of before: Zabbix. And although it has a few rough edges it seems way more professional than other common tools (the commercial tools I saw are even worse than the open-source variants). I tried it and after a lot of reading and trying it looks like it has a good potential to replace Nagios and Cacti. So I thought I d sum up my personal experiences with all of these tools. Nagios. Their makers claim that it s the industry-standard in IT infrastructure monitoring . Honestly it s a great tool but considering how many years it has been existing it barely evolved. During my diploma thesis I wrote an alternative software that I called MrNetwork that dealt with flaws that Nagios hasn t even fixed today. Still Nagios is a tool I have used for many years and it is very reliable. Advantages:

• open source
• large community
• many powerful plugins (and own plugins are easy to create: just write a program that prints a one-line string and set a certain return code)
• easy-to-use web frontend
• debugging plugins is moderately simple.
  
• many thought-out features like host groups or notification options that make your life easier
  
• dependencies (so that you don t get 100 alerts if a router between the Nagios server and other servers went down)
  
• nagvis plugin with a great interactive editor that draws nice management-suitable graphs (although I found the ndo2db interface hard to set up at first and a little flaky)
  

Annoyances:

• The focus is on availability checking you don t get fancy graphs on the values that are monitored (e.g how the CPU load was over time). So you ll need a second tool and set up the same checks there just to get graphs. But availability percentages are computed automatically.
• Textual configuration that has so many different settings that you need to look up the parameters often. A web-based configuration would probably be better (and is available as an add-on but I haven t tested it).
• Third-party plugins are often very badly programmed and barely documented that it appeared easier to reinvent the wheel. ( Look, ma, I can has plugins. )
• Some views on the web interface are not very obvious (e.g. clicking on the title of a host group gives a nice view of all hosts with all services).
• Many plugins don t have corresponding configuration entries so you have to find out how they work and write configuration entries yourself (and those which are preconfigured take some archeology to find out which parameters they expect). This is a huge time-waster for beginners. And in your services configuration you have to verify your checks configuration to understand the meaning of each parameter. Or do you remember what check_http 80!john!doe!10!30!body is supposed to check?
• Every set of parameters of a certain plugin requires a distinct configuration entry. The plugins have dozens of configuration switches that you may need one day. Want to set a timeout on HTTP checks? Write another check configuration. Want to check for a certain string in the HTTP response? Write another check configuration. And so on.
• Most checks are run from the Nagios server itself (the NRPE plugin to do the checks on the respective remote systems somehow refused to work properly here) which is suboptimal and puts a lot of load on the server.
• By default every alert triggers a notification. So if you can t define proper dependencies (e.g. if you want to check your web server in all 30 supported languages and there is some logical error) then you will get spammed with alerts.

Cricket. As Nagios does not support plotting graphs of the monitored values I was in need of another piece of software. Basically Cricket is a frontend to RRD (which stores data in a rotating/round-robin file that keeps data of the last X minutes/hours/days). It has a textual configuration that takes a lot of getting used to. It s main principle is inheritance of settings they call it configuration tree . Which means you have a master DEFAULTS file that contains general settings like how to query SNMP. In a subdirectory you define a certain class of devices that you want to monitor e.g. routers (the DEFAULTS are inherited to this level). Within the routers directory you can just define a list of routers you want to monitor. All settings are inherited from above (parent directories). It s more a geek tool for shell lovers. Advantages:

• very quick to monitor a large set of similar devices once the general device class is defined
• simple web interface
• very reliable
• can monitor SNMP values (it does this very well) or execute external scripts thus can be easily extended
• flexible graphing you can sum up values of two graphs into a new graph (aka mtargets multiple targets)
• different check frequencies can be configured for different subtrees through cron (by default values are collected every 5 minutes this can be set as low as one minute if needed)

Annoyances:

• the textual configuration is error-prone (leading to funny Perl errors that can be hard to debug)
• users may expect to see all parameters of a certain device instead of all devices having a certain parameters ( Give me the statistics of router42 instead of Let s see the temperature of all routers we have. )
• customizing the graphs (drawn by RRD tool) isn t trivial
• Frequency of checks is by default 5 minutes. Before RRD can draw the first value of a graph it needs three values. So you ll be waiting 15 minutes before you see any results.
• RRD rounds data by default. So the yearly graph doesn t show the peaks that the daily graphs do. (This can be fixed by not graphing the average values but the maximum values.) Long-time archiving of graphed data is not possible without throwing away the RRD files and manually customizing them. Changing the monitoring frequency (aka heartbeat ) is not possible either without throwing away the data and starting from scratch either.
• No proper built-in alerting in case certain thresholds are exceeded.

Cacti. Another frontend to RRD and a pretty sophisticated one. Nearly everything is configured through its web interface. And the result is beautiful. It s not entirely reliable though and SNMP support (at least in version 0.8.7b) is a big fail. I like Cacti for its user interface is much better than Cactis but it s less reliable and flexible. Advantages:

• Beautiful and (for most features) simple web interface. Nice features like graphs that can be zoomed using Javascript.
• Fine-grained permissions system. So a certain user may get read-only access to a certain subtree.
• The tree where graphs are placed can be configured freely so you get exactly the view you want.

Disadvantages:

• Doesn t hide the RRD magic very well. The user is easily confused by templates, data sources and the like.
• Graphing sometimes just stops working for no reason or values are missing although the server isn t overloaded and other software doesn t show such outages. According to a quick search on the lazyweb I m not the only one with such effects.
• Setting up many systems means a lot of clicking in the web interface. Setting up new kinds of checks (aka templates ) means even more clicking and is very error-prone.
• The quality of some third-party templates I tested was pretty bad. Creating new templates is tedious, error-prone, frustrating and close to black magic. Nothing for the casual user at least.
• Doesn t handle SNMP correctly (this is the biggest fail in my opinion and makes it unusable here). Although it knows how to query indexes (e.g. ifDescr to get the names of your network interfaces) it just seems to stored fixed OIDs. So once the SNMP tables change the order or number of items (which isn t unusual) then suddenly other parameters get graphed.
• Frequency of checks is by default 5 minutes. Increasing the frequency leads to missing data and wrong results.
• As it uses RRD and needs 3 valid values you won t see that your monitoring fails until you wait 3 5=15 minutes. Not suitable for impatient non-smokers like me.
• Debugging failed checks is close to impossible. If a check fails then I find myself clicking around randomly trying to find typos because the alternative is digging around in database entries.
• The web interface is sometimes confusing. A refresh of SNMP tables is done by clicking a unmeaning green circle icon. Adding new items to a list is done by clicking an inconspicious Add link that doesn t even look like a link.
• Another UI confusion: graphs are created from the devices view. But they are deleted from the graph management view.
• No alerting in case certain thresholds are exceeded. Another tool like Nagios would still be needed to notify you.
• Can t sum up multiple targets so monitoring failover clusters doesn t work well.
• RRD averages data when putting daily values into weekly values, weekly into monthly and monthly into yearly. So the yearly graph doesn t show the peaks that the daily graphs do. (This can be fixed by not graphing the average values but the maximum values.) Long-time archiving of detailed graphed data is not possible (RRD). Changing the monitoring frequency (aka heartbeat ) is not possible either without throwing away the data and startin from scratch.

Zenoss. People pointed me to Zenoss which is supposed to offer the same functionality as other monitoring systems but is much more integrated. So this short list is more a quick one-day-experimental expression than a thorough analysis. But in the end much of the fuss is just marketing. Advantages:

• Beautiful web interface
• Nice gimmicks like google maps integration to show you where your servers are down worldwide. Only makes sense when monitoring networks with several/many remote locations.
• Can partly discover parameters of systems automatically. That works for static routes (although I wonder why the heck I want to monitor static routes), file systems and network interfaces. On the other hand processes can t be discovered automatically and have to be set up manually.
• Does not come with a specific agent but plays rather well with plain old SNMP.
• Can monitor Windows through native WMI.
• Large fanbase.

Annoyances:

• Web interface feels slow (Zope is bloated)
• Opaque operation. It does not tell what s actually going on. You can add monitoring and check back later if anything worked they way you want.
• Questionable reliability. In a test here I was monitoring a running process. The process was said to be down and suddenly went to up after a while although nothing had changed on the system.
• Just one dashboard. Several dashboards similar to Zabbix screens would be nice.
• Configuration and data is spread across MySQL, the internal Zope database storage and RRD files on disk.
• The dependency graph is a nice Flash-based applet displaying how the systems are connected. But it does not show any details about the systems aside from whether they are up or down. And clicking on a host does not take you anywhere but center on the system. Beautiful but it could do so much more than just look beautiful.
• I dislike the way that things are configured. The context menu with the down arrow needs to be used. I d prefer simple add or delete actions instead of navigating the menu all the time. Looks like Javascript is used wrongly here.
• Limited open-source version. Full version needs to be paid for.

Zabbix. This is the tool I have been evaluating for the past two weeks and currently I m moving all checks away from Nagios and Cacti and put it into Zabbix. So the following list is more a first impression instead of a year-long experience. (I m using Zabbix 1.6.5 here. Debian Lenny s 1.4 version lacked some interesting features like proper SNMP handling so I backported the newer version to Lenny.) Zabbix seems very close to the perfect monitoring system I had always dreamt of. I would have designed it differently in some aspects though. Advantages:

• Availability Monitoring (like Nagios) and graphing (like Cacti) is combined into one tool.
• Highly configurable. User John may just get an SMS for problems of high severity during the weekend and on weekdays get a Jabber message. Even automatic actions like restarting services can be set up.
• The notifications actually help the person who gets the message. Low disk space on /var on web5 with an additional comment is pretty helpful even when sent via SMS.
• Very performant. The remote agent gathers the information on each system.
• Fast web interface.
• Sophisticated monitoring of web sites. Zabbix can follow a path of simulated mouse clicks on a web site and check for functionality and response time.
• Real-time graphs. Values are by default collected every 30 seconds. You quickly see where you are going.
• Permissions system. Certain users can be limited to certain views.
• Gathered data is stored in a database (MySQL, PostgreSQL, SQLite) instead of an RRD file. Storage periods (aka history ) can be configured freely. Backup up the database is all there is to be done.
• Agent for remote checks simply works (and is available on Debian Lenny) and can even call scripts or shell one-liners to gather information. The agent can gather a lot of information locally and send it to the Zabbix master server. Works on several operating systems and even on Windows (just set up monitoring of Windows services and processes successfully).
• Templates (that can even link to further templates) save time in setting up many checks.
• Graphs (plots of values over time) can be customized like which items are plotted and in what way. Even pie charts are possible.
• Even the parameters that don t get an explicit graph can be graphed at any time. E.g. the agent has tracked the CPU load on a system that you never cared about then you can graph that with one mouse-click.
• Screens and slide shows can be used for high-level views (aka dashboards ) or to be displayed on a big geeky display. They can combine textual display of the status as well as clocks, ad-hoc graphs or predefined graphs.
• Very flexible trigger expressions. For example you can tell a trigger to fire if the average system load over the last 15 minutes is above a certain value. As all measured parameters are stored in a backend database you can use all kinds of mathemical expressions. Like firing a trigger if the average number of running processes during the last half hour is above 50. All other software I tested just has access to the last value gathered.
• Customized alerting is simple through shell-based alert scripts.
• Remote monitoring made easy by using a Zabbix proxy.
• Paid support and paid custom programming available. But the software is completely open-sourced.
• 320 page PDF manual with screenshots and nice references. (Although I d personally prefer an online help. Currently the ? link within the web interface just points to the PDF that you can download.)

Annoyances:

• A lot of mouse moving and clicking is required to set up things. For example setting up an alert if the free space on a disk on a certain server is getting too low then you need to set up hosts, items, triggers and actions. Some of the clicking seems redundant. E.g. I didn t find a way to create triggers automtically for a set of checks. If I monitor how full the /home partition is then I d like to set a threshold in the same configuration step.
• Takes a little patience understanding the concepts although they make sense after half a day.
• The web interface is crammed full of features. For casual users it s confusing to navigate.
• The map editor is close to unusable. You can put icons and connections into the graph but have to define the X and Y coordinated manually. This feature is nice but desperately needs a good interactive map editor like nagvis uses.
• Does not detect the available assets in a monitored server automatically. Imagine that you want to monitor the space on all disk partitions on a system. You will have to copy over or create the check items manually or define all possible checks in a template and disable those you don t need. Cacti handles that better by offering you a list of partitions to monitor. Zenoss can do that partly. The zabbix agent should be able to handle such a service discover automatically. (The built-in Discovery feature rather seems to detect new servers in a given network range automatically. But that s something different.)

See also: Ben Rockwood s blog Further similar software I didn test thoroughly: Hyperic and Opsview (I ll surely update this blog post while I learn more about Zabbix.) All of the above tools are great. I m not meaning to say that Zenoss is total crap for example. The differences are subtle. And whether a piece of software suits your needs really depends on your expectations. I love that all this software is available as open-source. And a totally unscientific but fun analysis of the community is counting the number of active people in the respective channels on the Freenode IRC network:

• #nagios: 133 users
• #cacti: 58 users
• #cricket: 2 users
• #zenoss: 54 users
• #zabbix: 61 users

Either Nagios has the largest fanbase or perhaps that means that the majority of people needs help with it.

Over on Planet Ubuntu there is a VRMS meme winding down. Usually I tend to think web log aggregator memes are an abomination, however, this one is a little more interesting, especially if Debian folk pick it up. It seems like many Planet Ubuntu posters require some non-free stuff, examples include; graphics and other drivers/firmware, Flash (Gnash/Swfdec aren't there yet), Sun Java (despite OpenJDK), RAR 3.0 archives, LHZ archives (despite jlha), Microsoft fonts (despite ttf-liberation), icon themes, ICC profiles, mind mapping, support for codec DLLs from Windows, VirtualBox drivers for Windows, the Opera web browser, games and other things. For Debian users, the popcon information pretty much confirms the above issues. Some other stuff Debian users use that isn't in Debian include; patented codecs, skype, acroread, the non-free version of VirtualBox and more. Next, lets get in on the getting your meme on part. On my Dell Inspiron 6400 laptop I have following Debian non-free/contrib stuff installed; firmware-iwlwifi, gcc-doc, gcc-doc-base, gcc-4.3-doc, autoconf-doc, gdb-doc, make-doc. On my OpenMoko FreeRunner phone I have nothing installed from Debian non-free/contrib. My wireless router and ADSL modem don't run Debian. The wireless router is a MIPS-based Netgear WGR614L (that is marketed as supporting DD-WRT, OpenWRT and Tomato) so I could conceivably install OpenWRT etc or Debian on it, but I haven't gotten around to that yet. The ADSL modem is a MIPS-based Siemens SpeedStream 4200, but it looks like that would require a significant amount of effort to get running FLOSS. Lets ponder my laptop freeness a bit more. The docs are all FSF-free, but not DFSG-free. It appears they are not DFSG-free because the FSF will not allow 28 words of FSF/GNU advertising to be removed (I'm not sure if Debian or the FSF is more pedantic). I'm not aware of any DFSG-free alternatives to them and I don't particularly care about the restrictions. My wireless card is an Intel 3945ABG mini PCI card that requires the kernel to upload the iwlwifi firmware before it works. Intel refuse to release firmware source code under a free license, citing FCC requirements. I'm not aware of any free firmware (or firmware reverse engineering efforts) for this device so the only alternative is to use wired Ethernet or buy another wireless card. Ethernet wires everywhere would be very inconvenient, so I've been thinking about trying to replace the card or laptop as soon as I am able. Since Atheros has released GPLv2 firmware for its AR9170 802.11n USB chipset I'll probably get something released by Atheros. I wonder why the FCC didn't prevent Atheros from releasing modifiable firmware, perhaps Intel will change their position too. So, what other software is on my laptop apart from Debian? Lets see, nothing on the HD apart from GRUB 2, /boot containing a Linux image/initrd from Debian and the LUKS partition containing swap and the Debian ext3 rootfs. Coreboot laptop support seems to be in the very early stages so I have no alternative to the supplied Dell BIOS at the moment. There is also the CPU microcode and other firmware installed in the various devices on the system, at least the HD, Ethernet, Bluetooth, GPU, DVD drive etc. have some kind of firmware in them. Some of this firmware is updatable using software from the Dell Linux repositories. Lets ponder my phone freeness a bit more. On it I have two copies of the u-boot bootloader (one in the protected NOR flash), u-boot is free. The NAND flash has an old Qt Extended Improved (QTEI) install that I need to copy stuff off and wipe. I also have a 4GB microSD card containing QTEI, Debian, FDOM and SHR. Now that QtMoko has brought QTEI to Debian packages I can probably ditch the QTEI install. I read that OM2009 is being merged into SHR so I can get rid of FDOM. That leaves SHR and Debian/QtMoko (nothing non-free). IIRC SHR does contain some firmware for devices that the OpenMoko FreeRunner doesn't contain, but that will get removed as Linux upstream moves more and more of that into linux-firmware.git. Nothing more on the main flash or microSD storage, so what else? The OpenMoko contains lots of hardware, but the main problematic devices here are the AR6000 Wi-Fi chip and the TI Calypso GSM device. The AR6000 is an issue because the firmware embedded in the chip is non-free and FullMAC so it is hard to add new features like AP mode or improve the connection quality/etc. In addition the firmware currently on the device is a development version of a 2.0 version (1.x was apparently broken) and further development does not seem to be happening yet. Since Atheros GPLed the AR9170 firmware I idly wonder if they could be convinced to release the source and GPL the AR6000 firmware. Perhaps a life-size gold statue of RMS would help there. The GSM device is problematic because of various bugs in the firmware. It apparently is an ARM chip with Nucleos, so perhaps uCLinux or something could be ported to it or written for it. There has been at least one discussion about hacking the GSM firmware (thar be dragons). Searching for some keywords extracted from the end of the first mail in that thread lead to some interesting results indeed. Lets ponder my network freeness a bit more. I should allocate some time to investigate putting Debian on my wireless router and ADSL modem. These generally just work except the ADSL modem reboots itself a lot during the summer. The just-works status of these devices means that messing with them is problematic in case I brick them (luckily I have a spare for the ADSL modem though). I presume OpenWRT and similar have replacements for all the software on my wireless router, drivers may be an issue though. I use gmail for reading email lists, mainly because of intertia, cheapness and because the interface has no GUI FLOSS peers yet (sup-mail exists for console use). TangoGPS, OpenStreetMap and other tools are slowly replacing my use of Google Maps, although I imagine routing will remain an issue for many years yet. One of the tech groups I'm involved in uses Plesk as a hosting control panel, mainly out of intertia, I'm not familiar with any FLOSS replacement though. Some of the upstream FLOSS projects I'm involved with use sourceforge as a hosting platform. There is a certain level of lock-in with SF.net, most of the data is easily exportable though. Lets ponder my software freeness in comparison to Planet Ubuntu posters. I have no freeness issues with drivers at the moment, however firmware seems likely to have freeness issues for a long time to come. I've avoided the need for the Adobe Flash Player by using swfdec and clive for downloading videos for some sites. swfdec works OK for some sites, some that I care about where it doesn't work and clive cannot download video include Vimeo, abc.net.au and sbs.com.au. For Vimeo you can download raw video if you have a login, but I do not, bugmenot is unreliable and Vimeo doesn't seem to be accepting new registrations. SBS I watch on television and ABC I get no reception so I just don't watch it. Java isn't an issue now that OpenJDK exists. I sometimes need to extract RAR 3.0 archives, which I do by copying the files to a friend's Windows machine or using a RAR extractor website. There is no free implementation of a RAR 3.0 extractor yet. I seem to remember that there is source code for a non-free extractor so presumably a clean-room documentation and reimplementation effort could fix this. I don't encounter LHZ files, but jlha-utils is available for if I do. I feel no need to install Microsoft fonts since the Liberation fonts and DejaVu meet my general font needs and Debian contains fonts covering all the scripts of Unicode (except for about half the huge amount of Han characters). In my music collection, only the Voxware audio codec and DigiTrakker MDL files are not supported by Rhythmbox, but I don't really miss those few tracks. I rarely have audio/video codec issues with video from the Internet. I don't have a use for Opera. Thanks in part to the efforts of the Debian games team, games from Debian main (such as warzone2100) satisfy my gaming needs. I just don't use Skype. Overall, I'm generally happy with my level of software freedom. My main strategy for preventing regressions in my software freedom is to just avoid doing things that require non-free software. The most problematic FLOSS issues for me are Flash support (adoption of the HTML5 video tag will resolve this for me), RAR 3.0 support and firmware. Please feel free to contact me with any comments or questions. Happy Software Freedom Day everyone!

I was really happy a few years back, when Martin announced he'd start working on netconf. Not just because I agreed with his assessment that ifupdown needed replacement; also because I had been thinking about a good way to implement such a thing, and had been planning to start writing 'soon'. With Martin's announcement, I made a few suggestions to give him some input, and then put my plans away I had more important things to take care of, anyway. Unfortunately, netconf did not manage to reach the ambitious goals that Martin set out, mainly due to lack of contributors. I wanted to help out; I really did. But Martin's choice of python to implement a 'prototype' which then had to be reimplemented in C or C++ didn't get me very thrilled. Not just because I have a severe dislike of python; also because implementing something twice is, in my humble opinion, not quite the best way to do something like this. But hey, who am I, if I don't put my code out there. Never the less, I tried helping. I really did. During debconf8 at Mar del Plata, I approached Martin with the suggestion that I start reimplementing those bits that were unlikely to change anymore in C, as a first step toward something that could be in the base system. Unfortunately, that didn't happen. After trying for about two weeks, I just gave up. I thought I could read python code and reimplement that so that it would work the same way, but I couldn't. I thought I could implement python modules in C without knowing python, but as it turns out that's a laughable idea. For several months then, I didn't think about it anymore. In March of this year, however, I got a little bored. And what does one do when they get bored? Right, you find something to do. I my case, that was 'figure out how bison works'. I'd of course known about parser generators for quite a while, but just never had the time to dive in deep and figure out how they work. When I needed to do a config file for nbd, I instead hand-wrote my parser and used a lexer from libglib. But I wanted to learn how to do things properly, so I sat down. Long story short, when I had to choose something somewhat more complex to implement, I thought 'how about a hypothetical config file for a network configuration utility', and I started implementing that. It turned out to be fun, so I didn't stop after I knew enough about bison. And just last week, for the first time I successfully used it to bring up my wired interface using DHCP -- and bring it down again. It can also already manage static interfaces through netlink, though it isn't quite able to the equivalent of 'ip addr flush' yet. Fun. The system is quite flexible. It has a plugin interface (which needed some refinement that I did using some help on IRC just today), which should allow developers to implement extra functionality by just dropping a shared object in the right directory. In fact, I'm working on the wireless support (hence my blog post of last week) as a plugin. Obviously the code isn't remotely ready yet. The wireless code has a long way to go. The firewall module needs to be started. It will probably crash and burn if put to the test. But even so, it already has one feature that ifupdown, in its 10 years of existence, never acquired: it will not try to DHCP off an interface if it finds out that there is no cable connected to it unless you wrote a configuration that asks you to do so. I wanted things to be as simple as possible, and therefore the minimal valid configuration file that will do something useful is the following:

That's right, it's empty. This will cause the system to bring up the 'lo' interface at bootup, and do nothing else. If you say 'ifup eth0', where eth0 is a valid and existing interface, it will first check whether there is a link, and if so, DHCP off of it. There's much to do still, but I have now reached a point where I feel the system is ready to meet all challenges it should be able to meet (or so I hope), and where it might be nice if interested people check it out. The code is in a git repository that's mirrored on git.debian.org, github.com, and my own git.grep.be. The latter is my --mirror repository, but is not available for cloning to anyone else. If you're interested in checking it out, you may want to start with doc/tech.txt, and/or examples/ipcfg.cfg. I updated the former just yesterday, so it's quite up-to-date. On the other hand, while the latter shows the direction I would like the system to take, the code is a far cry from implementing everything shown there, and it has been a while since I last checked that file out, so the comments in there are, in some cases, somewhat outdated. But that's good, because I want to keep the file as a reference of the goals that I originally had, before I'd written this whole corpus of code, so that I can go back at some undefined point in the future and fix things. Comments are (more than) welcome.

Modern daemon implementations can be run in a variety of ways, in a range of contexts. The daemon software itself can be a useful tool in environments where the associated traditional system service is neither needed nor desired. Unfortunately, common debian packaging practice has a tendency to conflate the two ideas, leading to some potentially nasty problems where only the tool itself is needed, but the system service is set up anyway. How would i fix it? i'd suggest that we make a distinction between packages that provide tools and packages that provide system services. A package that provides the system service foo would need to depend on a package that provides the tool foo. But the tool foo should be available through the package manager without setting up a system service automatically. Bad Examples Here are some examples of this class of problem i've seen recently:

akonadi-server depends on mysql-server

akonadi is a project to provide extensible, cross-desktop storage for PIM data. It is a dependency of many pieces of the modern KDE4 desktop. Its current implementation relies on a private instantiation of mysqld, executed directly by the end user whose desktop is running. This means that a sysadmin who installs a graphical calendar application suddenly now has (in addition to the user-instantiated local mysqld running as the akonadi backend) a full-blown system RDBMS service running and potentially consuming resources on her machine.Wouldn't it be better if the /usr/sbin/mysqld tool itself was distinct from the system service?

puppetmaster depends on puppet

Puppet is a powerful framework for configuration management. A managed host installs the puppet package, which invokes a puppetd service to reconfigure the managed host by talking to a centralized server on the network. The central host installs the the puppetmaster package, which sets up a system puppetmasterd service. puppetmaster depends on puppet to make use of some of the functionality available in the package. But this means that the central host now has puppetd running, and is being configured through the system itself! While some people may prefer to configure their all-powerful central host through the same configuration management system, this presents a nasty potential failure mode: if the configuration management goes awry and makes the managed nodes inaccessible, it could potentially take itself out too. Shouldn't the puppet tools be distinct from the puppetd system service?

monkeysphere Build-Depends: openssh-server

The Monkeysphere is a framework for managing SSH authentication through the OpenPGP Web-of-Trust (i'm one of the authors). To ensure that the package interacts properly with the OpenSSH implementation, the monkeysphere source ships with a series of test suites that exercise both sshd and ssh.This means that anyone trying to build the monkeysphere package must pull in openssh-server to satisfy the build-depends, thereby inadvertently starting up a potentially powerful network service on their build machine and maybe exposing it to remote access that they didn't intend. Wouldn't it be better if the /usr/sbin/sshd tool was available without starting up the ssh system service?

Good ExamplesHere are some examples of debian packaging that already understand and implement this distinction in some way:

apache2.2-bin is distinct from apache2-mpm-foo

Debian's apache packaging recently transitioned to split the apache tool into a separate package (apache2.2-bin) from the packages that provide an apache system service (apache2-mpm-foo). So apache can now be run by a regular user, for example as part of gnome-user-share.

git-core is distinct from git-daemon-run

git-core provides the git daemon subcommand, which is a tool capable of providing network access to a git repo. However, it does not set up a system service by default. The git-daemon-run package provides a way for an admin to quickly set up a "typical" system service to offer networked git access.

vblade is distinct from vblade-persist

vblade offers a simple, powerful utility to export a single file or block device as an AoE device. vblade-persist (disclosure: i wrote vblade-persist) provides a system service to configure exported devices, supervise them, keep them in service across system reboots, etc.

Tools are not Services: a ProposalLet's consider an existing foo package which currently provides:

• the tool itself (say, /usr/sbin/foo) and
• sets up a system service running foo, including stock foo daemon configuration in /etc/foo/*, startup scripts at /etc/init.d/foo, service configs at /etc/default/foo, etc.

I suggest that this should be split into two packages. foo-bin would contain the tool itself, and foo (which Depends: foo-bin) would contain the service configuration information, postinst scripts to set it up and start it, etc. This would mean that every instance of apt-get install foo in someone's notes would retain identical semantics to the past, but packages which need the tool (but not the service) can now depend on foo-bin instead, leaving the system with fewer resources consumed, and fewer unmanaged services running. For brand new packages (which don't have to account for a legacy of documentation which says apt-get install foo), i prefer the naming convention that the foo package includes the tool itself (after all, it does provide /usr/sbin/foo), and foo-service sets up a standard system service configured and running (and depends on foo). Side Effects This proposal would fix the problems noted above, but it would also offer some nice additional benefits. For example, it would make it easier to introduce an alternate system initialization process by creating alternate service packages (while leaving the tool package alone). Things like runit-services could become actual contenders for managing the running services, without colliding with the "stock" services installed by the bundled tool+service package. The ability to create alternate service packages would also mean that maintainers who prefer radically different configuration defaults could offer their service instantiations as distinct packages. For example, one system-wide foo daemon (foo-service) versus a separate instance of the foo daemon per user (foo-peruser-service) or triggering a daemon via inetd (foo-inetd-service). One negative side effect is that it adds some level of increased load on the package maintainers if the service package and the tool package both come from the same source package, then some work needs to be done to figure out how to split them. If the tool and service packages have separate sources (like vblade and vblade-persist) then some coordinating footwork needs to be done between the two packages when any incompatible changes happen. Questions? Disagreements? Next Steps? Do you disagree with this proposal? If so, why? Unfortunately (to my mind), Debian has a long history of packages which conflate tools with services. Policy section 9.3.2 can even be read as deliberately blurring the line (though i'd argue that a cautious reading suggests that my proposal is not in opposition to policy):

Packages that include daemons for system services should place scripts in /etc/init.d to start or stop services at boot time or during a change of runlevel.

I feel like this particular conflict must have been hashed out before at some level are there links to definitive discussion that i'm missing? Is there any reason we shouldn't push in general for this kind of distinction in debian daemon packages? Tags: daemons, packaging, policy

Writing a network configuration tool is... fun. Well, mostly anyway.

/***************************** SCANNING *****************************/
/*
 * This one behave quite differently from the others
 *
 * Note that we don't use the scanning capability of iwlib (functions
 * iw_process_scan() and iw_scan()). The main reason is that
 * iw_process_scan() return only a subset of the scan data to the caller,
 * for example custom elements and bitrates are ommited. Here, we
 * do the complete job...
 */
/*------------------------------------------------------------------*/

(from iwlist.c) Not. The. Way. To. Do. It. Especially not if you're the person who writes both libiw and iwlist. Grmbl. Also, documentation for libiw seems to be, well, nonexistent. Fun. Not.

Order Without Law is a fantastic book by Robert Ellickson published in 1991. In a way, the book is an in-depth case study of the irrelevance of law. Subtitled, "how neighbors settle disputes," Ellickson shows how people solve complicated problems in an archetypal area of liability law without knowledge of the law. Ellickson shows that even when people know exactly what the law says, they often ignore it in favor of community norms and, in his examples, models of "neighborliness." Specifically, the book is about how neighbors in northern California settle disputes related to damage caused by roaming cattle, how neighbors construct and share costs of fences, and how, although the law is frequently debated in relation to classifying land as either open or closed to free grazing, the law tends to take a back seat to unwritten norms in the way that problems are actually solved. There is order and a shared understanding of rules in Ellickson's account; it just has very little to do with the law. As part of his description, Ellickson goes into some detail about the types of damage that cattle cause. For example, Ellickson describes how bulls, will often break down barbed wire fences and go on rampages impregnating heifers, eating vast amounts of food, and destroying crops and equipment. When I was on a long bike trip through northern California a few weeks ago, I was sitting under a tree waiting for my cyling partner to catch up. A giant black bull nearby noticed me. Shorting, mooing, and shaking ribbons of slobber from its mouth, it lumbered toward me. Only a rather weak-looking barbed wire fence separated us. All I could think about was Order Without Law.

Over the last few years, I've been pretty immersed in the world of wikis. A wiki is like a collaborative web site; it lets several people view and edit a web page and link to other pages, maybe new pages that they themselves create. Wikis let tech-savvy people all over the world collaborate on a single web page, a single document. That's a pretty powerful idea, really. The obvious thing to do once an idea like wikis appears is to attempt to cram all of humanity's knowledge into it. Well that's just crazy enough to work. Wikipedia is the gold standard of wikis. It's a collaborative encyclopedia with lots and lots of articles. As the gold standard, it influences people's thinking about wikis in ways that might not be quite true. If Wikipedia were the only wiki you'd ever heard of, you might think that wikis are used to create encyclopedias. If that were true, you'd really only need one of them. If Wikipedia were the only wiki you'd ever heard of, you might be under the false impression that all wikis everywhere are editable by anyone. So if you create a wiki page, it's no longer yours. Other people can scribble on it. Well, some kinds of documents perhaps just aren't suitable for being edited by anyone, and the good news is that you can build a wiki for you and your classmates, or you and your coworkers, and you can decide who gets to edit what. I want to think about a few distinct uses for wikis in different contexts. The most common use is of course to create a knowledge repository; a collection of information written by people who know what they're talking about. That's what Wikipedia is, mostly. That's what the Haskell Prime wiki (which I run) is. And the excellent wiki about the game of Go, Sensei's Library, is also a knowledge repository. But there are other uses for wikis that are pretty cool. I often have to work on proposals with my coworkers. One (horrible) way to do this is to write the first draft in MS Word, email it to everyone, who might edit it and email it back. Of course, only one person can actually edit it at a time, otherwise I have to figure out what everyone did and merge their changes by hand. That's what we, in the computer industry, call teh suck. A better way to do this is to use a version control system, which lets multiple people work on a document at one time. Of course, if you're editing evil document formats like Word, still only one person can edit it at a time, but at least you cut out email, so things are a bit better organized. But wikis are actually a really great way to work on a document with another person. Two or more people can each collaborate on the document using the wiki software over the web. What becomes of the document at that point depends on its ultimate use. Maybe you have to convert it to MS Word or PDF at the end. That's a bit of a bummer. So there are two uses of wikis that I've mentioned: building a knowledge repository, and collaborative writing. Google started a service for collaborating on documents called Google Docs & Spreadsheets, which is probably useful for collaborative writing, but not knowledge repositories. They also acquired a wiki startup a few days ago, so they are clearly trying to get all over this space. There's another very interesting use of wikis that I'll call situation awareness. You can get a really vivid picture of this use of wikis if you take a look at the wikipedia article on the July 2005 London bombings. By the way, I didn't discover this excellent example of situation awareness using wikis. I saw it in the slides for a talk, but I don't know the original source at this point. If you go to that link, you'll see an excellent and up-to-date encyclopedia article about the London bombings. It's not a news article. Go to Google News and try to find a news article from a major news source which is this detailed, up-to-date, and well organized. Most articles you'll find are a snapshot, taken at a particular point in time (when the article was published) that most likely contains an account of the latest developments, and usually brief background in case you missed previous stories. The wikipedia article isn't a snapshot, it's an article that explains things in a logical order, which isn't necessarily chronological. Each wikipedia article contains a link to its own history. You can look back over time and see the article evolve. If you look at the early edit history of that London bombing article, you'll notice something fascinating. The article was created shortly after the bombing, with an account of the information that was known at that time. Within an hour, many people had edited the article, adding information they know about. If you were refreshing that wikipedia entry over & over (or if you were subscribed to it using RSS), you'd see not an account of just the latest developments, but an evolving, logical understanding of the attacks. The article started evolving right away, and it continues to evolve to this moment. That's what I'm calling situation awareness. Don't get me wrong, the mainstream news media is quite good at situation awareness. They do it a bit differently, and their articles or TV stories don't become integrated into a knowledge repository like wikipedia. I've enumerated three somewhat overlapping uses for wikis that I think are pretty interesting to examine:

Knowledge Repositories, which are a lot like encyclopedias,

Collaborative writing, which is a bit like a more elegant version of emailing word documents, and

Situation awareness, which isn't so different from the news; it draws from the news as a source, it can also be authored by the eye-witnesses themselves, and each story becomes an integral part of a knowledge repository. I don't claim that these are an exhaustive, nor are they a partition: Wikis are also used for bug tracking and as web discussion forums, for instance. What other categories can you think of? Email me if you have ideas.

June is the last month before summer holidays, and at this phase in my study, courses take a lot of time: practical assignments, seminars to give, multiple exams. And mid-June, I'll be attending DebConf in Edinburgh, including delivering a talk about Mole, the subject of my Google Summer of Code project. In the period of April/May students participating in the Summer of Code project had the time to get more familiar with the project they were going to work for. Because I'm already somewhat familiar with Debian, I opted to organize an Etch release party together with some friends, because, well, what better way is there to get to know the community? During the past two weeks I've been really starting with the project. I updated the wiki page, added a new development page where I keep track of my work listing the subtasks I'm working on and their status. The first month will be mostly work on the internals, but starting with the second month I'll also be working on the web interface, meaning that there should be some stuff to show off. At the moment I'm refactoring some bits of Mole that are currently implemented in a hacky way, in order to make it all as extensible as it should be. This includes defining a config file format for mole tables, so that it's no longer necessary to modify the python code for new tables. It also involves some better way of stacking tables to each other, so that the package file extraction code can be moved away from the core of mole into a separate worker to keep mole itself pure and simple and generally applicable. I'll post irregular updates to my blog. I'm looking forward to seeing a lot of you again this Thursday, when I'll arrive in EDI!

The Debian Installer has a USB boot image that is supposed to be written raw to the USB stick. Problem is that means the image has a fixed size, 256 MB, and all the other gigabytes on your typical USB stick are unusable after writing this image to it. It's a bit of a cheat for me to blog about this, since I'm slighly responsible for the thing that I'm working around. But still. The real problem is not the fixed size of the image, but that it is written to the USB stick as a "super floppy", without any partition table. If there were a partition table, you could make a second partition in the rest of the space of the USB stick, and use it to store files, etc. So, let's set that up. It's not really hard:

1. Use your favorite partition tool to partition the USB stick. Make a 256 MB DOS partition, that will be used to hold the installer, and set it bootable. Make a second, or more partitions for your own use.
2. Unplug and replug the USB stick so the device files show up, and write the installer image to the first partition (/dev/sda1 in this example). zcat boot.img >/dev/sda1
3. Make the USB stick bootable by installing the mbr package, and running install-mbr /dev/sda
4. Copy a Debian ISO image to the USB stick somewhere. You can put it on the first partition, as ususal, or if you prefer, can put it on one of the later, larger partitions. That's useful if you want to use a larger ISO than the netinst. The installer should find it anywhere.

I m still working with SLHA files and noticed that people are using C/C++, Fortran, or also Mathematica programs for batch manipulation and processing of SLHA files (for example for repeatedly increasing some parameters in a SLHA file, feeding this file to some program and reading its output). For someone experienced with the shell (no, not that shell) this sounds like an easy task doable in a few lines of shell code. There is however no simple way to manipulate SLHA files on the command line.
The script I m presenting in this post, slha-io.sh, tries to close that gap. Its description is:

This script is a very simple (and stupid) tool to get or set data in SLHA (SUSY Les Houches Accord) files. It can output complete blocks or individual data lines and replace existing data lines with user given input. It is stupid because it does not check after replacing data lines if the resultant file conforms to the SLHA. So care must be taken when using the set action.

But as so often, it is best described by examples. The basic things you can do with slha-io.sh are (MSSM.in is the SLHA file in these examples):

$ slha-io.sh MSSM.in get modsel
Block MODSEL            # Select model
 1    0                 # MSSM
 4    1                 # R-parity is violated
Block SMINPUTS          # Standard Model inputs
$ slha-io.sh MSSM.in get sminputs 6
 6   1.729000E+02       # m_top(pole)
$ slha-io.sh MSSM.in get sminputs 6   awk '  print $2  '
1.729000E+02
$ slha-io.sh MSSM.in set sminputs 6 \
  " 6   1.730000E+02       # m_top(pole)"
$ slha-io.sh MSSM.in get sminputs 6
 6   1.730000E+02       # m_top(pole)

Here is another example of using slha-io.sh to create a batch of files where a parameter is constantly increased from 70 to 80 in steps of 0.5:

for NUM in  seq -f "%.8E" 70 0.5 80 ; do
    FILE = "m0_$NUM.in"
    cp MSSM.in "$FILE"
    slha-io.sh "$FILE" set minpar 1 " 1  $NUM  # m0"
done

Maybe this script is useful for one or two! By the way, more sophisticated tools for SLHA I/O are SLHALib and SLHAio.

I always wanted to write a short howto for Debian package maintainers, how to write a manual page in GROFF. This is useful for short documents (for longer documents, docbook/docbook-xsl can be a good choice). I already have written some parts. However, here is the first part of hints. More might follow later. The delimiter in the NAME section The name section is (probably) the only place, where exactly one hypen-minus \- must appear. The hyphen-minus is the delimiter between the command name and one-line description.

.SH NAME
foo-bar \- foos the template foo-what-bar

Typical mistakes regarding paragraphs Using a .PP macro directly following a .SH or .SS macro is useless. This macro should be used between paragraphs:

.SH OPTIONS
The program follows the usual GNU command line syntax, with long options starting with
two dashes ( \-').
.PP
A summary of options is included below.

Options/File descriptions To describe options or files it s usually useful to make use of the .TP macro.

.SH OPTIONS
.TP
.B \-f, \-\-force
Force the execution of the specified command.

.SH FILES
.TP
.I ~/.foobar
Per user configuration file.

To create more than one intended paragraph the .IP macro can be used.

.TP
.B \-f, \-\-force
Force the execution of the specified command.
.IP
This option has no effect in conjunction with \fB\-\-foo\fP.

Avoid hyphenation in URLs/URIs/paths Usually we don t want URLs or URIs to be hyphenated. This can be done using the \% sequence. Typical examples:

A short tutorial is available online at \fI\%http://foo.tld/some/path/here/manual.html\fP.

On a Debian system the complete text of the GNU General Public License
version 2 can be found in the file \fI\%/usr/share/common-licenses/GPL\-2\fP.

Referencing persons and their mail address The common markup is to write the person name in bold letters and the mail address in roman letters is put into angle brackets. It s usually a good idea to mark where the mail address starts and ends. We can use the \& sequence as shown in the following example:

\fBDaniel Leidert\fP <\&daniel.leidert@wgdd.de\&>

I released a new 0.9.7.2 version of DAViCal yesterday. This reflects quite a lot of stability and small fixes for some subtle problems, and quite a lot of work with the iPhone, adding the possibility of a simpler configuration experience for iPhone users. One long-standing, but less obvious problem (well, less obvious to me) was the handling of usernames with odd characters in them. In the round-trip these needed to be urlencoded, and that wasn't happening correctly in all places, but it hopefully is now. People who want to use e-mail addresses in usernames, or spaces in calendar names, or both, etc., should all work now. One newer problem I've been seeing the effects of a little recently is that PHP 5.3 deprecates call-time pass by reference, which I had used in a few places. That's fixed now, so it should be OK to upgrade to PHP 5.3 without breaking DAViCal or having to explicitly turn those warnings off. The iPhone is a curious beast. Lots of people love them, and it is about the only mobile device that supports CalDAV (though I'm hoping the Nokia N900 also will), so it's important to support it as transparently as possible regardless of my personal loathing for it's horribly locked down state. It was pointed out to me that it is possible for it to be configured by entering three pieces of data (domain name, username, password) so I decided to make DAViCal support that process. In order for it to work for you you will need to have DAViCal running on an SSL server on port 8443, which isn't too big a deal. You can also plain http on 8008 which it will also attempt after prompting when the SSL connection fails. You'll also have to enable mod_rewrite and use some rewriting rules like the example on the wiki. For myself, I use the free SSL certificates from CACert.Org so I had to browse to their root certificate page and load the class 1 and class 3 certs into the iPhone before it would talk happily to my DAViCal server. I'm hoping to continue with reasonably frequent incremental releases of DAViCal for the rest of the year, hopefully with 0.9.7.4 sometime in late October. What will be included there might also depend on how productive my visit to CalConnect XVI is in three weeks too, and I'm looking forward to that.

For those who aren t clinically dead, you may have heard of the BBC . The BBC are the state-ish-funded TV network in the UK, and the country s biggest broadcaster, alongside three other major terrestrial broadcasters who make their content widely available without payment ITV, Channel 4, and Five. These broadcasters also make some or all of their programming available for streaming over the Internets usually their home-grown programming only, not licensed stuff from America. Now, once upon a time, the online functionality was mostly offered via a proprietary P2P-and-DRM-based system called Kontiki. Kontiki was unpopular for various reasons for example, it was Windows-only, and banned by several Internet providers due to the use of P2P (e.g. I know such technology is still banned on the University of Oxford network). As a result, this lead to the introduction and eventual replacement of browser-based streaming solutions, starting with the changes made to BBC s iPlayer. iPlayer is nowadays a combination of an Adobe Flash service for web surfers, an unencrypted MP4 streaming service for users of mobile devices such as iPhones, and as of a couple of weeks ago, has streaming support built directly into the PlayStation 3 s user interface too. Oh, and on the PC, it supports DRM-based downloading courtesy of the Adobe Air platform, on Windows, Mac, and Air-capable (x86) Linux. Channel 4 s service 4od, and Five s Demand Five, are also based on Adobe Flash, and streaming-only. They re basic and functional. The final interesting one here is ITV s ITV Player. ITV Player was, until about a week ago, the only non-Flash service, instead making use of Microsoft Silverlight. They have now changed to be Flash-based, like their peers, meaning the entire market mandates use of the Adobe Flash plugin or, at a push, command-line utilities or browser plugins which grab the raw video files from the broadcasters servers, in violation of their licenses. Why is this interesting? Well, when ITV Player used Silverlight, we could watch TV using Free Software: Novell Moonlight 2.0 Beta 1.1 and above worked fine with it, on i386 and AMD64 (and other architectures with a recompile). Now that it s using Flash rather than Silverlight, where do we stand for watching streaming TV legitimately with Free Software? Let s take a peek! Firstly, some preamble. I m running Ubuntu Jaunty, and I hand-compiled a SVN (I think SVN? Maybe Bzr) snapshot of Gnash revision 11485 to ensure I had an up-to-date view of proceedings. It definitely seems to be working, as I m introduced to the world of Flash advertising via Gnash, and Youtube.com also works, more or less. And I m not discussing Adobe s Flash plugin here, for various reasons:

• It s proprietary. Boo.
• It s i386-only in several instances a beta of an AMD64 plugin exists, but your browser will crash on BBC iPlayer sites if you have the iPlayer Adobe Air app installed, as Air is i386-only. And the packages in Ubuntu install the i386 plugin alongside the abominable and crashy nsPluginWrapper.
• Did I mention it was proprietary? If the only way to surf the web on a Free OS is with proprietary software, then we re not doing very well.

So, Gnash it is. Firstly, it s a PITA to compile, as upstream seem to have misunderstood how AutoFoo works i.e. it detects that you don t have headers installed for certain features, and tells you so, yet still enables those features at configure-time (and obviously fails when building). Hey guys, if I don t have qapplication.h, take the hint and disable KDE support for me like every other bloody app does. With that out of the way, let s get to it! BBC iPlayer Well, what does the landing look like? Looks fine to me! Let s try playing a show! Oh. Um Never mind, then. Seems iPlayer s JavaScript to detect Flash presence doesn t pick up on Gnash, and it bails out. Score so far: 0/1 Demand Five Another former Kontiki partner, how do these guys fare? Hey, that looks pretty good to me! Perhaps we re onto a winner this time? Poot. Gnash is definitely being invoked see the context menu there but it sure isn t doing anything useful. Score: 0/2 Channel 4 4od How do those hip cats at Channel 4 fare? Hm Gotta confess, not feeling too hopeful about this one Oh, well, even worse than iPlayer. What you folks might not recognise is the missing Play buttons which are supposed to be to the right of the program descriptions in that list of episodes. Presumably more Javascript/Flash interop failure. 0/3 ITV Player Last, but not least, how does the newest entrant into Microsoft-free streaming fare? Now, now, in fairness, it s always been this bad, even when they used Silverlight weirdos that they are, ITV have always used Flash for their navigation, even when they used Silverlight for playback. The Flash-based navigation you (don t) see here is barely any better even with the proprietary Flash plugin. What the proprietary plugin does NOT do, however, is consume the 750 or so meg of RAM that Gnash did when sat idle on this screen. I mean, that s not a problem, that s why I have 6 gig in here, but still, not wise on Wifey s netbook. Oh. Well then. 0/4 it is. So? What annoys me here isn t so much that nothing works. I m used to there being temporary gaps in Free Software functionality, that s pretty normal. But I m greatly vexed that one of these four used to work on a Free platform, and now it doesn t and that places like UbuntuForums are filled with people celebrating that fact. Celebrating that ITV have stopped using the evil nasty Microsoft system which happened to have a functional Free replacement and that they ve now moved to a non-Microsoft system which mandates a proprietary plugin. It s not the first time either Major League Baseball in the USA used to use Silverlight for their HD streaming, and now they use a combination of Flash and a proprietary Windows-only extension to Flash to make it load WMV files this is considered a victory. It really isn t. We should NEVER be forced to use Proprietary software in order to surf the web yet now we ve gone from having access to 25% of the UK s streaming TV services via Free Software to 0% of them, and people are happy about it. So, I want you to think long and hard about this question, one which seems to get the oddball answer far too often: What is more important promoting Free Software, or demoting Microsoft Corp?