The brand new machine for Ultimate Debian Database motivated me to do UDD-related work again, so I implemented an old idea: build a maintainer/team-centric dashboard relying on UDD, the Debian Maintainer Dashboard. The idea is to expose as much useful information as possible about a maintainer s packages, both in the traditional (Developers Packages Overview-like) big table with all the info , but also in a task-oriented listing (to answer the ok, I have two hours for Debian, what should I do now? question). At this point, the Debian Maintainer Dashboard already brings several cool features (try it with my packages or pkg-ruby s packages, or your own!):

• Reports buildd issues (failed builds)
• Upstream status monitor that works (UDD has its own)
• Knows about teams VCS (if your team uses PET) and display the current version in the VCS
• Lists packages you maintain or co-maintain, and also the package you showed interest for by subscribing to them on the PTS

Of course, as of now, it s pretty ugly: that s the help needed part of the post. I really suck at web design, and would really appreciate if someone would help me to turn it into something nice to use. Besides adding links and colors everywhere, it could probably be nice to add sortable tables, and to create tabs for TODO items , Versions and Bugs (and possibly Lintian, for example). Contact me if you are interested.

Lately (as I earlier pointed out on my blog) I ve been working on improving GNOME s infrastructure monitoring services. After configuring XMPP it was time to find out a good way for sending out relevant notifications to our IRC channel hosted on GIMPNET. I achieved that with a nice combo: supybot + supybot-notify, all that mixed up with a few grains of Nagios command definitions. But here we go with a little step-by-step guide: Requirements 1. Install supybot and configure a new installation:

apt-get install supybot or yum install supybot
mkdir /home/$user/nagbot && cd /home/$user/nagbot
supybot-wizard (follow the directions to get the bot initially configured)

2. Install and load the supybot-notify plugin by doing:

git clone git://git.fedorahosted.org/supybot-notify.git && cd supybot-notify
mkdir -p /home/$user/nagbot/plugins/notify && cp -r * /home/$user/nagbot/plugins/notify

Finally, load the plugin. (this will require you to authenticate to the bot) Nagios configuration 1. Add the relevant command definitions to the commands.cfg file:

# 'notify-by-ircbot' command definition
define command 
    command_name    notify-by-ircbot
    command_line    /usr/bin/printf "%b" "#channel $NOTIFICATIONTYPE$ - $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$: $SERVICEOUTPUT$ ($$(hostname -s))"   nc -w 1 localhost 5050
     
# 'host-notify-by-ircbot' command definition
define command 
	command_name	host-notify-by-ircbot
	command_line	/usr/bin/printf "%b" "#channel $NOTIFICATIONTYPE$ - $HOSTALIAS$ is $HOSTSTATE$: $HOSTOUTPUT$ ($$(hostname -s))"   nc -w 1 localhost 5050
	 

* adjust the Netcat s host and port to your needs, in my case Supybot and Nagios were running on the same host. In the case of a Supybot running on a different host than Nagios, tweak Iptables to allow the desired port:

-A INPUT -m state --state NEW,ESTABLISHED -m tcp -p tcp --dport 5050 -j ACCEPT

2. Add a new entry on the contacts.cfg file:

define contact 
 	contact_name    nagbot
   	use             generic-contact
        alias		Nagios IRC Bot
        email           example@example.com
        service_notification_commands   notify-by-ircbot
 	host_notification_commands      host-notify-by-ircbot
 

3. Reload Nagios:

sudo /etc/init.d/nagios3 reload

And finally, enjoy the result:

 PROBLEM - $hostalias/load average is CRITICAL: CRITICAL - load average: 30.45, 16.24, 7.16 (nagioshost)
 RECOVERY - $hostalias/load average is OK: OK - load average: 0.06, 0.60, 3.65 (nagioshost)

It's been over a week since I publicly announced the Debian mirrors redirector.
It was covered in several places, including Linux Magazin and mentioned in the latest Debian Project News.

Thanks to all that, and quite a number of early testers of the wheezy image for the Raspberry Pi, http.debian.net has processed over one million requests from APT clients alone. This figure doesn't count users of apt-cacher/apt-cacher-ng/approx, or those who happen to be mirroring Debian with wget. Oh, and this is only the number of requests since last Sunday, when the logs were rotated.

By the way, if you are going to Debconf12 in Nicaragua, why don't you give http.debian.net a try and forget about switching mirrors?

I'm happy to see how it was welcomed and the many emails, blog comments, IRC messages, and other sorts of messages I've received about it. There are surely some rough edges but the service is still under development and all the feedback helps.

In some cases there appear to be some confusion as to how exactly it chooses the mirrors and why sometimes some mirrors are not used. I'll try to cover some of those topics in this blog, hoping the

P.S. If you've sent me an email about http.d.n and I haven't replied, please rest assured I try to take a quick look at them but I can't reply to every single message fast enough.

Thanks to everyone!

Whilst nobody was looking, Mono became the most important framework in the world of games. And I mean that with only a smidge more than my usual levels of hyperbole. It really is fantastically important, as far as game development goes. I blogged a long time ago about its use inside EA s multi-million-selling The Sims 3. And the massive performance gains in Second Life when they switched from their in-house scripting language to Mono are well documented. But I m not talking about those, I m talking about the now, the recent stuff. First up is Unity 3D. Not to be confused with the Ubuntu user interface, Unity 3D is a proprietary game development tool which began life for Mac, and has now managed to become the single most popular game engine amongst smartphone developers. It targets a bunch of new platforms not just Android and Windows and iOS, but in the new 4.0 release, Linux too for the first time. And every Unity 3D game is a Mono game the core of Unity is a fork of Mono, which enables it to have the performance and ease of development which it does. A reasonably large percentage of the cross-platform games on Kickstarter, such as Wasteland 2 and Super Retro Squad, are cross-platform because Unity 3D makes it easy and each of those games is powered by Mono. However, I have better things to do today than gush about a proprietary tool. No, far more important are the Free ways to develop games. So today I m going to talk about MonoGame. MonoGame is an implementation of Microsoft XNA a high level framework for developing in .NET languages, originally designed to allow indie developers access to the Xbox 360. XNA covers many of the bases covered by SDL sound, graphics, input, and so on, but can go onto platforms where SDL isn t an option, such as the aforementioned Xbox 360. And with MonoGame, any game written for XNA is a game which can not only run on Microsoft s approved platforms Windows, Xbox 360, Windows Phone but also on other platforms such as Mac OS X, iPhone, iPad, Android, and Linux. Cross-platform is good. More games are good. And MonoGame enables every Xbox 360 indie developer to throw a Linux port out of the door with a relative minimum of fuss. There are 2,521 games on the Xbox Live Indie Games store (games by individuals or small groups, with no major publisher) and each of those can become a game for Debian and Ubuntu with very little work. Not to mention the games with actual publishers (some portion of the 492 games on XBLA), not to mention the mobile games written for Windows Phone, Android and iOS. All of these developers have an easy route to Linux releases, as long as the effort remains low. And with today s publishing in the Debian archive of MonoGame 2.5.1, the effort is as low as can be. Anyone who purchased the recent Humble Indie Bundle 5, and played Bastion, has seen MonoGame in action Bastion began life as an XNA game for Xbox 360, and the Linux port was made possible only by MonoGame. Bastion s 100+ industry awards should indicate the possibilities open to a creative mind with decent developer tools. MonoGame is Free Software, built on top of other Free Software, with an active upstream development team genuinely responsive to the needs of Linux distributions. It s well worth a look not just for porting, but for new projects too. And I m delighted that it will be available to all users of Debian 7.0, as well as Ubuntu 12.10.

The bug webinterface of the Ultimate Debian Database currently knows about the following release critical bugs:

In Total:	1088
Affecting Wheezy:	697
Wheezy only:	128
Remaining to be fixed in Wheezy:	569

Of these 569 bugs, the following tags are set:

Pending in Wheezy:	21
Patched in Wheezy:	77
Duplicates in Wheezy:	57
Can be fixed in a security Update:	22
Contrib or non-free in Wheezy:	8
Claimed in Wheezy:	1
Delayed in Wheezy:	2
Otherwise fixed in Wheezy:	44

Ignoring all the above (multiple tags possible) 375 bugs need to be fixed by Debian Contributors to get Debian 7.0 Wheezy released. However, with the view of the Release Managers, 603 need to be dealt with for the release to happen. Please see Interpreting the release critical bug statistics for an explanation of the different numbers.

OASIS is a tool to integrate a configure, build and install system in your OCaml project. It helps to create standard entry points in your build system and allows external tools to analyse your project easily. It is the building brick of OASIS-DB, a CPAN for OCaml. This release takes almost 18 months to complete. This is too long and I will talk in another blog post on the way I am trying to improve this right now (esp. using continuous integration). This new release fixes a small bug (1 line) that prevents setup.ml to run with OCaml 4.00. If you have projects that was generated with a former release consider upgrading to OASIS 0.3.0. There are several big new stuff that comes with this release. It now supports Pack: field for libraries which allows to pack your library using -for-pack and so on. We are also compiling .cmxs (dynlink object for native) by default and we publish them in the META file. This feature was implemented in order to get more libraries to provide .cmxs and to help project like Ocsigen to take advantage of this. If you want to get rid of this at configure time, you can use ./configure --override native_dynlink false. We introduce two new default flags: --enable-tests and --disable-docs for configure. These are implicit flags that define if we will run Test sections or compile Document sections. They are especially useful to reduce the number of dependencies, because dependencies of Test will be excluded by default. We recommend to set Build$: flag(tests) to any Library or Executable sections that are only useful for tests. This allows you to really cut down the number of dependencies. The last change I want to introduce is about the old setup-dev subcommand which is now deprecated. It has been replaced by 2 different update schemes. I am pretty excited by this feature which in fact comes from OASIS user (esp. Lwt project). The former scheme was to have a big 'setup.ml' that always call the command oasis to update. This was complex and not very useful. We now have 2 mode: dynamic and weak. dynamic allow you to have a small setup.ml and to keep your VCS history clean but you need to install OASIS. weak need a big setup.ml but only need to call the command oasis if someone change something in _oasis. This mode is targeted to project that wants to be able to checkout from VCS an OCaml project without installing OASIS. The difference generated by weak mode doesn't pollute too much the VCS history because most the time they make sense. For example, you upgrade your package version number in _oasis and it produces a change with 6 lines where the version number changes in every META, setup.ml files and so on. This version has been tested on:

• Linux (Debian Sid and Debian Squeeze with GODI, OCaml 3.12)
• Windows Vista with MS Visual Studio 2008 toolchain, OCaml 3.12
• Mac OS X, OCaml 3.12 from GODI

You can download it here or use your favorite package manager: Debian (UPDATE: pending upload)

$ apt-get install -t experimental oasis

GODI

$ godi_console perform -build apps-oasis

odb.ml

$ ocaml odb.ml oasis

Here is the complete changelog: EXTREMLY IMPORTANT changes (read this)

• Fix bug with scanf %S@\n for ocaml 4.00. We were unfortunetaly using an undocumented tolerance of Scanf in the previous version. You should consider making new release using this version that fixed this.

PACKAGES uploaded to oasis-db will be automatically "derived" before OCaml 4.00 release (i.e. oUnit v1.1.1 will be regenerated with this new version as oUnit v1.1.1~oasis1). PACKAGES not uploaded to oasis-db need to be regenerated. In order not to break 3rd party tools that consider a tarball constant, I recommend to create a new version. Thanks to INRIA OCaml team for synchronizing with us on this point. Major changes:

• Handle the field Pack: true to be able to create packed libraries. It also installs .mli files for documentation into the target directory. The pack option is only supported for OASISFormat: 0.3, you will need to update the version of your _oasis file to match it.

• Introduce --enable-tests to disable tests and docs at oasis level. It seems a very common pattern to have a Flag tests to turn off by default the tests. This is now define as a standard var and you should remove you previous Flag tests but you can continue to use flag(tests) where needed.

You can now have the following example:

     ...
     Executable test_exec
       Install: false
       Build$: flag(tests)
       MainIs: testExec.ml
       BuildDepends: oUnit
     
     Test main
       Command: $test_exec
       TestTools: test_exec
     ...

The Run$: flag(tests) is implicit for the section Test main. The default value is false for tests. If all the executable for test are flagged correctly (Build$: flag(tests)), you'll get rid of the dependency on oUnit. It works the same for documentation, however the default is true. (Closes: #866)

• Allow to define interdependent flags

In order to allow interdependent flags, we transform back lazy values into unit -> string functions. This allows to change a flag value on the command line and to update all the dependent values. (Closes: #827, #938)

• Deprecate the subcommand oasis setup-dev in favor of oasis setup -setup-update none weak dynamic

It defines different ways to manage the auto-update of setup.ml:

• none: this is the default mode, and the one you should use when distributing tarballs. No update are performed at al.
• weak: the update is only triggered when something change in _oasis , we keep all files generated
• dynamic: the content of 'setup.ml' is ultra small (<2kB) and we only keep a small setup.ml, Makefile and configure.

The choice between weak and dynamic depends on your need with regard to VCS and to the presence of oasis. The weak allow to checkout the project from VCS and be able to work on it, without the need of installing 'oasis' as long as you don't change the file _oasis. But it clutter your VCS history with changes to the build system each time you change something in _oasis. The 'dynamic' mode gives you no VCS history pollution but makes mandatory to have installed oasis libraries.

• Don't copy executable in ocamlbuild

Avoid copying executable to their real name. It helps to call ocamlbuild a single time for the whole build rather than calling it n time (n = number of executable sections) and copying resulting exec. This speeds up the build process because ocamlbuild doesn't have to compute/scan dependencies each time. The drawback is that you have to use $foo when you want to call Executable foo, because $foo will be '_build/.../main.byte'.

• Change the way we parse command line like option (CCOpt, CCLib and the like). We have implemented a real POSIX command line parser, except that variables are processed by Buffer.add_substitute (except if correctly escaped, using Buffer.add_substitute escaping).

For example:

CCOpt: -DEXTERNAL_EXP10 -L/sw/lib "-framework vecLib"

Will be parsed correctly and outputed according to target OS.

• Minimize the dependencies of the project.

In order to ease building oasis, we have minimize the number of dependencies. You only need to install ocamlmod, ocamlify and ocaml-data-notation for a standard build without tests. Dependencies on pcre, extlib and ocamlgraph has been dropped. The remaining dependencies are hidden behind a flag tests.

• Implement proposal for handling plugin in META, using "plugin" extra directives

OASIS now produces .cmxs file by default and add them to META. Now a META looks like:

     ...
     archive(byte) = "oasis.cma"
     archive(byte, plugin) = "oasis.cma"
     archive(native) = "oasis.cmxa"
     archive(native, plugin) = "oasis.cmxs"
     ...

This will ultimately help to generate automatically .cmxs for all oasis enabled projects. We hope that this new feature will improve dynamic linking use in OCaml (esp. for project like Ocsigen). Other changes

• Produce only subcommands specific helps when asked (Closes: #783)
• Use pager for long textual output (Closes: #782)
• Give the help summary directly when running just "oasis" (Closes: #781)
• Remove pwd filename prefix to avoid too long command line on Windows (Closes: #1007)
• Install required .cmx when installing libraries (Closes: #686, #885)
• Retain blanks at the beginning of line in freeform (Closes: #811)
• Allow PreConfCommand to create setup.data with any variables inside (Closes: #823)
• Allow to use setup.data in shell script and Makefile (Closes: #982)
• Fix execute permissions (Closes: #980)
• Drop extra version in ocaml version (Closes: #964, #888)
• Always output info messages on stderr (Closes: #785)
• Fix interversion in version comparator lexer (Closes: #808)
• Document the fact that setup.data is mandatory (Closes: #946)
• configure translates getopt command line to Arg (Closes: #778, #804)
• Generate valid META when there are line breaks in synopsis (Closes: #933)
• Keep owner of generated files (Closes: #793)
• Set -I +threads for ocamlfind ocamldoc in ocamlbuild (Closes: #949)
• Exclude VCS directories by default (Closes: #847)
• Print the configure summary in the "normal" order.
• Merge findlib virtual containers with the same name
• Synchronize plugin data for section
• Improve documentation
• Verify validity of input when parsing in quickstart (Closes: #797)
• Fix stack overflow when there are twice the same library (Closes: #1130)
• Parse license examples for DEP5 specification (Closes: #1102)
• Be ultra-liberal regarding version syntax (allow '_' and ' ' in the version).
• Allow to add extra content to META files (XMETAExtraLines)
• Allow to use executable name that contains "-" as a variable (replacing "-" by "_").

Thanks to Anil Madhavapeddy, Pierre Chambart, Christophe Troestler, Jeremie Dimino, Ronan Le Hy, Yaron Minsky and Till Varoquaux for their help with this release. Also thanks to all the testers of the numerous release candidates. This was a long work and each time a tester has downloaded oasis has helped me to know that I was working for someone.

Today, OFCOM issued a press release entitled "New measures to protect online copyright and inform consumers". According to the announcement, internet users will be encouraged to download music and films through legal channels under the measures. Ofcom has published a draft code for consultation that would require large internet service providers (ISPs) to inform customers of allegations that their internet connection has been used to infringe copyright. There does seem to have been a number of concessions made, which is welcome, but I do still have a number of concerns. The code 'only' covers ISPs with more than 400k fixed lines - BT, Everything Everywhere, O2, Sky, TalkTalk Group and Virgin Media. This covers about 93% of the UK retail market. If an allegation of copyright infringement is made, the ISP would need to write to the customer with notice that an infringement has occurred, and in a new change, the number of copyright infringement reports connected to their account. If more than three letters are received in a year, anonymous information may be provided on request to copyright owners showing them which infringement reports are linked to that customer s account. The copyright owner may then seek a court order requiring the ISP to reveal the identity of the customer. Two more changes have been implemented:

• Copyright owners procedures for gathering evidence of infringement must now be approved by Ofcom, rather than by the copyright owners themselves.
• Ofcom has decided that subscribers should have 20 working days to appeal an allegation of infringement, but only on grounds specified in the Digital Economy Act.

Obviously the former of these will require further scrutiny, but I'm happier that this is not with the vested interest themselves. As for the latter, this gets a bit interesting. The Act states that

an appeal on any grounds must be determined in favour of the subscriber unless the copyright owner or internet service provider shows that a) the apparent infringement was an infringement of copyright, and b) the report relates to the subscriber s IP address at the time of that infringement.

This is good as it puts the burden of proof on the copyright owner, although it still uses IP addresses which is problematic. For example, New York Judge Gary Brown has ruled IP addresses are insufficient evidence to identify pirates, and has provided a lengthy and thoughtful explanation in his ruling. There has also been a consultation published on the how to share costs costs. The suggestion is fairly simple:

"Copyright Owners should bear all of the costs incurred by Ofcom, the majority of costs incurred by the appeals body, and 75% of the costs efficiently and reasonably incurred by Qualifying ISPs in carrying out their obligations."

This has the risk of driving up internet access prices as ISPs have to recover the costs of these notifications. Additionally:

"Under the Order, subscribers will have to pay a 20 fee to make an appeal against a report of infringement, which will be refunded in the event the appeal succeeds; the remaining costs of determining an appeal will be met by the Copyright Owner who submitted the CIR which has been appealed."

I disagree with the concept of having to pay for an appeal in this way. If accused, someone should be able to refute the argument without cost. This will introduce a barrier to entry, and even if it is refundable is something I believe should be avoided. Finally, the exact costs of implementing this have been set out. It looks like the Copyright Owners would have to pay 60p per letter (with the ISPs shouldering the remaining 20p), a similar split around capital costs - which are the ones that are actually significant. Taking the above, it seems that in total that content owners will have to shell out 15M to use the DEA scheme. The cost seems to be more or less fixed which could encourage content owners to send out frivolous accusations. There's an iterative process which will run to determine the estimated number of notifications, and if eventually the Content Owners don't agree with the pricing, they will have elected not to take advantage of the provisions of the Initial Obligations Code, as they have not been able to make binding commitments to fund CIR processing. So, in summary, this is something to keep an eye on. It certainly hasn't gone away, it's due to start in March 2014. The revised draft code and consultation, which closes on 26 July 2012, can be found here, and the sharing of costs consultation runs until 18th September 2012, found here.

I've been too busy at home, but finally I found time to wrap up another interview with the people behind Debian Edu and Skolelinux. This time we get to know Jos Luis Redrejo Rodr guez, one of our great helpers from Spain. His effort was the reason we added support for several desktop types (KDE, Gnome and most recently LXDE) in Debian Edu, and have all of these available in the recently published Debian Edu Squeeze version. Who are you, and how do you spend your days? I'm a father, teacher and engineer who is working for the Education ministry of the Region of Extremadura (Spain) in the implementation of ICT in schools How did you get in contact with the Skolelinux/Debian Edu project? At 2006, I verified that both, we in Extremadura and Skolelinux project, had been working in parallel for some years, doing very similar things, using very similar tools and with similar targets, so I decided it was time to join forces as much as possible. What do you see as the advantages of Skolelinux/Debian Edu? A community of highly skilled experts working together, with a really open schema of collaboration and work. I really love the concepts of Do-ocracy and Merit-ocracy and the way these concepts are been used everyday inside Debian Edu. What do you see as the disadvantages of Skolelinux/Debian Edu? Sometimes the differences in the implementations, laws or economical and technical resources in the different countries don't allow us to agree in the same solution for all of us, and several approaches are needed, what is a waste of effort. Also, there is a lack of more man power to be able to follow the fast evolution of the technologies in school. Which free software do you use daily? Debian, of course, and due to my kind of job I am most of my time between Iceweasel, Geany and Terminator. Which strategy do you believe is the right one to use to get schools to use free software? I think there is not a single strategy because there are very different scenarios: schools with mixed proprietary and free environments, schools using only workstations, other schools using laptops, netbooks, tablets, interactive white-boards, etc. Also the range of ages of the students is very broad and you can not use the same solutions for primary schools and secondary or even universities. So different strategies are needed. But, looking at these differences, and looking back to the things we've done and implemented, and the places were we have spent most of our forces, I think we should focus as much as possible in free multi-platform environments, using only standards tools, and moving more and more to Internet or network solutions that could be deployed using wireless. I think we'll see more and more personal devices in the schools, devices the students and teachers will take home with them, so the solutions must be able to be taken at home and continue working there.

I have recently read Test Driven Development with Embedded C by James W. Grenning and published by Pragmatic Programmers. I really enjoyed the book: while I was aware of the huge benefits of having a comprehensive test suite, I never studied seriously the principles behind Test Driven Development (TDD) and this book makes a good introduction to the topic. At the same time it focuses on the C language and contains lots of examples on how you can create tests even for projects which have to interact with hardware or other unpredictable components (the key is to create many abstractions) using all the possibilities that C offers. The author convincingly argues that developing code with TDD forces you to create a modular design that is easier to evolve when the underlying requirements change. He also highlights how the tests serve as reference documentation of the API. James W. Grenning recommends CppUTest as his xUnit test framework of choice. When I wanted to try this test framework, I discovered that it was not available in Debian. I decided to package it because it has some interesting features not offered by the contenders (at least not to my knowledge). It s now available in Debian and in Ubuntu. First, it doesn t require any explicit registration of tests and has a very lightweight syntax. The small downside is that CppUTest requires the usage of C++ for the tests. But C++ is compatible with C so it doesn t matter much if you have a C++ compiler for your target. On the contrary, usage of variables and methods scoped to the test group makes it easy to write clear tests. Here s a short sample of test code: To run those tests, you just need this boilerplate code in a main.cpp: Another interesting feature is its integrated memory leak detection system. Any test that hasn t released allocated memory at the end of the teardown process will be marked as failed. The upstream developers have made some unusual choices (static library only, installation in a private directory) but this will likely change with the switch to an automake and autoconf-based build system. I have reported the oddities that I found and I requested them to provide a pkg-config file to make it easier to compile and link unit tests exploiting CppUTest. I already used CppUTest to develop a small application running on an embedded Linux. At some point, I might try to use CppUTest for dpkg development. I believe that it makes for a good fit. dpkg is already C++ ready since dselect is written in C++ and reuses a good part of dpkg s code. In any case, if you like Test Driven Development and are writing C or C++ based applications, I invite you to try CppUTest.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Many years ago, while studying Computer Science at the University of Troms , I started collecting computer related songs for use at parties. The original version was written in LaTeX, but a few years ago I got help from H kon W. Lie, one of the inventors of W3C CSS, to convert it to HTML while keeping the ability to create a nice book in PDF format. I have not had time to maintain the book for a while now, and guess I should put it up on some public version control repository where others can help me extend and update the book. If anyone is volunteering to help me with this, send me an email. Also let me know if there are songs missing in my book. I have not mentioned the book on my blog so far, and it occured to me today that I really should let all my readers share the joys of singing out load about programming, computers and computer networks. Especially now that Debconf 12 is about to start (and I am not going). Want to sing? Check out Petter's Computer Science Songbook.

Participation and Results From June 15-17th we held a Debian BugSquashingParty in Salzburg, hosted and sponsored by conova communications GmbH. It was a fun and busy weekend, with 15-17 people from 5 countries being around, mainly working on RC bugs in Testing/Unstable. Gerfried Fuchs (rhonda) also worked on triaging the impact of RC bugs on the version in Squeeze, while Peter Palfrader (weasel) took care of Tor related things and Debian sysadmin work, including starting on the new bugs and udd hosts. Phillip Hug (hug) worked on the debian.ch infrastructure. Together with Miroslav Such from Red Hat Bernd Zeimetz (bzed) worked on the packaging of the necessary libraries and daemons to add (basic) Spacewalk client support to Debian. As soon as the packages passed NEW and #677871 was applied (thanks to the APT guys for working on that already), managing Debian clients with Spacewalk should work out of the box. Of course we also had a little keysigning party

Statistics

• about 68 bugs in unstable/testing were triaged/patched/fixed or at least pinged
• 54 bugs were tagged to show if they affect Squeeze, several other bugs were pinged to retrieve necessary information or to trigger an update in the next stable pointrelease.
• 5 packages were introduced into Debian (still in NEW, though) - the Spacewalk client related packages and libapache2-mod-auth-memcookie.

Accomodation Thanks to Debian funds we were able to provide accomodation for four participants in the JUFA youth hostel in Salzburg. We had paid in advance for eight, but changing to rooms with a higher category for only 4 people would have been equally or more expensive.

Press/Media coverage Additionally to being mentioned in the calendars on ProLinux and similar pages, we had some press coverage by the local newspaper and online magazines:

• Scan of Salzburger Woche / Stadt Nachrichten
• article on the Salzburger Nachrichten website
• meinbezirk.at
• Salzburg Cityguide - Photogallery

Fun facts We consumed 2kg of Leberkas, a big plate of "Buchteln mit Vanillesosse", about 16000cm^2 of Pizza, about 80 litres of coke, juice, beer and wine and I guess we drank at least the same amount of water. We had coffee made of 1.5kg coffee beans and managed to empty the (formerly well filled) icemaker in the fridge. Also we had successful training sessions of a standard Debconf game (rules won't be explained here obviously). Maybe we even successfully spread the game to the employees of a commercial linux distribution

The bug webinterface of the Ultimate Debian Database currently knows about the following release critical bugs:

In Total:	1167
Affecting Wheezy:	771
Wheezy only:	189
Remaining to be fixed in Wheezy:	582

Of these 582 bugs, the following tags are set:

Pending in Wheezy:	31
Patched in Wheezy:	82
Duplicates in Wheezy:	57
Can be fixed in a security Update:	21
Contrib or non-free in Wheezy:	11
Claimed in Wheezy:	2
Delayed in Wheezy:	5
Otherwise fixed in Wheezy:	47

Ignoring all the above (multiple tags possible) 376 bugs need to be fixed by Debian Contributors to get Debian 7.0 Wheezy released. However, with the view of the Release Managers, 671 need to be dealt with for the release to happen. Please see Interpreting the release critical bug statistics for an explanation of the different numbers.

Finally I've found time to look back at vacation in Rome and post some pictures. It has been great time there and I've really enjoyed that. I would like to thank to Luigi Gangitano for showing us some interesting places in the city. And now back to pictures:

Filed under: English Photography Travelling 0 comments Flattr this!

I will sound monothematic, but I have been devoting quite a bit of work to this topic lately: Trying to stop the advance of e-voting in Mexico, Latin America and the world. Why trying to stop it? Isn't technology supposed to help us, to get trustable processes? Yes, it's supposed to... but it just cannot achieve it, no matter how hard it is tried I won't get into explanations in this blog post, but there is plenty of information. Feel free to ask me for further details. Anyway Yesterday (Sunday, 2012-06-17) was the fifth simulated voting that will lead to the first wide-scale deployment of electronic voting booths in my country: About 10% of the population of the state of Jalisco (that means, ~500,000 people) will cast their votes on July 1st electronically. This particular case illustrates how simulated votings can be used to forge a lie: Pounce Consulting, the company that won the e-voting project for IEPC (Jalisco's voting authority), delivered their booths over 40 days late, just before the deadline for the project to be canceled. Oh, and by the way, it's the same company that just failed to deliver on time for another planned local authority (10% of the booths in the Federal District, where I live, where fortunately 100% of the votes will be cast on traditional, auditable and cheap paper). After this delay, five voting simulations were programmed, to get the local population acquinted with them. The first ones just failed to get the population's interest and had close to 40% failure rates (mainly regarding transmission). Several other "minor details" were reported, including mechanical details that allowed subsequent voters to see the vote of who had just left. Anyway, making long story short: The fifth and last simulation was held yesterday. Officially, it was finally successful (about time). As these booths include the "facilities" to communicate the results via the cellular network, but the populations where they are to be deployed do not yet have cellular coverage, 10% of the booths will have to be carried back to the Districtal Header (that can be a ~10hr trip) to be counted. Also, in all places, traditional paper stationery and paraphernalia will be printed just in case it is needed (and when will they now? When half of the votes are cast and lost?) Anyway... e-voting is still in its first stage in Mexico. Right now, I'm sure, no attempts to rig the election will be made (centrally). But every effort will be made (as it has been made) to dismiss the obviously big and nontrivial ways it has failed and will fail, and any problems will be labeled as "minor". And probably by 2018 we will be facing many more states (even nationwide) deployments. But propaganda fails to see the obvious: E-voting is more expensive, more complicated, leads to more possible failure states. E-voting should not be deployed in large-scale (i.e. more than a couple of hundred voters) elections. Electronic voting is insecure, violates secrecy, allows for fraud. No matter how many locks are put into it.

I've made some tweaks and fixes to my Gettings Things Done for Hackers booklet on personal productivity, and updated the website and the ebooks (see sidebar on the site). I've also published it on the Amazon Kindle store in case anyone wants to buy it ($1.99). The book remains licensed under the CC-BY-SA 3.0 license, and the Mobipocket version on my site is the same as the one Amazon publishes, modulo any changes Amazon's automation may have done. The Kindle store version is really there for only overly generous people.

A copy of this post is sent to soc-coordination@lists.alioth.debian.org as well as to debian-bootstrap@lists.mister-muffin.de. Diary June 4 I added the first version of basenocycles.ml to git. Given an initial set of cross built packages, it tries to compile as much as possible on the resulting system in multiple rounds. June 5 During June 3, I discovered and error in my program that would only come up when using the Debian Sid package lists as the input:

Fatal error: exception Assert_failure("common/edosSolver.ml", 610, 11)

On this day, June 5, I wrote a minimal test case for this problem. The same day, Pietro figured out that this is a bug in dose which will be fixed in the next release. Begin writing code to figure out how important a binary package is for the further build process. Try to use Depsolver.edos_install to find out what packages are needed to make debhelper available. Restructure basenocycles.ml, exclude source packages that already have been built, still trouble with already existing binary packages and Cudf.mem_installed, comment stuff better. June 6 I wrote some crude code (only estimate, not correct, fixed later) that would give a rough overview of how often a given binary package is directly required as a build dependency. Debhelper came out as the most needed package. It is architecture:all, so it does not have to be built but it has unfulfilled runtime dependencies. To make those packages available, 13 (actually 11, fixed later) packages have to be compiled on Ubuntu Natty. But those packages all (except gettext) require debhelper itself to be built. The first dependency cycle. This dependency cycle (actually, the 12 cycles) can be broken by either cross compiling those source packages or by making them build without debhelper. One goal of the program is to help decide what the easier option is, but this is not yet implemented. To play around a bit, I created the possibility to specify a list of packages that are additionally to the minimal set of cross compiled packages also cross compiled. I added the 13 packages found above to the list, thus making the binary packages they build available. This made debhelper available in the system. As a result, 1625 out of 3339 source packages can be built with just a minimal build system (priority:essential packages plus build-essential) and debhelper available. The next package that blocks the most other source packages from being built is cdbs. The next nine packages in that list also require cdbs so it seems to be the next important package to be available. Pietro's suggestions make me:

 - do not open BootstrapCommon but ExtLib, Common, Algo, Debian
 - do proper option parsing and logging
 - use Debcudf.ignore_essential = true
 - do Debcudf.init_tables (binlist@srclist)
 - use @ with shorter list first
 - use more List.rev_append instead of @
 - use CudfAdd.who_provides to find if a package is available

June 7 Pietro and Patrick suggest that for solving the debhelper cycles, one could provide a minimal debhelper version so that the above list of 12 packages can be built without debhelper. I try to figure out how to get a list of packages that are missing to make a package installable/buildable. This functionality should be provided in dose but I fail to find it. June 8 Lacking a solution of the problem of June 7, I write a mail to Pietro. I start my first graphs in ocaml using the ocamlgraph library. The graph I generate, starts off at a binary package. For each binary package it connects new vertices as its runtime dependencies. If a binary package is not arch:all and also not yet otherwise compiled, its source package is also added. The result is a graph in which set of source packages in it will make the initial package available, if those source packages would be cross compiled. The graph is extended further than the source packages. June 9 I refine a couple of functions, make univ_get_pkg_by_name return the package with the highest version number. I wrote a rather lengthy (1027 words) email to the list that explains my status as of this day. I can create graphviz dot files with ocaml, can create node and edge types and create the graph by an imperative pattern that I saw a lot in Pietro's code. Disjunctions are not yet handled correctly (see mail from June 8). The graphs generated look like the following: http://mister-muffin.de/p/8nyc.png June 11 I write a test case which shows how CudfAdd.who_provides doesnt find virtual packages. Automate the process of finding the packages that, if cross compiled, would make another package available. Add more predicates (identifying source packages) and improve input file reading code. Move build_compile_rounds which compiles as many source packages as it can in multiple rounds on a given minimal system a toplevel function and thereby abstract it more. Create a rudimentary text based menu to choose different actions to take for an analysis. Start writing an extended version of simple_dependency_graph for deeper analysis. Use xdot to show graphs from the text menu. Allow saving those graphs to a file. June 12 Move functionality from the extended version of simple_dependency_graph over to the normal version and delete the extended version. Add the new Closure vertex type. Create extended_dependency_graph which is supposed to not contain single binary package vertices but handle a package and its installation set as one vertex. The output of extended_dependency_graph is optionally reduced to the biggest (non degenerate) strongly connected component. User gets the option of choosing the exploration depth. June 13 Pietro replies to my mail from June 8 but apparently I failed to express myself well enough in my last mail, so I rephrase my question. Pietro replies to my email from June 11 and explains how the effect I see is due to "a nuisance of the debian to cudf encoding". As a result I change my code accordingly. June 14 Another lengthy (1130 words) email to the list. I explain what was done in the past days, what parts work and how they work. I list some rationales on why I did things the way I did them. The most important observation is, that after improving my code again and again, I ended up representing the dependency cycle problem in the same (very similar) way that Pietro suggested in the beginning. This is probably a good discovery. Lots of data of that email is now of only little use as of June 16, I make lots of improvements in correctness. As I dont have an answer to my other email to Pietro from June 13, I implement a very crude way to get an answer to the question of what packages are missing for a package to be available/compileable. I called it flatten_vpkgformula_best_effort and it suffers from many faults including disjunctions and package conflicts. Patrick spots a problem. As a result, I make sure that at no point, the source package of an arch:all package can be listed. June 15 As a reply to my mail from June 13, Pietro creates a new branch in the git and adds the code I needed to get a proper installation set. June 16 As a result of Pietro's efforts from June 15, I make great advancements on all fronts. Details of the current status follow in the next section. Results A big leap was made on June 16 due to Pietro's great help on making me understand how Depsolver.listcheck can be used for my purposes. My difficulties in finding the solution myself are rooted in many parts of the dose framework being poorly commented but Pietro did already a couple of documentation commits whenever things were unclear for me. Using Depsolver.listcheck makes it possible to be more distribution agnostic and I dont have to handle vpkgs, virtual packages and constraints myself anymore. The code also doesnt suffer anymore by wrongly analyzed dependencies and conflicts. The only thing that is not yet taken care of, is that Depsolver.listcheck only chooses one out of several possible installation set. A final version should be able to take into account that a different installation set could provide a better solution. Overall, in comparison to two weeks ago, I can now properly build, traverse and analyze graphs, can choose an installation set properly, understand more about dependencies, closures, dose and ocaml in general. Finding the importance of binary packages for building When calculating how many source packages are depending on the availability of a binary package I originally flattened the pkg.Cudf.depends list twice for a rough overview. This is of course wrong due to disjunctions and conflicts and also doesnt provide a deep dependency inspection. The new method is to calculate an installation set that is necessary to compile a source package for every source package. The resulting list of binary packages is then used to find out how often a binary package appears in an installation set. I see three drawbacks though:

• calculating an installation set for each source package in the archive is very slow
• if X packages build depend on A then also X packages will build depend on the installation set of A, resulting in lots of duplication
• only one installation set is selected though there are many

Removing simple graph The simple graph which contained single binary and source packages was removed. I realized it doesnt really serve any purpose to look at it. As a result, Bin vertices and InstallDep edges are also not part of the graph anymore. Since it was responsible for generating the list of source packages that have to be cross built to make a package available, I created a new function get_cross_source_packages which uses an installation to serve the same purpose. Fix extended_dependency_graph extended_dependency_graph now uses installation sets for generating the list of packages that is needed to compile a source package or install a binary package. The list of build dependencies does not include packages that are already installable. The list of runtime dependencies does not include packages that are otherwise available (cross built, arch:all...). Instead of checking for list membership all the time, I created hash tables for the list of installable as well as for the list of available binary packages. Future There are two big tasks for the next two weeks: Task one is to find a way to give hints on which packages to best consider for having reduced build dependencies. This would then probably finally make use of Pietro's cycle algorithms. Task two is to find a way to break cycles and create a build-DAG from a list of packages that already have staged build dependency information. Patrick is currently working on patching dpkg with Build-Depends-StageN dependencies as making perl cross compilable. If he doesnt need the ability to decide which packages to modify to have staged build dependencies in the near future, then task one is probably less urgent and therefor of less importance right now? On the other hand, I can easily generate fake reduced build dependencies so that doing task two right now would not be a problem. Also, having the solution for task two will make it possible to show the user what effect it would have to add reduced build dependencies to a package. For the reasons above (it's not urgent, task one profits from task two being solved) I will go and implement task two first (if there are no objections from my mentors). Another idea, that I discussed with wookey and Patrick yesterday, was that due to multiarch being used for more and more packages, there should exist a set of packages that is cross compilable without any change to the package. We agreed that I make a list of packages that, if cross compiled, would break dependency cycles and make other packages available. I created such a list of about 160 packages for Ubuntu Natty that, if cross compiled, made it possible to have 87% of Natty available (those numbers have to be treated with caution as I did not yet use the proper way of installation sets when generating that list, but the order of magnitude should be correct). Wookey can then try to cross compile those packages. If some packages of those "crucial" source packages are found to be cross compilable, then they should be cross compiled because it means that no work has to be done to break some cycles. Cross compiling all packages that are cross compilable out of the box is no solution, as only natively compiled packages can go into the archive. This is why the list of potentially additionally cross compiled source packages has to be kept as small as possible.

BEGIN PGP SIGNED MESSAGE
Hash: SHA256 , During the month of June, we experienced a service interruption which
has caused our mutual customer, School, an
inability to initiate outgoing calls. CenturyLink has also caused the
interruption of the school s DSL Internet service. During the interim
period, I have installed a VoIP phone and tied in to one of my backup
providers so that the school can function during the CenturyLink outage. My understanding is that will begin paying CenturyLink
directly for its service again due to the service interruption. Until
such time as I receive acknowledgment from you, we will continue paying
roughly $27/mo for the school s phone service. Once the transfer has
been completed to the satisfaction of the school board, Collier
Technologies LLC will discontinue payment for the DID - - ,
leaving a single DID, - - . I expect that the DID will be
moved back into the school s name within the next week. Please confirm
this or provide information as to why this will not happen. I have CC d
and , who represent the school board on this
email. Please include them in the reply. Since the DSL service has been interrupted at both the School s address
and at ours, and since CenturyLink has failed to provision the remaining
two $14.99/mo 10Mbit DSL circuits which Collier Technologies LLC has
been paying for despite their un-provisioned state, we will be reducing
the monthly payment from $150 to $70. My understanding is that $70 will
be more than sufficient to cover the two voice lines, inside wire
maintenance and regulatory fees. If this is not the case, please keep
us appraised of the situation. Do note that our decision to stop payment for service which is not being
provided does not, in our estimation, relieve CenturyLink from the three
year contract period negotiated verbally by you on its behalf on October
4th 2011. We currently consider CenturyLink to be in breach of
contract. Sincerely, C.J. Adams-Collier BEGIN PGP SIGNATURE
Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBCAAGBQJP3TqdAAoJEEyimPmPQm23qqoH/jRmiIVHNCnH8EqgzsNQiUIm
LVrFg7kvpRIM692mzrGgkZTxG9KlVK+1/dZl1aPsmMP5Q6ni3NigzRDnywJZI53a
yTwTZhid8B4Tf+JOrhUhKPpmm+2YoHTYaobwgxk+rRgUGUHEMCZMPnjKNGGPDCgO
2jNz5fSnI6IinRz4nXwhAtGQRUXVJWXGrUCX1kQXGH4pNvjbgE8n6yAsC6sixlPa
ljDaNjxUEjUY4PmQdKNetkzQLxlq/OGJgRHZXHzfg37cJEF4L28DJBYvf8CHbt2K
87ELXGs6nDn54yI4q4ojODln9k/GzC+NY6MDXYdof7dNbeEPujvjQCiyRgjLc9E=
=uLKs
END PGP SIGNATURE

I am transitioning my GPG key from an old 1024-bit DSA key to a new 4096-bit RSA key. The old key will continue to be valid for some time but I prefer all new correspondance to be encrypted with the new key. I will be making all signatures going forward with the new key. I have followed the excellent tutorial from Daniel Kahn Gillmor which also explains why this migration is needed. The only step that I did not execute is issuing a new certification for keys I have signed in the past. I did not find any search engine to tell me which key I have signed. Here is the signed transition statement (I have stolen it from Zack): For easier access, I have also published it in text format. You can check it with: To avoid signing/encrypting with the old key who share the same email addresses than the new one, I have saved it, removed it from the keyring and added it again. The new key is now first in both the secret and the public keyrings and will be used whenever the appropriate email address is requested. I now need to gather some signatures for the new key. If this is appropriate for you, please sign the new key if you signed the old one.

The bug webinterface of the Ultimate Debian Database currently knows about the following release critical bugs:

In Total:	1289
Affecting Wheezy:	862
Wheezy only:	180
Remaining to be fixed in Wheezy:	682

Of these 682 bugs, the following tags are set:

Pending in Wheezy:	37
Patched in Wheezy:	86
Duplicates in Wheezy:	57
Can be fixed in a security Update:	22
Contrib or non-free in Wheezy:	10
Claimed in Wheezy:	2
Delayed in Wheezy:	2
Otherwise fixed in Wheezy:	41

Ignoring all the above (multiple tags possible) 466 bugs need to be fixed by Debian Contributors to get Debian 7.0 Wheezy released. However, with the view of the Release Managers, 757 need to be dealt with for the release to happen. Please see Interpreting the release critical bug statistics for an explanation of the different numbers.

Today, my workstation sent me this email:

The following warning/error was logged by the smartd daemon:
Device: /dev/sda [SAT], 1 Currently unreadable (pending) sectors

and then a little later, this one:

The following warning/error was logged by the smartd daemon:
Device: /dev/sda [SAT], 1 Offline uncorrectable sectors

From the hard disk s SMART data, this is a clue that the drive is failing or will soon. Sigh. Incidentally, if smartmontools isn t installed on your machine, whether it s a laptop, desktop, or server, it should be. Although most of you know I run Linux on the metal on my machines almost exclusively, I do maintain a small drive with a Windows installation that I boot into every few months for various reasons. This is that drive. The drive is non-redundant (no RAID), and although it is backed up, the backup is made via backuppc from the NTFS filesystem mounted on Linux, and is a partial backup backing up certain data, not the OS. There are, of course, bare metal Windows backup solutions, but I generally don t want to back up Windows from within Windows on this machine. Restoring Windows isn t quite as simple as an mkfs, an untar, and a grub-install, either. So my first thought is: immediately save whatever of the drive I can. So I ran apt-get install gddrescue to install the GNU ddrescue tool. ddrescue is somewhat similar to dd, but deals much more intelligently with bad blocks on the drive. It will try to read them repeatedly, with decreasing block sizes, in an effort to get every last good byte off the disk that it can. If it ultimately fails to get certain bytes read, it will write placeholder data to the output file in place of the missing data, so that the output file maintains proper size and alignment. It also saves a log file that notes what it found (see info ddrescue for more on that.) So I created an LVM volume for the purpose (not enough free space on /home, and didn t want to have to shrink it somehow later), and ran:

ddrescue /dev/sda /mnt/sdasave.ddrescue /mnt/sdasave.logfile

Then I went to dinner. When I got back, I discovered there were 1 or 2 bad sectors, about halfway through the disk, but everything else was fine. So now, the question became: did I lose any data? If so, what? I needed to know if I had to revert to a backup for anything or not. To answer THAT question, first I had to figure out the offset of the bad spots on the disk. That s not too hard; the logfile gives it to me:

# Rescue Logfile. Created by GNU ddrescue version 1.15
# Command line: ddrescue /dev/sda /mnt/sdasave.ddrescue sdasave.logfile
# current_pos  current_status
0x3BBB8BFC00     +
#      pos        size  status
0x00000000  0x3BBB8BF000  +
0x3BBB8BF000  0x00001000  -
0x3BBB8C0000  0x38B5346000  +

what we see is that the bad sector starts at byte 0x3BBB8BF000 (256549580800 decimal) and extends for 0 1000 bytes (4096 decimal). Both the drive and NTFS use 512-byte sectors. So dividing by 512, we get sector 501073400 501073407 (4096 bytes is 8 sectors). As a check, I ran grep sector /var/log/kern.log and turned up a bunch of lines like this: Jun 14 21:39:11 hephaestus kernel: [35346.929957] end_request: I/O error, dev sda, sector 501073404 Which is within my calculated range. But this is an absolute sector on the disk. We need the sector within the partition, so for that, we have to enlist fdisk to make that calculation. fdisk shows, among other things:

Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048   976771071   488384512    7  HPFS/NTFS/exFAT

So the Windows partition starts at disk sector 2048. Let s just confirm that. If I use dd if=/dev/sda1 bs=512 count=1 hd head, I see a line beginning with .R.NTFS . Exactly the same as with dd if=/dev/sda bs=512 count=1 skip=2048 hd head, so I read the partition table information correctly. Subtract offset of 2048 from the earlier values, and I get relative sectors 501071352-501071359. That s enough to get some solid info from the filesystem via ntfscluster, part of Debian s ntfs-3g package. I pass -s to it, and ignoring some irrelevant stuff, get my answer:

ntfscluster -s 501071352-501071359 /dev/sda1
Inode 190604 /System Volume Information/ b4816feb-b609-11e1-a908-50e549b934f7 3808876b-c176-4e48-b7ae-04046e6cc752 /$DATA

I even reran it with a much larger sector range, just to be absolutely sure I had wiggle room in case calculations had an off-by-one error or something somewhere. This is really great news, because the file in question is pretty much useless I believe it s a system restore point, which I won t be needing anyhow. So at this point, all that remains is to reinstall this on a different drive. For that, I could just use my ddrescue image. I thought I would take a second image, just to be very extra careful, and use that; I used:

partclone.ntfs --rescue -c -s /dev/sda1 -o sda1.partclone

although ntfsclone would work just as well. This captures only the partition; I ll need the partition table as well, and perhaps also the space between the partition table and the first partition. I could capture it separately with dd, but it s already in the ddrescue image, so there s no need. (GRUB is installed on this drive, but there is no Linux filesystem on it, so it may well exceed the size of the MBR). Note that for Linux ext[234] filesystems, debugfs can provide the same (and more) info as I got from ntfscluster. I happen to have a drive of the right size sitting here, which I was about to install in a different machine. So a wipe and a swap and a restore later, and I should be good to go. This scenario is commonplace enough that I thought I d post how I dealt with it, in case anyone else ever has hard drive issues.