Both the procps and psmisc projects use Dejagnu for their testing. It s interesting to try to understand how it all works and learning a new language TCL. One of the big problems with Dejagnu is its documentation which is very sparse. So here are two things I ve picked up along the way for a reference for myself and others. Locale We recently had a bug report on the procps list where tests for w failed for someone when the headers were shown. After a some further debugging it was a locale problem, where the testsuite was looking for a number like 3.14 but the tester was seeing 3,14. There are many ways around this but for us it was to set the locale, with the following line in config/unix.exp after suggestions by Mike Frysinger should fix it. Cleaning Up
The documentation on this function is awesome! A title Cleanup Procedure and one line cleanup() .
That s all you get. The cleanup proc or function is called at the end of the testing. I use it
to remove temporary files and things like that.

A copy of this post is sent to soc-coordination@lists.alioth.debian.org as well as to debian-bootstrap@lists.mister-muffin.de.

Diary

June 18 Pietro suggests a faster way to generate installation sets for a list of packages. In my case, I need an installation set for every source package in the archive to find out how often a binary package is needed to build a source package. As a result, the speed is doubled in contrast to the original approach.

June 19

• adapt code to work with new dose release 3.0
• remove unneeded parts of code
• add different possibilities to find amount of source packages that need a binary package
• add code to get multiple installation sets using Depsolver_int.solve

June 20

• add ~global_constraints:false to Depsolver.listcheck, Depsolver.trim and Depsolver.edos_install calls
• adapt output graph to limited xdot capabilities

June 21 I formulate an email to the list, reporting of dependency graphs of debhelper, cdbs, pkg-config and libgtk2.0-dev. My current technique gets an installation set for a source package, removes all those that are already installable and adds the others as a dependency of that source package. This dependency will include an installation set of that binary as well minus all packages that are already available. The problem with that approach are dependency cycles created by long dependency chains. Example: src:A needs B needs C needs A. B and C would both be added as a dependency of src:A. B as well as C would also include their installation set which in both cases includes A. So now there are two cycles: src:A->B->A and src:A->C->A. For a real life example, look at the following situation of cdbs and src:sqlite3. It is created because src:sqlite3 needs cdbs needs python-scour needs python needs python2.7 needs libsqlite3-0. Therfor libsqlite3-0 is in the installation set of cdbs, python-scour, python and python2.7. This creates five cycles in the graph even though there is only one. It would be better to reduce the dependencies added to src:sqlite3 to its direct dependency which is cdbs. Package dependencies are disjunctions from which the solver chooses one or the other to build an installation set. To solve the problem above I would need to know which disjunction the solver chose and then only add the direct dependency of a package to the dependency graph.

• improve build_compile_rounds performance
• big overhaul of menu structure
• fix subgraph extraction code

June 22

• do not create a universe if not needed - use hashtables instead
• for sorting packages, generating difference of package sets and otherwise comparing packages, always use CudfAdd.compare
• as a custom list membership function, use List.exists instead of trying List.find
• more speedup for build_compile_rounds
• the number of source packages that can be built does NOT include the cross built packages
• print closure members in graph
• refactor code and move common functions to bootstrapCommon.ml
• add breakcycles.ml for future code to break cycles using staged build dependencies
• use more extlib functionality
• extended package list input format

June 23 After several emails with Pietro I learn about syntactic dependency graphs. To document my progress and prove my efforts I committed the code as commit 6684c13. But this code was soon found out to be unecessary so it will be removed later and just serves as documentation.

June 24 I came up with another (better?) solution to get the chosen disjunctions. It simply uses the calculated installation set to decide for each disjunction which one was taken by the solver. I reported that important step and the open questions involved with it in an email to the list. The problem always was, that an installation set can easily contain more than one package of a disjunction. In this case it is not clear which branch was chosen by the solver. I found, that in Ubuntu Natty there are only 6 such packages and for each of them the situation can be solved. It can be solved because in all of those cases it is that either one package of a disjunction provides the other or that both packages depend upon each other, which means that both have to be included.

June 27

• use installation set to flatten build dependencies of source packages
• refactor code and move common functions to bootstrapCommon.ml

June 25 I have to have an algorithm that finds all circuits in a given graph. This is necessary so that:

1. cycles can be enumerated for huge dependency graphs where cycles are hard to see
2. cycles can be enumerated to find a cycle that can be broken using staged build dependencies

It seems that Johnson's algorithm is the best way to do this complexity wise and Pietro already blogged about the problem together with an implementation of the algorithm in ocaml. Unfortunately it turns out that his code doesnt implement the algorithm correctly and hence misses out on some cycles. The fix seems not to be too trivial so I'm still investigating it.

June 28

• add crosseverything.ml to obtain a list of source packages that, if cross compiled, would make the whole archive available

Results While the first week was productive as usual, I had to work some time on a University project during the second week as well as attend a family meeting. I will catch up with the lost time over the course of the next week.

dose3 Using dose 3.0 (which fixes a bug about essential packages) the output of the algorithms is now likely less wrong then before.

performance Performance was improved in the generation of installation sets as well as in the code that tries out how many packages can be built in multiple rounds. This was achieved by more caching, less unnecessary operations in looping constructs, replacing lists with hashtables, not creating universes where not necessary.

user interface The main program, basenocycles.ml now has a much better menu structure.

input format The programs take two package file inputs. The list of source packages that has to be cross built for a minimal build system and the list of source packages that was chosen to be cross compiled in addition to that. Both files list one source package per line and now allow comments.

refactoring As more and more scripts are added, more and more functionality is moved to bootstrapCommon.ml which makes each script much cleaner.

what to test for cross building As discussed in the "Future" section of the last report, I now automated the process of finding out which packages, if they were cross compiled, would make the whole archive available because they break all cycles and allow native compilation of the rest. The outcome: to build 3333 out of 3339 packages in natty natively, at most 186 source packages must be cross compiled. The other six cannot be compiled because of version mismatches in the Natty Sources.bz2 and Packages.bz2. The code can be run from crosseverything.ml.

limit source dependencies to direct dependencies Reducing the dependencies of source packages from their full installation set to their direct dependencies by finding out which disjunction of their dependency list were taken, greatly simplifies the dependency graphs. The dependency graph created for libgtk2.0-dev could be reduced from 491 to 247 vertices for a depth of three. For cdbs it is now clearly visible that cdbs depends on libsqlite3-0 which builds from src:sqlite3 which depends on cdbs. Before: After: For pkg-config the graph also has been reduced to the one single cycle that matters: src:pkg-config needs libglib2.0-dev which depends on pkg-config which builds from src:pkg-config. Before: After:

Future I will prepare content for wookey's debconf talk on crossbuilding and bootstrapping. As this will include directions how to use the current code, I will kill two birds with one stone and write some proper documentation for my current source. The following two lists will be displayed after a dependency graph is calculated and reduced to its scc:

• those source packages that have the least build dependencies not fulfilled. Those might be candidates for easy staged build dependencies. Since the source package is part of the scc, it will definitely be involved in some cycle somewhere.
• those binary packages that most source packages depend upon. Those could be candidates for cross compilation as it might be easier to cross compile the source package than using staged build dependencies.

Patrick managed to cross build packages with sbuild now. So the list of packages that crosseverything.ml produces can now be checked efficiently for cross buildability. With this list, potentially more cycles can be broken out of the box. A feature will be added that allows the user to remove all packages from a dependency graph that can be cross compiled without any additional effort. Version mismatches between source and binary packages in Sources.bz2 and Packages.bz2 respectively in Ubuntu make the scripts fail and/or produce wrong results. Debian (even Sid) doesnt have this problem so I should find out where to report this problem to Ubuntu. I need to write a working version of Johnson's algorithm because much functionality depends upon it. I have the option to improve Pietro's version or write one from scratch. Writing one from scratch might be easier as I have Pietro's code as template as well as a Java implementation of Johnson's algorithm which seems to work. The following functionalities need working cycle enumeration:

• given source packages with staged build dependencies, an enumeration of cycles is needed to find out which cycles can be broken by building packages staged. It makes less sense to blindly build a package stage and then check if this makes more packages available.
• display cycles of a dependency graph to the user. After obtaining all cycles in the graph it makes sense to sort them by their length. The user would then investigate the situation of the smallest cycles first. This makes sense because breaking small cycles can potentially break bigger cycles. Since in the end, all cycles have to be eliminated anyway, it makes sense for the user to first tackle the small ones.
• display the feedback arc set to the user. The packages in the feedback arc set might be very good candidates for reduced build dependencies or cross compilation.

For the last 10 years, I've been working on many Free Software projects. From Debian to OpenStack, through awesome, Emacs, XCB and many more. This obviously allowed me to enhance my technical skills, but it also taught me about Free Software and Open Source development processes, and how to work with and close to the community. Working for almost 6 years at Easter-eggs taught me how to work in an autonomous manner, how to lead and manage a project. And how to run a company, thanks to the cooperative status of this great one. These are the reasons why I decided to leave my latest job and run my own company to work as a freelance consultant & developer specialized in Free Software, starting today. Therefore, I am now able and available to provide expertise and development on Free Software, including upstream contribution. Especially on projects I already worked on recently, like OpenStack.

I am in Berlin for the Wikipedia Academy, a very cool hybrid free culture community plus refereed academic conference organized, in part, by Wikimedia Deutschland. On Friday, I was very excited to have been invited to give the conference's opening keynote based on my own hybrid take on learning from failures in peer production and incorporating a bunch of my own research. Today, I was on a panel at the conference about free culture and sharing practices. I'll post talks materials and videos when the conference puts them online. I will be in Berlin for the next week or so before I head to directly to Washington, DC for Wikimania between the 11th and 15th. I'll be giving three talks there:

• Almost Wikipedia: A What Eight Collaborative Encyclopedia Projects Reveal About Mechanisms of Collective Action
• Can Social Awards Create Better Wikis? (with Aaron Shaw)
• The State of Wikimedia Scholarship (2011-2012) (with any poor souls I can recruit into helping me in the next week or so)

Between then and now, I'm taking the next week in Berlin to catch up on work, and with friends. If you're in either place and want to meet up, please get in touch and lets try to arrange something.

I recently had the sort of weird experience of a recent blog post being on the top of Hacker News and there being a fair amount of interest in the details of my wacky living situation. I could write a detailed post explaining everything, but that'd be boring .. instead I'll tease the stalkers with more oblique references to it. So here's my notes for a caretaker who will be here while I'm away at DebConf.

---

Please make yourself at home!

• The back bedroom bed is made
• That weird thing on a stand next to it that looks like a juice container with a wire coming out of it is a lamp. Plug it in to use it.
• Check battery voltage (press the green button on the Xantrex controller) and avoid using much power if it gets below 12.0 volts. (10.0 is absolute minimum)
• Orange inverter on stand in front bedroom has a switch to turn on, regular power cord on front porch.
• Plug in server computer in kitchen (under my portrait), wait 10 minutes, and you can get online with wireless. (Wireless network is "hollow", and is an ad-hoc network, with Cell id AA:AA:AA:AA:AA:AA. You may have to enter this information manually.) Slow is normal.
• Cook stove: Pull red handle behind stove to turn on gas. Light w/lighter. Turn off cook stove gas with red handle when not in use!
• Eat any staple foods in kitchen.
• Eat everything in fridge! (It won't keep; food lasts only half as long in it as you're used to.)
• Check once a day (especially on windy or stormy days) that the fridge is lit, by touching the black pipe sticking above it on the back. That's its chimney. It should be warm to the touch. Temperature can be adjusted in rear also. If fridge is blown out, follow instructions printed in back to re-light.
• Human drinking water is in large jugs in kitchen + red water cooler. Sorry it's so turbid right now, I'm using a neighbor's well.. but it's safe. I have not tested the water from the large cistern yet this year.
• Dishwashing water: Use water from any of these (non-potable) sources:
  • rain barrels
  • outdoor shower (on way to outhouse, not at parking area; that other one is fed from a spring that has dried up)
  • large pail on coffee table
• Please don't drink the distilled water; it's used in battery maintenance.
• Feed dog 3/4 of metal bowl each day (on porch, next to table)
• Cat: 1 dish full each day.
• Consider brushing the cat, with the folding saw, on the left-side porch steps, every other day. Hairballs..
• Chickens: 3/4 yoghurt container of feed per day (feeding station is across yard, down steps, on right; some feed is in container there)
• Eggs can be found in nest under bush in woods. Trail starts to left of shower, goes about 10 feet in. Eat eggs, or give to dog with food.
• Blueberries are currently available on the small hill near the greenhouse.
• Dog and chicken feed is in front bedroom.
• Keep solar powered fountian to the left of the sliding door full; it's used by the dog and chickens.
• Cat has large water pail in bathroom, good for about 2 weeks.
• Take animal water from rain barrels or from shower outside. (Can detach hose for easier access and watering.)
• Also water lemon and lime trees, in pots, if it's dry. (Feel free to eat lemons.)
• The house does best in hot weather if all windows are closed during the day, and opened during the night.
• If it gets too humid inside, drain water from trays of salt underneath the pantry shelves, and run the fan.

The brand new machine for Ultimate Debian Database motivated me to do UDD-related work again, so I implemented an old idea: build a maintainer/team-centric dashboard relying on UDD, the Debian Maintainer Dashboard. The idea is to expose as much useful information as possible about a maintainer s packages, both in the traditional (Developers Packages Overview-like) big table with all the info , but also in a task-oriented listing (to answer the ok, I have two hours for Debian, what should I do now? question). At this point, the Debian Maintainer Dashboard already brings several cool features (try it with my packages or pkg-ruby s packages, or your own!):

• Reports buildd issues (failed builds)
• Upstream status monitor that works (UDD has its own)
• Knows about teams VCS (if your team uses PET) and display the current version in the VCS
• Lists packages you maintain or co-maintain, and also the package you showed interest for by subscribing to them on the PTS

Of course, as of now, it s pretty ugly: that s the help needed part of the post. I really suck at web design, and would really appreciate if someone would help me to turn it into something nice to use. Besides adding links and colors everywhere, it could probably be nice to add sortable tables, and to create tabs for TODO items , Versions and Bugs (and possibly Lintian, for example). Contact me if you are interested.

Yesterday night, Regina and me arrived to Nicaragua. Ready to greet us, we found quite a good number of good friends. We had a nice pizza+beer dinner at Diana's house, and some of the foreigners among us were distributed among the houses of several locals. This morning, we woke up together with V ctor, Moray and Gaudenz in Norman's brother's beautiful house. We had breakfast with the family, were picked up to go to the hotel that will have the ho(n rr)or to host us all for the following two weeks, and walked to the Universidad Centroamericana (UCA) campus. Contrary to our usual practices... It seems everything is working fine! I mean, I'm sure we will stumble with some unforseen details and what not... But coming on the very first day to the university, to find that food is all sorted out, that we have food tickets (and they are all printed!), that network works (and it's by a fiber connection that was laid out expressly for us), that we have all the hardware I was worried about, that people are arriving and getting accepted at the hotel. I mean, things work! So, I'm quite optimistic this DebCamp will have everything ready to be a success And the DebConf following it as well, of course! If you have not yet arrived - See you soon!

So I slept in the Sheraton Heathrow hotel last night. It's a half-an-hour walk away from the airport, or a five minute bus drive. If you go to any service desk and ask for the best way to get to the hotel, they'll offer you a ticket for the 'HotelHoppa' service, which will stop right in front of your hotel entrance, and costs 4 for a one-way trip (even though the bus sides declare a 4.5 charge) However, if you take the regular "London United" bus service (which drives all over London, not just near the airport like the HotelHoppa service) then the same trip is completely free of charge. You may have to walk about a hundred yards more to reach your hotel, since the bus will stop at a regular bus stop rather than at the hotel entrance, but that's hardly a problem. What a ripoff. So, to clarify, don't take this bus: Instead, take this bus: This message brought by your friendly neighbourhood anti-ripoff service.

Thanks to a heads up from Bastian Blank, I learned that Lintian 2.5.7 and 2.5.8 were horribly slow on the Linux binaries. Bastian had already identified the issue and 2.5.9 fixed the performance regression. But in light of that, I decided to have a look at a couple of other bottlenecks. First, I added a simple benchmark support to Lintian 2.5.10 (enabled with -dd) that prints the approximate run time of a given collection. As an example, when running lintian -dd on lintian 2.5.10, you can see something like:

N: Collecting info: unpacked for source:lintian/2.5.10 ...
[...]
N: Collection script unpacked for source:lintian/2.5.10 done (0.699s)

When done on linux-image, the slowest 3 things with 2.5.10 are (in order of appearance):

[...]
N: Collection script strings for binary:linux-image-3.2.0-2-amd64/3.2.20-1/amd64 done (12.333s)
N: Collection script objdump-info for binary:linux-image-3.2.0-2-amd64/3.2.20-1/amd64 done (15.915s)
[...]
N: Finished check: binaries (5.911s)
[...]

(The mileage (and order) probably will vary a bit.) These 3 things makes up about 22 seconds of a total running time on approximately 28-30s on my machine. Now if you wondering how 12, 16 and 6 becomes 22 the answer is parallelization . strings and objdump-info are run in parallel so only the most
expensive of the two counts in practise (with multiple processing units). The version of linux-image I have been testing (3.2.20-1, amd64) has over 2800 ELF binaries (kernel modules). That makes the runtime of strings and objdump-info much more dominating than in your average package . For the fun of it I have done a small informal benchmark of various Lintian versions on the binary. I have used the command line:

# time is the bash shell built-in and not /usr/bin/time
$ time lintian -EvIL +pedantic linux-image-3.2.0-2-amd64_3.2.20-1_amd64.deb >/dev/null
# This was used with only versions that did not accept -L +pedantic
$ time lintian -EvI --pedantic linux-image-3.2.0-2-amd64_3.2.20-1_amd64.deb >/dev/null

With older versions of Lintian (<= 2.5.3) Perl starts to emit warnings; these have been manually filtered out. I used lintian from the git repository (i.e. I didn t install the packages, but checked out the relevant git tags). I had libperlio-gzip-perl installed (affects the 2.5.10 run). Most results are only from a single run, though I ran it twice on the first version (hoping my kernel would cache the deb for the next run). The results are:

2.5.10
real  0m28.836s
user  0m36.982s
sys  0m3.280s
2.5.9
real  1m9.378s
user  0m33.702s
sys  0m11.177s
2.5.8
real  4m54.492s
user  4m0.631s
sys  0m30.466s
2.5.7 (not tested, but probably about same as 2.5.8)
2.5. 0..6 
real  1m20s  - 1m22s
user  0m19.0s - 0m20.7s
sys  0m5.1s  - 0m5.6s

I think Bastian s complaint was warranted for 2.5. 7,8 . While it would have been easy to attribute the performance gain in 2.5.10 on the new parallelization improvements, it is simply not the case. These improvements only apply to running collections when checking multiple packages. On my machine, the parallelization limit for a package is effectively determined by the dependencies between the collections on my machine. Instead the improvements comes from reducing the number of system(3) (or fork+exec) calls Lintian does. Mostly through using xargs more, even if it meant slightly more complex code. But also, libperlio-gzip-perl shaved off a couple of seconds on binaries check. But as I said, linux-image is not your average package . Most of the improvements mentioned here are hardly visible on other packages. So let s have a look at some more other bottlenecks. In my experience the following are the worst offenders :

• unpacked (collection)
  • Seen on wesnoth-1.9 source. Here the problem seems to be tar+bzip2, so there is not really a lot to do (on the Lintian side). Though feel free to prove me wrong.
• file-info (collection)
  • Seen in eclipse/eclipse-cdt source. file(1) appears to spend a lot of time classifying some source files. For eclipse-cdt, I experience an approx. 10 second speed up (from 40s to 30s) if file are recompiled with -O2. (That would be #659355). However, even if file is compiled with -O2, the file-info collection is still the dominating factor.
• manpages (check)
  • Running man on manpages can be a dominating factor in certain doc packages. This is #677874 and suggestions for fixing it are more than welcome.

But enough Lintian for now time to fix some RC bugs!

The other day, I found myself confronted with a large number of large files. Which were presented in (gzip-)compressed ascii format---which R reads directly via gzfile() connections---as well as (compressed) numpy files. The numpy can be read very efficiently into Python. We can do the same in R via save() and load(), of course. But the trouble is that you need to read them first. And reading hundreds of megabytes from ascii is slow, no matter which language you use. Concerning R, I poked aound scan(), played with the colClasses argument and looked at the recent LaF package written just for this purpose. And all these solutions were still orders of magnitude slower than reading numpy. Which is no surprise as it is really hard to beat binary formats when you have to parse countless ascii tokens. So the obvious next idea was to read the numpy file in Python, and to write a simple binary format. One helpful feature with this data set was that it contained only regular (rectangular) matrices of floats. So we could just store two integers for the dimensions, followed by the total data in either one large binary blob, or a sequence of column vectors. But one minor trouble was that the Intertubes lead to no easy solution to unpack the numpy format. StackOverflow had plenty of question around this topic converned with, say, how to serialize in language-independent way. But no converters. And nobody local knew how to undo the "pickle" format underlying numpy. But a remote friend did: Laurent, well-known for his Rpy2 package, pointed me towards using the struct module and steered me towards the solution shown below. So a shameless plug: if you need a very experienced Python or R consultant for sciece work, consider his consulting firm. Finally, to round out this post, let's show the simple solution we crafted so that the next guy searching the Intertubes will have an easier. Let us start with a minimal Python program writing numpy data to disk:

#!/usr/bin/env python
#
# simple example for creating numpy data to demonstrate converter
import numpy as np
# simple float array
a = np.arange(15).reshape(3,5) * 1.1
outfile = "/tmp/data.npy"
np.save(outfile, a)

Next, the simple Python converter to create a binary file containing two integers for row and column dimension, followed by row times columns of floats:

#!/usr/bin/python
#
# read a numpy file, and write a simple binary file containing
#   two integers 'n' and 'k' for rows and columns
#   n times k floats with the actual matrix
# which can be read by any application or language that can read binary
 
import struct
import numpy as np
inputfile = "/tmp/data.npy"
outputfile = "/tmp/data.bin"
# load from the file
mat = np.load(inputfile)
# create a binary file
binfile = file(outputfile, 'wb')
# and write out two integers with the row and column dimension
header = struct.pack('2I', mat.shape[0], mat.shape[1])
binfile.write(header)
# then loop over columns and write each
for i in range(mat.shape[1]):
    data = struct.pack('%id' % mat.shape[0], *mat[:,i])
    binfile.write(data)
binfile.close()

Lastly, a quick littler script showing how R can read the data in a handful of lines:

#!/usr/bin/r
infile <- "/tmp/data.bin"
con <- file(infile, "rb")
dim <- readBin(con, "integer", 2)
Mat <- matrix( readBin(con, "numeric", prod(dim)), dim[1], dim[2])
close(con)
print(Mat)

That did the job---and I already used to converter to read a few weeks worth of data for further analysis in R. This obviously isn't the last word on possible solutions as the additional temporary file can be wasteful (unless it forms a cache for data read multiple times). If someone has nice solutions, please don't hold back and contact me. Thanks again to Laurent for the winning suggestion concerning struct, and help in getting the examples shown here to work.

Hope you used it well.

Already chilled by Wheezy freeze? It s been a long ride since the release of Squeeze, and your beloved FTP Team tried to assist our tireless developers and contributors at its best. Here are some hot stats to give you a figure about what happened behind the scenes. Since the release of Squeeze, 7462 .changes files with NEW components were processed by dak, with an average of 14.660 NEW packages per day. On the FTP Team side, we had 6877 accepts (13.511 per day), 641 rejects (1.259 per day) and 280 comments to maintainers (0.550 per day). This table represents the activity by single team member:

Login	Accepts	Rejects	Comments
ansgar	407 accepts (0.800 per day)	71 rejects (0.139 per day)	53 comments (0.104 per day)
dak	12 accepts (0.024 per day)	1 rejects (0.002 per day)	0 comments (0.000 per day)
dktrkranz	4319 accepts (8.485 per day)	381 rejects (0.749 per day)	104 comments (0.204 per day)
joerg	100 accepts (0.196 per day)	12 rejects (0.024 per day)	1 comments (0.002 per day)
mhy	214 accepts (0.420 per day)	14 rejects (0.028 per day)	5 comments (0.010 per day)
stew	67 accepts (0.132 per day)	16 rejects (0.031 per day)	7 comments (0.014 per day)
tolimar	1480 accepts (2.908 per day)	93 rejects (0.183 per day)	84 comments (0.165 per day)
twerner	278 accepts (0.546 per day)	53 rejects (0.104 per day)	26 comments (0.051 per day)

---

Who were the most prolific maintainers who got a NEW processing? Here is our special top ten:

1. Debian Perl Group (559 accepts)
2. Debian Haskell Group (491 accepts)
3. Debian Ruby Extras Maintainers (285 accepts)
4. Debian Java Maintainers (257 accepts)
5. Debian Med Packaging Team (164 accepts)
6. Debian Multimedia Maintainers (160 accepts)
7. Debian Fonts Task Force (156 accepts)
8. Debian Javascript Maintainers (137 accepts)
9. Debian Python Modules Team (129 accepts)
10. Debian Qt/KDE Maintainers (98 accepts)

That doesn t reflect the real developers, though. Here s our Changed-By top ten:

1. Clint Adams (216 accepts)
2. Jonas Smedegaard (208 accepts)
3. Ben Hutchings (203 accepts)
4. Joachim Breitner (153 accepts)
5. TANIGUCHI Takaki (112 accepts)
6. Alessio Treglia (101 accepts)
7. David Paleino (95 accepts)
8. Nicholas Bamber (76 accepts)
9. Mathieu Parent (68 accepts)
10. Jeff Breidenbach (63 accepts)

Clint rocks with tons of Haskell packages, followed by Jonas (mostly Perl packages), and Ben (kernel uploads). Italian cabal stands still, with Alessio and David respectively at 6th and 7th place

---

How long does a package stay in NEW? Some more, some less, but the average is 3 days, 15 minutes and 21 seconds. Now go and check your dak mails to see whether you had a fast processing or not liblog4ada 1.2-1 surely had, as it was accepted after 30 seconds! gsoap 2.7.17-1 was not so lucky, it took 103 days, 8 hours, 20 minutes and 43 seconds to clear NEW, but then made its way to the archive. Better late than never

---

FTP Team is not just accepting NEW packages, but also removing obsolete ones. Here are some details about this task:

• Source packages removed: 2159
• Binaries for architecture all: 8773
• Binaries for architecture alpha: 3646
• Binaries for architecture amd64: 14653
• Binaries for architecture arm: 3
• Binaries for architecture armel: 13283
• Binaries for architecture armhf: 3423
• Binaries for architecture hppa: 1418
• Binaries for architecture hurd-i386: 6923
• Binaries for architecture i386: 15655
• Binaries for architecture ia64: 12365
• Binaries for architecture kfreebsd-amd64: 10972
• Binaries for architecture kfreebsd-i386: 10998
• Binaries for architecture mips: 12478
• Binaries for architecture mipsel: 12479
• Binaries for architecture powerpc: 13945
• Binaries for architecture s390: 12374
• Binaries for architecture s390x: 3884
• Binaries for architecture sparc: 12295

---

FTP Team also took care of override changes:

• Number of priority changes: 853
• Number of section changes: 975

Do you ever have those afternoons you get a great idea and you've all the evening time for that task. The task is a relaxing one and won't need much attention, and you can watch a movie or something. But then, it happens that the evening turns into night as you realize a couple of little details adding complexity to the idea, and the task turns out to be much more invasive to your evening than you thought?

In this example, I got the great idea to upgrade my Debian running NAS device (thanks Martin for everything!) to use ext4 instead of ext3. The kind of idea that takes a long time for relatively little practical benefit, but it just feels like a nice thing do when you've the extra amounts of nerd time available. It's basically just opening the NAS device up, mounting its hard disk to a laptop via external case, running the tune2fs and fsck then putting the disk back. It just takes a long time for the initial fsck (to make sure everything's intact) and then the required fsck run to get ext4 mountable.

Only in this situation, it would have been beneficial to have the ext4 support in the flashed initramfs before the migration. So... before the photo below, I've already:

1. done the ext4 migration and fsck:s
2. screwed the disk back to the NAS case, attached cables and found that it doesn't boot
3. (on the laptop with the hard disk attached again tried manually unpacking initramfs and adding ext4 module... also had time to bind mount everything and chroot into the ARM system to run update-initramfs manually... also tried booting with those... until remembered the simple fact that the /boot partition is only for show and also the initramfs is loaded directly from flash)
4. copied the main root filesystem content from the original disk to another external disk with ext3 partition
5. attached the another disk (with same UUID:s) to the QNAP NAS device, booted, double-checked that I have now ext4 specified under /etc/initramfs-tools/modules, reconfigured the linux image that also regenerates initramfs and flashes it

And in the photo, what's happening is that:

1. I've again the original disk reattached and system booted with the initramfs generated and flashed from the ext3 disk
2. the NAS device is hanging in the air, cover open, from the closet where I've things stuffed in (normally secured with cable ties), and I need to support it with a knee or one hand since the 2TB disk is much heavier than the small SSD I used as the ext3 disk so the power cable and RJ-45 cable would have pretty heavy load
3. Since I've only one hand in use and can't use a laptop, I'm logging in via my Nokia N9 and then reflashing the kernel + initramfs from this original disk, just to make sure everything is now alright and also after that flashing it still boots (it does!). Note that I feel like the setup is secure enough for non-interrupted flashing so that I can indeed support the NAS with a knee, use one hand to keep N9 and another hand to take a photo with a camera.

And so we have had a productive and educating afternoon/evening/night once again. Does this ever happen to you?

Wheezy is frozen! Hooray! This also means you re probably too late in uploading NEW packages into the archive, so why don t you fix some RC bugs instead, contextually helping FTP Team keeping the queue low? Thanks, and happy bug squashing!

DebCamp starts tomorrow and the local team is relaxing and enjoying some free time with a cup of Debian Coffee. See you soon in Managua! (In case you wonder yes!!! both the Debian Coffee and the DebConf coffee mug will be for sale during DebConf12 in Managua)

I'm writing this from a hotel room near Heathrow airport, where I'll spend the night I don't like having to get up at early-o-clock in order to get to the airport, so I took a flight on the 30th in the comfortable afternoon, rather than one on the 1st in the uncomfortable morning. Of course that means I get to spend the night in London tonight. As it happens, it'll be my first time that I'll do that; I've been in London before, but it's always been when I was in transit changing planes or trains in Heathrow, or when I was on my way (by car) to Steve's last summer. But I've never been to London for London. Tonight, that isn't the case either, but at least this time I'll get to see more of this city than the airport, King's Cross and Waterloo station, the underground, and the highway network. And then tomorrow, again in the comfortable afternoon, I'll be on my way to Managua. I can hardly wait.

Lately (as I earlier pointed out on my blog) I ve been working on improving GNOME s infrastructure monitoring services. After configuring XMPP it was time to find out a good way for sending out relevant notifications to our IRC channel hosted on GIMPNET. I achieved that with a nice combo: supybot + supybot-notify, all that mixed up with a few grains of Nagios command definitions. But here we go with a little step-by-step guide: Requirements 1. Install supybot and configure a new installation:

apt-get install supybot or yum install supybot
mkdir /home/$user/nagbot && cd /home/$user/nagbot
supybot-wizard (follow the directions to get the bot initially configured)

2. Install and load the supybot-notify plugin by doing:

git clone git://git.fedorahosted.org/supybot-notify.git && cd supybot-notify
mkdir -p /home/$user/nagbot/plugins/notify && cp -r * /home/$user/nagbot/plugins/notify

Finally, load the plugin. (this will require you to authenticate to the bot) Nagios configuration 1. Add the relevant command definitions to the commands.cfg file:

# 'notify-by-ircbot' command definition
define command 
    command_name    notify-by-ircbot
    command_line    /usr/bin/printf "%b" "#channel $NOTIFICATIONTYPE$ - $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$: $SERVICEOUTPUT$ ($$(hostname -s))"   nc -w 1 localhost 5050
     
# 'host-notify-by-ircbot' command definition
define command 
	command_name	host-notify-by-ircbot
	command_line	/usr/bin/printf "%b" "#channel $NOTIFICATIONTYPE$ - $HOSTALIAS$ is $HOSTSTATE$: $HOSTOUTPUT$ ($$(hostname -s))"   nc -w 1 localhost 5050
	 

* adjust the Netcat s host and port to your needs, in my case Supybot and Nagios were running on the same host. In the case of a Supybot running on a different host than Nagios, tweak Iptables to allow the desired port:

-A INPUT -m state --state NEW,ESTABLISHED -m tcp -p tcp --dport 5050 -j ACCEPT

2. Add a new entry on the contacts.cfg file:

define contact 
 	contact_name    nagbot
   	use             generic-contact
        alias		Nagios IRC Bot
        email           example@example.com
        service_notification_commands   notify-by-ircbot
 	host_notification_commands      host-notify-by-ircbot
 

3. Reload Nagios:

sudo /etc/init.d/nagios3 reload

And finally, enjoy the result:

 PROBLEM - $hostalias/load average is CRITICAL: CRITICAL - load average: 30.45, 16.24, 7.16 (nagioshost)
 RECOVERY - $hostalias/load average is OK: OK - load average: 0.06, 0.60, 3.65 (nagioshost)

This Youtube video is an amusing satire of EULAs and copyright law as applied to uploading consciousness [1]. Washington s Blog has an insightful article about the way that the lack of trust in the US is killing their economy [2]. It seems that as long as the 1% are allowed to get away with breaking the law the US economy won t recover. It s something we should all keep in mind at election time, let s not be like the US. AirBnB is an interesting service to allow people to rent a room or an apartment, a quick scan indicates that it s a lot cheaper than hotels and offers many good locations [3]. It s probably of most interest to the more social people though which is a down side for me. Queensland s highest court has ruled that vilification of homosexuals is also vilification of bisexuals because an essential aspect of bisexuality is a sexual feeling of a person of the same sex, that is, homosexuality [4]. Anyone who didn t find that totally obvious could simply consult any dictionary or encyclopedia to find out. But the Australian legal system needed a 46 page ruling. We really need some sanity in the courts. Father Gregory Boyle founded an organisation named Homeboy Industries with the purpose of providing jobs for people with criminal records [5]. It s amazing the way he is helping people turn their lives around and it s apparently a lot cheaper than sending them to jail.

• http://www.youtube.com/embed/IFe9wiDfb0E
• http://www.washingtonsblog.com/2012/05/trust.html
• http://airbnb.com/
• http://tinyurl.com/7uc7ty8
• http://tinyurl.com/6nlex6k

Related posts:

1. Links February 2012 Sociological Images has an interesting article about the attempts to...
2. Links March 2012 Washington s Blog has an informative summary of recent articles about...
3. Links April 2012 Karen Tse gave an interesting TED talk about how to...

It's been over a week since I publicly announced the Debian mirrors redirector.
It was covered in several places, including Linux Magazin and mentioned in the latest Debian Project News.

Thanks to all that, and quite a number of early testers of the wheezy image for the Raspberry Pi, http.debian.net has processed over one million requests from APT clients alone. This figure doesn't count users of apt-cacher/apt-cacher-ng/approx, or those who happen to be mirroring Debian with wget. Oh, and this is only the number of requests since last Sunday, when the logs were rotated.

By the way, if you are going to Debconf12 in Nicaragua, why don't you give http.debian.net a try and forget about switching mirrors?

I'm happy to see how it was welcomed and the many emails, blog comments, IRC messages, and other sorts of messages I've received about it. There are surely some rough edges but the service is still under development and all the feedback helps.

In some cases there appear to be some confusion as to how exactly it chooses the mirrors and why sometimes some mirrors are not used. I'll try to cover some of those topics in this blog, hoping the

P.S. If you've sent me an email about http.d.n and I haven't replied, please rest assured I try to take a quick look at them but I can't reply to every single message fast enough.

Thanks to everyone!