It's been a couple of very interesting days. The thank.debian.net site was quite a success, and a lot of credit is due to Valessio's awesome balloons. Even though I didn't do much marketing about it, the site was linked on Slashdot, LWN, Ubuntu's and Debian's website, Planet, the BTS, and quite a number of blogs. We received almost 3000 thank you messages, from all over the world. Even though the site was in English, many people felt inspired to leave their thank you messages in their languages, if you browse over the site, you'll find messages in Turkish, Chinese, Russian and many other languages. On that matter, I was particularly amazed at this message in persian, which displays properly even inside mutt. Not too surprising, one of the things that people most value of Debian is the package management. The apt team got a lot of thank you messages, and many of the ones directed to the whole community also mention apt and upgrades as something very valuable I find it very interesting that many of the messages, came from Ubuntu users, stating that even though they use Ubuntu, they really value the work done by Debian. A number of posts also mention sidux, knoppix and mint among the many derivatives. Many people stated the many years that they use Debian, some of them have just started, some have been using Debian for quite a while.
I was happily surprised by a number of posts that included a thank you message to "guys/gals" or "guys and girls". And the image used in the "hackergotchi" of this message really brightened my day. Finally, I'm sorry that some people were annoyed by the thank you messages. For next year, we can try to do something a bit different and hopefully not annoy those who were annoyed this time. I do think that once in a while it's nice to be able to receive some of that much love that the users have for Debian and we rarely get to know about.

I've been talking about replacing my laptop since November. Now I've somewhat forced the issue by giving away my R200, leaving myself with just the EEE 901. I'm not planning to bring a desktop with me to California (well, I might ship bits for a media box, but probably not even a complete machine), so I've been looking more seriously at what my options are.

I like smaller laptops; the R200 is 12", I had a Compaq N200 previous which was 10". There seem to be a number of 13" options that are light, so I'm prepared to look at those.

This is going to be my main machine, so I need something with some grunt. A decent amount of RAM with a reasonable processor. I'm probably prepared to take the price hit in order to get SSD. Also a decent resolution screen along with built in wifi (does anything not have this these days?) + 3G would be nice too.

Of course, it turns out I can't have all of this. In particular the US market seems pretty dire for 3G support (no, something locked or that can't take a GSM SIM doesn't count. I expect to travel.) The UK market is a bit better, but there are still no perfect answers.

Lenovo X201s

(It's really hard for me to think of this as a Lenovo rather than an IBM Thinkpad.)

This comes in a 1440x900 resolution variant, which is nice for 12". I can have a touchpad too. What I can't have is 3G; with the 1440x900 screen there's apparently no space for the 3G antenna. Doh! Also Thinkpads are *ugly*.

Sony Vaio VPCZ12Z9E/X

I want this. Core i7, 1920x1080 screen, SSD, 3G (Gobi), sub 1.5Kg.

But. Sony. Ridiculously expensive. If it was half the price and made by Lenovo or Toshiba I'd have bought one by now. As it is there's no way I can reasonably expect it to last 3 years (and really I'd want 5 given the cost).

Toshiba R700-155

I've been extremely happy with my R200 and spent a long time eying the R500/R600. The -155 is a Core i7, 1366x768 screen, 13", 3G, sub 1.5Kg.

2 problems. One, I'd like a better resolution if I'm going to 13" - both the Lenovo and the Sony manage that. Secondly, and more of an issue, nowhere seems to have them in stock.

HP Elitebook 2540p

This has potential. Seems to be a little heavier than the other options, and it's only a 1280x800 screen (but in 12"). Available with Core i7, SSD & 3G. Oh, except once again stock is a problem.

Have I missed anything else out there? I can probably buy from the UK or US with equal ease.

After reading Do It Tomorrow, I was inspired to try some of the techniques that Mark Forster proposes. I commented in the review about the closed list system, which I'm still mostly using with some success. But I also started playing with the initiative slot. In his system, the initiative slot is first thing in the morning each (eligible) morning, which is used for a special project that you want to make forward progress on. There are several criteria for choosing a task for the initiative slot. One of the strongest is dealing with a backlog. Since I'd been feeling guilty about the backlog of Debian Policy work that had built up, that seemed like a good choice. I picked up Debian Policy backlog clearing as my initiative starting at the beginning of June after the AFS and Kerberos Best Practices Workshop, with a goal of getting the open bug count below 100. This has been very instructive, both from a time management perspective and from a Debian Policy perspective. Here are a few things that I learned on the Policy side:

• The open bugs against Debian Policy by and large represent real work that we should tackle in the Policy document. They're not all fully thought-out proposals, but I didn't find much that I could simply close (apart from a set of bugs requesting additional common-licenses, once I had an analysis tool). From time to time, we've thought about timing out old bugs that have not made any forward progress, but this experience leads me to think that would serve us poorly.
• It takes me about an hour on average to resolve a Policy bug. That includes all the fast five-minute typo bugs (well, ten minutes; I need to improve my Policy workflow to be a bit more efficient in the commit process). Some bugs take easily ten hours or more, such as #504880. I think this is on par with or a bit slower than resolving typical bugs in compiled software I package, and much slower than Lintian, where bugs take about a half-hour on average to resolve, I think.
• Most Policy bugs are making no forward progress because no one has written wording and started iterating on that wording. There are some exceptions, and this process seems to go better for me than it has for some other contributors, but if I pick up a bug and start drafting wording changes, I can usually bring the bug to resolution in about a week, often less. This implies to me that our process for making Policy changes is reasonable but we need more targetted help. This was discussed at the BoF at DebConf and on the list afterwards and I now have three people who have volunteered to supply some of that targetted help and need to get more organized to make use of that offer.
• The normative (rule-affecting) versus informative (informational and documentary) distinction in Policy is tricky and unclear, and most bugs end up becoming normative. While I think our system for dealing with normative bugs is reasonable, this does slow down fixes. I can commit informative changes without review and then tweak them if people object, and it would be nice to fix more wording issues that way, but it's hard to avoid getting tangled up in normative issues.

Things I learned on the time management side:

• I didn't work from a closed list, which is frowned on for dealing with backlogs. I'm not sure if this ended up being a problem or not. The correct approach, in theory, is to work out a system for dealing with new incoming work, make sure that system is in place, and then process backlog independently of all new work. I never did that. I never separated out the backlog into a closed list that would get smaller. I suspect I should have, although I'm not sure of a good way to do that in the BTS in an area where we're already using usertags for another purpose.
• The point of the initiative slot is to unblock something or to clear a backlog, but the initiative slot is supposed to rotate. It's not a good way to deal with something that's going to take a long time to resolve. In retrospect, I was excessively ambitious with my goal. Even apart from not working from a closed list, I can average about a half-hour a day on the initiative slot, which at an hour a bug and a starting point of 160 bugs would mean 120 days to reach my goal. That's too long for one task to dominate the initiative slot.
• The initiative slot really works. We brought the bug count on Policy down from about 160 to 126 over the course of those two months, and I don't think much changed other than my level of activity and of course the normal effect of other people getting more involved when someone is visibly doing work. The bug graph is impressive. It's very obvious when I started working, and also very obvious when the ramp-up to DebConf started and I stopped having time to have a daily initiative slot.
• Doing something that's unrelated to my day job in the initiative slot creates an odd tension, and I'm not sure how to deal with that. On one hand, one of my standard time management failure modes is to spend too much time on work and not enough time on non-work tasks. On the other hand, that's partly because I'm hypersensitive to not focusing enough on work and overcompensate, and taking time each morning to do non-work things doesn't help that. I'm not yet sure what to do about this.
• One of the failure modes of initiative slots that I'm already noticing is that when I move something out of that slot, rather than continuing to work on it at a normal pace, I have a tendency to stop working on it altogether. This doesn't work, particularly if it was used for backlog clearing. I just build up another backlog and then have to use the slot to clear it again. This means I need to give more attention to my time management outside of that slot.

I stopped using the initiative slot while travelling and haven't gotten back into the hang of regular time management since I've gotten back. I haven't yet decided whether to leave Policy work in the initiative slot for right now but change the goal, or whether to swap it out for something else. I'd like to keep momentum, but I'd also like to start using that slot for other things. Still pondering.

Last night, for the first time, Harry Dean Stanton sang backup vocals on The Weight.

Dear Debian, today you turn 17 ... ... happy birthday! I've already argued how you're in very good health now that you're close to the major age and how unique and important you are for the whole ecosystem of free software. So, no, I won't insist on that here. Still, today sounds like a good day to say thank you, Debian for being here promoting software freedom and showing to the world that all this can be done do-ocratically, democratically, relying on volunteer work and donations. If you too think Debian deserves being thanked for what it is giving to the world, just say thank you, Debian . My first (meta) thank you messages go to Marga, Maxy, Valessio, Raphael, Ana, and Rhonda for ideating and spreading the 1st Debian Appreciation Day ever. Update: fix 404 Penta b0rkage in the slides link

I have just read The Gift of Fear and Other Survival Signals that Protect Us From Violence by Gavin de Becker. Like many self-help books it has a concept that can be described in a paragraph and explained in a few pages. The rest of the book shares anecdotes that help the reader understand the concept, but which are also interesting for people who get it from the first chapter. When I read the book I considered the majority of the content to be interesting stuff added to pad it out to book size because the concept seemed easy enough to get from the start, but from reading some of the reviews I get the impression that 375 pages of supporting material aren t enough to convince some people maybe this is something that you will either understand from the first few chapters or never understand at all. Gavin s writing is captivating, he has written a book about real violent crime in a style that is more readable than many detective novels, from the moment I finished the first chapter I spent all my spare time reading it. I was a little disappointed at the lack of detailed statistics, but when someone has done all the statistical analysis chooses to provide the results in the form of anecdotes rather than statistics I m prepared to tolerate that especially when the anecdotes are so interesting. I spent quite a bit of time reading the Wikipedia pages relating to some of the people and incidents that are mentioned in this book. The basic concepts of his book are to cease worrying about silly things like airline terrorists (passengers won t surrender now so that s not going to work again) and to instead take note of any real fear. For example if you are doing the same things you usually do but suddenly feel afraid then you should carefully consider what you might have subconsciously noticed that makes you feel afraid and what you can do about it. This isn t going to change my behavior much as I have mostly been doing what the book recommends for a long time. I think that everyone should read this book.

Debconf10 has come and gone and its effects are still with me. This is not so surprising as this year I was not only a participant but on the local team. When we first started to hold in person meetings, I was a totally overwhelmed knowing that with each passing month, the demands on my time would grow and then explode. Then I got news that I landed a year long fellowship that would relieve me of all teaching for the 2010-2011 year and was able to breathe a little and enjoy what I don t get to do all that often in my academic job: work collectively. As Debcamp started, I was not sure whether I would enjoy the conference given that I had to pay attention to the nuts and bolts of the conference. I got my answer the first day as folks started to pour in hack lab. Due to the pressure cooker nature of my job, I did not attend the last two Debconfs. Seeing friends that I only see during these fairly intense weeks stirred up a lot of inside of me. It felt really good to be back . This was also my first Debcamp, and now I understand why people enjoy it so much. It gives you a chance to meet new folks instead of gravitating to those you already know and still get some work done. It also allows the orga team to hammer out some final details and test run some stuff before the swarm descends into the venue. One of the things I enjoyed the most during the week were the take out dinners. Logistically it was a bit of nightmare but it aligned very much with the culinary culture of NY and we had some nice outdoor picnics on the Columbia campus and one in Riverside Park. The pace picked up a lot during Debconf, which we kicked off with Debian Day. We managed to get a lot of NYC folks it was one of the best attended Debian Days. One of the highlights was Gale Brewer s talk. She is a public servant in the full sense of the word, dedicating thirty years of her life to this job and really understands what technology can and cannot due for New Yorkers. It was very inspirational talk. Aside from a few moments, such as the day trip and the wine and cheese party (btw, party does not capture the quantity of cheese featured and consumed ), Debconf itself has already become a bit of a blur, it all happened so quickly and under sleep deprived conditions. Nearly every vessel of time was filled, overflowing, at times bursting. I enjoyed a number of the talks Eben Moglen s, Joey Hess s CUT proposal, Marga s talk on Making Debian Rule, Zack s Bits from the DPL, the derivates BOF, and others. I also enjoyed preparing and moderating the welcome talk, which is not yet online (only the last bit is and the sound is distorted so I sound like a mouse). My sense is that the relationship between Ubuntu and Debian is improving, which is important. We need alliances and allies, not foes in the world of F/OSS (watch Eben s talk to get a sense of why this is so). I mostly ate on campus but managed two African based dinners with friends (Ethiopian and Senegalese) and only wished we had a little more time to talk. The late nights in the Carman lounge were pretty epic, if somewhat collegiate and during the end, hot and humid. Here are a few of the pictures I managed to take. I love Debconf for it so perfectly conjoins the comfort of the familiar with the thrill of the new. For those who experience it in its full intensity, it can be an artistic moment, at least in the way described by the poet John Keats: The excellence of every Art is its intensity, capable of making all disagreeables evaporate, from their being in close relationship with Beauty . . . And now, I am, of course, totally beat

Debconf10 has come and gone and its effects are still with me. This is not so surprising as this year I was not only a participant but on the local team. When we first started to hold in person meetings, I was a totally overwhelmed knowing that with each passing month, the demands on my time would grow and then explode. Then I got news that I landed a year long fellowship that would relieve me of all teaching for the 2010-2011 year and was able to breathe a little and enjoy what I don t get to do all that often in my academic job: work collectively. As Debcamp started, I was not sure whether I would enjoy the conference given that I had to pay attention to the nuts and bolts of the conference. I got my answer the first day as folks started to pour in hack lab. Due to the pressure cooker nature of my job, I did not attend the last two Debconfs. Seeing friends that I only see during these fairly intense weeks stirred up a lot of inside of me. It felt really good to be back . This was also my first Debcamp, and now I understand why people enjoy it so much. It gives you a chance to meet new folks instead of gravitating to those you already know and still get some work done. It also allows the orga team to hammer out some final details and test run some stuff before the swarm descends into the venue. One of the things I enjoyed the most during the week were the take out dinners. Logistically it was a bit of nightmare but it aligned very much with the culinary culture of NY and we had some nice outdoor picnics on the Columbia campus and one in Riverside Park. The pace picked up a lot during Debconf, which we kicked off with Debian Day. We managed to get a lot of NYC folks it was one of the best attended Debian Days. One of the highlights was Gale Brewer s talk. She is a public servant in the full sense of the word, dedicating thirty years of her life to this job and really understands what technology can and cannot due for New Yorkers. It was very inspirational talk. Aside from a few moments, such as the day trip and the wine and cheese party (btw, party does not capture the quantity of cheese featured and consumed ), Debconf itself has already become a bit of a blur, it all happened so quickly and under sleep deprived conditions. Nearly every vessel of time was filled, overflowing, at times bursting. I enjoyed a number of the talks Eben Moglen s, Joey Hess s CUT proposal, Marga s talk on Making Debian Rule, Zack s Bits from the DPL, the derivates BOF, and others. I also enjoyed preparing and moderating the welcome talk, which is not yet online (only the last bit is and the sound is distorted so I sound like a mouse). My sense is that the relationship between Ubuntu and Debian is improving, which is important. We need alliances and allies, not foes in the world of F/OSS (watch Eben s talk to get a sense of why this is so). I mostly ate on campus but managed two African based dinners with friends (Ethiopian and Senegalese) and only wished we had a little more time to talk. The late nights in the Carman lounge were pretty epic, if somewhat collegiate and during the end, hot and humid. Here are a few of the pictures I managed to take. I love Debconf for it so perfectly conjoins the comfort of the familiar with the thrill of the new. For those who experience it in its full intensity, it can be an artistic moment, at least in the way described by the poet John Keats: The excellence of every Art is its intensity, capable of making all disagreeables evaporate, from their being in close relationship with Beauty . . . And now, I am, of course, totally beat

My project. This part of this post assumes that the reader has read my last post. This post contains embarassing news: In my effort to organize the patches to make it easier to run a script that benchmarks each change to see how it affects performance, I made a very simple mistake, which mostly would disable Immix. This was done in the first patch of the stack, so all patches were compromised. I noticed this while I was attending Simon Marlow s request of measuring how many lines are being freed and allocated, and notice that it was none. My error was simply to set two different constants with the same value: BF_MEDIUM and BF_SWEPT. After the correction, I noticed two more errors that were introduced in this period, and that were not corrected because, as Immix was not activated, they were not causing any harm. Now, with everything in the stage I was before: a memory corruption that happens ocasionaly only in the nofib/real/maillist benchmark. As described in this link, I ve tried already a lot of different techniques to debug this one, but it s being hard. In the mean time, I m measuring again the stack of patches, in a lab using the complete nofib (except for nofib/real/maillist) and nofib/gc, and in my laptop only fibheaps, fulsom and constraints, from nofib/gc, which were used in my last post.

---

Debconf was great. I ve stayed in New York with my family doing tourism for some days before it, and New York city is fantastic. A very diverse architecture and people from all around the world. It s really impressive. I arrived in DebCamp on Friday and it was not easy for me to find where to check-in, and where to go after unpacking my stuff. I assume I haven t paid much attention to the e-mails explaining this, but I tried to search them when I got there, without success. It was good, on the other hand, to walk on Columbia searching: it s a very nice campus, not comparable to anything we have here in Brazil. During DebConf I tried to help the organization as much as I could, being Talk-meister in some talks, helping a little bit with food organization and participating in the video team. All of this was very fun. There were some talks that I couldn t attend to, and I watched the videos, or part of them, here on Monday: Making Debian rule again, FreeBSD, Anti-features I still want to watch some, like the Analysis of Debian mailling lists using machine learning, and there are others that I wanted to see but are not recorded, like the one about notmuch. I got my key signed by a lot of people, which certainly was not the case before, and it the GPG best practices BoF was great. And there was the proposal to host DebConf12 in Brazil, maybe in my city, Belo Horizonte. Which reminds me: Tomorrow Betim will host the Debian Day BH, in parallel to another event: FSL-BH. A BoF related to the organization of DebConf12 in Belo Horizonte is scheduled. See you there.

I am betting that Luakit will be more usable than Uzbl within a week. I am also betting that I will hear chirping about Luakit not integrating with the Monkeysphere, and though I agree that this is a problem, it seems to be difficult to solve. To do it correctly, one would need hooks or callbacks in WebKitGTK+, which, in turn, would need support in LibSoup, which would probably then need support in GnuTLS. Since apparently this will take years of bickering to accomplish, perhaps some sort of hack is called for. I suppose one could, for each HTTPS URL visited, make an OOB connection, download the certificate, and then run it through Monkeysphere validation, before then letting WebKitGTK+ make another connection and download (hopefully the same) certificate a second time. This strikes me as ugly and problematic. Ideas?

dctrl-tools 1.14 (targeting squeeze) has the following incomplete translations (as of right now in git): ca: 89 translated messages, 4 fuzzy translations, 18 untranslated messages.
cs: 108 translated messages, 1 fuzzy translation, 2 untranslated messages.
de: 111 translated messages.
en_GB: 89 translated messages, 4 fuzzy translations, 18 untranslated messages.
es: 89 translated messages, 4 fuzzy translations, 18 untranslated messages.
fi: 111 translated messages.
fr: 108 translated messages, 1 fuzzy translation, 2 untranslated messages.
it: 65 translated messages, 8 fuzzy translations, 38 untranslated messages.
ja: 89 translated messages, 4 fuzzy translations, 18 untranslated messages.
pl: 49 translated messages, 2 fuzzy translations, 60 untranslated messages.
pt_BR: 89 translated messages, 4 fuzzy translations, 18 untranslated messages.
ru: 108 translated messages, 1 fuzzy translation, 2 untranslated messages.
sv: 84 translated messages, 4 fuzzy translations, 23 untranslated messages.
vi: 89 translated messages, 4 fuzzy translations, 18 untranslated messages. I have put the relevant pot and po files up. This is not an archival URI, but I ll keep it available long enough for this. Submissions through the BTS are accepted, as usual. Debian developers and others with collab-maint access may, if they wish, push their updates directly to the Git repository. Please use the maint-2.14 branch and please read the README. I will NOT be gathering translations from Rosetta. All contributors and committers are asked to declare whether they can affirm the Developer s Certificate of Origin. The commit tag Signed-off-by is, by convention, interpreted as an affirmation of that document by the person identified on that tag line.

The unofficial rc bug tracker currently knows about the following bugs:

In Total:	448
Affecting Squeeze:	329
Squeeze only:	108
Remaining to be fixed in Squeeze:	221

Of these 221 bugs, the following tags are set:

Pending in Squeeze:	24
Patched in Squeeze:	43
Duplicates in Squeeze:	19
Contrib or non-free in Squeeze:	11
Claimed in Squeeze:	2
Delayed in Squeeze:	8
Can fixed in a security Update:	12
Otherwise fixed in Squeeze:	21

Ignoring all the above (multiple tags possible), 121 bugs need to be fixed by Debian Contributers to get Debian 6.0 Squeeze released. However, with the view of the Release Managers, 271 need to be dealt with for the release to happen. Please see my older post for an explanation of the different numbers.

1. Create an account on Flattr.com (mine is here);
2. Submit the URL of your free software project to the Flattr directory with a good description and the proper tags (here s my entry for dpkg);
3. Add a Flattr button on your website (I put one on wiki.debian.org/Teams/Dpkg);
4. Inform your users and reach out to free software users using Flattr by registering your project in Flattr FOSS.

And I m not alone, Joey Hess made 25 EUR in one week only. And right now only 30K people are using Flattr. I expect this number to increase significantly now that invites are no longer required to open a Flattr account (see the announce on Flattr s blog). It s time to spread the word about Flattr FOSS! 8 comments Support my work

Already available in both Maverick and Debian sid, it provides many interesting features. I m talkin about Font Manager, a small application written in C and Python by Jerry Casiano, which allows users to easily install, remove and compare font files on own system. Here are few nice screenshots:

To install the application, as usual type:

sudo apt-get install font-manager

Let me know what you think!

Links

• Project s homepage
• Debian PTS page

As far as I recall the standard for naming Linux Ethernet devices has always been ethX where X is a number starting at 0. Until fairly recently the interface names were based on the order that device drivers were loaded or the order in which the PCI bus was scanned. This meant that after hardware changes (replacing network cards or changing the BIOS settings related to the PCI bus) it was often necessary to replace Ethernet cables or change the Linux network configuration to match the renumbering. It was also possible for a hardware failure to cause an Ethernet card to fail to be recognised on boot and thus change the numbers of all the others! In recent times udev has managed the interface naming. In Debian the file /etc/udev/rules.d/70-persistent-net.rules can be edited to change the names of interfaces, so no matter the scanning order as long as an interface retains it s MAC address it will get the correct name or at least the name it initially had. One of the down-sides to the way this operates is that if you remove an old Ethernet card and replace it with a new one then you might find that eth1 is your first interface and there is no eth0 on the system this is annoying for many humans but computers work quite well with that type of configuration. I ve just renamed the interfaces on one of my routers by editing the /etc/udev/rules.d/70-persistent-net.rules file and rebooting (really we should have a utility like /sbin/ip with the ability to change this on a running system). I have decided to name the Ethernet port on the motherboard mb0. The PCI slots are named A, B, and C with A being the bottom one and when there are two ports on a PCI card the one closest to the left side of the system (when viewed from the front the right side when viewed from the read) is port 0 on that card. So I have interfaces pcia0, pcia1, pcib0, pcib1, and pcic0. Now when I see a kernel message about the link going down on one of my ports I won t have to wonder which port has the interface name eth4. I did idly consider naming the Ethernet devices after their service, in which case I could have given names such as adsl and voip (appending a digit is not required). Also as the names which are permitted are reasonably long I could have used names such as mb0-adsl, although a hyphen character might cause problems with some of the various utilities and boot scripts I haven t tested out which characters other than letters and digits work. I may use interface names such as adsl for systems that run at client sites, if a client phoned me to report Internet problems and messages on the console saying things like adsl NIC Link is Down then my process of diagnosing the problem would become a lot easier! Does anyone else have any good ideas for how to rename interfaces to make things easier to manage? I have filed Debian bug report #592607 against ppp requesting that it support renaming interfaces. I have also filed Debian bug report #592608 against my Portslave package requesting that provide such support although it may be impossible for me to fix the bug against Portslave without fixing pppd first (I haven t looked at the pppd code in question for a while). Thanks to Rusty for suggesting this feature during the Q/A part of my talk about Portslave for the Debian mini-conf in LCA 2002 [1].

• [1] http://www.linux.org.au/conf/2002/debiancon.html

At the beginning of DebCamp, Zack started the RCBC the Release Critical Bug squashing Contest. Two weeks later not only DebConf10 is over but we also have results for the RCBC:

In summary: It was a huge success, we managed to fix

• 139 RC bugs in testing/unstable
• 79 RC bugs in stable

And the winners are:

• Stable: Gerfried Fuchs (77 bugs)
• Testing/unstable: 1. Moritz M hlenhoff (32 bugs), 2. Nico Golde (17 bugs), 3. Alexander Reichle-Schmehl (16 bugs)

All details can be found at http://wiki.debconf.org/wiki/DebConf10/RCBC.

The winners who haven't collected their prizes at the closing ceremony will get them be snail mail.

Finally let me thank

• our donors of the prizes
• Micah and Zack for initiating the contest
• Mehdi for being my co-judge
• all participants who hugely improved Debian in the last two weeks

My project. This post assumes that the reader has read my last post. I ve been trying to find a reasonable method (which computer to use, single user or multi user) to benchmark for some time, and now an e-mail from Simon Marlow calmed me down. He suggests that in this stage it s not necessary to run a long benchmark, but only a short one, that will be enought to have an idea about what each change is causing to performance. For some time I was struggling with the benchmarks to get a reasonable result. I ve started more than once to write a blog post analysing the results I ve got, until in the middle of the post I notice the results are not consistent. This usually happens when there s a big distortion in benchmarks that shouldn t present no change. For instance, in a patch that only changes code that is executed with +RTS -w -RTS, there was a difference of 50% on execution time in default mode, that is, without +RTS -w -RTS. I think that only disabling CPU scaling and running a small number of tests, which make it easier to reproduce a possibly inconsistent result, will be enough to get a good data. I ve made a stack of patches, splitting each part of my work. The stack does not reflect my preferences between the patches, that is, the complete stack applied is not what I think that is the best change to GHC. It s only a set of patches applied linearly, in a way that it s easy to see the impact in performance of each change. The patches are: Don t check for swept blocks in -DS.

The checkHeap function assumed the allocated part of the block contained only alive objects and slops. This was not true for blocks that are collected using mark sweep. The code in this patch skip the test for this kind of blocks.

This patch is only needed to make it possible to run the binary with the parameters +RTS -w -DS -RTS. I ve already explained this patch. I didn t measure the difference in performance before this patch, since it only changes code that is not going to be executed in the benchmarks, since none of them are runned with -DS. Immix: allocate and free memory on lines. This is the main patch with the initial implementation of Immix. As it is the initial implementation, it has it s problems, that will be treated in the following patches. The comparison was done using 3 programs from nofib/gc: fibheaps, fulsom and constraints, as suggested by Simon Marlow. Two tests were made for each version of the compiler code: one using the default GC strategie (copy-collection with mark-compact if there s few heap space left), another one using mark-sweep (or Immix after my changes). When comparing the default GC strategie between before and after this change, the program got 0.4% slower, with the collection of the generation 0 being 0.1% slower and the collection of the generation 1 0.4% slower. Comparing the mark-sweep with Immix, it got 13.7% slower, 15% in GC0 and 7.2% in GC1. The change in memory used is irrelevant. Chose between allocating in line or blocks in todo_block_full The first improvement I ve made is to change the place where the allocation was searching for free lines. Most of the changes were done in the function alloc_for_copy(), in rts/sm/Evac.c. This is bad because this function is called a lot of times, and should be kept fast. So I moved them to ctodo_block_full() in rts/sm/GCUtils.c. It got about 9.9% and 9.6% faster in default and sweep, which makes it 1% faster and 2.8% slower than in the original code. Improvements in sweep() I was bothered that the code I ve written to free memory in lines was messy, and I thought it could be simplified and maybe even turn faster. The general execution time increased 0.4%, but the GC1 time, which is the only one that should be affected, was reduced by 0.6%. In the default GC strategie, there was also a 0.2% increase in time, so I guess this is inside the margin of change. I have to investigate better on this one, but it seems to have improved slightly the performance. Line before inscresing block size. In a conversatio with Simon Marlow, we thought about these two options: one is to search for a line after trying to increase the block size, and that s what I did in Chose between allocating in line or blocks in todo_block_full; the other is trying to search a line first, and just increase the block size if a block (and not a line) was being used for allocation and there is no line available. With this patch I ve changed it to use the second one. Using sweep, the GC time got 0.5% slower, and with default it got 0.1%. rts/sm/Sweep.c: Mark all BF_MEDIUM blocks as BF_FRAGMENTED In my first implementation, the blocks that contain objects bigger than a line (of medium size) are marked with BF_MEDIUM, and are treated as the usual block in mark-sweep: if the block is empty, it s freed; if it s very fragmented, it s marked to be collected using copy-collection. In this patch, instead of making it use the rules of mark-sweep, I just mark it as fragmented and make it be collected by copy-collection. This made the GC0 time be reduced by 3%, but caused an increase of 509.7% in the GC1 time. This was also the first time the memory used was reduced, by 25.4%. The results in the default mode are insignificant. What it seems to me is that the copy-collection code is more efficient with memory usage, but also more slow. Marking objects as BF_FRAGMENTED make this algorithm be used even for full objects. rts/sm/Sweep.c: Don t use 3/4 heuristics to mark as BF_FRAGMENTED The mark-sweep algorithm considered that an object was fragmented if more than 1/4 of the word groups are completely unmarked. I kept using this heuristic, and this change removes it, making more use of the Immix, and less use of copy-collection. This didn t change the memory used, and made the code 10.4% slower in sweep. In default GC mode, there was no relevant difference, as expected. Allocate in lines in minor GCs. In the initial implementation I was only allocating on lines only in major GCs, because I needed the mark stack, which was only available in these kind of allocations. In this change, I created the mark stack on all GCs, and used the allocation on lines. The results for the default mode are insignificant, and the code got 9.0% faster in sweep, using the same memory. Selection I ve made a selection with the presumably better patches, which are: Don t check for swept blocks in -DS, Immix: allocate and free memory on lines, Chose between allocating in line or blocks in todo_block_full, Improvements in sweep() and Allocate in lines in minor GCs. This is an attempt to achieve the best set of patches possible, to see how it improved the original code. Comparing the default strategie for this selection and the original code, it got 3.7% slower. Comparing both sweep, it got 3.9% slower. Comparing the original default with the final sweep, it got 4.2% slower and uses 4.7% more memory. There s a lot of room for improvement, and I m willing to hear suggestions of what I could change in the code to achieve this improvement.

---

The results are available. Every time I had doubt about a comparison, I d run both versions again to check. This is way there are some backup files. The data presented here is not from all the most recent versions of each measurement, but from the ones that I thought were more similar in conditions.

(This blog has slept since I wrote the previous post. That post had to wait for a year to be made public! Anyway, here I go again ) One of the ways people characterise science and technology is by saying that science is about prediction and technology is about design. This is not wrong but I feel that it misses something. An astrologer too claims to predict the future and claims to tell you how to re-design your future. So the above explanation of science and technology could make it indistinguishable from magic! One thing that does distinguish science is that the prediction and design is done by calculation ; this is not restricted to mathematical calculation but includes recipes like mix so much of this reagent with so much of that solvent . In fact, the notion of calculation can be expanded to anything that I can teach someone else to do even an idiot like a computer. This leads us to the view that science is not only about making predictions and designing things but also about teaching others to do the same. At the very first level, this is done by providing formulas which other people can use to make predictions and design things. However, this is not quite far enough, since it would still make the act of creating these formulas a magical thing! So we must take this a step further and be able to teach others how to create their own formulas. In other words, we want to provide formulas for creating new formulas. It does look as if we could be in infinite regression here, for who will then create formulas to provide formulas for creating new formulas! Luckily, the monadic thinking of Turing helps us here. He realised that formulas for creating new formuals are themselves just (slightly more complicated) formulas. Just as a gift-wrapped box containing a gift can be thought of as just a box containing a gift.

What do you do when you have a technical question that you're embarrassed to ask? The first Sunday of Debconf, I led a birds of a feather (BoF) session called Debian for Shy People. The conference team scheduled it on "Debian Day," a pre-conference day that was open to the public and still had plenty of Debian Developers in attendance. I just uploaded the slides to the "Penta" page for the talk. I led it because of my own experience. In 2004 or so, I saw Debian as the cool kids' club, that awesome project that I wished I could be a part of. By 2006, I managed to get over myself, read the New Maintainer's Guide, and find a way to get involved. As of mid 2009, I am a full-blown Debian Developer. I have real ultimate power. But I sometimes do still feel hesitation akin to "Imposter Syndrome". (A bunch of people at Debconf didn't really believe I'm "shy," since I asked a lot of questions at the conference. At core, I don't naturally believe that the things I say are worth hearing, but I patch over this hesitation. Sometimes I speak too much, and then I feel ashamed of burdening everyone. But anyway, this is about Debconf not, me -- so moving on....) In the past year of being a Developer, one thing I've seen is that other contributors ask me privately for help. Rather than blast the public lists like debian-mentors, they email or IRC private-message me, or SMS me, or find me at a Linux Users Group event. I'm lucky to know these people, and they're lucky to have me as a safe person to ask questions of. Moreover, Debian is better because these people could move past their confusion to make a technical contribution. I began the BoF session by talking about when someone asked me for help. Then I asked, "How many of you have someone you can ask embarrassing questions of?" Of the forty people crammed into Schapiro 414, two people' raised their hands. One person put it plainly, "I don't know anyone else who does Debian." It reminded me of a fact that Karen discovered when she was doing market research for us at OpenHatch: the vast majority of free software programmers know zero other people who do free software. I had seen the figure; we even used it in a talk to try and convince venture capitalists to fund OpenHatch last year. But I didn't really feel it until I heard it from a room full of Debian contributors. I structured the BoF in two parts: First, I talked in front of some slides to set the tone properly, and then we enjoyed open discussion. As I was preparing thoes slides, Daniel Morais asked me, "What's the point of having the session? Why not just come up with some ideas, implement them, and not bother also talking about it at the conference?" I had considered this; I decided I wasn't self-confident enough to start implementing ideas without talking to people to make sure I wasn't the only one who saw a problem. But I discovered another benefit of giving the talk: people who want to make Debian more welcoming knew to reach out to me. So here are some thoughts that came from our discussion (and later discussions during the conference):

• We need better mentorship in Debian. Truly, mentorship is a personal relationship. Perhaps matching people up into one-on-one groups would be a good idea. KiBi said that doing so using OpenHatch (if we add mentorship-oriented features!) would be a reasonable way to do it.
• I've heard from quite a few people who would be willing to have more one-on-one relationships with mentees.
• Everyone (except Manoj) knows that Debian contributors are not born experts. It takes a process of learning and teaching to bring them up to speed.

I set up an Etherpad document on cjb's OpenEtherpad.org. This is what we learned together:

• "If your level of understanding might be low, but you want development help, then you might not know if #debian-devel is only for people who are already super involved and know everything and therefore you should feel embarrassed."
• People get anxious over their ability to speak English.
• Bikeshedding over patches can be a major detractor.
• "Asheesh talks too fast."

One idea I had before the BoF was to create a discussion area that was safe for all questions, even if they seem silly. We talked for a while about what name that would take, if it were to become a new IRC channel. We reached something of a conclusion, but in the conference that followed Emmet Hickory offered to help make the debian-mentors IRC channel friendlyer. I think that's the best direction to take things, so the next step is for him and me to write up what we want and send a note to the debian-mentors email list explaining our vision. In the Etherpad document, people discussed the idea of doing Debian discussion over XMPP (also known as Google Talk, also known as Jabber). We weren't sure how such a place would get critical mass; someone briefly mentioned the idea of an IRC/XMPP gateway. I actually think this discussion is along a very reeasonable path, namely discovering what discussion method(s) Debian contributors want to use. (That might explain why I'm now an admin on forums.debian.net.) We also briefly discussed the idea of an anonymous question-answering service. I realize now that I'm not going to be able to have time to run that, but I still think it'd be a really cool idea. Biella would remind me that Debian is already successful at bringing in new contributors. I agree! As a free software project, we have an enormous number of participants. This is a really good thing, and we're clearly doing something right. The purpose of this talk was to figure out how to make contributing to Debian less stressful for those who participate. Truly, a "Debian for Shy People" effort isn't about shy people. It's about the moments of self-doubt we all have in which we don't know what to do and are too embarrassed to ask. I think that if the project more friendly, we can find more participants, make better use of our current ones, and see improvements to our diversity. Whew, that was long. What do you think of all this?

Apparently some people misunderstood Christian's recent statistics how Debian Developers are distributed over countries a bit. Especially the the sentence We now have 62% of active DDs while we had 73% last year. is kind of misleading, if you don't know how an active developer is defined. And especially, if you ignore the following sentence: This year's campaign of MIA work has been somehow "successful", apparently So, let's start with the active developers. To be precise: Neither Christian (nor did I back then) tried in any way to decide which developer is active, in the meaning of: He is actively contributing to Debian. That's pretty difficult data to get. Instead we just took the data available, and looked for activated accounts. And that's an important difference. See, if an Debian Developer leaves the project, his account doesn't get deleted (for various reasons), it just gets deactivated in our central account setup. So, when Christian wrote, he counted active developers he basically said: I didn't counted the deactivated accounts. So far so good. But indeed quite a lot accounts got disabled last year. Is that a bad sign (as some journalist say)? Actually no. As it's quite complex to check, who's still using his account (see http://wiki.debian.org/MIA for some explanation), it is quite possible for someone to still have his account active, who isn't using it. During the last year many of these Karteileichen (nominal members, if you will) have been deactivated (after being pinged several times). Or, to write it this way: While indeed quite some accounts have been disabled, Debian hasn't lost much through at, as these account's weren't used!