The first Debian-NYC Novice Night was a great success. We had about 25 people, about fifteen of which we had never seen before. We've been trying hard to reach out to new groups, and it seems to have worked even though we have a lot yet to do. One idea we'll probably use in the future is Novice Nights targeted to specific groups. A group of tech teachers in NYC has already contacted us and we will probably do a night in conjunction with them in the near future.People came for all sorts of reasons. We had a lot of installations, some people came for help configuration or upgrading, and some people came just to talk and learn more. Those who came to help learned a lot, too. Most everyone wants to come back.As with all Debian-NYC's activities, our plans and what we have learned are online so that anyone can adapt and reproduce what we do.We have a social event on December 10th, and will have another workshop, and another Novice Night, early next year.Tags: debian-nyc

It was unfortunate when Sony decided to pull out the Other OS support from PS3. One of the reasons of convincing myself to buy it was this feature. With that feature gone, the PS3 stood as nothing much but mostly a media center and an occasional game box. But with the Move, I think they have compensated it. It is a greatly engineered product with very good accuracy. It has been a week and I've been thoroughly enjoying it. I liked the packaging too. One single compilation of Sports Center contains like 5 games in it. Just the Table Tennis alone is worth it. But there are more that I need to try. Another surprising factor was pricing. United States pricing for the starter pack is expensive than at what I got here. This was something I didn't expect at all. Thank you Sony for creating a good product.

The screenshots.debian.net site is a public repository of screenshots taken from applications contained in the Debian GNU/Linux distribution and its derivates like Ubuntu. It was created by Cristoph Haas to help people get an impression of what a certain software will look like on your desktop before you install it. Everybody can take screenshots and upload them through the upload form. So far there are about 3100 screenshots, which is actually a small number if you consider Debian Squeeze contains about 15000 source packages After I hacked a bit on KSnapshot, the KDE snapshot tool, Sune asked me to add an export to screnshots.debian.net command. I had not thought of it before but that looked like a great idea, it would make the process or taking-screenshot, visit-upload-form, upload-each-screen so much easier!. So I started to develop my first KIPI plugin. Here is the resulting Debian Screenshots plugin, as you would use it in KSnapshot:

Being a KIPI plugin, it is available not only from KSnapshot but also from Digikam, Gwenview and the other applications which support KIPI plugins:

Which makes me think: what about having a debshots (the software powering screenshots.debian.net) installation over at kde.org and have screenshots for all the KDE applications? What do you think? Promo team? Sysadmins? Setting it up shouldn t be difficult: I m no Python or Pylons expert (I m more of a Ruby and Wt C++ guy and got it running in a VM in an afternoon, even solving some SQLAlchemy 0.6 issues.

Tuesday, MIT Kerberos had a bad hair day one of those days where you re looking through your hair and realize that it s turned to Medusa s snakes while you weren t looking. Apparently, since the introduction of RC4, MIT Kerberos has had significant problems handling checksums. Recall that when Kerberos talks about checksums it s conflating two things: unkeyed checksums like SHA-1 and message authentication codes like HMAC-SHA1 used with an AES key derivation. The protocol doesn t have a well defined concept of an unkeyed checksum, although it does have the concept of checksums like CRC32 that ignore their keys and can be modified by an attacker. One way of looking at it is that checksums were over-abstracted and generalized. Around the time that 3DES was introduced, there was a belief that we d have a generalized mechanism for introducing new crypto systems. By the time RFC 3961 actually got written, we d realized that we could not abstract things quite as far as we d done for 3DES. The code however was written as part of adding 3DES support. There are two major classes of problem. The first is that the 3DES (and I believe AES) checksums don t actually depend on the crypto system: they re just HMACs. They do end up needing to perform encryption operations as part of key derivation. However the code permitted these checksums to be used with any key, not just the kind of key that was intended. In a nice abstract way, the operations of the crypto system associated with the key were used rather than those of the crypto system loosely associated with the checksum. I guess that s good: feeding a 128-bit key into 3DES might kind of confuse 3DES which expects a 168-bit key. On the other hand, RC4 has a block size of 1 because it is a stream cipher. For various reasons, that means that regardless of what RC4 key you start with, if you use the 3DES checksum with that key, there are only 256 possible outpus for the HMAC. Sadly, that s not a lot of work for the attacker. To make matters worse, one of the common interfaces for choosing the right checksum to use was to enumerate through the set of available checksums and pick the first one that would accept the kind of key in question. Unfortunately, 3DES came before RC4 and there are some cases where the wrong checksum would be used. Another serious set of problems stems from the handling of unkeyed checksums. It s important to check and make sure that a received checksum is keyed if you are in a context where an attacker could have modified it. Using an md5 outside of encrypted text to integrity protect a message doesn t make sense. Some of the code was not good about checking this. What worries me most about this set of issues is how many new vulnerabilities were introduced recently. The set of things you can do with 1.6 based on these errors was significant, but not nearly as impressive as 1.7. A whole new set of attacks were added for the 1.8 release. In my mind, the most serious attack was added for the 1.7 release. A remote attacker can send an integrity-protected GSS-API token using an unkeyed checksum. Since there s no key the attacker doesn t need to worry about not knowing it. However the checksum verifies, and the code is happy to go forward. I think we need to take a close look at how we got here and what went wrong. The fact that multiple future releases made the problem worse made it clear that we produced a set of APIs where doing the worng thing is easier than doing the right thing. It seems like there is something important to fix here about our existing APIs and documentation. It might be possible to add tests or things to look for when adding new crypto systems. However I also think there is an important lesson to take away at a design level. Right now I don t know what the answers our, but I encourage the community to think closely about this issue. I m speaking about MIT Kerberos because I m familiar with the details there. However it s my understanding that the entire Kerberos community has been thinking about checksums lately, and MIT is not the only implementation with improvements to make here.

So, DuckDuckgo now has !bang search goodness for Debian packages (!dpackages) and Debian BTS (!dbugs). The BTS search actually launches the "search", while I'd have preferred just http://bugs.debian.org/%s, so that !dbugs src:python-numpy would just take me directly to the numpy bug page, but it's a start. The search engine creator is very receptive to comments, though, and I am sure you can reason with him if you feel most people would find it useful if it is changed to something else; I wasn't sure what the best course of action was, so I haven't talked to him yet.Here is the full list of !bang searches in DuckDuckGo, and a submission form to request for new ones. Enjoy!

LWN has an article by Valerie Aurora titled The dark side of open source conferences [1] which is about sexual harassment and sexual assault at Free Software conferences. Apparently some conferences create such a bad environment that some people won t attend, it s a well researched article that everyone in the community should read.The Autism DerailmentThe comments have the usual mix of insight, foolishness, and derailment that you expect from such discussions. One derailment thread that annoyed me is the discussion about men on the Autism Spectrum started by Joe Buck [2]. Joe seems to believe that the 1% of males on the Autism Spectrum (and something greater than 1% but a lot less than 50% in the Free Software community) are a serious part of the problem because they supposedly hit on women who aren t interested in them in spite of the fact that the article in question is about women who are being insulted, harassed, and groped at at open source conferences . The article had no mention of men who try to chat up women presumably this was a deliberate decision to focus on sexual assault and harassment rather than what Joe wanted to talk about.In response Mackenzie made the following insightful point:I don t think any autistic person who is high-functioning enough to A) contribute to open source B) want to be at an event with so many people and C) carry on any sort of conversation is low-functioning enough not to understand stop or no. If you can understand your patch has been rejected, you can likely understand don t do that again. Understanding how Other People FeelBruce Perens claimed What they [Aspies] don t understand is how the other person in the situation feels . Like many (possibly most) people Bruce doesn t seem to get the fact that no-one can really understand how other people feel. The best logical analysis of this seems to be the Changing Emotions article on Less Wrong [3]. While Less Wrong deals with Male to Female conversion as the example (which may be relevant to the discussion about the treatment of women) the same logic also applies to smaller changes. Anyone who even thinks that if they would always be able understand how their identical twin felt (if they had one) probably hasn t considered these issues much. As an aside, having a psychologist diagnose you as being on the Autism Spectrum and therefore by implication thinking differently to 99% of the population really makes you consider the ways in which other people might have different thought processes and experiences.Every time we have a discussion about issues related to sexism in the Free Software community we get a lot of documented evidence that there are many people who are apparently neuro-typical (IE not Autistic) who don t understand how other people think in many cases they go so far as to tell other people what their emotional state should be.What Really HappensNix said However, in that situation our natural reflex is to *get out of there*, not to jump on women like some sort of slobbering caveman which is a really good summary.In more detail, I think that the vast majority of guys who are on the Autism Spectrum and who are able to do things like attend computer conferences (*) realise that chatting up a random girl that they meet is something that just isn t going to work out. Generally people don t attempt things that they expect to fail so I don t think that Autistic guys are going to be hitting on girls at conferences.(*) Having never met any Autistic people who aren t capable of attending such conferences I can t speak for them. I really doubt that the Low Functioning Autistic guys are as much of a problem as some people claim, but lack evidence. In any case the actions of people who don t attend conferences aren t relevant to a discussion about things that happen at conferences.Update: It Keeps GoingDion claims that the misogyny at conferences is due to socially inept people, he also casually switches between discussing people who misunderstand when someone is flirting and people who hire almost-naked booth-babes (two very different classes of action) [4]. Several people asked for supporting evidence, naturally none was provided.In response njs posted a link to Marissa Lingen s blog post Don t blame autism, dammit [5]. Marissa points out that people who offend other people due to lacking social skills will tend to do so in times and places that are likely to get a bad reaction if you don t know that you are doing something wrong then there s no reason to hide it. If someone offends a senior manager at a corporate event then it could be because they are on the Autism Spectrum (I ve apparently done that). If someone offends junior people at a times and places where there are no witnesses but is always nice to managers and other powerful people then it s not related to Autism.One final note, I have little tolerance for anyone who claims to be an Aspie when they do something wrong. You are either on the Autism Spectrum all the time or none of it. Anyone who wants any sympathy for me for an occasion where they stuffed up due to being an Aspie can start by making a clear statement about where they are on the Autism Spectrum.Update2: Yet More from Bruce PerensBruce wrote IMO, the kind of men who go in to software engineering suffer a lack of healthy interaction with women who are their peers, and it may be that the high incidence of empathy disorders in our field is involved (which seems to be part of the inspiration for Joe Buck later in that thread) and now claims Nobody here was trying to connect Asperger s or autism with the touching incidents or violent crime .Matthew Garrett responded to that with If you weren t trying to say that the high incidence of empathy disorders in our field was related to a lack of healthy interaction with women who are their peers, and that that has something to do with incidents of sexual harassment or assault at conferences, what were you trying to say? Because that sounds awfully like We wouldn t have so many problems if it weren t for all the autists .Bruce s latest comment is If you choose to read something that nasty into my writing, that s your problem. Get therapy .Through this discussion I ve been unsure of whether to interpret the statements by Bruce et al the way Matthew does or whether I should consider them as merely a desperate attempt to derail the discussion. I can t imagine any possible way of interpreting such comments in connection with the discussion of sexual assault as anything other than either trivialising violent crimes against women (suggesting that they are no worse than asking out someone who s not interested) or claiming that anyone who lacks social skills should be treated as a violent sexual predator. It s just not reasonable to believe that every single person who wrote such comments referring to Autism was misunderstood and really meant something nice.As a general rule I don t think that it s the responsibility of other people to try and find a non-offensive interpretation of something that one might say. I don t think that all the people who strongly disagree with the most obvious and reasonable interpretations of Bruce s comments should get therapy. I think that Bruce should explain what he means clearly.

• [1] http://lwn.net/Articles/417952/
• [2] http://lwn.net/Articles/418209/
• [3] http://lesswrong.com/lw/xe/changing_emotions/
• [4] http://lwn.net/Articles/418281/
• [5] http://mrissa.livejournal.com/746160.html

I have this project in my head, I want to work on it but I always lack the time. In order to go forward, I thought I could write about it, at least it would let me clarify my ideas and the core goals. So here I am, I will present you Librement (I have registered the alioth project but it s empty). The core goal is to make it easy for every user to contribute to free software in some way. I will now present the main features that I envision. Defining skills and interests In order to propose tasks that the user can do, we must have an idea of his skills. So on the first run (and later through a preferences menu) the user will be invited to define his skills:

• his native languages (multiple allowed)
• other languages he can understand
• programming languages he knows
• version control systems he can use
• markup language he knows (HTML, DocBook, Wiki-like formats, etc.)
• etc.

Maybe we can also ask which skills he would like to learn. Because contributing to free software is a nice opportunity to learn new skills! We should also find out what the user is interested in. What are his favorite free software projects? What kind of contributions would he like to do (documentation, translation, coding, bug fixing, bug triaging, creating artwork, donations, etc.)? Choose activities and pick concrete tasks Based on the user s skills and his interests, the software shows a list of possible activities. The user can then sort that list, from the most interesting one to those that he doesn t want to do. Each activity can generate concrete tasks. For example, the activity Do translation for Debian could generate a task Translate strings in debconf/fr.po or Review translations in partman/fr.po . Work on tasks When the user decides to work on a task, a step-by-step assistant helps him/her. It can automate some steps and provide explanations for the remaining ones, for example in the case of a translation for Debian:

• grab the PO file (from a VCS, from an HTTP URL, from a translation server, etc.);
• select and install a software to work with PO file (if not already done);
• edit the PO file with the preferred program;
• check the PO file (is it complete? is there no mistakes like missing substitutions?);
• send back the completed PO file in a mail to the Debian bugtracking system.

If the tasks is not completed in one go, the user can resume it the next time. Each free software project must provide some meta-information describing the various workflows involved for contributing to the different parts of the project. If necessary the project can also provide new plugins to support new operations that are not available in the default library. Setting goals In order to keep the user motivated, the software could track how much time he spent contributing to free software and it could verify if the user reached the goals he picked up for himself. Maybe it can also hook into the OMG Trophy Awarding System. The sky is the limit I hope that you now have a clearer idea of what this desktop application is supposed to be. There are literally hundreds of ways to contribute to free software and I like the idea that we can streamline the process for most users. All the plugins implementing activities can use local information (list of packages installed with their versions, configuration settings, etc.) to propose tasks targetted to the user and highly beneficial for the corresponding free software projects. For example, a bug tagged unreproducible might benefit from a few more users trying to reproduce it. The software could direct the user to this bug report if it detects that he/she runs the same version on the same architecture and that this software is regularly run on the system. Many projects have created operations or events to encourage people to contribute, they could all be implemented as dedicated activities in Librement. I m thinking of stuff like Gnome Love, Ubuntu s 5-a-day, Ubuntu s 100 papercuts, etc. Even for people who have no time to contribute, the application can still be useful by referencing the various ways to donate money (or material) to projects that they are using. Feedback welcome I m excited by the potential of such an application, but it s normal since it s my idea. Do you believe it can be useful and popular? Do you have ideas of exciting activities that such a framework can offer? PS: If you wonder how I came up with the name Librement , here s the explanation. It s a French word which means freely . And users who want to give back are trying to live up to the principles of free software, which I sum up by they are trying to live freely .

Follow me on Identi.ca, Twitter and Facebook. Or subscribe to this blog by RSS or by email.

11 comments Liked this article? Click here. My blog is Flattr-enabled.

Bruce Schneier has written an absolutely powerful essay in his blog, with the modest proposal that in response to the security worries at the Washington Monument, we should close it. If you haven t read it yet, run, don t walk, to his blog and read it. Then if you live in the States, write to your congresscritters, and ask them to reinsert the backbone which they have placed in a blind trust when they got elected, and tell the TSA that they have a new mandate; to provide as much security as possible without compromising our freedom, privacy, and American Ideals. Right now, they have an impossible job, because they have been asked to provide an absolute degree of security. And in trying to provide the impossible, the terrorists have already won No related posts.

Ok, so micro rant time: this is the effect of not taking things upstream: hardware doesn t work Out Of The Box. Very briefly, I purchased a Vodafone prepaid mobile broadband package today, which comes with a modem and SIM. The modem is a K3571-Z, and Ubuntu *thinks* it knows how they work (it doesn t). So it fails to connect in NetworkManager with a rather opaque NO CARRIER message. Thanks to excellent assistance from Matt Trudel, we tracked this down to a theory that perhaps modemmanager is using the wrong serial port and voila, it is. From there, the config file (/lib/udev/rules.d/77-mm-zte-port-types.rules) was an obvious next step and indeed there is no entry in there for the 19d2:1010 the K3571-Z. Google found one immediately though, on a Vodafone research site. The awful shame is this: that was committed to the bcm project in March this year. If Vodafone had shipped off a patch to modemmanager, we could have had that in 10.10, and possibly even in 10.04. There are plenty of users having trouble on Whirlpool etc with this model who would have had a better experience helping Vodafone s users be happier. All it would have taken is an email I m sure Vodafone want a great experience for their users, but I think they re failing to separate out platform improvements share and share alike, and branding / custom facilities. The net impact is harmful, not helpful. Anyhow, Natty will support this modem.

This is a mental note really, since my certificates last two years and I ve always forgotten what to do about it.

1. Generate a new request:
   openssl req -new -key <keyfile> -out <csrfile>
   StartSSL throw away all properties of the request except the key, so any answers will do.
2. Re-use the request you sent last time (thanks Noel).
3. Get the certificate signed.
4. Dovecot expects a key in /etc/ssl/private/dovecot.pem and a certificate chain in /etc/ssl/certs/dovecot.pem. Build the chain, CERTIFICATE FIRST:
   cat <crt> sub_class2.pem ca.pem > /etc/ssl/certs/dovecot.pem
5. Reload Dovecot and test from somewhere remote:
   openssl s_client -connect <server>:imaps
6. Coffee time.

Lighttpd is basically the same, but additionally expects the key to be in the top of the certificate chain.

(This is a part of a series of posts on my recent experience with Windows. See Windows (I) for the first post.)

For the past month, I got disconnected from basically all of my usual activities. Not only from Debian work (well, yes, I followed up on a couple of important bugs and releases, but I was in a very low power consumption mode of sorts), DebConf organization/following (some people have contacted me regarding the DebConf 12 venue decision, for which we should start working and have a presentation soon) Even the always-so-important Real Life got pushed aside. I barely participated in this year's EDUSOL (On-line Encounter of Education and Free Software). Even work suffered I asked my boss to hold any incoming requests for me (save, of course, for urgent stuff). I am now shyly coming up from the hole I dug myself in for this time. Why did I do such a thing? I do not have a formal education beyond high school. Due to several personal issues, I was accepted to the best university in my country, and to the first generation of the study program I would definitively love to have followed, but didn't attend. Of course, I had a fair bit of computing background, enough to be competitive and start working. After amazingly few years, I didn't only get a decent job, but I got a job precisely at the very university I wanted to study at (and where my father has worked for the past 40 years). And, even with some changes along the years, here I am with 10 years of laboral history in this university, with an academic position (although the smallest academic position, but still). It is one of my life's achievements, one of the things that makes me proudest of. Still, I don't want to stay at this level for the rest of my life. And an institution such as this one has rules Yes, flexible enough to allow me in, but rules whose importance I cannot deny. I aspire to become a professor/researcher at some point, and that is just not possible without filling in the prerequisites. Besides, I want to try my hand at teaching at a universitary level. So, what choices did I have besides devoting at least four years to full-time study? Well, there is a government entity that evaluates and regulates formal education given by smaller universities CENEVAL. For probably around ten years now, they have offered the ACREL (Acuerdo 286 Licenciatura) program to validate professional knowledge acquired through work experience And I'm writing this detail as several people have asked me for details on it. So, once you are accepted in the program (basic requisites: Having a high school certificate, over 30 year old, and with over 10 years of provable work experience), they can grant a Licenciatura degree (i.e. the equivalent to a four-year study program). So, in order to get it granted, I have to:

• Present a written exam, covering all of the areas of study. In my case (it varies depending on the program you are presenting), it was a 12 hour exam (over three sessions IIRC I finished each session in around three hours of the four allowed). Questions are all multiple choice. For the Software Engineering degree, you are allowed to enter each session with up to five reference books. Rules vary depending on the area. The exam was long, although not too hard. And I'm proud to say that I reached outstanding level on each of the areas of the exam
  If this information intersts you, do take a look at the links under the Study guides bullet at CENEVAL's information page. The areas to study, the style of the questions, and the general information on the exams are very close to what you will actually receive.
• The second phase (oral exam) is split in two sub-phases. The first one, the one I have just finished, is the longest one: I was assigned a practical case to develop. In my case, I was requested to write a document detailing the methodology for writing a request-tracking system for a restaurant. I was given a month for this task And in part, the reason that kept me so busy is that I subscribe to the idea that the best methodology to write an application is to write it, not just to document what you are about to do (and then fail to recognize you missed so many details). Then, there is yet another period (between one and three months, assigned randomly AFAICT) during which I would have to develop the application and a presentation, and defend it against a jury.
• Some study areas (mainly those related to health) have a third step: A practical exam. Of course, I cannot go into details on this one.

Anyway... Enough of a blog posting. I hope this is useful for somebody. And I hope it explains why I have disappeared so much lately. If everything goes as planned, I expect to be receiving my "licenciatura" title around March, and afterwards... Well, we shall see! Now... Back to work. And back to life. And back to my huge backlog of other pending things :)

We ve come a long way since the times when you needed to configure 2 X servers in XDM just to be able to use 2 X sessions at once. However there was still some way to go until recently. A number of bugs that could be wrongly attributed to bugs in the X server or in the desktop environment were actually caused by the display manager doing crap. GDM up to 2.20 Since the introduction of the flexible X servers feature, GDM hadn t evolved much on the matter of user switching. What it used to do was pretty straightforward:

• a specific protocol can be invoked by the gdmflexiserver command;
• the gdm daemon spawns a new X server on an empty console;
• it initiates another login process in it;
• when the session exits, or if the user clicks on Quit instead of logging in, the X server exits.

It is interesting to note that VT (console) switching is purely handled by the X server. When starting, the new server switches the current VT to where it is. When exiting, it automatically switches back to the VT from which it was launched. While very simple, this idea fails to work correctly every time you try to do something more complicated than starting a temporary session for a guest and exiting it. For example, if you start two of them, there is a chance that, when the X server switches back to the console it was run from, there is nothing left running in this console, leaving you with the funny Control-Alt-Fn shortcuts to find your way back to a X server. You will also meet interesting race conditions when trying to switch back to an existing session from the login window. GDM 2.28 and above In the process of rewriting the code entirely, the GDM developers tried to address a number of those shortcomings, making use of D-Bus and ConsoleKit. The new design is slightly more complicated, however.

• The gdmflexiserver tool will first try to look for an existing login window in another X server, and just switch to the VT it is in if it finds one.
• Otherwise, the daemon starts a new slave process with a new X server and a new login window, in a very similar way to what older versions did.
• When logging in as a user with an existing session, it switches to the VT it is in, but leaves the login window and its X server running.
• When going into a new session, the X server is simply left to die at the end of the session, and to switch back to the VT from which it was launched.

Not killing the X server in some cases partly addresses the problems caused by letting it switch back to the original VT when exiting. However in several ways the cure is worse than the disease.

• First of all, it will leave unused X servers, with all processes used by the login window - and that makes quite a number of them, with GDM now using a minimal GNOME session.
• When there is such a login window remaining, ConsoleKit will refuse to let you shut down your computer, being lured into thinking there is someone else using it.
• It doesn t solve the inconsistency issue. When you leave a session, you can find either of: a login window, a screensaver unlock dialog, or a black screen.

Getting it to work The modular architecture of GDM makes it possible to improve the situation. (Possible but not easy because of the millefeuille of classes.) However, it is merely a band-aid unless you fix the root issue: the X server knowing better than you which VT it should switch to when exiting. Fortunately Xorg now features an option to avoid that behavior: -novtswitch. So the first step is to enable it, and let the GDM daemon (or slave) handle VT switching through ConsoleKit. With that, the following changes are possible.

• When switching to an existing session, don t leave a X server behind. You can now kill it safely without risking a VT switch.
• On the opposite, when exiting a session, always respawn a login window on the same VT.
• The last step is to stop making a difference between the first launched X server (called a static server) and the flexible servers. The only remaining difference between a static display and a flexible one is that the static one honors automatic login/timed login settings.

The result With all these changes the behavior of the display manager is finally completely consistent.

• When exiting a session, regardless of it, you always find the same login window.
• There is never an unused process left.
• You will never find yourself facing a black screen with only keyboard shortcuts to leave you out of it.

Interestingly enough, this is very similar to what user switching looks like on Vista or MacOS X. So what now? These changes are stabilized for Debian squeeze, but of course it has been long overdue to get them accepted upstream, along with the very large number of Debian-specific changes that still lie in our packages.

I have been helping with an overdue web app project lately, and the project uses ASP.NET MVC, so I have had to use Windows for most of my day. Since I had been using GNU/Linux almost exclusively since 2003, this is quite an adventure (imagine trying out GNU/Linux after a seven year break). It is not nearly as bad as I thought it would be, even though I am using the not-so-well-liked Windows Vista which came pre-installed with my computer.

The first thing that struck me is how much less important the OS is when compared to the situation a few years ago: a lot of the applications I depend on are web-based (GMail, Google Calendar, Google Docs, Springpad), and the ones that are not web-based are multiplatform and installable with a few clicks (GVim comes to mind). In fact, I do not think there is a single piece of software that I really missed in Windows. (An interesting corollary of this observation is that it should be equally easy to migrate to GNU/Linux nowadays.)

The one thing where the operating system matters is hardware support, and I do not think I have to say much here. After years of whack-a-mole GNU/Linux upgrades which would fix support for one component but break another (for things as essential as video and sound), it was absolutely weird having all hardware work properly at the same time (sound after suspend! no video glitches! high quality webcam picture!). There is no point recounting the reasons why hardware support in Linux is as it is, but now I am starting to think that Linux users do tend to suffer from a sort of Stockholm syndrome regarding hardware support; I know I did.

The other important thing that operating systems do is resource management, and I did not have problems here either. In fact, I liked the virtual memory behaviour much better: when Windows ran out of memory (I had filled up the smallish Windows partition and there was no space left for the swap file to grow), it started rejecting malloc calls from applications, but the system was kept alive and responsive, and I could close apps and clean up the mess without much trouble. In Ubuntu, with the default settings, whenever a runaway process ate too much RAM, the system would start thrashing so hard that it would be very difficult to fix. My standard procedure (switch to text console, log in, find and kill app using command line tools) would take minutes and was so annoying that I would sometimes just reset the computer instead. And in the end, even if I did not reset, an important app would possibly be killed anyway. I know there are kernel parameters for tweaking this behaviour (swappiness?), but I never got around to figuring them out as this situation does not arise that frequently, and testing is painful.

I did not like the Windows FS security model. ACLs are much more complicated and thus less transparent that the Unix approach, even though they are more powerful. My main gripe with them is that I do not know a good way to display them easily (as with ls -l in Unix). I have not had to deal with them much though.

When I accidentally looked at my personal Debian repository, I realized that it is time to remove obsolete things from it. There were packages for polld, for which which I don't see need anymore (it is now properly handled by kernel), hacked acpid packages, which were doing special things for Toshiba M30 notebook (which I don't use with Linux anymore and nobody else probably knew about these packages) or wmmp, a Window Maker applet for MPD, which I haven't used for quite a long time. All those are gone now and I don't think anybody will miss them (if you do, please tell me). What stayed there are daily snapshots of dictionaries for StarDict and gnaughty package, which was rejected as possibly causing troubles.

Filed under: Debian 0 comments Flattr this!

You probably know about top , in Debian and Ubuntu part of the procps package. Ever wanted to see CPU and memory usage as bars and not numbers? Ever wanted to kill a process from inside top by just selecting its row instead of having to type its pid? Ever tried to press a cursor key inside top? It makes a noise and says Unknown command - try h for help . Short said: top is not that interactive. Ever wanted top to be more colorful? Well, there is a solution to all these issues. It s called htop (Debian package) and is some kind of colorful, ncurses based cross-over between top and a Midnight Commander for processes:

More screenshots of htop at screenshot.debian.net and at the project s site at SourceForge. It can do many things, top can t do:

• Different color themes including fore- and background (includes also a monochrome theme for people considering ANSI colors being eye cancer :-)
• Highlighting the current user s processes.
• Scrolling up, down and sidewards.
• Interactively selecting processes with cursor keys as well as with the mouse.
• Kill or renice the selected process without having to enter its pid
• Show the CPU usage for each core and a CPU usage summary at the same time.
• Show CPU, memory and swap usage as textual bars, either encoded in colors or characters.
• Easy interactive configuring while running.
• Automatically saves the current state (sort order, color scheme, etc.)
• Easy access to the most often used functions through F-keys like in Midnight Commander.

One more cool thing about htop: It s also available on the Nokia N900 as app . The only thing I found so far that top can do, but htop can t, is top s batch mode (e.g. with top -b -n 1 ) where it runs non-interactively and its output can even be piped to other processes. The Xymon/Hobbit monitoring system uses that as input for some machine statistics.

In this post I'll describe the changes made to the kernel and some of the Squeeze packages for the Freaky Wall. The plan is to submit whishlist bugs to the BTS on the hope of having all what is needed for this project available on Debian after the Squeeze release, as my feeling is that a freeze is not the right time to push this changes... ;) I'm giving access here to all the changes made to the source packages, but if anyone wants the binary packages (amd64 only) send me an email and I'll give you the URL of an apt repository that contains all the modified packages (it's the one at work, that contains other modified packages) or, if there is interest, I can put them on people.debian.org. Kernel To be able to build the firewall we need a kFreeBSD kernel with some options not compiled on the version distributed with Debian. To compile the kernel I've followed the procedure described on the following debian-bsd mailing list post: http://lists.debian.org/debian-bsd/2010/09/msg00023.html Basically I've done the following:

    apt-get build-dep kfreebsd-8
    apt-get source kfreebsd-8
    cd kfreebsd-8-8.1
    cat >> debian/arch/amd64/amd64.config << EOF
    # Add pflog, pfsync, ALTQ and CARP support
    # ----------------------------------------
    # http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/firewalls-pf.html
    device      pf
    device      pflog
    device      pfsync
    options         ALTQ
    options         ALTQ_CBQ        # Class Bases Queuing (CBQ)
    options         ALTQ_RED        # Random Early Detection (RED)
    options         ALTQ_RIO        # RED In/Out
    options         ALTQ_HFSC       # Hierarchical Packet Scheduler (HFSC)
    options         ALTQ_PRIQ       # Priority Queuing (PRIQ)
    options         ALTQ_NOPCC      # Required for SMP build
    # http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/carp.html
    device      carp
    EOF
    vi debian/changelog 
    dpkg-buildpackage -B -uc

Once the package was built I installed the new kernel package and rebooted the machine. Utilities To be able to use some utilities related to pf I have built patched versions of three packages:

• freebsd-utils: I have included pflogd and ftp-proxy on the package and have added some options to allow the use of additional interface types with ifconfig (carp, pfsync, lagg, bridges, ...). There were a lot of changes needed:
  1. The pflogd and ftp-proxy Makefiles are missing from the original tarball, I modified the get-orig-source of the debian/rules, but I build my packages against the original tarball, with the extra files included directly on the package .diff.gz.
  2. The pflogd daemon needs a _pflogd user and group and a /var/run/_pflogd directory, so I've added the directory and the creation of the user and group to the package post-install.
  3. The ftp-proxy daemon uses the proxy user when dropping privileges and I've modified the code to make it use the /var/run/ftp-proxy directory for the chroot.
  4. Some C header files that are not available on -dev packages were missing and I put them on the packages' debian directory. I've created a debian/include directory and moved there the original debian/net directory and added the headers debian/include/net/if_bridgevar.h and debian/include/net/if_lagg.h to add bridging support to ifconfig and the debian/include/pcap-config.h and debian/include/pcap-int.h libpcap private headers to be able to compile the pflogd binary.
  As I'm not familiar with the way people changes code for kFreeBSD some of the patches are a little bit dirty, but at least things work; besides, probably I should also have had to include init.d scripts for pf, pflogd and ftp-proxy, but I have not done it at the package level yet, as what I got was enough to work with the tools. The debdiff against the freebsd-utils-8.1-2 source package is available here or here.
• libpcap: A test had to be removed in order to be able to support pflog on the library; the debdiff against the libpcap-1.1.1-2 package is available here or here.
• tcpdump: The package also had to be modified to include the code to print the pflogd entries on the pcap file; the debdiff against tcpdump_4.1.1-1 is available here or here.

On the next post I'll describe how I've configured the system, the network interfaces and the different utilities patched and compiled on this post.

I knew walking all over the Pyrenees during a whole month would come with some side effects. I could imagine having a few muscular issues in my legs and some back pain for a while after getting back; after all we did over 7 hours of exercise every single day during a month. What I got after our hike was totally unexpected... I should have run a half marathon in Gandia on the 21st, and instead I stayed at home doing some assignments. When we came back, I was in a really good condition, and wanted to keep the good shape we had built the month before. Given I haven't been able to swim for nearly a year now, due to the Misterious Shoulder Injury and I don't have enough time to go out cycling regularly, I centered my efforts in running, with the idea of starting to get prepared for cross/mountain races this season. Training had to wait a little because some days after getting back, I got a sudden pain on both knees, which even made climbing stairs difficult. It stuck for a while, and when it finally went away, more or less, I went ahead and tried going out for a run. I haven't been training for over two years, so I indended to increase the distances proggressively. The first day I went out, I ran really nicely for the first 25 minutes, and suddenly I started getting an intense pain in the outer side of my right knee, to the point I could barely walk back home. I've tried letting them rest for weeks, and every time I try them, the pain comes back. My physiotherapist says my body has developed quite a few asymmetries, probably caused by the many days I had to walk crippled by a big blister in my right foot. As time passes, I feel I'm losing all the physical improvements I had developed during the summer, and I can't do anything about it. The new approach to tackle this is yoga. I know my body isn't too elastic, and the lack of stretching during the walk made it even worse, so I'll try to forget about real traning during some time and focus on healing my muscles. Hopefully, this will help me resolve the nastiest physical problem I've ever come across.

Debugging a computer program is such an interesting activity because it's not really a matter of fixing a program. It's a matter of fixing your own understanding to the point that the cause of the bug becomes obvious. So debugging means constantly challenging your assumptions, constantly looking for the overlooked insignificant thing that turns out to be crucial. I mostly debug by print statements. I don't like it much, but it works. I add some print statements around the problem area and see if what's going on matches what the model in my head thinks should be going on. If it doesn't, I try to fix the model, and if it does, I focus the print statements in tighter and tighter, deeper and deeper, until I finally reach the bug. It's nasty and ah-hoc, but it works really well, mostly. And I know I'm in good company doing it. But that method can fail; I myself can end up caught in a loop. Like today, tracking down an obscure bug in Branchable's auditing subsystem, when I found myself at what was apparantly the point of a bug: Function A was supposed to call function B, but B never seemed to be called. So surely A was buggy, and I instrumented it to bits with print statements and stuff, and started wondering if the libraries it relied on were somehow breaking it in non-obvious ways before it could call B, or maybe there was a bug in the runtime language that was breaking the call to B. Going deeper and deeper didn't help, because the thing I was overlooking was up at the top level -- one of the first print statements I inserted showed that all this code was running twice. Why? Didn't seem relevant to the bug so I ignored it. Of course what I was ignoring was a symptom of the real problem: The first run was by another subsystem, that happened to redefine function B temporarily.. Actually, one of the nice things about Haskell is that print statements are not very helpful in debugging pure code. Pure code tends to get loaded up in ghci and tested interactively, and the bug is then clear. Or a quickcheck test case gets written with an invariant that the pure code should satisfy. I spent a whole day a few weeks ago writing an inverse form of a buggy function in git-annex just so I could feed them both into quickcheck and automate debugging. (Most of that time was spent wrestling with unicode decomposition .. ugh.) Being used to getting down in the buggy code with print statements, that at the time felt rather like a waste of time unrelated to getting on with making my program work, but it was very much worth it since I got my program working 100% in even crazy edge cases, and got a test suite for free, too. Most of my debugging of Haskell code so far has not involved bugs in pure code. The most memorable bug was actually a bug in ghc's IO manager, which I'd have never tracked down without Josh's help. The problem with finding bugs in the runtime environment or compiler is that after you've found enough over the years, you're more likely to go down false paths distrusting runtime environments, like I did with the bug today. The beginner's assumption is that the runtime environment just works, but I by now may have over-challenged that assumption. :)

Every so often, we get bug reports about Varnish leaking memory. People have told Varnish to use 20 gigabytes for cache and they discover the process is eating 30 gigabytes of memory and they get confused about what's going on. So, let's take a look. First, a little bit of history. Varnish 2.0 had a fixed per-object workspace which was used for both header manipulations in vcl_fetch as well as for storing the headers of the object when vcl_fetch was done. The default size of this workspace was 8k. If we assume an average object size of 20k, that is almost 1/3 of the store being overhead. With 2.1, this changed. First, vcl_fetch doesn't have obj any longer, it only has beresp which is the backend response. At the end of vcl_fetch, the headers and other relevant bits of the backend response are copied into an object. This means we no longer have a fixed overhead, we use what we need. Of course, we're still subject to malloc's whims when it comes to page sizes and how it actually allocates memory. Less overhead means more objects in the store. More objects in the store, means, everything else being equal, more overhead outside the store (for the hash buckets or critbit tree and other structs). This is where lots of people get confused, since what they see is just Varnish consuming more memory. When moving from 2.0 to 2.1, people should lower their cache size. How much depends on the amount of objects they have, but if they have many and small objects, a significant reduction might be needed. For a machine dedicated to Varnish, we usually recommend making the cache size be 70-75% of the memory of the machine. A reasonable question to ask at this point is what all this overhead is being used for. Part of it is a per-thread overhead. Linux has a 10MB stack size by default, but luckily, most of it isn't allocated, so it only counts against virtual, not resident memory. In addition, we have a hash algorithm which has overhead and the headers from the objects are stored in the object itself and not in the stevedore (object store). Last, but by no means least, we usually see an overhead of around 1k per object, but I have seen up to somewhere above 2k. This doesn't sound like much, but when you're looking at servers with 10 million objects, 1k of overhead means 10 gigabytes of total overhead, leading to the confusion I talked about at the start.