Dan Rosenberg pointed me to a paper from the 2010 WOOT conference that mentions my work to implement the CryptProtect function in Wine. Their research is awesome, and it was fun to compare my attempts at identifying the blob structure to what they discovered. Looks like I got the structure pretty well, but that was easy; they totally broke the encryption itself. Now those native blobs can be decrypted, opening the door to full NTFS interoperability, offline forensics of Windows encrypted files, etc. (For designers of future symmetric encryption methods: please don t store the keys (in any form) on disk with the cipher text ) What I found most alarming about this is a comparison to eCryptfs, and how it is implemented with the user s login passphrase. In both cases, a hash of the passphrase is used to perform additional work that results in keying the final encryption. In eCryptfs, this hash is calculated to unlock the main key that is used for eCryptfs and is then thrown away (it can always be regenerated when the user logs in). If the user changes their passphrase, they must decrypt and re-encrypt the eCryptfs key (this is done automatically by PAM). Under Windows, to deal with potential user login passphrase changes, they instead decided to store all prior (SHA1) hashes of the user s passphrases, even lacking a salt. So all the clear-text user login passphrases are recoverable with a standard rainbow table, in parallel. O_o

1. Create an account on Flattr.com (mine is here);
2. Submit the URL of your free software project to the Flattr directory with a good description and the proper tags (here s my entry for dpkg);
3. Add a Flattr button on your website (I put one on wiki.debian.org/Teams/Dpkg);
4. Inform your users and reach out to free software users using Flattr by registering your project in Flattr FOSS.

And I m not alone, Joey Hess made 25 EUR in one week only. And right now only 30K people are using Flattr. I expect this number to increase significantly now that invites are no longer required to open a Flattr account (see the announce on Flattr s blog). It s time to spread the word about Flattr FOSS! 8 comments Support my work

Here are my notes about the very insteresting BOF on distributed bugtrackers held at DebConf This is just some elements of report (some taken from the gobby edits, and written down by me after watching the video (unfortunately missing 2 mins of sound in the beginning of the video stream/recording)). This is not a full report of all the discussions, as some discussed elements weren t of interest IMHO. You ll find here many ideas of mine, links and pointers (resulting of our work in the context of the Helios project on similar topics), so this is in no way a report of what was said exactly during the BOF. Aspects of interest:

• for Debian bugs
  • Sharing state between upstream, downstream and sidestream
    • Forwarding upstream
  • Sharing comments between the same bugs
  • merging / conflict resolution (technically challenging, as well as in order to match people s habits/process)
• Same problem in general
  • distributed bug tracking
    • working offline (modify locally, push, etc.)
  • but also distributed version/commit tracking : tracking bug presence or fix along the branches of commits in distributed VCSes
• Interop with other bugtrackers:
  • method to pass data back and forth between different bugtrackers
  • standard method / format / API : lack of standard widely used
  • visibility of issues known upstream (or elsewhere) to Debian users while using reportbug
  • scraping other trackers

Note that these topics may in principle be discussed on the Distributed bug trackers mailing list: http://kitenet.net/cgi-bin/mailman/listinfo/dist-bugs Tools / implementation ideas :

• First, there s a wiki resource about lots of software tools : http://dist-bugs.kitenet.net/software/
• SD (presented later by Christine Spang see my other blog post about her great presentation)
• LaunchPad s tracker already knows about lots of other bugs in other distros including Debian bugs (through bug tasks )
• bts-link watches links between bugs and status changes (mainly through scraping)
• Generic scraping tools could be used :
  • Scraperwiki (http://scraperwiki.com/)
  • or WebReformatter (http://code.google.com/p/webreformatter/
• Virtuoso spongers to perform on the fly conversion of bugs XML feeds to RDF to inject them in a RDF store (ex: search bugzilla inside http://www.openlinksw.com/dataspace/dav/wiki/Main/RDFMappersUseCases)

Storage of repositories / databases of remote bugs :

• Mail lots of suggestions, but to be discarded IMHO
• in users Virtuoso RDF stores, eventually in an integrated way with the Semantic Desktop (KDE 4.4 / Tracker / Zeitgeist) which may be an option (integrating bugs in other desktop tools aware of the semantic desktop).
• through desktopcouch (as used by the bughugger client for LP tracker see the bughugger presentation at RMLL/LSM 2010)

About such storage, and the RDF store option, here s a prototype of ours that exposes UDD s bug facts through a SPARQL interface, allowing to publish bug facts as RDF (setup in the frame of the Helios project). It includes both Debian and Ubuntu bugs, and allows interlinking them in a standard way (with Semantically described links in RDF). See http://testforge.int-evry.fr/d2r-server/resource/spackage/evolution for an example. The problem is not how to distribute (distributed database systems exist), but the semantics of bug properties, and the different implementations of interfaces of different tools : which standard to implement, or how many converters are necessary if no standard ? We believe that navigation/referencing/tracking of external bugs would be much easier if bugs were LinkedData citizens, i.e. they would be exposed with (standard) formats like Semantic Web ontologies served with Semantic Web standards (like RDF). The worst case is currently to have to perform web scraping on bugtracker interfaces. How to avoiding scraping ? A solution could be to publish RDFa / microformats using common standard semantics, directly from the bugtracker pages at bugs URLs (see my wishlist for debbugs on that subject : 590931: Would be great to integrate RDFa metadata into debbugs pages). Now some pointers to historical details on the same subject and Debian (old posts by Lars Wirzenius) :

• Distributed bug tracking
• Bug exchange format

Overall, the discussion was quite interesting, but didn t reach any sensitive progress regarding the state of the art. Much needs to be worked on, mainly on standardizing APIs, semantic descriptions of bugs, and implementation in tools like SD for instance. Stay tuned for more progress.

Already available in both Maverick and Debian sid, it provides many interesting features. I m talkin about Font Manager, a small application written in C and Python by Jerry Casiano, which allows users to easily install, remove and compare font files on own system. Here are few nice screenshots:

To install the application, as usual type:

sudo apt-get install font-manager

Let me know what you think!

Links

• Project s homepage
• Debian PTS page

I was in New York for three full weeks, before, during and after DebConf 10 took place. Unfortunately, I wasn't able to attend talks or go hang out with folks as much as I would've wanted. Most of the time I was busy at the office with a new product launch we have been preparing for the last few months. However, I was able to come and say hi to a few people, attend the C&W party and talk with friends and ran into. I was, however, following all conversations on IRC so I was pretty informed of all of what was going on. Some random thoughts:

• Being surrounded by so many amazing Debian people made me realize how much I've been apart from the project. My involvement decreased pretty much since I moved to New York. But because of that, I'm now slowly retaking my activity, trying to work on actual things that I use on my daily work instead of just packaging tons of shit that nobody really cared about.
• DebConf is pretty awesome. After attending six out of the last seven Debian conferences, you get to establish a nice connection with people and they become your friends. This year, even if I only came a couple of nights, I made stronger friendships, and for that I'm thankful. It is only within Debian that this human part of bonding really takes place.
• I really regret missing a couple of events: First, the Day trip to Coney Island and the Cyclones game; and the group photo. Both because of work, yes.
• It was interesting to hear the thoughts from one of the Bosnian attendees about next year's DebConf there. I'm really sorry I can't recall his name, but it shocked a bit that he stated that they might try to find better speakers next year, because a lot of them "sucked a lot" here. All I can say is that, even though everybody's opinions are valid, this might come from an outsider, not used to Debian conferences. Where are you going to find better speakers? Us developers are the only speakers out there, good or bad, we are the ones making the conferences by and for.
• I like Zach's attitude as DPL, even when I introduced him to a very nice friend of mine that came to the party to hang out.

DebConf 10 rocked and I appreciate everybody's participation in it. See you in Banja Luka, next year!

At the beginning of DebCamp, Zack started the RCBC the Release Critical Bug squashing Contest. Two weeks later not only DebConf10 is over but we also have results for the RCBC:

In summary: It was a huge success, we managed to fix

• 139 RC bugs in testing/unstable
• 79 RC bugs in stable

And the winners are:

• Stable: Gerfried Fuchs (77 bugs)
• Testing/unstable: 1. Moritz M hlenhoff (32 bugs), 2. Nico Golde (17 bugs), 3. Alexander Reichle-Schmehl (16 bugs)

All details can be found at http://wiki.debconf.org/wiki/DebConf10/RCBC.

The winners who haven't collected their prizes at the closing ceremony will get them be snail mail.

Finally let me thank

• our donors of the prizes
• Micah and Zack for initiating the contest
• Mehdi for being my co-judge
• all participants who hugely improved Debian in the last two weeks

(This blog has slept since I wrote the previous post. That post had to wait for a year to be made public! Anyway, here I go again ) One of the ways people characterise science and technology is by saying that science is about prediction and technology is about design. This is not wrong but I feel that it misses something. An astrologer too claims to predict the future and claims to tell you how to re-design your future. So the above explanation of science and technology could make it indistinguishable from magic! One thing that does distinguish science is that the prediction and design is done by calculation ; this is not restricted to mathematical calculation but includes recipes like mix so much of this reagent with so much of that solvent . In fact, the notion of calculation can be expanded to anything that I can teach someone else to do even an idiot like a computer. This leads us to the view that science is not only about making predictions and designing things but also about teaching others to do the same. At the very first level, this is done by providing formulas which other people can use to make predictions and design things. However, this is not quite far enough, since it would still make the act of creating these formulas a magical thing! So we must take this a step further and be able to teach others how to create their own formulas. In other words, we want to provide formulas for creating new formulas. It does look as if we could be in infinite regression here, for who will then create formulas to provide formulas for creating new formulas! Luckily, the monadic thinking of Turing helps us here. He realised that formulas for creating new formuals are themselves just (slightly more complicated) formulas. Just as a gift-wrapped box containing a gift can be thought of as just a box containing a gift.

On my last night in New York, I didn't sleep much. At 6am, I said farewell to Central Park by running round the reservoir, which I hadn't yet done. There was a very nice red sunrise to be seen from the west side. Unfortunately I didn't sleep much on the flight home either. The British accents sounded quite unusual when we landed in Heathrow, and it was quite confusing not being able to find a Starbucks. Once I was back home, I crashed, and woke up at 10pm. I spent last night clearing the pkg-perl review queue - gregoa is taking a short break after DebConf. Then I went running at sunrise again. This is quite a different experience to Central Park - first, you have to run 2.5km just to get to Southampton Common, and secondly it is raining quite heavily. I dug out some winter gear that had turned out to be completely inappropriate for New York.

So, DebConf time is over once again. The two weeks worth of fifty weeks waiting are left behind once again, and it's back to get back to normal. DebConf was great Yes, it always is, and that's what we are all saying, but hey - Seriously! Being in the same building than 300 crazed developers is always fun, and it's always better than last year's fun. A good highlight this year is that, given the number of Free Software and Free Culture groups that exist in USA's north-eastern coast, we had the opportunity to join a large crowd which has never been part of DebConf. Also, I must agree that the USA bid for DebConf was aiming to attract as many Debian people (developers, maintainers, or just happy users) which had not yet been to a DebConf before as possible. And it was a great success! I finally met several people I have long read in the mailing lists, in blogs or in IRC. A much higher proportion than usual, I'd venture to say. Another interesting phenomenon /methinks is that this year's DebCamp started much more staffed than usual: I arrived on the first day, Sunday 25, and there were ~40 people there already; I don't have the actual numbers, but we quickly grew, and the number started to stabilize past mid-week, only to (sharply) rise in the weekend, in time for DebianDay and DebConf start. Great time! But, they say, nobody can go to the USA without buying some sweet toys, right? Well, being the proud owner of six very hairy cats, I have thought into entering the looming and weaving industry... But cat hair, while abundant, I have heard is untreadable... Maybe due to the indisciplined, natural and independent personality of the cats (catonality should I say?)... So I had two choices: Clean up my home quite often, or live in a literally hairy mess. Enter choice #3: The Roomba! I had been waiting to buy this thing for several years, as they refuse to send to Mexico or charge Mexican cards. So, I walked across Manhattan and got my very own robot cleaner! For my further surprise, although I have not yet tried it (I don't even have a suitable cable yet), I found this: Yay, the Roomba is actually hackable (via a 7 pin miniDIN serial port)! Wikipedia says that:

Roomba comes with a Mini-DIN TTL serial interface, which is incompatible with standard PC/Mac serial ports and cables, both electrically and physically. However, third-party adapters are available to access the Roomba's computer via Bluetooth, USB, or RS-232 (PC/Mac serial). New, 500-series, and 410/420 series Roombas upgraded with the OSMO hacker device allow the user to monitor Roomba's many sensors and modify its behavior. The Roomba Open Interface (formerly "Roomba Serial Command Interface") API allows programmers and roboticists to create their own enhancements to Roomba. ( )

My first impressions? Well, the Roomba lazily charged its battery throughout the day today, and was hungry and ready when I arrived home. It is a but louder than what I expected, and of course my cats were not thrilled by the presence of a eighth animated and apparently sentient being at home. Their initial reaction was of course to be verrry alert of the thing. Twelve eyes were constantly pointing at the Roomba while mine alternated between them. As they measured the thing's speed and (I guess) inferred its movement patterns, they started escaping upstairs A flat, round thing with no legs to be seen will not likely be able to climb the stairs. And they were completely right. At first, only Chupchic remained downstairs. After a bit, I went up to show them the Roomba didn't jump on us to eat our brains, and after a bit, Santa and Macusa joined. The Roomba roombed for maybe 90 minutes (this space is large, and decided it was enough... And slowly, the rest of them started coming down. I would not say Roomba's cleaning is perfect, of course. Its room discovery algorithm is funny, and it even seems it's based on the mere chance of covering most (never all) of the space it has to clean. I had, of course, not fully studied it (after a single run, how could I?). It does make a honestly good attempt at cleaning under coaches, chairs and tables. It collected a fair amount of dust (on a house that seemed quite clean to me, I cannot imagine what would happen on a messy one). I have not yet played with the virtual walls (infrared transmitters which limit rooms as if a door was closed), but given the size of this house (and that I don't want it to clean around the cats' designated bathroom area), I guess I will end up using them regularly. During DebConf, I heard one bad (stupid useless noisy thing) and two very good (it has radically changed my life) comments on the Roomba. I hope to shift the balance towards 3/4 and not towards 2/2! Anyways... Thanks to each and every one of you. DebConf is great. Always great. Always a success. I cannot even thank specific teams. Debian Rules, and DebConf Rocks!

DebConf was really an awesome conference. I m happy that I have finally met so many of the fellow Debian people I only knew from the mailing lists or planet. There where lot s of interesting talks, the hacklabs where always busy, and the overall atmosphere was very good. Now that I m back in Berlin, I m already missing it I m by the way also missing my luggage which is apparently still at the JFK airport. This was my first DebConf, but definitively not my last. I encourage people who never made it to DebConf: Get your butt up and register for the next conference! You don t know what you re missing. If money is an issue: there is a sponsoring which you can apply for and chances are good that you ll get at least partially sponsored. Big thanks to all the volunteers who helped to organize and run this conference and to SPI for sponsoring my trip, food and accommodation! I m looking forward seeing some of you again in three weeks at Steve s place in Cambridge for a nice BBQ and the rest of you next year in Bosnia and Herzegovina!

So, I have finally visited my first Debconf, and it was a phenomenal experience. To meet persons who have guided you, persons work with, and persons you respect face-to-face and have conversations with them about several topics ranging from their vision for Debian to their views on topics totally unrelated to software is something truly great. I am now convinced that events like this, which provide the right environment to socialize and discuss various issues concerning Debian, will go a long way into getting several aspects of the project moving along in the right directions, as well as create awareness to the outside world about what the project is about and how it benefits users.In terms of discussions, the Debian Science track was a nice way to see how contributors are putting Debian to good use for scientific research, and it was a nice opportunity to list out the things to be done to fill in the gaps and take Debian as a science platform forward. The Debian Python BoF was also a good place to meet many contributors face-to-face, and learn more about the technical concerns which people have in regard to the future of Python in Debian.While I didn't hack much during the conference, Christian's presence made me finish the Hindi translation of the Debian Installer. We can now shift focus to other aspects, such as getting packages in order for a short freeze-to-release cycle.Columbia University provides for a very nice venue, with easy access while maintaining a quaint environment within itself. Since this is my second visit to New York City, I had an idea of the common tourist locations, so I did not go around much during the conference. However, since I have a buffer day, I might try my hand at some photography around the place.Have a safe trip home, and a wonderful time ahead! I leave you with a photo I took when I went with a friend across the Brooklyn Bridge after Debconf.

If you get messages like this:

ACPI Error (psparse-0537): Method parse/execution failed [\_SB_._OSC] (Node ffff8801e8c62b30), AE_AML_BUFFER_LIMIT

then you've fallen foul of one of the less appealing aspects of ACPI. _OSC methods are defined as methods to allow the operating system and the firmware to handshake over their support of optional features. Different _OSC methods apply to different types of hardware. CPU _OSC methods allow the operating system to inform the firmware that it supports ACPI 3.0 throttling states, while PCIe _OSC methods allow the operating system to indicate that it can manage native PCIe hotplug. The firmware can choose whether or not to give up control of these features, and the OS then has to cope.

The problem arises when we get to the _OSC method on the system bus. This wasn't specified until ACPI 4.0, leaving an attractive mechanism for vendors to add OS/firmware integration. To that end, we now have at least three different _SB_._OSC methods in the wild:

1. The ACPI specified _OSC. This exists purely for the OS to tell the firmware what it supports, without the firmware having the opportunity to disagree. As such it passes 8 bytes of data to the method.
2. The Microsoft WHEA _OSC. This exists to allow the firmware and Windows to handshake over whether or not the firmware supports Microsoft's hardware error reporting.
3. HP's PCC _OSC method, designed to allow handshaking between the OS and the firmware in order to determine whether they support OS interaction with the firmware-level CPU scaling

This wouldn't be enough to be a problem in itself. The ACPI spec requires _OSC methods to have GUIDs in order to protect users from exactly this kind of situation - Windows can attempt to enable WHEA on a machine with a spec-compliant _OSC, and the _OSC method will return an immediate failure because the GUID doesn't match. Except that the WHEA and PCC versions of _SB_._OSC pass 12 bytes of data in the third argument, against the ACPI spec version's 8. And many _OSC implementations attempt to access the region between bytes 9 and 12 before checking the GUID, resulting in the AE_AML_BUFFER_LIMIT error.

This is made even more annoying due to the fact that argument 2 contains the number of parameters being passed in argument 3, making it straightforward to avoid this kind of failure. Firmware authors, tbh.

The svn repository used to draft the Debian Project News just passed the sixth hundreds commit. If my calculations are correct, that's quite more than the number of edits done, back when the DPN was drafted in the wiki. I take it as a hint, that the new work flow works better than the old one. However, playing a bit with statistics (done by horrible shell one liners ;) I noticed, that - while every Debian Developer may commit to the subversion repository - not that many commits from other DDs where made: Of the 602 commits so far, only 70 where done by other DDs:

alex@melusine:~$ svn log --xml svn://svn.debian.org/svn/publicity \
 xmlstarlet sel -t -m "/log/logentry/author" -v "concat(.,' ')"   \
sed -e "s/ /\n/g" sort uniq -c sort -n grep -v guest grep -v tolimar
      1
      1 abe
      1 hertzog
      1 holger
      1 mika
      1 paravoid
      2 pabs
      3 alfie
     15 zobel
     20 gio
     25 spaillard

To give credit where credit is due, here are the guest commits:

     16 tpeteul-guest
     21 jeremiah-guest
     25 gmascellani-guest
     36 madamezou-guest
    122 taffit-guest

Many thanks so far, but I would like to advertise the Debian Project News and invite every DD to help us and to commit directly. It's quite easy: Run svn co svn+ssh://svn.debian.org/svn/publicity/dpn/en/current, edit the index.wml file, and then commit your changes back to the repository. (Well, ideally you would also honor Status flag and won't commit, if it's not open-for-edit.) As for the format: While we indeed use wml, you just need some basic HTML knowledge. To add a paragraph just use the following with proper content:

<a name="X"></a>
<h2>Fancy title</h2>
<p>More details about the topic.</p>

Usually the articles are kind of sorted by importance, so unless you are pretty sure it might be the easiest to just add your article at the end of the regular news, just before the other news sections. Should you not be able to fill an entire paragraph, feel free to just a one or two sentences to the other news section (just add a <p>...</p> with your content to the end of the other news). Don't worry about style, your English or syntax: It won't end up on the web page directly, it's reviewed and checked, so you can't do anything wrong. More details are available in the wiki at http://wiki.debian.org/ProjectNews/HowToContribute. Feel free to ask any questions unanswered on the publicity list at debian-publicity@lists.debian.org. Feel also free to contact us there, if you would like to help by translations or reviews. The more people help, the less work it's for everyone :) If you are not a Debian Developer, but still would like to help us, it's no problem. All you need is an account on our Alioth System (very easy to get, you just need to ask) and request to join the publicity project. See http://wiki.debian.org/ProjectNews/HowToContribute#Becominganeditoryourself for the respective links.

My rule to see if a framework is worth of attention I came up with a little rule:

In order to be worth of any attention, a framework must be stable enough that I can charge money to train people to use it.

This probably applies to other kinds of software stacks, libraries, development environments and, well, to most software applications. In the context of python web frameworks, this means that:

• If it changes API all the time it is not worth of attention, because my customers won't get value for their money, as they'd continuously need retraining and rewriting their software.
• If I see lots of DeprecationWarnings it is not worth of attention, because my customers will see them and blame me for teaching them deprecated stuff.
• If fixes for bugs affecting the stable version are only distributed "in a recent git" or "in the next development version", and they are not backported into a new bugfix-only stable release, then it is not worth of attention, because:
  • my customers' business is to develop their own products based on the framework.
  • My customers' business is not to be maintaning in-house stable updates of the framework. Although if the framework's community is nice enough they might end up giving a hand.
• If it requires virtualenv or can only be obtained through easy_install it is not worth of attention, because:
  • my customers are not interested in maintaning custom deployment environments over time.
  • My customers are not interested in tracking each and every single library's upstream development to keep their production system free of bugs.
  • My customers are used to getting software through a proper distribution which also takes care of security updates.
  • I am paid to teach them how to use a framework, not a custom python-only package management system.
  • In my experience, if distributions have trouble keeping packages up to date, upstream is doing something fundamentally wrong.

In light of this rule, I regret to notice that I see very few python web frameworks worth of any attention.

Eben Moglen's talk at DebConf10 has inspired a flood of responses and activity at DebConf10. The original talk overran the allotted time within the venue and then outside the venue for something like an hour, HackLab discussions migrated to the single theme and brought in more and more people over lunch. When the Skype talk was left without a speaker in the afternoon, it morphed into a FreedomBox BOF with maybe 100 delegates planning and contributing to the ideas and another BOF is planned for Friday. A rough outline is appearing on the wiki page for those who want to follow it and the video of the original talk is available, linked from that Wiki page.

What gets people excited about this project is that we already have all the components to do the work, people have been doing most of this stuff at home, on their own, for some time and it needs to be brought together as a series of packages and meta packages and other middleware glue to get it working. (multistrap is part of that glue - it's already being used commercially in a similar role. Debian Pure Blends is another part of the glue. This can be a Debian solution, it doesn't have to be a separate project.)

The software and glue doesn't have to be dependent on a single hardware platform, this is about getting the software right and deploying it on whatever hardware can run it - albeit that the architecture itself is likely to be ARM-something.

It is time for social networking to be freed from the man-in-the-middle - time to withdraw user data from central servers owned by some third-party and share directly with your friends (whichever network they use). Aggregate all data from all your friends, keep all your data in your own plug and access it from wherever you are using whatever devices you have to hand. What's more, your friends - especially the non-techy friends - can just buy a cheap beige box that hides away in the corner of their home and have all the same access too - everyone gets to choose exactly which pieces of data get shared and to whom and can then use their friends plugs to store encrypted backups of their other data. Or maybe the final solution will be slightly different - it's up to people in Debian to make it happen. The tools are in our own hands, if we want this, we just need to do it.

What is needed now is some direction, there are choices to be made about how the glue is designed and how this is turned into a "self-discoverable" distributed network service. Yes, IPv6 will make this easier but only once ordinary users can deploy the device transparently and without configuration, so an interim solution will be needed.

Who's up for the challenge?

I dunno why, but hearing bagpipes always gives me the creeps, probably because their melodies sound like a whole country was just brutally slaughtered and is being mourned. Sure enough, this morning, on my way to pick up my new national ID card matching my new citizenship, I ran into a Scottish regiment warming their bagpipes in preparation for the Police marching band festival of Helsinki and instantly had goosebumps, to the point of running a mental list of everyone among my relatives who might have crossed the Styx during the last few hours. While my mother's recent ailments have indeed been a source of concern, I simply couldn't think of any reason for them to degenerate to a fatal extent, so I mentally crossed that possibility out. Still, even after marching a few blocks further downtown towards the central police station, I simply couldn't shake the deep sadness that got onto me upon hearing the bagpipes rehearsal. By the time I reached my destination, an answer came: before I could get my new ID card, I'd have to surrender my old Foreigner's ID card. As odd as it might sound, I felt sad to let go of that pink-tinted little bugger. See, as much as I've hated being a foreigner all these years, that pinkish ID card was my only legal tie to this country and the photo of me it bears represented an important phase of struggles in my life, a phase that I would have loved to document in any possible way. Alas, it was not possible, so I gave one last look at my old card, handed it over and took my new blue-tinted citizen's ID card in exchange. Walking back across downtown towards my home, all I could do was stare at my new ID card in disbelief: the Citizenship line indeed said FIN. I suppose that was the real message: C' tait la fin d'une citoyennet et le d but d'une autre. Une petite mort, une grande (re)naissance. Seh n on hyv vaihtokauppa, miun puolesta.

Debian always was known for its communication "style". There were even shirts sold in memory of Espy Klecker with a quote he is known for: Morons. I'm surrounded by morons. Yes, I bought me one of those shirts too in the early days. And there were the talks that promoted Debian as a place to have Good flamewar training. And people considered that to be the fun part. After some years it got tiring. It got stressful. It got annoying. Bad feelings popped up, stirred you into the next flamewar, and it went down the gutter from there. It was almost becoming impossible to not be the target of a flamewar when one was doing more than just basic maintenance. Snide and extreme terse responses became the standard. In the end people are starting to give up and leave. The Ugly thing about this is that human resources are crucial. They aren't endless and can't be replaced as easily as broken hardware, especially when capable people or when people leave who invested an enormous amount of their spare time and effort. And given that a fair amount of people do put their heart into Debian, it feels like a small suicide to them and the public thinking about leaving is meant as a call for help which wasn't and isn't given. The solution to this death swirl? I'm not sure. When one looks over the edge of the plate and ignores for a moment all the bad feelings they one might have built up against Ubuntu because of their success and possibility to find new contributors on a regular basis one is able to find a much friendlier and productive environment there. This might be attributed to the Code of Conduct about which I wrote about last year already and which is an extremely well intended and useful document (the point I raised in there is already solved for a while, so I became a MOTU). And even if it might be hard to follow it at times, Mark Shuttleworth reminds and encourages its contributors to stick to these principles even in tough times. The result? When following the planets, one finds on Planet Ubuntu a very good rate of blog posts on things that had been done, compared to the good rate of blog posts of rants on Planet Debian. And even though people regularly complain about the communication style within Debian, the answers of this year's DPL candidates to the question about a code of conduct for Debian were rather rather disappointing. So it is just well too understandable that people go the path that hurts themself, take a cut and leave the project behind in its mess. For myself? I'm not too far from that point on a regular basis, and I can understand those who did the final step only too well. Regular abuse, especially when doing stuff that others neglect on a regular basis but needs to be done anyway, being belittled on that grounds and not being taken serious and getting disrespectful responses isn't improving the situation. It happens to way too many people, and the only thing that still keeps me on tracks is that I do not want to give in yet, that I don't think that it would improve Debian to leave the grounds to various destructive people. On the other hand, there is only so much abuse one can take...
ObTitle: Ennio Morricone - The Good, The Bad and The Ugly

/debian permanent link Comments: 5

Debian always was known for its communication "style". There were even shirts sold in memory of Espy Klecker with a quote he is known for: Morons. I'm surrounded by morons. Yes, I bought me one of those shirts too in the early days. And there were the talks that promoted Debian as a place to have Good flamewar training. And people considered that to be the fun part. After some years it got tiring. It got stressful. It got annoying. Bad feelings popped up, stirred you into the next flamewar, and it went down the gutter from there. It was almost becoming impossible to not be the target of a flamewar when one was doing more than just basic maintenance. Snide and extreme terse responses became the standard. In the end people are starting to give up and leave. The Ugly thing about this is that human resources are crucial. They aren't endless and can't be replaced as easily as broken hardware, especially when capable people or when people leave who invested an enormous amount of their spare time and effort. And given that a fair amount of people do put their heart into Debian, it feels like a small suicide to them and the public thinking about leaving is meant as a call for help which wasn't and isn't given. The solution to this death swirl? I'm not sure. When one looks over the edge of the plate and ignores for a moment all the bad feelings they one might have built up against Ubuntu because of their success and possibility to find new contributors on a regular basis one is able to find a much friendlier and productive environment there. This might be attributed to the Code of Conduct about which I wrote about last year already and which is an extremely well intended and useful document (the point I raised in there is already solved for a while, so I became a MOTU). And even if it might be hard to follow it at times, Mark Shuttleworth reminds and encourages its contributors to stick to these principles even in tough times. The result? When following the planets, one finds on Planet Ubuntu a very good rate of blog posts on things that had been done, compared to the good rate of blog posts of rants on Planet Debian. And even though people regularly complain about the communication style within Debian, the answers of this year's DPL candidates to the question about a code of conduct for Debian were rather rather disappointing. So it is just well too understandable that people go the path that hurts themself, take a cut and leave the project behind in its mess. For myself? I'm not too far from that point on a regular basis, and I can understand those who did the final step only too well. Regular abuse, especially when doing stuff that others neglect on a regular basis but needs to be done anyway, being belittled on that grounds and not being taken serious and getting disrespectful responses isn't improving the situation. It happens to way too many people, and the only thing that still keeps me on tracks is that I do not want to give in yet, that I don't think that it would improve Debian to leave the grounds to various destructive people. On the other hand, there is only so much abuse one can take...
ObTitle: Ennio Morricone - The Good, The Bad and The Ugly

/debian permanent link Comments: 8

It s widely regarded that to solve reliability problems you can just install a cluster. It s quite obvious that if instead of having one system of a particular type you have multiple systems of that type and a cluster configured such that broken systems aren t used then reliability will increase. Also in the case of routine maintenance a cluster configuration can allow one system to be maintained in a serious way (EG being rebooted for a kernel or BIOS upgrade) without interrupting service (apart from a very brief interruption that may be needed for resource failover). But there are some significant obstacles in the path of getting a good cluster going. Buying Suitable Hardware If you only have a single server that is doing something important and you have some budget for doing things properly then you really must do everything possible to keep it going. You need RAID storage with hot-swap disks, hot-swap redundant PSUs, and redundant ethernet cables bonded together. But if you have redundant servers then the requirement for making one server reliable is slightly reduced. Hardware is getting cheaper all the time, a Dell R300 1RU server configured with redundant hot-plug PSUs, two 250G hot-plug SATA disks in a RAID-1 array, 2G of RAM, and a dual-core Xeon Pro E3113 3.0GHz CPU apparently costs just under $2,800AU (when using Google Chrome I couldn t add some necessary jumper cables to the list so I couldn t determine the exact price). So a cluster of two of them would cost about $5,600 just for the servers. But a Dell R200 1RU server with no redundant PSUs, a single 250G SATA disk, 2G of RAM, and a Core 2 Duo E7400 2.8GHz CPU costs only $1,048.99AU. So if a low end server is required then you could buy two R200 servers that have no redundancy built in which cost less than a single server that has hardware RAID and redundant PSUs. Those two servers have different sets of CPU options and probably other differences in the technical specs, but for many applications they will probably both provide more than adequate performance. Using a server that doesn t even have RAID is a bad idea, a minimal RAID configuration is a software RAID-1 array which only requires an extra disk per server. That takes the price of a Dell R200 to $1,203. So it seems that two low-end 1RU servers from Dell that have minimal redundancy features will be cheaper than a single 1RU server that has the full set of features. If you want to serve static content then that s all you need, and a cluster can save you money on hardware! Of course we can debate whether any cluster node should be missing redundant hot-plug PSUs and disks. But that s not an issue I want to address in this post. Also serving static content is the simplest form of cluster, if you have a cluster for running a database server then you will need a dual-attached RAID array which will make things start to get expensive (or software for replicating the data over the network which is difficult to configure and may be expensive), so while a trivial cluster may not cost any extra money a real-world cluster deployment is likely to add significant expense. My observation is that most people who implement clusters tend to have problems getting budget for decent hardware. When you have redundancy via the cluster you can tolerate slightly less expected uptime from the individual servers. While we can debate about whether a cluster member should have redundant PSUs and other expensive features it does seem that using a cheap desktop system as a cluster node is a bad idea. Unfortunately some managers think that a cluster solves the reliability problem and therefore you can just use recycled desktop systems as cluster nodes, this doesn t give a good result. Even if it is agreed that server class hardware is used for all servers so features such as ECC RAM are used you will still have problems if someone decides to use different hardware specs for each of the cluster nodes. Testing a Cluster Testing a non-clustered server or some servers that use a load-balancing device at the front-end isn t that difficult in concept. Sure you have lots of use cases and exception conditions to test, but they are all mostly straight-through tests. With a cluster you need to test node failover at unexpected times. When a node is regarded as having an inconsistent state (which can mean that one service it runs could not be cleanly shutdown when it was due to be migrated) it will need to be rebooted which is sometimes known as a STONITH. A STONITH event usually involves something like IPMI to cut the power or a command such as reboot -nf , this loses cached data and can cause serious problems for any application which doesn t call fsync() as often as it should. It seems likely that the vast majority of sysadmins run programs which don t call fsync() often enough, but the probability of losing data is low and the probability of losing data in a way that you will notice (IE it doesn t get automatically regenerated) is even lower. The low probability of data loss due to race conditions combined with the fact that a server with a UPS and redundant PSUs doesn t unexpectedly halt that often means that problems don t get found easily. But when clusters have problems and start calling STONITH the probability starts increasing. Getting cluster software to work in a correct manner isn t easy. I filed Debian bug #430958 about dpkg (the Debian package manager) not calling fsync() and thus having the potential to leave systems in an inconsistent or unusable state if a STONITH happened at the wrong time. I was inspired to find this problem after finding the same problem with RPM on a SUSE system. The result of applying a patch to call fsync() on every file was bug report #578635 about the performance of doing so, the eventual solution was to call sync() after each package is installed. Next time I do any cluster work on Debian I will have to test whether the sync() code seems to work as desired. Getting software to work in a cluster requires that not only bugs in system software such as dpkg be fixed, but also bugs in 3rd party applications and in-house code. Please someone write a comment claiming that their favorite OS has no such bugs and the commercial and in-house software they use is also bug-free I could do with a cheap laugh. For the most expensive cluster I have ever installed (worth about 4,000,000 UK pounds back when the pound was worth something) I was not allowed to power-cycle the servers. Apparently the servers were too valuable to be rebooted in that way, so if they did happen to have any defective hardware or buggy software that would do something undesirable after a power problem it would become apparent in production rather than being a basic warranty or patching issue before the system went live. I have heard many people argue that if you install a reasonably common OS on a server from a reputable company and run reasonably common server software then the combination would have been tested before and therefore almost no testing is required. I think that some testing is always required (and I always seem to find some bugs when I do such tests), but I seem to be in a minority on this issue as less testing saves money unless of course something breaks. It seems that the need for testing systems before going live is much greater for clusters, but most managers don t allocate budget and other resources for this. Finally there is the issue of testing issues related to custom code and the user experience. What is the correct thing to do with an interactive application when one of the cluster nodes goes down and how would you implement it at the back-end? Running a Cluster Systems don t just sit there without changing, you have new versions of the OS and applications and requirements for configuration changes. This means that the people who run the cluster will ideally have some specialised cluster skills. If you hire sysadmins without regard to cluster skills then you will probably end up not hiring anyone who has any prior experience with the cluster configuration that you use. Learning to run a cluster is not like learning to run yet another typical Unix daemon, it requires some differences in the way things are done. All changes have to be strictly made to all nodes in the cluster, having a cluster fail-over to a node that wasn t upgraded and can t understand the new data is not fun at all! My observation is that the typical experience of having a team of sysadmins who have no prior cluster experience being hired to run a cluster usually involves learning experiences for everyone. It s probably best to assume that every member of the team will break the cluster and cause down-time on at least one occasion! This can be alleviated by only having one or two people ever work on the cluster and having everyone else delegate cluster work to them. Of course if something goes wrong when the cluster experts aren t available then the result is even more downtime than might otherwise be expected. Hiring sysadmins who have prior experience running a cluster with the software that you use is going to be very difficult. It seems that any organisation that is planning a cluster deployment should plan a training program for sysadmins. Have a set of test machines suitable for running a cluster and have every new hire install the cluster software and get it all working correctly. It s expensive to buy extra systems for such testing, but it s much more expensive to have people who lack necessary skills try and run your most important servers! The trend in recent years has been towards sysadmins not being system programmers. This may be a good thing in other areas but it seems that in the case of clustering it is very useful to have a degree of low level knowledge of the system that you can only gain by having some experience doing system coding in C. It s also a good idea to have a test network which has machines in an almost identical configuration to the production servers. Being able to deploy patches to test machines before applying them in production is a really good thing. Conclusion Running a cluster is something that you should either do properly or not at all. If you do it badly then the result can easily be less uptime than a single well-run system. I am not suggesting that people avoid running clusters. You can take this post as a list of suggestions for what to avoid doing if you want a successful cluster deployment.

After coming back from holiday where I made lots of photos and videos, I needed the possibility to import all those from SD cards to my fileserver running Debian unstable. To import the photos I used Digikam on Linux, that was no problem. But for the videos I needed iMovie and/or FinalCut Express on my Mac. I've looked into Kino and Cinelerra, but those Linux apps are awful. The problem was that iMovie is able to directly import from our Panasonic HD camcorder to the network storage. I was always forced to first import the fotoage to the drive in my MacBook Pro and copy it then to the Linux box onto a Samba share. The solution for that problem is to use netatalk/AFP. The Linux box appears as a Mac Xserve then. However, I quickly found some HowTos for configuring netatalk. One HowTo is from Mike Hughes and the other one can be found on disgruntled-dutch.com. If you follow these two HowTos you're quite close to have a working AFP service to interact with your Mac via Bonjour/Avahi. There are mainly two problems when you follow them. First, Mike Hughes has some illegal chars in his sample /etc/avahi/services/afpd.servicefile:

<!DOCTYPE service-group SYSTEM avahi-service.dtd >
<service-group>
<name replace-wildcards= yes >%h</name>

The quotation marks are the problem. Instead of you'll need normal " quotation marks. Or you can use the file on disgruntled-dutch.com. Next problem is the configuration of the shares. On both sites the examples are using cdb as CNID backend:

~/ "Home Directory" cnidscheme:cdb options:usedots,upriv

But this will give the following error:

Aug 3 23:30:29 muaddib afpd[8151]: Can't open volume "/home/ij" CNID backend "cdb"
Aug 3 23:31:26 muaddib afpd[8151]: Cannot find module named [cdb] in registered module list!

The solution can be found on forum.ubuntuusers.de (German): you'll need to change the cdb to dbd. So, the Home Directory share will look like this then:

~/ "Home Directory" cnidscheme:dbd options:usedots,upriv

Quite simple and easy. Now restart your avahi-daemon and netatalk services and have fun! When you want to use a share on your Linux server for TimeMachine backups, you'll need to add a "tm" to the options of your share.