It took longer than usual, but I believe it's worth.

I just released Cupt 2.2.0~rc1 to Debian experimental. To experimental, because there are important changes and I would appreciate "in-field" testing before final 2.2.0.

Here are the major changes since 2.1.x:

• Dpkg action sequencer improvements. There are multiple improvements, both in quality and speed.


• Repository index deltas. Or, in APT terminology, "PDiffs". Finally, there is an initial support for them. They are switched on by default (if you have Recommends disabled, make sure you have 'ed' package installed). To switch off, set the option 'cupt::update::use-index-diffs' to 'no'.


• Action override options. New console front-end's options '--install', '--remove', '--purge', '--satisfy', '--unsatisfy' allow to mix any of corresponding actions to the single command. See the tutorial and the man page for examples and details.


• Logging. By default, system changes are now logged to /var/log/cupt.log. Refer to the tutorial section 'Logging' for tuning.

Enjoy and send the bug reports.

This is just a quick post to remind myself in the morning, as soon as I've made it I intend to turn my computer off and leave it off until I can re-organize my office. I've been using mutt for my email for the past few years. Nothing compares to the flexibility of procmail/sieve for organizing server-side mail, and then mutt is ideal for reading them. With the addition of the mutt-patched sidebar mode you can even go for a few days before realizing you're not in a graphical environment. But one thing I do long for is the ability to execute scripts at various times. Thus far I've not actually planned what I'd like to do, but as a starting point imagine being able to execute a hook when new mail arrives? Or when you send a message matching a pattern in some fashion? There are some things out there, such as the various hacks which are designed to abort sending a message if you mention "See attachment" in a message body but fail to add one before sendign the message. These hacks generally abuse the sendmail configuration such that they're extremely ad-hoc and hard to chain/nest. I've mellowed out over the years and I have no interest in attempting to write a mail-client (though at the same time how hard can it be? Just restrict yourself to using inotify on ~/Maildir and offload delivery to exim and you're almost done? I guess the hard part is the UI, though I do like the mutt + sidebar layout. Write the whole thing in some scripty language?) I'll re-examine notmuch and gnus over the next week or two, but I suspect both will continue to disappoint in various ways. Anyway, for the moment I'm just pondering. But threading is an obvious concern. Most current mutt hooks relate to the local folder, or the local message. If I were viewing a message in one directory and a new mail notification fired for a delivery to both ~/Maildir and ~/Maildir/.people.foo I'd need to either serialise them or thread them. Ponder ponder. In other news I've been doing more photography recently. Nothing cohesive except for my recent experiment with shooting a "street-girl" outdoors in falling light, but that was an interesting challenge and the results were sufficient to make me want to try shooting outdoors in an organized fashion again. (Some random images have been linked to from my wee twitter page.) ObFilm: "She doesn't get eaten by the eels at this time " - The Princess Bride

I'm not very good with dates. For the last two years now, Joshua's anniversary has snuck up on me and caught me by surprise. I don't think that I'd really appreciated until this year that we lost him on the Labor Day long weekend. I'll have to remember that for next year so I don't forget. I don't know what my problem was this year. I just didn't realise that it was this early in September that we lost him. Last year I was just too exhausted from dealing with a new baby to realise when his anniversary was, and I felt terrible for missing it. This year, I just felt terrible for not realising when it was, but at least Sarah pointed it out before the fact. For added irony, the 4th of September this year is Father's Day in Australia. Yesterday, I was looking at photos of Joshua to remember him, and Gallery gave me a random photo of Zoe in the corner. That was something. I'm not sure what. I feel like Zoe being a wonderful child is some sort of consolation prize from the universe for taking Joshua, but I often wonder what Joshua would be like today if he were almost four.

This post is a small follow-up on yesterday's post about the Linux graphics stack. I'm still keeping to high level stuff and won't dig too deep, but a few questions arose (others reached me by e-mail), so, here we go. Gallium3D is a framework, initially developed by Tungsten Graphics, which were bought by VMware. The idea behind Gallium3D is, that many functions can be shared between drivers for different GPUs and that the effort to support a new platform/system is for classic drivers pretty high. With Gallium3D you just need a state tracker and a target. The state tracker is code, that gives the pipeline of Gallium3D drivers access to a platform/system, like OpenGL or X on various operating systems. A target is the required as glue between a state tracker and the actual driver. For example: there is a state tracker for video playback acceleration (the recently added g3dvl). The state tracker itself offers the interface and common functions (with some help from the auxilliary stuff) between the Gallium3D pipeline and the userland libraries like libvdpau1. Targets like vdpau-[DRIVER] glue the driver and the state tracker together. The winsys part of Gallium3D is a similar concept but it abstracts the windowing system on the target platform away (e.g. Wayland, X or GDI). I didn't write about the proprietary drivers yesterday, as I don't consider them a real part of the Linux graphics stack, more like parasites, and didn't want to encourage anyone to use them (yes, I know, there are some cases in which you might be better off with them, but generally I'd avoid them). Anyway, I got some questions about those (some by e-mail, the other I'm aware off, can be found behind the link at the beginning of this entry). So, where do the proprietary drivers come in? They are basically the entire stack in a closed form, plugging into the public interfaces of the Kernel (whether that's legal is disputed), some other parts like X (by building a DDX) and building their own libGL. Especially the last part can be a cause for a lot of pain, when you don't use the proprietary driver packages offered by the distribution (e.g. Debian's fglrx packages divert the libGL and restore the default one on unistallation, with the vendor build scripts and tools this is not really guaranteed, which might leave you with a broken OpenGL setup). Another question was, why Mesa was lacking support for a recent OpenGL version (i.e. 4.x). In my opinion there are a few reasons, though I don't really feal like the best person to answer this. The first are design issues by some internal representations (e.g. missing integer support in the intermediate representations). The second is the available number of developers being able to implement new OpenGL functions. Apart from a not all too clear specification the Mesa code base is huge (even if you ignore most of the drivers), which makes it difficult for new developers to start coding away. Then you need an understanding of how modern GPUs operate to find good solutions for the problems faced. If you want to get an idea for this, take a look at Zack Rusin's or Alex Deucher's blog. Then, there is a sad split of developers, on the one hand we've got the Gallium3D-using developers on the other we've got the ones staying with the classic driver model. This means, that you have to implement e.g. GLSL extensions for TGSI and Mesa's IR respectivly. There might be other reasons, I'm totally missing, but this should give you some insight into why it takes so long, to get current OpenGL versions supported. I hope this clears some of the remaining questions, if not, feel free to write me an e-mail, but please remember, that this is not an in-depth analysis.

While other people are squashing RC bugs I was using this week to fix (or investigationg) some more kFreeBSD issues -- mostly looking at failed build logs and trying to fix the problems and after some nice fish for dinner writing things up.

• First issue this week was #639178 a build failure in tar I had reported earlier and didn't manage to process the response. After sending some findings to the bug I noticed Petr was faster and did actually find out a lot more detail. Short story: success in that test suite requires linux behavior and the failure on kfreebsd is covered by what POSIX allows
• #640156 multiarch related changes resulting in a nonfunctional ldd breaking clutter-gst build
• #640012 postfix is hard-coding kFreeBSD versions up to 7 and therefore won't build on a 8.2 kernel. It also doesn't handle absence of NIS on Hurd and kFreeBSD #545970
• #640159 iozone3 just needed a bit of massaging to combine the FreeBSD backend with the linker flags needed for kFreeBSD
• Installing the build depends for openjdk-* resulted in a installation failure for some time. Looking closer it turned out a minimal testcase was installing menu and python2.6 together. Turned out dash's test builtin wasn't working #640334 because it was relying on the intuitive but not POSIX mandated behavior of the faccessat syscall #640325
• #640341 ed decided not to build on kfreebsd-i386 in the 40 minutes between -2 and -3 upload. Without any actual source changes. Just trying agan tricked it to build again but probably someone should look what went wrong actually
• #640378 leveldb needed enabling the FREEBSD_OS kind of build with the linux style of linker flags (additional -lrt)
• #640385 owfs was failing to some symbol difference (but otherwise building although being a *fs ;))
• the gcc family of packages still has some heisenbug repeatedly failing when doing regular builds on the buildds. Independent which one. Multiple times in a row. Building on my test VM or my notebook doesn't show that problem (but takes ~10h). Building on the same buildd in the same chroot with the same sbuild flags and it's still building fine.

[UPDATE 2011-09-05]I received some questions (or saw them), which I tried to answer with a follow-up entry.[/UPDATE] As I tried to explain this a few times in the past to others and had trouble myself, when I started using Linux, I thought I take some time today and write down what parts comprise the Linux graphics stack and how they interact. I'll ignore some parts which aren't too interesting to an average user at the moment, to keep this post as consise as possible. Apart from that, I hope I'm able to put all the relevant data into one spot. Let us start our little journey in the kernel. There, in a directory named gpu you'll find the drm directory, which contains all DRM drivers. In this post, we'll focus on those. The drivers in that directory are the kernel side of the Direct Rendering Infrastructure (DRI) and are responsible for managing concurrent access to the graphics hardware. They also provide interfaces to pass commands and data to the GPU. The DRI wiki explains the three main purposes of the DRM modules. The DRM module is also the part that decides whether KMS or UMS is used. Other acronyms you might hear with regard to graphics acceleration on Linux and are referring to the Kernel part are:

• GEM: an Intel-developed memory management system for GPUs.
• TTM: refers to the memory management system used by almost all non-Intel drivers (e.g. AMD's Radeon drivers). The TTM interface is "GEM-ified", meaning it offers the same API as GEM.

Apart from that, every driver has its hardware-specific parts like utilizing power-saving functions of the respective GPUs, loading firmware, if required, etc. Now we move from Kernel space to user land. We'll ignore any non-X and non-Mesa parts (like acceleration for TTYs). That means, that our next GPU specific part is most likely the DDX. (Please note, that the classic DDX might be replaced by a Gallium3D state tracker, currently that would be the xorg state tracker, but xorg will be replaced by XA. This is only an option if the Gallium driver for your GPU supports one of those state trackers, see the Gallium status page for further details.) The DDX for your GPU offers plain (2D) X acceleration on your hardware. The respective DDX' are generally named xf86-video-[NAME], where [NAME] is replaced by e.g. ati or intel. The DDX decides which X acceleration paths are available. Acronyms you might hear, referring to the DDX part of the graphics stack, are:

• EXA: even though it looks like an acronym, the X.Org wiki Glossary defines it as:

  acceleration architecture with no well-defined acronym

  EXA superseded the older XAA and offers 2D acceleration, often in conjunction with XRender.
• UXA: more or less EXA for Intel GPUs.
• GLX: actually not a part of the DDX, but an extension for X. It provides a way to draw with OpenGL into an window managed by X.
• XRandR: An extension for X which needs to be supported by the DDX to work. It offers a standardized way to set up (multiple) heads (e.g. resolution, position, rotation, etc.).

We're almost through: the last important part is the 3D acceleration offered by a Mesa driver. Mesa drivers come in two flavours: classic (e.g. all Intel drivers which are supported by Intel) and Gallium3D-based (like the drivers for Radeon chips from the R300 onwards). The difference here is, that the Gallium3D drivers try to share as much code as possible (and thus simplify writing a new driver supporting many platforms, systems and technologies), while the classic drivers are specific to certain platforms, chips, and technologies. They share much less code among each other. All parts of Mesa need direct access to the graphics hardware (DRI) The Mesa drivers use the 3D pipeline of the GPUs for accelerated drawing, even for non-3D applications like video acceleration (recently VDPAU, XvMC and VA support were added to Gallium3D, driver support followed quickly). Otherwise the Mesa drivers are mainly used for OpenGL acceleration. Users of OpenGL include scientific programs, CAD and of course games. This concludes our little journey through the various parts of the Linux graphics stack. I hope it helps in understanding the building blocks.

I often hear criticism of "negative campaigning" in the free software movement. For example, in reply to a blog post I once wrote about an FSF campaign, several people argued against, "negative campaigning of any sort, in any realm." Drawing an analogy to political smear campaigns, some members of the free software community have taken the position that negative campaigning in general is not useful and that negativity has no place in our advocacy. First, it is important to be clear on what we mean by a negative campaigns. I believe that there is a fundamental difference between speaking out against policies or actions and smear campaigns that employ untrue claims, ad hominem attacks, and that attempt to avoid a real conversation about issues. I will categorically condemn the latter form of smear campaigning in campaigns for software freedom or for anything else. That said, negativity directed at negativity has had a positive effect in many social movements. I have supported and participated in "negative" campaigns against proprietary software, software patents, DRM, centralized network services, and the firms behind these practices. I've done so because I believe that if one is taking an ethical position, it is justified, and often necessary, to not only speak about the benefits of freedom but against acts of dispossession and disenfranchisement. In some of the most effective social movements, unambiguously negative messages have been central. Should a campaign for abolishing child labor talk only about how valuable adult workers are to their employers or how happy kids are when they don't work? Should a campaign trying to abolish land mines talk only about the benefits of bomb-free fields and intact lower limbs? Should a free speech organization only speak out about the social welfare brought by a free press and never against acts of censorship? These may seem like outlandish comparisons but you can find people writing, only a couple centuries ago, about how slavery should be abolished by arguing in favor of the benefits of paid labor. Even if the economic arguments in favor of paid work are strong, these arguments seems irrelevant and offensive today. Whether slavery is more or less efficient is a moot point. Society has rejected it because it is wrong. We have made important strides toward eliminating injustices like child labor and slavery because activists waged decidedly negative campaigns against them and convinced others to join in opposition. In doing so, activists declared the status quo unconscionable and created an ethical responsibility to find alternatives and to redefine what was "realistic." While I will not suggest that the movement for software freedom is comparable in ethical weight to these other causes, I know that the free software mission is similar in kind. Of course, if one does not think that user control over technology is an ethical issue but is instead merely a matter of choice, one will probably oppose negative campaigns. It is also possible that a particular negative campaign is tactically unwise in that it is unlikely to reach a large audience, unlikely to change people's minds, or be difficult to carry out successfully. But such campaigns are a bad idea because they are ineffective, not because they are negative. Additionally, a movement that is purely negative and offers no reasonable alternative to the stated ill may also be unlikely to succeed. This is why, for example, I believe it is good that the FSF uses the large majority of its resources in the "positive" role of supporting free software. For those that do treat technological empowerment as an ethical ideal, it is both justified and essential to condemn the systematic disempowerment of others through non-free software just as we celebrate the benefits of software freedom. "Negative" campaigns against proprietary software, software patents, and DRM in music have already led our community to important -- if incomplete -- victories. The desire to right wrongs has been a critical part of our movement's success and of many others'. We would be wise not to give it up.

(Photo credit: Obey Arthur Liu; originally on Picasa, license.) Three weekends ago, I participated in a Debian bug squashing party. It was more fun than I had guessed! The event worked: we squashed bugs. Geoffrey Thomas (geofft) organized it as an event for MIT's student computing group, SIPB. In this post, I'll review the good parts and the bad. I'll conclude with beaming photos of my two mentees and talk about the bugs they fixed. So, the good:

• Attendance. As Luke put it, when he arrived, there was hardly anyone there. Then people showed up! And then mentorship and pedagogy happened! In all, 14 people attended, seven of whom are active in Debian or its derivatives (Ubuntu, Debathena).
• We took photos. Arthur took a wide panoramic photo, and I snapped a few good ones, too. (Thanks to Jessica McKellar for letting me borrow her camera.) Here's my full photo set.
• Good, free pizza. Props to geofft for ordering from Beauty's, and props to SIPB for picking up the tab.
• We fixed bugs! I mentored two new Debian contributors through their first release-critical bug fix. Both of them said they would attend similar events in the future. One of my mentees is now idling and asking questions in the #debian-mentors IRC channel.
• The Debian Events team republished the event. MadameZou posted the event to debian.org, giving it a nice stable URL.
• geofft made a good list of bugs to work on. He worked from the release-critical bug list, categorized the bugs by expected difficulty, and made somee short notes about how to fix the bug. This turned out quite useful.
• People printed music to the speakers. If this sounds weird, you should read the SIPB documentation for how to use Gutenbach (written by Liz Denys).
• We had a good ratio of developers to newcomers. This was hugely helpful; from what I saw, new people rarely waited very long for their questions to be answered. Thanks to Luke Faraone, Sam Hartman, Obey Arthur Liu, Karl Ramm, and Christine Spang from Debian, and Geoffrey Thomas from Debathena, for sharing knowledge.

The event was a success, but as always, there are some things that could have gone more smoothly. Here's that list:

• We hadn't prepared for people to do Debian development on Ubuntu. In particular, it took Luke and me about half an hour to come up with a pbuilder command line that worked.
• We didn't contribute to Ubuntu, just Debian. I actually think this is fine, but geofft did originally want people to work on both. He was disappointed, and we could have managed to do that if we had done more prep.
• We could have reached more prospective attendees. I know earlier in the year, daf organized a highly popular Debian packaging tutorial elsewhere on campus. I didn't see any of those people at the event, and I think some would have attended if they had known about it.
• Some people wanted a bit more context. In particular, attendees asked me about what makes a bug release-critical and what it means to do a non-maintainer upload. We could have taken some time to explain these to the whole group; instead, we dove right into hacking. Because we had so many mentors, I think people did get their questions answered, but we can't sustain that if the event grows in size.
• Mentees didn't always get high-quality help. geofft suggested Jessica work on one bug, and when she had a question, she asked Luke Faraone. Luke wasn't familiar with the bug, and he skimmed it, not realizing that the bug changed issues midway through the bug log. As a result, his initial help was unhelpful. After Luke, Jessica, and the bug were on the same page (so to speak), Jessica received clear, sound answers to questions.

Still, it turned out well! I did three NMUs, corresponding to three patches submitted for release-critical bugs by my two mentees. Those mentees were: Jessica McKellar is a software engineer at Ksplice Oracle and a recent graduate of MIT's EECS program. She solved three release-critical bugs. This was her first direct contribution to Debian. In particular:

• #636201: While updating the package for multiarch support, the maintainer of wildmidi presumably made a typo. A single character was missing from a "Replaces:" declaration in debian/control. This bug caused upgrades to fail. Sam Hartman uploaded an NMU based on a patch Jessica prepared.
• #626420: The scribus-doc package failed to build from source due to a missing build-depends entry. This one was super easy, and served as a good test package for setting up and using pbuilder.
• #565000: The switch to gcc-4.5 left hubbub unable to build from source due to new GCC warnings that made -Wall fail on the source, although it would succeed with earlier versions of the package. This led to long philosophical discussions about the particularly most reasonable fix: removing -Wall, or adding a more specific exception.

Jessica has since gotten involved in the Twisted project's personal package archive. Toward the end of the sprint, she explained, "I like fixing bugs. I will totally come to the next bug squashing party." Noah Swartz is a recent graduate of Case Western Reserve University where he studied Mathematics and played Magic. He is an intern at the MIT Media Lab where he contributes to DoppelLab in Joe Paradiso's Responsive Environments group. This was definitely his first direct contribution to Debian. It was also one of the most intense command-line experiences he has had so far. Noah wasn't originally planning to come, but we were having lunch together before the hackathon, and I convinced him to join us. Noah fixed #625177, a fails-to-build-from-source (FTBFS) bug in nslint. The problem was that "-Wl" was instead written in all lowercase in the debian/rules file, as "-wl". Noah fixed that, making sure the package properly built in pbuilder, and then spent some quality time with lintian figuring out the right way to write a debian/changelog. That's a wrap! We'll hopefully have one again in a few months, and before that, I hope to write up a guide so that we run things even more smoothly next time.

The kernel.org compromise has people talking about the security of git's use of sha1. Talking about this is a good thing, I think, but there's a lot of smug "we're cryptographically secure" in the air that does not seem warranted coming from non-cryptographers like me. Two years ago I had a discussion on my blog about git and sha1, that reached similar conclusions to what I'm seeing here: It seems that current known sha1 attacks require somehow getting an ugly colliding binary file accepted into the repository in the first place. Hard to manage for peer reviewed source code. We all hate firmware in the kernel, so perhaps this is another reason it's bad. ;-) Etc. Well, not so fast. Git's exposure to sha1 collisions is broader than just files. Git also stores data for commits, and directory trees. Git's tree objects are interesting because they're a bag of bytes that is rarely if ever manually examined. If there was a way to exploit git such that it ignored some trailing garbage at the end of a tree object, then here's an attack injection vector that would be unlikely to be caught by peer review. If you can change the content of a tree without changing its sha1, you can simply make it link to an older version of a file that had an exploitable problem. Or you can assemble a combination of files that results in an new exploitable problem. (For example, suppose a buffer size was hardcoded in two files in the kernel, and then the size was changed in both -- make a tree that contains one change and not the other.) Now, git's tree-walk code, until 2008, mishandled malformed data by accessing memory outside the tree buffer. Was this an exploitable bug in git? I don't know. It is interesting that the fix, in 64cc1c0909949fa2866ad71ad2d1ab7ccaa673d9 relied on the parser stopping at a NULL -- great if you want to put some garbage after the tree's filename. With that said, the particular exploit I describe above probably won't work -- I tried! Here's all the code that stands between us and this exploit:

        if (size < 24   buf[size - 21])
                die("corrupt tree file");
        path = get_mode(buf, &mode);
        if (!path   !*path)
                die("corrupt tree file");

Any good C programmer would recognise that this magic-constant-laden code needs to be careful about the size of the buffer. It's not as clear though, that it needs to be careful about consuming the entire contents of the buffer. And C programmers involved with git have gotten this code wrong before. tldr: If git is a castle, it was built just after cannons were invented, and we've had our fingers in our ears for several years as their power improved. Now the outer wall of sha1 is looking increasingly like one of straw, and we're down to a rather thin inner wall of C code.

Are you using DEP-5 in copyright file? Are you doing it right? Thanks to the work of Jakub Wilk, Lintian 2.5.3 will spot some of the regular mistakes (including syntax errors) and check if you are using the newest revision. This is far from the only thing Jakub Wilk has added to Lintian in this development cycle. Most of his contributions in 2.5.3 are listed next to a little [JW] in the changelog and I hope to see a lot more of those. Together with Jeremiah Foster, we have created a small POD document that hopefully will help aspiring developers. The file currently explains how the source code is divided, some core concepts and how to run specific tests in the test suite. The plan is to make it available on lintian.d.o along with a link to the git source code (#639974). Last time, I mentioned we have gotten rid of collection/fields by a simple improvement to our Lintian::Collect API. Turns out the same improvement could also be used on collection/source-control-file. This means we no longer create a file per field in the d/control of the source package. To avoid (any more) confusion, Lintian will now inform you if the package carries an override for a non-overridable tag (except if you use --quiet). On a related note, Lintian will now error out if it sees an unknown field in a vendor profile. Previously it would just silently ignore the field. There is one more change I would like to highlight and this one is probably best served with an example:

$ lintian --show-overrides ../lintian_2.5.2.dsc
N: We build-depend on cdbs for the test suite
O: lintian source: unused-build-dependency-on-cdbs
N: We build-depend on quilt for the test suite
O: lintian source: quilt-build-dep-but-no-series-file
N: We don't have a patch system for lintian itself
O: lintian source: patch-system-but-no-source-readme

Lintian will now attempt to extract and print the comment related to the override from the overrides file. The comment will be printed above each tag the override applies to (so you may see some repeated comments). The full documentation for how Lintian finds the comments (or why it does not find the comment) is available in the Lintian User Manual. The exact format of the comments being outputted may change in the near future. Particularly, we may add a marker to assist the tools generating lintian.d.o to find the comments. Hopefully we will be able to close #512901 soon. Beyond the changes above, I am very happy to see the bug count drop below 180 unfixed bugs[1]. We have not been this low there since the 2.5.0~rc1 upload (in February) and I hope we can keep it this time. Finally if you ever wanted to run your own little lintian.d.o , then you may find the Lintian harness proposal interesting. The basic idea the is to clean up the current tools and make a proper frontend out of them. Should you be interested, feel free to come with suggestions or requests. [1] Lintain bug page I use the number next to Outstanding under Status near the bottom. So pending bugs are considered fixed in my numbering.

In light of the fact that it seems possible to fit the plastic caps of a Debian branded Swiss Army Knife (Last orders today!) on an existing Swiss Army Knife (German written howto as PDF), I started to think about which Victorinox Cybertool would be the best fitting for me. And because the Victorinox comparison page doesn t really show diffs, just columns with floating text which are not very helpful for generating diffs in your head, I used command line tools for that purpose: wdiff Because the floating texts are not line- but just whitespace-based, the tool of choice is not diff but wdiff, a word-based diff. It encloses additions and removals in + + and [- -] blocks. (No, those aren t Japanese smileys although they look a lot like some. ^^). The easiest and clearest way is to copy and paste the texts from Victorinox comparison page into some text files and compare them with wdiff:

$ wdiff cybertool34.txt cybertool41.txt
+Schraubendreher 2.5mm,+ Pinzette, N hahle mit Nadel hr, +Holzs ge,+ Bit-Schl ssel( 5 mm Innensechskant f r die D-SUB Steckverbinder, 4 mm Innensechskant f r Bits, Bit Phillips 0, Bit Phillips 1, Bit-Schlitzschrauben 4 mm, Bit Phillips 2, Bit Hex 4 mm, Bit Torx 8, Bit Torx 10, Bit Torx 15 ), Kombizange( H lsenpresser, Drahtschneider ), Stech-Bohrahle, Kugelschreiber( auch zum DIP-Switch verstellen ), Mehrzweckhaken (Pakettr ger), +Metalls ge( Metallfeile, Nagelfeile, Nagelreiniger ),+ Dosen ffner( kleiner Schraubendreher ), Kleine Klinge, Grosse Klinge, Ring, inox, Mini-Schraubendreher, Kapselheber( Schraubendreher, Drahtabisolierer ), +Holzmeissel / Schaber,+ Bit-Halter, Stecknadel, inox, Schere, Korkenzieher, Zahnstocher

So this already extracted the information which are the seven tools which are in the Cybertool 41, but not in the Cybertool 34. Nevertheless the diff is still not easily recognizable on the first glance. There are several ways to help here. First wdiff has an option --no-common (the according short option is -3) which just shows added and removed words:

$ wdiff -3 cybertool34.txt cybertool41.txt
======================================================================
 +Schraubendreher 2.5mm,+ 
======================================================================
  +Holzs ge,+ 
======================================================================
  +Metalls ge( Metallfeile, Nagelfeile, Nagelreiniger ),+ 
======================================================================
  +Holzmeissel / Schaber,+ 
======================================================================

This is already way better to quickly recognize the actual differences. But if you still also want to see the common tools of the two knifes you need some visual help: One option is to use wdiff s --terminal (or short -t) option. Added words are then displayed inverse and removed words are shown underlined (background and foreground colors hardcoded as there is no invert colors style in CSS or HTML):

$ wdiff -t cybertool34.txt cybertool41.txt
Schraubendreher 2.5mm, Pinzette, N hahle mit Nadel hr, Holzs ge, Bit-Schl ssel( 5 mm Innensechskant f r die D-SUB Steckverbinder, 4 mm Innensechskant f r Bits, Bit Phillips 0, Bit Phillips 1, Bit-Schlitzschrauben 4 mm, Bit Phillips 2, Bit Hex 4 mm, Bit Torx 8, Bit Torx 10, Bit Torx 15 ), Kombizange( H lsenpresser, Drahtschneider ), Stech-Bohrahle, Kugelschreiber( auch zum DIP-Switch verstellen ), Mehrzweckhaken (Pakettr ger), Metalls ge( Metallfeile, Nagelfeile, Nagelreiniger ), Dosen ffner( kleiner Schraubendreher ), Kleine Klinge, Druckkugelschreiber, Grosse Klinge, Ring, inox, Mini-Schraubendreher, Kapselheber( Schraubendreher, Drahtabisolierer ), Holzmeissel / Schaber, Bit-Halter, Stecknadel, inox, Schere, Korkenzieher, Zahnstocher

But some still like to to use color instead of the contrast-rich inverse and the easily to oversee underlining. This is where colordiff comes into play: colordiff colordiff is like syntax highlighting for diffs on the command line. I works with classic and unified diffs as well as with wdiffs and debdiffs (the debdiff command is part of the devscripts package).

$ wdiff cybertool34.txt cybertool41.txt colordiff
+Schraubendreher 2.5mm,+ Pinzette, N hahle mit Nadel hr, +Holzs ge,+ Bit-Schl ssel( 5 mm Innensechskant f r die D-SUB Steckverbinder, 4 mm Innensechskant f r Bits, Bit Phillips 0, Bit Phillips 1, Bit-Schlitzschrauben 4 mm, Bit Phillips 2, Bit Hex 4 mm, Bit Torx 8, Bit Torx 10, Bit Torx 15 ), Kombizange( H lsenpresser, Drahtschneider ), Stech-Bohrahle, Kugelschreiber( auch zum DIP-Switch verstellen ), Mehrzweckhaken (Pakettr ger), +Metalls ge( Metallfeile, Nagelfeile, Nagelreiniger ),+ Dosen ffner( kleiner Schraubendreher ), Kleine Klinge, Grosse Klinge, Ring, inox, Mini-Schraubendreher, Kapselheber( Schraubendreher, Drahtabisolierer ), +Holzmeissel / Schaber,+ Bit-Halter, Stecknadel, inox, Schere, Korkenzieher, Zahnstocher $ wdiff cybertool29.txt cybertool41.txt colordiff
+Schraubendreher 2.5mm,+ Pinzette, N hahle mit Nadel hr, +Holzs ge,+ Bit-Schl ssel( 5 mm Innensechskant f r die D-SUB Steckverbinder, 4 mm Innensechskant f r Bits, Bit Phillips 0, Bit Phillips 1, Bit-Schlitzschrauben 4 mm, Bit Phillips 2, Bit Hex 4 mm, Bit Torx 8, Bit Torx 10, Bit Torx 15 ), +Kombizange( H lsenpresser, Drahtschneider ),+ Stech-Bohrahle, +Kugelschreiber( auch zum DIP-Switch verstellen ), Mehrzweckhaken (Pakettr ger), Metalls ge( Metallfeile, Nagelfeile, Nagelreiniger ),+ Dosen ffner( kleiner Schraubendreher ), Kleine Klinge, [-Druckkugelschreiber,-] Grosse Klinge, Ring, inox, Mini-Schraubendreher, Kapselheber( Schraubendreher, Drahtabisolierer ), +Holzmeissel / Schaber,+ Bit-Halter, Stecknadel, inox, +Schere,+ Korkenzieher, Zahnstocher

(Coloured Screenshots done with ANSI HTML Adapter from the package aha.) Some, especially those who are used to git, are probably confused by the default choice of diff colors. This is easily fixable by writing the following into you ~/.colordiffrc:

newtext=green
oldtext=red
diffstuff=darkblue
cvsstuff=darkyellow

(See also /etc/colordiff for the defaults and hints.) colordiff has by the way two operating modes:

• Without parameter it reads diffs from standard input as seen above.
• With parameters it works as drop-in diff replacement including all diff options as shown below.

So now let us compare the Cybertool 29 with Cybertool 34 in a normal diff (by using the texts from above and replacing all commata with newline characters) with git-like colors:

$ colordiff cybertool29-lines.txt cybertool34-lines.txt
12a13,14
> Kombizange( H lsenpresser
> Drahtschneider )
13a16,17
> Kugelschreiber( auch zum DIP-Switch verstellen )
> Mehrzweckhaken (Pakettr ger)
16d19
< Druckkugelschreiber
25a29
> Schere

Or as unifed diff with some context:

$ colordiff -u cybertool29-lines.txt cybertool34-lines.txt
--- cybertool29-lines.txt     2011-08-31 20:55:37.195546238 +0200
+++ cybertool34-lines.txt   2011-08-31 20:55:11.667710504 +0200
@@ -10,10 +10,13 @@
 Bit Torx 8
 Bit Torx 10
 Bit Torx 15 )
+Kombizange( H lsenpresser
+Drahtschneider )
 Stech-Bohrahle
+Kugelschreiber( auch zum DIP-Switch verstellen )
+Mehrzweckhaken (Pakettr ger)
 Dosen ffner( kleiner Schraubendreher )
 Kleine Klinge
-Druckkugelschreiber
 Grosse Klinge
 Ring
 inox
@@ -23,5 +26,6 @@
 Bit-Halter
 Stecknadel
 inox
+Schere
 Korkenzieher
 Zahnstocher

So if you want nicely colored diffs with Subversion like you re used to with git, you can use svn diff colordiff.

I saw this link on Hacker News:
http://techcrunch.com/2011/08/30/speaking-of-dropbox-heres-my-no-1-phone-prank/
And couldn't help but thinking it was singularly unfunny. He changed the guy's password - ha ha! I'm sure most people can do better. Here's one off the top of my head: I was due to leave a company in a few days, prior to our move to Austria. One of my colleagues in the programming department was tasked, in that period, with doing a bunch of work with Drupal, which he found singularly unpleasant and a pain in the neck to work with, which was compounded by the boss asking often about the status of the project. My colleague tended to be an early riser and would get into the office to get some work done early, before it got noisy and people started hassling him about this, that or the other thing. So on my soon-to-be-abandoned workstation, I set up a cron job that would, during those hours, randomly play a sound file I recorded while he was out, with my impression of a ghostly voice saying "druuuuuuuppaaaaaaallll". He said it scared the pants off him the first couple of times it went off.

AltOS 1.0 TeleMini support and a host of new features Bdale and I are pleased to announce the release of AltOS version 1.0. AltOS is the core of the software for all of the Altus Metrum products. It consists of cc1111-based microcontroller firmware and Java-based ground station software. AltOS Firmware TeleMini support, Kalman Filtering and more Support for the new TeleMini altimeter is included in version 1.0 along with a wealth of other new features:

• Change telemetry to be encoded in multiple 32-byte packets. This enables support for TeleMini and other devices without requiring further updates to the TeleDongle firmware. Previous versions of the firmware used a large monolithic telemetry packet 95 bytes long. The old single-packet format included all of the rapidly updating data coming from the on-board sensors, along with slowly changing GPS data and never-changing configuration data. Not only were the packets large (reducing the reliability of reception), they were also device-specific, encoding precisely the set of information available in TeleMetrum. The new telemetry system uses mulitple different formatted telemetry packets, all 32 bytes in length, transmitting rapidly changing data often and other data more slowly. This should improve reception, but more importantly, allows for different devices to send different sets of data. Within the TeleDongle, this change was implemented by removing all of the telemetry decoding logic from the embedded device and moving it to the AltosUI java code running on the host. This means that the TeleDongle can now receive arbitrary telemetry packets without having new firmware installed.
• Support operation of TeleMetrum with the antenna pointing aft. Previous firmware versions required the antenna to be pointing upwards, now there is a configuration option allowing the antenna to point aft, to aid installation in some airframes. A TeleMetrum user, DK Duncan, changed the firmware on his board to flip the accelerometer ADC values around. This produced something that worked on the ground, so he went and flew it. And it just worked. I added this as a configuration option, including all of the Java code to change it. Nice that any Altus Metrum user can try things like this out and then have them get added in the next version of the software.
• Ability to disable telemetry. For airframes where an antenna just isn t possible, or where radio transmissions might cause trouble with other electronics, there s a configuration option to disable all telemetry. Note that the board will still enable the radio link in idle mode. Terry Lee is taking a TeleMetrum board for a ride during LDRS this year, but with TeleMetrum mixed in with a pile of other electronics, he didn t want us transmitting during flight and causing potential RFI issues with other boards in the air frame. Instead of building a custom version of the firmware, we just made this a configurable mode.
• Arbitrary frequency selection. The radios in Altus Metrum devices can be programmed to a wide range of frequencies, so instead of limiting devices to 10 pre-selected channels , the new firmware allows the user to choose any frequency in the 70cm band. Note that the RF matching circuit on the boards is tuned for around 435MHz, so frequencies far from that may reduce the available range.
• Kalman-filter based flight-tracking. The model-based sensor fusion approach of a Kalman filter means that AltOS now computes apogee much more accurately than before, generally within a fraction of a second. In addition, this approach allows the baro-only TeleMini device to correctly identify Mach transitions, avoiding the error-prone selection of a Mach delay. Developing this feature made extensive use of the simulator which runs the flight management code using sensor data captured on previous flights. Somehow, we ve managed to collect log data from over 100 flights; replaying them on the ground means that even before the first flight with the new firmware, we were confident that it would work.

AltosUI New Features AltosUI has also seen quite a bit of work for the 1.0.1 release. Of course, many of the changes in AltosUI are to accomodate the new TeleMini altimeter and changes in the AltOS firmware for TeleMetrum. In addition, we ve also added lots of new features in response to user requests.

• Add main/apogee voltage graphs to the data plot. This provides a visual indication if the igniters failed before being fired.
• Scan for altimeter devices by watching the defined telemetry frequencies. This avoids the problem of remembering what frequency a device was configured to use, which is especially important with TeleMini which does not include a USB connection.
• Monitor altimeter state in Idle mode. This provides much of the information presented in the Pad dialog from the Monitor Flight command, monitoring the igniters, battery and GPS status withing requiring the flight computer to be armed and ready for flight.
• Pre-load map images from home. For those launch sites which don t provide free Wi-Fi, this allows you to download the necessary satellite images given the location of the launch site. A list of known launch sites is maintained at altusmetrum.org which AltosUI downloads to populate a menu; if you ve got a launch site not on that list, please send the name of it, latitude and longitude along with a link to the web site of the controlling club to the altusmetrum mailing list.
• Flight statistics are now displayed in the Graph data window. These include max height/speed/accel, average descent rates and a few other bits of information. The Graph Data window can now be reached from the Landed tab in the Monitor Flight window so you can immediately see the results of a flight.

FreeSmartphone.Org (FSO), Openmoko, Debian's FSO group, SHR, QtMoko et cetera are a few of the community based intertwined projects to bring free software to smartphones. They have a relatively long and colorful history of doing this, and have nowadays been approaching multiple target devices despite limited resources and for example the losing of Openmoko Inc. in 2009. I've been using Debian on my Neo FreeRunner phone for over two years now, and over three years of FreeRunner use altogether. FSO2, the next generation freesmartphone.org stack, is finally coming into Debian now, extending the basic phone support besides Openmoko phones to eg. Palm Pre, Nexus One, Nokia N900 and a few HTC phones. It needs a lot of tweaking and eg. a proper kernel, but still.

Four years after beginning of sales of the first Openmoko device (Neo1973), we're still in the pioneering phase of free distributions for mobile phones. There is no "Ubuntu for phones" so to speak, not for even a selected models. Meanwhile, like so often in the wide world, competing free software approaches have arrived. Android is the obvious one, and has seen a port to Neo Freerunner among else. Android is not as open a project as one could like, and replaces everything we've known with its own code, but nevertheless it requires to be noted and is completely usable in its free software form. But since there are limitations to its approach, and since it's more of an own separate world from the rest of Linux distributions, it is not as interesting to me as the others in the long run, at least in its current shape.

A more similar competitor to FSO and distributions using FSO is MeeGo and its middleware, and to be more precise so far specifically Nokia's efforts on it. Obviously there is the strong competitor for the best general population smartphone of the year, the Qt based Nokia N9, but its default software is more of a proprietary thing even though it has a neatly rock solid free software foundation with separate free/non-free repositories and all that. Nice and great for the (GNU/)Linux in general, but the Harmattan software is not exactly on topic for this blog post. It's however in my opinion the best marketing to vendors around the world GNU/Linux + Qt can get as Android Linux has been taking most of the limelight. But meanwhile, with significantly smaller focus and resources, Nokia has also been sponsoring to try to create a truly community based mobile phone software stack at the MeeGo upstream, nowadays called "MeeGo Community Edition" or MeeGo CE for short. It co-operates with the MeeGo Handset target of MeeGo (and Intel) that hasn't got actual target consumer hardware at the moment, although that might change soon (?), but has been doing some nice applications recently. CE has the target hardware and additional device specific and non-specific software cooking. It used to target Nokia N900 only, but nowadays the project has added N950 (the for-developers-only phone) and N9 to the targets, and it is starting to seem there should be no showstoppers to bring MeeGo CE (or other distributions later on) to them, despite some earlier doubts. A few needed bits to void the warranty we all want to do are still missing, though, but coming. After that it's just developing free software. Usual caveats about specifics of a few kernel drivers apply, as the devices were not designed with the sole purpose of free software drivers in mind. Hopefully mobile 3D gets into a better shape in the coming years, but that's another story.

MeeGo (CE) smartphone middleware, of course, shares nothing with FSO *). While FSO is a "handle everything" in itself with its wide variety of daemons, MeeGo consists of more separate software like Intel's and Nokia's oFono for modem support. FSO demo UIs and SHR UIs have traditionally focused on using Enlightenment 17 for its suitability to low power machines, while in MeeGo everything is written in Qt. As Qt/QML is becoming faster and faster, and it's very powerful to write, there might be quite a bit of useful software emerging from there also for other distributions besides MeeGo itself. Actually, there is already a Debian MeeGo stack maintainer group available, although it hasn't yet focused on the UIs as far as I can see (if I'll have free time I'll join the effort and see for myself in more detail). There is also the QtMoko distribution, based on the original, canceled Trolltech/Nokia Qt Extended (Qtopia) project, but ported to newer Qt:s and put on top of Debian.

*) Although, correct me if I'm wrong and I might be, the Nokia N900 isi modem driver in FSO was ported/learned from oFono.

MeeGo CE is not only a project to bring a proper MeeGo distribution to a few smartphones, but also to shake out bugs in the MeeGo project's contributions and community processes. It is acting as a completely open MeeGo product contributor, and investigating how things should be optimally done so that everything gets integrated into the MeeGo properly, and that proper MeeGo software releases can be done for the target devices. Therefore it's also an important project for the vitality of the whole MeeGo.

MeeGo CE has so far had a whole team in Nokia working on it, but for obvious strategy related reasons community cannot rely on it lasting forever. The real hurdle is for the wider free smartphones community to be ready for really embracing a project that is already a community project but to outsiders might seem like a company project. I believe it's never easy to grow a volunteer community if the starting setup is a paid company team, but it mainly requires a) the interested people and b) the few smart ones to take control and communication responsibility so that it's not anymore seen as a company project. MeeGo CE never was a traditional company project, everything being done in the open and together with volunteers, but I just know how people perceive these things by default. People tend to assume stuff as someone else's responsibility

Whatever happens, I hope there are enough people interested in free software for mobile phones to carry on all these different approaches to the software needed. I hope MeeGo / MeeGo CE will have a great future, and I hope that both the middleware like FSO and MeeGo's components, and the UIs like FSO, SHR, QtMoko and MeeGo Handheld UIs continue to develop. I also hope other distributions like Debian will gather a strong suite of packaged software for smartphones. I know I have had some hard time to find suitable apps for my phone at times.

For those interested about how I use Debian as my mobile phone OS, see http://wiki.openmoko.org/wiki/User:TimoJyrinki

There are many SSH multiplexers in Debian and most of them have one or two features which make them unique and especially useful for that one use case. I use some of them regularily (I even maintain the Debian package of one of them, namely pconsole :-) and I ll present then and when one of them here. For non-interactive purposes I really like parallel-ssh aka pssh. It takes a file of hostnames and a bunch of common ssh parameters as parameters, executes the given command in parallel in up to 32 threads (by default, adjustable with -p) and waits by default for 60 seconds (adjustable with -t). For example to restart hobbit-client on all hosts in kiva.txt, the following command is suitable:

$ parallel-ssh -h kiva.txt -l root /etc/init.d/hobbit-client restart
[1] 19:56:03 [FAILURE] kiva6 Exited with error code 127
[2] 19:56:04 [SUCCESS] kiva
[3] 19:56:04 [SUCCESS] kiva4
[4] 19:56:04 [SUCCESS] kiva2
[5] 19:56:04 [SUCCESS] kiva5
[6] 19:56:04 [SUCCESS] kiva3
[7] 19:57:03 [FAILURE] kiva1 Timed out, Killed by signal 9

(Coloured Screenshots done with ANSI HTML Adapter from the package aha.) You easily see on which hosts the command failed and partially also why: On kiva6 hobbit-client is not installed and therefore the init.d script is not present. kiva1 is currently offline so the ssh connection timed out. If you want to see the output of the commands, you have a two choices. Which one to choose depends on the expected amount of output: If you don t expect a lot of output, the -i (or --inline) option for inline aggregated output is probably the right choice:

$ parallel-ssh -h kiva.txt -l root -t 10 -i uptime
[1] 20:30:20 [SUCCESS] kiva
 20:30:20 up 7 days,  5:51,  0 users,  load average: 0.12, 0.08, 0.06
[2] 20:30:20 [SUCCESS] kiva2
 20:30:20 up 7 days,  5:50,  0 users,  load average: 0.19, 0.08, 0.02
[3] 20:30:20 [SUCCESS] kiva3
 20:30:20 up 7 days,  5:49,  0 users,  load average: 0.10, 0.06, 0.06
[4] 20:30:20 [SUCCESS] kiva4
 20:30:20 up 7 days,  5:49,  0 users,  load average: 0.25, 0.17, 0.14
[5] 20:30:20 [SUCCESS] kiva6
 20:30:20 up 7 days,  5:49, 10 users,  load average: 0.16, 0.08, 0.02
[6] 20:30:21 [SUCCESS] kiva5
 20:30:21 up 7 days,  5:49,  0 users,  load average: 3.11, 3.36, 3.06
[7] 20:30:29 [FAILURE] kiva1 Timed out, Killed by signal 9

If you expect a lot of output you can give directories with the -o (or --outdir) and -e (or --errdir) option:

$ parallel-ssh -h kiva.txt -l root -t 20 -o kiva-output lsb_release -a
[1] 20:36:51 [SUCCESS] kiva
[2] 20:36:51 [SUCCESS] kiva2
[3] 20:36:51 [SUCCESS] kiva3
[4] 20:36:51 [SUCCESS] kiva4
[5] 20:36:53 [SUCCESS] kiva6
[6] 20:36:54 [SUCCESS] kiva5
[7] 20:37:10 [FAILURE] kiva1 Timed out, Killed by signal 9
$ ls -l kiva-output
total 24
-rw-r--r-- 1 abe abe  98 Aug 28 20:36 kiva
-rw-r--r-- 1 abe abe   0 Aug 28 20:36 kiva1
-rw-r--r-- 1 abe abe  98 Aug 28 20:36 kiva2
-rw-r--r-- 1 abe abe  98 Aug 28 20:36 kiva3
-rw-r--r-- 1 abe abe  98 Aug 28 20:36 kiva4
-rw-r--r-- 1 abe abe 102 Aug 28 20:36 kiva5
-rw-r--r-- 1 abe abe 100 Aug 28 20:36 kiva6
$ cat kiva-output/kiva5
Distributor ID:	Debian
Description:	Debian GNU/Linux 6.0.2 (squeeze)
Release:	6.0.2
Codename:	squeeze

The only annoying thing IMHO is that the host list needs to be in a file. With zsh, bash and the original ksh (but neither tcsh, pdksh nor mksh), you can circumvent this restriction with one of the following command lines:

$ parallel-ssh -h <(printf "host1\nhost2\nhost3\n ") -l root uptime
[ ]
$ parallel-ssh -h <(echo host1 host2 host3     xargs -n1) -l root uptime
[ ]

In addition to parallel-ssh the pssh package also contains some more ssh based tools:

• parallel-scp and parallel-rsync for parallel copying files onto a set of hosts.
• parallel-slurp for fetching files in parallel from a list of hosts.
• parallel-nuke to kill a bunch of processes in parallel on a set of machines.

I though think that parallel-ssh is by far the most useful tool from the pssh package. (Probably no wonder as it s the most generic one. :-)

I have a pathological use of browser tabs: I use a lot of them. A lot is probably an understatement. We could say I use them as bookmarks of things I need to track. A couple weeks ago, I was saying I had around two hundred tabs opened. I now actually have much more. It affected startup until I discovered that setting the browser.sessionstore.max_concurrent_tabs pref to 0 was making things much better by only loading tabs when they are selected. This preference has/will become browser.sessionstore.restore_on_demand. However, since I only start my main browser once a day, while other applications start and while I begin to read email, I hadn t noticed that this was still heavily affecting startup time: about:startup tells me reaching the sessionRestored state takes seven seconds, even on a warm startup. It also affects memory usage, because even when tabs are only loaded on demand, there is a quite big overhead for each tab. And more importantly, it gets worse with time. And I think the user interface is actively making it worse. So, to get an idea how bad things were in my session, I wrote a little restartless extension. After installing it, you can go to the about:tabs url to see the damage on your session. Please note that the number of groups is currently wrong until you open the tab grouping interface. This is what the extension has to say about my session 2 days ago, right after a startup:

• 556 tabs across 4 groups in 1 window
• 1 tab has been loaded
• 444 unique addresses
• 105 unique hosts
• 9 empty tabs
• 210 http:
• 319 https:
• 14 ftp:
• 2 about:
• 2 file:
• 55 addresses in more than 1 tab
• 39 hosts in more than 1 tab

The first thing to note is that when I filed the memory bug 4 days earlier, I had a bit more than 470 tabs in that session. You can see 4 days later, I now have 555 tabs (if excluding the about:tabs tab). The second thing to note is something I suspected because it s so easy to get there: a lot of the tabs are opened on the same address. Since Firefox 4.0, if I m not mistaken, there is a great feature in the awesomebar, that allows to jump to an existing tab matching what you type in the urlbar. That is very useful, and I use it a lot. However, there are a lot of cases where it s not as useful as it could be. One of the addresses I visit a lot is http://buildd.debian.org/iceweasel. It gives me the build status of the latest iceweasel package I uploaded to Debian unstable. That url is particularly known in my browsing history, and is the first hit when I type buildd in the urlbar (actually, even typing b brings it first). Unfortunately, that url redirects to https://buildd.debian.org/status/package.php?p=iceweasel through an HTTP redirection. I say unfortunately because when I type buildd in the urlbar, I get 6 suggestions for urls in the form http://buildd.debian.org/package (I also watch other packages build status), and the suggestion to switch to the existing tab for what the first hit would get me to is 7th. Guess what? The suggestion list only shows 6 items ; you have to scroll to see the 7th. The result is that I effectively have fifteen tabs open on that url. I also keep a lot of bugzilla.mozilla.org bugs open in different tabs. The extension tells me there are 255 of them for 166 unique bugs. Largely, the duplicate bug tabs are due to having these bugs open in some tab, but accessing the same bugs from somewhere else, usually a dependent bug or TBPL. I also have 5 tabs opened on my request queue. I usually get there by going to the bugzilla home page and clicking on the My Requests link. And I have several tabs opened on the same bug lists. For the same reason. When I started using tab groups, I splitted in very distinct groups. Basically, one for Mozilla, one for Debian, one for stuff I want to follow (usually blog posts I want to follow comments from), and one for the rest. While I was keeping up with grouping at the beginning, I don t anymore, and the result is that each group is now a real mess. Firefox has hundreds of millions users. It s impossible to create a user experience that works for everyone. One thing is sure, it doesn t work for me. My usage is probably very wrong at different levels, but I don t feel my browser is encouraging me to use it better, except by making my number of opened tabs explode to an unmanageable level (I already have 30 tabs more than when I started writing this post 2 days ago). There are a few other things I would like to know about my usage that my extension hasn t told me yet, either because it doesn t tell, or because I haven t looked:

• How many tabs end up loaded at the end of a typical day?
• How many tabs do I close?
• How many duplicate tabs do I open and close?
• How long has it been since I looked at a given tab?
• How do the number of tabs and duplicates evolve with time?

Reflecting on my usage patterns, I think a few improvements, either in the stock browser, or through extensions, could make my browsing easier:

• Auto-grouping tabs: When I click on a link to an url under mozilla.org, I most likely want it in the Mozilla group. An url under debian.org would most likely go in the Debian group.
• Switch to an existing tab when following a link to an already opened url: That might not be very useful as a general rule, but at least for some domains, it would seem useful for me that the browser switches to an existing tab not only through the urlbar, but also when following links in a page. If I m reading a bug, click on a bug it depends on, and that bug is already opened in another tab, get me there. There would be a history problem to solve, though. (e.g. where do back and forward bring?)

Maybe these exist as extensions, I don t know. It s hard to find very specific things like that through an add-on search (though I haven t searched very hard). [Looks like there is an experiment for the auto tab grouping part] I think it would also be interesting to have something like Test Pilot, but for users that want to know the answer to How do I use my browser? . As I understand it, Test Pilot can show individual user data, but it only can do so if there is such data, and you can t get data for past studies you didn t take. In my case, I m not entirely sure that, apart from the pinned tabs, I use the tab bar a lot. And even for pinned tabs, most of the time I use keyboard shortcuts. I m not using the menu button that much either. I already removed the url and search bar (most of the time) with LessChrome HD. Maybe I could go further and use the full window for web browsing.

I stopped my story in early 1994, after my Big Switch to the mysterious and new world of Linux. My home server, kheops, is now running Debian and is fed with news and mail by FrMug, formerly French Minix Users Group and now run by AFAU, the "Association Fran aise des Amateurs de Usenet". As I'm still console-challenged, I'm reading mail and news on another machine I have at home, named "khephren", which is running an, ahem, beta copy of Microsoft Windows 95. Yeah, I know...:-) These 1994-1999 years were mostly the Usenet years, for me. I gradually became more and more involved in several fr.* newsgroups, either related to the Linux world...or to the Windows world. The latter was professionnally a bonus for me as 1994 is also the year where I switched from the Material Science department at Onera to the Network and Computing Department. My mission there was to accompany the increasing prevalence of personal desktop workstations in the daily life of what was formerly the Big Castle of Computing at Onera, hosting various generations of Cray computers, most of which being in the Top500 world supercomputers (we even had one in the Top10, very briefly, at somepoint). So, I was "the PC and Windows guy" in this world of supercomputing, Unix and network wizards. But my hobby at home helped me keeping and growing skills in both "worlds" which, over the years, has proven to be a not so common profile..:-) Meanwhile, at home, I was spending hours and nights reading those newsgroups...or getting involved in fr.* hierarchy management..or expanding my friends network to many people in the news, Linux, BSD worlds. That lasted for about 3 years with ups and downs (with three young kids at home, you end up having slightly less time...). I was gradually improving my home server use, adding more functionalities to it, for instance setting it up as a file and print server with Samba 1.9.something (this will be the beginning of a long story with samba!). Unfortunately, my dear Ren had passed in the meantime. Ren did so much for Linux in France that he'll be the only link I give in this article. If you happen to go through P re Lachaise cemetery in Paris, go to its Memory garden instead of visiting Jim Morrison's grave and drink a whisky bottle in memory of Ren . Or just install one of the most funnily named Debian packages: le-dico-de-rene-cougnenc and, again, drink something to his memory (whisky is highly preferred). We then reached 1997 and a specific event: my father (who was very sick for years) passed during summer, some time after my grand-father passed too. As it apparently happens for many people, it triggerred me into the interest for family history. After all, those people who are going away are family memories. And, in some way, I wanted to keep track of this memory. As a consequence, I started collecting family memories, interviewing my close relatives (the oldest ones) and ending, of course, in tracking all this with my computers. That was the start of a long genealogy research that sent me on the track of the Perrier family (of course), but also all other branches, on my mother and father side, as well as reconstructing the tree of cousins we have, etc, etc. As anybody who did that knows, this is quite an endless task but I went very committed to it. I quickly ended up in joining research groups that were using the recent growth of personal access to Internet communication. Those groups were, in these 90's, building elaborated systems where people can help together in their research instead of having to travel to places where archives are stored. I participated to many such groups and even ended up animating one who was doing research in archives of the "Yvelines" "d partement", my place of living (even though I have no family branch there). At the same time, other groups were sending me research results for my own family in the French region where my roots are. Collective effort, help others for free, you can recognize things that are a constant all over these years, indeed. It actually helped me in tracking my Perrier ancestors up to about 1680, with 11 generations of men holding that name (from Jean-Baptiste, my son, up to tienne Perrier, who is born about 1640-1650, in the small village named Briant in Sa ne et Loire, in France). Or even climbing up to Laurent Polette, born circa 1510, mentioned in a document date February 1562. He his my oldest known ancestor.Those folks being 15 generations before me. My oldest ancestor is born under Fran ois the First, King of France, protector of Leonardo Da Vinci, at the time La Joconde was painted or the Chambord castle was built. Doh. Unfortunately, genealogy software, at that time, was quite poor for Unix systems. Lifelines was a quite strong development but wasn't that user-friendly and I ended up using a Windows-based "shareware" software written by an individual. Still, my Linux machine was used quite intensively: databases were stored there, and saved there, all important informations were there, my mail was received there, along with dozens of mailing lists I was reading. I even hosted a few mailing lists on my home server. Still, in the meantime also, again in 1997, in October, another event happened. We had a pollution peak in Paris area and October 1st was the first (and as of now the only ever) day where driving restrictions were applied. Up to that moment, I was driving to/from work (30 kilometers one way). I decided to use this opportunity to really test public transport, for one full month. As I knew it would take me much more time than driving, I tried to imagine a way to use that time. How about reading mail and news on a laptop instead of only at home? Done... In a few days, I found a used Compaq Aero laptop (486 16MHz), I installed Debian "bo" on it, switched my mail and news workflow to console programs (mutt and slrn) and here we go! So, finally, in October 1997, I was using Linux nearly all time long, the only exception being genealogy data handling. That laptop was named "mykerinos" (the small pyramid) and....my laptop is still named mykerinos, though it's now the 5th mykerinos I have..:-) Ah, and since then, I nearly always use public transport to go to/from work. Debian addiction is good for the planet! At some point (I don't remember exactly when...about 1999, I think), DSL connections appeared and one of the first DSL FAI emerged on France: Free. I was already using their services for dialup access, but DSL was then a revolution: persistent connection! And nearly immediately, static IP! I became one of their first DSL customers (and I'm still one). Moreover, I had then friends working there...and they had many Debian machines to host their services. So, with the help of my Freenix friends, some DNS hosting on their machines and the still tireless eu.org entirely free domain, perrier.eu.org was born. Immediately, this gave me opportunities for doing many interesting things with my home server: setting up www.perrier.eu.org and use it....for learning web things and sharing genealogical data. For instance, I ended up hosting some data sent by other genealogists such as digital copies of archives for some French regions, all served through my 512/128kbps DSL line (quickly upgraded to 2Mb, IIRC). And, very quickly, I felt the need to share my family tree, of course! And here comes Geneweb. Written (in Ocaml) by Daniel de Rauglaudre, Geneweb was (and still is) by far one of the best tools to share genealogy data over the Net. So, I ended up installing it on my home machine. Guess how I did that? "apt-get install geneweb", of course. Yes, it was packaged. Indeed by Brent Fulgham who did the initial release on July 22nd 2000. Here, my memory is a little bit vague. I think I actually packaged Geneweb for my own use and sharing it on upstream FTP site, forking Brent's package as a start, but updating to latest upstream release. Then I quickly ended up having discussions with Brent about this new upstream release, and that lead to:

geneweb (3.10-3) unstable; urgency=low
  * Merging with unofficial package from Geneweb ftp site
    (in preparation of new maintainer [Christian Perrier] takeing
    over responsibility).  Note that all of this work is from
    Christian.
  * Default database dir is now writable by members of geneweb group
  * New geneweb group added at installation
  * Package tries to properly handle former installations of the
    unofficial Geneweb package available on Geneweb ftp site
  * Debconf dialogs translated in french
  * More languages available
  * Creation of /var/geneweb/etc, /var/geneweb/cnt, /var/geneweb/images
    with proper permission so that authorised users may use these also
  * When de-installed, all databases are exported with gwu (should allow
    smooth upgrades when the databse format changes)
 -- Brent A. Fulgham <bfulgham>  Mon, 22 Jan 2001 19:30:47 -0800

See? Already "debconf translation to French"! "More languages available". Here we are. As discussed with Brent, he propsoed me to take the package over. So, on January 11th 2001, I had applied in the New Maintainer Process. I'm amazed how quickly it went indeed: there was a tool I needed, it needed to be packaged, I could handle the packaging...and then I decided to be involved. Here we are: bubulle is on his way to become one of those magicians who are building the best universal operating system. Yes, me, the clueless material scientist..:-) How did it go? You'll know this in next episode, of course...

Finally I managed to write up a transition statement for my not so new, but stronger GnuPG key. See below:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1,SHA256
I am transitioning my GPG key from an old 1024-bit key to a new 4096-bit key.
The old key will continue to be valid for some time, but I prefer all new
correspondence to be encrypted for the new key, and will be making all
signatures going forward with the new key.
If you have signed my old key, I would appreciate signatures on my new key as
well, provided that your signing policy permits that without reauthenticating
me.
The old key, which I am transitioning away from, is:
pub   1024D/99E141B4 2004-02-10
      Key fingerprint = 25FE 4741 4770 0558 949D  1DB1 58DD 3FE2 99E1 41B4
The new key, to which I am transitioning, is:
pub   4096R/51B85139 2009-06-18
      Key fingerprint = A3EB B41F C5AB D675 CEE4  1C45 EA6C A6B9 51B8 5139
Thanks in advance.
Cheers,
Micha Lenk
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
iEYEARECAAYFAk5aOnkACgkQWN0/4pnhQbTxPgCgzRhREZlQiKJyI9UIdJLLs3Zq
bH4AnA1myFxgDWM7aUMHXgvvsujLTjiWiQIcBAEBCAAGBQJOWjp5AAoJEOpsprlR
uFE5gCsP/0dtCUPl9aQHV1MbQl7+bMofpsC2ikkpdZmrzi68jTG16We49BuzY+PV
S8FhXqg17/YxhKYDnDNTowfztUOyjAOJxy5vrqm3X5xiLwTqN3js9mra+vb4s35k
EVbKMzLLDhj3i0FjeargWEmJmm9cVhaZWKvOvQUhDJAilqvEQ0/50P7B8I+1YvtV
UHoQKbweTljVSlK5R1YPPy9i6r2/oZBYxK4nrknWwS+qPQ5luqelJd+mZdgQ6tow
7HIvtmPgCblJ+hYZWFpoZK6vxs8RaBbuCQcKwYArNhZT/v4TeD/LAaUmIkbQMyKV
J2TKuEHya4+5GMbtg6BGKeiZpleEHPnAq1AfvGpz6opkxjxCLG3RO3X8D3EuM3RW
mkq60mWM8+Zwu1yKbb62iHplp6jpyiQgdjJlB6eHjX7SdY7CvHgYZxGDx4kclP0/
HAdig2U1T+nG6Nn5XflmmKwvNLuKlQlIwJ5NeXyCONRnYvdomQn2hgvkjwMLCdFh
ulIhxa4UvDY7/aQNPZeOrvDHb2XYpiV3TwA9hLgQXXWd0FPmUMVVPpKpRjilaWth
Mtq/QiMGP5Mq/YxgLInRZHGyajDtE67RD4+RgHYOP50cP3UGPoB4ncc2EEtM2kfe
BzJrXPHmdtyGEA6Korl0YwUTRnYsqkqkY1VqDsO0UkOLlV7RAzb9
=9vw1
-----END PGP SIGNATURE-----

Yet another useful tool of which at least I heard quite late in my Unix career is watch . For a long time I wrote one-liners like this to monitor the output of a command:

while :; do echo -n " date  "; host bla nameserver; sleep 2; done

But it s way shorter and less error-prone to use watch from Debian s procps package and just write

watch host bla nameserver

The only relevant difference is that I don t have some kind of history when the output of the command changed, e.g. to calculate the rate with which a file grows. You can even track the output of more than one command:

watch 'ps aux   grep resize2fs; df -hl'

Also a nice way to use watch is to run it inside GNU Screen (or tmux or splitvt) and split up the terminal horizontally, i.e. show the output of watch in one window and the process you re tracking with the commands run by watch in the other window and see both running at the same time. Update, Sunday, 28th of August 2011, 17:13h I never found a useful case for watch s -d option which highlights changes to the previous run (by inverting the changed bytes), but until now three people pointed out the -d option in response to this blog-posting and weasel also had some nice examples, so here are they: Keep an eye on the current network routes (once per second) of a host and quickly notice when they change:

watch -n1 -d ip r

Watch the current directory for size or time stamp changes of its files:

watch -d ls -l

The option -d only highlights changes from the previous run to the next run. If you want to see all bytes which ever changed since the first run, use --differences=cumulative. Thanks to Klaus Mowgli Ethgen, Ulrich mru Dangel, Uli youam Martens and Peter weasel Palfrader for comments and suggestions.

This is a direct followup on my previous blog posting about calculating IPs and netmasks with the tools netmask and prips. Kurt Roeckx (via e-mail) and Niall Donegan (via a comment to that blog posting) both told me about the package sipcalc, and Kurt also mentioned the package ipcalc. Thanks for that! And since I found both useful, too, let s put them in their own blog posting: Both tools, ipcalc and sipcalc offer a get all information at once mode which are not present in the previously presented tool netmask. ipcalc ipcalc by default outputs all information and even in ANSI colors:

$ ipcalc 192.168.96.0/21
Address:   192.168.96.0         11000000.10101000.01100 000.00000000
Netmask:   255.255.248.0 = 21   11111111.11111111.11111 000.00000000
Wildcard:  0.0.7.255            00000000.00000000.00000 111.11111111
=>
Network:   192.168.96.0/21      11000000.10101000.01100 000.00000000
HostMin:   192.168.96.1         11000000.10101000.01100 000.00000001
HostMax:   192.168.103.254      11000000.10101000.01100 111.11111110
Broadcast: 192.168.103.255      11000000.10101000.01100 111.11111111
Hosts/Net: 2046                  Class C, Private Internet

(Coloured Screenshots done with ANSI HTML Adapter from the package aha.) You can suppress the bitwise option or directly output HTML via commandline options. For example ipcalc -b -h 192.168.96.0/21 outputs the following content:

Address:	192.168.96.0
Netmask:	255.255.248.0 = 21
Wildcard:	0.0.7.255
=>
Network:	192.168.96.0/21
HostMin:	192.168.96.1
HostMax:	192.168.103.254
Broadcast:	192.168.103.255
Hosts/Net:	2046	Class C, Private Internet

Yes, that s an HTML table and no preformatted text, just with a monospaced font. (I just removed the hardcoded text color from it, otherwise it would not look nice on dark backgrounds like in Planet Commandline s default color scheme.) Like netmask, ipcalc can also deaggregate IP ranges into largest possible networks:

$ ipcalc 192.168.87.0 - 192.168.110.255
deaggregate 192.168.87.0 - 192.168.110.255
192.168.87.0/24
192.168.88.0/21
192.168.96.0/21
192.168.104.0/22
192.168.108.0/23
192.168.110.0/24

(ipcalc -r 192.168.87.0 192.168.110.255 is just another way to write this, and it results in the same output.) To find networks with at least 20, 63 and 30 IP addresses within a /24 network, use for example:

Address:   192.0.2.0            
Netmask:   255.255.255.0 = 24   
Wildcard:  0.0.0.255            
=>
Network:   192.0.2.0/24         
HostMin:   192.0.2.1            
HostMax:   192.0.2.254          
Broadcast: 192.0.2.255          
Hosts/Net: 254                   Class C
1. Requested size: 20 hosts
Netmask:   255.255.255.224 = 27 
Network:   192.0.2.128/27       
HostMin:   192.0.2.129          
HostMax:   192.0.2.158          
Broadcast: 192.0.2.159          
Hosts/Net: 30                    Class C
2. Requested size: 63 hosts
Netmask:   255.255.255.128 = 25 
Network:   192.0.2.0/25         
HostMin:   192.0.2.1            
HostMax:   192.0.2.126          
Broadcast: 192.0.2.127          
Hosts/Net: 126                   Class C
3. Requested size: 30 hosts
Netmask:   255.255.255.224 = 27 
Network:   192.0.2.160/27       
HostMin:   192.0.2.161          
HostMax:   192.0.2.190          
Broadcast: 192.0.2.191          
Hosts/Net: 30                    Class C
Needed size:  192 addresses.
Used network: 192.0.2.0/24
Unused:
192.0.2.192/26

sipcalc sipcalc is similar to ipcalc. One big difference seems to be the IPv6 support:

$ sipcalc 2001:DB8::/32
-[ipv6 : 2001:DB8::/32] - 0
[IPV6 INFO]
Expanded Address        - 2001:0db8:0000:0000:0000:0000:0000:0000
Compressed address      - 2001:db8::
Subnet prefix (masked)  - 2001:db8:0:0:0:0:0:0/32
Address ID (masked)     - 0:0:0:0:0:0:0:0/32
Prefix address          - ffff:ffff:0:0:0:0:0:0
Prefix length           - 32
Address type            - Aggregatable Global Unicast Addresses
Network range           - 2001:0db8:0000:0000:0000:0000:0000:0000 -
                          2001:0db8:ffff:ffff:ffff:ffff:ffff:ffff

(Thanks to Niall for the pointer to RFC3849. :-) It can also split up networks into smaller chunks, but only same-size chunks, like e.g. split a /32 IPv6 network into /34 networks:

sipcalc -S34 2001:DB8::/32
-[ipv6 : 2001:DB8::/32] - 0
[Split network]
Network                 - 2001:0db8:0000:0000:0000:0000:0000:0000 -
                          2001:0db8:3fff:ffff:ffff:ffff:ffff:ffff
Network                 - 2001:0db8:4000:0000:0000:0000:0000:0000 -
                          2001:0db8:7fff:ffff:ffff:ffff:ffff:ffff
Network                 - 2001:0db8:8000:0000:0000:0000:0000:0000 -
                          2001:0db8:bfff:ffff:ffff:ffff:ffff:ffff
Network                 - 2001:0db8:c000:0000:0000:0000:0000:0000 -
                          2001:0db8:ffff:ffff:ffff:ffff:ffff:ffff
-

Similar thing with IPv4:

sipcalc -s27 192.0.2.0/24
-[ipv4 : 192.0.2.0/24] - 0
[Split network]
Network                 - 192.0.2.0       - 192.0.2.31
Network                 - 192.0.2.32      - 192.0.2.63
Network                 - 192.0.2.64      - 192.0.2.95
Network                 - 192.0.2.96      - 192.0.2.127
Network                 - 192.0.2.128     - 192.0.2.159
Network                 - 192.0.2.160     - 192.0.2.191
Network                 - 192.0.2.192     - 192.0.2.223
Network                 - 192.0.2.224     - 192.0.2.255

sipcalc also has a show me all information mode with the -a option:

$ sipcalc -a 192.168.96.0/21
-[ipv4 : 192.168.96.0/21] - 0
[Classfull]
Host address            - 192.168.96.0
Host address (decimal)  - 3232260096
Host address (hex)      - C0A86000
Network address         - 192.168.96.0
Network class           - C
Network mask            - 255.255.255.0
Network mask (hex)      - FFFFFF00
Broadcast address       - 192.168.96.255
[CIDR]
Host address            - 192.168.96.0
Host address (decimal)  - 3232260096
Host address (hex)      - C0A86000
Network address         - 192.168.96.0
Network mask            - 255.255.248.0
Network mask (bits)     - 21
Network mask (hex)      - FFFFF800
Broadcast address       - 192.168.103.255
Cisco wildcard          - 0.0.7.255
Addresses in network    - 2048
Network range           - 192.168.96.0 - 192.168.103.255
Usable range            - 192.168.96.1 - 192.168.103.254
[Classfull bitmaps]
Network address         - 11000000.10101000.01100000.00000000
Network mask            - 11111111.11111111.11111111.00000000
[CIDR bitmaps]
Host address            - 11000000.10101000.01100000.00000000
Network address         - 11000000.10101000.01100000.00000000
Network mask            - 11111111.11111111.11111000.00000000
Broadcast address       - 11000000.10101000.01100111.11111111
Cisco wildcard          - 00000000.00000000.00000111.11111111
Network range           - 11000000.10101000.01100000.00000000 -
                          11000000.10101000.01100111.11111111
Usable range            - 11000000.10101000.01100000.00000001 -
                          11000000.10101000.01100111.11111110
[Networks]
Network                 - 192.168.96.0    - 192.168.103.255 (current)

Thanks again to Kurt and Niall for their contributions!

Now listening to the schreimaschine and fausttanz submissions for the interactive competition at the B nzli/DemoDays in Olten (Switzerland)