This week, I've finished my OAuth 2.0 client implementation for GNU Emacs. I have imported it into GNU ELPA so Emacs 24 users will be soon able to install it using the new Emacs packaging system. OAuth 2.0 can be used to access, among others, Google APIs or the Facebook Graph API.
Flattr me!

An obvious question is why Linux doesn't support UEFI secure booting. Let's ignore the issues of key distribution and the GPL and all of those things, and instead just focus on what would be required. There's two components - the signed binary and the authenticated variables.

The UEFI 2.3.1 spec describes the modification to the binary format required to produce a signed binary. It's not especially difficult - you add an extra entry to the image directory, generate a hash of the entire binary other than the checksum, the certificate directory entry and the signatures themselves, encrypt that hash with your key and embed the encrypted hash in the binary. The problem has been that there was a disagreement between Microsoft and Intel over whether this signature was supposed to include the PKCS header or not, and until earlier this week the only widely available developer firmware (Intel's) was incompatible with the only widely available signed OS (Microsoft's). There's further hilarity in that the specification lists six supported hash algorithms, but the implementations will only accept two. So pretty normal, really. Developing towards a poorly defined target is a pain. Now that there's more clarity we'll probably have a signing tool before too long.

Authenticated variables are the other part of the puzzle. If a variable requires authentication, the operating system's attempt to write it will fail unless the new data is appropriately signed. The key databases (white and blacklists) are examples of authenticated variables. The signing actually takes place in userspace, and the handoff between the kernel and firmware is identical for both this case and the unauthenticated case. The only problem in Linux's support here is that our EFI variable support was written to a pre-1.0 version of the EFI specification which stated that variables had a maximum size of 1024 bytes, and this limitation ended up exposed to userspace. So all we really need to do there is add a new interface to let arbitrary sized variables be written.

Summary: We don't really support secure boot right now, but that's ok because you can't buy any hardware that supports it yet. Adding support is probably about a week's worth of effort at most.

comments

vi users have their . (dot) redo command for repeating the last command. The article Repeating Commands in Emacs in Mickey Petersen s blog Mastering Emacs explained Emacs equivalent for that, namely the command repeat, by default bound to C-x z. I though seldomly use it as I mostly have to repeat a chain of commands. What I use are so called Keyboard Macros. For example for the CVE-2011-3192 vulnerability in Apache I added a line like Include /etc/apache2/sites-common/CVE-2011-3192.conf to all VirtualHosts. So I started Emacs with all the relevant files: grep CVE-2011-3192 -l /etc/apache2/sites-available/*[^~] xargs emacs & To remove those Include lines again M-x flush-lines is probably the easiest way in Emacs. So for every file I had to call flush-lines with always the same parameter, save the buffer and then close the file or in Emacsish kill the buffer. So while working on the first file I recorded my doing as a keyboard macro:

C-x (

Start recording

M-x flush-lines<Enter>CVE-2011-3192<Enter>

flush all lines which contain the string CVE-2011-3192

C-x C-s

save the current buffer

C-x C-k<Enter>

kill the current buffer, i.e. close the file

C-x )

Stop recording

Then I just had to call the saved macro with C-x e. It flushed all lines, saved the changes and switched to the next remaining file by closing the current file with three key-strokes. And to make it even easier, from the second occasion on I only had to press e to call the macro directly again. So I just pressed e for a bunch of time and had all files edited. (In this case I used git diff afterwards to check that I didn t wreck anything by half-automating my editing. :-) Of course there are other ways to do this, too, e.g. use sed or so, but I still think it s a neat example for showing the power of keyboard macros in Emacs. More things you can do with Emacs Keyboard Macros are described in the EmacsWiki entry Keyboard Macros. And if you still miss vi s . command in Emacs, you can use the dot-mode, an Emacs mode currently maintained by Robert Wyrick which more or less automatically defines keyboard macros and lets you call them with C-..

A new version of the digest package (which generates hash function summaries for arbitrary (and possibly nested) R objects using any of the standard md5, sha-1, sha-256 or crc32 algorithms) is now on CRAN. Bryan Lewis, who is doing very stuff with websockets and Html5 as he demoed last week the Chicago R User Group meeting, needed raw support and was kind enough to send a patch. After some more testing, this change set is now in the new version which went onto CRAN earlier today. CRANberries provides the usual summary of changes to version 0.5.0. As usual, our package is available via the R-Forge page leading to svn and tarball access, my digest page and the local directory here.

Anyone who works with building something with a few libraries will quickly become familiar with rpath's; that is the ability to store a path inside a binary for finding dependencies at runtime. Slightly less well known is that the dynamic linker provides some options for this path specification; one in particular is a path with the special variable $ORIGIN. ld.so man page has the following to say about the an $ORIGIN appearing in the rpath:

  ld.so understands the string $ORIGIN (or equivalently $ ORIGIN )
  in  an  rpath specification to mean the directory containing the
  application  executable.  Thus,  an   application   located   in
  somedir/app   could   be  compiled  with  gcc  -Wl,-rpath,'$ORI 
  GIN/../lib' so that it finds an  associated  shared  library  in
  somedir/lib  no matter where somedir is located in the directory
  hierarchy.
  

As a way of a small example, consider the following: Now, wherever you should choose to locate main, as long as the library is in the right relative location it will be found correctly. If you are wondering how this works, examining the back-trace from the linker function that expands it is helpful: Just from a first look we can divine that this has found the RPATH header in the dynamic section and has decided to expand the string $ORIGIN. At this point it proceeds to do a readlink on /proc/self/exe to get the path to the currently running binary, effectively does a basename on it and replaces the value of $ORIGIN. Interestingly, if this should fail, it will fall back on the environment variable LD_ORIGIN_PATH for unknown reasons. This might be useful if you were in a bad situation where /proc was not mounted and you still had to run your binary, although you could probably achieve the same thing via the use of LD_LIBRARY_PATH just as well. This feature should be used with some caution to avoid turning your application in a mess that can't be packaged in any standard manner. One common example of use is probably your java virtual machine to find its implementation libraries. There is a small but interesting security issue. If an attacker can hardlink to your binary that is using $ORIGIN then any path expansion is now relative to the attackers directory. Hence it is possible to make the linker read in arbitrary libraries and hence run arbitrary code. Clearly if your program is running setuid as someone else, such as root, you've just given up the keys to the house! Luckily, the dynamic linker will not let you shoot yourself in the foot like this, and prevents expansion of the $ORIGIN field if the program is running setuid. However, this is a good example of why you might want to keep users home and temp directories on separate file systems away from where binaries live. There is also a similar variable $PLATFORM, which describes the current running system. This is gained via the auxiliary vector, which I have written about here and here. Setting LD_SHOW_AUXV=1 will allow you to examine this value.

Funnily two of my daily web comics recently featured interesting things you could do with toilet paper: Zits on 19th of September 2011 involving a fan and Calvin and Hobbes on 13th of September 2011 involving flushing the toilet. Although both experiments are obviously resource wasting, they look like quite some fun and I m tempted to actually try them both at least once. (I though don t plan to try this, too. :-)

As the administrator for several MTAs I've received my fair share of support requests for "unreadable" e-mails. They generally have an attachment called winmail.dat. This is what all (for exceptions see the rest of this blog post) non-Outlook MUAs can display, when they receive an e-mail with the application/ms-tnef (or application/vnd.ms-tnef) MIME type. TNEF is a propietary format cooked up in Redmond. When chosen, the e-mail in question isn't sent as text/plain or at least text/html, but the entire e-mail is put into an TNEF-encoded attachment, most often called winmail.dat, in which you (most commonly) find the content of the e-mail in RTF, possible attachments are also put into the winmail.dat. The easieast solution for all involved parties is to convince the sender, that her Outlook and/or Exchange setup needs fixing (disabling TNEF in Outlook and Exchange 2003). Sadly this won't work all the time. The other solutions are to either filter the e-mails on your e-mail server and run them through something like ytnef_smtpd.py (please note, that this might be legally problematic for you as a service provider, opening an e-mail, even automatically, is a violation of communication privacy rights in many jurisdictions). Or, if the legal problem is too big or can't be solved through an local server installation (e.g. because there is no server running 24/7), you're left with either using Outlook (probably what Microsoft intended) or you can use a plug-in/add-on for your client. I've tested the LookOut add-on for icedove and it seems to work well. There are of course plug-ins for many other clients, but I can't honestly tell whether they work or not. [UPDATE 2011-09-20]: Fellow Debian Developer Josselin Mouette wrote me, that Evolution is capable of reading TNEF-encoded e-mails out of the box, but he also said that [TNEF e-mails should] be banned by all means.[/UPDATE 2011-09-20] Now, before I close, please always try to convince the sender/postmaster to fix their setup. It has only advantages: text/plain e-mails are smaller, they're less likely to contain malicious code (i.e. no parts hidden away from the virus scanner on the mail server (amavis can read TNEF e-mails, if libconvert-tnef-perl is installed)) and all recipients can open the e-mails with the client of their choice.

I m listening to quite a few podcasts recently. Here are my current favourites:

• Cyberunions which covers the tech/work crossover space and is appearing as Ogg near the start of most weeks. I reviewed an earlier episode and it s kept on improving. It s under CC-BY-NC-SA, so you can share it, but it isn t free software (FOSS) itself. Thanks to John Atherton for the tip.
• The Command Line is more tech, but with a leaning towards activism too. This podcast is actually FOSS sadly the only one of this round-up! It s recently dropped to once a week on Sundays, which is a shame in one way, but suits me better. I don t remember how I got started on this.
• More or Less: Behind the Stats is about numbers in the news. Maybe my love of statistics influences me, but I think this is brilliant, puncturing the pompous politicians who try to mislead (figures don t lie, but liars do figure). Imagine the FactCheck Blog with its own show and a sense of humour. It s just gone on a break, but there s plenty of recent episodes to catch up on.
• Real Peloton is about pro cycling, by reporter-presenter Ned Boulting and journalist Matt Rendell. They probably should not be left alone to make podcasts, but it s great that they do. They appear sporadically, depending on other work schedules of the presenters. The Banjo Cycles team including Matt Rendell won the IG Markets Hot Lap during the final stage of the Tour of Britain in London and the world championships are this week so maybe they ll have something to say soon.
• Answer Me This appears on Thursdays (although it has just gone on a month s break) and gives right-but-sideways answers to listener questions.

And now for number six, what are you listening to? Anything you d recommend?

I searched forever for a way to just build the examples in the Android NDK, preferably from the command line. Here's how. First step: build the C++ code:

cd /usr/src/android-ndk-r6/samples/san-angeles/
/usr/src/android-ndk-r6/ndk-build

Simple enough. Second step:

You will then need to generate your final .apk through the usual means. -- Android NDK Dev Guide

Thanks a ****ing lot! In addition the documentation is very much Eclipse-centered - and I can't be bothered to install and configure all the Eclipse stack for a mere Java wrapper for the above-generated .so file XD We first need to use the 'android' wrapper and download the "SDK Platform Android 2.2, API 8, revision 3" - aka "android-8", the first environment to support OpenGL ES 2.

/usr/src/android-sdk-linux_x86/tools/android

It appears we first need to "manage the project", more precisely import from an existing project: http://developer.android.com/guide/developing/projects/projects-cmdline.html So we create a project by importing the sources from the current directory:

/usr/src/android-sdk-linux_x86/tools/android update project \
  --name san-angeles --path . --target "android-8"

Then we use ant to build the thing: http://developer.android.com/guide/developing/building/building-cmdline.html

apt-get install ant openjdk-6-jdk
ant debug  # avoid messing with signing

We now have a bin/san-angeles-debug.apk and we can run it in the emulator. Create a new "Virtual device" using the "android-8" platform we downloaded above, and launch it. Given the time needed to boot the OS, I suggest enabling snapshots. Scaling to display resulted in a very tiny, dark and slow display, in a first step I suggest using a non-scaled one.

/usr/src/android-sdk-linux_x86/tools/android

At last we can run it! Install it in the emulator:

/usr/src/android-sdk-linux_x86/platform-tools/adb install -r bin/san-angeles-debug.apk

You'll find a DemoActivity in your apps directory

---

Note: I tried to compile the "hello-gl2", showcasing OpenGL ES 2.0, but sadly it crashes:

Sorry! The application GL2JNI (process com.android.gl2jni) has stopped unexpectedly. Please try again.

The emulator apparently doesn't advertise EGL_OPENGL_ES2_BIT (checked in in GL2JNIView.java). If you try to by-pass it, you'll get loads of:

E/libEGL ( 333): called unimplemented OpenGL ES API

It's worth noting the OpenGLES20 sample from the "API Demos" application uses a fall-back OpenGL ES 1 renderer if v2 is not available - it made me mistakenly believe that OpenGL ES 2 was supported only from Java >( OpenGL ES 2 is not supported in the emulator as of 2011-09-17 - period.

A while ago I've been interviewed for a forthcoming article in the LinuxUser magazine on the topic of why there are so many Debian derivatives and on the potential impact, on them, of the (potential) advent of the so called "Debian Rolling" experiment. Below you can find my take on those topics. Questions are by Ferdinand Thommes, who has been involved in various Debian derivatives, including the forthcoming Siduction.

---

1.) Why do you think people base distributions on Debian? Looking at how many there is, since Corel and Libranet set sail in 1999, there must be a reason why, for one, people fork Debian in the first place, and secondly, why they choose Debian to do so and not some other distribution.

There are always advantages in creating a so called "derivative" distribution with respect to creating one from scratch. The main one is the possibility of reusing the packaging work which is already done in the "upstream" distro, focusing derivatives' efforts on customization, usually with clear target public in mind. Debian pushes most of those advantages (for derivatives) at their extremes.

• The Debian archive is huge (about 30'000 packages with Squeeze), meaning that the amount of packaging work that can be reused is similarly huge.
• Debian has a long history and a well established community, which reduces the risk of "betting" the efforts of derivatives on a distro that might disappear anytime soon.
• Debian is well known for its obsession with package quality, meaning that the risk of trusting pre-existing packages is greatly reduced, and that package behavior is on average very dependable as well as documented in standard documents such as the Debian Policy
• Debian is also well known for its obsession with licensing, at the point of being considered by many one of the authoritative sources for deciding what constitutes Free Software and what does not. For derivatives that implies that they do not need to worry---as long as they trust Debian, of course---neither about legal redistribution risks nor about software freedom claims on their distro.

But there are also a couple of "political" reasons for basing derivatives on Debian. One is quite subtle and applies mostly to commercial distributions. If you are designing one such commercial distro, you have to be based on an independent distro with no commercial interests, lest risking that petty (technical or otherwise) choices might be made just to undermine your business. Among "popular" GNU/Linux distros, Debian is essentially the only one which is both volunteer-based and not ascribable to any specific company. The last reason why I think many derivatives are based on Debian is our vocation to universality. Since we are not targeting any specific public, but rather trying to provide an operating system suitable for several different use cases (e.g. the tasks supported by the Debian installer or the choice of Debian Pure Blends), we offer an ideal starting point for those who are going to have customization as their main business.

2.) What could be the benefits for Debian and for its Derivates, if the discussions about a second Debian Release based on the Testing branch really come to be reality?

We are considering the possibility of adding a new Debian suite which is "rolling" like Testing, in the sense that new software releases flow in regularly rather than only at periodic bumps corresponding to major distribution releases. There seem to be user demand for that and I personally believe that such a scheme could address the needs of both advanced desktop users and of developers, which are now strand to mix and match Debian suites ending up using package combinations which benefit, as a whole, of very little quality check. The main issues to face to get there is how to make sure that adding such a suite would not get in the way of preparing high quality Stable releases, in terms of quality control, developer focus, etc. Note that such a hypothetical "rolling" suite needs does not necessarily need to be based on Debian Testing, although that is a good starting point. The reason why it's not a good endpoint is that Testing has been created (circa 2000), and is still used, as an internal tool to prepare the forthcoming Debian Stable release. As such, it is not entirely suitable for final user consumption. For instance, during the Debian "freeze" period the flow of new software releases that reach Testing is greatly reduced up to stopping completely just before the release of a new Debian Stable. I've been told that users of testing-based (derivative) rolling releases regularly complain about this aspect during Debian freezes, which is not surprising. The main advantage for derivatives which are already offering a Debian-based "rolling" release will then be the possibility of basing their work on a suite less bound to the life cycle of Debian Stable than Testing is at present. On the converse side, that would also mean that derivatives will need to differentiate more from Debian than by just saying they are "rolling", which is a good thing: differences among distros are healthy and drive innovation. With those derivatives, on the other hand, which will recognize their goals as being aligned with Debian's goals---Debian Stable today, maybe Debian "rolling" tomorrow---we will be happy to join efforts.

Fixing the Sandbrige MacBook Air display initialization We left our hero with Debian installed on the MacBook Air, but with the display getting scrambled as soon as the i915 driver loaded. As was reported to Matthew, the problem is as simple as a lack of the right mode for the eDP panel in the machine. This mode is supposed to come from the panel EDID data, but for some reason the driver wasn t able to query the EDID data, and so it decided to try some random panel timings it dug out of the VBT tables, which are generally supposed to be used by LVDS panels. Apple helpfully stuck valid data there, but for some other panel one that is 1280x800 pixels instead of the 1366x768 pixel panel in the MacBook Air. I heard rumors that some machines would get a black screen when the i915 driver loaded. I was fortunate my machine simply displayed a 1366x768 subset of the programmed 1280x800 mode. A bit of garbage on the right side, and a few scanlines missing at the bottom. Quite workable, especially after I ran fbset -yres 768 to keep the console in the visible portion of the screen. DDC failure Looking through the kernel logs, the Intel driver tries to access the EDID data and times out, as if DDC is just completely broken. This is rather unexpected; the eDP spec says that the panel is required to support DDC and provide EDID. Now, we ve seen a lot of panels which don t quite live up to the rigorous eDP specifications, but it s a bit surprising from Apple, who generally do VESA stuff pretty well. We ve heard reports about panels reporting invalid EDID data, or EDID data which didn t actually match the panel (causing us to prefer the VBT data on LVDS machines). But I ve not heard of an eDP panel which didn t have anything hanging off of the DDC channel. But X works fine? During early debugging, I happened to start X up. Much to my surprise, X came up at the native 1366x768 mode. Digging through the kernel logs after that, I discovered that EDID was successfully fetched from the eDP panel while X started up. At this point, I knew it was all downhill the EDID data was present, it just wasn t getting picked up during the early part of the driver initialization when the console mode is initialized. eDP power management The CPU is given complete control over the power management of the eDP panel; sequencing through various power states and waiting appropriate amounts of time when things change. Given the goal of keeping power usage as low as possible, this makes a huge amount of sense. The eDP spec is quite clear though, without power, the panel will not respond to anything over the aux channel, and that includes EDID data. The eDP panel power hardware in the Sandybridge chip has a special mode for dealing with this requirement. If the panel is not displaying data, you can supply power for the aux channel stuff by setting a magic bit in the panel power registers. When initializing the frame buffer, the kernel driver turns off the panel completely so that it has all of the hardware in a known state (yeah, this is not optimal, but that s another bug). When X started, the panel was already running with the console mode. Given the difference between these two states EDID querying with the panel off failed, while EDID querying with the panel on worked, it seemed pretty clear that the panel power wasn t getting managed correctly. So, it seemed pretty clear that the magic power the panel bit wasn t getting turned on at the right times. Getting the power turned on. I stuck a check inside all of the aux channel communication functions to see where things were broken. This pointed out several places missing the panel power calls. This wasn t quite sufficient to get EDID data flowing. The remaining problem was that the code wasn t waiting long enough after turning the panel power on before starting the aux channel communication. A few msleep calls and huzzah! EDID at boot time and the console had the right mode. Making it faster However, it turns out that the driver does this a lot, and the msleeps required were fairly long the eDP panel wants a 500ms delay from turning the panel power off before you can turn it back on. I fix this by simply delaying the panel power off until things were idle for a long time. Now mode setting goes zipping through, and a few seconds later, the bit to force panel power on gets turned off. Getting these bits for yourself I ve pushed out the code to my (temporary) kernel repository git://people.freedesktop.org/~keithp/linux in the fix-edp-vdd-power branch. I d love to hear if you ve tried this on either a MacBook Air or any other eDP machine from Ironlake onwards.

This morning, I was one of a mere 900 or so marathoners in the second (?) annual Fox Valley Marathon, along with a 20 mile training race (as Chicago's big marathon is coming up) and a half marathon. This all took place along the Fox river, which is about an hour west of Chicago. The race went well enough. I had been racing twice a year at this distance, but took a break after running with some minor injury and generally not too well (see e.g. my writeup from last year's Boston and Chicago marathons). So I wasn't even sure I'd run one this year at all, but the bug got me again when two friends signed up for this, and off we went to train over the summer. So with a few more miles from training, I was pretty antsy and went out too fast as usual, and paid with an as-usual slower half and few short walking breaks past mile 20. But I beat the hope-for target time of 3:25 quite handily with a 3:19:03 -- or a 7:36 min/mile pace. Which (as I noted in a quick post to Google+ started right after the race, and since edited/expanded -- prods to Google+) should hopefully get me back to Boston next spring.

Installing Linux on Sandybridge MacBook Air Matthew Garrett got a report from someone who bought a brand new Sandybridge MacBook Air and was trying to install Fedora on it. The screen went black as soon as the i915 driver tried to set the initial mode. I bought one to try and help out. Just getting Linux installed turned out to be a minor adventure and I thought I d write a few notes for the next person who comes along and tries to do this. The obvious method I downloaded a collection of Debian ISO images, live images, netboot images from testing and CD-1 of squeeze. I downloaded a Fedora 16 alpha XFCE live image. I burned all of these to actual CDs, and then stuck them in a CD drive and held the c key while booting the machine. Nothing worked. The CD would spin up, the screen would switch from white to a black text mode with a blinking cursor at the upper left and then then CD would stop. rEFIt attempts rEFIt provides a boot selection menu and various configuration bits to provide for a multi-boot environment. This is necessary to leave the OS-X install on the disk and also install Linux. It also lets you boot from removable media, or at least that was the story. rEFIt would list removable USB flash storage devices, but it would not list CD drives, even after the CD drive spun up and did stuff for a while. This makes me wonder if the c key technique wasn t actually trying to boot from the CD. However, rEFIt wouldn t boot from a grub2-enabled USB key. ISO on USB flash key What rEFIt will boot is a Fedora or Debian ISO image copied directly to the USB flash device. The Ubuntu 11.04 live image and it did not work. Fedora F16 Alpha The live image came up and started X in frame buffer mode. However, when I went to install the image to the hard disk, it got stuck trying to discover devices. I waited about 20 minutes and it never finished, although it did chew up a lot of CPU time. Debian Squeeze I copied CD-1 of the 32-bit Squeeze distribution to my flash key. That booted up to text mode and went through the install. Nothing really unusual until it came to installing grub. I let it install grub to /dev/sda4, the new partition I had created for Linux. I figured installing grub to /dev/sda would be a bad idea (I think it would have actually worked just fine in retrospect). A slight out-of-sync diversion here I couldn t get the 32-bit kernel to talk to any keyboard or USB networking device, so I eventually installed a 64-bit kernel and that works just fine. You can run a 32-bit user space with a 64-bit kernel without any trouble. Making EFI boot Naturally, I couldn t get the machine to boot to Linux after this. rEFIt, the multi-boot tool for Mac OS X appears only interested in booting EFI images, or at least was not interested in booting my BIOS-based Grub2 installation on /dev/sda4. I used the rescue mode on the USB flash drive to get back to my installed Linux image and installed grub-efi-amd64. The OS X EFI version does not support 32-bit EFI code, which seems entirely reasonable to me. What took a while was figuring out that all of the grub files need to be readable through EFI, and that the Mac EFI code can only read FAT (and HFS?) partitions. So, the trick appears to be to stick the grub files on the Mac OS X efi partition (/dev/sda1). I created a directory over there, /EFI/grub and then mounted that as /efi and made a symlink from /boot/grub to /efi/EFI/grub. Now, grub-install sticks the files in the right place, and update-grub will even place grub.cfg in the directory where it needs to go. At this point, rEFIt will happily find the core.efi and grub.efi files and show them in the list of possible operating systems to boot. I don t know why there are both core.efi and grub.efi files; I ve only ever picked grub.efi and that has worked nicely. Broadcom WiFi I haven t gotten that working yet; the b43 supported chip list says it works in kernel 3.1. I m busy building a 3.1-rc4 kernel as I type this, so perhaps that will just work when I boot that. i915 KMS The whole reason for doing the install is to fix problems with the eDP EDID discovery which is currently preventing the machine from running X. I ll be working on that next; a quick kludge to just read the current mode from the device should get something on the screen (we already do that for LVDS panels). Beyond that, I ll be trying to figure out whether I can make EDID actually work, or whether it s just not available in this machine. Somehow the kernel frame buffer ended up as 1280x800, which was quite annoying. As a temporary kludge, I ran fbset -yres 768 to at least avoid having text disappear off the bottom of the screen. Once the panel size is discovered correctly, that should be fixed. Status This is sure not like installing Linux on a regular PC. However, I m reasonably hopeful that the devices in the machine will be working pretty well under Debian within the next few days.

Sounds easy doesn t it? You got a nice debian computer filled with mp3s and you got an iPod Shuffle. You want to get the mp3s onto the Shuffle, simple really! Well, no. I ve got a 2nd Generation iPod Shuffle; its a little blue thing the size of a postage stamp with no screen. Specifically a model A1204. It uses the headphone socket with a special connector to get it connected via USB. It also steadfastly refused to play anything I transferred onto it. gtkpod was first. I knew I was in trouble when my model doesn t appear in their list of supported devices. 2nd Generation Shuffles do, just not an A1204. After a lot of fiddling around and playing with special files and firewire IDs I got a file onto it. Great, but then turning the iPod on I got the green and orange sequence of lights which means no file found. Next, was Rhythmbox. It is supposed to understand iPods too. The funny thing was that it saw the music file I put on the iPod by gtkpod, but adding any file gave me the same colour sequence. What was worse is if you tried to look at the information about the iPod, Rhythmbox crashed. It seemed I was doomed to use iTunes on the windows computer in the living room. A program that seems designed not to work the way I want. (It tries to load all the mp3 files from my son s games into the iPod but ignores my music on the samba share, for starters). I just wanted the files on the player and the player to recognise and play them. It didn t seem like a too hard a task, but I was stuck. That was until I found a small (643 lines) python script called rebuild-db that builds the special iPod files with my music. I found the file at the rebuild-db sourceforge page and it was very simple process. Now I plug the iPod in, wait for the window to appear, push my songs onto the iPod and run the script. It s quick and simple and more importantly it works. Not bad for a program that hasn t been developed since April 2006.

Since it was Software Freedom Day again, I figured I should revisit my personal software freedom and see what has changed since my post two years ago. There hasn't been a vrms meme again this year so I don't have anyone else to compare with. A small survey on IRC indicates folks still need non-free nVidia drivers, embedded software, GNU documentation, Java and more. Since then I have switched my laptop from the Dell Inspiron 6400 to the Thinkpad X201 Tablet. Nothing really changed with my laptop, GNU documentation is still non-free and the new Intel WiFi chip on my new laptop still uses non-free embedded software. To remind myself of the non-free bits embedded in the hardware (CPU microcode, BIOS, EC etc), I have installed intel-microcode and microcode.ctl. Since then I didn't get any new phone, still the same OpenMoko FreeRunner. I'm now using QtMoko on a 4GB microSD card and Debian and SHR on other cards for testing. QtMoko is based on Debian and is the latest incarnation of Qtopia. It is probably free but I haven't done any audit of it. One interesting thing that changed with the FreeRunner in the past two years is that Harald Welte and friends started OsmocomBB, a project to create free software for the GSM modem built into the FreeRunner and many old feature-phones. I haven't tried it yet due to lack of time. While at the Chaos Communication Camp 2011 I learnt about the blackbox that SIM cards are (video), which hopefully OsmocomBB has a chance to protect against. AFAIK nothing else has changed to improve software freedom on the FreeRunner, the WiFi, GPS and other parts still contain non-free software with no chance of a replacement in sight. Since then I am still using the same wireless router and ADSL modem and I wasn't brave enough to try replacing their software. I still use gmail out of intertia, but I at least have started using offlineimap to prepare for the day when I will move away from it. I still use Google Maps, mainly for searching for public transport routes. With my recent travels in Europe I encountered many places where OpenStreetMap has much better coverage than Google. One nice thing about Google Maps adding public transport information is that public transport organisations have been publishing public data feeds for Google to consume. Apparently there is also a realtime variant. I also found one website using a JSON API for Google street view. The combination of these gives me hope for better support for spatial data in free software. The data (aerial and streets) will obviously remain a much longer term freeness issue though. For flash video on the web get-flash-videos appeared and made it even easier to ignore flash, especially since I got a few patches added to it. In addition Swfdec died but Gnash improved enough to replace it. Lightspark was also created but has so far been pretty buggy and unstable. Free RAR v3 format support appeared in the form of TheUnarchiver. The Voxware audio codec and DigiTrakker MDL files are still not supported by Rhythmbox. Unicode has been updated and so now Debian doesn't support every script fully. I still manage to avoid Skype. The nouveau drivers have matured enough to provide 3D support, multi-monitor support and general stability for the one nVidia card I used in the past two years. Overall, I'm reasonably happy with my level of software freedom. My main strategy for preventing regressions in my software freedom remains to just avoid doing things that require non-free software. The most problematic FLOSS issues for me are embedded software, spatial data support and Flash support. Please feel free to contact me with any comments or questions. Happy Software Freedom Day everyone!

Brown-bag release time for littler. One of the <emph>minor cleanups</emph> in the 0.1.4 release from Thursday actually introduced a nasty little bug as you can't call Rf_KillAllDevices() when you do not have any graphics device. Doh. So with apologies for the lower quality release, and the two hour malfuntioning window for CRANberries that it cause, a repaired version is now out there. The very brief (upstream) ChangeLog is as follows:

2011-09-17  Dirk Eddelbuettel  <edd>
        * configure.in: Mark as release 0.1.5
        * littler.c (littler_CleanUp): Remove call to Rf_KillAllDevices() 

littler provides r (pronounced littler), a shebang / scripting / quick eval / pipelining front-end to the the R language and data analysis environment. As usual, our code is available via our svn archive or from tarballs off my littler page and the local directory here. A fresh package is in Debian's incoming queue and will hit mirrors shortly.

So I decided, once the disk was in place, that the whole system should move to a RAID1 over the two disks. Basically this is not that hard as it may seem at a first glance, but I had some problems due to a new sector size in some recent harddisks, which is called Advanced Format.

1. Install mdadm with apt-get. Make sure to answer 'all' to the question which devices need to be activated in order to boot the system.
2. Partition the new disk (almost) identical.
   Because the new drive is somewhat bigger that wouldn't make sense, but at least the two partitions which should be mirrored on the second disk, need to be identical.
   Usually this is achieved easily by using

   sfdisk -d /dev/sda sfdisk /dev/second/sdb

   
   
   In this case, it wasn't that easy. But I will come to that in a minute.
3. Change the type of the partitions to 'FD' (Linux RAID autodetect) with fdisk
4. Erase evidence of an eventual old RAID from the partitions, which is probably pointless on a brand-new disk, but we want to be sure:

   mdadm --zero-superblock /dev/sdb1

   mdadm --zero-superblock /dev/sdb2

5. Create two DEGRADED raid1 arrays from the partitions:

   mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdb1 missing

   mdadm --create /dev/md1 --level=1 --raid-devices=2 /dev/sdb2 missing

6. Create filesystem on the first raid device, which will become /boot.
7. Mount that filesystem somewhere temporary and move the contents of /boot to it:

   mount /dev/md0 /mnt/somewhere

8. Unmount /boot, edit fstab to mount /boot from /dev/md0 and re-mount /boot (from md0)
9. Create mdadm configuration with mdadm and append it to /etc/mdadm/mdadm.conf:

   mdadm --examine --scan >> /etc/mdadm/mdadm.conf

10. Update the initramfs and grub (no manual modification needed with grub2 on my system)
    and install grub into the MBR of the second disk.

    
    

    update-initramfs -u

    update-grub

    grub-install /dev/sdb

11. The first point to pray: Reboot the system to verify it can boot from the new /boot.
12. Create a physical volume on /dev/md1:

    pvcreate /dev/md1

13. Extend the volume group to contain that device:

    vgextend <volgroup_name> /dev/md1

14. Move the whole volume group physically from the first disk to the degraded RAID:

    vgmove <volgroup_name> /dev/md1

    
    
    (Wait for it to complete... takes some time ;)
15. Reduce first disk from the VG:

    vgreduce <volgroup_name> /dev/sda2

16. Prepare it for addition to the RAID (see step 3 and 4) and add it:

    mdadm --add /dev/md0 /dev/sda1

    mdadm --add /dev/md1 /dev/sda2

17. Hooray! Watch into /proc/mdstat. You should see that the RAID is recovering.
18. When recovery is finished pray another time and hope that system is still booting with it running from the RAID entirely. If it does: Finished :-)

There is some action taking place with the hardware vendors to move to a new sector size. Physically my new device has a size of 4096 bytes per sector. Somewhat different to the 512 bytes disks used to have the last decade.

Device Boot Start End Blocks Id System

/dev/sdb1 2048 291154 144553+ fd Linux raid autodetect

/dev/sdb2 291160 625139334 312424087+ fd Linux raid autodetect

I just download this tool here. After finishing a really simple setup wizard, which includes choosing your blog service provider, filling in your username&passwd info, I m now writing my first blog with this tool. Before I download and begin to use it, I knew from reviews that this is NOT a WYSIWYG tool. I really doubt it that it could be useful for a person like me who have nearly no HTML knowledge. But so many reviews mention this tool and all give it high options that I decide to give it a try. The tool looks pretty neat, isn t it? It s simple for me to use any buttons without guessing what they will do. And you can see that there s Preview tab, in which you can check the real layout when published. The preview tab(it seems the first picture is not uploaded successfully?) In all, I begin to love this editor even this is my first time using it! It s simple yet completely enough for users like me. And with this editing HTML directly, I can even learn a little bit about its syntax. Don t mention all the merits an offline desktop editor can provide. Now, I ll try the last and most important function Publish! See you on my blog!

Triaging bugs is one of the easiest way to start contributing to Debian. I ll teach you the basics in this article. 1. Prerequisites All interactions with the Debian Bug Tracking System (BTS) happen through email so you need to have an email account with an address that you re willing to make public. All the mail that you send to the BTS will be archived and publicly available through its web-interface. This also means that you should have some spam filters in place because it will inevitably be harvested by spammers. To ensure that this email address is consistently used by the various tools that we re going to use, it s a good idea to put this email address in the DEBEMAIL environment variable. You can also specify your full name in DEBFULLNAME (in case you don t want to use the name associated with your Unix account). You usually do this by modifying ~/.bashrc (if you use bash as login shell):

export DEBEMAIL="hertzog@debian.org"
export DEBFULLNAME="Rapha l Hertzog"

You should also install the devscripts package, it provides the bts command that we re going to use. 2. Find a package or a team with too many bugs You can literally pick any popular software that s in Debian, they almost always get more bug reports than the maintainers can handle. Instead of picking a package, you can also select a packaging team and concentrate your efforts on the set of packages managed by the team. In any case, it s important to receive the bug traffic for the packages that you re going to work on. If you went for a specific package, you should subscribe to the package via the Package Tracking System (there s a subscribe box on the bottom left corner once you selected the source package of interest). If you decided to help a team, there s usually a dedicated mailing list receiving all bug traffic. You can browse a list of packages with the most bugs if you have troubles finding a package to work on. 3. Triage bugs! Bug triaging is all about making sure that bugs are correctly classified so that when a developer looks at the bug list, he can quickly find bugs with all the information required to be able to fix them! 3.1 Adding information to bug Adding supplementary information is easily done just by sending a mail to XXXX@bugs.debian.org (replace XXXX with the bug number). But often you want to reply to a message in the bug history, in that case bts --mbox show XXXX is for you. It will grab the corresponding mailbox and open a mailer (mutt by default) on it. Now you can directly reply in your favorite mailer. 3.2 Classifying bugs The Debian BTS uses tags (click the link and read the doc!) to classify bugs. bts tag XXXX + foo will add the foo tag (replace the + with a to remove a tag). If you want to explain why you re adding a tag, you should instead reply in the bug log as explained above, put control@bugs.debian.org in Bcc (Blind Carbon Copy) and start the body of your message with your tag command:

tag XXXX + foo
thanks

But what tag should you add? When a bug is submitted, you should try to reproduce the bug. If you can reproduce it, then tag the bug confirmed (example in #641710). If you can t, you should request more information (ex: a sample document triggering the bug, a configuration file, the output of some relevant command, etc.) until you can reproduce it or conclude that it was a user mistake. When you request supplementary information due to this, you should tag the bug unreproducible moreinfo (example in #526774). moreinfo should be later dropped when the requested information are provided, and unreproducible should be dropped if those information were enough to actually help reproduce the bug (example in #526774). During that initial evaluation, it s also worth differentiating packaging bugs (which are specific to Debian) from upstream bugs (which are relevant also for non-Debian users). The latter should be tagged upstream (and forwarded upstream if the bug is reproducible or contains enough information for the upstream developers, example in #635112). If you saw a (viable) patch in the bug log, the bug should be tagged patch . This is usually done by the patch submitter but sometimes it s forgotten (example in #632460). Take care though to not reinstate the patch tag if it was initially set but then dropped by the package maintainer after a review of the patch. If the title of the bug report is not descriptive enough, you can change it with a retitle XXXX new-title command (example in #170850). You can also change the severity of the bug report depending on the impact of the problem (with a command severity XXXX new-severity , what a surprise!). Request for new features are wishlist , most documentation problems are minor . On the other side of the scale, you can use important for bugs that are very annoying but that should not block a release. serious , grave and critical are used for release critical bugs, check the official definitions of the severities (examples in 502738 or #506498). 3.3 Closing non-bugs and bugs that are already fixed If your analysis of the bug report is that it s not really a bug but a user mistake, then you should close it by sending a mail to XXXX-done@bugs.debian.org with some explanations of the user s mistake so that he can get past his problem (example in #592853). If the problem was a real bug, but one that is apparently already fixed, you should try to quickly find the version that fixed the bug. If you can t find it in the changelog (there s a link to it in the PTS, or you can use /usr/share/doc/package/changelog.Debian.gz), you ll make the safe assumption that the upstream version you re currently using is the first one where this is fixed. Then you send a mail to XXXX-done@bugs.debian.org but you start your mail with Version: version-that-fixed-the-bug and continue with a small explanation of why you believe the bug to be fixed by this version (example in #122948). 3.4 Reassigning misfiled bug reports Bug reports are not always filed against the proper package. Users file bugs against applications where they experience the bugs, but the real bug might be in an underlying library or application. When that happens, you should use the reassign XXXX correct-package version command to get it filed against the correct package. The version parameter is optional but should be provided if possible, it should be the oldest version that we know to have the problem (example in #626232). 3.5 Forwarding bugs Forwarding bugs means opening bug reports in the upstream bug tracker for issues that have been reported in Debian but that applies to the upstream (unmodified) source code. Be sure to include all the relevant information and a link to the corresponding Debian bug. Depending on the upstream bug tracker, you might have to open an account to be able to file new bug reports. On the Debian side, you must record that a bug has been forwarded with bts forwarded XXXX upstream-bug-url . upstream-bug-url is the URL corresponding to the upstream bug report you created (ex: http://projects.ciarang.com/p/feed2omb/issues/21/ recorded in #609345 ). If the upstream authors fix the bug you reported, you can tag the Debian bug with fixed-upstream so that it s easier to find bugs to close when the next upstream release comes out (example in #637275). 3.6 Updating version information The Debian BTS uses version tracking to know which package versions are affected by a given bug. It s particularly important to have correct version information for release critical bugs since it might affect the migration of packages to testing. You can learn more on this topic here: http://wiki.debian.org/HowtoUseBTS. 4. More advice Colin Watson wrote a constructive rant explaining some mistakes that bug triagers are often doing. While it refers mainly to Ubuntu s launchpad, the advice apply equally as well to Debian. Check it out to become a better bug triager! Note that you can refer to this article with this shorter URL: http://raphaelhertzog.com/go/bugtriaging/

Do you want to read more tutorials like this one? Click here to subscribe to my free newsletter, you can opt to receive future articles by email.

One comment Liked this article? Click here. My blog is Flattr-enabled.

Happy 10th birthday Mozilla bug #98160, I hope you won t make it to your 11th, but who knows I hope the same for your younger brothers bug #166240 which is only 9 years old. Many thanks for all the people who tried to help RTL users switch their textarea directionality and alignment easier. Another bug in the it sometime sucks to be an RTL user department is bug #119857, soon to be age 10. I m lucky to have BiDi Mail UI as a good workaround, otherwise I couldn t read emails in Hebrew with the proper alignment/directionality. In another department, I must say I love daylight saving time, as I have more hours of sun, but this comes on the expense of my schedule as due to bug #504299 all my events aren t on the right local time, but an hour earlier. Although the post is written cynically, I would like to call the Mozilla community to help with fixing these issues. I also want to thank the people who are already helping with these issues or helped in the past.
Filed under: Free software applications, i18n & l10n, Mozilla