I've just released git-annex version 3, which stops cluttering the filesystem with .git-annex directories. Instead it stores its data in a git-annex branch, which it manages entirely transparently to the user. It is essentially now using git as a distributed NOSQL database. Let's call it a databranch. This is not an unheard of thing to do with git. The git notes built into recent git does something similar, using a dynamically balanced tree in a hidden branch to store notes. My own pristine-tar injects data into a git branch. (Thanks to Alexander Wirt for showing me how to do that when I was a git newbie.) Some distributed bug trackers store their data in git in various ways. What I think takes git-annex beyond these is that it not only injects data into git, but it does it in a way that's efficient for large quantities of changing data, and it automates merging remote changes into its databranch. This is novel enough to write up how I did it, especially the latter which tends to be a weak spot in things that use git this way. Indeed, it's important to approach your design for using git as a database from the perspective of automated merging. Get the merging right and the rest will follow. I've chosen to use the simplest possible merge, the union merge: When merging parent trees A and B, the result will have all files that are in either A or B, and files present in both will have their lines merged (and possibly reordered or uniqed). The main thing git-annex stores in its databranch is a bunch of presence logs. Each log file corresponds to one item, and has lines with this form:

timestamp [0 1] id

This records whether the item was present at the specified id at a given time. It can be easily union merged, since only the newest timestamp for an id is relevant. Older lines can be compacted away whenever the log is updated. Generalizing this technique for other kinds of data is probably an interesting problem. :) While git can union merge changes into the currently checked out branch, when using git as a database, you want to merge into your internal-use databranch instead, and maintaining a checkout of that branch is inefficient. So git-annex includes a general purpose git-union-merge command that can union merge changes into a git branch, efficiently, without needing the branch to be checked out. Another problem is how to trigger the merge when git pulls changes from remotes. There is no suitible git hook (post-merge won't do because the checked out branch may not change at all). git-annex works around this problem by automatically merging */git-annex into git-annex each time it is run. I hope that git might eventually get such capabilities built into it to better support this type of thing. So that's the data. Now, how to efficiently inject it into your databranch? And how to efficiently retrieve it? The second question is easier to answer, although it took me a while to find the right way ... Which is two orders of magnitude faster than the wrong way, and fairly close in speed to reading data files directly from the filesystem. The right choice is to use git-cat-file --batch; starting it up the first time data is requested, and leaving it running for further queries. This would be straightforward, except git-cat-file --batch is a little difficult when a file is requested that does not exist. To detect that, you'll have to examine its stderr for error messages too. Perhaps git-cat-file --batch could be improved to print something machine parseable to stdout when it cannot find a file. Takes some careful parsing, but straightforward. Efficiently injecting changes into the databranch was another place where my first attempt was an order of magnitude slower than my final code. The key trick is to maintain a separate index file for the branch. (Set GIT_INDEX_FILE to make git use it.) Then changes can be fed into git by using git hash-object, and those hashes recorded into the branch's index file with git update-index --index-info. Finally, just commit the separate index file and update the branch's ref. That works ok, but the sad truth is that git's index files don't scale well as the number of files in the tree grows. Once you have a hundred thousand or so files, updating an index file becomes slow, since for every update, git has to rewrite the entire file. I hope that git will be improved to scale better, perhaps by some git wizard who understands index files (does anyone except Junio and Linus?) arranging for them to be modified in-place. In the meantime, I use a workaround: Each change that will be committed to the databranch is first recorded into a journal file, and when git-annex shuts down, it runs git hash-object just once, passing it all the journal files, and feeds the resulting hashes into a single call to git update-index. Of course, my database code has to make sure to check the journal when retrieving data. And of course, it has to deal with possibly being interrupted in the middle of updating the journal, or before it can commit it, and so forth. If gory details interest you, the complete code for using a git branch as a database, with journaling, is here. After all that, git-annex turned out to be nearly as fast as before when it was simply reading files from the filesystem, and actually faster in some cases. And without the clutter of the .git-annex/ directory, git use is overall faster, commits are uncluttered, and there's no difficulty with branching. Using a git branch as a database is not always the right choice, and git's plumbing could be improved to better support it, but it is an interesting technique.

At beginning of the last two weeks, I was working on the final package of JTReg. I was working on generating man pages and I finally found a solution. I decided to use help2man to generate the man pages but I had to patch the two scripts jtreg.sh and jtdiff.sh . Basically, these two scripts launch main classes of the jtreg.jar JAR file. The problem with them is that they don t know where they can find the JAR file on Debian. So I patched the two scripts shell so they can locate JAR files in /usr/share/java .
To be able to generate man pages with help2man, I needed to be able to use the scripts. But with jtdiff.sh and jtreg.sh using jtreg.jar which is not in /usr/share/java during the build, I had to find a way to make the scripts work. I decided to patch the scripts (again) and make them depend of some environment variables. So in each script, there are 4 used variables:

• JAVA_HOME, which is the path where Java is,
• JTREG_HOME, which is the path where jtreg.jar is,
• JTHAR_HOME, which is the path where jtharness JAR are, and,
• JAR_HOME, which is the path where other JAR are.

So during the packaging only JTREG_HOME needed to be changed to be able to use the script. Once the scripts patch done, I finally wrote the rules file of the package. Once again, I used javahelper so the rules file contains the following code:

#!/usr/bin/make -f
JAVA_HOME = /usr/lib/jvm/default-java

override_dh_auto_build:
ant -f make/build.xml
dh_auto_build
JTREG_HOME=./dist/jtreg/lib/ help2man \
--name="Regression Test Harness" \
--help-option="-help" \
./dist/jtreg/linux/bin/jtdiff > jtdiff.1
JTREG_HOME=./dist/jtreg/lib/ help2man \
--name="Regression Test Harness" \
--help-option="-help" \
./dist/jtreg/linux/bin/jtreg > jtreg.1

override_dh_auto_clean:
rm -r dist :
rm -r build :
rm jtdiff.1 :
rm jtreg.1 :
dh_auto_clean

%:
dh $@ --with javahelper

As you can see it is not really complicated. The most interesting part is the one when the man pages are generated. The JTREG_HOME variable is set to ./dist/jtreg/lib/ because it is the path where jtreg.jar is once it is built. I also used the help-option option of help2man because the help option of jtreg and jtdiff is -help and not help .
The jtreg package is now on the Debian Java team SVN so anyone can get it using:

svn checkout svn+ssh://$ SVN_USER @svn.debian.org/svn/pkg-java/trunk/jtreg

Thanks to Sylestre Ledru, jtreg is now available in Sid and the ITP is now closed.

The second part of my work consisted in testing Jigsaw and fixed tests if necessary. All the packaging work is now useful since the tests need JTReg to be run. The problem with the current build system of Jigsaw is that it is enough generic to be used on any system. So I had to patch several makefiles. To find them:

cd jigsaw-tests
find . -name 'Makefile' grep '/test/Makefile'

Then, the path to find jtreg has to be modified to match /usr/bin/jtreg . After that it is possible to run the tests with the following command:

make test

Passing all the tests is something which takes a lot of time. After that, I identified 42 (that s a cool number right?) failing tests. Some of them came from the lack of X11 server and other from unreachable network hosts. Some hackers of IcedTea gave me some ideas to fix several tests. They told me to look here and here because I just had the same problems that they had before.
With this, I was able to fix the tests related to unreachable hosts and to the lack of X server. I used Xvfb to fix the X server depending tests.
After running the tests again, only 13 tests were still failing. That is a good result. Tom Marble and me decided to apply the patch of Alan Bateman and re-run the tests to see if it will break something:

cd jigsaw-tests
mkdir patches && cd patches
wget http://cr.openjdk.java.net/~alanb/jigsaw-mp-prototype1/webrev/jdk.patch
cd jdk
patch -p1 < ../patches/jdk.patch
make clean
make sanity
make all
make modules
xvfb-run -e xvfb-errors -a -s -ac make test -k

15 tests failed against 13 before. The 2 new failing tests are:

• java/lang/management/MemoryMXBean/LowMemoryTestConcMarkSweepGC.sh
• java/util/Locale/Bug6989440.java

I didn t go further but I ll try to understand why we have 15 failing tests in few days. All the modifications that I have made on the source code of Jigsaw can be found in a patch format here:

This 2 weeks were also dedicated to the understanding of Jigsaw and its modules. Reading the jigsaw-dev mailing list, I have found some interesting conversation about jigsaw and how to write modules. So I decided to write a module to see how it works. I followed the instruction of the quick start guide but when compiling the module didn t work. So I tried to understand why and then to fix it. I eventually compiled and installed my module and here is how I did it:

mkdir -p src/com.greetings/com/greetings/
mkdir -p src/org.astro/org/astro/
mkdir modules
vim src/com.greetings/module-info.java
vim src/com.greetings/com/greetings/Hello.java
vim src/org.astro/module-info.java
vim src/org.astro/org/astro/World.java
./jigsaw/build/linux-amd64/bin/javac -d modules \
-modulepath ./jigsaw/build/linux-amd64/modules/modules \
-sourcepath src find src -name '*.java'
./jigsaw/build/linux-amd64/bin/jmod create -L mlib
./jigsaw/build/linux-amd64/bin/jmod install \
modules org.astro com.greetings -L mlib
./jigsaw/build/linux-amd64/bin/java -L mlib -m com.greetings

And here it is, we have a nice hello world module. The documentation says to put the module-info.java of org.astro in src/org.astro/org/astro/ but it didn t work for me so I tried to put it in src/org.astro/ and it worked. Also the quick start guide says that we have to use the -modulepath option to specify where the modules are. I first tried to use ./jigsaw/build/linux-amd64/modules/ but javac told me he could not find java.lang so I noticed that there is a subdirectory called modules in ./jigsaw/build/linux-amd64/modules/ so I used it as module path and it worked.

Writing a module and seeing how Jigsaw modules are made started to make me think about the packaging of Jigsaw. Dependencies being available, knowing how to build and test Jigsaw and seeing how modules are made, I think that I will soon try to start the packaging of Jigsaw and getting started with the packaging will be my goal for the next two weeks.

I think the old-fashioned coredump is a little under-appreciated these days. I'm not sure when it changed, but I even had to add myself to /etc/security/limits.conf to raise my ulimit to even create one. Anyway, debugging __thread variables from coredumps is a bit of a pain. For the uninitiated, the __thread identifier specifies a thread-local variable, i.e. every thread gets its own copy of the variable automatically. The implementation of this is highly architecture specific. The reason is that TLS entries need to be accessed via a register kept as part of the thread state, and thus every architecture chooses their own register and builds their own ABI. On x86-32, which is very register-limited, you certainly don't want to dedicate a register to a pointer to TLS variables and take it out of operation. Luckily there is the hang-over from the 70's (60's? 50's?) segmentation. Without going into real detail, segment registers can be used to offset into a region of memory based on a look-up of a region descriptor stored in a table. Above, you see a simplified example of the %gs register loaded with the index value 2, and thus when you access %gs:20 what you are saying is "find entry 2 in the global descriptor table (GDT), follow it and offset 20 into that region. The kernel gives each thread its own GDT (i.e. the GDT register is part of the thread-state). Thus __thread variables are stored based on segment offsets and voila thread-local storage. Now, there's a few tricks here. For various reasons, a process can not setup entries in the GDT; this is a privileged operation that must be done by the kernel. There is actually a special system call for threads to setup their TLS areas in the GDT set_thread_area. When a new thread starts, the thread-library and dynamic linker conspire to allocate and load any static TLS data (i.e. if you have a global __thread variable initialised to some value, then every thread must see that value when it starts) and then calls this to make sure the variables are ready to go. After that, the gs register is filled with the index of that GDT entry, and all TLS access goes via it. That, in a nut-shell, is TLS for x86-32. Now, to the problem. Take, for example, the following short program: We start a few threads and then abort to make it dump core. But, if you try and examine foo: It seems that gdb doesn't know how to find the value of foo because its not a variable in the usual sense ("this target", in this case, means a coredump). It relies on accessing via the gs register, which relies on the current processes' GDT state, which has since been destroyed. If you care to consult the canonical source of TLS info, you can find out exactly why this is so hard to figure out generically. However, with some work, we can start to figure out the value by hand. A coredump is really a ELF file full of just two things: a bunch of LOAD segments that are just dumps of the process memory regions, and a NOTE section that includes a bunch of notes that the kernel dumps out for us such as the current register state, the process id, the signal that killed us, etc. Here's an example of a core file under readelf You can examine the notes; with readelf we see: This shows 5 notes of type NT_PRSTATUS; it should be little surprise that each of these notes describes the process status of one running thread. When gdb pops up [New Thread 4973] that's because it just hit another note that describes the new thread. To actually make sense of the note, however, we need some other tools that look deeper. The elfutils based tools give us a better description of the various notes in the coredump; more akin to what GDB is interpreting them as. Below I've extracted one thread's info: What's particularly interesting here is that the note type that was previously unknown (Unknown note type: (0x00000200)) has been resolved for us it is in fact of type NT_386_TLS and is a dump of the GDT entries for the thread. So, if we examine how our function is accessing our TLS variable by disassembling the thread function: Examining the disassembly we can see that foo is accessed via an offset of -4 from %gs (this is OK, as our limit value is maxed out. See the TLS ABI doc for more info). Now, we can examine gs and see which selector it is telling us to use: Above we shift out the last 3 bits, as these refer to the privilege level (bits 0 and 1) and if this is a GDT or LDT reference (bit 2). Thus, looking at the GDT descriptor for index 6: we can finally do some maths from the base-address to figure out the value: Thus we have found our TLS value; in this case for thread 3 the value is indeed 3. I caution this is the simplest possible case; other "models" (see the TLS doc, again) may not be so simple to work out by hand, but this would certainly be how you would start.

Yeah! It's this time of the year: Friday evenings after work with your friends having some cool beer on the streets, Saturdays around the nearby mountains for a good hike and swimming in a lake or river, and good beach Sunday in a Valencian beach. And for a great ending of a Summer weekend, a good indie movie in your neighbourhood, reclaiming the streets and going back to our roots, when people perceived the public spaces as theirs, and would bring foldable chairs out, would gather with their neighbours and had a good after-dinner chat a la fresca. The always active Associaci de Ve ns i Ve nes de Benimaclet has organized, for the fifth year four cinema projections in Benimaclet's square, which are open for anyone who wants to share good moments with us. The program this year includes Soul Kitchen (3rd of July), When the Wind Blows (10th), Concursante (17th) and Moon (24th). Before every movie, we'll enjoy live music by local bands, and projections of good short films. We'll be happy to see you there, and remember you only need a chair and some dinner... but be sure to be there a bit before 22:00: last year this got so popular some people started having issues to find good spots for their chairs!

So as Alpha 2 approaches, I find myself working towards porting the alternate installer/d-i to the pandaboard to support the netboot installer. There's not a lot of documentation that describes the internals of d-i, nor what bits are platform specific.

This is especially true when working towards creating a new subarchitecture since lots of little places have to be touched, kernels usually have to be tweaked, and all other sorts of odds and ins. This post isn't a comprehensive guide to what's necessary, but just little tidbits of what I did, just some random odds and ends.

The first step of any enablement is to have something you can run and boot. The netboot images, as well as the alternate kernel and ramdisk are built out of the debian-installer package. In the debian-installer package, several config files for driving the process are located in build/config/$arch/$subarch. For omap4, we have the following files:

boot/arm/generate-partitioned-filesystem
build/config/armel.cfg
build/config/armel/omap4.cfg
build/config/armel/omap4/cdrom.cfg
build/config/armel/omap4/netboot.cfg

boot/arm/generate-partitioned-filesystem is a shell script that takes a VFAT blob, and spits out a proper MBR and partition table.

build/config/armel.cfg simply is a list subarchitectures to build, and some sane-ish kernel defaults for armel.

build/config/armel/omap4.cfg is also a simple config file which specifies the type of images we're building, and the kernel to use in d-i. This file looks like this:

MEDIUM_SUPPORTED = netboot cdrom

# The version of the kernel to use.
KERNELVERSION := 2.6.38-1309-omap4
# we use non-versioned filenames in the omap kernel udeb
KERNELNAME = vmlinuz
VERSIONED_SYSTEM_MAP =

As a point of clarification, 'cdrom' is a bit of a misdemeanor; it refers to the alternate installer kernel and ramdisk used by alternate images, and not the type of media. Other types of images exist such as 'floppy' and 'hd-install', but these are specialized images, and out of scope for this blog post.

Each file in build/config/armel/omap4/* is a makefile thats called in turn for each image that created. The most interesting of this is the netboot.cfg

MEDIA_TYPE = netboot image
SUBARCH = omap4
TARGET = $(TEMP_INITRD) $(TEMP_KERNEL) omap4
EXTRANAME = $(MEDIUM)
INITRD_FS = initramfs

MANIFEST-INITRD = "netboot initrd"
MANIFEST-KERNEL = "kernel image to netboot"
INSTALL_PATH = $(SOME_DEST)/$(EXTRANAME)

omap4:
 # Make sure our build envrionment is clean
 rm -rf $(INSTALL_PATH)
 mkdir -p $(INSTALL_PATH)

 # Generate uImage/uInitrd
 mkimage -A arm -O linux -T kernel -C none -a 0x80008000 -e 0x80008000 -n "Ubuntu kernel" -d $(TEMP_KERNEL) $(INSTALL_PATH)/uImage
 mkimage -A arm -O linux -T ramdisk -C none -a 0x0 -e 0x0 -n "debian-installer ramdisk" -d $(TEMP_INITRD) $(INSTALL_PATH)/uInitrd

 # Generate boot.scrs
 mkimage -A arm -T script -C none -n "Ubuntu boot script (serial)" -d boot/arm/boot.script-omap4-serial $(INSTALL_PATH)/boot.scr-serial
 mkimage -A arm -T script -C none -n "Ubuntu boot script (framebuffer)" -d boot/arm/boot.script-omap4-fb $(INSTALL_PATH)/boot.scr-fb

 # Create DD'able filesystems
 mkdosfs -C $(INSTALL_PATH)/boot.img-fat-serial 10240
 mcopy -i $(INSTALL_PATH)/boot.img-fat-serial $(INSTALL_PATH)/uImage ::uImage
 mcopy -i $(INSTALL_PATH)/boot.img-fat-serial $(INSTALL_PATH)/uInitrd ::uInitrd
 mcopy -i $(INSTALL_PATH)/boot.img-fat-serial /usr/lib/x-loader/omap4430panda/MLO ::MLO
 mcopy -i $(INSTALL_PATH)/boot.img-fat-serial /usr/lib/u-boot/omap4_panda/u-boot.bin ::u-boot.bin
 cp $(INSTALL_PATH)/boot.img-fat-serial $(INSTALL_PATH)/boot.img-fat-fb
 mcopy -i $(INSTALL_PATH)/boot.img-fat-serial $(INSTALL_PATH)/boot.scr-serial ::boot.scr
 mcopy -i $(INSTALL_PATH)/boot.img-fat-fb $(INSTALL_PATH)/boot.scr-fb ::boot.scr
 boot/arm/generate-partitioned-filesystem $(INSTALL_PATH)/boot.img-fat-fb $(INSTALL_PATH)/boot.img-fb
 boot/arm/generate-partitioned-filesystem $(INSTALL_PATH)/boot.img-fat-serial $(INSTALL_PATH)/boot.img-serial

 # Generate manifests
 update-manifest $(INSTALL_PATH)/uImage "Linux kernel for OMAP Boards"
 update-manifest $(INSTALL_PATH)/uInitrd "initrd for OMAP Boards"
 update-manifest $(INSTALL_PATH)/boot.scr-fb "Boot script for booting OMAP netinstall initrd and kernel from SD card. Uses framebuffer display"
 update-manifest $(INSTALL_PATH)/boot.scr-serial "Boot script for booting OMAP netinstall initrd and kernel from SD card. Uses serial output"
 update-manifest $(INSTALL_PATH)/boot.img-serial "Boot image for booting OMAP netinstall. Uses serial output"
 update-manifest $(INSTALL_PATH)/boot.img-fb "Boot image for booting OMAP netinstall. Uses framebuffer output"

The vast majority of this is fairly straightforward. TARGET represents the targets called by make. There are tasks for creating a vmlinuz and initrd that must be included. The omap4 target then handles specialized handling for the omap4/netboot image.

omap4 requires a VFAT boot partition on the SD card with a proper filesystem and MBR. The contents of the filesystem are straightforward:

MLO - also known as x-loader, a first stage bootloader
u-boot.bin - u-boot binary, second stage bootloader, used to book the kernel
uImage - linux kernel with special uboot header (created with mkimage)
uInitrd - d-i ramdisk with special uboot header
boot.scr - special boot script for u-boot for commands to execute at startup.

MLO and u-boot.bin are copied in from x-loader-omap4-panda and u-boot-linaro-omap4-panda which are listed as build-deps in the control file for d-i. boot.scr is generated from a plain-text file:

fatload mmc 0:1 0x80000000 uImage
fatload mmc 0:1 0x81600000 uInitrd
setenv bootargs vram=32M mem=456M@0x80000000 mem=512M@0xA0000000 fixrtc quiet debian-installer/framebuffer=false console=ttyO2,115200n8
bootm 0x80000000 0x81600000

These are u-boot commands that simply load the uImage/uInitrd into RAM, set the command line, and then boot into it.

When porting the installer, it is mostly a task of putting your subarchitecture name in the right places, then adding the necessary logic in places to spit out an image that boots. This provides a sane base to start working on porting other bits of the installer. When d-i is uploaded to Launchpad, these files end up in http://ports.ubuntu.com/ubuntu-ports/dists/oneiric/main/installer-armel/current/images/

My next blog post will go a bit into udebs, and understanding how d-i does architecture detection, and introducing flash-kernel.

Let data be textarea->value and browser be (firefox or chrome).I want the user to be able to sign the data he entered in the textarea as I do not trust the website to store the data without modification.So far I found a couple of GnuPG/PGP based solutions:

• FireGPG, orphaned(not compatible with Firefox4+), there is fork on github
• gpgAuth
• GPGTools Mobile

Didn t test any of them yet, so I am asking you, dear Lazyweb: are these any good? Are there any more such tools? What about X.509 client certificates? Can I abuse them for signing in the browser too? So far I found login stuff only. Pointers highly appreciated.

5 comment(s) this blog is flattr enabled

I often get requests of persons who would like to contribute to Debian but who don t know where to start. Let s try to answer this question properly so that I can give out this URL the next time that I am asked. The Debian website has a page explaining how to help Debian. While it provides no less than 10 suggestions in a daunting text-only list, it s difficult to know what to do next once you picked up something that you could do. I will try to fix this by providing concrete information for each cases in upcoming articles but in the mean time I propose you another approach to start with. Before answering your question ( what can I do for Debian ), we need to know some information about you. What motivates you? You re a volunteer, you re not doing stuff for Debian because someone told you so. You must have some intrinsic motivation and the ultimate motivation is usually that you re enjoying what you re doing. So what are you enjoying and/or what are your motivations ?

• Is there something that you would like to learn? A new programming language? Packaging? Coding? System administration? A specific software?
• Do you want to interact with smart people?
• Do you like to help users?
• Do you like to fix software just so that it works for you?
• Do you like to build something remarkable and useful for millions of people?

On the opposite, make sure to know what you hate and what you want to stay far away from. Maybe you dislike a programming language so much that you don t want to be involved in a project where you would have to use it, etc. Write down the answers to the questions, you might need them later when you ll ask other Debian contributors how you can help. What are your skills? If you re not interested in learning new skills, then you must obviously select a task where your current skills are sufficient. Again make a list of your skills and in particular of skills that you d like to practice! Here s a non-exhaustive list of skills to consider:

• What languages are you fluent with? Are you confident to write documentation or translate documentation in those languages?
• Are you a programmer? If yes, which languages do you know?
• Can you diagnose problems? Can you debug problems with strace and/or gdb?
• Can you triage bugs?
• Do you know some Debian packaging?
• Are you an artist and/or a web designer?
• Do you know how to work with VCS (subversion, git, bzr, )?

How much time can you spend on Debian? This is the last important information that you need to communicate whenever you re asking someone else what you could do for Debian. There s no point giving you a big task if you can only spend 30 minutes every week. On the opposite if you can work on Debian full time during a week (because you re between 2 contracts or because you re in vacation), it s equally important to know. In general contributing to Debian requires time, you should be ready to spend at least several hours per week and possibly more at the start while you re learning everything. Find something to do At this point you have a generic idea of what you d like to do but you re still missing a concrete objective. Let s try to find one, we ll explore several ways to do this. Scratch your itch The best objectives are those that satisfy your own needs. Here are some examples:

• Did you notice a missing feature? Try to implement it.
• Have you been annoyed by a bug? Try to fix it.
• Did you lose too much time on something because there was no documentation? Write the missing documentation and submit it where appropriate.
• File bug reports for the things that you can t fix yourself. Even wishlist bug reports for new features.
• Do you use software that are not packaged for Debian? Create the package(s) and maintain it/them.
• Do you need a newer version of a package compared to what s in Debian unstable? Contact the maintainer and propose your help to update the package.
• Do you need a newer version of a package compared to what s in Debian stable? Contact the maintainer and propose your help to create a backport.

If you re a good Debian citizen, you have already filed bugs for issues that bugged you. Then you can browse http://bugs.debian.org/from:hertzog@debian.org to find out some ideas of stuff to do (obviously replace hertzog@debian.org by your own email address). Join a team If you don t have a specific itch to scratch, you might want to focus your work on a specific team. Head over to wiki.debian.org/Teams and browse the list of teams. You ll surely find one that works in an area that you like. If you select a packaging team, pick one that works on packages that you re actually using. Some of the teams have instructions for newcomers, follow them when that is the case. Otherwise join the mailing list and the IRC channel, and get a feel of how the team works. See if it suits you, you can follow several teams at the same time and pick the one that you prefer after a few days/weeks. Once you have lurked a bit, if you still don t know how you could help, then just ask on the mailing list. Include all the answers you have collected to the 3 questions on your motivations, your skills and your time available. Focus on a specific package You can concentrate your work on a specific package even if you re in a team, it s often a good idea. But I list it separately because not all packages are team maintained and you might want to help maintain a package where there s a single maintainer currently. I leave it up to you to find a way to select a package that interests you Then head over to the package tracking system: http://packages.qa.debian.org/dpkg (replace dpkg by the name of the package that interests you) Fill in the form in the bottom-left corner with your email and select opts in the drop-down list, then click go . You get a new form where you can select the information that you ll receive, I recommend you to keep everything except upload-binary and to validate the form. From now on, you get the same mails than the maintainer (and a bit more actually), and it s a good idea to inform the maintainer that you subscribed and that you re going to help a bit. Maybe he s willing to grant you commit rights immediately, maybe he will ask you to send patches for a start. The important thing is to create a good relationship. In any case (even if you did not get any answer from the maintainer), you should be free to help triage existing bugs and to help deal with the flow of incoming bugs (including forwarding bugs when appropriate). Help a Debian developer Paul Tagliamonte once blogged Hey, DDs, need help?. He offered his help to overworked Debian developers . His post was missing all the information required (cf motivations/skills/time) but the approach is a good one. The best way to start contributing is to work with existing Debian developers. Even if you just want to be sponsored for your own pet package, you should consider that mentoring is a burden for many Debian developers and that you re more likely to get a sponsor if you have an existing relationship with a Debian developer that you helped. Pick a developer that works in an area that is of interest, and offer him your help. To simplify things even further, I have created a wiki page where you can find out how you can help me and thus build a relationship with me. That s it for now, I hope this article will help you to start contributing. Don t forget to subscribe to my newsletter to not miss future articles for new contributors. Note that you can refer to this article with the following URL: http://raphaelhertzog.com/go/contributing/ (easier to type and remember).

13 comments Liked this article? Click here. My blog is Flattr-enabled.

Today I was made aware of bug #607839, and it makes me sad. I think its a great example about how trademarks and free software do not coexist well together. This bug started out as a question of the copyright license of a file, but quickly turned into a discussion about the GNOME [0] trademark, and the GNOME Foundation Trademark Usage Guidelines for Third Parties I believe that the GNOME project would be standing on solid legal ground if they wanted to actually try to aggressively protect their trademark. It is easy for me to find hundreds of places we use their mark in ways that are expressly forbidden by their trademark license. Here are some examples:

grep -i "gnome's" /var/lib/dpkg/available  # many package descriptions invalidly using a possessive
grep -i "gnome[^ ]" /var/lib/dpkg/availabe # returns nothing, trademark symbol is required

Probably to be in strict compliance, we would need to allow uppercase letters in package names. (I doubt this would happen, especially not for a reason like this). I am sure that a lawyer would tell us that the safest legal place for Debian would be to abandon all use of their marks. This would unfortunately be a herculean effort on Debian's part; we'd have to rename tons of packages, create and maintain tons of patches, create new logos, etc. And, I believe it would have the unfortunate side-effect of having a negative impact on both projects from a publicity standpoint. As with the firefox/iceweasel situation, the company trying to protect its trademark looks like a bully, and Debian looks like out-of-touch zealots. From the tone of the conversation in the bug report, I do not believe that this is something that the GNOME Foundation would force on us, and I certainly don't believe it would be an outcome they would hope for. They could give us written permission to use their marks in the ways we are already using them, however, lots of people would point at DFSG#8, and this would fail the 'tentacles of evil' test. (We are, of course, already under the grasp of these tentacles, as their current trademark license is subject to change) Better for us would be if they modified their trademark license agreement in order to allow our current usages of their mark. I feel that this quite unlikely to happen. It would certainly weaken their hold on the mark. Trying to apply the DFSG in these situations also doesn't work as well as it should. The DFSG is written with copyrights in mind and not patents and trademarks. This feels to me similar to situations we have with patent encumbered software which is under a perfectly free license. We can distribute the software without fear of a claim that we are violating copyrights, but the software is still not perfectly free since as I user, my rights to use and distribute the software might be severely limited from a legal standpoint. Also by Debian choosing to distribute software which has these kinds of potential problems, we might be putting a legal liability on members of the project and people that run mirrors for us. Of course, to add to the conflict I feel, Debian has its own trademarks, which it is poor at enforcing. On one hand, its bad for all the reasons that this situation is bad. On the other hand I sometimes wish that Debian did a better job of protecting its own trademark. It's clear from how many people show up in #debian expecting support for linuxmint or aptosid claiming "linux mint debian edition IS Debian, it says so on linuxmint.org" that we have allowed our mark to be diluted. NOTE: I became aware of this bug because of Zack CCing the FTP team. I serve on this team as an assistant, however, My comments here come from me personally without any discussion with other members of FTP team and should not be taken as any kind of offical word from the FTP team. [0] I suddently feel compelled to mention that The GNOME name is a registered trademark of GNOME Foundation in the United States or other countries.

While some folk look down on fakeraid (that is BIOS based RAID-until-OS-takes-over) solutions, I think they are pretty neat: they let a user get many of the benefits of dedicated controller cards at a fraction of the cost. The benefits include the usual ones for RAID more spindles to handle IO, tolerance of disk failures. And unlike pure LVM solutions, you can boot from a degraded RAID 1 / 5 / 10 set because the BIOS knows how. In some ways this is better than dedicated cards, because we have the software take over, so we can change the algorithms for IO dispatch all the way down to the individual devices However, these RAID volumes are in a pretty awkward spot for installers and bootloaders: inside a running Linux environment they look like software RAID which cannot be depended on for booting, but at boot time they look like hard disks which cannot be looked under the hood. I recently got a new desktop machine which has one of these motherboards, and fortuitously my old desktop I was replacing had the same size disks so I had 4 disks and the option of using a RAID setup. Apparently I m a sucker for punishment because I went for a RAID 10 (that is two RAID volumes made up of two-disk mirrors (the RAID 1 component), and then those two volumes are combined via striping (the RAID 0 component). This has the potential for pretty nice performance: in principle any read can come from one of 2 disks, and every 64KB (the stripe size) of linear data will switch to the other mirror set, giving a nice boost. Writes need to write to 2 disks always, but every 64KB worth of data will alternate mirror sets, also giving a boost. Sadly we (Ubuntu) aren t ready for this yet: there are two key bugs that make this layout almost impossible to install into. This blog post is for my exo-memory, I want to be able to figure out what I did next time around . Firstly parted_devices, a helper used by Ubiquity and debian-installer to determine which block devices are actually disk drives that one can partition and install onto, has a confused heuristic when dealing with dmraid it looks for devices which are not layered on other dmraid devices. This handily excludes partitions, but has the undesirable effect of excluding that striped device because it is layered on the two mirrored devices. Bug 560748 was filed about that, and I ve added a workaround to it basically disabling the filtering, so its not suitable as a long term fix, but it will let one select the RAID volume correctly. Secondly, grub2, which needs to figure out what the name at boot time of the RAID volume will be currently gets confused. I don t know enough to really explain and be correct in my explanation but I do have a fugly patch which worked for me. Bug 803658 tracks this defect. The basic approach I took was to say that dmraid devices should be an abstraction layer we don t peek under: if it claims to be a disk, well then its a disk. As grub does actually work that way - it talks to INT 13h the BIOS support for booting off of the RAID volume is entirely sufficient. Sadly neither bug is at the point where the patches can be rolled into Ubuntu itself, but the workaround should let folk get up and running. In both cases, build the package locally in the installer, install it, then after than run ubiquity and things should install. After the install, you will need to reapply the patch in the resulting installed environment, or things like update-grub will die on you! (huge thanks to cjwatson and ev for giving me some tips while I investigated this)

When traveling I usually try local brands of beer. This became some sort of hobby, and usually gives me a good possibility to get in contact with the locals. For DebConf11 so far I have been suggested to try Tuzlanski (Tuzla) and Nektar (Banja Luka). Any other suggestions what should be tried?

... is now working. It isn't finished yet; there are still some loose ends. But at least it works; if you combine this kernel with this debian-installer image, you'll be able to install to an NBD target. Known bugs:

• if you install your root device on an NBD device, the installer will attempt to install a boot loader to the local hard disk. Make sure to bring it into expert mode before the tasksel part to avoid overwriting your MBR -- or do what I did, and install to a (virtual) diskless computer.
• Also, due to an issue with the initscripts package, halt will make the kernel panic rather than power down the system but at that point it's safe to switch off the power, anyway.
• The installer currently fails to add a nbdroot= parameter to the kernel command line when installing root to NBD. When you're rebooting into the installed system, you should add something like nbdroot=<server>,<port> to your kernel command line, otherwise the installed system will not find the root device.

Other than that, it should work fine; testing is welcome (but note standard development code caveats apply).

I am involved in developing something coined as the FreedomBox. Explaining it to my mum the other day, she wisely asks if it, albeit clearly an exciting challenge we've picked, really is doable? Surely the World has gone sour, but is such radical change even possible? Annoying question! And clever For some years, tech media has tried predict when Linux have reached momentum for ordinary users. That current or next year was to become the Year of the Linux Desktop. Funny thing, seen in restrospect, is how "the year" kept being postponed, and when finally OLPC paved the way for the boom of Netbooks and arguably we got there, the World had moved on: Now Free Sofware is as common and as usable on desktops as commercially driven systems. It is taken for granted, not praised, and we look forward for the Next Big Challenge (as geeks) or Next Big Excitement (as users). Perhaps a similar fate is to be expected for FreedomBox: Initially when sparking our interest, and repeatedly since although we still today have nothing concrete to show - indeed even before we started hacking on it or knew the name of our dreams - our Prophet declared the Year of the FreedomBox. Not explicitly, but the cleverly phrased "right now." I am excited and proud to be working on FreedomBox, and foolishly hope it will be ready for worldwide consumption in a very recent "right now" - well aware that most likely it won't happen like that. Thing is, I don't really care how it happens, if only something does. This is due to the way we work: Hacking may appear from outside as larger projects, but really is juggling piles of small pieces for an eternal jigsaw puzzle, with each piece usable in multiple ways and across projects. I do not work only on FreedomBox, just as I did not work only on Sugar before that, or only with Debian as my platform: I work on Freedom-enabling technologies and ways to frame them for the Real World to use them with a vengeance. I sure hope you take the results for granted. That's true success!

I have bought a Samsung Galaxy S Android phone some time ago. One of the first thing that you try to do with such a phone is to switch to some alternate firmware (the coined term is ROM ), like CyanogenMod. Flashing a firmware is a whole new universe. You are told about a lot of proprietary tools, you need to download a lot of things from MegaUpload and friends and nobody really explains what s going on under the hood. I was a bit lost at first. I will not provide a step-by-step guide since I cannot afford to stay up-to-date. Such a guide is usually posted on some forum. It assumes you are running Windows and provides appropriate download links. You must follow it (and check that it is recent enough). I will just provide you some background to understand those guides.

Android firmwares There are several Android firmwares. Most of them are specific to a given phone (or its close variations). They can be sorted in three categories:

1. Official firmwares from the phone manufacturer include a kernel with appropriate drivers and, usually, a customized interface (HTC Sense for HTC, TouchWiz for Samsung). Some manufacturers will lag a lot behind Android schedule and you may not get all the new shiny features.
2. Modified firmwares are built from the official firmwares. They usually add a lot of hacks on top of the original firmware, remove some cruft but are still closely related to the official firmware.
3. Firmwares built from Android Open Source Project (the open source version of Android) do not share nothing with the official firmware except some bits related to the kernel (drivers, hacks).

Darky ROM Darky ROM is a modified firmware. It takes the original firmware from Samsung, adds a custom kernel (based on the source of the official kernel), adds a lot of hacks, some additional applications and removes some unwanted cruft. This is a very nice firmware which is quite snappy. This is a drop-in replacement for the default firmware: you will only get improvements. The latest version is 10.1 and there is a neat guide on how to install it.

CyanogenMod CyanogenMod is a firmware built from AOSP. It supports several devices and provide an unified firmware on all of them. Support for the Samsung Galaxy S is recent and is not present in the stable build. However, some RC build is now available. Be sure to read the FAQ (most notably the part saying that you need to install from Froyo 2.2 because the bootloader provided by Samsung with Gingerbread 2.3 is not compatible). You can also grab some information from the wiki. The major drawback of using this firmware instead of going for a modified firmware like Darky ROM is that the battery life may suck. I now need to charge my phone every day instead of once every two days. On the plus side, you get a very clean firmware with a lot of features and almost no cruft. For example, the notification bar can be customized and OpenVPN is available out of the box. Moreover, almost everything is open source (if you don t install Google apps). You can see this firmware as an open source distribution for your phone. There is no need to download random files linked from a random forum to make it work. There is no package manager, though. Despite the bad battery life, I prefer this firmware.

Prerequisites Before trying to flash a new firmware, there are some tools that you need to have. Note that you don t need root access to flash a new firmware.

Download & recovery mode Before trying to install any firmware, you must check that your phone can access download and recovery modes. The download mode allows to flash any new firmware, even if the current firmware does not boot at all. As long as this mode is available, there is very little chance that you brick your phone. To access those modes, you need to use the three-button combo . Here are the steps (your phone should not be plugged):

1. Power off the phone (wait for the phone to vibrate).
2. Hold the Volume down button for download mode. Hold the Volume up button for recovery mode.
3. Hold the Home button.
4. Hold the Power button. The phone will start.
5. Once you have the Samsung logo displayed on the screen, release the Power button.
6. Only release the two other buttons when you enter the recovery mode or download mode. The recovery mode displays a menu. The download mode displays a big yellow Android logo.

If after ten tries, you are unable to access those modes, maybe they have been disabled. Keep reading, there is a solution.

Heimdall Heimdall is a tool to flash a new firmware to your phone. This is not a mandatory tool but you will have to use it if something goes wrong. Most guides will tell you to use Odin. It is a leaked Samsung tool running on Windows only. You can use Heimdall instead.

$ git clone https://github.com/Benjamin-Dobell/Heimdall.git
Cloning into Heimdall...
remote: Counting objects: 423, done.
remote: Compressing objects: 100% (271/271), done.
remote: Total 423 (delta 220), reused 332 (delta 138)
Receiving objects: 100% (423/423), 5.91 MiB   962 KiB/s, done.
Resolving deltas: 100% (220/220), done.
$ cd Heimdall/heimdall
$ sudo apt-get install libusb-1.0-0-dev
$ ./configure && make
[...]
$ sudo cp 60-heimdall-galaxy-s.rules /etc/udev/rules.d
$ sudo /etc/init.d/udev reload

You need to use this tool while in download mode. You can test it, while in download mode, with:

$ ./heimdall print-pit

This will display the partition table of your phone. This is enlightening to know what it contains:

• IBL+PBL is the boot loader, be extra careful with it;
• PIT is the partition table;
• EFS contains some important data like IEMEI, be extra careful with it;
• SBL is the second boot loader;
• SBL2 is the backup of the second boot loader;
• PARAM contains some images used as various stage of the boot process;
• KERNEL is the kernel;
• RECOVERY is the backup of the kernel;
• FACTORYFS contains /system file system which is some kind of root file system;
• DBDATA contains /data file system which is the database to store application parameters;
• CACHE contains /cache file system which contains the Dalvik cache;
• MODEM contains the firmware for the modem.

As we will see, Heimdall is able to flash any of those partitions with flash subcommand.

adb adb stands for Android Debug Bridge. Among other things, this tool allow you to drive your phone through the USB connection. You can get a shell, reboot into download or recovery mode, push files to your Android device, etc. To get this tool, you need to download the full Android SDK.

$ tar zxvf ~/download/android-sdk_r11-linux_x86.tgz
$ cd android-sdk-linux_x86
$ tools/android update sdk
[select Android SDK Platform-tools]
$ platform-tools/adb shell
# uname -a
Linux localhost 2.6.35.7-I9000XWJVH-CL184813 #Dark_Core_2.4 PREEMPT Mon May 16 15:02:38 EEST 2011 armv7l GNU/Linux

The important commands are:

• shell to get a shell on the device;
• reboot recovery to reboot in recovery mode;
• reboot download to reboot in download mode;
• push to put a file on the phone;
• pull to retrieve a file;
• install to install an application from its APK;
• logcat to see the logs and what happens, very useful!

Restore download mode If your phone is missing download mode, you can recover it with the help of adb and heimdall. You need to grab some official firmware and extract boot.bin and Sbl.bin from it.

$ adb reboot download
$ heimdall flash --primary-boot ~/download/boot.bin --secondary-boot ~/download/Sbl.bin

Be extra careful here. You may brick your phone. Double check that you grab the correct files and that they are really for your phone (there exists some variants). It worked for me.

Installing a new firmware While you can switch from one firmware to another without losing your data, it is better to wipe your system and your data before switching firmware. This means that you will lose all your applications and the data inside them (list of contacts, SMS, preferences). However, nothing will ever modify your data in the internal SD card or in the external SD card (which are both inside the phone). They usually contains your music and your photos. There are two ways to install a firmware:

1. via Heimdall, or
2. using the recovery.

Heimdall When a guide tells you to use Odin, you can use Heimdall instead. Odin takes a TAR file. You should untar it and uses its content with Heimdall. For example:

$ ./heimdall flash --repartition --pit s1_odin_20100512.pit \
>                  --factoryfs factoryfs.rfs \
>                  --cache cache.rfs --dbdata dbdata.rfs \
>                  --param param.lfs --modem modem.bin \
>                  --kernel zImage --secondary-boot Sbl.bin 
Claiming interface again...   Success
Setting up interface...   Success
Beginning session...
Handshaking with Loke...   Success
Uploading PIT
PIT upload successful
Uploading SBL
100%
SBL upload successful
Uploading KERNEL
100%
KERNEL upload successful
Uploading MODEM
100%
MODEM upload successful
Uploading PARAM
100%
PARAM upload successful
Uploading FACTORYFS
100%
FACTORYFS upload successful
Uploading DBDATAFS
100%
DBDATAFS upload successful
Uploading CACHE
100%
CACHE upload successful
Ending session...
Rebooting device...

You may be interested to update your modem using Heimdall. This is usually independent of the firmware you want to use. Also, no need to provide a PIT file if you don t want to repartition. Heimdall can read the current one and flash only one of the partitions.

Recovery mode The base recovery shipped with the Samsung Galaxy S does not have a lot of options. Moreover, it may be locked and will accept to install only signed firmwares. Most firmwares come with ClockWorkMod, a more advanced recovery. Once you get it, you will be able to use any ZIP file to install a new firmware. My advice is to try to get ClockWorkMod as soon as possible. If you are root, you can install ROM Manager that will install it for you. If you are not root, most custom firmwares will provide ClockWorkMod. Therefore, you can just install a custom firmware with Heimdall. There exists special firmwares just for this, like EZbase. If you want to install CyanogenMod, be sure to download a Froyo firmware, not a Gingerbread one. Keep in mind that the recovery is located inside the kernel. It is not self-contained in the bootloader (unlike download mode). If you don t want to flash a whole firmware (and lose everything on your phone) just to get the recovery, you can just flash the kernel. The ClockWorkMod recovery comes with an utility to backup your whole phone. This way, you can easily restore your previous firmware if you don t like the current one. This is what is called a Nandroid backup.

Lagfix A lot of guides will tell you to enable or disable lagfix. Samsung shipped the Galaxy S with some crappy filesystem called RFS (which is some kind of enhanced FAT). It is slow and seems responsible for some lags that may happen on the interface. A lagfix is a way to replace this filesystem by something better, like EXT4 or YAFFS2. The process is pretty straighforward and consist to backup all the partition to the SD card, reformat it with another filesystem and restore the backup. The lagfix may be applied automatically with some firmware (like CyanogenMod) or you may request it in the recovery mode (most modified firmwares work this way). In this case, it will be applied automatically on the next boot. You should disable it before installing another firmware with the recovery mode (no need if you install it with Heimdall).

Applications You can find many free (as in free speech) applications for Android. Unfortunately, Android Market does not allow to use this criteria as a filter. F-Droid is an alternate market featuring only FOSS applications. Here is a short selection of free applications:

• ADW.Launcher is a home replacement app. It is very configurable and themable.
• K-9 Mail is a very nice email client.
• Xabber is a XMPP/Jabber client.
• ConnectBot is an SSH client (as well as a terminal).
• DroidWall is an application firewall (using Netfilter). It lets you select which application can access to Internet depending on your connection (wifi or 3G).
• Barcode Scanner decodes 1D/2D barcode including QR codes.
• OsmAnd is a replacement for Google Maps using OpenStreetMap. The free version on Android Market is out-dated. Buy the plus version or download it throught F-Droid.

Another must-have application is Titanium Backup which allows to backup any application with data. Unfortunately, this is not a free (as in free speech) application.

I have bought a Samsung Galaxy S Android phone some time ago. One of the first thing that you try to do with such a phone is to switch to some alternate firmware (the coined term is ROM ), like CyanogenMod. Flashing a firmware is a whole new universe. You are told about a lot of proprietary tools, you need to download a lot of things from MegaUpload and friends and nobody really explains what s going on under the hood. I was a bit lost at first. I will not provide a step-by-step guide since I cannot afford to stay up-to-date. Such a guide is usually posted on some forum. It assumes you are running Windows and provides appropriate download links. You must follow it (and check that it is recent enough). I will just provide you some background to understand those guides.

Android firmwares There are several Android firmwares. Most of them are specific to a given phone (or its close variations). They can be sorted in three categories:

1. Official firmwares from the phone manufacturer include a kernel with appropriate drivers and, usually, a customized interface (HTC Sense for HTC, TouchWiz for Samsung). Some manufacturers will lag a lot behind Android schedule and you may not get all the new shiny features.
2. Modified firmwares are built from the official firmwares. They usually add a lot of hacks on top of the original firmware, remove some cruft but are still closely related to the official firmware.
3. Firmwares built from Android Open Source Project (the open source version of Android) do not share nothing with the official firmware except some bits related to the kernel (drivers, hacks).

Darky ROM Darky ROM is a modified firmware. It takes the original firmware from Samsung, adds a custom kernel (based on the source of the official kernel), adds a lot of hacks, some additional applications and removes some unwanted cruft. This is a very nice firmware which is quite snappy. This is a drop-in replacement for the default firmware: you will only get improvements. The latest version is 10.1 and there is a neat guide on how to install it.

CyanogenMod CyanogenMod is a firmware built from AOSP. It supports several devices and provide an unified firmware on all of them. Support for the Samsung Galaxy S is recent and is not present in the stable build. However, some RC build is now available. Be sure to read the FAQ (most notably the part saying that you need to install from Froyo 2.2 because the bootloader provided by Samsung with Gingerbread 2.3 is not compatible). You can also grab some information from the wiki. The major drawback of using this firmware instead of going for a modified firmware like Darky ROM is that the battery life may suck. I now need to charge my phone every day instead of once every two days. On the plus side, you get a very clean firmware with a lot of features and almost no cruft. For example, the notification bar can be customized and OpenVPN is available out of the box. Moreover, almost everything is open source (if you don t install Google apps). You can see this firmware as an open source distribution for your phone. There is no need to download random files linked from a random forum to make it work. There is no package manager, though. Despite the bad battery life, I prefer this firmware.

Prerequisites Before trying to flash a new firmware, there are some tools that you need to have. Note that you don t need root access to flash a new firmware.

Download & recovery mode Before trying to install any firmware, you must check that your phone can access download and recovery modes. The download mode allows to flash any new firmware, even if the current firmware does not boot at all. As long as this mode is available, there is very little chance that you brick your phone. To access those modes, you need to use the three-button combo . Here are the steps (your phone should not be plugged):

1. Power off the phone (wait for the phone to vibrate).
2. Hold the Volume down button for download mode. Hold the Volume up button for recovery mode.
3. Hold the Home button.
4. Hold the Power button. The phone will start.
5. Once you have the Samsung logo displayed on the screen, release the Power button.
6. Only release the two other buttons when you enter the recovery mode or download mode. The recovery mode displays a menu. The download mode displays a big yellow Android logo.

If after ten tries, you are unable to access those modes, maybe they have been disabled. Keep reading, there is a solution.

Heimdall Heimdall is a tool to flash a new firmware to your phone. This is not a mandatory tool but you will have to use it if something goes wrong. Most guides will tell you to use Odin. It is a leaked Samsung tool running on Windows only. You can use Heimdall instead.

$ git clone https://github.com/Benjamin-Dobell/Heimdall.git
Cloning into Heimdall...
remote: Counting objects: 423, done.
remote: Compressing objects: 100% (271/271), done.
remote: Total 423 (delta 220), reused 332 (delta 138)
Receiving objects: 100% (423/423), 5.91 MiB   962 KiB/s, done.
Resolving deltas: 100% (220/220), done.
$ cd Heimdall/heimdall
$ sudo apt-get install libusb-1.0-0-dev
$ ./configure && make
[...]
$ sudo cp 60-heimdall-galaxy-s.rules /etc/udev/rules.d
$ sudo /etc/init.d/udev reload

You need to use this tool while in download mode. You can test it, while in download mode, with:

$ ./heimdall print-pit

This will display the partition table of your phone. This is enlightening to know what it contains:

• IBL+PBL is the boot loader, be extra careful with it;
• PIT is the partition table;
• EFS contains some important data like IEMEI, be extra careful with it;
• SBL is the second boot loader;
• SBL2 is the backup of the second boot loader;
• PARAM contains some images used as various stage of the boot process;
• KERNEL is the kernel;
• RECOVERY is the backup of the kernel;
• FACTORYFS contains /system file system which is some kind of root file system;
• DBDATA contains /data file system which is the database to store application parameters;
• CACHE contains /cache file system which contains the Dalvik cache;
• MODEM contains the firmware for the modem.

As we will see, Heimdall is able to flash any of those partitions with flash subcommand.

adb adb stands for Android Debug Bridge. Among other things, this tool allow you to drive your phone through the USB connection. You can get a shell, reboot into download or recovery mode, push files to your Android device, etc. To get this tool, you need to download the full Android SDK.

$ tar zxvf ~/download/android-sdk_r11-linux_x86.tgz
$ cd android-sdk-linux_x86
$ tools/android update sdk
[select Android SDK Platform-tools]
$ platform-tools/adb shell
# uname -a
Linux localhost 2.6.35.7-I9000XWJVH-CL184813 #Dark_Core_2.4 PREEMPT Mon May 16 15:02:38 EEST 2011 armv7l GNU/Linux

The important commands are:

• shell to get a shell on the device;
• reboot recovery to reboot in recovery mode;
• reboot download to reboot in download mode;
• push to put a file on the phone;
• pull to retrieve a file;
• install to install an application from its APK;
• logcat to see the logs and what happens, very useful!

Restore download mode If your phone is missing download mode, you can recover it with the help of adb and heimdall. You need to grab some official firmware and extract boot.bin and Sbl.bin from it.

$ adb reboot download
$ heimdall flash --primary-boot ~/download/boot.bin --secondary-boot ~/download/Sbl.bin

Be extra careful here. You may brick your phone. Double check that you grab the correct files and that they are really for your phone (there exists some variants). It worked for me.

Installing a new firmware While you can switch from one firmware to another without losing your data, it is better to wipe your system and your data before switching firmware. This means that you will lose all your applications and the data inside them (list of contacts, SMS, preferences). However, nothing will ever modify your data in the internal SD card or in the external SD card (which are both inside the phone). They usually contains your music and your photos. There are two ways to install a firmware:

1. via Heimdall, or
2. using the recovery.

Heimdall When a guide tells you to use Odin, you can use Heimdall instead. Odin takes a TAR file. You should untar it and uses its content with Heimdall. For example:

$ ./heimdall flash --repartition --pit s1_odin_20100512.pit \
>                  --factoryfs factoryfs.rfs \
>                  --cache cache.rfs --dbdata dbdata.rfs \
>                  --param param.lfs --modem modem.bin \
>                  --kernel zImage --secondary-boot Sbl.bin 
Claiming interface again...   Success
Setting up interface...   Success
Beginning session...
Handshaking with Loke...   Success
Uploading PIT
PIT upload successful
Uploading SBL
100%
SBL upload successful
Uploading KERNEL
100%
KERNEL upload successful
Uploading MODEM
100%
MODEM upload successful
Uploading PARAM
100%
PARAM upload successful
Uploading FACTORYFS
100%
FACTORYFS upload successful
Uploading DBDATAFS
100%
DBDATAFS upload successful
Uploading CACHE
100%
CACHE upload successful
Ending session...
Rebooting device...

You may be interested to update your modem using Heimdall. This is usually independent of the firmware you want to use. Also, no need to provide a PIT file if you don t want to repartition. Heimdall can read the current one and flash only one of the partitions.

Recovery mode The base recovery shipped with the Samsung Galaxy S does not have a lot of options. Moreover, it may be locked and will accept to install only signed firmwares. Most firmwares come with ClockWorkMod, a more advanced recovery. Once you get it, you will be able to use any ZIP file to install a new firmware. My advice is to try to get ClockWorkMod as soon as possible. If you are root, you can install ROM Manager that will install it for you. If you are not root, most custom firmwares will provide ClockWorkMod. Therefore, you can just install a custom firmware with Heimdall. There exists special firmwares just for this, like EZbase. If you want to install CyanogenMod, be sure to download a Froyo firmware, not a Gingerbread one. Keep in mind that the recovery is located inside the kernel. It is not self-contained in the bootloader (unlike download mode). If you don t want to flash a whole firmware (and lose everything on your phone) just to get the recovery, you can just flash the kernel. The ClockWorkMod recovery comes with an utility to backup your whole phone. This way, you can easily restore your previous firmware if you don t like the current one. This is what is called a Nandroid backup.

Lagfix A lot of guides will tell you to enable or disable lagfix. Samsung shipped the Galaxy S with some crappy filesystem called RFS (which is some kind of enhanced FAT). It is slow and seems responsible for some lags that may happen on the interface. A lagfix is a way to replace this filesystem by something better, like EXT4 or YAFFS2. The process is pretty straighforward and consist to backup all the partition to the SD card, reformat it with another filesystem and restore the backup. The lagfix may be applied automatically with some firmware (like CyanogenMod) or you may request it in the recovery mode (most modified firmwares work this way). In this case, it will be applied automatically on the next boot. You should disable it before installing another firmware with the recovery mode (no need if you install it with Heimdall).

Applications You can find many free (as in free speech) applications for Android. Unfortunately, Android Market does not allow to use this criteria as a filter. F-Droid is an alternate market featuring only FOSS applications. Here is a short selection of free applications:

• ADW.Launcher is a home replacement app. It is very configurable and themable.
• K-9 Mail is a very nice email client.
• Xabber is a XMPP/Jabber client.
• ConnectBot is an SSH client (as well as a terminal).
• DroidWall is an application firewall (using Netfilter). It lets you select which application can access to Internet depending on your connection (wifi or 3G).
• Barcode Scanner decodes 1D/2D barcode including QR codes.
• OsmAnd is a replacement for Google Maps using OpenStreetMap. The free version on Android Market is out-dated. Buy the plus version or download it throught F-Droid.

Another must-have application is Titanium Backup which allows to backup any application with data. Unfortunately, this is not a free (as in free speech) application.

Periodically Free Software people from other countries visit Melbourne on business trips. Usually when someone is sent any distance on a business trip (IE to Australia from anywhere other than NZ) they will stay in a good hotel (4 star or better), this generally means that they have a pool in their hotel. 5 star hotels and the newer serviced apartment hotels tend to have really good pools (1/4 olympic size isn t uncommon). Hotel pools are very under-utilised, their main purpose AFAIK is to boost the hotel star rating my experience is that it s not common to meet other people in a hotel pool. While food and drinks are often banned in the pool area my observation is that the only rule which matters is no glass . So hotel pools are almost ideal for pool parties, you just need to drink from cans or from drinks in plastic bottles that are poured into plastic cups. If any Free Software person finds themself staying in a hotel in Melbourne Australia with not much to do in the evenings or weekends then one option is to call for a pool party. I ve asked on a local mailing list and it seems that there is enough interest for a small party, the local mailing lists can be used to arrange a party. Also one thing to note is that some hotels have outdoor pools, while Australia is generally a warm place the southern parts of Australia (such as Melbourne) get quite cold in winter, an outdoor pool is not going to be fun for the colder half of the year. So getting a hotel with an indoor pool is very important during the April-September period.

I've taken a tiny step towards getting together a prototype toolset for developing an initial development iteration of the NAS functionality of the FreedomBox. Here's what I am trying to do:

• create a disk image for a system that provides simplistic NAS functionality, with no configuration, including no authentication: just serve whatever disks are available over SMB
• use a TDD approach to developing it

In essence, this means I want to specify the functionality of the system I am building using automated tests. As soon as those tests pass, I will write more tests, and make them pass. When I can't think of new relevant tests, I know I'm done. I'm building a simple tool, systest, that will execute the tests. It is built on top of Python's unittest library, because that makes my implementation simple. I've now developed systest far enough that I can almost automatically do this:

• build a new image, using vmdebootstrap, plus a customization script
• start the image with KVM controlled by libvirt
• run the tests using systest

(The command line would be something like this: ./build-fbxnas && virsh start fbxnas && sleep 30 && ~/systest/systest --target=192.168.122.89 --user=tomjon ~/systest/tests-debian-base-install.py --user-ssh-private-key=tomjon.secret
--log=foo.log --user-password=password1 where build-fbxnas is a wrapper script around vmdebootstrap. But I didn't actually run that command, and it's lacking the virt-install command to actually set up the virtual machine the first time.) systest contains very little code, but to get that far, I had to shave some yaks, and that made writing it rather an unpleasant experience. The tests so far really simple, and don't test NAS stuff at all, just that the basic Debian install works: only the ssh port is open, and basic networking and such things work. However, now that I've gotten this much to work, I can start writing NAS tests. (smbclient may be my next yak. This would be a lot easier if I had ever used Samba for anything real...) systest and vmdebootstrap are on Gitorious. No .debs yet, this stuff is much too raw, but you can run them from the checkouts, no install necessary. PS. If anyone knows how to query KVM or libvirt for the IP address assigned by their internal DHCP server to the virtual machine guest, automatically, without asking the actual virtual machine, please tell.

I m really happy to be able to attend DebConf11. It will be my third DebConf. I hope it will be as productive and enjoyable as past editions I ll arrive on July 20th and stay until the end of DebConf. See you all there soon!

Hello The last version (1.247, available in Debian unstable as libconfig-model-perl) of Config::Model provides a brand new model for /etc/LCDd.conf. With this model, you can edit or validate your lcdproc configuration. To edit this configuration the first time, you will have to run:

config-edit -application lcdproc -force

The -force option is required to load the file the first time. (because the default LCDd.conf file contains configuration options for all drivers, even though only one them if usefull). You will get this screen:
In this screen, you can setup the driver for your adress with a right click on Driver parameter. Once youv e selected the driver and clicked on store , the driver will be listed in the left part of the window so you can tune the parameters of this specific driver: In this screen, I ve turned the backlight off and added a small note to remember why. This small note will be writted in a comment in /etc/LCDd.conf. Once all required modifications are done, it s time to save the configuration file using the menu entry File -> save Now, let s see the resulting file (which is a bit long due to the amount of comments in the original LCDd.conf file):

## This file was written by Config::Model
## You may modify the content of this file. Configuration
## modifications will be preserved. Modifications in
## comments may be mangled.
# LCDd.conf -- configuration file for the LCDproc server daemon LCDd
#
# This file contains the configuration for the LCDd server.
#
# The format is ini-file-like. It is divided into sections that start at
# markers that look like [section]. Comments are all line-based comments,
# and are lines that start with '
#
# The server has a 'central' section named [server]. For the menu there is
# a section called [menu]. Further each driver has a section which
# defines how the driver acts.
#
# The drivers are activated by specifiying them in a driver= line in the
# server section, like:
#
#   Driver=curses
#
# This tells LCDd to use the curses driver.
# The first driver that is loaded and is capable of output defines the
# size of the display. The default driver to use is curses.
# If the driver is specified using the -d  command line option,
# the Driver= options in the config file are ignored.
#
# The drivers read their own options from the respective sections.
[server]
# Where can we find the driver modules ?
# IMPORTANT: Make sure to change this setting to reflect your
#            specific setup! Otherwise LCDd won't be able to find
#            the driver modules and will thus not be able to
#            function properly.
# NOTE: Always place a slash as last character !
DriverPath=/usr/lib/lcdproc/
# Tells the server to load the given drivers. Multiple lines can be given.
# The name of the driver is case sensitive and determines the section
# where to look for further configuration options of the specific driver
# as well as the name of the dynamic driver module to load at runtime.
# The latter one can be changed by giving af File= directive in the
# driver specific section.
#
# The following drivers are supported:
#   bayrad, CFontz, CFontz633, CFontzPacket, curses, CwLnx, ea65,
#   EyeboxOne, g15, glcdlib, glk, hd44780, icp_a106, imon, imonlcd,
#   IOWarrior, irman, joy, lb216, lcdm001, lcterm, lirc, lis, MD8800,
#   mdm166a, ms6931, mtc_s16209x, MtxOrb, mx5000, NoritakeVFD, picolcd,
#   pyramid, sed1330, sed1520, serialPOS, serialVFD, shuttleVFD, sli,
#   stv5730, svga, t6963, text, tyan, ula200, xosd
Driver=imonlcd
# Tells the driver to bind to the given interface
Bind=127.0.0.1
# Listen on this specified port; defaults to 13666.
Port=13666
ReportLevel=3
# Sets the reporting level; defaults to 2 (warnings and errors only).
# ReportLevel=3
# Should we report to syslog instead of stderr ? [default: no; legal: yes, no]
# ReportToSyslog=yes
# User to run as.  LCDd will drop its root priviledges, if any,
# and run as this user instead.
User=nobody
Foreground=no
Hello=    Hello
Hello=    LCDproc!
GoodBye=    GoodBye
GoodBye=    LCDproc!
# The server will stay in the foreground if set to true.
# Foreground=no
# Hello message: each entry represents a display line; default: builtin
# Hello="  Welcome to"
# Hello="   LCDproc!"
# GoodBye message: each entry represents a display line; default: builtin
# GoodBye="Thanks for using"
# GoodBye="   LCDproc!"
# Sets the default time in seconds to displays a screen.
WaitTime=5
# If yes, the the serverscreen will be rotated as a usual info screen. If no,
# it will be a background screen, only visible when no other screens are
# active. The special value 'blank' is similar to no, but only a blank screen
# is displayed. [default: on; legal: on, off, blank]
# ServerScreen=no
# Set master backlight setting. If set to 'open' a client may control the
# backlight for its own screens (only). [default: open; legal: off, open, on]
# Backlight=open
# Set master heartbeat setting. If set to 'open' a client may control the
# heartbeat for its own screens (only). [default: open; legal: off, open, on]
# Heartbeat=open
# set title scrolling speed [default: 10; legal: 0-10]
# TitleSpeed=10
# The "...Key=" lines define what the server does with keypresses that
# don't go to any client.
# These are the defaults:
ToggleRotateKey=Enter
PrevScreenKey=Left
NextScreenKey=Right
ScrollUpKey=Up
ScrollDownKey=Down
# ScrollUpKey=Up
# ScrollDownKey=Down
# If you have only 4 keys, you can choose to use this:
# ToggleRotateKey=Enter
# PrevScreenKey=Up
# NextScreenKey=Down
# If you have only 3 keys, you can choose to use this:
# ToggleRotateKey=Enter
# PrevScreenKey=Up
[menu]
# You can configure what keys the menu should use. Note that the MenuKey
# will be reserved exclusively, the others work in shared mode.
# The following works excellent with 4 keys or more.
MenuKey=Escape
EnterKey=Enter
UpKey=Up
DownKey=Down
LeftKey=Left
RightKey=Right
[imonlcd]
Protocol=0
OnExit=2
Device=/dev/lcd0
# My fishes dont like the glow
Backlight=off
DiscMode=0

You will notice a few things:

• Comments from the original file in section [server] and [menu] were kept
• Unused drivers sections are gone
• The fishy note about backlight was saved as a comment above the backlight parameter

If you need to update the file, you will have to run:

sudo config-edit -application lcdproc

The -force option is no longer needed. If you don t like to click, you can run:

config-edit -application lcdproc -ui none -save

For instance:

config-edit -application lcdproc -ui none -save 'server driver=imonlcd - imonlcd Device="/dev/lcd1" '

Further reading If you want to know more about lcdproc model, you can view:

• LCDd model and all models for the drivers
• LCDd documentation and the drivers documentationgenerated from these models

Under the hood I was too lazy to write myself the models of all drivers (about 3500 lines for them all). All models were generated from the LCDd.conf template file provided by lcdproc project. This blog explains how this was done. But the result is far from perfect. Event though LCDd.conf comments give a lot of semantic information, parameter types could be better adjusted and some of the examples provided in LCDd.conf confuse the model generator.
So the current comments are good, but not enough. In the future, we could decide on better structural conventions for the comments that would give users the information they need and give a model generator enough information to generate better models. This will hopefully be the topic of yet another blog All the best

Two weeks ago, I updated the packages containing the BioPerl modules bioperl and bioperl-run, which allowed to resume the work done on GBrowse, one of the genome browsers available in Debian. As many Perl modules, BioPerl has extensive regression tests. Some of them need access to external services, and are disabled by default as the Internet is not available in our build farm. Nevertheless, they can be triggered through the environment variable DEB_MAINTAINER_MODE when building the package locally. Bioperl successfully passes all off-line tests, and a part of the on-line tests is already corrected its development branch. In contrary, BioPerl-Run fail the tests for Bowtie, DBA, EMBOSS, Genewise, Hmmer, PAML, SABlastPlus, T-Coffee and gmap-run. In some cases, like EMBOSS, it is because a command has been renamed in Debian. In other cases, in particular DBA et Genewise, it is much more difficult to figure out on which side is the problem. Regression tests are essential to determine if a package works or not on ports others than the one used by the packager (amd64 in my case). Even simple ones can be useful. In the case of T-Coffee, that has a rudimentary test that I have activated recently, it showed that the packages distributed on armel are not working at all. Running regression tests when building packages have advantages, in particular to have the results published for each port automatically as part of the build logs. But is also cause problems. First, a package would need to build-depend on every software it can interact with, in order to test it comprehensively. In the example of bioperl-run, it makes it impossible to build on armel as long as t-coffee is broken there. Second, this approach does not help the user to test similarly a program installed on his computer. Maybe the Debian Enhancement Proposal 8, to test binary packages with material provided by their source packages, will solve these two problems.

LDAP's "Country" entry reads "France" The correlation among quiet times for a blog and quiet times in its owner's life is often negative. My past two months are no exception. Beside my regular Debian activities, I've used a good chunk of my day time to (try) settle down and stop wandering around Europe, one temporary academic position after another. In the end, it has turned out very well. Starting September 1st, I'll be joinining the PPS laboratory as a Ma tre de Conf rences, a tenured academic position roughly equivalent to US' Associate Professor (at least according to Wikipedia; see the high quality references that tenured people resort to use?). More specifically, I'll continue to work in the context of IRILL (Research and Innovation on Free Software), an exciting new entity that I've helped bootstraping over the past couple of years. Thanks to IRILL and to his director Roberto Di Cosmo, I've been able not only to pursue quite some research on FOSS distros, but I've also been supported (rather than discriminated) in my Debian activities, including acting as DPL around the world. Last but not least, within IRILL I have also been able to offer support to several Free Software communities such as: Debian itself (hosting Sprints and sponsoring DebConf hardware), GNU, OSI, OCaml, Darcs, and many others. I realize how lucky I've been ending up in a place like IRILL and I'm glad to know I'll be around here for quite some time. The main change I'll expect to happen from September on is that I'll get back to do some teaching, trading-off time from research activities. I've been teaching only on and off since I've moved to France in 2008 and I miss that quite a bit. It looks like that next year I'll be teaching various classes on FOSS-related (or at the very least UNIX-y) topics. Time looks scarce, but who knows, maybe I'll also find a way to mix-in the nice packaging tutorial that Lucas has recently put together. Wish me luck!