I like so much the git feature branch workflow, that in the early days of development on Python with Mercurial I created some named branches; well, that is something you should not do.

In Mercurial, the changeset contains the branch name, so you cannot develop on a separated (named) branch and then merge on default and hope that branch goes away, because it will stay.

What do I do now? Python Mercurial repository is quite big (around 200Megs) so I wanted to avoid to re-check it out. Thanks to the help of the folks on #mercurial (on freenode IRC network) I found my solution: strip the branch!

Please note that strip is dangerous. Use it only as last resort, and mind you can lose data with it. That said, it's a very powerful tool :) My main aim was to remove completely those named branches, leave no traces, and lose the changes I made on them. Another important aspect is that I didn't merged those branches on default.

So, how to get rid of a named branch:

$ hg strip "branch($ BRANCHNAME )"

and re-iterate for all the branches you have, that's it. Now, to be completely sure they were removed and no spurious changes are in the repository, you can:

$ hg pull -u
$ hg outgoing

and if it says "no changes found" you're sure that those branches are really gone.

Linux will be 20 years old this year. I've been drafting an outline for an article of my memories of the early days, but that's going to take a lot of writing before it is ready. Just in case I don't finish the article, I'll point at Linux Anecdotes, the transcript of a talk I gave in 1998, when Linux was only seven years old. It might still be entertaining to some. If nothing else, you can laugh at my naivete about the future, now our past.

Thanks to the tremendous growth of social applications over the past five years, we have our pick of services for collecting, saving and sharing our experiences online. We each have collections of photos, contacts, messages and more, spread across multiple popular services like Twitter, Facebook, LinkedIn, as well as many less popular services which address particular needs or preferences. We re also producing a wealth of exhaust data through our browsing history, mobile sensors, transactions and other activity streams that we rarely if ever examine directly. This ecosystem is becoming so complex that it s easy to lose track of what you ve created, shared or seen. We need new tools to manage this complexity, to make the most of the wealth of information and connections available to us through various services. John Battelle calls these metaservices , and points to growth in the number of connections between the services we use. I expect that this next age of information tools will center around data rather than services. Data is a common denominator for these online experiences, a bridge across disparate services, technologies, social graphs, and life cycles. Personal data, in particular, has this property: the only thing that links together your photos on Flickr, Facebook, Picasa and Twitpic is you. So where s your data center ? I don t anticipate the emergence of a single service where you do everything. There will continue to be innovation in the form of new and specialized services which meet a particular need very well. There won t be a single service which is everything to everybody. Instead, I foresee us wanting to track, save, use and control all of our stuff across the web. That s why my new colleagues and I are working to make that possible. There s open source code available on github, a vibrant IRC channel (#lockerproject on Freeenode), and lots more I d like to write about it. But it s time to get back to work for now

Yesterday I begun using Debian GNU/kFreeBSD squeeze in thorin, my main workstation. During the last few weeks I had to work through some of the limitations that were holding me back, such automated driver load and FUSE. I was lucky enough that other people filled the missing pieces I wanted, such as NFS client support and a GRUB bugfix that broke booting from Mirrored pools. I have to say that I m very satisfied. Barring a pair of small nuissances here and there, the result is quite impressive:

• System responsiveness has noticeably improved, specially on disk-intensive operations such as loading my browser. I believe lightweight ZFS compression (the default algorithm is fast enough that it saves time rather than spending it) has a big influence on this.
• Partition management became a lot simpler. Instead of using a fixed size like legacy file systems, with ZFS the storage is shared. This allows me to add, remove or resize file systems or swap volumes from my running system without having to worry about where they will be allocated (I have two pools, one that is mirrored and one that isn t, and I can add or remove things from either. That s as far as I m concerned with allocation).
• I no longer have to go through long wait periods when adding a new physical disk, or replacing a broken one. I have a lot of old used disks at my disposal due to various reasons, and they tend to break easily, so I used to run badblocks to them before I begun using them, to be sure they were safe (reliability is very important to me). Now I simply plug them in and trust ZFS checksuming to do the right thing.
• No more waiting for long fsck. Specially annoying when I m in a hurry and chance wanted that this was the 30th time I boot my system without fsck.

That s basically my personal experience as newbie Debian GNU/kFreeBSD user. Of course my perspective is very limited because I just started and yes, I am biased. Anyway, what about yours? If you have installed Debian GNU/kFreeBSD, was it meant for production or just a toy machine ? If you considered using it on production, did it succeed at satisfying your needs, or did something hold you back? Leave your comment!

Documentation! That is what the next version of Lintian will have. In 2.5.1, Lintian
had support for adding options in the lintianrc. Unfortunately, the
user had to guess the syntax, which makes the feature less useful.
With the next release, the manpage and the example config file[1]
will contain documentation and examples. The User Manual[2] also saw some improvements. Particularly, it will
mention that lintian can process a .changes file and that it does
cross-package checks. Of course, it also documents the format of the
new Vendor Profiles. Though even with these improvements, the current
documentation still leaves much to be wanted. The last unpack script (unpack-srcpkg-l1) was finally removed and
was replaced by a smarter version of index collection. This means
that Lintian can now schedule index in parallel with a few other
collections (namely unpacked, which is the current bottleneck). The speed improvement is probably not noticeable, but it was also a
good time to upgrade our index collection to properly handle
multi-tarball source packages. Previously Lintian only extracted the
index from the base tarball. The fields collection also saw some attention. Previously it
extracted fields from the control file (in the .deb, etc.) into a
directory. Each field became a file and the value of the field would
be written in the field. Today, the collection basically looks like
this:

[...]
if (-d 'fields') 
    system('rm', '-rf', 'fields') == 0
        or die 'Could not remove old fields directory';
 
[...]

It has been deprecated the minute Lintian::Collect API saw a little
upgrade to fetch these fields itself. This change should remove some
unnecessary file I/O in a normal run[3]. If you have a friend with his/her own little home-grown Lintian check
(or collection)[4], they will have to update it to A) not depend on
fields, B) use the Lintian::Collect API for accessing fields and C)
depend on index for source packages if it uses the index information.
For the latter, Lintian::Collect has grown support for source package
index. Lintian has also seen a lot of decrufting in the 2.5.2 development
cycle. The new test suite now features skeletons ( standard base
environments ) and template support for all package test suites[5]. This means it takes a lot less of writing (or copy/waste) to generate
a simple specialized test in the low-level test suites. As a part
of cleaning up the test-suites, I got to run:

$ git rm t/ source,debs /*/copyright

without breaking a single test. I have to admit that commit 8584996
is among my favorite ones in the 2.5.2 development cycle. Finally, if you are one of the few that have ever used the
--packages-file option, then I am pleased to inform you that it will
be deprecated in 2.5.2 and replaced by --packages-from-file. The
old option had some specialized format, whereas the new option parses
the file (or stdin) as one package per line . The latter being far
more "find -name '*.deb' "-friendly. In case you are wondering why --packages-file is deprecated, here is
the packages-file parser from lintian 2.5.1:

    while (my $line = <$pkgin>)  
	chomp($line);
	my (undef, undef, undef, $file) = split(/\s+/, $line, 4);
	$pool->add_file($file);
     

- [1] Installed as /etc/lintianrc [2] Installed as /usr/share/doc/lintian/lintian. html/,txt.gz [3] Namely, 2 opens + 2 closes + 1 read and one write for each field
in every package (the fields were cached, once they were read). Not
to mention a directory listing and having to remove the files again
later. [4] Of course, you do not have such a thing Because we all know
that no one (except this not too close friend of yours) messes with
Internal APIs of other projects [5] The exception being that the changes suite does not have a
standard base environment, since it would be completely unused.

Wondering what is going on with the port of Debian to the Minix 3 microkernal? Here's an update. dpkg Ported Well, the big news as I mentioned in my last post is that dpkg has been successfully ported. I haven't sent the patches to the maintainers just yet as they need review and clean up but I will do that ASAP. The architecture name is defined as minix_i386. (There is no k because I am using the Minix libc as well as the kernel.) I am currently targetting squeeze. A Maze of Twisty Dependencies All Different Armed with my new dpkg, I have begun building packages which has been an interesting experience. The Debian packaging systems greatest asset dependencies is also its most frustrating during initial bootstrapping. Take debhelper for instance; nowadays most packages depend on it and I personally wouldn't think of creating a .deb without it but it build-depends e.g. po4a which build-depends on...debhelper. Also some programs which are assumed to be present on linux such as ldconfig are simply not available on Minix. So currently theres a lot of dpkg --force-depends and even --force-all going on. Of course this is only a temporary problem until I get a full set of base packages built. The equivs package is coming in handy too. In lieu of working .debs, I have been using Minix native packages to fill in the gaps. They use NetBSD pkgsrc which is a new experience for me. I can't say I'm that impressed but they work and, being uptodate, are a good source of patches. Works in Progress The two big packages I need to work on next are perl 5.10.1 and gcc 4.4.5. I haven't touched the latter yet. Perl is failing in its second-stage bootstrap for reasons I haven't gotten to the bottom of just yet. I suspect it has more to do with deficiencies in my build environment than a problem in perl itself. libc Deficiencies As I mentioned last time, the situation has improved tremendously since Minix switched over to using the NetBSD libc but I am still running into problems with unimplemented APIs. Take getpwent(3) and getpwuid(3) for instance. Being able to map between user ids and names is kind of important you know? Even worse instead of being completely absent some of these exist as stubs. So they compile and link without warning but always return ENOFILE and so on. I was surprised to find that there is no standardized API for getting information on mounted filesystems. (Which e.g. df(1) needs.) Linux uses getmnt(3) similar but incompatible with System V. The BSDs use getmntinfo so probably this is what Minix will implement. GNU programs are good about portability but a lot of Free software is unfortunately Linux-centric. Hopefully the Debian FreeBSD port already has patches for these cases. There are several annoyances like this but I suppose I shouldn't complain too much. I am using an unreleased version of Minix which is under heavy development and I must say the upstream developers have been good about responding to my concerns. Looking forward: fakeroot, grub2, ext2 fakeroot is a very handy development tool that frees a developer from having to create .debs as root. Alas it won't work under Minix as it depends on LD_PRELOAD and Minix doesn't even support shared libraries. fakeroot-ng sounds a bit more promising as it uses the ptrace(2) system call which is implemented in the Minix kernel though I don't know how well. Minix is now multiboot compliant so it is possible to use grub2 instead of the Minix boot loader. Definitely something I'm going to look at soon. Minix now supports ext2 (though not ext3 or ext4) in addition to its native file system. For compatability this should become the default in the Debian port. While the kernel support is part of the standard Minix system, the userland support (e2fsprogs) is not for some reason. So I'll have to see about porting that. Can I Help the Porting Effort? Sorry not yet. Though if you are determined I'll try and help you get set up. Hopefully in a month or less I will be able to distribute a pre-alpha version so I suggest waiting for that. I am definitely going to be looking for outside help.

NOTE: This post uses MathML. You will need a modern browser to view things correctly. I just had a fairly annoying experience with pocketronics and have left negative feedback for the first time. Basically they expected the customer (my mother in this case), to contact them when Australia Post failed to deliver. And, rather than sending out another item as they had to anyway they wanted to wait until Australia Post reported back. Which takes 10 working days. So for about ~AUD$30 they decided it was worth throwing away their reputation. Obviously a company with a different view of the life time value of a customer and the earning of goodwill than most. With eBay generally almost never leave 'positive' feedback because, frankly, that should be the default case. Anyway. But some feedback about eBay's feedback system. Limiting to 80 characters is pointless. I could have put in a bit.ly link to a twitter status as my feedback for pocketronics. Or a blog post (like this), linking to their site (as above). If I want to write an article, then let me. And just as I can respond as many times as like via other media, I ought to be able to do the same via the internal ones. The score calculation is weighted towards positive feedback. As explained scores are: $\frac{\mathrm{positive}}{\mathrm{positive}+\mathrm{negative}}$ Which for pocketronics comes to: $\mathrm{99.7}=\frac{\mathrm{5071}}{\mathrm{5071}+\mathrm{16}}\mathrm{100}$ If the calculcation instead took into account neutral feedback too, then things change - not by much, but enough. To wit: $\frac{\mathrm{positive}}{\mathrm{positive}+\mathrm{negative}+\mathrm{neutral}}$ And it becomes: $\mathrm{99.0}=\frac{\mathrm{5071}}{\mathrm{5071}+\mathrm{16}+\mathrm{36}}\mathrm{100}$ I think that also by default feedback to be neutral by default, it would reduce the gaming of feedback. To have a positive score you would need to be exceptional. Not just run of the mill. Which is what the current system encourages.

It has been a long time I want to prepare a pure Debian virtual image for the Amazon elastic computer cloud, that would contain the bioinformatics tools that we package in Debian Med. Most methods discussed in the ec2debian group use debootstrap, and finish the preparation with external scripts. Now that Amazon Machine Images can boot on their original kernel, I am exploring the use of the debian installer to setup a pristine system on a elastic block storage volume. The Debian installer can be booted with GRUB and preseeded through a file downloaded early after starting. In the Amazon cloud, this file can be put at http://169.254.169.254/latest/user-data with the other instance metadata. Since I am not much experienced in this field, I progress slowly on automating the procedure. For the moment, preseeding is not comprehensive, but at least the installer's SSH console is started. Ideally, one should connect using a key, but for the moment I go with a password. Currently, I am stuck with the partitioning of the hard drive: No root file system is defined . Here are a couple of technical details. I use the cheapest instances, t1.micro (32 bits) for the tests. I downloaded debian-installer on an elastic volume of 1 Gb, that I formatted in ext2. Not knowing if the device name will stay stable (/dev/sda1 or /dev/xvda1), I flagged the partition, as I have seen in Ubuntu virtual machines.

ARCH=i386
DIST=squeeze
DI_VERSION=20110106+squeeze3
MIRROR=jp
BASEURL=http://ftp.$MIRROR.debian.org/debian/dists/$DIST/main/installer-$ARCH/$DI_VERSION/images/netboot/xen
mke2fs -L debian-installer /dev/sdb -F
mount LABEL=debian-installer /mnt/ && cd /mnt/
wget $BASEURL/initrd.gz $BASEURL/vmlinuz
mkdir -p boot/grub
cat > boot/grub/menu.lst <<__END__
default 0
timeout 3
title  Debian Installer ($DI_VERSION $ARCH)
root   (hd0)
kernel /vmlinuz root=LABEL=debian-installer ro console=hvc0 auto=true priority=critical url=http://169.254.169.254/latest/user-data
initrd /initrd.gz
__END__

A snapshot of this volume can then be registered as an image machine. The kernel to use will depend on whether the system has been installed in the whole volume or a partition. Debian distributes tools for all these operations, such as euca2ools.

When I think about Margarita, I always remember her as a friendly and welcoming person. Like most of the Debian Women members by the way. But she likes to spread some love and organized a Debian Appreciation Day for example. I think I met her in real life for the first time at Debconf 6 in Oaxtepec (Mexico). She deeply cares about Debian in general. She has proven it multiple times with her DPL candidacy and by giving talks like Making Debian rule again. One last thing, Debconf11 is just over and you will see that Debconf4 has had a big influence on Marga. My advice is simple: next time there s a Debconf on your continent, make sure to take a few days off and come to meet us! It really gives another picture of the Debian community. Now let s proceed with the interview. Raphael: Who are you? Margarita: I m Margarita Manterola, a Software Developer from Argentina. I work developing software in Python in a Debian-friendly company during the day, and teach programming at a local university during the evenings. I m married to Maximiliano Curia who is also a Debian Developer, most of our Free Software work has been done together. I only maintain a handful of packages in Debian, I m more interested in fixing bugs than in packaging new software. I ve also been a part of the organizing team of many of the previous Debian Conferences. One of the biggest commitments and the biggest success of my participation in Debian was being part of the organizing team of DebConf8, in Argentina. Raphael: How did you start contributing to Debian? Margarita: I started using Debian around 2000. Soon after we had learned the grips of general GNU/Linux usage, Maxy and I started giving an introductory course at our local university, and became quite involved with the local LUG. At some point in 2002/2003 I became a Debian Bug Reporter : most of my friends would report bugs to me, and I would then write them in the proper form to the BTS. I would also be very attentive about reporting any bugs that I might encounter myself trying to create good bug reports. The turning point in my participation in Debian was DebConf4 in Porto Alegre, Brazil. Being so close to Argentina meant that we felt specially invited to be there, and Maxy and I decided to go to DebConf for our honeymoon. We didn t really know much about DebConf dynamics, but we were really eager to learn more about Debian and become more involved. What happened was that meeting with DDs from all over the world transformed our lives, we became part of the Debian family and wanted to be more and more involved. Soon after that we both started maintaining packages and not long after that, applied to become Developers. The Debian Women project also meant a lot to me. I felt encouraged all along the way, encouraged to learn, to ask questions and to lose the fear of making mistakes. I became a Debian Developer on November 2005. Since then, Debian has always been one of the most important things I do in my life. Raphael There was a Debian Women BoF during debconf. What are the plans for Debian Women in the upcoming months? Margarita: I was not there in person, but thanks to the awesome work of the video team, and of Christian Perrier s typing efforts when something failed, I was able to experience much of what was discussed. One of the many points that came up during the BOF is that many people Want to help but don t know where to start or how to go about it. It s a challenge for the Debian Women project to find a way to allow these people to become involved in Debian through Mini projects or something like that. Another of the subjects that was brought up was the Debian Women mentoring project, which has been going on for quite a while now, but lacks enough publicity. So, we need to reach more people about it, and maybe also improve it with some templates, similar to the New Maintainer templates, so that mentees that don t know where to start have some sort of general path to follow. Raphael: You created very useful diagrams documenting how package maintainer scripts are invoked by dpkg. How did you do it and was that a useful experience? Margarita: I did those diagrams to be able to answer one of the questions in the NM templates, regarding the order of the maintainer script execution. Answering the question in text was basically copying and pasting the part of the Debian Policy that explained it, which wasn t really too clear for me, so I decided to go and make a diagram of it, so that I could really understand it. I did it by the best of all debugging techniques: adding prints to each of the maintainer scripts, and testing them in all the different orders that I could think of. It was a useful experience at the time, because I learned a lot of how maintainers scripts work. I didn t expect the diagrams to become so famous, though, I only did them to answer one NM question, that I assumed most other people had already answered before Raphael: You participated in a DPL election. This is a big commitment to make. What were your motivations? Margarita: As I said, I was part of the organizing team of DebConf8, in Argentina. Which was quite a success, a lot of people enjoyed it and praised the good work that had been done by the local team. During said DebConf8, I had a dream (it was almost a nightmare, actually): I woke up and just like that, I was the DPL. I spoke to some people about this dream and to my complete surprise many said that I should actually do it. After giving that possibility a year and a half of thoughts, during the 2010 campaign I was talked into participating myself as a candidate, and it was a very interesting experience. However, I m very glad that Zack got elected and not me, I think he makes a much better DPL that I would have made. Raphael: What s the biggest problem of Debian? Margarita: I think the main problem that we have is our communication, both inside the project and outside the project. Most of us are very technical people, our skills lay in the technical part of Debian (preparing packages, fixing bugs, writing software, administering systems) not in the social part. And thus, we lack a general empathy that is quite needed when interacting with people from all over the world. Raphael: Do you have wishes for Debian Wheezy? Margarita: Not particularly. I do want it to be a great release with good quality, stable software. I would also like to keep making Debian more and more universal with each release, making it more user friendly, more accessible, and more robust than any other previous release. Raphael: Is there someone in Debian that you admire for their contributions? Margarita: I admire a lot of people in Debian. There s a lot of people that contribute a lot of time to Debian, amounts of time that I can t begin to understand how they can afford. I admire Stefano Zacchiroli, our current project leader. And Steve McIntyre, the project leader before him. Also Bdale Garbee, who s also been a DPL in the past. Making this list I realize that Debian has been blessed by quite a number of great leaders in the past. I admire Holger Levsen, for his contributions to the DebConf video team, that have made it possible year after year for the whole project to participate in DebConf remotely. I admire Steve Langasek and Andreas Barth (etch is still my favourite release). I admire Christian Perrier for his work on internationalization. I admire Joerg Jaspert for the incredible amounts of time that he puts into Debian. And actually, I could go on admiring people all night long. I admire so many people that this interview could become a very boring list of names. I guess it s better to leave it at saying that Debian is lucky to have quite a lot of excellent hackers around.

---

Thank you to Marga for the time spent answering my questions. I hope you enjoyed reading her answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

14 comments Liked this article? Click here. My blog is Flattr-enabled.

Sometime it happens, that there are school holidays. As my lovely girl used to be a teacher, she has some time off at that period. So she left me with our kids yesterday to visit the grandparents.
It was two hours after leaving, when I was really surprized that I was sitting around and thinking about what to do next, as normaly the weekend are family time.
So I decided to take my bike and a paper map to get of for a trip. Since some years, I usually use my bike to escort the girls or when going to work, which is just a 5 minutes ride, so I thought that would be a great opportunitiy.
I took a backpack with raining clothes, mounted the bike helmet on it, in case I needed to go cross country, some water and my music player.
It was a great ride, beside that I realized I m not in the same condition as some years ago. Unfortunately this will likely not change in the next 12 mounth, as we have some projects in the queue we need to deal with. Hopefully we will solve that satisfactorily and have a bit more spare time afterwards.

Anyways .. 27,65km in 1h40 is not so bad, as it was hot and humid.
Ride summary:

rsync is often used to back up systems, with options such as:

-a --archive

recurse and preserve usual attributes: symlinks, devices and special files, user and groups ownership, permissions and times;

-H --hard-links

detect and preserve hard links;

-A --acls

preserve ACLs, if you use them;

-X --xattrs

preserve extended attributes, if you use them.

In addition to these common options, rsync has plenty others, so everyone has his own recipe, but I would like to share two useful options I discovered.--numeric-ids By default, rsync uses literal UIDs and GIDs if it can find matches in both the source and the destination. That is, if you have users jacquouille (1002) on your source system Reblochon, and a user jacquouille (1005) on your destination system Neufch tel, rsync will map them: files with an UID of 1002 will be backed up with an UID of 1005. This may or may not be wanted, but anyway you must be aware of it. For instance, if you have to restore Reblochon's home directories and do that by attaching a new drive locally to Neufch tel, you will end with mixed up UIDs. This option --numeric-ids forces rsync to use numeric IDs instead of trying to map them. It is notably needed for backups of jailed systems (BSD jails, OpenVZ, VServer, lxc ) which appear to have bogus IDs when seen from their host system because they have their own ID maps. --fake-super When you back up a full system or anything that requires keeping file ownership or device information, you need to have root privileges on the destination system. Without them, rsync would be unable to correctly chown() the destination files. This option --fake-super offers a way to back up privileged information without requiring root privileges. It works by adding special extended attributes such as user.rsync.%stat containing al this information. Of course, it still requires a destination file system that supports extended attributes. This provides a good way to push system backups without weakening your security by opening a root access on you destination system. For instance, I personally use it with rsyncd running as an unprivileged user.

The first day of the Desktop Summit was yesterday, Saturday, August 6. I loved it and gave a presentation. There are two very different stories I can tell on the topic of gender equality in free software. I'll start with the bad.

---

It's pretty easy, in the U.S. at least, to get people of privilege to stop using terms that evoke centuries of oppression from slavery. It's harder to ask people to stop doing that to women. I'm writing this post to ask for that. There were two different times that people I generally respect used words that historically have been used to hurt and minimize women. "Cygnus Solutions is prostitutes." Dave Neary was delivering a talk (that I found impressively informative) called, "The cost of going it alone". When the talk covered the cost (in time and money) of getting your corporation's code into the community branch of an open-source project, he pointed out you could hire someone with the skills. Cygnus was the go-to company for that in the 1990s; to explain that Cygnus does not care who they work for, he said the sentence in bold. "The OpenSuSE Build Service is a slut." I was at a party at a hackerspace last night. Someone who I admire for his work in free in free software, both technical and community, was joking with me about that no one should compile (or use) GTK. I riffed on the joke and remarked, "The OpenSuSE build service will build anything." He replied with the sentence in bold. "Slut" and "prostitute" are terms that recall the objectification of women. They're terms that attempt to measure a woman's worth as a sex object. It's not nice to the many women in attendance to bring that up. You might not have thought this through. You might want to read one woman's take "On Sluts, Rape, and Fuckery". Give it a read. Then, give it a rest. The thing is, this matters all the time. That's why I have to call you two out on it. Now for the happy story.

---

While watching the Desktop Summit's intern showcase, I was floored. One hacker implemented Off-The-Record instant messaging for Telepathy. Another implemented pluggable back-ends for Getting Things GNOME, a task manager. I heard about overwhelming documentation and usability improvements to GNOME mainstays Cheese and Anjuta. In a short summer, these students made huge changes. For most of them, this was the first time they delivered any sort of presentation. Every single talk was delivered in earnest and enthusiasm. They told us about the work they had done and what might happen in the future. The other remarkable thing about the intern presentations was the demographics. I didn't keep count, but it seemed like as many women as men presented. We heard about hugely-important changes to documentation, code, and usability. People from central Europe, South Asia (yay), and Brazil took the stage. I hope that is the future of free software.

---

There are two things I want to see for our community. I want to know that people are respected and not reminded of centuries of oppression. I also want to see our community grow in size and diversity. I treat these issues as separate. We should choose respectful words when we speak not because we want more women to show up, but because it part of the expectation of decency that we should be able to expect from each other. And I failed the community when I did not make it clear there and then that this kind of language is not okay with me. It took me a day to understand this failure, so here I am writing this blog post. P.S. Thanks to Karen Rustad for her feedback while writing this post.

I got a call from Mum today to inform me that my grandmother had passed away last night, Brisbane time, at 94 years of age. If you want to get all genealogically correct, she's technically my step-maternal grandmother, but she's the grandmother I spent the most time with, growing up, so I'm not going to split hairs. It's come as a bit of a shock, because despite having had progressively worse dementia for probably over a decade, she's been in incredibly good physical condition. I think it was in her 70's that she had a heart valve replaced, and I for one had been expecting that to be the thing to wear out and take her down slowly. I last saw her in the flesh in January, when the above photo was taken, and more recently a few weeks ago on Skype. She was recently moved from an aged-care hostel in Toowoomba to a nursing home in Brisbane, as her dementia had progressed beyond the point that she could be cared for in the previous facility. Mum had visited her earlier in the day yesterday, and she was fine, but apparently the staff found her unconscious at around 5pm when they went to get her for dinner. She was taken to the Royal Brisbane and Women's Hospital where it was determined she had suffered from a stroke, and she passed away later that evening at around 11:30pm. So all in all, she had a very good innings, and of all the ways to leave this world, this was one of the quicker and more painless ways to do it. That's the problem with a quick exit - it's usually completely unexpected. It's looking like the funeral will be on Friday, and I've got a flight tentatively booked to leave on Wednesday and get in on Friday morning and return again on Monday. At the time I arranged this, I'd completely forgotten that I was flying to Calgary on Monday night to babysit the son of friends, while they go to the US Consulate for a visa appointment. Fortunately this travel doesn't conflict with that travel, it just means I'll be away from work for longer, and spending a lot of time inside pressurized tubes. I'm fairly confident with all of the short international trips, I'm going to look like a drug mule or something, and someone's going to thoroughly cavity search me.

FreedomBox activities at Debconf11 I spent the last two weeks of July 2011 in Banja Luka. The occasion was the annual Debian developer's conference, Debconf11 and preceding work week known as Debcamp. This was my tenth successive year attending Debconf, and I had a very productive and pleasant time! The facilities were good, the local team was friendly, enthusiastic, and very helpful, and in addition to giving three talks and hosting a couple panel discussions, I managed to put a burst of energy into work on FreedomBox. Several other developers working on FreedomBox were also present, and a good number of Sheeva and Dream plugs were evident in the hacklabs sporting new FreedomBox stickers. Working together in the same place for several days, we made good progress on several projects, and also had some great discussions about what we want to do going forward. image building tools For some time, I've been working towards a light-weight tool set to build FreedomBox software images. Shortly before Debconf started, I chose the name 'freedom-maker' for this tool and shared a link to a readable copy of my git repository with other developers I expected to work with in Banja Luka. With input from Bert Agaz and Jonas Smedegaard during Debcamp, freedom-maker went from almost useful to actually useful. It still deserves work to be more useful to others, but I have now pushed a copy of the git repository to git.debian.org so that we can take advantage of the tools supported there to enable others to more easily contribute to the code. Very soon, Bert plans to add support to freedom-maker for using Lars Wirzenius' vmdebootstrap to build x86 images suitable for testing in a virtualized environment. At the same time, we plan to refactor the existing code slightly to enable lists of desired packages for the various image flavors we expect to produce independently of the configuration for each specific image building tool. Jonas continued in parallel to work on his alternate packaging toolset boxer. It offers some potentially interesting features for the future, and we may eventually merge some or all of it into freedom-maker, but for now it remains a separate utility. uAP user space tools Several weeks ago, we received from Marvell the source code to two user space programs that are necessary for configuring and monitoring the binary firmware provided for the uAP wireless chip used in the DreamPlug. Early during my stay in Banja Luka, I packaged these for Debian as uaputl and uapevent, and I am pleased to note that they were quickly accepted into the archive and are now present in Debian mirrors. u-boot Another bit of code received very shortly before Debconf started was the source for the version of u-boot shipped by Globalscale in the DreamPlug units we're working with. During Debcamp, Clint Adams passed a copy of this source to Jason Cooper, who was already trying to add support for the DreamPlug to upstream u-boot, but had stalled due to a lack of information. Jason has now merged his own work with the sources we got from the manufacturer, and is making good progress towards merging DreamPlug support into upstream u-boot. Once that happens, we should be able to flash our Sheeva and Dream plug devices with a u-boot image built from the source in the Debian u-boot package, in the process enabling things that matter to us like the ability to boot from an ext2 partition, and hopefully the ability to execute command scripts from that partition instead of having to hard-code kernel filenames in flash. This will allow us to support the ongoing effort in Debian to move away from the need for kernel symlinks. DreamPlug kernels With respect to kernels, another work stream at Debconf primarily involving H ctor Or n and Nick Bane was to analyze the current state of the patches from Marvell and Globalscale used to support the DreamPlug against both upstream and current Debian kernel sources. To my surprise and our collective pleasure, the remaining patch set required against current upstream kernels is much smaller than we previously believed! There are still several patches critical to us that are not merged upstream, but the work remaining to be able to build images for our devices from mainline and Debian kernel source trees now seems like something we might be able to complete before Debian's next stable release. One of our discoveries during the u-boot and kernel work during Debconf was that Globalscale did not obtain a new machine id for the DreamPlug, but instead re-used the one for the GuruPlug series, despite there being some differences in the hardware that require at least one additional driver. After much discussion, we plan to continue using the existing machine id instead of requesting another, particularly because the ARM kernel community has apparently stopped issuing new ids for the moment. We will add a new kernel config option for the DreamPlug, however, and are likely to build distinct Sheeva and Dream kernel packages that do not require initrd for use in FreedomBox images, even if doing so is not strictly necessary. This will allow us to optimize both the in-memory footprint and boot times for our devices. software configuration Another area of investigation in Banja Luka was technology for package configuration. Mirsal Ennaime performed various tests using debconf and Config::Model, with some results reflected in this commit relating to configuring the bitcoind daemon in the bitcoin package for Debian. identity and trust management While we did not actually do any FreedomBox specific work on the trust management layer we know is necessary, after several rounds of conversation, I am now more convinced than ever that the right path forward is to base our trust relationships on OpenPGP keys using GnuPG and Monkeysphere as starting software elements. Our thinking to date is captured on an Identity Management page in the wiki. communication services Another thing that became fairly clear to me during discussions at Debconf is that in the near term, planning to build communication services around XMPP is the approach most likely to give good results. Investigating the software choices available to build an interesting XMPP infrastructure is now a high priority for me. Jonas has done some work towards configuring and integrating ejabberd or Prosody, I've started studying yate as a possible call manager and VoIP server choice with XMPP/jingle support, and we await with great interest a release from the Buddycloud developers to evaluate as a possible basis for deploying social network services. Some of these software choices will lead us to use Apache as our web services base technology because of the need for features that only it supports well among daemons that are Free Software. Jonas completed packaging GNU Sip Witch for Debian, and it is now available in the mirror network. Tzafrir Cohen and Jonas did some initial testing on its use. documentation A number of new wiki pages were written (or at least started) in order to sum up ideas, design various aspects of FreedomBox, and reflect discussions that happened during DebConf11. A lot of work is needed to complete these pages though, as well as others to capture more of the current state of the project. press coverage Finally, while in Banja Luka I got some great press coverage for FreedomBox! On Sunday the 24th, I was interviewed by the main television network serving the Republika Srpska. This led to a couple of minutes of coverage near the top of the national news program that night, immediately following the lead story about the President and several ministers appearing at Debian Day that morning to help open the conference. This interview was later re-used in another TV program that summarized Debconf11. On the morning of Thursday the 28th, I was part of a small group that spent more than an hour meeting with the Minister of Science and Technology in his office, and the relationship between Debian and our work on FreedomBox was one of the items of discussion in that meeting and the associated press conference. I'm told this resulted in more press coverage, but if true I have not seen it yet. summary On Friday afternoon the 29th, I gave a talk in the main Debconf program containing a FreedomBox Progress Report . In it, I talked about the structure of the FreedomBox Foundation, progress the foundation has made, and the work that was still then underway in Banja Luka. It was streamed live over the internet, and replays are available online. The reaction from Debian developers present was very positive, which was good to learn since by that time my energy level was quite low after the nearly two weeks of intense technical and social interaction that is Debconf! All in all, we got lots of work done on FreedomBox in Banja Luka, enough that I think at least the next few steps along the road towards an eventual "1.0" release of a reference implementation are now much clearer than they were two weeks ago!

The DSPAM project has recently released the 3.10.0 version of free content-based spam filter. Besides various bug fixes, the most important change is the switch to the AGPL-3 license (from GPL-2). The Debian packages are almost ready, but still require further testing as they have been largely amended, especially to add multiarch support. I expect to upload them to unstable within the next week or so, and, if no major problem happens in the meantime, will upload them to squeeze-backports as soon as they reach testing.

Thanks to Lars Wirzenius who has written a nice how-to on GnuPG subkeys on the Debian wiki, I am now using a signing subkey. If you happen to have my key 0x258E26B1 in your public keyring, please update it: This will allow me to keep my master private key offline while still being able to sign mail or Debian packages for upload.

When you care for somebody, it is difficult to tell them "no." When you trust somebody, you will tell them.

I don't want to keep my GPG private keys on my laptop, so I've been keeping them on an encrypted USB drive instead, and loading them onto the laptop only when needed. That's cumbersome, when making Debian packages, and needing to sign the packages frequently. So I'm going to start using subkeys instead. However, I couldn't find good documentation for that, so after figuring things out, I wrote something that will hopefully be useful for others. It's at http://wiki.debian.org/subkeys for your fixing pleasure.

Todays addition to the tips series has a funny history. A friend of mine was stuck in an envrionment, where she couldn't install Perl or CLive but PHP with some PECL modules was installed. She needed to download a few videos for a presentation, but didn't know how to fetch the videos from YouTube. So I was asked to help. ;-) After some pondering of the problem, I came up with the following script:

<?php
/*
 * Copyright (C) 2011  Kai Wasserb ch <kai@dev.carbon-project.org>
 *
 * Small little script to help out in the absence of Perl/Clive but with
 * PHP present.
 *
 * License: GPLv3 <http://www.gnu.org/licenses/gpl-3.0.txt>
 *
 */
if(php_sapi_name() !== 'cli')
  die("This script needs to be run on the CLI!\n");
if($argc !== 2 )
  die("No URL given on CLI or too much parameters!\n");
// Script expects a URL like http://www.youtube.com/watch?v=[ID]
if(filter_var($argv[1], FILTER_VALIDATE_URL, FILTER_FLAG_QUERY_REQUIRED) === false)
  die("Parameter given as URL does not seem to be an URL or contain a query!\n");
if(!$ch = curl_init($argv[1]))
  die("Couldn't init cURL handle for  ".$argv[1]." \n");
curl_setopt($ch, CURLOPT_HEADER,          false );
curl_setopt($ch, CURLOPT_RETURNTRANSFER,  true  );
$htmlsource = curl_exec($ch);
$error = curl_error($ch);
curl_close($ch);
if($htmlsource === false)
  die('cURL returned an error: '.$error."\n");
unset($error);
if(!preg_match('/url_encoded_fmt_stream_map=([a-zA-Z0-9%\.,\-_]+)&/',
               $htmlsource, $matches))
  die("No matches found for url_encoded_fmt_stream_map!\n");
$final = NULL;
foreach(explode(',', urldecode($matches[1])) as $decoded_url_map)  
  parse_str($decoded_url_map, $tmp);
  if($tmp['quality'] === 'hd1080')  
    $final = $tmp;
    $final['url']   = urldecode($tmp['url']);
    // $final['type']  = urldecode($tmp['type']); // Not used
    break;  // Not really needed, but should save us some
            // iterations, since 1080p seems to come always first.
   
 
if(!is_array($final)   is_null($final))
  die("No 1080p version of the video was found!\n");
if(!preg_match('/<meta property="og:title" content="(.*)"/', $htmlsource, $title))
  die("Couldn't determine title!\n");
$ftitle = preg_replace('/[^\w\d]*/', '', $title[1]);
printf("Found hd1080-quality video for  %s .\n", $argv[1]);
printf("Downloading to %s.mp4 ...", $ftitle);
if(!$ch=curl_init($final['url']))
  die("\nCouldn't create cURL handle for  ".$final['url']." \n");
if(!$fh = fopen(getcwd().DIRECTORY_SEPARATOR.$ftitle.'.mp4', 'wb'))  
  curl_close($ch);
  die("\nCouldn't create  ".$ftitle.".mp4 .\n");
 
curl_setopt($ch, CURLOPT_HEADER,         false );
curl_setopt($ch, CURLOPT_FILE,           $fh   );
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true  );
if(!curl_exec($ch))  
  $error = curl_error($ch);
  fclose($fh);
  curl_close($ch);
  die("\nCouldn't download video. cURL error: ".$error."\n");
 
curl_close($ch);
fclose($fh);
printf(" done.\n");
?>

You can download it from my website or you can just copy the code given above into a file. However you get the code, the execution will look like: php -f /path/to/script.php http://www.youtube.com/watch?v=[ID]. Now, before I get a lot of e-mails telling me, what could be improved on the script or its other limitations (like only being able to download 1080p videos), please remember that it was hacked together for a simple use case with no real intent on my part to use it for anything else. It only later occurred to me, that others might be in a similar pinch sometime. Thus I'm releasing the above code "as is" under the GNU GPLv3. Therefore you're free to adapt it to your needs. Please also note, that I'm not recommending using the above script instead of CLive. If you can install CLive, do so and don't bother with the above.

Hello, this is the fourth report for the project. First, I d like to
apologize for being late, I took a short trip over the weekend and
missed the deadline. This time I ve good news for my project. As
expected I ve made the necesary changes to have Eucalyptus support ARM
images. The approach was extending and exposing an arch field that
will be used by libvirt s XML domain file. I m working currently on
having the involved functions working with the new extension and I
expect to have this done over the middle of the next week. What was done so far

- Extended the ncInstance_t struct on util/data.h adding the archId
parameter. This struct keeps the information regarding a node (nc),
say RAM, Kernel, etc. Currently it has no way to set an arch in order
to be used later. typdef struct ncInstance_t
...
char reservationId[CHAR_BUFFER_SIZE];
char userId[CHAR_BUFFER_SIZE];
char archId[CHAR_BUFFER_SIZE]; // arch field added
int retries;
...

The Eucalyptus team was OK with this approach that I ve proposed few
weeks ago. For all my changes the approach is maintain compatibility
with existing features.
We still have to define if we are setting a default value, say amd64,
or keep it unset until the user sets one. For now we will be supporting
amd64 and arm as valid fields. - Modified the allocate_intance() function to support the archId
field. This involved adding a new parameter and detecting whether its
present and setting de value accordingly. This function is later used
by the doRunInstance() function in node/handlers_kvm.c - Modified the involved functions that are called by the KVM handlers
and result in passing parameters to the XML file generator for being
used by libvirt s and KVM and have the image running under
Eucalyptus. The new parameters and field are not required and only
being used if they are set with a value. The archId parameter has been
added as the last one, so existing function calls can work. This could
be rearranged later, but will involve a more extensive revision with
Eucalyptus developers. - I ve set a git local repo in order to keep track of the changes to
Eucalyptus. Since this part is a sort of a Eucalyptus branch I took
this approach. Steffen and I are discussing whether to release this as
a pkg-eucalyptus branch (as dpatch, using existing packaging). Either
way I m putting the diff files form my git repo over the wiki and will
put them on my site too. I plan to use git-svn in order to send them
later to the pkg-eucalyptus repo once we are settled with this. - Updated the project s wiki[1] page with information regarding all that s
involved on my work, so it can be re-played or resumed later. 1- http://wiki.debian.org/SummerOfCode2011/BuildWithEucalyptus/ProjectLog What I plan to to over the next weeks
-
- Finish the modifications to the relevant functions. Expected dealine:
next week.
- Do integration tests:
Write a test program, there s an example under node/ that could be
helpful.
Generate the XML domain definition from Eucalyptus functions. The
full cycle: handler_kvm -> allocate_instance -> get_instance_xml ->
have the XML file properly generated -> qemu-kvm runs the instance
using the proper emulator (arm in our case).
- Test runnning the image I ve worked on the project s begining.
- Fixing bits and cooperate with the pkg-eucalyptus team on the
packaging bits.
- Talk to Eucalyptus so they adopt our patches.
- Rock and roll :) What would be nice to have after the project ends
-
- The Debian packaging *needs* work. I ve already discussed this on the
pkg-eucalyptus mailing list. I ve also committed myself to help on
that side even after the SoC ends.
- Talk with upstream and Ubuntu to coordinate the packaging efforts and
modifications to the software.
- Get adopters to use our work :) By now, few weeks about to finish the SoC, I can say that it s been a
good experience and as I stated I m planning to keep maintaing/working
on the software after that. I ve already joined the pkg-eucalyptus team
and plan to contribute actively. I m sad I didn t managed to get Debconf
this year, time was a constraint again, hopefully we can meet in 2012! I
guess that all by now! Best regards,
Rud