Hello World. I am very pleased to announce apt-offline, version 1.0. This release adds a Graphical User Interace to apt-offline. Big thanks to Abhishek Mishra and Manish Sinha who did all the development work to make this GUI happen. Help: I was wondering if there is a logo for APT that I could use in the big blank space on the main window. Apart from the GUI, there are a bunch of bug fixes in this release (which have already been made available for the Squeeze release also). I look forward to your feedback and comments on the GUI front. Note: The GUI in its current form is basic and does not provide all the features that are available in apt-offline.

Wishing everyone a happy, auspicious, and prosperous Gujarati new year. (Vikrama samvata 2067 called "Shubhakrt") For Diwali I got a smartphone after enduring years of pointing and laughing at my previous decidely dumb phone. It's a T-Mobile G1 with Google or in other words a rebranded HTC Dream. I didn't want an iPhone (evil) or the phone the sales people were trying to push, the G2 (very evil.) As evil is the default in the telecom industry I especially wanted the ability to run CyanogenMod and this phone, though it doesn't have all the latest whiz-bang features, is perfect for that. (Btw, people are working on porting Cyanogen to the G2 but why let your dollars encourage the carriers and their shennanigans?)

Security is more than bug fixing. Security fixing/updating, the thing most people are exposed to, is reactive security . However, a large area of security work is proactive where defensive abilities are put in place to try and catch problems before they happen, or make classes of vulnerabilities unexploitable. This kind of security is what a lot of people don t understand, and I think it s important to point out so the distinction can be clearly seen. In the Linux kernel, there s yet another distinction: userspace proactive security and kernel proactive security. Most of the effort in kernel code has been protecting userspace from itself (things like Address Space Layout Randomization), but less attention has been given to protecting the kernel from userspace (currently if a serious enough flaw is found in the kernel, it is usually very easy to exploit it). One project has taken great strides with proactive security for the Linux kernel: PaX and grsecurity. There hasn t been a concerted effort to get its pieces upstream and it s long overdue. People are starting to take proactive kernel security more seriously, though there is still plenty of debate. While I did my best to push some userspace protections upstream earlier in the year, now it s time for kernel protections. What to help? Here is the initial list of things to do. Dan Rosenberg has started the information leaks discussion, and I ve started the read-only memory discussion. Hopefully this will go somewhere good.

When doing porter uploads, one must not forget to pass -m"My Name <my@email.addr>" to dpkg-buildpackage, e.g. with --debbuildopts for cowbuilder. Thanks Aur lien and sorry to everyone who got the upload mails. (How are the rules for sponsored uploads? I get conflicting info on these, and indeed, the one I sponsored never showed up on my QA DDPO page )

Debian mailing lists' standard policy is to allow posts from unsubscribed users. That of course fits our standards for opened development but has the drawback of allowing a lot of spam to reach our mailing lists. This spam also clutters the mailing lists archives. However, 1.5 years ago, listmasters established a well-working system to clean out spam from the mailing lists. It basically works in four steps:

1. first, spam can be "reported" by anyone meeting it. While you are reading the mailing lists, reporting spam is as simple as bouncing the offending mail to a dedicated address: report-listspam@lists.debian.org. I, for instance, use a mutt macro to set ESC-L as a "report spam" hotkey. Spam can also be reported when browsing the mailing lists archives and clicking on the "Report as spam" button;
2. when a given mail has been reported as spam more than five times, it is then "nominated" for review;
3. nominated mails must the be reviewed by volunteers in order to confirm they really are spam. The review algorithm is complex but, in short, when a mail has been confirmed as spam by 3 reviewers without being qualified as "ham" by anyone else, then it is considered spam;
4. such mails are then removed from the mailing lists archives.

Steps 2 and 4 are performed by a weekly script that runs on Sunday mornings (5:00 UTC, IIRC) How can you help? Process nominated spam We need more reviewers for step 3. At this moment 2 people are doing this on a regular basis (every Sunday with the morning coffee). Some others are working on this in a less frequent way. It seems that 1 or 2 more regulars reviewers would guarantee that we can cope with the nominated spam incoming flow. Only Debian developers can do that step. They should read instructions, ask for a password, then start reviewing. The first 1 or 2 people joining this will have a quite big amount of nominated spam to process, for their first effort. However, as most of these have been classified as spam by at least the two people doing this for months....the amount will slowly decrease. I, for instance, now only spend 5-10 minutes reviewing the nominated mails every week. Report spam Everybody can work on reporting spam. You can start by doing this on mailing lists you read by just bouncing spams to the spam report address; If you want to do more, you can join existing spam cleaning efforts, such as the one running for debian-boot or the one running in debian-l10n-french (the latter probably more for French speakers). Or you can of course start a similar effort for your pet list. To be efficient, this effort needs to gather 5-6 people who will process the mailing lists archives systematically. Let's clean out these archives!

Prioritising packages for the Debian Edu / Skolelinux DVD, which is supposed provide a school with all the services and user applications needed on the pupils computer network has always been hard. Even schools without Internet connections should be able to get Debian Edu working using this DVD. The job became a lot harder when apt and aptitude started installing recommended packages by default. We want the same set of packages to be installed when using the DVD and the netinst CD, and that means all recommended packages need to be on the DVD. I created a patch for debian-cd in BTS report #601203 to do this, and since this change was applied to the Debian Edu DVD build, we have been seriously short on space. A few days ago we decided to drop blender, wxmaxima and kicad from the default installation to save space on the DVD, believing that those needing these applications are few and can get them from the Debian archive. Yesterday, I had a look what source packages to see which packages were using most space. A few large packages are well know; openoffice.org, openclipart and fluid-soundfont. But I also discovered that lilypond used 106 MiB and fglrx-driver used 53 MiB. The lilypond package is pulled in as a dependency for rosegarden, and when looking a bit closer I discovered that 99 MiB of the 106 MiB were the documentation package, which is recommended by the binary package. I decided to drop this documentation package from our DVD, as most of our users will use the GUI front-ends and do not need the lilypond documentation. Similarly, I dropped the non-free fglrx-driver package which might be installed by d-i when its hardware is detected, as the free X driver should work. With this change, we finally got space for the LXDE and Gnome desktop packages as well as the language specific packages making the DVD more useful again.

My recent experience with Mozilla ThuderBird, IceDove. With KDE 4.x, the KDE team took a radical step of ripping apart most of the stuff and rethinking many of the designs. Quite a bold move. Many people appreciated KDE's efforts to start afresh while others moved away from KDE. For some reasons, I decided to stick to KDE. Maybe it was because of the awesome flexibility KDE provided provides to customize the DE to one's personal taste. Or maybe because I was too used to the KDE way of doing things. I stayed with KDE while 4.0 was released and stayed with it up till very recently. I started to lose my patience with the PIM Suite. I know the PIM team is also going to some very radical changes which will bring very innovative stuff later. But, at the moment, the KDE PIM suite is very broken. Broken not in the first impress, but broken when you regularly use it. It leaks memory like anything and keeps doing lots and lots of I/O. I hope some day the KDE team decides that they do need a core team, a core team that could take care of important tasks, making sure that the imporatant tasks are Continuously Usable. It takes time to earn the reputation but it takes a lot lesser time to lose it. Anyways, having been a KDE (PIM) user for long, I had been bearing the PIM torments up until, I recently saw a colleague using IceDove. There was a time when the Mozilla suite was in itself terribly slow. But things seem to have changed a lot. Both, the browser and the PIM suite, have improved a lot lot in terms of functionality and performance. Performance is very important. What good is a feature if its performance is terrible and it hinders the usability. Some of the things that really impressed me were:

• Indexing
• Organization

Indexing: Nepomuk might be good one day but that day is yet to be seen. I patiently wait for that day to come. Well, don't have much choice. Have been patiently waiting for it since KDE 4.0 was released. KDE has great ideas with Nepomuk which is good. But realisticly, what all do you want to see indexed ? There has to be a realistic line drawn. The browsers already have indexing for the history. Amarok, the player, also does indexing. It can tell you when your last favorite song was played. For pictures, I have the awesome KPhotoAlbum that cannot be beaten by anything. But above all, the most important thing to index is your conversation. Those emails that you send daily. And indexing is no good if you can't find the information you need, later. IceDove has filled that place. It does an excellent indexing (in terms of performance) and presents a very user intuitive way to narrowing it down to, when looking for a particular information. Organization: The other great thing about KDE PIM is its ability to break down its applications into small parts and glue them together into a new, well integrated application. Yes, Kontact. It is used to be the best PIM application. Used common libraries to make the suite more efficient than the rest. But all this was used to be. Today, kpart itself might be Kontact's problem. We have different applications glued together that if built with a common design, could have benefited a lot more. Take for example: kmail, knode and akregator. They are all very important parts of the PIM suite. Yet all three are different. The only thing common amonst them is is that they are available from the same Kontact shell. kmail has a different navigation. akregator has very nice aggregated folders but the same cannot be available in knode and kmail. knode, while still okay, has been rotting for some time. So it was time to see how IceDove performed when testing it up against this Use Case. The good thing about IceDove is is that it has one single uniform interface to most of the PIM needs. I can use the same window and the same interface for all my rss feeds, my emails and my leafnode newsgroups. That has a big benefit in itself. I have only 1 integrated interface to look at and only 1 interactive method to learn. There aren't different keystrokes for different applications. All is one in IceDove. And I think one of the main reasons for Mozilla's success is its plugin architecture. It is very difficult to satisfy everyone's needs. In the same way, it is very difficult for one group of developer's to be able to innovate differently. This is where Mozilla rocked. They provided a solid foundation with basic standard interface and let new fresh minds to do the rest of the innovation. Turns out it has worked well. So, with my PIM needs satisfied, I thought KDE was serving as nothing but just a mere shell. So, now was the right time to do the thing I always thought of doing. Switch to GNOME. GNOME looks elegant at first look but that is it. I wanted to take a screenshot of an application to report a bug. I fired up PrntSc key to let the screen capture utility pop-up. It did not have the opiton to select just the application window. Hmmm! Time to return back to KDE and use the Mozilla PIM suite and hope the KDE PIM team learns and does the right thing.

For some reason, 60G of free space on my notebook suddenly turned into 60G of movies. Bownian motion, perhaps. Unfortunately, in this particular case, I absolutely don't like to let a movie (or book, for that matter) unfinished, so I watched Fear and Loathing in Las Vegas (directed by Terry Gilliam, starring Johnny Depp; so far, so good) from start to finish. After the first half hour I started wondering if the plot is going to start. The reptile party in the hotel lobby was a short entertaining interlude, but to the end, no plot manifests. I'm left wondering what this was all about. Quantum of Solace on the other hand is good entertainment. Probably not one of the best, but still worth watching if you like her majesty's secret agent. Going back to movies I've seen before is something I like to do as well, so having another evening in company with the unforgettable Leon (the professional) was time well spent. I didn't give up on Terry Gilliam yet and met The Fisher King. While the story is completely different, it does feel a bit similar to 12 Monkeys in terms of set design and atmosphere. Speaking of Terry Gilliam: I'm curious how his current attempt at The Man Who Killed Don Quixote turns out; so far I've only seen him losing this movie in the documentary about the aborted last attempt: Lost in La Mancha. On the funny side, Dogma and especially The Wedding Crashers were worth watching. If you have to pick one, it's the latter one. Dogma feels a bit forced in some places. Returning back to the opening theme: seeing Inland Empire was another evening that left me confused. The main story is interesting enough, but I have to admit that I just couldn't follow where all the other sub plots tie in, or if they even are supposed to. And I'm not only talking about the rabbit family (those felt a bit like the Middle of the Film in The Meaning of Life and were quite in order.)

Thanks to a colleague, we hereby present the multi-strace:

The Russian l10n team is the first team to reach one of our translators' Grails: all debconf templates translations are now fully translated in Russian, in testing. Russian is the first language to reach this ever (I formerly announced 100% completion for a few languages, but that was in unstable: we never tracked testing up to now). It means that, in theory, on can do a full installation of all packages in squeeze (in case some crazy folks want to do this!) and always be prompted in Russian. French and German are lagging very close to this, the only missing bit being the udev package, which had two consecutive urgent uploads that couldn't include a full translation coordination. For these, we're indeed waiting for another udev upload, but I already nagged Marco about this..:-) Portuguese should make it, though I still have to track 7 packages for it. Swedish is only missing 2 packages. I'm less sure that Czech and Spanish (the other two teams working hard on 100% completion) will make it.

/etc/fstab files tend to be an unreadable mess of unaligned fields.

# /etc/fstab: static file system information.
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
proc            /proc           proc    defaults        0       0
/dev/mapper/benz-root /               ext3    errors=remount-ro 0       1
/dev/sda1       /boot           ext3    defaults        0       2
/dev/mapper/benz-home /home           ext3    defaults        0       2
/dev/mapper/benz-swap_1 none            swap    sw              0       0
newton:/home        /nfs        nfs defaults,soft,intr,users    0 0
/dev/scd0       /media/cdrom0   udf,iso9660 user,noauto     0       0

Let's remove some whitespace in the third line:

#<filesystem> <mountpoint>   <type>  <options>       <dump>  <pass>

And then pipe everything from line 3 to the end through column -t:

# /etc/fstab: static file system information.
#
#<filesystem>            <mountpoint>   <type>       <options>                 <dump>  <pass>
proc                     /proc          proc         defaults                  0       0
/dev/mapper/benz-root    /              ext3         errors=remount-ro         0       1
/dev/sda1                /boot          ext3         defaults                  0       2
/dev/mapper/benz-home    /home          ext3         defaults                  0       2
/dev/mapper/benz-swap_1  none           swap         sw                        0       0
newton:/home             /nfs           nfs          defaults,soft,intr,users  0       0
/dev/scd0                /media/cdrom0  udf,iso9660  user,noauto               0       0

Thanks to SP8472 for bringing this to my attention.

My fellow Republicans and I got a nice early Diwali present on Tuesday. ~65 seats in the House ~6 in the senate, ~18 governorships and control of ~20 state legislatures. (Some races haven't been formally called yet.) A few disappointments though. no majority in the Senate and Harry Reid and Barney Frank were not defeated. Still overall its an election to celebrate. Here are a few thoughts. The Election System Is An Embarassment Once again there are several races where the outcome is in doubt because of technical problems with voting. Whatever politics we have, can we agree that it is damn stupid that we still can't get something as simple as counting ballots right? Prediction: The 2012 Republican presidential candidate will come from the Tea Party. While I may be a Republican, I am also a Cosmopolitan East Coast Elitist so the Tea Party didn't move me much. There were a couple of cases (NY and DE atleast) where their candidates were subpar and cost us victory but the best of the new Republicans also came out of this movement. I would rather vote for one of them than Romney, Huckabee, McCain, Gingrich etc. (With the exception of Sarah Palin who is just too polarizing I'm afraid.) The Tea Party itself will subside in a few years as all populist movements do. (Ask the Hope'n'changers.) Young People Are Dumb Part of leftist mythology is the idea that one day "the kids" are going to rise up against the mean old corporate overlords. It ain't going to happen. There was a huge upsurge in first time voters in 2008 but that was because Obama was a novelty act not because they had suddenly discovered the merits of Keynesianism. Now Obama is old and busted so they won't vote anymore. (Not until they have kids and a mortgage sometime in the future.) They'll spend time make cool ironic signs for the Rally to Promote Comedy Central but even the prospect of legalizing pot won't get them to the polling booths in sufficient numbers to defeat the oldies. Why? Because young people are dumb. Prediction: War With Iran If you're President Obama what do you do now? Bipartisanship? Yeah right. Look at what happened to Rep. Cao of Louisiana, the only Republican to vote for Obamacare and one of only two GOP congressmen to lose on Tuesday. Obama actively campaigned for his challenger. Any of the new Republicans who try and compromise with the President will end up under the bus sooner or later as they well know. The President could go full retard with the socialism (progressivism, liberalism whatever. It's all the same.) All his initiatives will be shot down but he can claim to be a martyr to the party of No. (Who will be joined in the Senate by Lieberman and whatever centrist Democrats who are up for reelection in 2012.) But this seems to be the path to a one-term presidency to me. Maybe that's what he wants. He could retire comfortably to the deanship of some Ivy League law school and make a lucrative killing on the rubber chicken circuit. But lets assume he wants to get back in the White House in 2012. One way to get back in touch with the voters is to start getting serious about foreign policy which has either been a joke (Remember the reset button?) a travesty or just copied from the Bush administration. In particular the Obama administration has completely failed in stopping Iran from gaining nuclear technology, sponsoring terrorism in Iraq, Afghanistan and Lebanon, and oppressing its own people. War with Iran would enhance his standing with Conservatives (hey it works for me!) most liberals would go along with it and the extreme Code Pink type loonies can safely be ignored. I say it happens before November 2012.

About a month ago, at the end of August, I finished a 10-day Vipassana meditation course. This was probably one of the more extreme activities I have ever participated in, and it's well worth a blog post. I found out about the course in a ridiculously trivial way, by following a link in a Reddit comment (I wonder if I could track down that comment...). The website referenced by the comment contained (almost) no new-agey mumbo-jumbo and looked legit in general. I also appreciated the fact that this was a relatively serious and intensive course, which should be enough to form a reasonably founded opinion on meditation in general. When learning sporadically, in small bits, it is hard to say whether the material itself is no good, or you just do not understand it yet and need to study more; I was quite skeptical towards the practice, but wanted to give it fair trial. I was quite impressed that there was a course available in Lithuania, a few hours by train from where I live. Finally, the course was free (by tradition Vipassana is taught without monetary compensation). I went ahead and signed up for a standard introductory 10-day course. Vipassana (also called insight meditation) is a meditation technique allegedly coming back from Gautama Buddha himself (well, all the traditions like to claim that origin...). It has been experiencing a renaissance, partly due to an organization set up by a meditation teacher S. N. Goenka which has founded meditation centers all over the world. The essence of Vipassana is a method of teaching your mind to act instead of merely reacting to outside stimuli such as pain, anger of other people or hardships. The technique provided is to monitor your sensations, watch them with equanimity and suspend any reaction, and slowly it sinks in that all things are temporary and you can choose what to react to and how. That's the basic idea; I don't want to go deep into exposition of Vipassana as there is plenty of information about that (also at the same time the information is almost useless because you have to experience it to really understand it). Instead, I will sure some of the more personal remarks and experiences. The course took place in an cottage near Kretinga, housing around 50 people. The living conditions were absolutely great, and this would have been a true holiday if not for the regime and the rules. The day would start very early, at 4 AM (ending at around 9:30 PM), and we'd do about 12 hours of meditation in total inbetween, with occasional breaks, but the most gruesome part is that no communication at all was allowed with anyone except the manager of the camp (and the teacher during a specified time). That gets to you after a few days, but still, there are people around you, so it's bearable; on the other side, the senses (touch, smell) become very acute. The food was taken care of by volunteers (students who have finished the course), and I could say it was quite good actually. The goal, of course, was to keep the environment from interfering with meditation, and I think it was done very well. The presentation of the actual technique also did not disappoint. Each day a new step would be introduced, with the whole day to practice before moving on. Looking back, the pacing worked well. Every day in the evening there were lectures on theory. They were somewhat informative, but also sometimes annoyingly primitive and verbose. The little mistakes, holes in logic and slight doses of mumbo-jumbo almost drove me nuts by the middle of the course, until I learned not to react to them. Despite the shortcomings of presentation, the simplicity and lucidity of the practice and lightness of theoretical baggage really appealed to me. The whole organisation in general gave the impression of complete transparency, with no strings attached, no financial interests in play and no agitation for further involvement. While there are many meditation techniques, I am happy that I've gone with Vipassana in particular for one reason: Vipassana meditation uses no artificial tools to help you concentrate, such as mantras, visualisations, counting, etc. Such methods make concentration much easier, but they also create a dependence on the methods themselves. I appreciate the purity of Vipassana in this regard. Not everything in the practice is roses though. First, a particular worldview supported by Vipassana is presented as "scientific" and supposedly "the right way to live", and if even if you don't agree, "the truth" of Vipassana will become obvious to you with time. This is a distinctive mark of ideologies. The thing that makes everything break is the reincarnation, which is generally assumed as true, even if it's not shoved down the student's throat. Every rationalization of a world view has a "white lie" (usually related to the concept of eternity somehow) at the core, and in this context reincarnation is it. Given what we know now about the relationship between the mind and the body, reincarnation is absolutely ridiculous and completely untenable, and that breaks all "moral" reasoning for practicing meditation as opposed to doing something else. I'm not the first to bring up this problem, there's plenty of material about it online. The second slippery idea is that the concept of "negative happiness" is central in Vipassana, that is: happiness is defined as lack of suffering. This point is not that unreasonable actually, from personal experience I could indeed call the state of non-suffering happiness, but there is definitely something missing when compared to the state of mind when I'm truly, positively happy. Whether you consider that something important or not, it's definitely there. Third, Vipassana suggests that it is only you who is at fault when your inner state is disturbed; the world affects you as much as you let it, and in order to find happiness you should learn to control the "interface" rather than the outside world, which will usually not behave as you want. This may be correct, but sometimes it could just be easier to fix the world rather than change yourself, and I have the suspicion that it this is much more frequently the case in the modern world compared to the pre-modern environment. (This is one of the points which break when reincarnation is rejected, as with reincarnation you have the whole eternity to fix yourself.) While the complaints are serious, they do not mean complete rejection of Vipassana but rather integration among the various other techniques available. The actual meditation technique is still great for gaining and maintaining mental composure, reducing anxiety and improving well-being. Now, if I could only get used to meditating regularly... If you have the chance, I highly recommend you try out an introductory 10-day course. There is little to lose and a lot to gain. And if you do go ahead, I have two hints. First, do not even think about leaving the course before it's over. Any hardships are temporary and will be overcome quickly. Second, use the time with the teacher, those interactions were very helpful to me. Also, given the "noble silence", talking to someone once in a while will not hurt.

During my recent trip to Massachusetts for CalConnect XIX I passed back via New York, surfing a few nights on the couch at the Washington Cube Garden¹. This was just enough time for my new Davis VantagePro2 to arrive by UPS ground (phew!) causing me much consternation, as the box was about twice as wide as I expected it to be. On opening it I discovered that the reason for the size was the physical dimensions of the rain gauge, leading me to realise that I had actually bought exactly what I wanted: a high quality weather station. Also leading me to wonder how the hell I was going to get it halfway around the world with me the next day. Unfortunately that extra wide carrying case was in no way going to fit inside my suitcase. Fortunately it came with a handle. And those wonderful people at Air New Zealand gave me a Koru Gold upgrade for a 50th birthday present, so it was time to put it to the test... Travelling on the subway out to JFK was it's own little adventure, but I made it out to terminal 5 and wandered around a bit before concluding I needed to be in terminal 7. Once I was in the right place I found the fancy people's check in counter and the nice lady there was only too happy for me to carry it to New Zealand, though she did aver that perhaps the security scanners would be dubious. Not a hitch there though: I guess they're used to letting everything through that's not specifically denied, so weather stations are fine because nobody would be silly enough to even think of doing that... Being Koru Gold really helped at this point though, because it enabled me to queue jump use the priority boarding line when getting onto the plane, and every airline in the US seems to have gone all-out for this "checked baggage costs more" approach, so the planes all fly with empty baggage holds and totally overstuffed overhead lockers. If this progresses I shouldn't wonder that the planes will get so top-heavy they roll over, but through the miracles of priority boarding I was able to commandeer all the locker space I needed before anyone else had even made it past row 40. I confess that I was a little worried that it wouldn't be quite such plain sailing getting through LAX. I mean we all know what reputation this delightful little exit port has. In the event it was totally anitclimactic. If it has a handle on it, and it's about the same size and shape as a largish carryon then it's fine. There have been no recorded stabbings with wind vanes anyway - at least on aeroplanes - so clearly they are perfectly safe. Finally back in my country of birth, thinking I was home free, I breathed a sigh of relief transferred my bags and nipped off to the Koru lounge for a much needed shower and change of clothes after I snuck the station through another scanner. By now the handle had broken, so it was looking decidedly more 'box' like and less 'carryon' like, and when I eventually rolled up at the gate with my boarding pass the attendant made a valiant attempt to relieve me of it, but she was too slow and I danced around her and swanned up the gangway to take my seat. I was slow, this time, having missed the first boarding call, and all overhead lockers were crammed full. There was no seat in front of me to put the weather station under. Fortunately a nearby cupboard proved fit for purpose and it was secreted there and we were underway. At this point the guy two seats over said "didn't I see that weather station in New York?". Of course being bright yellow does make it a little conspicuous I guess, and I'm not renowned for my own inconspicuousness myself. So that's how I got my new weather station home, and you can see right away that doing things the easy way would never even occur to me. Possibly this is why I run Linux: there just isn't enough challenge when you run a different operating system. On Windows someone has always written a program for your new weather station, and a Mac user wouldn't be seen dead with it because it's black and beige and comes in a bright yellow box! So now I had to find some code to write. I looked around and very rapidly discovered that some software called vproweather was available, which would happily talk to my weather station and download the data from it, write web pages, stuff it into a MySQL database and so forth. All, obviously, far too easy, so I thought aha! I could convert that to PostgreSQL, and then I would be happy and relax with the thought of a job well done. So I downloaded the code in question an impressive several hundred kb of C code, cranked open a text editor and started to crawl through it and see where I could add the magic pixie dust to make it work with PostgreSQL. But it was not to be. My eyes glazed over or scales fell from them or something, and I realised that this particular edifice was beyond my help. Back to square one. I downloaded the Vantage Serial Protocol docs from the Davis Instruments website did some quick googling and decided I should write a module to talk this protocol. And I should write it in Perl (it was a Thursday, and Perl always seems more sensible on a Thursday, and not because of Douglas Adams, either). I beavered away into the night. Rain threatened, so I raced outside and installed the new weather station on the roof in the hopes that it's magnificent rain collector would collect. I raced back inside and was able to count the first (and so far, only) tip of the bucket. And so I now release upon the world Davis::VantagePro my first perl module. Well, the first one I care to share with the world, anyway. I have stressed it with very little testing, lumbered it with no planning or design whatsoever and so I feel it only fair that I should cast it upon the world with very little thought for it's survival or existential goals.

¹ I seem to recall one night we came up with some fantastic names for Micah & Biella's Washington Square apartment, but I couldn't remember any of the good ones and had to slap a new one on there. I should also mention my stay at the Acetarium in Boston, which was a fabulous few days, but it would pad this blog post needlessly, and I've already done that.

In our first few years, Ubuntu experienced explosive growth, from zero to millions of users. Because Ubuntu is an open project, these people don t just use Ubuntu, but can see what s happening next and influence it through suggestions and contributions. The volume of suggestions quickly became unmanageable through ad hoc discussion, because the volume of feedback overwhelmed the relatively few people who were actively developing Ubuntu.

In order to better manage user feedback at this scale, Ubuntu Brainstorm was created in 2008. It s a collaborative filtering engine which allows anyone to contribute an idea, and have it voted on by others. Since then, it s been available to Ubuntu developers and leaders as an information source, which has been used in various ways. The top ideas are printed in the Ubuntu Weekly Newsletter each week. We experimented with producing a report each release cycle and sharing it with the developer community. People have been encouraged to take these suggestions to the Ubuntu Developer Summits. We continue to look for new and better ways to process the feedback provided by the user community. Most recently, I asked my colleagues on the Ubuntu Technical Board in a meeting whether we should take responsibility for responding to the feedback available in Ubuntu Brainstorm. They agreed that this was worth exploring, and I put forward a proposal for how it might work. The proposal was unanimously accepted at a later meeting, and I m working on the first feedback cycle now. In short, the Technical Board will ensure that, every three months, the highest voted topics on Ubuntu Brainstorm receive an official response from the Ubuntu project. The Technical Board won t respond to all of them personally, but will identify subject matter experts within the project, ask them to write a short response, and compile these responses for publication. My hope is that this approach will bring more visibility to common user concerns, help users understand what we re doing with their feedback, and generally improve transparency in Ubuntu. We ve already selected the topics for the first iteration based on the most popular items of the past six months, and are organizing responses now. Please visit brainstorm.ubuntu.com and cast your votes for next time!

I ve been looking at git dpm. It s a tool for managing Debian packages in git. The description promises the world: use of git to maintain both a set of patches to some upstream software along with the history of those patches both at the same time. It also promises the ability to let me share my development repositories. The overhead is much less than topgit. My feelings are mixed. It does deliver on its promise to allow you to maintain your upstream patches and to use git normallyish. It produces nice quilt patch series. In order to understand the down side, it s necessary to understand a bit about how it works. It maintains your normal branch roughly the way you do things today. There s also a branch that is rebased often that is effectively the quilt series. It starts from the upstream and has a commit for every patch in your quilt series containing exactly the contents of that patch. This branch is merged into your debian branch when you run git dpm update-patches. The down side is that it makes the debian branch kind of fragile. If you commit an upstream change to the Debian branch it will be reverted by the next git dpm update-patches. The history will not be lost, but unless you have turned that change into a commit along your patches branch, git dpm update-patches will remove the change without warning. This can be particularly surprising if the change in question is a merge from the upstream branch. In that case, the merged in changes will all be lost, but since the tip of the upstream branch is in your history, then future merges will not bring them back. If you either also merge your changes into the patches branch, or tell git dpm about a new upstream, then the changes will reappear at the next git dpm update-patches. The other fragility is that rebasing your debian branch can have really unpleasant consequences. The problem is that rebase removes merge commits: it assumes that the only changes introduced by merge commits are resolution of conflicts. However, git dpm synthesizes merge commits. In particular, it takes the debian directory from your debian branch, plus the upstream sources from your upstream branch, plus all the patches you applied and calls that your new debian branch. There s also a metadata file that contains pointers to the upstream branch and the quilt patch series branch. In addition, debian/patches gets populated. Discarding this commit completely would simply roll you back to the previous state of your patches. However, this loses history! There is no reference that is left pointing to the patches branch; the only reason it is still referenced is that it s a parent of the commit rebase is throwing away. This isn t really all that unusual; if you merged in any branch, deleted the branch and rebased away both the merge and the commits from the branch, you d lose history. I do find it a bit easier to do with git dpm. However, it s more likely that you ll manage to rebase and keep the commits from the patches branch. In particular, if the horizon of your rebase includes a merge of the patches branch,then for every patch that involves changes to the debian branch from the point where rebase starts rewriting history, you will see a commit included by your rebase. My suspicion is that this commit is more likely to generate conflicts than usual because it s a patch on top of the upstream sources being applied to a partially patched debian branch. However I have not fully worked through things: it may be that such a patch is exactly as likely to cause conflicts as it would in a git dpm update-patches operation. If that succeeds and you do not manage to fail to pick one of these commits, you ll end up with roughly the same debian sources you started with. There are two key exceptions: the git dpm metadata file is not updated and your debian/patches is not consistent with your sources. The first means you re in the state of the previous paragraph: git dpm update-patches will blow away your changes. The second means that any source package you produce is going to be kind of funky if not out right broken. It is possible to recover from git dpm plus rebase. I think if you git dpm checkout-patches, then cherry pick the patch related commits from your debian branch that were introduced by the rebase, then git dpm update-patches, you ll be in a sane state. Obviously you could also recover by finding the pre-rebase commit and resetting there. I m assuming though that you actually needed to rebase for some reason. Of course git dpm does involve a space penalty. You generate a commit for every patch in your quilt series for every version of the upstream sources you deal with. Also, you will introduce and store every version of the quilt series directly in your debian directory. The extra commits probably aren t a big deal: the blobs probably delta compress well. I m not sure though that the quilt patch blobs will compress well with other things. In my opinion the ability to get nice quilt patches is well worth the space penalty. In conclusion, git dpm is a sharp tool. Actually, I think that s understating things. Git dpm involves a bunch of rotating blades. You stick your face and foot between the blades, and if you use it correctly, there s exactly enough clearance that nothing gets damaged and you get some fairly neat results. Other outcomes are available. I m still trying to evaluate if it is worth it. I think the tradeoff might be different for something like the krb5 package which is maintained by two very experienced maintainers than say for Shibboleth which seems to involve several more maintainers. I m not sure that there is much git dpm can do to make things better. Even detecting loss-of-foot events might be kind of tricky.

Using the apt-xapian-index catalogedtime plugin apt-xapian-index has recently gained a catalogedtime plugin, which stores the timestamp of when the package was first cataloged. At first glance this does not mean much, so I'll explain with an example. aptitude can show a list of "New Packages": those packages that it sees now for the first time. It is a very useful feature, but you cannot ask it "what packages became available during last week?" With the catalogedtime plugin, the apt xapian index can now answer that question. You can also sort package results by "newness", or implement a user interface with some sort of "newness timeline" package filter option. Of course you will find documentation about it in /var/lib/apt-xapian-index/README, since the index is self-documenting. As usual in my apt-xapian-index posts, here is a code example:

#!/usr/bin/python
# coding: utf-8
# Show packages that were NEW in the last N days
# (C) 2010 Enrico Zini <enrico@enricozini.org>
# License: WTFPL version 2 (http://sam.zoy.org/wtfpl/)
import axi
import xapian
import time
import sys
class App:
    def __init__(self, days):
        self.db = xapian.Database(axi.XAPIANINDEX)
        # Read the contents of /var/lib/apt-xapian-index/values
        self.values, self.descs = axi.readValueDB()
        # Find the value ID for the catalogedtime timestamps
        self.cattime_vid = self.values["catalogedtime"]
        # Filter packages first seen more than than X days ago
        self.query = xapian.Query(xapian.Query.OP_VALUE_GE, self.cattime_vid,
                    xapian.sortable_serialise(time.time() - 86400*days))
    def main(self):
        # Perform the query
        self.enquire = xapian.Enquire(self.db)
        self.enquire.set_query(self.query)
        # Sort by cataloged time
        self.enquire.set_sort_by_value(self.cattime_vid, True)
        # Print the first 200
        matches = self.enquire.get_mset(0, 200)
        for m in matches:
            name = m.document.get_data()
            val = m.document.get_value(self.cattime_vid)
            fval = xapian.sortable_unserialise(val)
            tm = time.localtime(fval)
            timestr = time.strftime("%c", tm)
            print "%s %s" % (timestr, name)
if __name__ == "__main__":
    try:
        days = int(sys.argv[1])
    except:
        print >>sys.stderr, "Usage: %s days" % sys.argv[0]
        sys.exit(1)
    app = App(days)
    app.main()
    sys.exit(0)

Enjoy!

Last week, I wrote a couple of patches to add support for transparency when Emacs is displaying GIF images. Until now, it was displaying the color used to define transparency in the file data. Now it displays the image correctly by using the frame color as the transparency color, like it's done for other image formats. The patches have not been merged yet, but will probably be soon.

I recently added a solid-state drive to my desktop computer to take advantage of the performance boost rumored to come with these drives. For reliability reasons, I've always tried to use software RAID1 to avoid having to reinstall my machine from backups should a hard drive fail. While this strategy is fairly cheap with regular hard drives, it's not really workable with SSD drives which are still an order of magnitude more expensive.

The strategy I settled on is this one:

• continue to have all partitions (/, /home and /data) on my RAID1 hard drives,
• put another copy of the root partition (/) on the SSD drive, and
• leave my /tmp and swap partitions in RAID0 arrays on my rotational hard drives to reduce the number of writes on the SSD.

This setup has the benefit of using a very small SSD to speed up the main partition while keeping all important data on the larger mirrored drives.

Resetting the SSDThe first thing I did, given that I purchased a second-hand drive, was to completely erase the drive and mark all sectors as empty using an ATA secure erase. Because SSDs have a tendency to get slower as data is added to them, it is necessary to clear the drive in a way that will let the controller know that every byte is now free to be used again.

There is a lot of advice on the web on how to do this and many tutorials refer to an old piece of software called Secure Erase. There is a much better solution on Linux: issuing the commands directly using hdparm.

Partitioning the SSDOnce the drive is empty, it's time to create partitions on it. I'm not sure how important it is to align the partitions to the SSD erase block size on newer drives, but I decided to follow Ted Ts'o's instructions anyways.

Another thing I did is leave 20% of the drive unpartitioned. I've often read that SSDs are faster the more free space they have so I figured that limiting myself to 80% of the drive should help the drive maintain its peak performance over time. In fact, I've heard that extra unused unpartitionable space is one of the main differences between the value and extreme series of Intel SSDs. I'd love to see an official confirmation of this from Intel of course!

Keeping the RAID1 array in sync with the SSDOnce I added the solid-state drive to my computer and copied my root partition on it, I adjusted my fstab and grub settings to boot from that drive. I also setup the following cron job (running twice daily) to keep a copy of my root partition on the old RAID1 drives (mounted on /mnt):

nice ionice -c3 rsync -aHx --delete --exclude=/proc/* --exclude=/sys/* --exclude=/tmp/* --exclude=/home/* --exclude=/mnt/* --exclude=/lost+found/* --exclude=/data/* /* /mnt/

Tuning the SSDFinally, after reading this excellent LWN article, I decided to tune the SSD drive (/dev/sda) by adjusting three things:

• Add the discard mount option (also know as ATA TRIM and introduced in the 2.6.33 Linux kernel) to the root partition in /etc/fstab:

/dev/sda1  /  ext4  discard,errors=remount-ro,noatime  0  1



• Use the noop IO scheduler by adding these lines to /etc/rc.local:

echo noop > /sys/block/sda/queue/scheduler
echo 1 > /sys/block/sda/queue/iosched/fifo_batch



• Turn off entropy gathering (for kernels 2.6.36 or later) by adding this line to /etc/rc.local:

echo 0 > /sys/block/sda/queue/add_random

Is there anything else I should be doing to make sure I get the most out of my SSD?

Today I bought two Ikea boxes. Possibly the best 8 I have ever spent - I still have a load of bits of paper to sort out, but now they're all hidden from sight in a sleek, black and chrome rivet-effect container. More than simply reclaiming my sofa from the piles of notes, they seem to point to a higher truth; they fit their purpose in a way that is hard to articulate, like some platonic Form. Taking complexity, and making it manageable; but more than that, elegant. At some point, I lost the ability to explain to my friends and family what it is I spend my spare time doing. I feel isolated; I cannot express what motivates me, or even what it is I do. In fact, sometimes I can't even describe it to myself. Perhaps I never could. I don't believe that you can buy happiness; but if I surround myself with metaphors, maybe I can buy clarity.