My current employer has some internal program that it allows employee to devote two days a year for volunteering work.
It doesn t matter what it is (work for hospice, planting trees, helping in some foundation) so I thought about devoting this time for the Debian project. Either fixing RC bugs or finally give my packages enough attention and take care of their updates. Anyway the employer needs the certificate that I took participation in such event.
So the question is if it is possible to get some official document/certificate that I devoted some time to work on Debian? Bureaucracy has to be fulfilled unfortunately.

I just uploaded Debian Policy 3.9.4.0, which includes the implementation of the Technical Committee decision to make build-arch and build-indep mandatory targets and which significantly reworks the section on shared libraries. Nothing here is release-critical for wheezy, and uploads targeted for wheezy should not attempt to update packaging for a new version of Policy. I went into some more detail on debian-devel-announce. The version number probably should have been 3.10.0.0 given the build-arch and build-indep changes and the work on symbols files. Ah well. Here's the upgrading checklist:

2.4

New tasks archive section.

4.9

build-arch and build-indep are now mandatory targets in debian/rules.

5.6.26

New section documenting the Vcs-* fields, which are already in widespread use. Note the mechanism for specifying the Git branch used for packaging in the Vcs-Git field.

7.1

The deprecated relations < and > now must not be used.

7.8

New Built-Using field, which must be used to document the source packages for any binaries that are incorporated into this package at build time. This is used to ensure that the archive meets license requirements for providing source for all binaries.

8.6

Policy for dependencies between shared libraries and other packages has been largely rewritten to document the symbols system and more clearly document handling of shared library ABI changes. symbols files are now recommended over shlibs files in most situations. All maintainers of shared library packages should review the entirety of this section.

9.1.1

Packages must not assume the /run directory exists or is usable without a dependency on initscripts (>= 2.88dsf-13.3) until the stable release of Debian supports /run.

9.7

Packages including MIME configuration can now rely on triggers and do not need to call update-mime.

9.11

New section documenting general requirements for alternate init systems and specific requirements for integrating with upstart.

12.5

All copyright files must be encoded in UTF-8.

Some of my Romanian readers might know that for the last two years I've got involved in the skeptical movement to such a degree that I am a co-producer of a bi-weekly podcast on science and skepticism (in Romanian) called Skeptics in Romania . Some might even be regular listeners of the show.

(There isn't much to see now visually on the site, but me and the other people behind the project have some ongoing plans to change that.)

In spite of our modest site, up until now we had some successes, one of them being the publication of an article on us in a known Romanian printed publication and another being the invitation to a live show face to face with Oreste Teodorescu, a well known Romanian mysticist and woo promoter.

During that live show we managed to show a demonstration (video below, in Romanian) of how astrology gives the impression of working, without actually working, and, taking into account we had no prior TV camera experience and that it was a live show, I think we managed an honourable presence.


We also have a series of interviews in English with some really interesting people: Dr. Eugenie Scott, Prof. Christopher French, Prof. Edzard Ernst, Samantha Stein and others. We did these interviews at Denkfest 2011, in Zurich, and we integrated the translated (voice over) interviews in our podcast. The conlusion is that most of our activities revolve around the podcast, so let me tell you more about that.

The podcast has a somewhat fixed structure, it starts with a conversation between ourselves, then we have a segment on the history of science, technology, skepticism and woo, and then we have a segment called The dangers of not being skeptical . In this segment we present cases of people who lost their lives, their health, their money or any combination of the former because they were duped into some scam, science-y sounding non-science, unfounded claim or some other woo.

Having lost recently my brother-in-law to a form of cancer known as Hodgkin lymphoma, I became especially sensitive about miracle-cure claims for cancer, and this section of the show has lately seen its fair share of such cases. Honestly, if there could be a way to prosecute the irresponsible, ignorant and/or cynical people promoting all sorts of quack "therapies", especially for cancer*, I would really like to see it happen. But there isn't, and we're trying the best accessible approach, informing the public.

During my brother-in-law's last two years of his life, he went through lots of chemotherapy and radiotherapy sessions, repeated periods of hospitalisation, and lots of drugs. This is the best of what we currently have for treating and curing most forms of cancer, and too many times this isn't enough. I can't even imagine how stressful and discouraging it must feel when the best of what we have doesn't help.

Here is where the desperation and hopes of patients and their families meet the purely irresponsible cynical or ignorant promoters of woo and quack therapies. Because it takes either an ignorant or a really cynical (I really feel this word isn't enough) person to prey on the suffering of other people to make easy money under the false pretence of offering a cure.

It almost happened to my brother-in-law and his family, because they almost went for some herbal concoction promoted as a cancer cure on some forum, blog or page of a seller of this fake therapy. It was really hard for me to make them understand why using such a product it not advisable, not even in parallel with the medical treatment due its possible counter effect or interactions with the real medical treatment, without them getting the wrong idea that I wasn't trying to help. While trying to be brief and informative not to lose their attention, I told them how "natural" doesn't necessarily mean "good" (uranium, lead and Irukandji's venom are all natural), and how plants are drugs because they all contain chemical substances (and no, "chemical" does not mean "human made" or "artificial") which could interact with the medical treatment.


But most people don't even have the chance of having close by a person with a more science-leaning thought process and a skeptical mind. Those unfortunate people are the most vulnerable people and constitute the biggest chunk of the victims of baseless pseudo-cures or pseudo-treatments.

On our last show, I presented the case of Yvonne Main, a cancer suffering patient who mistook an invasive carcinoma for a cyst, and irdologist Ruth Nelson for a real healthcare giver.

Yvonne Main, died from an invasive carcinoma after seeking help from a iridologist, and delaying real medical treament for 18 months

Yvonne, after seeking medical advice from a person that essentially promotes the dead idea of guessing diseases by looking at the eyes**, used natural treatments for about 18 months and, after all this time, her carcinoma grew to a size of 10 to 11 cm, eating through her skull and causing damage which was later attempted to be countered through bone transplant from her ribs.

Ruth Nelson wasn't prosecuted in any way and continues her practice of quackery unharmed.

This is not the only case, nor even one case from a select few where woo and quackery lead to grave consequences for patients. There are many, many more; they're so many that even after splitting them in categories they seem too many per category, especially when you realise these are only the findings of, essentially, a single man:




This is part of what I have been doing in the last few years, instead of working on Debian. Is it a good thing? Is it a bad thing? Maybe it's good. I want to know what do you think?


* you will, most likely, never hear such a promoter of non-therapies say that there isn't just one cancer, and that, in fact, cancer is a name for a certain family of diseases which are all called cancer - that's a first sign that you might be dealing with quack
** probably in the line of thought that the eyes are the gates to the soul so they must tell something significant about health

I recently blogged about 3 languages reaching 100% translation for debconf templates in wheezy. Today, two more languages joined the club: French and Portuguese. We're still on our way to get seven or even eight complete translations in wheezy. Czech is now only missing "linux-latest" and Spanish waits for sysvinit (which I just NMU'ed) and nova (unfortunately an error in former translation was unnoticed and I discovered it quite late). David Pr vot is still trying to get Danish complete, by doing many fast update rounds and NMUs. I hope he succeeds in this.

AltOS 1.1 Bug fixes and some nice new features Bdale and I are pleased to announce the release of AltOS version 1.1. AltOS is the core of the software for all of the Altus Metrum products. It consists of cc1111-based micro-controller firmware and Java-based ground station software. We ve spent the last flying season chatting with people flying TeleMetrum and TeleMini boards and they came up with some great ideas to add to the system. AltOS Firmware Features and fixes There are bug fixes in both ground station and flight software, so you should plan on re-flashing both units at some point. However, there aren t any incompatible changes, so you don t have to do it all at once. New features:

• Apogee-lockout timer. For situations where the normal apogee determination algorithm could be fooled, we ve added a timeout value to prevent premature firing of the apogee charge. Normal flights won t need this, but a couple of users asked for this feature.
• RSSI value for Monitor Idle mode. The TeleDongle firmware has been updated to report signal strength information for data received from the altimeter. This allows the user to see how well the radio is working without having to switch to flight mode.
• Force the radio to 434.550MHz. This is useful with TeleMini devices where the only way to talk to the device is through the radio. If you don t know the frequency, it s really hard to make that work.

Bug fixes:

• Stale telemetry data reported when switching frequencies. TeleDongle would accidentally re-transmit old telemetry packets when the radio frequency was changed. This would be harmless except that when scanning to find the frequency used by an altimeter, you d appear to get packets at every frequency.

AltosUI Easier to use AltosUI has also seen quite a bit of work for the 1.1 release. There aren t any huge new features, but some activities are restructured to make them easier to navigate. And, of course, we ve fixed a bunch of bugs. New features:

• Configure Ground Station activity. This provides a way to set the default radio frequency for each TeleDongle. This replicates the frequency menu present in the Monitor Flight activity, but doesn t also bring up that giant window.
• Support the apogee lockout timer. This just adds another entry in the dialog for configuring the altimeter to configure the new timer. By default, the timer is disabled, allowing the apogee detection code in the flight computer to operate normally.
• Add imperial units option. When enabled, this uses imperial units (feet and miles) for all values on the screen and in the voice announcements.

User interface changes:

• Make the look-n-feel configurable. Java offers many different user interface styles on each platform. This exposes the available set and lets the user pick one. By default, we continue to use the native platform appearance.
• Add an Age element to the Monitor Flight UI. This shows how long it has been since the last valid telemetry packet was received, making it easy to tell when communications are lost.
• Change flight data downloading. Instead of selecting which to download and which to delete at the same time, the interface now has separate steps for downloading and then deleting files. This makes it easier to verify that the files were downloaded before deleting flights from the on-board memory.
• Re-compute boost and landing times. Given the whole flight history, it s easy to find the time when the rocket started and stopped flying. Having these get recomputed means the boost time, acceleration values and main descent rates are computed more accurately.

Bug fixes:

• Wait for 10 valid GPS messages before marking GPS as ready. Before this fix, GPS was marked as ready when 10 valid packets were received from the flight computer after the first valid GPS data arrived. This waits for 10 valid GPS packets instead.
• Fix Google Earth file export. The format requirements for Google Earth files became more strict in recent releases; this patch changes how the files are formatted to make them work again.
• Make AltosUI run on Mac OS X Lion . Apple changed the default heap size for Java applications with this release, dramatically reducing the memory available to applications. This would cause map tiles to fail to load and other random problems.
• Improve COM port handling on Windows. This eliminates the need to wait 5 seconds between closing and re-opening devices, and also eliminates other spurious errors when opening devices.

Today was a black day for democracy in Germany. The German constitutional court ruled in favour of the European Stability Mechanism. In combination with last week s announcement by the European Central Bank to purchase government bonds without limits (breaking the No-Bail-Out clause at the core of their mandate more obviously and irreversably than ever before), the German people have lost a good deal of democracy today. Why? you may ask because from now on, fiscal and financial policy will be made in Brussels, by people enjoying full immunity, but who are not elected democratically by the European people, let alone the Germans, and they will freely decide over who has to pay and be liable for whom. I am talking about people like Klaus Regling, who was already involved the very first time the Maastricht Criteria were violated. He is now at the front of the largest and most powerful financial weapon ever conceived. With immunity. And people like Mario Draghi, whom I would possibly call the most corrupt person I know. His announcement to save the Euro at whatever cost accidentally came only a day before his motherland Italy had to go to the market for more money and was able to place a bond at such ridiculously low interest rates that anyone who s kept up to speed with Italy s development had to rightfully ask how that was possible. While in the past, for whatever reason, the European people have let the ECB get by saying that they are not bailing out countries when they buy bonds on the secondary market (wtf!), they have finally dropped that restriction (the law). And as of today, the ESM is ready to go, along with the fiscal pact. Germany is now liable for more than quarter of all of the Eurozone s past and future debts. And no citizen will be able to have any more influence in this, or reverse it. Budget, fiscal policy and currency control are forever gone. Not that parliamentarian democracies were ever direct. Yet, in the past, one could at least vote for those people whose promises one was inclined to believe the most. You can still do that in the future, but those people won t be able to influence fiscal or financial policy anymore. There is no way back. The ESM and its employees enjoy full immunity, and the ESM is forever-binding. There is no exit clause. Thanks to the ECB s law breaking and the ESM, which I consider highly unconstitutional, at least in Germany, Eurozone-countries may refinance their debts at interest rates that are in no way related to their ability to pay back loans. All other countries foremost Germany are henceforth liable for others debts. The fundamental rule of the EU that no country would have to stand up for another country, is gone with the wind. Within an hour, the markets reacted. Germany, which previously had to pay negative interest (a sign of stability) saw interests on its bond shoot up. And Spain, Portugal, Greece and others who couldn t previously refinance their old debts, are now getting fresh money cheaper than ever. Spain s president Rajoy today didn t even bother beating around the bush anymore, he s now going to apply for fresh money but won t bother with any saving schemes or other restructurings. Monti in Italy has suggested the same. Wouldn t you take money if you were offered it for free, without the need to pay it back? This is more than inflation, in my opinion. What is currently happening in Europe is active depreciation of individual wealth. Our heads of state are actively working against the people. The Euro has lost all credibility and everyone knows it. It is only a question of time until it will tremble and fall. Meanwhile, the market celebrates and continues their gambles while they still can, on the backs of our currency and our wealth. Most affected are the people who have savings in Euros, whose life insurances are decreasing in worth and who cannot afford to diversify into other asset classes or currencies. On the other hand, those who let their money do the work are being saved. Whoever previously invested into bonds of struggling states, hoping to reap massive interest gains, is now proven right. Brussels has eliminated the risk factor. What kind of message does this send??? Hands up if you thought that our politicians are even interested in closing the rapidly widening gap between rich and poor. Really? That s naive. The Eurozone is corrupt, and our currency has never been as virtual as today. Nobody can say whether saving the Euro at all cost is the right thing and noone knows whether what s currently happening is just bad. I would have wished that our politicians had taken the crisis as an incentive to fix the system in the interest of the people and with a long-term focus:

• Europe should have returned to the law it gave itself, without exceptions: The No-Bailout-Clause, as well as the Masstricht Criteria. In general, you cannot spend more than you earn, and this principle seems to have been forgotten entirely, and those who had most actively forgotten it, are being rewarded.
• A shared currency requires a shared fiscal policy. This policy has to be in place before the currency, the currency can only ever be the result of a common strategy. The EU has done this the wrong way around and it failed. Trying to put a fiscal policy in place now is patchwork that is guaranteed to fail. And putting fiscal policy into the hands of undemocratically chosen people in Brussels is not representing the people.

But on the contrary! Europe s policiticans are making it crystal clear that the foundation upon which it was built, the laws and rules, the promises and guarantees, no longer apply. The people were not asked. The promises once made were broken. Our politicians have ruled over our heads. More debts are being made, and more debts to pay off debts, and so on. It s long gotten out of control, now the process is institutionalised. I feel sorry for our kids. I find it irresponsible what is being done to them (in addition to the way we rape the environment). I also feel deeply with the people in the struggling countries who are being screwed by the crisis and are not at fault. What our politicians are doing is unfortunately not going to help long term. The problems are just postponed, and with every day, the inevitable crash will be more painful. I am sorry. Today is a black day for democracy. We have lost souvereignity. We have lost control over our currency. We have lost our budget rights. And I have lost my faith in the last instance of the German government that I trusted. As of today, I know that the German constitutional court is nothing more than a puppet in the hands of the politicians (who are themselves puppets of Brussels and the banks). The limit they imposed (Germany s liability must not increase beyond 190 billion Euros without the federal parliament s consent) is worthless. Soon the politicians will explain to us why it s inevitable that we must raise this limit. Not that the people could prevent it, but still I had hoped for a fundamental ruling. They should not have touched numbers. The EU had a no-bailout-clause from day one. It was conditional from the start. If one of the fundamental principles of a contract is broken, the contract becomes invalid. Not only did I expect the court to rule against socialised debt, I would have wished them to go a step further. The German national bank gave up control over the currency to the ECB only because the ECB incorporated the principles of the German national bank. Once the ECB overturned those principles, Germany should have reclaimed their souvereignity. But noone else in Europe would have wanted that. Merkel became a puppet herself. I am grateful that our daughter has dual citizenship. NP: Porcupine Tree: Live at Atlanta 2010

After several months of effort by the i18n team (and quite a bit from /me), some NMUs, a lot of help by the release team to accept many unblocks, we finally reached 100% translation for debconf screens, in wheezy, for three languages. And, no, French is not among them (yet). The first to reach this heaven are Russian, German and Swedish with translations for all translatable debconf screens for packages in Debian testing (which will become the next Debian release). Three more languages (French, Czech and Portuguese) are waiting for one package to reach testing and another (Spanish) is waiting for three packages. We should then soon reach 7 complete languages. David Pr vot is even trying to get Danish as complete as possible, but it requires pushing for about 15 packages, with many NMUs and unblocks to ask. I started this work in early April, so it took about SIX months to reach this and be ableto happily make a lot of noise about such achievement. You have no idea how it is appreciated by translators....so you maybe have a better idea why I can be so noisy when some uncoordinated upload (for instance with modified localized material) breaks this... We'll have much more news about achievements in l10n for wheezy in the upcoming weeks. We had a lot of things that deserve some trumpets, bells or whistles..:-)

I think perhaps Alonzo Horton overstated the case but San Diego was a perfectly pleasant place to spend a week. Recently I was fortunate and attended the Linux Plumbers conference (and bits of Kernel Summit) and as on previous occasions there were some interesting sessions and some less helpful diversions.

Melodie went along with me, our first trip away together without the kids in a long time. We did the tourist thing for a few days including the USS Midway aircraft carrier tour (recommended) and the harbour boat tour (also recommended).

The conference was co-located with several other events and there seemed to be a lot of people around not connected with KS or LPC. For me the hallway track was, as usual, much more rewarding and I got to catch up with several old friends and make some new acquaintances. A lot of my colleagues were about attending the Gstreamer and Linuxcon conferences so there was the obligatory work evening event (which was pizza and beer, just done brilliantly).

A track that did stand out for me was the Clang / LLVM presentations. They gave an excellent overview of their progress towards making the Kernel compile with their tools, such innovation appears to be making both projects stronger.

One thing that occurred to me was the blandness of KS and LPC this time, usually there are at least some loud disagreements but I failed to attend even one session where there was more than small differences which were quickly resolved.

Perhaps such conferences, like the majority of people attending them, are becoming middle aged and a little complacent. An observation like that does make me wonder where the next change will come from and what it will look like. Just as open source software (including Linux) has disrupted the proprietary software industry in the last decade what will come along and disrupt us? Or is open source the end of the line and we will just continue to evolve?

Continuing the ZFS evaluation journey (which I will summarize here with the things that I learned), I was able to fit about 2.5TB of data on a single 2TB drive, with deduplication enabled. Unfortunately, even moving files around in ZFS (which you would think would be a cheap operation) takes ages. Removing files also takes ages. In a completely unloaded Phenom II X4 910 system running Linux 3.5.2 (actually, from Debian's experimental), and zfsonlinux version 0.6.0.71-0ubuntu1~precise1 (recompiled from source to work with my Debian system), I tried to remove a subdirectory that had about 4000 files. According to time, it took:

real    42m51.487s
user    0m0.188s
sys     0m16.769s

Which is a bit too slow. I just (read: "yesterday") upgraded this machine from 4GB to 6GB of RAM (well, I would have updated it to 8GB of RAM as that is what I ordered, but it one of the RAM chips arrived dead here and I simply took one of the new 4GB card and one of the older 2GB card). But, honestly, I don't see change in upgrading the RAM from 4GB to 6GB and, unless something magic happens with 2GB more, I wouldn't expect a whole different story with 8GB of RAM. Oh, no swap is being used (well, it is an unloaded machine after all), and the Linux kernel swappiness knob is at its default of 60. What I do see, though, is that even for a single file removal, the disk is thrashing a lot. I mean like crazy and that is, of course, the prime suspect of the bad performance (of course, of course). So, there must be some really crazy metadata churning going on here and I guess that the people from the LLNL would like to know of their modules producing this behavior. Keep tuned for some future impressions.

According to my Amazon purchase history, it was Sunday, October 11, 2009, that we were lazing around with our friends Dan and Sue and we decided we'd just hang out and watch the first season of Dollhouse. I bought the whole thing on Amazon's Video on Demand, and we started watching episodes. It got close to dinner time, and Dan was going through a home made pizza phase at the time, and we thought about making some pizzas. I wasn't sure if we had all the ingredients in the house, but it went something like this:

Dan: Got flour? Me: Yes Dan: Yeast? Me: Yes Me: Oh, and we've got a big bag of grated mozzarella from Costco. And we have a pizza stone and a peel, and a bunch of random stuff we can throw on a pizza.

So we totally ninja-ed up some pizzas from stuff we had lying around the house. It was awesome because it was totally spontaneous. You couldn't have planned it better, and we pulled it off brilliantly without having the leave the house. Anyway, Dan had to calculate the ratios of water, flour, salt and sugar, and he wrote them all down on the magnetic whiteboard we kept on our fridge. Fast forward almost 3 years later, and we've moved to our new home, and have a stainless steel fridge that we haven't been able to put this whiteboard on, so it got stashed away in a cupboard. The pizza dough recipe is still written on it, though. So with Sarah away, Bill and Victoria are coming around for dinner tonight to hang out, and I thought trying to make pizza again might be a fun experiment. So I dug out the whiteboard with the trusty recipe, and Zoe and I made some dough this morning, and it's currently proofing in the oven. Unfortunately the whiteboard only had the quantities and not the method on it, and from perusing other recipes on the Internet, I may not have mixed things properly (I just kind of threw things together and made it into a ball), so this first batch might be a total disaster. I might go and make a second batch while Zoe's napping. Here's the contents of the whiteboard, for posterity (I'm hoping it says 10g of salt and not 18g, not that I think it'll affect the outcome too much):

A couple of months ago I was experimenting with adding no-new-privileges to various systems I run. Unfortunately I was surprised a few weeks later at unintended breakge. My personal server has several "real users", and several "webserver users". Each webserver user runs a single copy of thttpd under its own UID, listening on 127.0.0.1:xxxx, where xxxx is the userid:

steve@steve:~$ id -u s-steve
1019
steve@steve:~$ sudo lsof -i :1019
COMMAND  PID    USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
thttpd  9993 s-steve    0u  IPv4 7183548      0t0  TCP localhost:1019 (LISTEN)

Facing the world I have an IPv4 & IPv6 proxy server that routes incoming connections to these local thttpd instances. Wouldn't it be wonderful to restrict these instances, and prevent them from acquiring new privileges? Yes, I thought. Unfortunately I stumbled across a down-side: Some of the servers send email, and they do that by shelling out to /usr/sbin/sendmail which is setuid (and thus fails). D'oh! The end result was choosing between:

• Leaving "no-new-privileges" in place, and rewriting all my mail-sending CGI scripts.
• Removing the protection such that setuid files can be executed.

I went with the latter for now, but will probably revisit this in the future. In more interesting news recently I tried to recreate the feel of a painting, as an image which was successful. I think. I've been doing a lot more shooting recently, even outdoors, which has been fun. ObQuote: "You know, all the cheerleaders in the world wouldn't help our football team." - Bring it On

This is my monthly summary of my Debian related activities. If you re among the people who made a donation to support my work (88.41 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. This month has again been a short one since I have mostly been in vacation during the last 2 weeks. Dpkg Things are relatively quiet during the freeze. I only took care of fixing 3 bugs: a regression of 3.0 (quilt) (#683547), a segfault of dpkg-query -W -f (commit) and a bad auto-completion for French users (#685863). Testing the upgrade to wheezy We got several reports of wheezy upgrade that failed because dpkg ran the trigger while the dependencies of the package with pending triggers are not satisfied. Unfortunately fixing this in dpkg is not without problems (see #671711 for details) so Guillem decided to defer this fix for Jessie. My suggestion of an intermediary solution has fallen in limbo. Instead we now have to find solutions for each case where this can fail (example of failure: 680626). Another way to avoid those errors is to ensure that triggers are run as late as possible. We can improve this in multiple ways. The first way is to modify most triggers so that they use the interest-noawait directive. In that case, the packages activating the trigger will be immediately marked as configured (instead of triggers-awaited ) and the trigger will thus not need to be run as part of further dependency solving logic. But as of today, there s no package using this new feature yet despite a nudge on debian-devel-announce. The second way is to modify APT to use dpkg no-triggers, and to let the trigger processing for the end (with a last dpkg configure -a call). I requested this early in the wheezy timeframe but for various reasons, the APT maintainers did not act on it. I pinged them again in #626599 but it s now too late for wheezy. I find this a bit sad because I have been using those options for the entire wheezy cycle and it worked fine for me (and I used them for a dist-upgrade on my wife s laptop too). It would have been good to have all this in place for wheezy so that we don t have to suffer from the same problems during the jessie upgrade, but unless someone steps up to steer those changes, it seems unlikely to happen. Instead, we re back to finding klumsy work-arounds in individual packages. Packaging I prepared security updates for python-django (1.4.1 for unstable,
1.2.3-3+squeeze3 for stable). I packaged a new upstream version for cpputest (3.2-1). I reviewed ledgersmb 1.3.21-1 prepared by Robert James Clay and asked him to prepare another version with further fixes. I released nautilus-dropbox 1.4.0-2 with supplementary changes of my own to support https_proxy and to display better diagnostic information when the download fails. With the help of Paul van der Vlis and Michael Ziegler, we did what was required to be able to migrate python-django-registration 0.8 to Wheezy even though it s a new upstream version with backwards incompatible changes. Thanks to Adam D. Barratt who unblocked the package, we now have the right version in Wheezy despite the fact that I missed the freeze deadline. Debian France Julien Cristau reminded the board of Debian France that we have to elect officers (President, Secretary, Treasurer) as the current officers have withdrawn. I was somewhat afraid that nobody would take over so I pinged each member to try to get new volunteers. We now have volunteers (me, Julien Danjou and Sylvestre Ledru) and we re waiting until Julien finds some time to run the election. Misc With the help of DSA, I setup antispam rules for the owner@packages.qa.debian.org alias because I was getting tired by the amount of spam. In the process, they asked me to write a wiki page for dsa.debian.org to document everything so that they can refer to it for future queries. I did it but it looks like that they did not apply my patch yet. I also tested an upstream patch for gnome-keyring (see bugzilla #681081) that reintroduces the support of forgetting GPG passphrases after a specified amount of time. Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Next weekend I will hold an advanced Haskell talk at the Meta Rhein-Main ChaosDays in Darmstadt. It will mostly be a guided tour through the main memory of a running Haskell program, showing how things are represented and how they change over time. For this I cannot avoid to touch the issue of garbage collection, so I created a short animation: (If you do not see it above, then you can try on YouTube.) It is created with Synfig, feel free to use the sources. It seems that Synfig, although it has some features that allow you to link position in various objects, or calculate with them, is not really meant for programmers. Take the arrows: They are all quite similar, and defined by starting point, end point, and how much the initial segment sticks out to the right. I could even create them so that when I move the first or last point, the rest adjusts automatically. So one would expect that I can combine these shapes in one re-usable object that has these three values as parameters, but unfortunately, that is not the case. I guess next time I want to animate something as structured as this, I use either tikz (and generate one page per frame, convert them to an image and then to a video) or a Haskell library like gloss or diagrams. Update: I made some minor improvements to the video. Youtube does not allow me to change the video, so you ll see the new version only here.

Flattr this post

I claim that the way we conduct discussions on debian-devel has gone quickly downhill this year. The discussions around the default init implementation are particularly sad examples of how Debian debates technical topics. I've whined about this before, and people have suggested various things. Here's a list of things that I think we can conclude do not work:

• "Do not feed the troll" does not work, because there is always someone who can't resist responding. Also, some of the bad behavior is not trolling. Further, having everyone ignore people who behave badly does not give negative feedback for bad behavior, and does not build shared social norms for what is acceptable behavior.
• Unorganized private feedback (people mailing in response to public list posts) sometimes works, but it's a bit hit-and-miss, and again does not help build shared social norms. In other words, new members of the community see that others behave badly, and then nothing happens as far as they can see, whereas in private there may be quite a lot of activity.
• Up/down-voting a la Reddit or Stack Overflow works on those sites, but it is not the only thing that keeps their sites in shape. It also works badly on mailing lists, I think, and we can't very well switch away from the lists for all sorts of reasons. Voting can also result in a stronger "echo chamber", where the popular people or opinions are upvoted, regardless of technical merit.

Things that can work:

• Real-life meetings between participants. Debconf, sprints, FOSDEM, other such conferences. Unfortunately, this is expensive, and we can't reach everyone.
• Active public, but constructive confrontation when people behave badly. This needs good social skills to be done well.

Things that might be worthwhile to try:

• Changes to how our mailing list software works. Not sure what they would be, but Clay Shirky's Group as User: Flaming and the Design of Social Software may be a good source of inspiration.
• If not technical changes to the list software, possibly ways in which we use the tools we have. For example, a friend of mine suggested a while ago that when there is a topic on which consensus is difficult to achieve, each side of the debate (there may be any number) writes up their position and arguments for it and against the others on a wiki page. This might avoid going in circles in the discussions, repeating the same arguments over and over, during weeks and months of discussion.
• Public, but anonymous, feedback tools, a la Reddit and Stack Overflow. Perhaps as simple as adding "+1" and "-1" buttons to the web archive pages for messages, along the "Spam" button, and then publishing aggregate statistics? (A voting system, if you wish.)
• A consensus on the rough bounds of acceptable behavior and some group of people who are delegated the authority to enforce that.

The current situation, where a few hard-nosed, uncompromising people can prevent any rational discussion about init systems cannot be allowed to continue.

Some months ago I have run into a pretty interesting bug while working on a Ubuntu-based remote desktop system. The whole OS was installed on a server somewhere and users could access their desktop remotely. Some call this stuff Desktop-as-a-Service. The operating system we chose was Ubuntu Oneiric (11.10) and the remote access part was implemented with x2go, which uses nxagent to provide NX transport of X sessions. Users could access their Ubuntu machines remotely, with sound, video, youtube, and all you would expect from a desktop machine. The whole thing was working quite well. Now, as I said that was in May. Ubuntu 12.04 LTS was available, and the choice of upgrading to it sounded pretty much obvious. So we upgraded our test system to Precise and everything seemed to work smoothly. Till we tried to open a PDF document, actually. It took evince about 50 seconds to display a relatively small document. Same story with images opened with the default image viewer, eog. The fix delivered to our users was simple: we have set shotwell-viewer as the default image viewer, and epdfview as the default PDF viewer. Everybody happy. In the meantime, obviously, I was interested in this issue so I ve tried to run run evince from a terminal, getting the following output:

(evince:15833): GRIP-WARNING **: Failed to initialize gesture manager.

Funny. On another test system running Debian Sid (unstable) everything was working smoothly. The diff between Ubuntu s version of evince and Debian s is a 6MB monster. Among other changes, I noticed that Ubuntu s version build-depends on libgrip-dev, which depends on libutouch-geis. Multitouch stuff. Why should multitouch support break my remote session? So on May the 10th I filed a bug on launchpad. How this issue got handled is in my opinion one of the many fine examples of the inherent superiority of free software, coupled with a we won t hide problems mindset. For an example of how bad is the proprietary approach, just check a random bug in Adobe s bug tracking system. But let s go back to the evince bug. Other users reported that their VNC sessions were also affected by the same problem. After a few days it was clear that the culprit was utouch-geis, and a patch appeared. Unfortunately it did not actually address the issue. Somebody else reported that RDP sessions were broken too. At the beginning of June Precise was still affected. Finally, on August the 6th a working patch was submitted by Bradley M. Froehle and included by Chase Douglas (thank you guys). End of August, fixed version of geis accepted into precise-updates, case closed. Now for some considerations. The problem was clearly of a certain importance. A Long Term Support, stable version of Ubuntu shipped with broken PDF and image viewing functionalities. It got fixed properly, even though 3 good months are quite a long time for such a bug to get solved. However, the issue only affected a pretty limited number of users, also certainly not Ubuntu s main target audience. This bug never affected Debian, simply because utouch-geis has not made its way into the archive yet. It takes longer to make changes like this in Debian, but for some categories of users stability is more important than new, cool features. Choice is a good thing.

ELKI is a data mining software project that I have been working on for the last years as part of my PhD research. It is open source (AGPL-3 licensed) and avilable as both a Debian package and Ubuntu package in the official repositories. So a simple aptitude install elki should get you going and give you a menu entry for ELKI. These packages come with the scripts elki to launch the MiniGUI and elki-cli to run from command line. The key feature that sets ELKI apart from existing open source tools used in data mining (e.g. Weka and R) is that it has support for index structures to speed up algorithms, and a very modular architecture that allows various combinations of data types, distance functions, index structures and algorithms. When looking for performance regressions and optimization potential in ELKI, I recently ran some benchmarks on a data set with 110250 images described by 8 dimensional color histograms. This is a decently sized dataset: it takes long enough (usually in the range of 1-10 minutes) to measure true hotspots. When including Weka and R in the comarison I was quite surprised: our k-means implementation runs at the same speed as Rs implementation in C (and around twice that of the more flexible "flexclus" version). For some of the key agorithms (DBSCAN, OPTICS, LOF) we are an order of magnitude faster than Weka and R, and adding index support speeds up the computation by another factor of 5-10x. In the most extreme case - DBSCAN in Weka vs. DBSCAN with R-tree in ELKI - the speedup was a factor of 330x, or 2 minutes (ELKI) as opposed to 11 hours (Weka).
The reason why I was suprised is that I expected ELKI to perform much worse. It is written in Java (as opposed to R's kmeans, which is in C), uses a very flexible architecture which for example does not assume distances to be of type double and just has a lot of glue code inbetween. However, obviously, the Java Hotspot compiler actually lives up to its expectations and manages to inline the whole distance computations into k-means, and then compiles it at a level comparable to C. R executes vectorized operations quite fast, but on non-native code as in the LOF example it can become quite slow, too. (I would not take Weka as reference, in particular with DBSCAN and OPTICS there seems to be something seriously broken. Judging from a quick look at it, the OPTICS implementation actually is not even complete, and both implementations actually copy all data out of Weka into a custom linear database, process it there, then feed back the result into Weka. They should just drop that "extension" altogether. The much newer and Weka-like LOF module is much more comparable.) Note that we also have a different focus than Weka. Weka is really popular for machine learning, in particular for classification. In ELKI, we do not have a single classification algorithm because there is Weka for that. Instead, ELKI focuses on cluster analysis and outlier detection. And ELKI has a lot of algorithms in this domain, I dare to say the largest collection. In particular, they are all in the same framework, so they can be easily compared. R does of course have an impressive collection in CRAN, but in the end they do not really fit together. Anyway, ELKI is a cool research project. It keeps on growing, we have a number of students writing extensions as part of their thesis. It has been extremely helpful for me in my own research, as I could quickly prototype some algorithms, then try different combinations and use my existing evaluation and benchmarking. You need some time to get started (largely because of the modular architecture, Java generics and such hurdles), but then it is a very powerful research tool. But there are just many more algorithms, published sometime, somewhere, but barely with source code available. We'd love to get all these published algorithms into ELKI, so researchers can try them out. And enhance them. And use them for their actual data. So far, ELKI was mostly used for algorithmic research, but it's starting to move out into the "real" world. More and more people that are not computer scientists start using ELKI to analyze their data. Because it has algorithms that no other tools have. I tried to get ELKI into the "Google Summer of Code", but it was not accepted. But I'd really like to see it gain more traction outside the university world. There are a number of cool projects associated with ELKI that I will not be able to do myself the next years, unfortunately.

• A web browser frontend would be cool. Maybe even connected to Google Refine, using Refine for preprocessing the data, then migrating it into ELKI for analysis. The current visualization engine of ELKI is using SVG - this should be fairly easy to port into the web browser. Likely, the web browers will even be faster than the current Apache Batik renderer.
• Visual programming frontend. Weka, RapidMiner, Orange: they all have visual programming style UIs. This seems to work quite well to model the data flow within the analysis. I'd love to see this for ELKI, too.
• Cluster/Cloud backend. ELKI can already handle fairly large data sets on a big enough system. If someone spends extra effort on the index structures, the data won't even need to fit into main memory anymore. Yet, everybody now wants "big data", and parallel computation probably is the future. I'm currently working on some first Hadoop YARN based experiments with ELKI. But this is a huge field, turning this into true "elk yarn". I will likely only lay some foundations (unless I get funding to continue as a PostDoc on this project. I sure hope to get to do at least a few years of postdoc somewhere, as I really enjoy working with students on this kind of project)
• New visualization engine. The current visualization engine, based on Apache Batik and SVG is quite okay. It does what I need, which is to get a quick glance at the results and the ability to export them for publications in print quality. (in particular, I can easily edit the SVG files with Inkscape) But it is not really something fancy (although we have a lot of cool visualizations). And it is slow. I havn't found a portable and fast graphics toolkit for Java yet that can produce SVG files. There is a lot of hype around processing, for example, but it seems to be too much about art for me. In fact, I'd love to use either something like Clutter or Cairo. But getting them to work for Windows and Mac OSX will likely be a pain.
• Human Computer Interaction (HCI). This is in my opinion the biggest challenge we are facing with all the "big data" stuff. If you really go into big data (and not just run Hadoop and Mahout on a single system; yes - a lot of people seem to do this), you will at some point need to go beyond just crunching the numbers. So far, the challenges that we are tackling are largely data summarization and selection. TeraSort is a cool project, and a challenge. Yet, what do you actually get from sorting this large amount of data? What do you get from running k-means on a terabyte? When doing data mining on a small data set, you quickly learn that the main challenge actually is preprocessing the data and choosing parameters the right way so that your result is not bogus. Unless you are doing simple prediction tasks, you often don't have a clearly defined objective. Sure, when predicting churn rates, you can hope to just throw all the data into a big cloud and hope you get some enlightement out. But when you are doing cluster analysis or outlier detection - unsupervised methods - the actual objective by definition cannot be hardcoded into a computer. The key objective then is learn something new on the data set. But if you want to have your user learn something on the data set, you will have to have the user guide the whole process, and you will have to present results to the user. Which gets immensely more difficult with larger data. Big data just does no longer look like this. And neither are the algorithms as simple as k-means or hierarchical clustering. Hierarchical clustering is good for teaching the basic ideas of clustering. But you will not be using a dendrogram for a huge data set. Plus, it has a naive complexity of O(n^3) and for some special cases O(n^2) - too slow for truly big data.
  For the "big data future" once we get over all the excitement of being able to just somehow crunch these numbers we will need to seriously look into what to do with the results (in particular, how to present them to the user), and how to make the algorithms accessible and usable for non-techies. Right now, you cannot expect a social sciences researcher to be able to use a Hadoop cluster. Yet to make sense of the results. But if you are a smart guy to actually solve this, and open up "big data processing" to the average non-IT user, this will be big.
• Oh, and of course there are just hundreds of algorithms not yet available (accessible) as open source. Not in ELKI, and usually not anywhere else either. Just to name a few from my wishlist (I could probably implement many of them in a few hours in ELKI, but I don't have the time to do so myself, plus they are good student or starter project to get used to ELKI): BIRCH, CLARA, CLARANS, CLINK, COBWEB, CURE, DOC, DUSC, EDSC, INSCY, MAFIA, P3C, SCHISM, STATPC, SURFING, ... just to name a few.

If you are a researcher in cluster analysis or outlier detection, consider contributing your algorithms to ELKI. Spend some time optimizing them, adding some documentation. Because, if ELKI keeps on growing and gaining popularity, it will be the future benchmark platform. And this can give you citations, which are somewhat the currency of science these days. Algorithms available in the major toolkits just do get cited more, because people compare to them. See this list for an overview of work cited by ELKI - scientific work that we reimplemented at least to some extend for ELKI. It is one of the services that we provide with ELKI for researchers: not only the algorithm, but also the appropriate citation.

This article documents how I'm currently building my firewalls. It builds on netfilter-based-port-knocking, and tries to integrate several components of a firewall as gracefully as possible. For some context: I'm getting involved with servers where the firewall policy goes beyond a handful of SSH from my home and my other servers rules. There are many different network streams that need to be policed, some of them are common across several (potentially many) servers, and so on. So I'm gradually giving up on my hand-made scripts, and trying out higher-level tools. I settled on FWbuilder, which seems nice. However, it only allows static policies, and I still want to keep dynamic policies such as what fail2ban provides, as well as my own port-knocking system. The problem I had was that fail2ban isn't really made to play nice as part of a complex firewalling setup, my port-knocking system was too tightly integrated within my firewall script, and FWbuilder wasn't too flexible when it came to delegating part of the firewall policy to something external. Fortunately, this was only a perceived problem (or a real problem in my understanding), because it is actually possible to have all three blocks playing nicely together. More context: as usual, I'm focusing on Debian-like systems. More precisely, on those with a Linux kernel; it may be that FreeBSD's firewalling subsystem has a feature comparable to Linux's recent module, but I don't know. Let's start with FWbuilder. This is not the place for a manual, the official documentation is rather complete. I'll assume you have defined most relevant blocks in there: firewall, hosts, IP addresses, services, and so on. You define your static policy with the standard rules. From then on, we want to integrate the external tools for dynamic rules. Step 1: Integrating fail2ban We want fail2ban to have its own playground, so that it doesn't overwrite anything in the standard policy. The trick is to define a new policy rule set named fail2ban. Leave it empty in FWbuilder. So far so good, but fail2ban (the daemon) still operates on the INPUT chain in the firewall, and could therefore still mangle the static rules. Fortunately, starting with fail2ban 0.8.5 (available from Debian Wheezy, or in the backports for Squeeze), you can define what chain to operate on: with a configuration item such as chain = fail2ban, fail2ban (the daemon) will now only add its rules to fail2ban (the firewall chain), and won't be able do damage the other chains. The missing part is to send some of the traffic to it using the standard policy: i defined a rule sending the incoming SSH connections to the fail2ban policy ( branching in FWbuilder jargon). Voil : the static policy delegates part of the decision-making to a sub-policy controlled by the fail2ban daemon. Step 2: Integrating port-knocking This is a bit trickier, but we'll use a similar method. First, the traffic used for port-knocking needs to be directed to the chain that does the listening. Define a policy rule set named portknocking, and leave it empty in FWbuilder. It'll be used by the dynamic rules to track progression of source IP addresses through the port-knocking sequence, so you'll need to send ( branch ) incoming traffic there, probably after the rules allowing incoming connections from known hosts. The dynamic part of this will only concern the refreshing of this listening chain , which we assume will do its work and mark IP addresses with PK_ESTABLISHED once the sequence is completed. What we do with these marked IP addresses will still be defined within the FWbuilder policy. We're going to need some complex rules since we want to filter according to this PK_ESTABLISHED bit and according to destination port, for instance; unfortunately FWbuilder doesn't allow combining filter criteria with and, so we define a new policy rule set called accept_if_pk_ok. This ruleset has two rules: the second is an ACCEPT and should be easy to understand. The first rule needs to ensure the ACCEPT is only reached for connections coming from PK_ESTABLISHED addresses, so it's going to be a bit tricky.

• The service needs to be a custom service (I called it PK not established ), since FWbuilder doesn't know about the marking feature in iptables. Use -m recent ! --rcheck --seconds 86400 --name PK_ESTABLISHED for the definition (change the duration to the number of seconds the door should stay open after the port-knocking sequence has been completed). Note the exclamation mark.
• The action is also going to be custom, defined as -j RETURN. Again, this feature is iptables-specific and FWbuilder doesn't provide any UI for it.

(Explanation: the first rule matches packets coming from IP addresses not marked as PK_ESTABLISHED, and returns them to the calling policy. Packets remaining after this rule are those coming from the appropriate addresses, and they go on to the ACCEPT. We could have had the first rule match on IP addresses that are marked, and branch to yet another ruleset with the ACCEPT part, but that would make it harder to read, I feel.) Now let's get back to the main policy and add rules concerning what kind of traffic we want to allow once the port-knocking sequence completed. For instance, we define a rule matching on the SSH service , where the action is to branch to accept_if_pk_ok. When an incoming packet tries to establish a connection to the SSH port, it's passed to the accept_if_pk_ok ruleset. If it comes from the same IP as a recent port-knocking sequence, it goes on to be ACCEPTed. If not, it returns to the main policy. Maybe static rules further on will allow it to go through. Step 3: tying it all together Now that we have all the pieces, the rest is plumbing.

• Get FWbuilder to compile a script from the data. I called mine $hostname.fw, and stored it into /usr/local/sbin.
• Write a /usr/local/sbin/port-knocking script that operates on the portknocking chain and manages the PK_ESTABLISHED bit. It need not do more than what's described in netfilter-based-port-knocking.
• Write an initialisation script that calls both /usr/local/sbin/$hostname.fw and /usr/local/sbin/port-knocking. I called mine /etc/init.d/firewall.
• Make sure that fail2ban's initialisation script is called after ours. Either with boot sequence numbers, or with the LSB dependency pseudo-headers: I made my firewall script to Provides: iptables; since fail2ban's script declares that it Should-Start: [ ] iptables, we're fine.
• Run /usr/local/sbin/port-knocking every hour, or as often as needed to recalculate the port numbers.

With this setup, at boot time, the $hostname.fw script creates the static policy and the extra playgrounds; then the port-knocking script implements the listening for the magic sequence; then fail2ban inserts its own rules. And there we are: three different parts for the firewall policy, all integrating nicely. Mission accomplished! Note: (Mostly copy-and-pasted from the previous article) This article is deliberately short on details and ready-to-run scripts. Firstly because firewall scripts vary wildly so any script would have to be adapted anyway, but mostly because security is best handled with one's brain switched on. Fiddling with a firewall can easily open gaping holes or lock everyone out. So please make sure you understand what goes on before blindly pasting stuff into your own setup. Some bits are left as an exercise to the reader.

Several months ago, I wrote about grep everything and listed grep-like tools which can grep through compressed files or specific data formats. The blog posting sparked several magazine articles and talks by Frank Hofmann and me. Frank recently noticed that we though missed one more or less mighty tool so far. We missed it, because it s mostly unknown, undocumented and hidden behind a package name which doesn t suggest a real recursive grep everything : deepgrep deepgrep is part of the Debian package strigi-utils, a package which contains utilities related to the KDE desktop search Strigi. deepgrep especially eases the searching through tar balls, even nested ones, but can also search through zip files and OpenOffice.org/LibreOffice documents (which are actually zip files). deepgrep seems to support at least the following archive and compression formats:

• tar
• ar, and hence deb
• rpm (but not cpio)
• gzip/gz
• bzip2/bz2
• zip, and hence jar/war and OpenOffice.org/LibreOffice documents
• MIME messages (i.e. files attached to e-mails)

A search in an archive which is deeply nested looks like this:

$ deepgrep bar foo.ar
foo.ar/foo.tar/foo.tar.gz/foo.zip/foo.tar.bz2/foo.txt.gz/foo.txt:foobar
foo.ar/foo.tar/foo.tar.gz/foo.zip/foo.tar.bz2/foo.txt.gz/foo.txt:bar

deepgrep though neither seems to support any LZMA based compression (lzma, xz, lzip, 7z), nor does it support lzop, rzip, compress (.Z suffix), cab, cpio, xar, or rar. Further current drawbacks of deepgrep:

• Nearly no commandline options, especially none of the common grep options
• No man-page or other documentation
• Exit code not related to search results, you have to check the output to see if something has been found

deepfind If you just need the file names of the files in nested archives, the package also contains the tool deepfind which does nothing else than to list all files and directories in a given set of archives or directories:

$ deepfind foo.ar
foo.ar
foo.ar/foo.tar
foo.ar/foo.tar/foo.tar.gz
foo.ar/foo.tar/foo.tar.gz/foo.zip
foo.ar/foo.tar/foo.tar.gz/foo.zip/foo.tar.bz2
foo.ar/foo.tar/foo.tar.gz/foo.zip/foo.tar.bz2/foo.txt.gz
foo.ar/foo.tar/foo.tar.gz/foo.zip/foo.tar.bz2/foo.txt.gz/foo.txt

As with deepgrep, deepfind does not implement any common options of it s normal sister tool find. Dependencies The package strigi-utils doesn t pull in the complete Strigi framework (i.e. no daemon), just a few libraries (libstreams, libstreamanalyzer, and libclucene). On Wheezy it also pulls in some audio/video decoding libraries which may make some server administrators less happy. Conclusion Both tools are quite limited to some basic use cases, but can be worth a fortune if you have to work with nested archives. Nevertheless the claim in the Debian package description of strigi-utils that they re enhanced versions of their well known counterparts is IMHO disproportionate. Most of the missing features and documentation can be explained by the primary purpose of these tools: Being backend for desktop searches. I guess, there wasn t much need for proper commandline usage yet. Until now. ;-) 42.zip And yes, I was curious enough to let deepfind have a look at 42.zip (the one from SecurityFocus, unzip seems not able to unpack 42.zip from unforgettable.dk due a missing version compatibility) and since it just traverses the archive sequentially, it has no problem with that, needing just about 5 MB of RAM and a lot of time:

[ ]
42.zip/lib f.zip/book f.zip/chapter f.zip/doc f.zip/page e.zip
42.zip/lib f.zip/book f.zip/chapter f.zip/doc f.zip/page e.zip/0.dll
42.zip/lib f.zip/book f.zip/chapter f.zip/doc f.zip/page f.zip
42.zip/lib f.zip/book f.zip/chapter f.zip/doc f.zip/page f.zip/0.dll
deepfind 42.zip  11644.12s user 303.89s system 97% cpu 3:24:02.46 total

I though won t try deepgrep on 42.zip. ;-)

SSDs have been dropping in price recently so I just bought four Intel 120G devices for $115 each. I installed the first one for my mother in law who had been complaining about system performance. Her system boot time went from 90 seconds to 20 seconds and a KDE login went from about 35 seconds to about 10 seconds. The real problem that she had reported was occasional excessive application delay, while it wasn t possible to diagnose that properly I think it was a combination of her MUA doing synchronous writes while other programs such as Chromium were doing things. To avoid the possibility of a CPU performance problem I replaced her 1.8GHz E4300 system with a 2.66GHz E7300 that I got from a junk pile (it s amazing what s discarded nowadays). I also installed a SSD in my own workstation (a 2.4GHz E4600). The boot time went down from 45s on Ext4 without an encrypted root to 27s with root on BTRFS including the time taken to enter the encryption password (maybe about 23s excluding my typing time). The improvement wasn t as great, but that s because my workstation does some things on bootup that aren t dependent on disk IO such as enabling a bridge with STP (making every workstation a bridge is quieter than using switches). KDE login went from about 27s to about 12s and the time taken to start Chromium and have it be usable (rather than blocking on disk IO) went from 30 seconds to an almost instant response (maybe a few seconds)! Tests on another system indicates that Chromium startup could be improved a lot by purging history, but I don t want to do that. It s unfortunate that Chromium only supports deleting recent history (to remove incriminating entries) but doesn t support deleting ancient history that just isn t useful. I didn t try to seriously benchmark the SSD (changing from Ext4 to BTRFS on my system would significantly reduce the accuracy of the results), I have plans for doing that on more important workloads in the near future. For the moment the most casual tests have shown a significant performance benefit so it s clear that an SSD is the correct storage option for any new workstation which doesn t need more than 120G of storage space. $115 for SSD vs $35 for HDD is a fairly easy choice for a new system. For larger storage the price of hard drives increases more slowly than that of SSD. In spite of the performance benefits I doubt that I will gain a real benefit from this in the next year. The time taken to install the SSD equates to dozens of boot cycles which given a typical workstation uptime in excess of a month is unlikely to happen soon. One minor benefit is that deleting messages in Kmail is an instant operation which saves a little annoyance and there will be other occasional benefits. One significant extra benefit is that an SSD is quiet and dissipates less heat which might allow the system cooling fans to run more slowly. As noisy computers annoy me an SSD is a luxury feature. Also it s good to test new technologies that my clients may need. The next thing on my todo list is to do some tests of ZFS with SSD for L2ARC and ZIL. Related posts:

1. How I Partition Disks Having had a number of hard drives fail over the...
2. Xen and Swap The way Xen works is that the RAM used by...
3. big and cheap USB flash devices It s often the case with technology that serious changes occur...

One major change in GNOME 3.6 is Pango s shaper engines replacement with HarfBuzz. Only language engines (for word break analysis, for example) are retained. So, I m checking how this affects Thai/Lao rendering and what to do next. Over all, Behdad has put a good effort to make it right. Most Uniscribe behaviors have been achieved for compatibility. He even cares enough to cover some widespread Thai fonts in which the language tag 'latn' is used instead of 'thai', as seen in Mozilla #719366. Unfortunately, this font set has been declared as standard fonts in official documents. The workaround seems inevitable. Supported Fonts In my experiments with some existing Thai OpenType fonts, the new Pango still renders well without regression. Loma font from fonts-tlwg (glyph positioning with GPOS, rearrangement with GSUB): Arundina Sans font from Fonts-SIPA-Arundina (positioning by substitution, only GSUB, no GPOS): But for legacy fonts without OpenType features, it renders badly: In addition, according to Behdad, PUA glyphs in legacy fonts are not supported yet. This means there will be regression on fonts designed for Windows XP or below. But modern fonts designed for Windows 7 should be fine. Changes on Bugs The engine replacement from scratch certainly affects existing bugs. Some become obsolete, some still remain. Here are the summary for Thai/Lao engine, as resolved upstream: Closed bugs:

• GNOME #616495 (Debian #620001) regarding Lao MAI KONG rendering, which was caused by a flaw in my code. I have proposed a patch for a while, but no action is taken yet. And patched debs have been locally distributed as a workaround. However, with HarfBuzz replacement, the bug has now gone.
• GNOME #378001 regarding minority languages supports. I hadn t worked on this because I was waiting for WTT 3.0, a local standard, to be drafted. Anyway, with HarfBuzz replacement, the old WTT 2.0 clustering has been dumped and replaced with Unicode guidelines. Therefore, I assume it should be now possible to render minority languages with Thai script, provided that the font has the required OpenType features.
• GNOME #393307, #677090 regarding wrong rendering of zero-width marks like ZWJ, ZWSP. This bug has also been dumped with the HarfBuzz replacement.

Questionable bugs:

• GNOME #583718 (Debian #620002) regarding the rendition of Thai SARA AM (U+0E33) on VTE with excessive dotted circle. So far, I have disagreed with Behdad whether this bug should be treated along with Indic scripts. IMO, there is an easy path for Thai by rendering monospace fonts differently, which is also in accordance with widespread practice everywhere else, albeit XTerm, Emacs, or framebuffer TTYs. But Behdad doesn t like the idea and insists that it should be treated along with Indic scripts, which would complicate things a lot. So, the bug has been there for many years. Meanwhile, I have also provided a workaround in the aforementioned patched debs. BTW, the situation has been changed a little bit after the HarfBuzz replacement. Firstly, let s see the problem with current Pango: One can easily spot the dotted circle glitches. And here is how I workaround it, which is like how it's rendered on other terminals: With the HarfBuzz replacement, here is how it renders: That is, although it s still wrong, it s more tolerable. So, the question for users is: Could they tolerate this until VTE is redesigned for Indic scripts supports?

Remaining bugs:

• GNOME #576156 (Debian #620004) regarding weird cursor movements caused by Unicode UAX #29. Many amendment efforts have been pushed to Unicode from different sources, until it was finally accepted in Unicode 6.1.0. However, no action has been taken in Pango yet. We still have to push it further. Again, a fix has also been provided in the patched debs.