Looking through the Libravatar access logs, I found that most of the traffic we currently serve consists of 302 redirects to Gravatar. Optimising that path is therefore very important.

While Apache allows admins to provide custom error pages for things like 404 or 500, it's not quite that straightforward for 30x return codes.

Standard 301 / 302 responsesBy default, Apache (and most web servers out there) returns a fairly large HTML page along with a 30x redirection. Try it for yourself by disabling automatic redirections in Firefox (Preferences Advanced General Accessibility) or by installing the Request Policy add-on.

The 302 responses sent by Libravatar looked like this:

$ curl -i http://cdn.libravatar.org/avatar/12345678901234567890123456789012
HTTP/1.1 302 Found
Date: Wed, 21 Sep 2011 01:51:52 GMT
Server: Apache
Cache-Control: max-age=86400
Location: http://www.gravatar.com/avatar/12345678901234567890123456789012.jpg?r=g&s=80&d=http://cdn.libravatar.org/nobody/80.png
Vary: Accept-Encoding
Content-Length: 310
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href="http://www.gravatar.com/avatar/12345678901234567890123456789012.jpg?r=g&s=80&d=http://cdn.libravatar.org/nobody/80.png">here</a>.</p>
</body></html>

As you can see, the body of the response is just as large as the headers and isn't really necessary.

Body-less 301 responsesAfter reading about the ErrorDocument directive, I created an empty file called 302 in the root of the web server and included this directive in my vhost configuration file:

ErrorDocument 302 /302

which made the responses look like this:

$ curl -i http://example.com/redir
HTTP/1.1 302 Found
Date: Wed, 21 Sep 2011 03:39:26 GMT
Server: Apache
Last-Modified: Wed, 21 Sep 2011 03:39:17 GMT
ETag: "8024d-0-4ad6b52201036"
Accept-Ranges: bytes
Content-Length: 0
Content-Type: text/plain

This one does have a completely empty body, however, there's an important problem with this solution: the Location header is missing! Not much point in reducing the size of the redirect if it's no longer working.

Custom 302 response pageThe next thing I tried (and ended up settling on) is this:

ErrorDocument 302 " "

which results in a 1-byte response (a single space) in the body:

$ curl -i http://example.com/redir
HTTP/1.1 302 Found
Date: Wed, 21 Sep 2011 03:37:50 GMT
Server: Apache
Location: http://www.example.com
Vary: Accept-Encoding
Content-Length: 1
Content-Type: text/html; charset=iso-8859-1

There is still a little bit of unnecessary information in this response (character set, Vary and Server headers), but it's a major improvement over the original.

If you know of any other ways to reduce this further, please leave a comment!

Even though some three months have passed since DebConf11 has successfully ended, I still wanted to give you just a glimpse on how some parts were played through the eyes of a lunatic (read: organizer). Of course, blog post can t come close near of explaining anything but at least it should give some insight and hopefully some pointers to the future DebConf organizers. If you don t feel like reading this much text you can listen to last episode of This Week In Debian podcast and after it just head to Beginning of the end part. DebConf11 First of all I d like to apologize if I offended or hurt anyone in this whole process as it wasn t my intention and during DebConf organization this is nothing irregular, for you to get hurt or you hurting somebody else. One thing that got stuck with me this whole time is when Mart n Ferrari approached me after I did bid proposal of Bosnia/Herzegovina for DebConf11 back on DebConf9 and told me you have no idea what you just got yourself into I played cool and said something along the line of coure I do and boy did I lie. During DebConf organization you re bound to make some of your decisions instantaneously in which you ll lose something, the most you can do is assess what s the thing you can cut the cord on, even though if that thing or a person for that matter might have meant something to you. You ll lose things, relationships, friendships, contacts at the times sense of humor and maybe even common logic. Some of it comes back to you, some the very next day and for some it may take longer. Some of it was lost irreversibly, but even in that case it wasn t lost and could even be traced to some other cause. One thing that I m thankful for is that I didn t lose, not lose but even get distanced from those close to me, and that I didn t lose guy who was with me in this from very first day, my right hand, my partner in crime Velimir ( aroundthfur ). It s not even losing I m talking about here, because even by losing you re gaining something and this is part of your everyday life, it s just that your life frame kinda gets stuck on fast forward button. So once the ride s over you may feel bit dizzy asking yourself what just happened. During this time I was also contacted by Google recruiter regrading software engineer internship but to make a long story short I didn t make it due to my heavy involvement and my lack of time (read: fail). But I don t find this as losing either, first time I was contacted by them, next time they ll be contacted by me maybe even for different position this time (project manager? . Even this DebConf wasn t our first bid we lost DebConf7 bid to Edinburgh, but it is perseverance that got us here. The moral I m trying to tell here is if you have a goal never back down from it. What I found to be our biggest problem and what I personally found as most painstaking in all of this was having and reporting to so many different sides and different groups while trying to keep them in same communication channel. Even my right hand (Velimir) was apart from me (Vienna), then place where two of us are coming from is not the place where DebConf is getting organized in. As in the end we ended up having all the parties separated and pulling to their own side, from Government of Republika Srpska to NGO to Debian to local team and to those who remained as undeclared . Option of cutting the cord in this case was missing as these were the components that DebCon11 was made of. Of course this meant you ll be missing workforce and there were days you d wake up driving >= 200 km/h (literally) with one hand on the wheel, other on the phone and your other right hand (Velimir) holding your RedBull. But you wouldn t do this just because you had to and there was no one else to do it, you d do it because you re responsible for the sake of the project and won t be just sitting there because by some unwritten hierarchy rules you re supposed to sit and supervise things. I d advise you do the same thing when you see there s no one else to do something, even if it means making coffee for one of the teams. I m not trying to scare you away from organizing your own DebConf, on the contrary after all I ve been through in past two years if someone asked me would I do it if I knew it was like this, without a doubt my answer would still be fsck yea . All the previous DebConf organizers told me about all of this, but why didn t I give up? For the same reason you won t give up and for the same reason why we believe in the same goal. If anything this is one of the stories you ll want to tell your kids about so When I was your age I used to sleep naked in the snow would turn to When I was your age I organized DebConf11 in Maybe this blog post wasn t even necessary and some things weren t supposed to be said, I guess sometimes I just can t shut up For those who were on DebConf11 I hope you enjoyed and had a great time, otherwise I m saying all of this in vein. As organizer all you see is problems and you miss most of the confrence, but by what I know there are 11 candidate bids for DebConf13 and for me hearing that tells me the results of our efforts. DebConf11 after effects I could write a whole novel about this, but to keep it as short as possible, for last two years as a side project I was working on an idea of Government or some of its institutions migrating to Linux. At first I was somewhat loud about it, then after Microsoft heard about it and after they tried stopping the idea by trying to scare me by trying to interfere with my private life; as that didn t work its lobbyist came even near of obstructing the whole conference within the Government. For the sake of the conference, I convinced the Government that by supporting DebConf it doesn t mean they need to move to Linux and publicly stopped talking about it. I also convinced them that our only goal was to have successful conference and promote alternative options and open ideas. I wasn t lying as I saw this as new opportunity of them concluding on their own why they should or shouldn t not move, the better conference was the more chances of success we had. That s why I tried pushing as many representatives from various companies as in this case we would use reverse psychology where basically no one or few know what Linux or Debian for that matter is, but everybody knows who Google is, so if you have participants from i.e: Google or Austrian E-Health care system talking about how they are using your technology is better way to explain what s it all about really. Eventually Microsoft even had their first ever conference in Bosnia/Herzegovina and you wanna take a wild guess where it was held? In the end we had a great conference, after the conference we were the ones that were approached by some big local companies interested in future co-operation and in the end a meeting with Mark Shuttleworth and President along with the core of Government was scheduled. Topic? Migration to Linux. For me personally this meeting went better then I could possibly even picture it, many topics were discussed and basically it was up to us/me to make a draft of the project plan and submit request for proposal to the Government. There was still some lobbying but it seemed as it all disappeared, runway was clear and open for the lift off. Necessary contacts with potential future partners were made and before any meetings were scheduled I announced I ll be taking a short break. I missed this years vacation, for matter of a fact I missed going to a vacation for last two years. I decided going to Dubrovnik for few days, just to recollect my thoughts, recompile my kernel and switch to lower gear as I was stuck in 6th gear for way too long. Beginning of the end Being on this vacation , even if it was only for few days this gave me an opportunity to look at it all from a different prospective, different angle and that s when it came to me. I realized that even before I was finished with one project without even realizing I was already involved with another one, one that would take much more time and one that might leave me wondering ~5 years later with question what if? After all if get involved with something I m not going to back out, it s either failure is not an option or I won t start doing it at all. Even if it brought me everything I wanted, how would you know if that s what you wanted if you haven t thought about it first? Having DebConf for the first time in this region already showed some (positive) results, but for seeing the actual results it may take a bit longer. We sowed the seeds now the best thing we can do is leave them as it is and see if anything grows out. Of course I could lean over them and protect them from any natural or any other type of hazards, this would also mean when I m not around it wouldn t know how to protect itself on its own. Besides all, interfering with natures way of doing things is rarely a good option. I consider myself to still be relatively young and threatening my well being over safe keeping some seeds and watching over if they grow into something or not doesn t really seem that attractive to me. Maybe one day I ll decide to leave the land, maybe I won t even get to see if anything grows out after all. Maybe some day someone will find this fruit, recognize its worth and claim it as it s own, maybe over time everybody forgets everything and even that this land once was a fertile ground. Maybe, maybe not. I m still a student, I was supposed to graduate last year but I took less classes last year so I wouldn t snap under all the pressure. That s why this year I have less classes and am supposed to graduate in May, that s less then a year until then and I just want to devote some time to myself, figure out what I want with me life, learn the things I want to learn, do the things I want to do. I want to turn back to working on Eclipse, I have some good plans for Eclipse and Android developers, lately I ve been slacking off way too much but I think I have a good excuse In the end, all I know is that I still want to change the world, whether it s doing that inside of Google or in my own company, or some third place; or I ll just be laughing at this statement only time will show. I m hoping that now with all of this out of me I can finally finish the DebConf11 chapter of my life.

Lindsay Holmwood has written about the benefits of a standing desk and how to buy one [1]. The case for avoiding sitting is strong, but I couldn t stand up all day. One thing that s been on my list of things to do if I had an unreasonably large amount of spare time or money is to make a reclining computer station. The idea would be to take a bed and mount a TFT display above it so that it can be viewed while lying down. Then a split keyboard would be required so that each hand can be used for half the keys (this would be difficult or expensive). If the keyboard halves were aligned correctly then it would reduce the carpal tunnel problems associated with computer use (which has been a big problem for me in the past and is something that I will probably never fully recover from [2]). As far as I am aware the risk of back problems is eliminated when lying down, so two of the major problems with regular computer desks would be avoided. I don t think that lying down all day would be that great and it wouldn t work for collaborative projects. But as monitors are so cheap nowadays it would be viable to have a second monitor at a desk connected to the same computer. Then I could spend about half my computer time lying down.

• [1] http://holmwood.id.au/~lindsay//2011/10/03/standing-desk-adventures/
• [2] http://etbe.coker.com.au/2010/06/16/carpal-tunnel-getting-better/

If so, please tell me in the comments, how and for what purpose. I am going to remove this configuration feature as being an unnecessary complication, unless I hear compelling user stories.

Before I start, I'll admit that I'm not a real RPM packager. Maype I'm approaching this from completely the wrong direction, what do I know? I'm in the process of packaging Varnish 3.0.2 which includes mangling the spec file. The top of the spec file reads:

%define v_rc
%define vd_rc % ?v_rc:-% ?v_rc 

Apparently, this is not legal, since we're trying to define v_rc as a macro with no body. It's however not possible to directly define it as an empty string which can later be tested on, you have to do something like:

%define v_rc % nil 
%define vd_rc % ?v_rc:-% ?v_rc 

Now, this doesn't work correctly either. % ?macro tests if macro is defined, not whether it's an empty string so instead of two lines, we have to write:

%define v_rc % nil 
%if 0% ?v_rc  != 0
%define vd_rc % ?v_rc:-% ?v_rc 
%endif

The 0 ?v_rc != 0 workaround is there so that we don't accidentially end up with == 0 which would be a syntax error. I think having four lines like that is pretty ugly, so I looked for a workaround and figured that, ok, I'll just rewrite every use of % vd_rc to % ?v_rc:-% ?v_rc . There are only a couple, so the damage is limited. Also, I'd then just comment out the v_rc definition, since that makes it clear what you should uncomment to have a release candidate version. In my naivety, I tried:

# %define v_rc ""

# is used as a comment character in spec files, but apparently not for defines. The define was still processed and the build process stopped pretty quickly. Luckily, doing # % define "" seems to work fine and is not processed. I have no idea how people put up with this or if I'm doing something very wrong. Feel free to point me at a better way of doing this, of course.

The Occupy Wall St blog has an informative summary of attempts to reclaim the American political process which has been pwned badly by financiers in recent times [1]. The basic concept is that people who represent the 99% of the population who aren t super rich have protests in Wall St and now other business areas. Care2 has an interesting article about US marines opposing the brutal actions of police against Occupy Wall St protesters [2], apparently they treated Iraqis better than US police are treating Americans. The movement has spread to other locations, the OccupyTogether.org site has information on related events all around the world [3]. We have an ongoing event in Melbourne, Australia. It s been going for a week and yesterday Robert Doyle (the Mayor of Melbourne) ordered police to disperse the protest, so riot police and mounted police forced the protesters out of the city square [4]. According to the news reports there were only 100 people there at the time, here is a Google Maps link to the location, as you can see 100 people would not take up much of that space, not even with banners etc. The smart move would have been for the government to ignore it all until the protesters got bored. Now of course we will have more and bigger protests. The use of riot police will probably be considered as a good thing by some of the more aggressive protesters, but anyone who doesn t want to make the government (and the corporations that control it) look bad would consider it a gross error. Robert Doyle needs to be replaced, the liberal reason for replacing him is that we just don t want unnecessary force used against peaceful protesters. The conservative reason for replacing him is that he s grossly incompetent, he transformed a small protest that wasn t getting much media attention and appeared to be losing interest into a large protest with a lot of media attention. It will be interesting to see what happens next.

• [1] http://occupywallst.org/
• [2] http://www.care2.com/causes/u-s-marines-protect-occupy-wall-street-protesters-video.html
• [3] http://www.occupytogether.org/
• [4] http://www.theage.com.au/victoria/doyle-accused-of-heavyhandedness-in-halting-occupation-20111021-1mck9.html

A common question about hosting is whether to use a dedicated server or a virtual server. Dedicated Servers If you use a dedicated server then you will face the risk of problems which interrupt the boot process. It seems that all the affordable dedicated server offerings lack any good remote management, so when the server doesn t boot you either have to raise a trouble ticket with the company running the Data-Center (DC) or use some sort of hack. Hetzner is a dedicated server company that I have had good experiences with [1], when a server in their DC fails to boot you can use their web based interface (at no extra charge or delay) to boot a Linux recovery environment which can then be used to fix whatever the problem may be. They also charge extra for hands-on support which could be used if the Linux recovery environment revealed no errors but the system just didn t work. This isn t nearly as good as using something like IPMI which permits remote console access to see error messages and more direct control of rebooting. The up-side of a dedicated server is performance. Some people think that avoiding virtualisation improves performance, but in practice most virtual servers use virtualisation technologies that have little overhead. A bigger performance issue than the virtualisation overhead is the fact that most companies running DCs have a range of hardware in their DC and your system (whether a virtual server or a dedicated server) will be on a random system from their DC. I have observed hosting companies to give different speed CPUs and for dedicated servers different amounts of RAM for the same price. I expect that the disk IO performance also varies a lot but I have no evidence. As long as the hosting company provides everything that they offered before you sign the contract you can t complain. It s worth noting that CPU performance is either poorly specified or absent in most offers and disk IO performance is almost never specified. One advantage of dedicated servers in this regard is that you get to know the details of the hardware and can therefore refuse certain low spec hardware. The real performance benefit of a dedicated server is that disk IO performance won t be hurt by other users of the same system. Disk IO is the real issue as CPU and RAM are easy to share fairly but disk performance is difficult to share and is also a significant bottleneck on many servers. Dedicated servers also have a higher minimum price due to the fact that a real server is being used which involves hardware purchase and rack space. Hetzner s offers which start at E29 per month are about as cheap as it s possible to get. But it appears that the E29 offer is for an old server new hardware starts at E49 per month which is still quite cheap. But no dedicated server compares to the virtual servers which can be rented for prices less than $10 per month. Virtual Servers A virtual server will typically have an effective management interface. You should expect to get web based access to the system console as well as ssh console access. If console access is not sufficient to recover the system then there is an option to boot from a recovery device. This allows you to avoid many situations that could potentially result in down-time and when things go wrong it allows you to recover faster. Linode is an example of a company that provides virtual servers and provides a great management interface [2]. It would take a lot of work with performance monitoring and graphing tools to give the performance overview that comes for free with the Linode interface. Disk IO performance can suck badly on virtual servers and it can happen suddenly and semi-randomly. If someone else who is using a virtual server on the same hardware is the target of a DoS attack then your performance can disappear. Performance for CPU is generally fairly reliable though. So a CPU bound server would be a better fit on the typical virtual server options than a disk IO bound server. Virtual servers are a lot cheaper at the low end so if you don t need the hardware capabilities of a minimal dedicated server (with 1G of RAM for Hetzner and a minimum of 8G of RAM for some other providers) then you can save a lot of money by getting a virtual server. Finally the options for running a virtual machine under a virtual machine aren t good, AFAIK the only options that would work on a commercial VPS offering are QEMU (an x86 CPU instruction emulator), Hercules (a S/370 S/390, and Z series IBM mainframe emulator), and similar CPU emulators. Please let me know if there are any other good options for running a virtual machine on a VPS. Now while these emulators are apparently good for debugging OS development they aren t something that are generally useful for running a virtual machine. I knew someone who ran his important servers under Hercules so that x86 exploits couldn t be used for attacking them, but apart from that CPU emulation isn t generally useful for servers. Summary If you want to have entire control of the hardware or if you want to run your own virtual machines that suit your needs (EG one with lots of RAM and another with lots of disk space) then a dedicated server is required. If you want to have minimal expense or the greatest ease of sysadmin use then a virtual server is a better option. But the cheapest option for virtual hosting is to rent a server from Hetzner, run Xen on it, and then rent out DomUs to other people. Apart from the inevitable pain that you experience if anything goes wrong with the Dom0 this is a great option. As an aside, if anyone knows of a reliable company that offers some benefits over Hetzner then please let me know. What I would Like to See There is no technical reason why a company like Linode couldn t make an offer which was a single DomU on a server taking up all available RAM, CPU, and disk space. Such an offer would be really compelling if it wasn t excessively expensive. That would give Linode ease of management and also a guarantee that no-one else could disrupt your system by doing a lot of disk IO. This would be really easy for Linode (or any virtual server provider) to implement. There is also no technical reason why a company like Linode couldn t allow their customers to rent all the capacity of a physical system and then subdivide it among DomUs as they wish. I have a few clients who would be better suited by Linode DomUs that are configured for their needs rather than stock Linode offerings (which never seem to offer the exact amounts of RAM, disk, or CPU that are required). Also if I had a Linode physical server that only had DomUs for my clients then I could make sure that none of them had excessive disk IO that affected the others. This would require many extra features in the Linode management web pages, so it seems unlikely that they will do it. Please let me know if there is someone doing this, it s obvious enough that someone must be doing it. Update: A Rimuhosting employee pointed out that they offer virtual servers on dedicated hardware which meets this criteria [3]. Rimuhosting allows you to use their VPS management system for running all the resources in a single server (so no-one else can slow your VM down) and also allow custom partitioning of a server into as many VMs as you desire.

• [1] http://www.hetzner.de/
• [2] http://www.linode.com/
• [3] https://rimuhosting.com/order/vps-on-dedicated.jsp

[UPDATE 2011-10-26] The story has developed.[/UPDATE] Recently I was talking with somebody about the WEB.DE toolbar, and today I finally got around to look something up I was told then. And while I searched for the information on the download page for the WEB.DE toolbar, I stumbled over the download URL for the installer:

http://wa.ui-portal.de/webde/webde/s?produkte.browserdownload.link.download.ff7&bd_mc=undef_undef&ns_type=pdf&ns_url=http://dl.web.de/browser/firefox/WEB.DE_MFF7_Setup.exe

There I was magically attracted by the ns_url parameter, because it smelled like something where you could just pass any URL along. And indeed you can: http://wa.ui-portal.de/webde/webde/s?produkte.browserdownload.link.download.ff7&bd_mc=undef_undef&ns_type=pdf&ns_url=http://bundestrojaner.net/inhalt-bundestrojaner-gratis-download-3.html. That should redirect you to http://bundestrojaner.net/inhalt-bundestrojaner-gratis-download-3.html instead of http://dl.web.de/browser/firefox/WEB.DE_MFF7_Setup.exe. This is a serious flaw, as it might allow an attacker to request a username and password combination from an unsuspecting user. With a harvest page designed to look like another WEB.DE page you should have a high return rate. This is, by the way, a flaw on the OWASP Top 10 (2010 edition). I've informed somebody working for United Internet and expect the bug to be resolved pretty soon, hopefully it won't work anylonger when you read this.

Red Hat s Matthew Garrett let the cat out of the bag about a month ago: when UEFI Secure Boot is adopted by mainboard manufacturers to satisfy Microsoft Windows 8 requirements, it may very well be the case that Linux and others (BSD, Haiku, Minix, OS/2, etc) will no longer boot. Matthew has written about it extensively and seems to know very well what the issues are (part I, part II), the details about signing binaries and why Linux does not support Secure Boot yet. The Free Software Foundation has also released a statement and started a campaign, which is, as usually, anti-Microsoft instead of pro-solutions. Now let me express my opinion on this matter: this is not Microsoft s fault. Facts Let s see what are the facts in this controversy:

• Secure Boot is here to stay. In my humble opinion, the idea is good and it will prevent and/or lessen malware effects, especially on Windows.
• Binaries need to be signed with a certificate from the binaries vendor (Microsoft, Apple, Red Hat, etc)
• The certificate that signs those binaries needs to be installed in the UEFI BIOS
• Everybody wants their certificate bundled with the UEFI BIOS so that their operating system works out of the box
• Given that there are many UEFI and mainboard manufacturers, getting your certificate included is not an easy task: it requires time, effort and money.

Problem The problem stems from the fact that most Linux vendors do not have the power to get their certificates in UEFI BIOS. Red Hat and Suse will for sure get their certificates bundled in server UEFI BIOS. Debian and Ubuntu? Maybe. NetBSD, OpenIndiana, Slackware, etc? No way. This is, in my humble opinion, a serious defect in the standard. A huge omission. Apparently while developing the Secure Boot specification everybody was busy talking about signed binaries, yet nobody thought for a second how the certificates will get into the UEFI BIOS. What should have been done The UEFI secure boot standard should have defined an organization (a Secure Boot Certification Authority ) that would issue and/or receive certificates from organizations/companies (Red Hat, Oracle, Ubuntu, Microsoft, Apple, etc) that want their binaries signed. This SBCA would also be in charge of verifying the background of those organizations. There is actually no need for a new organization: just use an existing one, such as Verisign, that carries on with this task for Microsoft for kernel-level binaries (AuthentiCode). Given that there is no Secure Boot Certification Authority, Microsoft asked BIOS (UEFI) developers and manufacturers to include their certificates, which looks 100% logical to me. The fact that Linux distributions do not have such power is unfortunate, but it is not Microsoft s fault at all. What can we do? Given its strong ties with Intel, AMD and others, maybe the Linux Foundation could start a task force and a Temporary Secure Boot Certification Authority to deal with UEFI BIOS manufacturers and developers. This task force and TSBCA would act as a proxy for minorities such as Linux, BSD, etc distributions. I am convinced this is our best chance to get something done in a reasonable amount of time. Complaining will not get us anything. Screaming at Microsoft will not get us anything. We need to propose solutions. Wait! Non-Microsoft certificates? Why? In addition to the missing Secure Boot Certification Authority, there is a second problem apparently nobody is talking about: what is the advantage mainboard manufacturers get from including non-Microsoft certificates? For instance: why would Gigabyte (or any other mainboard manufacturer) include the certificate for, say, Haiku? The benefit for Gigabyte would be negligible and if someone with ill-intentions gets Haiku s certificate, that piece of malware will be installable on all Gigabyte s mainboards.This would lead to manufacturer-targetted malware, which would be fatal to Gigabyte: oh, want to be immune to the-grandchild-of-Stuxnet? Buy (a computer with) an MSI mainboard, which does not include Haiku s certificate Given that 99% of desktops and laptops only run Windows, the result of this (yet unresolved) problem would be that manufacturers will only install Microsoft certificates, therefore they would be immune to malware signed with a Slackware certificate in the wild. If we are lucky, mainboard manufacturers will give us an utility to install more certificates under your own risk. The solution to the first problem looks easy to me. The solution to the second looks much more worrying to me.

Let's start that a new meme on planet, haven't seen one for a while. The Austrian president starts the relaunched web presence with a "Putin" style parachute jump.

 lynx -dump -head http://www.bundespraesident.at/ 
HTTP/1.1 200 OK
Date: Thu, 20 Oct 2011 08:42:35 GMT
Server: Apache/2.2.16 (Debian)
X-Powered-By: PHP/5.3.3-7+squeeze3

So apparently the used typo3 installation is newer, but the used OS for the Austrian presidential web site is: Debian Squeeze.

I've been wanting to sponsor a child in Africa for many, many years. I remember seeing a photo of a sponsored child on a couple of sets of friends' mantelpieces in Canberra, and saying to myself how I should really do that too. Of course, for no really good reason, fast-forwarding to 6+ years later, I still hadn't gotten around to it. Until last month. I was walking out of the Apple Store in Palo Alto. I was accosted by one of those people on the street, with the clipboard and all that. I usually manage to brush these sorts of people off. I really hate solicited donations. I'd rather consciously make a donation to a charity, than do it because someone on the street was paid to ask me to, or knocked on my door, or whatever. But I was distracted, or in a good mood, or something, and so I missed the initial opportunity to deflect her. I'm really glad I did, because as I said, this is something that I've wanted to do for years, so I barely hesitated, and signed up on the spot. I received the welcome kit in the mail a couple of days ago. I've been allocated a 6 year old boy named Armando, from Mozambique. I'm looking forward to corresponding with him. Our friends, and former neighbours, the Harvetts have moved back to South Africa, and I hope one day to be able to visit them. Since we'll be in the vicinity, I hope one day to be able to visit Armando as well. I'm pleased to see that Save the Children gets a good rap in Charity Navigator. I was worried that I'd get home and find out I'd signed up with one of those charities that blows half their money on overhead, but that's not the case.

I received recently a mail about my attempt to provide Grsecurity kernels in Debian. The sender found the bug by accident, and asked me why I didn't do some more publicity here. So here we are. I won't go into details on what grsecurity is, it's fairly complex. But it's basically a hardening patch for the Linux kernel, with three main components:

• the PaX patch, which purpose is to harden the memory layout of the Linux kernel and improve existing options: enforcing of non-executable memory pages (userland and in kernel), W^X (no page marked as writable and executable), ASLR, prevention of invalid userland pointers dereference, copies between userland and kernel memory
• RBAC (Role Based Access Control), an implementation of Mandatory Access Control
• various hardening features: /proc restrictions, chroot restrictions, kernel symbols hiding etc.

A lot of this touches low level stuff in the kernel, especially memory management. Ideally this patch would be pushed upstream, but Brad Spengler (grsecurity main developper) already said he wasn't interested in upstreaming it and upstream already said the patch was too huge and invasive to include it like that (especially since the original authors aren't interested in maintaining it upstream). There's an ongoing effort to split the patch and merge things little by little, but in the meanwhile having a mid-term solution would be nice. I know Debian users rebuilding grsecurity-patched kernels themselves, and I know some of them would appreciate having them included in the Debian kernel. Fortunately, the linux-2.6 source package has a nice feature which is called featureset. Basically it's a way to build some (binary) packages using a different set of patches and a different config. For example this was used to provide xen/openvz/vserver patchsets, and is now used to provide rt kernels. So I though it'd be nice to provide a grsec featureset, and starting doing the work. I have a working setup for producing those kernels, so I've opened a wishlist bug against the kernel (#605090) to have this merged. Those packages follow the sid kernel. There's an ongoing work for Squeeze, but it's a bit harder there because both the grsecurity patchset and the Debian kernel ship a whole lot of backports to the Linux kernel, meaning the grsecurity patch doesn't apply directly to the Debian source package. Basically I need to remove some of the hunks (since they are already applied to the source) and port some others (since there are some backported code not present in the vanilla 2.6.32, for example the drm code). Until the patches are merged and the bug is closed, I host some of the built packages at:

deb http://molly.corsac.net/~corsac/debian/kernel-grsec/packages/ sid/ The repository is signed by my key which you can add to your apt setup using apt-key add. If you want to rebuild the packages yourself, here's the method:

mkdir kernel-grsec
cd kernel-grsec
svn checkout svn://svn.debian.org/svn/kernel/dists/sid/linux-2.6
git clone git://anonscm.debian.org/users/corsac/grsec-patches.git
wget http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.0.tar.bz2
wget http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.0.tar.bz2.sign
gpg --verify linux-3.0.tar.bz2
cd linux-2.6
apt-get build-dep linux-2.6
export QUILT_PATCHES=../grsec-patches
quilt push -a
python debian/bin/genorig.py ../linux-3.0.tar.bz2
debian/rules orig
fakeroot debian/rules source
fakeroot make -f debian/rules.gen binary-arch_amd64_grsec_amd64 You could also do dpkg-buildpackage, pdebuild or whatever. Kernel handbook is a nice reading too if you want more information on how to rebuild Debian kernels. The quilt push -a may fail if you checkout an svn version more recent than mine. I try to keep patches up to date but I usually have some delay. Note that installing the kernel will require installing linux-grsec-base package. Binary is not yet available on my mirror but you can easily build it. Source can be found on git.debian.org. If you're interested by this, don't hesitate to mail me or the bug.

I'm building a debian package, CDAT. The latest version 6.0.alpha uses CMAKE to build, rather than configure. The trouble is that CMake doesn't build. It doesn't even fail.

$ mkdir build
$ cd build
$ cmake ..
()-- Configuring done
-- Generating done
-- Build files have been written to: /home/amckinstry/deb-packages /cdat/cdat-6.0.alpha/build
$ make
$

The problem is, 'make' does nothing. A Makefile is generated by CMake, which calls cmake which calls make again on ./CMakeFiles/Makefile2 ... which does nothing useful. Apparently the CMake is supposed to put useful stuff in there, but doesn't. What puts stuff into Makefile2, and where should I pick up the bugs trail? Tags lazyweb, cmake, debian

Since the 3.5.0 release, Openbox automatically starts applications with .desktop files in /etc/xdg/autostart. This was already the case with previous releases when setting the . $GLOBALAUTOSTART instruction in the autostart script. This instruction is now deprecated. This has puzzled me as some applications were started twice in my Openbox session: those launched via my ~/.config/openbox/autostart script which are also launched through xdg autostart. I also like to use notify-osd when in openbox, but it seems the new gnome-fallback session requires notification-daemon. In my Openbox session, I have noticed that notification-daemon was launched whereas my autostart script specifies to use notify-osd. I then discovered that /etc/xdg/autostart/notification-daemon.desktop was the cause of this. I had to drop a notification-daemon.desktop file in my ~/.config/autostart/ directory with the following contents: The last two lines allow me to not start notification-daemon only when using Openbox ie. it still starts automatically when launching a GNOME session. You can check what applications are automatically started through xdg with: The results are quite amazing, most applications being only configured to start in GNOME or KDE, but nothing seems to be very coordinated amongst maintainers. I find this quite misleading, and would prefer openbox to stick with the previous default behaviour. Who said diehard ?!

This post is made in a personal capacity. While it represents the outcome of multiple discussions on the topic with a variety of people, nothing written below should be construed as Red Hat's corporate position

The FSF have released a statement on UEFI secure boot. It explains the fundamental issue here, which isn't something as simple as "will OEMs let me install Linux". It's "Does the end user have the ability to manage their own keys".

Secure boot is a valuable feature. It does neatly deal with the growing threat of pre-OS malware. There is an incentive for it to be supported under Linux. I discussed the technical aspects of implementing support for it here - it's not a huge deal of work, and it is being worked on. So let's not worry about that side of things. The problem is with the keys.

Secure boot is implemented in a straightforward way. Each section of a PE-COFF file is added together and a hash taken[1]. This hash is signed with the private half of a signing key and embedded into the binary. When you attempt to execute a file under UEFI, the firmware attempts to decrypt the embedded hash. This requires that the firmware have a either a copy of the public half of the signing key in its key database, or for there to be a chain of trust from the signing key to a key in its key database. Once it has the decrypted hash, it generates its own hash of the binary and compares them. If they match, the binary is executed.

What happens if it doesn't match? Per the UEFI specification, the firmware can then prompt the user and ask if they want to execute it anyway. If the user accepts then the hash of the binary is remembered[4] and can continue to be executed in future. This is similar to what you get when you visit a self-signed https site, or when you connect to an ssh server for the first time - the user must explicitly state that they trust the software that is being booted.

This is a little awkward for a couple of reasons. First, it means that any updates to the bootloader would require the user to manually accept the binary again. Second, as we've seen with https, there's the risk of uesrs just blindly accepting things. If it's acceptable to do this off internal media[5] then malware could just rely on some number of users saying "yes". And thirdly, we've been told that Microsoft's Windows logo requirements forbid this option.

The first is problematic but manageable with appropriate documentation. The second is a real risk that makes this approach far less compelling. And the third is probably a showstopper.

So signed but untrusted binaries are probably out, on technological, UI and policy grounds. What else?

We can sign binaries with a trusted key. This gets us interoperability, but comes at a cost. The first issue here is that we still don't know what the process for getting a trusted key is going to be. One option is for something like a generic Linux key that's provided by all firmware vendors. As long as the OEM doesn't remove it, then that Linux key could be used to sign other keys that are used by individual Linux vendors. The main difficulty involved in this is having the Linux key be managed by an authority that OEMs can trust to only provide keys to trustworthy organisations and maintain control of the private half of the key. Remember, if any malware is signed with any of these keys, it'll boot on any machine carrying this key. It's a lot of trust. It's not obvious that there's an organisation with sufficient trust at the moment.

The second approach is for each Linux vendor to generate their own key and get every OEM to carry it. Let's rule this out on the basis that it's an approach that doesn't scale at all. Red Hat could probably get its key into a bunch of OEM systems. Canonical too. Oracle? Probably for the vendors they care about. It's likely to be much harder for everyone else, and you still end up with the risk that you've just bought a computer that'll boot Red Hat but not Ubuntu.

The third approach is for each Linux vendor to obtain a key from someone who's already trusted. The most obvious authority here is Microsoft. Ignoring the PR unpleasantness involved with Linux vendors having to do business with Microsoft to get signed, so far Microsoft haven't given us any indication that this will be possible.

So every approach that involves us signing with a trusted key comes with associated problems. There's also a common problem with all of them - you only get to play if you have your own trusted key. Corporate Linux vendors can probably find a way to manage this. Everyone else (volunteer or hobbyist vendors, end users who want to run their own OS, anyone who wants to use a modified bootloader or kernel) is shut out.

The obvious workaround is for them to just turn off secure boot. Ignoring the arguments over whether or not OEMs will provide that option[6], it benefits nobody for Linux installation to require disabling a legitimate security feature. It's also not likely to be in a standard location on all systems and may have different naming. It's a support nightmare. Let's focus on trying to find a solution that provides the security and doesn't have obvious scaling issues.

There's a pretty simple one. If the problem centres around installing signing keys, why not have a standardised way of installing signing keys?

What we've proposed for this is an extension of the current UEFI standard for booting off removable media. There's a standardised location for putting the bootloader for USB sticks or optical media. For 32-bit PCs, it's EFI/BOOT/BOOTIA32.EFI. For 64-bit, it's EFI/BOOT/BOOTX64.EFI. Our proposal is to specify a path for key storage on the media. If the user attempts to boot off an item of removable media and finds a key file, we would like the firmware to prompt the user to install the key. Once the key is installed, the firmware will attempt to boot the bootloader on the removable media.

How does this avoid the problems associated with prompting the user to boot untrusted binaries? The first is that there's no problem with updates. Because a key has been imported, as long as future bootloader updates are signed with the same key, they'll boot without prompting the user. The second is that it can be limited to removable media. If malware infects the system and installs itself onto the hard drive, the firmware won't prompt for key installation. It'll just refuse to boot and fall back on whatever recovery procedures the OEM has implemented. The only way it could get on the system would involve the user explicitly booting off removable media, which would be a significant hurdle. If you're at that stage then you can also convince the user to disable secure boot entirely.

This proposal has been brought up with the UEFI standards working group, and with the UEFI plugfest taking place next week we're hoping it'll be discussed. It seems to satisfy the dual requirements of maintaining system security and ensuring that users have the flexibility to choose what they run on their system. But as with all technical proposals, feedback is entirely welcome.

[1] Hilariously, Microsoft's signing tool gets this wrong by also adding the contents of gaps between sections in direct contravention of their own specification. This is fine for binaries generated by Microsoft's toolchain because they don't have any gaps, but since our binaries do contain gaps[2] and since the standard firmware implementation[3] does implement the specification correctly, any Linux-generated binaries signed with the Microsoft tool fail validation. Go team.
[2] Something that is, as far as we can tell, permitted by the PE-COFF specification
[3] Written by Intel, not Microsoft
[4] Note that this means that only the specific binary is considered trusted. No key is imported, and any other binaries signed with the same key will also need to be manually trusted in the same way.
[5] As would be required if you ever want to be able to update your bootloader
[6] And to forestall panic, at this point we expect that most OEMs will provide this option on most hardware, if only because customers will still want to boot Windows 7. We do know that some hardware will ship without it. It's not implausible that some OEMs will remove it in order to reduce their support burden. But for the foreseeable future, you'll be able to buy hardware that runs Linux.

comments

OpenXenManager cleared the NEW queue and is now available in the archive. What it is:read more

I ve recently been to the LibreOffice Conference 2011 in Paris. It s very nice to finally meet in person all those names from the malinglist and the bug reports. Although meeting most people for the first time, I left the conference with a strong feeling of a community (which I m proud to be a member of), this is a very motivating feeling. During the conference I had the chance to work on LibreOffice related issues:

• Hebrew translation for version 3.5 (up by 2%).
• Fixes to the Hebrew translation for version 3.4 (noticed a few string changes in 3.3 not included in 3.4).
• Patch to fix a small typo in the code (noticed while trying to find the meaning of the string before translating).
• Patch to fix the appearance of the BrOffice logo in RTL UI instead of LibreOffice (fix to the 3.3 branch, already fixed in 3.4).
• 3 patches (1, 2, 3) to cleanup unused .chaos file types with Michael Meeks s encouragement and help with doing some digging in the code.
  Apparently, even non programmers can write patches with the appropriate help.
• Check about 30 RTL and Hebrew related bug reports for my RTL issues presentation, verifying them with 3.4.3.

The 3 patches began when I noticed a few weird strings to translate in the context of installation media good luck to those who will try to install LibreOffice from 5.25 disk, 3.5 disk or a tape drive. These 3 strings are now removed from the code with other 36 unused strings. So translators please take the strings to translate with a grain of salt. Post conference tasks (always ending up with more stuff to do after a conference):

• Recreate my RTL issues presentation so that the live example of bugs could be presented as slides.
• Keep advancing with the Hebrew translation (3.5 and 3.4 fixes).
• Finish the fdo bugzilla search for RTL bugs. I found out some other people who reports bugs (and try to coordinate with them).
  A list of RTL related bugs in LibreOffice I know of is here: http://wiki.hamakor.org.il/index.php/ _ _ (still work in progress, in Hebrew).
• Create a meta bug for the RTL bugs, and prioritize the issues to focus on what s important to make sure is fixed for version 3.5.
• Start testing version 3.5 for RTL bugs (daily / weekly builds).
• Help Christoph Noack with finding which PaperCuts we have in LibreOffice and try to get them fixed.

Filed under: LibreOffice, Openoffice.org

Rapha l Hertzog recently announced a new dpkg-buildflags interface in dpkg that at long last gives the distribution, the package maintainers, and users the control they want over the build flags used when building packages. The announcement mail gives all the gory details about how to invoke dpkg-buildflags in your build to be compliant; but the nice thing is, if you're using dh(1) with debian/compat=9, debhelper does it for you automatically so long as you're using a build system that it knows how to pass compiler flags to. So for the first time, /usr/share/doc/debhelper/examples/rules.tiny can now be used as-is to provide a policy-compliant package by default (setting -g -O2 or -g -O0 for your build regardless of how debian/rules is invoked). Of course, none of my packages actually work that way; among other things I have a habit of liberally sprinkling DEB_MAINT_CFLAGS_APPEND := -Wall in my rules, and sometimes DEB_LDFLAGS_MAINT_APPEND := -Wl,-z,defs and DEB_CFLAGS_MAINT_APPEND := $(shell getconf LFS_CFLAGS) as well. And my upstreams' build systems rarely work 100% out of the box with dhauto* without one override or another somewhere. So in practice, the shortest debian/rules file in any of my packages seems to be 13 lines currently. But that's 13 lines of almost 100% signal, unlike the bad old days of cut'n'pasted dh_* command lists. The biggest benefit, though, isn't in making it shorter to write a rules file with the old, standard build options. The biggest benefit is that dpkg-buildflags now also outputs build-hardening compiler and linker flags by default on Debian. Specifically, using the new interface lets you pick up all of these hardening flags for free:

-fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -Wl,-z,relro

It also lets you get -fPIE and -Wl,-z,now by adding this one line to your debian/rules (assuming you're using dh(1) and compat 9):

export DEB_BUILD_MAINT_OPTIONS := hardening=+pie,+bindnow

Converting all my packages to use dh(1) has always been a long-term goal, but some packages are easier to convert than others. This was the tipping point for me, though. Even though debhelper compat level 9 isn't yet frozen, meaning there might still be other behavior changes to it that will make more work for me between now and release, over the past couple of weekends I've been systematically converting all my packages to use it with dh. In particular, pam and samba have been rebuilt to use the default hardening flags, and openldap uses these flags plus PIE support. (Samba already builds with PIE by default courtesy of upstream.) You can't really make samba and openldap out on the graph, but they're there (with their rules files reduced by 50% or more). I cannot overstate the significance of proactive hardening. There have been a number of vulnerabilities over the past few years that have been thwarted on Ubuntu because Ubuntu is using -fstack-protector by default. Debian has a great security team that responds quickly to these issues as soon as they're revealed, but we don't always get to find out about them before they're already being exploited in the wild. In this respect, Debian has lagged behind other distros. With dpkg-buildflags, we now have the tools to correct this. It's just a matter of getting packages to use the new interfaces. If you're a maintainer of a security sensitive package (such as a network-facing daemon or a setuid application), please enable dpkg-buildflags in your package for wheezy! (Preferably with PIE as well.) And if you don't maintain security sensitive packages, you can still help out with the hardening release goal.

Full stats are here Status for D-I level 1 (core D-I files):

• 18 languages 100% for all sublevels: bg cs de eo es fa fr it ja ko nl pl pt ru ta te th zh_CN
• 2 languages 99%: si
• 2 language 98%: gu sv
• 1 language 97%: eu
• 2 languages 96%: hi tr
• 5 languages 95%: be bn dz sk uk
• 4 langauges 94%: ast bs ga ro
• 3 languages 93%: hu lo nb
• 2 languages 92%: id sl
• 5 languages 91%: et mr pa vi zh_TW
• others are 90% or below

Status for D-I level 2 (packages that have localized material that may appear during default installs, such as iso-codes, tasksel, etc.):

• 8 languages 100%: de es fa fr pt ru si zh_CN
• 14 languages 99%: be cs da eo id it ja kk nl sk sl sv th uk
• 4 languages 98%: bg ca eu ro
• 1 language 97%: pt_BR
• 4 langauges 95%: el fi he nb
• 7 languages 94%: ar ast dz gl hr vi zh_TW
• 1 language 93%: is
• 7 languages 92%: bn bs hu ko ne pl tr
• 9 languages 91%: ga gu ka km lt mk mr sr ta
• others are 90% or below

Status for D-I level 3 (packages that have localized material that may appear during non-default installs, such as win32-loader)

• 6 languages 100%: cs fa fr nl pt sv
• 19 languages 99%: be bg bs ca de el eo es fi ga gl he is ja ru sk sv th tr zh_CN
• 2 languages 97%: hu uk
• others are 90% or below

So, full 100% completeness (hall of fame) for: Persian (fa), French (fr), Portuguese (pt).