The new ncurses library that is currently sitting in the NEW queue will have a significant change that, if we ve done it right, should be an invisible change. If we ve got it wrong, then the BTS (and us) may be a little busy. What has happened is that the low-level terminfo library has been split out from the general ncurses library making a new package called libtinfo5. This also means that there only needs to be one libtinfo5 package because at this low level it doesn t care about wide characters, so there is no libtinfo5w (and that gets important later). libncurses is linked to libtinfo so from the point of view of using libncurses there should be no major change in how things are done. So, why bother? The way ncurses was built before causes a nice long conga line of trouble . There really are (was) two main ncurses library packages, libncurses5 and libncursesw5. The w meant it was for wide characters. So anything that uses ncurses needs to decide which one to link to or perhaps both. A problem came up with the readline library that needs to link to ncurses because programs that link to IT (such as python) want a wide library but other programs do not. One solution is to have two readline libraries, but then any other library would need to be compiled and built twice as well and so on and so on. readline didn t really want the full ncurses library, it only wants the terminfo part but there was (until now) no way of picking up that bit only. Once this new version of ncurses gets released then readline can link to libterminfo which is agnostic to wide and non-wide characters which means it is as well which them means everything else should, in theory work. If you are using ncurses for some low-level terminfo work only, you may want to have a look to see if you can link to libtinfo5 only which will mean depending on libtinfo5 and not libncurses5 or libncursesw5. I m just writing this up mainly and uploading the packages, the bulk of the work was done by Sven and Matthias as well as some help from our ever-helpful upstream Thomas.

There's loads of music on YouTube. However, when I want to listen to music, I'm not necessarily interested in slideshows. And yes, that's a euphemism. Luckily, extracting the video and audio from a youtube clip isn't very hard.

• First, download the clip. Yes, that's possible. This will get you a movie clip, by default in the highest quality format available for the particular clip you're trying to download.
• You want to know which codecs the clip is using. Mplayer to the rescue:

  wouter@carillon:~$ mplayer -frames 1 -v <file> 2>&1 grep -i 'stream.*audio'
  [lavf] stream 1: audio (vorbis), -aid 0
  

  meaning, this particular file has Vorbis audio.
• Next, extract the audio:

ffmpeg -i <file> -vn -acodec copy <output>.ogg

The critical bit is the '-vn' part, which tells ffmpeg not to encode the video to the output file. The 'mplayer' step isn't strictly required; but since youtube only uses lossy audio codecs, re-encoding the audio to a different codec will cost you audio quality. You may not want to do that; hence the mplayer step (to figure out which codec the source file is using), and the '-acodec copy' argument to ffmpeg's command line (which tells ffmpeg to not re-encode the audio, but just store the unmodified original audio stream in a different container file). Tadaa, music without video.

Today, German politicians decided, that Germany be liable for up to 211 billion Euros for the debts of other EU countries. Or, put differently: the politicians put the money of the current and future generations on the line for a country that lived way above its capabilities for too many years. When the EU was founded, it was explicitly stated that no country needs ever be liable for any other. I understand that politics is hard, and letting Greece (and others) fall down might carry heavy, unforeseen consequences. Also, I understand that the Greek people are mostly innocent in all this and that the fault lies with their politicians and other corrupt entities in the nation. However, what s happening these days is beyond the comprehensible. If I were in a non-Germany EU country, I d rejoice and continue making debts. It is likely that this won t be the last time that our politicians cave in to pressure by other nations who have a lesser understanding of budgeting and saving. Since I am German, I can only shake my head, look to Berlin and ask myself whether this is the final straw that broke the camel s back. How the heck do the people over there ever want to regain the trust of their people? Politics has become the game of pleasing each other, who cares about the people? And the German politicians are (once again, remember credit default swaps?) at the forefront of this stupidity. To me, there is only one solution to Greece s debts: make sure that what happened can never happen again, and then cut the debts, or slice them in half. Let the banks carry the weight, for it was them who gave out the loans too liberally. And if this forces a bank or two to default, let it happen, for fuck s sake. The consequences might be dire, but they ll subside. And that s surely better than trying to pretend that we can keep juggling this heavily inflated financial system. Instead, the executives, elected to carry the trust of the people, are setting precedents for other countries to follow Greece, for it is likely that they wil be bailed out. By us. That is not the way to teach anyone the basics of economy: you can only spend as much as you earn, without exceptions. Debts will only come around to hurt you. I could puke. NP: Godspeed You Black Emperor!: F# A#

Besides FusionForge stuff, I also kept busy with other things.

• A new electronic drumkit: I upgraded from a Roland TD-4K to the new TD-9KX2. Apart from the improvements of the sound module and the mesh-head pads, it has two goodies. First, an extra trigger input line. Which means I just had to plug a cymbal pad from my Guitar Hero drum controller (with a standard jack cable), so I get an extra pad. More cowbell! Second, it now has a MIDI-in port. Which means that I "just" have to write some code (probably using libcwiid) to get the signals from the GH controller to a convenient computer nearby, then turn them into MIDI signals and send them to the Roland kit. So that I get not one but six extra pads (actually, five and a pedal). Special effects, lasers, symphonic chords, cymbals of all sizes, steel drums and more cowbell!
• Big mistake: I borrowed a Zelda Twilight Princess game. I'm ashamed to admit that it ate my week-end, and it's likely to eat quite a bit of my time again in the near future, although I'll try to put it on a diet. Yeah, okay, I had sort of guessed that games had improved a lot since the days of Might & Magic, but wow, I didn't realise how much.
• Rapha l and I started the fundraising campaign for the translation of our Debian book to English. Pre-orders/pledges are collected on Ulule; we'll only get them if the threshold is met, but the counterpart is that we won't actually do the translation if the threshold isn't met: translation and proofreading and publication of a book take a tremendous amount of time, and that's going to be time spent away from our respective clients. Note that there's a second threshold, for the "liberation fund"; if that one is met, the result of the book will be released under a Free license (most probably a CC-BY-SA + GPL dual-license). There are a whole lot of options for contributing, with rewards according to the amount you choose: e-book, paperback book, updates to the e-book, dedication, and even a dinner with the authors (limited offer, while supplies last).

A summary of my FusionForge-ish activities this month:

• FusionForge 5.1 is now available in Debian testing ("wheezy"). 5.0 is still in Debian stable ("squeeze"), so support continues; nevertheless, the improvements are significant enough that I recommend new users to run the version from testing if possible. Note that you can install the packages from testing onto a system running stable, either by hand or with APT pinning (there are only a handful of packages you need to grab from testing to satisfy the dependencies).
• This was made possible by some bugfixing in the 5.1 branch of FusionForge. Nothing major in itself, but the accumulated fixes will probably be worth a 5.1.1 release soon.
• I didn't do much on the trunk apart from merging from 5.1, but others did: several new plugins appeared, notably as part of the Coclico project (which is approaching its end fast).
• Also, not yet pushed to trunk are two local branches I have: one is a plugin to integrate MoinMoinWiki (with permissions managed through the FusionForge roles), developed with a client of mine. The other will end up in reuniting the FusionForge database upgrade process across installation methods (the *.rpm packages and the source installation currently use a PHP script, the *.deb ones use a Perl script with transactions). It'll probably break many things when initially committed, so I'll keep it on a local branch until I feel it's mature enough.
• I updated the "sandbox virtual machine" image; it now uses Debian wheezy, and the testsuite now runs a more recent version of Selenium; more importantly, the testsuite now passes.

The following is a quick HOWTO for the brave Debian users who want to upgrade to GNOME 3. Assuming you have an up to date system running sid, and experimental listed in your APT sources, perform the following complicated steps to end up having a functional GNOME 3 desktop:

apt-get install -t experimental gnome

Thanks go to Joss for putting together new GNOME 3 meta-packages, and the rest of the Debian GNOME people for months of hard planning and packaging work, and painful testing transition handling. Before you ask, yeah, not all of GNOME 3.x is in unstable yet, but will soon be, as precedent transitions start clearing the way. And yeah, GNOME 3.2 will come just after the two remaining package sets enter testing. To compensate, you'll find that you have some GNOME leaf packages pending an upgrade to 3.2.0-1 while you read this.

This weekend we'll be having the 4th version of the Colombian Mini-Debconf. It will be held in the Otraparte Museum, in the city of Medellin. The cool thing this year is that we'll be having mostly Debian contributors than just users, with the kind participation of Anibal Monsalve, who is currently in the country.

There will be several 10-min lightening talks, followed by a key signing party and some actual work in packaging, and of course we'll be having some beers with Debian friends.

More info (in Spanish): http://wiki.debian.org/DebianColombia/MiniDebconf2011

Session reuse is one of the most important mechanism to improve SSL performance: by submitting an appropriate blob to the server, a client is able to trigger an abbreviated handshake, improving latency and computation time. There exists two distinct ways to achieve session reuse: session identifiers as described in RFC 5246 and session tickets as depicted in RFC 5077. A quick note: when I say SSL, unless I mention a version number, you should read TLS instead. Look at the article for TLS on Wikipedia for some background on this.

Theory of operation To establish a SSL connection, four messages need to be exchanged between client and server. With a latency of 50 ms, we have a 200 ms overhead to establish the connection (plus TCP handshake). Moreover, to share a common secret, both the client and the server needs to achieve some public-key cryptographic operations which are costly, computation-wise. To avoid a full SSL handshake each time a client request a resource, it is possible for the client to request an abbreviated handshake, saving a complete round-trip (100 ms) and avoiding the costliest part of the full SSL handshake. Two mechanisms can be used to accomplish an abbreviated handshake:

1. When the server sends the Server Hello message, it can include a session identifier. The client should store it and present it in the Client Hello message of the next session. If the server finds the corresponding session in its cache and accepts to resume the session, it will send back the same session identifier and will continue with the abbreviated SSL handshake. Otherwise, it will issue a new session identifier and switch to a full handshake. This mechanism is detailed in RFC 5246. It is the most common mechanism because it exists since earlier versions of SSL.
2. In the last exchange of a full SSL handshake, the server can include a New Session Ticket message (not represented in the handshake described in the picture) which will contain the complete session state (including the master secret negotiated between the client and the server and the cipher suite used). Therefore, this state is encrypted and integrity-protected by a key known only by the server. This opaque datum is known as a session ticket. The details lie in RFC 5077 which supersedes RFC 4507.

The ticket mechanism is a TLS extension. The client can advertise its support by sending an empty Session Ticket extension in the Client Hello message. The server will answer with an empty Session Ticket extension in its Server Hello message if it supports it. If one of them does not support this extension, they can fallback to the session identifier mechanism built into SSL. RFC 5077 identifies situations where tickets are desirable over session identifiers. The main improvement is to avoid to maintain a server-side session cache since the whole session state is remembered by the client, not the server. A session cache can be costly in term of memory and difficult to share between multiple hosts when requests are load-balanced across servers.

Browser support Session identifiers are part of SSL and are therefore supported since a long time by clients and servers. However, session tickets are an optional TLS extension and therefore, the support is not as widespread as for session identifiers. Support for tickets was added in OpenSSL 0.9.8f (October 2007). In GnuTLS, this is version 2.9.3 (August 2009). For NSS, the set of libraries behind most browsers, support of RFC 5077 is in version 3.12 (June 2008). Schannel, Microsoft implementation of TLS, does not currently supports tickets. To check if a browser supports session resume without and with tickets, I have written a web server listening to several ports with different configurations (session cache disabled or enabled, ticket support disabled or enabled). With the help of some Javascript, it is possible to interact with this server to determine what kind of session resume a browser support. It is difficult to get extensive results this way since you need to install a lot of browsers to get something valuable. Surprisingly, Android 2.3.4 browser (as shipped by Cyanogen Mod) does not support any session resume. Another way to check if a browser supports session resume is to use a sniffer and to spot Client Hello messages. If a client tries to resume a session, it will use an appropriate session identifier or send a ticket. A simple program parsing PCAP files allows to automate this task and get some interesting statistics. Here are random facts from more than 300 000 requests (38 000 clients) to a customer care service of an important French telco:

• 35% of requests exhibit support of tickets;
• 53.3% of requests ask to resume without tickets; 25.6% with tickets;
• 67.2% of requests exhibit support of SNI;
• 86.7% of requests use TLS 1.0; the remaining uses SSL 3.0; almost no TLS 1.1/1.2;
• in average, a client execute 8 SSL handshakes;
• ciphers supported by all clients are 3DES-SHA, RC4-MD5 and RC4-SHA.

With some log analysis to match each requests with the appropriate user agent, we can split the world in two parts: browsers supporting RFC 5077 (Chrome and Firefox) and browsers not supporting RFC 5077 (Internet Explorer, Opera 9.80, Safari for MacOS and iOS and Android browser).

Web server support There is no unique recipe to configure a web server to handle appropriately and efficiently session resume. Here are some generic directions :

• if the web server uses multiple processes, you need to configure a shared memory session cache; usually, tickets will work just fine in this setup;
• if you use local load-balancing, one easy way to avoid any problem is to use ensure that one IP is mapped to one server (either statically with hashing or dynamically with a stick table); if your load-balancer understands SSL, you can also use session identifier (and disable tickets¹) to build a stick table;
• if you don t want to apply the previous advice, you need to share the session cache between the servers with something like memcached and disable tickets;
• if you use global load-balancing (DNS based), there is no need to ensure that session resumption works across geographic pools since a user will usually stick to one pool.

Remember that only one client out of three supports TLS tickets. Therefore, you cannot rely on them to ensure proper session resume. You need a session cache.

Setting a session cache with Apache & nginx Apache features two different SSL engines. The first one is mod_ssl and uses OpenSSL as backend. It is possible to setup a shared session cache with SSLSessionCache but the memcached-enabled session cache is available only in the development branch. You can also get one by switching to mod_gnutls backend. The cache is enabled with GnuTLSCache. In this case, do not enable tickets because they cannot be shared. With nginx, you need to enable the shared session cache with ssl_session_cache. Currently, it lacks a memcached-enabled session cache but Matt Palmer has some patches to add it to the 0.8 branch. However, those patches can have a serious impact on nginx performance since retrieving a session from memcached is done synchronously (mostly because a limitation in OpenSSL design who does not allow to register asynchronous callbacks with SSL_CTX_sess_set_get_cb()). As a proof of concept, I have also added a similar feature to stud, the scalable TLS unwrapping daemon, a very efficient network proxy terminating SSL connections. The same limitation as for nginx applies: performances will suffer. Look at the pull request for more details.

Sharing tickets RFC 5077 tells tickets allow to load-balance requests across servers. However, to the best of my knowledge, there is currently no web server able to share tickets across a pool. Actually, when a server initializes its SSL stack, it will randomly generate some keys that will be used to protect and encrypt tickets. In OpenSSL, from ssl/ssl_lib.c:

    /* Setup RFC4507 ticket keys */
    if ((RAND_pseudo_bytes(ret->tlsext_tick_key_name, 16) <= 0)
              (RAND_bytes(ret->tlsext_tick_hmac_key, 16) <= 0)
              (RAND_bytes(ret->tlsext_tick_aes_key, 16) <= 0))
            ret->options  = SSL_OP_NO_TICKET;

Therefore, with a round-robin load-balanced pool of servers, tickets have to be disabled because servers would not accept tickets from neighbors. One way to solve this is to generate keys deterministically by hashing some common secret with the private key. I have implemented this approach for stud. Here is a simplified version (no error handling, no allocation) of the pull request for this feature:

    unsigned char keys[48];
    EVP_PKEY *pkey = grab_private_key();
    /* To get our key, we sign the seed with the private key */
    unsigned int siglen;
    unsigned char sign[LARGE_ENOUGH];
    EVP_MD_CTX mdctx;
    EVP_MD_CTX_init(&mdctx);
    EVP_SignInit(&mdctx, EVP_sha256());
    EVP_SignUpdate(&mdctx, some_secret, strlen(some_secret));
    EVP_SignFinal(&mdctx, sign, &siglen, pkey);
    /* And we keep only the first bytes. */
    memcpy(keys, sign, sizeof(keys));
    /* Tell OpenSSL to use those keys */
    SSL_CTX_set_tlsext_ticket_keys(ctx, keys, sizeof(keys));

Testing While tests can be done with just openssl s_client and openssl sess_id, testing all aspects can take a lot of time. For this purpose, I have written a client testing session resume with and without tickets. Here is a typical result: The program will try to establish five consecutive SSL connections for each IP without and with tickets. For each IP, the last four SSL connections should always reuse the first SSL session. Here, Twitter web servers seem properly configured: for each IP, session resume is successful. Their web servers do not support tickets or they may have disabled it because they share the session cache between servers inside the same pool.

---

1. RFC 5077 describes interactions between session identifiers and session tickets. When using tickets, the server should send back an empty session identifier. The client may present an empty session identifier or generate one. However, for all practical purposes, it seems that servers send back a non-empty session identifier and clients just stick to this identifier. Please, investigate if you want to use SSL session identifiers for load-balancing while keeping tickets enabled.

* Life: As usual. * Debian: Lots of stuffs going on. We ve yet another MiniDebConf edition, this time at Mangalore. (Not really, it is at: NMAM Institute of Technology, Nitte) And, yes, we ve some nice international guests this time. (Ah, hello, bubulle!). I ve booked ticket etc. even I m heading there during Diwali time, I hope that it will be more fun than usual crackers + sweets etc this time! gwibber still sucks. * There is nothing apart from Life, Debian that needs update. Aye. Aye.

We just announced the launch of the fundraising campaign for the Debian Administrator s Handbook. We wanted to use this opportunity to let people contribute money both to our project but also to Debian itself. That s why we have setup a special link that you can use to participate. 15% of any donation made through this link (after VAT has been subtracted) will be given back to the Debian Project. Here s the link: http://debian-handbook.info/go/ulule-debian/ Feel free to use this link when promoting the project to your friends, so that even more money goes back to Debian. You can also embed a special widget on your website where any visitor that ends up becoming a supporter will also contribute 15% to the Debian project.

Help us spread the word about the project, and help raise money for Debian!

5 comments Liked this article? Click here. My blog is Flattr-enabled.

A bit more than a year elapsed since we announced our plans to translate our Debian book into English and to try to get it published under a license compatible with the Debian Free Software Guidelines. But we re now ready to go to the next step. Completing the translation of 450 pages book is a huge work, we estimate it s going to take roughly 3 full time months for both Roland and me. Since we re freelancers, we can take the required time provided that we have a minimum income during that period. That s where you come into play: we have setup a crowdfunding campaign on Ulule.com and we need your support to raise 15000 (this is the absolute minimum for us to be able to commit the required time). Even if a good and up-to-date book on Debian is a great perspective, we want to go further than that by adding the perspective of getting a DFSG-free Debian book. That s why we have transformed the crowfunding campaign in a liberation campaign. When you support the project, you can pledge money towards a liberation fund and if this fund reaches 25000 then the book will be published under the GPL-2+ and CC-BY-SA 3.0 licenses. On top of this, when you support this project you can select a reward that goes from a copy of the ebook to a dinner with the authors (only 10 places for the latter)! Among the other rewards, there s obviously a paperback version of the book, but also an individual one-hour mentoring session with me a nice way to get you hooked as a new Debian contributor (limited to 40 persons, don t miss the opportunity!). We also need your help to spread the news and get as many supporters as possible. Share the link with your friends, write an article on your blog, put a widget on your website, etc. Thank you very much!

Click here to go to the fundraising page and to learn more about this project.

10 comments Liked this article? Click here. My blog is Flattr-enabled.

Dad, will you and mom stay up all night decorating the house for me?

That was Jacob s question to me at bedtime the evening before his fifth birthday. Jacob had already had his birthday party a few weeks ago. When scheduling means that parties happen that far away from the boys real birthdays, they get a smaller celebration with just Terah and me where we open their presents from us. A low-key thing, so we weren t planning to decorate the house. I said, Well, I don t think we ll be doing all that since you already had your birthday party. And the look of eager anticipation on his face turned to a very sad and disappointed look, and made me feel really bad. Uhoh. So while Jacob was sleeping, I mentioned it to Terah. We decided we should improvise something simple, so she found some old streamers and we taped them up, running them through several rooms in the house and across his door. It took a few minutes using supplies we already had, but the joy the next morning was priceless. Oh dad, you said you wouldn t stay up all night, BUT YOU DID! Oh I am SO HAPPY! YAY YAY YAY! And he ran through the house to discover what else was set up. Then he ran to find Oliver and gave him a tour of everything. Then we sat down to open his presents. Here he is, holding a present from Terah and me: That s a copy of The Lorax. My friend Jonathan had brought a copy along during our road trip in Mexico, and Jacob was really excited about it. And didn t really want to give it up, because You can only get The Lorax in Mexico. He (and Oliver) really enjoyed all his presents he also got a train book from us (which he said, Oh, yay, it s the book they have at preschool! ), and a game and some other presents from the distant relatives. But the highlight was something of an impulse buy. I was at the RadioShack in Derby a place I ve written about before, It is what a RadioShack should (and used to) be. It has a large amateur radio section, sells all sorts of coax by the foot, and provides astonishingly good post-sale service. Well, I was there with a radio question, and Mark (the owner) who is an excellent salesman in a positive way pointed me to a display of snap kits. I noticed their Electronics 101 Snap-Kit (a rebranded Snap Circuits Jr. SC-100). A ham in Pennsylvania had suggested them to me once, and as Jacob s birthday was coming up, I gave it some thought. The kit said ages 8 and up. I asked Mark what he thought about a boy just turning 5. He said, Well, probably not normally. But knowing you, if you re there to work on them with him, I think he d enjoy it. But I wouldn t have him work on it by himself. I agreed and we bought it. I pondered how to explain the concept of this thing to Jacob. Eventually I decided I would call it a toy-building kit. He understood that. Jacob and I spent hours together working with it. He would flip through the book, either picking circuits that looked interesting or telling me what kind of circuit he wanted to build. Then I would tell him what to put where, and he d snap them together and play with them. He only played with each finished product a few minutes before he was ready to try another. Once I got him very excited with my offer to show him how to hook up two switches in parallel for the fan he built (and later introduced the parallel vs. series concept by hooking them up in series instead.) Here we are working on it together. Jacob repeatedly called Terah over to look at the things he built. He was very excited that he assembled it himself. Eventually, Oliver (age 2) came over wanting to help. So he sat on my lap, and handed parts to Jacob, then Jacob put them on the grid. Oliver really enjoyed being involved in this way, even though I had to keep him from doing things like ripping the capacitor off its mount. We tend to be modest in terms of the number of things we give the boys and their cost, reasoning that we, like many, already have too many toys in our house, and that greater cost doesn t necessarily equate with a better experience for the boys. I particularly look for things with lasting value and unique experiences for them, and I think we succeeded this year. But I realized quickly that the greatest value of this kit wasn t electronics. It was having a great way for me to spend a lot of time doing things with the boys which all of us enjoyed. Those hours building things together were as much a present for me as for Jacob, I m quite sure. Jacob s real party was a few weeks ago at the Great Plains Transportation Museum in Wichita. They let people rent a historic caboose to use for a birthday party for children. So we did that for Jacob this year. That was a huge hit for the boys. Jacob got to help his grandpa make some pie (instead of cake) for the party. He enjoyed eating it, of course. He enjoyed opening his presents high up on the observation chair in the caboose. And the boys got to play on all the other equipment in the museum. Jacob enjoyed playing tour guide for family since most of them hadn t been there. He also enjoyed watching freight trains pass on the other side of the fence from the museum preferably while sitting in one of the museum s engines. Oliver certainly didn t get left out. Train-watching is serious business, after all. Jacob has long talked about going to the train museum and the airplane museum (Kansas Aviation Museum) on the same day, so one of his birthday surprises was that we went to the airplane museum after his party. His favorite item there is a retired FedEx 727. Here he is walking down the rear of the plane. And, of course, they played captain and co-pilot in several different planes.

With no clear plans for Hackweek this year, I decided to play even more with N950. As I quite lack some GPS application I decided to write it :-). The feature set will be based on things I would use, though I still think it will be useful for others:

• Display basic GPS info (coordinates, speed, etc.)
• Compass
• Moon and sun rise and set calculations for current location

I've named the application as GePeS and you can find sources on Gitorious: https://gitorious.org/gepes During first day I've managed to implement basic things, check screenshots:

Filed under: English Meego Suse 0 comments Flattr this!

So one my favorite fieldwork photos is the one above and I would love to include it in my book but I have no idea where I got it from much less who took it. The photo was taken during a San Francisco protest against the DMCA, and more urgently at the time, also calling for the release of Dimitry Sklyrov (and we were all listening to Richard Stallman who was giving a small speech and I am on the right hand side of the photo, furiously scribbling notes) I am looking for a high quality photo from one of the many Dmitry Sklyarov protests that were held in San Francisco or San Jose a photo that is either under CC license or given to me with formal permission to publish (I have forms to get official permissions). As I mentioned in my previous post, the book will be published with Princeton University Press with a CC license.

So my first book, Coding Freedom: The Ethics and Aesthetics of Hacking is mostly sort of done. All the arguments are in place and I am doing the final, very annoying, seemingly endless work of adding a few final citations, hunting for permissions, and tweaking about 10 paragraphs that I feel can be slightly improved (down from about 20 last week). The book is based on the dissertation but re-framed significantly with lots of new material and best of all a lot of the cruft has been zapped away, I think. The book will include two published pieces, although slightly revised. One of them is one of my all time favorite pieces on the hacker conference (which is the least academic/jargony of the peer reviewed articles). I thought it might be fun to change some of the photos for my book: so if you have some awesome, high quality photos that exude the joy of conferencing from past Debconfs or ANY hacker conference (preferably with computers in the background) please share. I am open to anything although I am looking for a high quality/sharp photo of people hacking away in the hack lab. The photo I have is good but I would like one with better lighting/sharper focus. The book will be published under a CC license (the most restrictive one, but heck as a junior scholar, I took what I could get and it is the first book that Princeton University Press is releasing under such a license). If you hold the copyright, I will need more official permission, otherwise, if it is under a CC or copyleft license, I will be adding the correct attribution/notice in the book.

Despite being on sabbatical last academic year, life was hectic and chaotic leaving me with next to no time to write entries here. Anonymous, which I had worked on for a few years, exploded. I was excommunicated from my house (due to a flood/something likely mold making be very sick) and had to commute to Princeton from the far reaches of Manhattan where I crashed at a friends house for 5 months, while fighting my landlords. I then decided to go on a job hunt primarily as I did not know what would happen with my housing situation in NYC but I was also pursuing some interesting opportunities. Though the housing issue was eventually resolved, in May a rare and enticing opportunity arose. Come January 2012 I will be joining the Art History and Communication Department at McGill University as the Wolfe Chair in Scientific and Technological Literacy. The decision to leave MCC NYU was torturous. The faculty is stellar. The graduate students are top notch and cream of the crop. And the digital media faculty is a force of nature, perhaps one of the deepest and most dynamic departments when it comes to the study of digital media. They also know how to treat their junior faculty right, which made it a place I respected and am deeply indebted to. But this type of opportunity is rare and it helped that NYC as wonderful as it is, is not my cup of tea. I wanted to eventually settle in one of the large Canadian cities, and Montreal was on the top of my list. I get to join one of the best universities in the world and a department whose work on digital media, STS, and politics converges with much of my own. The position provides resources, time, and room to cultivate scholarly work, help build a program of some sort in Science Technology Studies at McGill, and also engage with the public about the ethics and politics of science and technology. I will be teaching a class to joint BA/BS majors on scientific and technological controversies, a topic close to my heart. I also hope to create an open courseware website for my hacker course, which I will teach sometime in the 2012-2013 academic year. It is all that I can ask for, well, except for the weather. I would be lying if I did not admit that the winter scares the heck out of me I know some days I will feel just like this .

This week, I've finished my OAuth 2.0 client implementation for GNU Emacs. I have imported it into GNU ELPA so Emacs 24 users will be soon able to install it using the new Emacs packaging system. OAuth 2.0 can be used to access, among others, Google APIs or the Facebook Graph API.
Flattr me!

An obvious question is why Linux doesn't support UEFI secure booting. Let's ignore the issues of key distribution and the GPL and all of those things, and instead just focus on what would be required. There's two components - the signed binary and the authenticated variables.

The UEFI 2.3.1 spec describes the modification to the binary format required to produce a signed binary. It's not especially difficult - you add an extra entry to the image directory, generate a hash of the entire binary other than the checksum, the certificate directory entry and the signatures themselves, encrypt that hash with your key and embed the encrypted hash in the binary. The problem has been that there was a disagreement between Microsoft and Intel over whether this signature was supposed to include the PKCS header or not, and until earlier this week the only widely available developer firmware (Intel's) was incompatible with the only widely available signed OS (Microsoft's). There's further hilarity in that the specification lists six supported hash algorithms, but the implementations will only accept two. So pretty normal, really. Developing towards a poorly defined target is a pain. Now that there's more clarity we'll probably have a signing tool before too long.

Authenticated variables are the other part of the puzzle. If a variable requires authentication, the operating system's attempt to write it will fail unless the new data is appropriately signed. The key databases (white and blacklists) are examples of authenticated variables. The signing actually takes place in userspace, and the handoff between the kernel and firmware is identical for both this case and the unauthenticated case. The only problem in Linux's support here is that our EFI variable support was written to a pre-1.0 version of the EFI specification which stated that variables had a maximum size of 1024 bytes, and this limitation ended up exposed to userspace. So all we really need to do there is add a new interface to let arbitrary sized variables be written.

Summary: We don't really support secure boot right now, but that's ok because you can't buy any hardware that supports it yet. Adding support is probably about a week's worth of effort at most.

comments

Microsoft have responded to suggestions that Windows 8 may make it difficult to boot alternative operating systems. What's interesting is that at no point do they contradict anything I've said. As things stand, Windows 8 certified systems will make it either more difficult or impossible to install alternative operating systems. But let's have some more background.

We became aware of this issue in early August. Since then, we at Red Hat have been discussing the problem with other Linux vendors, hardware vendors and BIOS vendors. We've been making sure that we understood the ramifications of the policy in order to avoid saying anything that wasn't backed up by facts. These are the facts:

• Windows 8 certification requires that hardware ship with UEFI secure boot enabled.
  
• Windows 8 certification does not require that the user be able to disable UEFI secure boot, and we've already been informed by hardware vendors that some hardware will not have this option.
  
• Windows 8 certification does not require that the system ship with any keys other than Microsoft's.
  
• A system that ships with UEFI secure boot enabled and only includes Microsoft's signing keys will only securely boot Microsoft operating systems.

Microsoft have a dominant position in the desktop operating system market. Despite Apple's huge comeback over the past decade, their worldwide share of the desktop market is below 5%. Linux is far below that. Microsoft own well over 90% of the market. Competition in that market is tough, and vendors will take every break they can get. That includes the Windows logo program, in which Microsoft give incentives to vendors to sell hardware that meets their certification requirements. Vendors who choose not to follow the certification requirements will be at a disadvantage in the marketplace. So while it's up to vendors to choose whether or not to follow the certification requirements, Microsoft's dominant position means that they'd be losing sales by doing so.

Why is this a problem? Because there's no central certification authority for UEFI signing keys. Microsoft can require that hardware vendors include their keys. Their competition can't. A system that ships with Microsoft's signing keys and no others will be unable to perform secure boot of any operating system other than Microsoft's. No other vendor has the same position of power over the hardware vendors. Red Hat is unable to ensure that every OEM carries their signing key. Nor is Canonical. Nor is Nvidia, or AMD or any other PC component manufacturer. Microsoft's influence here is greater than even Intel's.

What does this mean for the end user? Microsoft claim that the customer is in control of their PC. That's true, if by "customer" they mean "hardware manufacturer". The end user is not guaranteed the ability to install extra signing keys in order to securely boot the operating system of their choice. The end user is not guaranteed the ability to disable this functionality. The end user is not guaranteed that their system will include the signing keys that would be required for them to swap their graphics card for one from another vendor, or replace their network card and still be able to netboot, or install a newer SATA controller and have it recognise their hard drive in the firmware. The end user is no longer in control of their PC.

If Microsoft were serious about giving the end user control, they'd be mandating that systems ship without any keys installed. The user would then have the ability to make an informed and conscious decision to limit the flexibility of their system and install the keys. The user would be told what they'd be gaining and what they'd be giving up.

The final irony? If the user has no control over the installed keys, the user has no way to indicate that they don't trust Microsoft products. They can prevent their system booting malware. They can prevent their system booting Red Hat, Ubuntu, FreeBSD, OS X or any other operating system. But they can't prevent their system from running Windows 8.

Microsoft's rebuttal is entirely factually accurate. But it's also misleading. The truth is that Microsoft's move removes control from the end user and places it in the hands of Microsoft and the hardware vendors. The truth is that it makes it more difficult to run anything other than Windows. The truth is that UEFI secure boot is a valuable and worthwhile feature that Microsoft are misusing to gain tighter control over the market. And the truth is that Microsoft haven't even attempted to argue otherwise.

comments