Almost a year has passed since my talk at pgconf.eu 2011 in Amsterdam on Connecting the Debian and PostgreSQL worlds, and unfortunately little has happened on that front, mostly due to my limited spare time between family and job. pgapt.debian.net is up and running, but got few updates and is lagging behind on PostgreSQL releases. Luckily, we got the project moving. Dimitri Fontaine and Magnus Hagander suggested to do a face-to-face meeting, so we got together at my house for two days last week and discussed ideas, repository layouts, build scripts, and whatnot to get all of us aligned for pushing the project ahead. My employer sponsored my time off work for that. We almost finished moving the repository to postgresql.org infrastructure, barring some questions of how to hook the repository into the existing mirror infrastructure; this should get resolved this week. The build server running Jenkins is still located on my laptop, but moving this to a proper host will also happen really soon now. We are using Mika Prokop's jenkins-debian-glue scripts for driving the package build from Jenkins. The big plus point about Jenkins is that it makes executing jobs on different distributions and architectures in parallel much easier than a bunch of homemade shell scripts could get us with reasonable effort. Here's a list of random points we discussed:

• We decided to go for "pgdg" in version numbers and distribution names, i.e. packages will have version numbers like 9.1.5-1.pgdg+1, with distributions wheezy-pgdg, squeeze-pgdg, and so on.
• There will be Debian-testing-style distributions called like wheezy-pgdg-testing that packages go into for some time before they get promoted to the "live" distributions.
• PostgreSQL versions out of support (8.2 and below) will not be removed from the repository, but will be moved to distributions called like wheezy-pgdg-deprecated. People will still be able to use them, but the naming should make it clear that they should really be upgrading.
• We have a slightly modified (compared to Debian unstable) postgresql-common package that sets the "supported-versions" to all versions supported by the PostgreSQL project. That will make the postgresql-server-dev-all package pull in build-dependencies for all server versions, and make extension module packages compile for all of them automatically. (Provided they are using pg_buildext.)
• There's no Ubuntu support in there yet, but that's mostly only a matter of adding more cowbuilder chroots to the build jobs. TBD soon.

We really aim at using unmodified packages from Debian as much as possible, and in fact this project doesn't mean to replace Debian's PostgreSQL packaging work, but to extend it beyond the number of server versions (and Debian and Ubuntu versions covered) supported. The people behind the Debian and Ubuntu packages, and this repository are mostly the same, so we will claim that "our" packages will be the same quality as the "original" ones. Big thanks go to Martin Pitt for maintaining the postgresql-common testsuite that really covers every aspect of running PostgreSQL servers on Debian/Ubuntu systems. Stay tuned for updates! :)

A while back mdz blogged about challenges facing Ubuntu and other Linux distributions. He raises the point that runtime libraries for Python / Ruby etc have a unique set of issues because they tend to have their own packaging systems. Merely a month later he attended Debconf 2010 where a presentation was given on the issues that Java packages have on Dpkg based systems. Since then the conversation seems to have dried up. I ve been reminded of it recently in discussions within Canonical looking at how we deploy web services. Matt suggested some ways forward, including:

• Decouple applications from the core
• Treat data as a service (rather than packages) get data live from the web rather than going web -> distro-package -> user machines.
• Simplify integration between packaging systems (including non-packaged things)

I think its time we revisit and expand on those points. Nothing much has changed in how Ubuntu or other distributions approach integration with other packaging systems but the world has kept evolving. Internet access is growing ever more ubiquitous, more platforms are building packaging systems clojure, scala, node.js, to name but three, and a substantial and ever growing number of products expect to operate in a hybrid fashion with an evolving web service plus a local client which is kept up to date via package updates. Twitter, Facebook and Google Plus are three such products. Android has demonstrated a large scale app store on top of Linux, with its own custom packaging format. In order to expand them, we need some background context on the use cases that these different packaging systems need to support. Platforms such as antivirus scanners, node.js, Python, Clojure and so forth care a great deal about getting their software out to their users. They care about making it extremely easy to get the latest and greatest versions of their libraries. I say this because the evidence is all around us: every successful development community / product has built a targeted package management system which layers on top of Windows, and Mac OSX, and *nux. The only rational explanation I can come up for this behaviour is that the lower level operating system package management tools don t deliver what they need. E.g. this isn t as shallow as wanting a packaging system written in their own language, which would be easy to write off as parochialism rather than a thoughtful solution to their problems. In general packaging systems provide a language for shipping source or binary form, from one or more repositories, to users machines. They may support replications, and they may support multiple operating systems. They generally end up as graph traversal engines, pulling in dependencies of various sorts you can see the DOAP specification for an attempt at generic modelling of this. One problem that turns up rapidly when dealing with Linux distribution package managers is that the versions upstream packages have, and the versions a package has in e.g. Debian, differ. They differ because at some stage, someone will need to do a new package for the distribution when no upstream change has been made. This might be to apply a local patch, or it might be to correct a defect caused by a broken build server. Whatever the cause, there is a many to one relationship between the package versions that end users see via dpkg / rpm etc, and those that upstream ship. It is a near certainty that once this happens to a library package, that comparing package versions across different distribution packages becomes hard. You cannot reliably infer whether a given package version is sufficient as a dependency or not, when comparing binary packages between Red Hat and Debian. Or Debian and Ubuntu. The result of this is that even when the software (e.g. rpm) is available on multiple distributions (say Ubuntu and RHEL), or even on multiple operating systems (say Ubuntu and Windows), that many packages will /have/ to be targeted specifically to build and execute properly. (Obviously, compilation has to proceed separately for different architectures, its more the depedency metadata that says and build with version X of dependency Y that has to be customised). The result of this is that there is to the best of my knowledge no distribution of binary packages that targets Debian/Ubuntu and RHEL and Suse and Windows and Mac OS X, although there are vibrant communities building distributions of and for each in isolation. Some of the ports systems come close, but they are still focused on delivering to a small number of platforms. There s nothing that gives 99% coverage of users. And that means that to reach all their users, they have to write or adopt a new system. For any platform X, there is a strong pressure to have the platform be maintainable by folk that primarily work with X itself, or with the language that X is written in. Consider Python there is strong pressure to use C, or Python, and nothing else, for any tools that is somewhat parochial, but also just good engineering reducing variables and making the system more likely to be well maintained. The closest system I know of Steam is just now porting to Ubuntu (and perhaps Linux in general), and has reached its massive popularity by focusing entirely on applications for Windows, with Mac OSX a recent addition. Systems like pypi which have multi platform eggs do target the wide range of platforms I listed above, but they do so both narrowly and haphazardly: whether a binary or source package is available for a given platform is up to the maintainer of the package, and the packages themselves are dealing with a very narrow subset of the platforms complexity: Python provides compilation logic, they don t create generic C libraries with stable ABI s for use by other programs, they don t have turing complete scripts for dealing with configuration file management and so forth. Anti virus updaters similarly narrow the problem they deal with, and add constraints on latency- updates of anti virus signatures are time sensitive when a new rapidly spreading threat is detected. A minor point, but it adds to the friction of considering a single packaging tool for all needs is the different use cases of low level package management tools like dpkg or rpm vs the use cases that e.g. pypi has. A primary use case for packages on pypi is for them to be used by people that are not machine administrators. They don t have root, and don t want it. Contrast that with dpkg or rpm where the primary use case (to date) is the installation of system wide libraries and tools. Things like man page installation don t make any sense for non-system-wide package systems, whereas they are a primary feature for e.g. dpkg. In short, the per-platform/language tools are (generally):

1. Written in languages that are familiar to the consumers of the tools.
2. Targeted at use on top of existing platforms, by non-privileged users, and where temporary breakage is fine.
3. Intended to get the software packaged in them onto widely disparate operating systems.
4. Very narrow they make huge assumptions about how things can fit together, which their specific language/toolchain permits, and don t generalise beyond that.
5. Don t provide for security updates in any specific form: that is left up to folk that ship individual things within the manager.

operating system package managers:

1. Are written in languages which are very easy to bootstrap onto an architecture, and to deploy onto bare metal (as part of installation).
2. Designed for delivering system components, and to avoid be able to upgrade the toolchain itself safely.
3. Originally built to install onto one operating system, ports to other operating systems are usually fragile and only adopted in niche.
4. Are hugely broad they install data, scripts, binaries, and need to know about late binding, system caches etc for every binary and runtime format the operating system supports
5. Make special provision to allow security updates to be installed in a low latency fashion, without requiring anything consuming the package that is updated to change [but usually force-uninstalling anything that is super-tightly coupled to a library version].

Anti virus package managers:

1. Exist to update daemons that run with system wide escalated privileges, or even file system layer drivers.
2. Update datasets in realtime.
3. Without permitting updates that are produced by third parties.

Given that, lets look at the routes Matt suggested Decoupling applications from the core as a strategy makes an assumption that the core and applications are partitionable. If they are not, then applications and the core will share common elements that need to be updated together. Consider, for instance, a Python application. If you run with a system installed Python, and it is built without zlib for some reason, but the Python application requires zlib, you have a problem. A classic example of this problem is facing Ubuntu today, with all the system provided tools moving to Python 3, but vast swathes of Python applications still being unported to Python 3 at all. Currently, the Python packaging system virtualenv/buildout + distribute don t provide a way to install the Python runtime itself, but will happily install their own components for everything up the stack from the runtime. Ubuntu makes extensive use of Python for its own tools, so the system Python has a lot of packages installed which buildout etc cannot ignore this often leads to issues with e.g. buildout, when the bootstrap environment has (say) zope.interfaces, but its then not accessible from the built-out environment that disables the standard sys.path (to achieve more robust separation). If we want to pursue decoupling, whether we build a new package manager or use e.g. virtualenv (or gem or npm or ), we ll need to be aware of this issue and perhaps offer, for an extended time, a dedicated no-frills, no-distro-packages install, to avoid it, and to allow an extended supported period for application authors without committing to a massive, distro sponsored porting effort. While its tempting to say we should install pip/npm/lein/maven and other external package systems, this is actually risky: they often evolve sufficiently fast that Ubuntu will be delivering an old, incompatible version of the tool to users well before Ubuntu goes out of support, or even befor the next release of Ubuntu. Treating data as a service. All the cases I ve seen so far of applications grabbing datasets from the web have depended on web infrastructure for validating the dataset. E.g. SSL certificates, or SSL + content checksums. Basically, small self-rolled distribution systems. I m likely ignorant of details here, and I depend on you, dear reader, to edumacate me. There is potential value in having data repackaged, when our packaging system has behind-firewall support, and the adhoc system that (for instance) a virus scanner system has does not. In this case, I specifically mean the problem of updated a machine which has no internet access, not even via a proxy. The challenge I see it is again the cross platform issue: The vendor will be supporting Ubuntu + Debian + RHEL + Suse, and from their perspective its probably cheaper to roll their own solution than to directly support dpkg + rpm + whatever Apple offer + Windows the skills to roll an adhoc distribution tool are more common than the skills to integrate closely with dpkg or rpm What about creating a set of interfaces for talking to dpkg / rpm / the system packagers on Windows and Mac OSX ? Here I think there is some promise, but it needs as Matt said careful thought. PackageKit isn t sufficient, at least today. There are, I think, two specific cases to cater to:

1. The anti-virus / fresh data set case.
2. The egg/gem/npm/ specific case.

For the egg/gem/npm case, we would need to support a pretty large set of common functionality, on Windows/Mac OSX / *nux (because otherwise upstream won t adopt what we create: losing 90% of their users (windows) or 5% (mac) isn t going to be well accepted . We d need to support multiple installations (because of mutually incompatible dependencies between applications), and we d need to support multiple language bindings in some fashion some approachable fashion where the upstream will feel capable of fixing and tweaking what we offer. We re going to need to support offline updates, replication, local builds, local repositories, and various signing strategies to match the various tradeoffs made by the upstream tools. For the anti-virus / fresh data case, we d need to support a similar set of operating systems, though I strongly suspect that there would be more tolerance for limited support in that most things in that space either have very platform specific code, or they are just a large-scale form of the egg/gem/npm problem, which also wants easy updates. What next? We should validate this discussion with at least two or three upstreams. Find out whats missing I suspect a lot and whats wrong I hope not much . Then we ll be in a position to decide if there is a tractable, widespread solution *possible*. Separately, we should stop fighting with upstreams that have their own packaging systems. They are satisfying different use cases than our core distro packaging systems are designed to solve. We should stop mindlessly repackaging things from e.g. eggs to debs, unless we need that specific thing as part of the transitive runtime or buildtime dependencies for the distribution itself. In particular, if us folk that build system packaging tools adopt and use the upstream application packaging tools, we can learn in a deep way the (real) advantages they have, and become more able to reason about how to unify the various engineering efforts going into them and perhaps even eventually satisfy them using dpkg/rpm on our machines.

For anybody that doesn't know - Ken Starks is the man behind the Helios Project - recycling computers, putting Linux on them, donating them to youngsters and the needy in Austin Texas.

He has a cancer diagnosis but his disability pension is enough to defeat his state provided insurance. Now he himself needs the help of the community.

More details here: http://linuxlock.blogspot.co.uk/2012/08/something-ive-been-putting-off-all-day.html and his partner's post above.

and here: Thomas A. Knight, who is setting up donations.

http://thomasaknight.com/blog.php?id=71

UPDATE:

There is now also a donate button at the Blog of Helios (linuxlock.blogspot.com).

Enough has been raised so far to buy Ken an appropriate monitor - because of the effects of radiotherapy, he's been unable to tolerate using a monitor with backlighting.

One donor has also stepped in to buy a month's drug treatment.

The initial target -US $5000 - has been met but The cost of an operating theatre session is at least US $50,000. All donations continue to be gratefully received.

Should efforts for raising funds for Ken himself be unsuccessful, all surplus funds raised will be ploughed back into the Helios Project (now known as Reglue following a recent merger with another local computer charity).

The Helios Project has been associated with Software in the Public Interest [SPI - also the umbrella organisation supporting Debian] for some years.

UPDATE2 - From Thomas A. Knight's blog referenced above:

By Thomas A. Knight on Sun 19 Aug 2012 07:33:49 am $11,000 so far, and still going strong. The outpouring of support for this is amazing. My most humble thanks to all contributors.

Even though we are still far from our goal, this has been far more successful than I ever thought possible. Thank you.

UPDATE3 - From Ken Starks, Sunday 26 August 2012

The amount of money needed has been raised. Donations to Helios Project/Reglue are welcome, as ever - but Ken's immediate needs are satisfactorily met and he himself doesn't wish to profit further from the community's goodwill. As he says, he can't thank us enough
More news at http://linuxlock.blogspot.com is due on 27/08/2012

This vision of the future at Sam's No. 3 in Denver suggests that we will have ample blackboards after the apocalypse. And that the contrast will be greatly improved in direct sunlight.

There are plenty of times where it is nice to have Linux transmit things out a radio. One obvious example is the digital communication modes, where software acts as a sort of modem. A prominent example of this in Debian is fldigi. Sometimes, it is nice to transmit voice instead of a digital signal. This is called voice keying. When operating a contest, for instance, a person might call CQ over and over, with just some brief gaps. Most people that interface a radio with a computer use a sound card interface of some sort. The more modern of these have a simple USB cable that connects to the computer and acts as a USB sound card. So, at a certain level, all that you have to do is play sound out a specific device. But it s not quite so easy, because there is one other wrinkle: you have to engage the radio s transmitter. This is obviously not something that is part of typical sound card APIs. There are all sorts of ways to do it, ranging from dedicated serial or parallel port circuits involving asserting voltage on certain pins, to voice-activated (VOX) circuits. I have used two of these interfaces: the basic Signalink USB and the more powerful RigExpert TI-5. The Signalink USB integrates a VOX circuit and provides cabling to engage the transmitter when VOX is tripped. The TI-5, on the other hand, emulates three USB serial ports, and if you raise RTS on one of them, it will keep the transmitter engaged as long as RTS is high. This is a more accurate and precise approach. VOX-based voice keying with the Signalink USB But let s first look at the Signalink USB case. The problem here is that its VOX circuit is really tuned for digital transmissions, which tend to be either really loud or completely silent. Human speech rises and falls in volume, and it tends to rapidly assert and drop PTT (Push-To-Talk, the name for the control that engages the radio s transmitter) when used with VOX. The solution I hit on was to add a constant, loud tone to the transmitted audio, but one which is outside the range of frequencies that the radio will transmit (which is usually no higher than 3kHz). This can be done using sox and aplay, the ALSA player. Here s my script to call cq with Signalink USB:

#!/bin/bash
# NOTE: use alsamixer and set playback gain to 99
set -e
playcmd ()  
        sox -V0 -m "$1" \
           "  sox -V0 -r 44100 $1 -t wav -c 1 -   synth sine 20000 gain -1" \
            -t wav -   \
           aplay -q  -D default:CARD=CODEC
 
DELAY=$ 1:-1.5 
echo -n "Started at: "
date
STARTTIME= date +%s 
while true; do
        printf "\r"
        echo -n $(( ( date +%s -$STARTTIME) / 60))
        printf "m/$ DELAY s: TRANSMIT"
        playcmd ~/audio/cq/cq.wav
        printf "\r"
        echo -n $(( ( date +%s -$STARTTIME) / 60))
        printf "m/$ DELAY s: off         "
        sleep $DELAY
done

Run this, and it will continuously play your message, with a 1.5s gap in between during which the transmitter is not keyed. The screen will look like this:

Started at: Fri Aug 24 21:17:47 CDT 2012
2m/1.5s: off

The 2m is how long it s been going this time, and the 1.5s shows the configured gap. The sox commands are really two nested ones. The -m causes sox to merge the .wav file in $1 with the 20kHz sine wave being generated, and the entire thing is piped to the ALSA player. Tweaks for RigExpert TI-5 This is actually a much simpler case. We just replace playcmd as follows:

playcmd ()  
        ~/bin/raiserts /dev/ttyUSB1 'aplay -q -D default:CARD=CODEC' < "$1"
 

Where raiserts is a program that simply keeps RTS asserted on the serial port while the given command executes. Here's its source, which I modified a bit from a program I found online:

/* modified from
 * https://www.linuxquestions.org/questions/programming-9/manually-controlling-rts-cts-326590/
 * */
#include <stdio.h>
#include <stdlib.h>
#include <termios.h>
#include <unistd.h>
#include <sys>ioctl.h>
#include <sys< sys="">types.h>
#include <sys>stat.h>
#include <fcntl.h>
static struct termios oldterminfo;
void closeserial(int fd)
 
    tcsetattr(fd, TCSANOW, &oldterminfo);
    if (close(fd) < 0)
        perror("closeserial()");
 
int openserial(char *devicename)
 
    int fd;
    struct termios attr;
    if ((fd = open(devicename, O_RDWR)) == -1)  
        perror("openserial(): open()");
        return 0;
     
    if (tcgetattr(fd, &oldterminfo) == -1)  
        perror("openserial(): tcgetattr()");
        return 0;
     
    attr = oldterminfo;
    attr.c_cflag  = CRTSCTS   CLOCAL;
    attr.c_oflag = 0;
    if (tcflush(fd, TCIOFLUSH) == -1)  
        perror("openserial(): tcflush()");
        return 0;
     
    if (tcsetattr(fd, TCSANOW, &attr) == -1)  
        perror("initserial(): tcsetattr()");
        return 0;
     
    return fd;
 
int setRTS(int fd, int level)
 
    int status;
    if (ioctl(fd, TIOCMGET, &status) == -1)  
        perror("setRTS(): TIOCMGET");
        return 0;
     
    status &= ~TIOCM_DTR;   </sys>* ALWAYS clear DTR */
    if (level)
        status  = TIOCM_RTS;
    else
        status &= ~TIOCM_RTS;
    if (ioctl(fd, TIOCMSET, &status) == -1)  
        perror("setRTS(): TIOCMSET");
        return 0;
     
    return 1;
 
int main(int argc, char *argv[])
 
    int fd, retval;
    char *serialdev;
    if (argc < 3)  
        printf("Syntax: raiserts /dev/ttyname 'command to run while RTS held'\n");
        return 5;
     
    serialdev = argv[1];
    fd = openserial(serialdev);
    if (!fd)  
        fprintf(stderr, "Error while initializing %s.\n", serialdev);
        return 1;
     
    setRTS(fd, 1);
    retval = system(argv[2]);
    setRTS(fd, 0);
    closeserial(fd);
    return retval;
 

This compiles to an executable less than 10K in size. I love it when that happens. So these examples support voice keying both with VOX circuits and with serial-controlled PTT. raiserts.c could be trivially modified to control other serial pins as well, should you have an interface which uses different ones.

Continuing on from my second EFI progress report... My second alpha release of a Wheezy netinst CD for amd64 EFI seemed to work OK for me and a few others, but it wasn't 100% good, with (at least) two issues:

• It didn't work as a hybrid (CD or USB) image, just working from CD for EFI
• It didn't work for BIOS booting due to a silly bug

In the couple of days since then, I've fixed both of these issues. Yay! Hybrid boot was easier than I expected - I've just added a tiny FAT partition to the image containing the grub EFI bootloader and that seems to work fine. I also found that I made a mistake in the ordering of the El Torito boot records in the last image. BIOS boot seems to be limited to just the first image available on the machines I've tested with, wherease EFI will happily work as a secondary image. I've swapped them around in my code, and things look much better now. So, time for another CD alpha release for testing. Grab the image from http://cdimage.debian.org/cdimage/unofficial/efi-development/upload3/ if you'd like to help test it. Please do, and let me know via debian-boot/debian-cd how you get on. As previously, the "bits" subdirectory contains all the tweaked d-i packages I've played with, in both source and binary form. My debian-cd changes are still in my branch but are just about ready for merging I think. I've posted my full set of d-i patches to the debian-boot list for review now, and hopefully we'll be able to get those changes merged in after the Wheezy d-i beta 2 build is done. Full speed ahead for EFI in beta 3!

Do you remember at school, when the teachers mentioned that something would be on your "permanent record"? The big secret is that there isn't one. Apologies to all teachers, but it's not likely this blog's demographic will reach your students. However, since the advent of the internet, a permanent record does exist. Everywhere from the Wayback Machine to Google's cache contains copies of all public activity you post online. At Collabora, we hire some of the best open source software engineers to work for our clients. It should be no surprise that a quick Ohloh and search for someone's name is a standard part of the hiring process. This normally comes up with all sorts of exciting things which people have done, both with software and further abroad. A tricky situation will occur if you have lost your temper online, as Patrick Ryan has done recently. He made a comment on #node.js on the Freenode network that:

[20.16.34] < TheEmpath> anyone else have a hot programmer girl in their office? o____O

This was quite rightly called out as being inappropriate for the channel, and quickly decided to have a large and abusive rant about this. This was then continued with him opening a number of bugs against the projects owned by the person who called him out, with titles such as "Hyperactive Leftist Power Trip not included" and "Not anti-Republican enough". This then gets worse. Once the log is posted (link above) there is yet more abuse and unfounded accusations by Mr Ryan in the comments. So, what happens when you search for his name? You get a log with someone being entirely inappropriate and then large rants. Not the best move for a software consultant.

Sometimes, very rarely, I am happy that the spam filters in my personal mail server are so far from perfect. Many of us were amazed, some days ago, news broke that Amelia Earhart's plane was found in the Southwestern Pacific. The news is amazing and worthy in itself. But today, I got this mail, proof that Amelia lived through all those years: (all addresses obscured, of course)

Date: Wed, 22 Aug 2012 23:38:59 -0700
From: Amelia Earhart <some-mail@gmail.com>
To: undisclosed-recipients: ;
Subject: Hello
Reply-To: some-other-mail@live.com Hello friend, With due respect to humanity, i am compelled to write to you on a humanitarian ground, however, it has been my wish to execute in life. Please, I am contacting you based on my desire to create a charitable fund in your country with this sum of EURO 3.5 Million which I inherited from my late husband (Dr. Ahrens Earhart). It is my desire to see that this money is invested in any organization of your choice in your country and distributed each year among the charity organizations, the house mother of the child, mosques, churches, schools, support for disadvantaged older men and women or in whatever way you intend helping the less privileged. I took this decision because I was raised out of the house of mother and baby at the moment, i am admitted in a hospital in London were i am taking severe treatment to my cancer illness. Please answer as soon as possible to confirm your agreement with this proposal, so I ll give you all the necessary information that will enable you to have access to this funds my personal accountant will be aware of my arrangements with you. Thank you for your attention and I hope to hear from you soon. Please contact me through my this email- some-other-mail@live.com the nurse helping me will attend to you i have been restricted from touching the computer due to my illness i will direct you through her she is a nice person. Sincerely,
Ms. Amelia Earhart.

In this choc-filled post, I ve got a few things I d love to share. debian.js First, a quick side-hack called debian.js. It s not entirely a serious project, but it s interesting. Here s some example code (and output from the terminal) Here s some of the client code (but slightly modified for clarity)

debian.on('bug', function(bug)  
    console.log(bug);
 );

which produces the following:

  sender: 'John Johansen <john.johansen@canonical.com>',
  package: 'vidalia',
  id: '<503577E5.2080502@canonical.com>',
  bugno: '682431',
  action: 'followup',
  class: 'bug',
  description: 'Bug#682431: [apparmor] Factorize Qt and GTK themes abstractions?'  

(and later, after I filed for unblock of desktop-base (more on that later))

  sender: 'Paul Tagliamonte <paultag@debian.org>',
  package: 'release.debian.org',
  id: '<20120823011639.GA8561@leliel.localdomain>',
  bugno: '685657',
  action: 'report',
  class: 'bug',
  description: 'Bug#685657: unblock: desktop-base/7.0.3'  

Legit. Anyway, hopefully more hax on this later. I d love to see Debian and Node.js get along better. dpu Nextly, I ve been working on something called dpu , with the (epic) Niels Thykier which will be a replacement for Lintian s test-runner. Progress is good, but not feature-complete yet. The end goal is to allow all projects which deal with the process of generating a debian package to start testing their code. If you have requirements, let me know! desktop-base I ve uploaded version 7.0.3, which features a switch to xz, and slight changes to existing theme elements. Mad props to Eshat Cakar for some slick KDE updates, and to Jo Carter for the Plymouth updates. Props!

[ once again, I am translating somebody else's material In this case, my good Costa Rican friend Carolina Flores. Please excuse my stylistic mistakes My English is far from native as you well know. But this material is worth sharing, and worth investing some tens of minutes doing a quick translation. If you can read Spanish, go read Caro's original entry ] Have you been to a music record store lately? I did so last Saturday, as a mere excercise. I was not planning on buying anything but I wanted to monitor things and confirm my suspicions. What was I suspicious of? First, that I would only find old records. And so it was: The only recent record I found was ...little broken hearts by Norah Jones. the second, that I would only find music for over 50 year old people. so it was: Were I there to look for a present for my father, I would have walked out with 10 good records. Third, that in the store nothing worth commenting would happen. About that last point, I should point out it was around 10 AM and the store had just opened its doors. Lets concede the benefit of doubt. I don't think many of you will remember, but in Barrio La California (where there is now a beauty parlour, almost in front of AM.PM) there was Auco Disco. In Auco Disco there was a guy specialized in rock (Mauricio Alice) and another one specialized in jazz (I don't remember his name). In that record you could always find rare records, but if they were not there, at least you were sure to find somebody to say: "No, we don't have that, but that's an excellent record, it's the best that [insert group here] have ever recorded because just afterwards they switched their guitar player, they had gone a bit south but with that record they are flying. But no, we don't have it; I can recommend you this record by [insert another group] because it has a guitar solo in track six that is amazing". It would happen more or less like that, which means, one would arrive to Auco Disco at 10 AM and leave around 5 PM with three new records, after having listened to a spectacular music selection. What happened to those stores? Were they killed by The Pirate Bay? That's the simplistic answer from the recording industry! The answer is that those stores never got anything from the industry but an invoice. The industry specially in prescindible markets such as ours was limited to hiring artists, taking care of them recording a sellable product, producing the object called record, and that's it. The more commercial radio stations were paid to program those songs as it cannot be casual that "Mosa, mosa" is the summer hit in all of Latin America, can it? but, record stores? Nothing. Lets carry on with that idea: Radio stations are paid to program said music. This idea should not lead us to believe that recording companies are to blame for bad taste. I won't reveal my sources, but I know the success of the "Locura autom tica" song by La Secta was a real example. Nobody paid for it. That song got to the number one because of its own merits(?) (you don't know the effort it took to find that thing, I cannot recommend it to you). Same thing happens with other stations that don't program reggaet n, that try to save the species, and where they play what we do like. But the thing is, everything we like is not available in any record store in this country. Then, even if we wanted to buy a record or give it as a gift to somebody, it is plain impossible. And don't tell me it's the same to present as a gift a link or a CD full of downloaded MP3 as it is to give a record with cover and booklet, wrapped in gift paper. I might be old-school, but the fetish object record still exists, not only because of its cover, but because of its sound. A 3MB MP3 is akin to drinking coffee dripping from a bag that has been used eight times with the same coffee beans. That format is the worst that has ever happened to music, and if we had any bit dignity we would never purchase digital files in Amazon or iTunes safe for MP3 with an acceptable compression level. That, if we could buy them, because not only that is allowed to us. As the musical industry has no interest in resolving ITS problem (that is not our problem, it is those companies') it has not even been resolved how to charge for a MP3 download that includes import fees (well, if downloading from here a MP3 from a USA-based file server can be considered importing goods into Costa Rica!!!) so we don't have to get dizzy entering into the nineties to Titi Online to discover there is nothing by Muse, Andrew Bird, The Killers, Death Cab for Cutie, Paramore, Bj rk... (believe me, I looked them all up, even Norah jones and La Secta. They also were not there). This all leads me to the question, which I present with all due respect (NOT): What the fuck do they expect us to do??? It is outrageous; above all because in the best case they will sell us a watery coffee download that won't allow us to get all the details a vinyl or less compression would give us. In the worst case, post-MP3 groups will end up recording music with no harmonics or hidden sounds, because, what for? Nobody will hear it. They even admit it: "Some musicians and audio engineers say the MP3 format is changing the way studios mix their recordings. They say the MP3 format "flattens" dynamics differences in tone and volume in a song. As a result, a great deal of new music sounds very much alike, and there is nothing as focusing to create a dynamic listening experience. Why working so hard in creating complex sound if nobody can detect it?" (Rolling Stone, The Death of High fidelity, December 26 2007, taken from here). That's why I am not surprised by Adri n's post regarding the sales of old records. The price has nothing to do with it. The causes are related to the fetish object record and what it means or does not mean for people that have never purchased one. Adri n also asks if somebody here keeps buying records. I answered that I would if the stores sold anything I like. I do it even after the nausea I feel while reading "This phonogram is an intellectual work protected in favor of its producer COPYING IT IN WHOLE OR PARTIALLY IS FORBIDDEN " (like that, uppercased, yelling to whoever is only guilty of having bought a record and defending the producer, not the artist). But I am sure that almost nobody buys records because doing so is no longer a gratifying experience; because if buying a record is clicking to wait 15 days for it to reach the mailbox, we prefer clicking on the download link. But there is another reason for people not to buy records anymore. In one of my talks regarding the dictatorship of the all rights reserved, I asked the 30 twenty-something-year-old students if any of them had ever bought a record. One answered he had, because he is an author and performer (cantautor in Spanish) and understands the effort that producing a record entails. The rest of them had never done so. Is it possible that said young people have never listened to real music? Is it possible that, were it not for concerts, what they consider music is a set of washed-out MP3 that are about to fill up 1TB of their computer? Does people no longer buy records because they cannot differentiate one sound from the other? It is not very clear for me where do I want to get to. The recording industry is despicable. An industry that instead of innovating sets its energy on suing adolescents for downloading songs, trying to pass laws restricting our freedoms in Internet, putting up DRMs making us hostages to our devices* and forcing us to listen just the aroma of music, deserves my whole contempt. If we add ot this that said industry won't allow us to legally download their breadcrumbs because it has not understood that Internet does not need a van crossing borders, besides my contempt they deserve my pity and my heartfelt condolence. But the condolence is also for music, real music, that which is not compressed under the shoe using a terrible format. It is also for independent musicians that have not realized that begging for a bit of space to that industry they just add to themselves the "despicable" tag, given they deserve the fruits of their work to enter their bank account. However, there are good things that have come out of this absurdity. Be it for those that have joined projects such as Aut mata (even if it is in MP3) and for dreams come true such as Musopen (that have achieved that the music that's in theory Public Domain becomes so in practice as well). Good for the Electronic Frontier Foundation and the list of lawyers willing to defend people accused of ilegally downloading music in the USA. Good for the Creative Commons licenses that allow free sharing. All those are growing solutions, although none of them allows me to buy the Panamanian Carlos M ndez's record. Thankfully, a friend of mine who knows I will never give a dime to Apple, bought the files for me in iTunes. I thank him deeply, although I would have prefered to go to Auco Disco and have Mauricio tell me that the 2007 EP I have from Carlos is better than the record he did on 2009. * My devices don't have DRM because I use free software. I also use the ogg file format. Image by verbeeldingskr8

IT S FINALLY IN! With this last upload of the MiniUPnP daemon 1.7-3, I ve reached a first milestone: all of the MiniUPnP libraries, daemons and clients are finally in Debian, in a shape which I am not ashamed anymore. What is UPnP for? Back in the days (around Linux 2.0.3x), the only way to share a single Internet connection was to run a Linux box as your gateway. These days are far gone, and nearly every home connection is done through a mini home router, which also often provides a small switch and a WiFi access point. Every computer on the LAN gets its connectivity through the router thanks to NAT, and to be more accurate, masquerading. That works very well for outbound connections (browse the web, send a mail, etc.), but it is then a lot harder that something connects to a server located in the LAN. If you run a permanent server with a fixed LAN IP address, that s fine, you just configure your router to do some port forwarding, so that any inbound connection will be forwarded to the computer of your choice in the LAN. But if one application needs to listen on the public IP address just for a while (for example, IRC using DCC connection), so that something can connect to it, it becomes a way more tricky. That s when UPnP comes into play. UPnP stands for Universal Plug and Play. It s a protocol so that any application on the LAN can open ports on the public IP address of the gateway. Note that an UPnP router is also often referenced as UPnP IGD (Internet Gateway Device). Why using MiniUPnP, don t we have already linux-igd? Before my packaging work, the only UPnP server that was available in Debian was linux-igd, which has a dependency on libupnpX. That library is quite big for what it does (about 500KiB, on top of which you have to install the 200KiB daemon itself), while MiniUPnP is designed to be lightweight (only 158K, with dependencies on iptables, iproute, uuid-runtime and libc6 only). That seems ridiculous, but when you have only 8MiB of flash in your old router, saving few hundreds KiB is really important, and here, we re saving more than 500 KiB, with a full implementation of the UPnP protocol, and the Microsoft NATPmP implemented as well (yes, Microsoft decided that one protocol wasn t enough, and that they should implement their own (sic!)). Who is using MiniUPNP? Probably, you already have the libminiupnpc installed in your SID/wheezy desktop, because the transmission bittorent client which is installed with Gnome by default, depends on it, and minissdpd might be in your desktop too, if you installed the Recommends: packages as well. Which is why these are scoring quite high already in popcon. Previously, transmission was embedding its own version of the MiniUPnP client library, so it s nice to have it as a standalone shared library now. Also, because of its very light weight, and the fact that UPnP is often needed for gaming (both XBOX 360 and PSP need, and can use, the MiniUPnP daemon that runs on your IGD), many ISP who ship a router to their customer, install MiniUPnP in it. For example, at least, the 2nd and 3rd largest Internet access provider in France (Free Telecom and SFR) are shipping MiniUPnPd in their box (one of them using OpenWRT on their router). So probably as well, you are also using MiniUPnP daemon without knowing it. But isn t UPnP unsafe? To this question, I d answer that connecting to Internet is unsafe. It isn t less safe than connecting to Internet on a WiFi hotspot for example. So adding UPnP support on your gateway doesn t make it less safe, especially since MiniUPnP has some features to help with security (like an ACL with IP and port lists, or a secure mode were the router will only open ports to the client requesting it). Also, we are running a distribution where dependencies are easy to check, so it s easy for you to check if a client uses UPnP or not, and decide to trust it (or not). I personally have no issue trusting transmission, x-chat or warzone2010 (more exactly: I don t trust that incoming connectivity will make it less safe for these applications, there are so many other things that could go wrong). MiniUPnP consists of what? MiniUPnP is a set of 4 source packages to handle all of your UPnP needs: - The client library, libminiupnpc (and the corresponding -dev package), with its sidekick miniupnpc client program (you can use it to check the public IP of your gateway, or manually open ports). - The UPnP IGD daemon, miniupnpd, which just reached Debian Experimental. The latest version, 1.7-3, also includes debconf configuration so that you can choose the NIC and IP address to bind on. If you installed previous version, I strongly recommend upgrading to fix previous problems. - The libnatpmp client library (and the -dev package) for implementing the Microsoft protocol as a client (it also has a client package binary for your tests). - The minissdpd daemon to keep memory of all UPnP devices that announced themselves, and speed-up using the UPnP protocol (used by the MiniUPnP client library). A long process to have MiniUPnP in Debian All these 4 pieces of the puzzle were not as easy as it seems to be packaged in Debian. Especially, the IGD daemon, MiniUPnPd, used to depend on some header files that were not packaged in Debian. This was a decision of the iptables maintainers, which even after a long discussion, refused to add the necessary headers because it wasn t supposed to be a kernel API. While this might really be truth, the issue was that there was no public API available in the Linux kernel at all, and other distributions (Fedora, Gentoo, etc.) really did supply these needed files. As a consequence, during a very long time (years literally), shamefully, it was impossible to build the MiniUPnP daemon in Debian, while it was no problem on other distributions. Note that I do not want to start a troll discussion here, I m just stating facts, both parties have very strong and respectable arguments. Anyway, this needed API was finally embedded in MiniUPnPd itself thanks to a contributor to the project, and now it builds on both the Squeeze and the Wheezy (that is Linux 3.2) kernel without any issue. Call for testing Unfortunately, I don t have a Linux box as my Internet gateway. So I did the tests that I could: I installed MiniUPnPd on one computer on my LAN, and checked that I could open ports from another computer. So it seems to work, but I had no way to really test it live , with some real clients. So if you have an Debian SID box as your Internet gateway, feedback would be highly appreciated. What is remaining to do? A transition from libminiupnpc5 to libminiupnpc8 has to happen. Of course, this is too late for Wheezy, so this will happen right after the release. There s not so many reverse dependencies, so it should be ok, but I (yet) have zero experience with transitions. BTW, I had hard time to find out what this transition process was, and maybe some more effort for documenting it would help others (I m thinking about adding some text in the Debian wiki for that). Also, I have called for help on debian-devel, because I m currently the only one maintaining all these 4 source packages, and I m trying to have all my packages checked by others, and team maintained. So if you want to give a hand, or just declare yourself as a backup if I m busy, please raise your hand! Upstream is a friend of mine, and is very reactive to any requests (it very rarely took more than 48 hours to get a reply). Enjoy MiniUPnP and report bugs! Again, feedback would be appreciated. :)

In the process of installing and configuring wanna-build, I had lots of problems. Because there was almost no documentation on the subject (and some of the existing was outdated), I had to figure out how everything works by reading the source code and asking people on IRC and mailing lists. I use the most recent version (from git) and since it s not stable, there were some problems and bugs. I fixed several bugs in order to set the system up. I also documented every step so now there s documentation on everything I have done. I uploaded the documentation on installing and configuring wanna-build [0]. I installed and configured reprepro on a local machine in order to use it with wanna-build. In the process I made step-by step setup instructions [1]. I installed and configured sbuild. The lastest version of sbuild contains filters but they don t work. Their purpose is to keep only the necessary variables in the environment, but apparenly the configuration file which should set them is never read. That leaves the filter list empty which causes the environment to be empty. I had to disable them to keep my work going. I installed buildd. I had some problems setting it up there was a bug in the scripts related to parsing the configuration file. I was pointed to a helpful bug report on the topic and with some additional changes I eventually got it working. In the process I had to read some of the source code of buildd. I made a patch that fixes a bug in the manual page of reprepro and got it approved. I contacted the Alioth admin team via the web site, requesting to use their server for my project Clang for build services . I got approved and I created a git repository where I m now uploading my work. The repository that I use to upload changes to sbuild and buildd is [2]. I add the name of the used compiler to the control file and to the name of the deb package. I wrote a function for parsing the debian/rules file looking for hard-coded compiler there. If there is such compiler, which is different from the one chosen as a default, the building proccess exits with a clear error. I made changes to wanna-build, dpkg, sbuild and buildd to fix bugs in them and make it possible to use a compiler different from gcc for the build process.
After my work a build compiler can be set easily. If a package can not be built with that compiler, the build process fails with a clear error. I also modified the pgstatus web interface to display build logs and create reports on user request. I wrote complete documentation about setting up wanna-build, reprepro and buildd ([0] and [1]). This makes a fully functional buildd infrastructure. Now we have a full step by step tutorial. Me and Sylvestre Ledru who was my great mentor, modified the buildd daemon so it will be possible to be run several instances of it for different architectures on a single physical server. I made a lot of changes to pgstatus this is a web interface to the wanna-build database. I had to add some features and fix several problems. In the process I read some of the code of pgstatus and buildd-tools. Now I have a better understanding of how they work. I added a new feature in pgstatus that allows the user to create a full report of the state of all builds. When a designated reports page is opened, a table with the current state of the builds is shown, together with a list of all the reports saved to files for the current distribution and architecture. The reports can be downloaded directly from the
interface. They are stored in JSON format. The log files were not accessible from pgstatus because of lack of implementation and bugs. I created a script that reads emails sent by wanna-build and extracts log files from them. I made changes to wanna-build so that it feeds the database with the necessary information about package history. I updated the wanna-build patch to include these changes. As a result the buildd logs can now be opened from the web interface of pgstatus. I applied a patch that fixes bug [3] so that config_schroot.sh (this script installs and sets the default compiler) starts with the necessary privileges. I added a new feature to sbuild that allows scripts to be run right after the dependencies are installed in the chroot and before the actual package starts building. Now anybody can configure sbuild to run such a script by modifying the appropriate configuration file. By adding the following lines: $external_commands =
chroot-pre-build-commands => [
['/root/purge-gcc.sh'],
],
; The script /root/purge-gcc.sh is run by sbuild in the chroot. This script can be necessary to set everything in the environment so that packages are built exactly as needed. Me and my mentor used this to replace gcc with a script that removes all gcc-specific packages so that we can be sure whether a package uses clang or not. I have a fully functional buildd infrastructure installed on my development system. In order to get it running, I made several patches: dpkg.patch The main patch with a lot of changes related to compiler support for building.
wanna_build.patch Fixes bugs and makes wanna-build feed the database with package history information.
sbuild.patch Fixes bugs and work-arounds the problem that some sbuild scripts are called without reading the configuration file of sbuild. Adds configuration option that allows scripts to be run right after the dependencies are installed in the chroot and before the actual package starts building.
pgstatus.patch Fixes bugs and adds reports page to pgstatus. Me and my mentor contacted the Debian wanna-build team to connect our new service with Debian. We explained what the current state of the project is and what we need from them. I thank my mentor Sylvestre Ledru for his presentation on DebConf 12 ([4], [5]). He showed our project and it was seen that people are interested in it. A nice resume of his talk has been made by Michael Larabel on Phoronix [6]. I tested everything thoroughly to make sure my changes don t break anything. I am happy to say that the project is now finished. ==== [0] http://wiki.debian.org/DebianWannaBuildInfrastructureOnOneServer
[1] http://wiki.debian.org/SetupBuildServiceForWanna-build
[2] http://anonscm.debian.org/gitweb/?p=pkg-llvm/clang-build.git;a=summary
[3] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=608840
[4] http://sylvestre.ledru.info/blog/media/5/rebuild-debian-presentation-clang-July-10.pdf
[5] http://meetings-archive.debian.net/pub/debian-meetings/2012/debconf12/high/884_Build_Debian_with_another_compiler.ogv
[6] http://www.phoronix.com/scan.php?page=news_item&px=MTEzODQ

github has become the place where a lot of code is made available. It turns out that not everyone has the time or inclination to answer queries / questions or apply pull requests. So, whenever I am investigating using something I've found, I always look to see what the forks are; if there are common issues, etc. In the past, I've done this by hand, for small repositories it isn't too time consuming. Today, though, I sat down and wrote find-forks.py which will do this automatically. Now, I find a project, clone it, run ./find-forks.py and settle down with gitk to examine the history and various other forks. <script src="http://gist.github.com/3405534.js"></script> This works well for me, feel free to use or improve as you like. I hereby place this in the Public Domain.

So far this year, I ve read somewhere in the neighborhood of 5000 pages. As I ve started to read more, I ve started to watch TV, movies, and Youtube less, because they are simply boring and shallow in comparison. War and Peace, in particular, deeply touched me. Lately I have been reading the Wheel of Time series, which has its own unique characteristics. Whether an epic (or super-epic, such as Wheel of Time) novel, or the Sherlock Holmes series, or nonfiction works, there is something magical about reading a book. We often see characters, real or fictional, that rise from obscurity to do great things for the world. We are transported in time and place to a time or place we will never be able to experience, perhaps because it is long past, or perhaps because it never was. But in any case, we can be inspired. I am reminded of this quote: The people crazy enough to think they can change the world are the ones that do. If someone told me that a street vendor in Tunisia would, in less than a year, cause the overthrow of 4 dictatorships and reform in a handful more, I would have, yes, thought that was crazy. And while Mohamed Bouazizi isn t a household name in much of the world, he managed exactly that. But not just him. It took crazy unarmed people to occupy Tahrir Square, some to die, for progress to be made in Egypt. This story is written all over history. People have done the impossible, have defied all odds, through sheer belief that they could. Civil rights have been granted due to the leaders we all know, but also due to the millions of marchers we don t. Changing the world doesn t have to mean that the world knows you. It just has to mean that you love the world, as Tolstoy pointed out:

Love hinders death. Love is life. All, everything that I understand, I understand only because I love. Everything is, everything exists, only because I love. Everything is united by it alone. Love is God, and to die means that I, a particle of love, shall return to the general and eternal source.

Whatever your stance on religion, this is a powerful quote. Sometimes particles of love might look crazy. But isn t it then that they are the most alive? Isn t it then that they are the greatest hindrance to death and despair?

I have some long running server processes that I am going to launch soonish, and I want to be alerted when they're done. Using Gmail and Android, it's pretty easy:

1. In Gmail, set up a new label, "Alerts" or something like that.
2. Set up a filter in Gmail that matches messages along the lines of codered@example.com, and adds the Alerts label, does not skip the inbox and are always marked as important.
3. Now, on your Android phone, in Gmail -> Settings -> Account settings (select the account) -> Sync inboxes and labels, and select your Alerts label.
4. Then, go back and select Labels to Notify in settings, select "Alerts", and add it to "Notify in Status bar", select an appropriate ringtone, set vibrate to 'always', and deselect 'Notify once' - because we want to be notified any time an alert comes in.

That should do it - on your server, set up an alias in something like /etc/aliases that redirects email from the alert alias - codered in this case - to your own email address. Now, you can script alerts that will get their own ringtone on your phone like so: echo "Dave, I can't let you do that" mail -s "Warning, computer malfunction" codered@example.com Pretty simple and effective if you need a simple way for your computer to let you know that it needs attention right now.

You can use a PBO solver to clean up your system from unneeded automatically installed packages. First of all, you convert the system state to PB, and add an optimization function telling it to remove as many automatically installed packages as possible. Then you run this thing through a solver (such as clasp, which seems the fastest solver for PBO instances in the Debian archive) and convert its output to human-readable package names. Code is provided at http://anonscm.debian.org/gitweb/?p=users/jak/cleanup.git, under the MPL 2.0. You need to have python-apt and clasp installed to use it. There is potential minisat+ support, but it s currently a bit broken. To use, run python program_builder.py, and it will tell you which packages are no longer needed on your system. It ignores Suggests, if you want those in, you have to hack the code and replace Recommends by Recommends , Suggests . You can also turn of such dependencies by setting Program.hard_softdeps to False.
Filed under: APT2, Debian, Python

There's been a few links to this story of someone buying a system that turned out to have UEFI firmware and also turned out not to boot. Given all the press, it's unsurprising that people would assume that problems they have with UEFI booting are related to Secure Boot, but it's very unlikely that this is the actual problem here. First, nobody's shipping an appropriately signed operating system yet. A hardware vendor that enabled secure boot out of the box would be selling a machine that wouldn't boot any OS you could buy. That's a poor way to make money. Second, the system booted a Fedora 17 CD. Fedora 17 isn't signed, so if the firmware booted it then the firmware isn't enforcing Secure Boot. Third, it didn't boot the installed OS. That's really at the point of it sounding like a hardware problem - selling systems that don't run the OS you sold them with is a guaranteed way of getting enough support calls that you wouldn't make any money on them, ever.

To be fair, Linux compatibility with UEFI systems is still not as good as it is with BIOS systems. Fedora 18 will be using a new UEFI boot process and so far in our testing it's been significantly more reliable than Fedora 17. There's still some remaining issues that we're aware of and working on, but right now it's hugely more likely that failures to boot Fedora 17 on UEFI systems are down to our bugs rather than Secure Boot.

comments

Happy 19th birthday, Debian! The Debian project is 19 years old now. Following Francesca's invitation (and Rapha l's lead, and using Leandro's image, collaboratively as it always happens here), I will tell a bit of my memories: How I got to Debian. I am a Debian user since ~2000, and a Debian Developer since April 2003. But, just as Rapha l's, my history must go somewhat further back in time. In 1992, I got my first 1200bps modem, and almost immediately became an avid BBS user (what's that? Javier Matuk talks [in Spanish] about BBSes in his newspaper column back in 1994). By mid 1993 I started operating my own BBS, CatarSYS. One of the key points that defined CatarSYS is that my focus was large-scale communication. I started connecting to several BBS networks, allowing messages to be relayed to distant people, mainly in the USA and Spain, and getting some feeds that could be seen as the poor man's Usenet. At some point during the year I kept CatarSYS going, I got connected through a strange set of gateways to pure gold: A UUCP feed! This means, during some months, I operated the first (free, hobbyist) service that offered its users a free Internet mail address in Mexico. Yes, it was completely different to what we are used to today. I tried to connect to my provider at least three times a week, but this meant less than one week turnover time for messages sent to people anywhere in the world! But, back to Debian: Via this UUCP feed, I also got some real Usenet newsgroups Including several on the comp.os category. I remember reading about Linux back then, and learning some commands, but didn't really get hooked into it. I was in Israel from July 1994 to June 1995, doing completely non-computer-related tasks. Came back in 1995, and due to my father being an academic, got dialup internet access at home. WOW, *real* Internet! It didn't take me long to start downloading Linux information and floppy images. One failed after another. But before the end of the year, I found in a bookshop a book (that included a CD-ROM) called "Build your own web server with Linux". WOW again Remember this was still 1995! I bought it, and shortly afterwards, I had a Slackware system (Linux kernel 1.0.9) running. That meant many sleepless nights full of joy and frustration (as getting hardware to work was cloe to impossible). By 1996, I got (within one week) my two first real jobs: A systems administrator at a small ISP and a highschool teacher. At the ISP, I got a spare computer to play with Linux, as of course the Big Server was running with Windows NT 3.51. Poor little machine... By then I was already a part of the Mexican Linux User Group. This group had just printed a batch of Linux CDs RedHat 4.0. This was the first release that really made me happy and allowed me to do good work. Together with a friend I took to work with us, Juan Pablo Romero, we installed over a weekend a full replacement for our buggy NT machine, in much cheaper hardware. Of course, Linux was nowhere near corporate-recognition, and our project remained a project, not touching the Windows machine. Anyway... Several years passed, and I was happy with my RedHat choice. I won't mention the milestones and job changes, as it would get boring and leave the point completely aside. By the year 2000, I was quite more involved with the LUG, as well as with the computer security group in DGSCA-UNAM. I became also an OpenBSD user, and had got so hooked up in free software that I felt the need to collaborate: To be a little part of one of those Big Projects that had given me so much. But which one? I have never been much of a programmer Yes, I can solve my everyday needs and have fun with it, and sometimes a bit beyond that. I enjoy programming. But all of my projects have begun little and stayed little. I wanted to join OpenBSD, as it was a community I really believed in, but given my skillset (and given a flame-prone, aggressive developer community), I lost motivation to do so. By 2000, I had also lost faith in RedHat. I don't have the exact dates, so I might be some months off But after RedHat's IPO, I felt a sharp change. Version 7.0 was really demotivating It tried to offer a polished desktop experience, but was really buggy, unstable, and full of bad decisions. In Mexico, Pepe Neif had taken up the job of making a derivative distribution of RedHat (called LinuxPPP), pressing hundreds of CDs and making a teaching program I was part of several times. Talking with Pepe (who continued to release based on RedHat 6), he told me he was interested in switching over to become a Debian-based distribution, but the job of migrating his installed base made the project stall LinuxPPP reached only version 6.4. But I installed Debian in early 2000, and loved it. I started getting familiar with its social philosophy and foundation documents at the same time I started migrating my servers from RedHat to Debian This must have been by Spring 2000, as I installed Potato while it was frozen but not yet stable. By January 2002 I applied for NM. My process took a long time, as my AM got MIA when he had already approved me (but before sending the AM report), so basically I had to go through AM twice And by April 16, 2003, I got accepted as a DD. Contrary to what is acceptable today, I requested the full process to be done before starting to maintain any packages, as I didn't want to bother people with package sponsorship requests, so my whole process was done evaluating packages I would eventually upload. Since becoming a DD, my main involvement in the project has been in packaging groups (I was a pkg-perl founder and member for many years, and am currently working in the pkg-ruby-extras group). But, as I said, my main strength is not programming So my main involvement in Debian has been more social than technical: I have been a DebConf organizer since 2005, a very interesting, stressing, rewarding and (for some months) time-demanding role, and since 2009 I am part of the keyring maintainence team, which is much easier workload, although carries important ramifications. So, after 19 years of Debian, and after nine years of me being part of it, Debian is clearly my strongest link to the Free Software community, a project I have grown to love and whose way of being I share and enjoy studying and explaining. And it is a technically excellent product, and a great place to start and keep learning both about how every aspect and layer of an operating system works, and how human-to-human interaction works in such a diverse, almost impossible environment happens. Congratulations, Debian!

Attachment	Size
debian-19_500.png	149.81 KB

The biggest objection that most people have to UEFI Secure Boot as embodied in the Windows 8 certification requirements is the position of Microsoft as the root of trust. But we can go further than that - putting any third party in a position of trust means that there's the risk that your machine will end up trusting code that you don't want it to trust. Can we avoid that?

It turns out that the answer is yes, although perhaps a little more complicated than ideal. The Windows 8 certification requirements insist that (on x86, at least) the key databases be completely modifiable. That means you can delete all the keys provided by your manufacturer, including Microsoft's. However, as far as the spec is concerned, a system without keys is in what's called "Setup Mode", and in this state it'll boot anything - even if it doesn't have a signature.

So, we need to populate the key database. This isn't terribly difficult. James Bottomley has a suite of tools here, including support for generating keys and building an EFI binary that will enrol them into the key databases. So, now you have a bunch of keys and the public half of them is in your platform firmware. What next?

If you've got a system with plug-in graphics hardware, what happens next is that your system no longer has any graphics. The firmware-level drivers for any plug-in hardware also need to be signed, and won't run otherwise. That means no graphics in the firmware. If you're netbooting off a plug-in network card, or booting off a plug-in storage controller, you're going to have similar problems. This is the most awkward part of the entire process. The drivers are all signed with a Microsoft key, so you can't just enrol that without trusting everything else Microsoft have signed. There is a way around this, though. The typical way to use Secure Boot is to provide a list of trusted keys, but you can also provide trusted hashes. Doing this involves reading the device ROM, generating a SHA256 hash of it and then putting that hash in the key database.

This is, obviously, not very practical to do by hand. We intend to provide support for this in Fedora by providing a tool that gets run while the system is in setup mode, reads the ROMs, hashes them and enrols the hashes. We'll probably integrate that into the general key installation tool to make it a one-step procedure. The only remaining problem is that swapping your hardware or updating the firmware will take it back to a broken state, and sadly there's no good answer for that at the moment.

Once you've got a full set of enrolled keys and the hashes of any option ROMs, the only thing to do is to sign your bootloader and kernel. Peter Jones has written a tool to do that, and it's available here. It uses nss and so has support for using any signing hardware that nss supports, which means you can put your private key on a smartcard and sign things using that.

At that point, assuming your machine implements the spec properly everything that it boots will be code that you've explicitly trusted. But how do you know that your firmware is following the spec and hasn't been backdoored? That's tricky. There's a complete open implementation of UEFI here, but it doesn't include the platform setup code that you'd need to run it on any x86 hardware. In theory it'd be possible to run it on top of Coreboot, but right now that's not implemented. There is a complete port to the Beagleboard hardware, and there's instructions for using it here. The remaining issue is that you need a mechanism for securing access to the system flash, since if you can make arbitrary writes to it an attacker can just modify the key store. On x86 this is managed by the flash controller being switched into a mode where all writes have to be carried out from System Management Mode, and the code that runs there verifies that writes are correctly authenticated. I've no idea how you'd implement that on ARM.

There's still some work to be done in order to permit users to verify the entire stack, but Secure Boot does make it possible for the user to have much greater control over what their system runs. The freedom to make decisions about not only what your computer will run but also what it won't is an important one, and we're doing what we can to make sure that users have that freedom.

comments