Background I ve previously written about the claim that people use Autism as an excuse for bad behavior [1]. In summary it doesn t and such claims instead lead to people not being assessed for Autism. I ve also previously written about empathy and Autism in the context of discussions about conference sexual harassment [2]. The main point is that anyone who s going to blame empathy disorders for the widespread mistreatment of women in society and divert the subject from the actions of average men to men in minority groups isn t demonstrating empathy. Discussions of the actions of average men are so often derailed to cover Autism that the Geek Feminism Wiki has a page about the issue of blaming Autism [3]. The Latest Issue Last year Shanley Kane wrote an informative article for Medium titled What Can Men Do about the treatment of women in the IT industry [4]. It s a good article, I recommend reading it. As an aside @shanley s twitter feed is worth reading [5]. In response to Shanley s article Jeff Atwood wrote an article of the same title this year which covered lots of other things [6]. He writes about Autism but doesn t seem to realise that officially Asperger Syndrome is now Autism according to DSM-V (they decided that separate diagnosis of Autism, Asperger Syndrome, and PDD-NOS were too difficult and merged them). Asperger Syndrome is now a term that refers to historic issues (IE research that was published before DSM-V) and slang use. Gender and the Autism Spectrum Jeff claims that autism skews heavily towards males at a 4:1 ratio and cites the Epidemiology of Autism Wikipedia page as a reference. Firstly that page isn t a great reference, I fixed one major error (which was obviously wrong to anyone who knows anything about Autism and also contradicted the cited reference) in the first section while writing this post. The Wikipedia page cites a PDF about the Epidemiology of Autism that claims the 4.3:1 ratio of boys to girls [7]. However that PDF is a summary of other articles and the one which originated the 4.3:1 claim is behind a paywall. One thing that is worth noting in the PDF is that the section containing the 4.3:1 claim also references claims about correlations between race and Autism and studies contradicting such claims it notes the possibility of ascertainment bias . I think that anyone who reads that section should immediately consider the possibility of ascertainment bias in regard to the gender ratio. Most people who are diagnosed with Autism are diagnosed as children. An Autism diagnosis of a child is quite subjective, an important part is an IQ test (where the psychologist interprets the intent of the child in the many cases where answers aren t clear) to compare social skills with IQ. So whether a child is diagnosed is determined by the psychologist s impression of the child s IQ vs the impression of their social skills. Whether a child is even taken for assessment depends on whether they act in a way that s considered to be obviously different. Any child who is suspected of being on the Autism Spectrum will be compared other children who have been diagnosed (IE mostly boys) and this will probably increase the probability that a boy will be assessed. So an Aspie girl might not be assessed because she acts like other Aspie girls not like the Aspie boys her parents and teachers have seen. The way kids act is not solely determined by neuro-type. Our society expects and encourages boys to be louder than girls and take longer and more frequent turns to speak, this is so widespread that I don t think it s possible for parents to avoid it if their kids are exposed to the outside world. Because of this boys who would be diagnosed with Asperger Syndrome by DSM-IV tend to act in ways that are obviously different from other kids. While the combination of Autism and the the social expectations on girls tends to result in girls who are quiet, shy, and apologetic. The fact that girls are less obviously different and that their differences cause fewer difficulties for parents and teachers makes them less likely to be assessed. Note that the differences in behavior of boys and girls who have been diagnosed is noted by the professionals (and was discussed at a conference on AsperGirls that my wife attended) while the idea that this affects assessment rates is my theory. Jeff also cites the book The Essential Difference: Male And Female Brains And The Truth About Autism by Professor Simon Baron-Cohen (who s (in)famous for his Extreme Male Brain theory). The first thing to note about the Extreme Male Brain theory are that it depends almost entirely on the 4.3:1 ratio of males to females on the Autism Spectrum (which is dubious as I noted above). The only other evidence in support of it is subjective studies of children which suffer from the same cultural issues this is why double blind tests should be used whenever possible. The book Delusions of Gender by Cordelia Fine [8] debunks Simon Baron-Cohen s work among other things. The look inside feature of the Amazon page for Delusions of Gender allows you to read about Simon Baron-Cohen s work [9]. Now even if the Extreme Male Brain theory had any merit it would be a really bad idea to cite it (or a book based on it) if you want to make things better for women in the IT industry. Cordelia s book debunks the science and also shows how such claims about supposed essential difference are taken as exclusionary. The Problem with Jeff Atwood Jeff suggests in his post that men should listen to women. Then he and his followers have a huge flame-war with many women over twitter during which which he tweeted Trying to diversify my follows by following any female voices that engaged me in a civil, constructive way recently . If you only listen to women who agree with you then that doesn t really count as listening to women. When you have a stated policy of only listening to women who agree then it seems to be more about limiting what women may feel free to say around you. The Geek Feminism wiki page about the Tone Argument [10] says the following: One way in which the tone argument frequently manifests itself is as a call for civility. A way to gauge whether a request for civility is sincere or not is to ask whether the person asking for civility has more power along whatever axes are contextually relevant (see Intersectionality) than the person being called incivil , less power, or equal power. Often, people who have the privilege of being listened to and taken seriously level accusations of incivility as a silencing tactic, and label as incivil any speech or behavior that questions their privilege. For example, some men label any feminist thought or speech as hostile or impolite; there is no way for anybody to question male power or privilege without being called rude or aggressive. Likewise, some white people label any critical discussion of race, particularly when initiated by people of color, as incivil. Writing about one topic is also a really good idea. A blog post titled What Can Men Do should be about things that men can do. Not about Autism, speculation about supposed inherent differences between men and women which are based on bad research, gender diversity in various occupations, etc. Following up a post on What Can Men Do with discussion (in blog comments and twitter) about what women should do before they are allowed to join the conversation is ridiculous. Jeff s blog post says that men should listen to women, excluding women based on the tone argument is gross hypocrisy. Swearing Jeff makes a big deal of the fact that Shanley uses some profane language in her tweets. This combines a couple of different ways of silencing women. It s quite common for women to be held to a high standard of ladylike behavior, while men get a free pass on doing the same thing. One example of this is the Geek Feminism article about the results of Sarah Sharp s request for civility in the Linux kernel community [11]. That s not an isolated incident, to the best of my recollection in 20+ years my local Linux Users Group has had only one debate about profanity on mailing lists in that case a woman (who is no longer active in the group) was criticised for using lesser profanity than men used both before and after with no comment (as an experiment I used some gratuitous profanity a couple of weeks later and no-one commented). There is also a common difference in interpretation of expressions of emotion, when a woman seems angry then she invariably has men tell her to change her approach (even when there are obvious reasons for her anger) while when a man is angry the possibility that other people shouldn t make him angry will usually be considered. The issues related to the treatment of women have had a large affect on Shanley s life and her friend s lives. It s quite understandable that she is angry about this. Her use of profanity in tweets seems appropriate to the situation. Other Links Newsweek s Gentlemen in Technology article has a section about Jeff [12], it s interesting to note his history of deleting tweets and editing his post. I presume he will change his post in response to mine and not make any note of the differences. Jacob Kaplan-Moss wrote a good rebuttal to Jeff s post [13]. It s a good article and has some other relevant links that are worth reading.

• [1] http://etbe.coker.com.au/2012/04/23/autism-as-an-excuse/
• [2] http://etbe.coker.com.au/2010/12/08/empathy-autism-and-geeks/
• [3] http://geekfeminism.wikia.com/wiki/Autism_is_to_blame
• [4] https://medium.com/tech-culture-briefs/a1e93d985af0
• [5] https://twitter.com/shanley
• [6] http://blog.codinghorror.com/what-can-men-do/
• [7] http://idea.library.drexel.edu/bitstream/1860/2632/1/2006175339.pdf
• [8] http://en.wikipedia.org/wiki/Delusions_of_Gender
• [9] http://www.amazon.com/Delusions-Gender-Society-Neurosexism-Difference/dp/0393340244
• [10] http://geekfeminism.wikia.com/wiki/Tone_argument
• [11] http://geekfeminism.org/2013/09/05/tone-policing-a-tool-for-protecting-male-power/
• [12] http://www.newsweek.com/today-tabs-meet-gentlemen-technology-248696
• [13] http://jacobian.org/writing/what-can-men-do/

I hate the days when I wake up at odd hours for no good reason. This morning I woke up at 4:30am briefly, with no idea why. It makes me wonder if there was some sort of external noise, and that's what woke Zoe up yesterday morning. I got myself going at a reasonable hour this morning, and decided to walk to the Bulimba post office to collect a package that was waiting for me. It was a nice enough morning for a walk, and I'm glad I decided to do it. Along the way, I observed multiple grandmothers walking their grandchildren to school. I've been thinking a lot about next year lately. I feel very strongly about Zoe not having to go into after school care. It seems that one of the new societal norms is the grandparents have to step in with the school transportation and after school care. That's not going to be particularly practical in our situation though, so I'm currently leaning towards seeking some sort of flexible employment next year that will allow me to be able to be available after Zoe finishes school. I was fairly motivated to get stuck into my to do list after my walk. A long standing item was to cancel one of my remaining US credit cards, and the time difference worked out well to do that, so I finally took care of that. I then realised it was a good time for the US East Coast, so I did a Hangout with one of my best friends in New York. It was great to have a catch up. After that, I finished off my US tax return, and also sorted out some other stuff that needed finishing off. I had to go to the Valley to find a FedEx location, so I drove over there, and on the way back stopped off in Bulimba to get some Mother's Day cards. By that stage it was about time to pick up Zoe from Kindergarten, so I quickly grabbed some lunch and then drove over to pick her up. She didn't nap again today, so that made our departure from Kindergarten nice and easy. I'd booked both of us in for haircuts at 3pm, so we got home with enough time for Zoe to ride her scooter to the hairdresser. We had our haircuts and Zoe had a bit of a play in the play corner, and we headed back home. I'd bought a venus fly trap from Bunnings last week while we were picking up other bits and pieces. I wanted to show Zoe how they worked, so we watched a few YouTube videos. This one was particularly good. After that, we went down to the rubbish skip with our fly trap to see if we could catch any flies in it, but Zoe quickly lost interest. So we headed back upstairs, and triggered one with a toothpick. I probably could have done this on a Friday and called it a Science Friday activity with a botany bent. After that, we just watched some TV together on the couch until Sarah arrived to pick her up.

Distributing software efficiently accross your platform can be difficult. Every distribution comes with a package manager which is usually suited for this task. APT can be relied upon on when using Debian or a derivative. Unfortunately, the official repositories may not contain everything you need. When you require unpackaged software or more recent versions, it is possible to setup your own local repository. Most of what is presented here was setup for Dailymotion and was greatly inspired by the work done by Rapha l Pinson at Orange.

Setting up your repositories There are three kinds of repositories you may want to setup:

1. A distribution mirror. Such a mirror will save bandwidth, provide faster downloads and permanent access, even when someone searches Google on Google.
2. A local repository for your own packages with the ability to have a staging zone to test packages on some servers before putting them in production.
3. Mirrors for unofficial repositories, like Ubuntu PPA. To avoid unexpected changes, such a repository will also get a staging and a production zone.

Before going further, it is quite important to understand what a repository is. Let s illustrate with the following line from my /etc/apt/sources.list:

deb http://ftp.debian.org/debian/ unstable main contrib non-free

In this example, http://ftp.debian.org/debian/ is the repository and unstable is the distribution. A distribution is subdivided into components. We have three components: main, contrib and non-free. To setup repositories, we will use reprepro. This is not the only solution but it has a good balance between versatility and simplicity. reprepro can only handle one repository. So, the first choice is about how you will split your packages in repositories, distributions and components. Here is what matters:

• A repository cannot contain two identical packages (same name, same version, same architecture).
• Inside a component, you can only have one version of a package.
• Usually, a distribution is a subset of the versions while a component is a subset of the packages. For example, in Debian, with the distribution unstable, you choose to get the most recent versions while with the component main, you choose to get DFSG-free software only.

If you go for several repositories, you will have to handle several reprepro instances and won t be able to easily copy packages from one place to another. At Dailymotion, we put everything in the same repository but it would also be perfectly valid to have three repositories:

• one to mirror the distribution,
• one for your local packages, and
• one to mirror unofficial repositories.

Here is our target setup:

Initial setup First, create a system user to work with the repositories:

$ adduser --system --disabled-password --disabled-login \
>         --home /srv/packages \
>         --group reprepro

All operations should be done with this user only. If you want to setup several repositories, create a directory for each of them. Each repository has those subdirectories:

• conf/ contains the configuration files,
• gpg/ contains the GPG stuff to sign the repository¹,
• logs/ contains the logs,
• www/ contains the repository that should be exported by the web server.

Here is the content of conf/options:

outdir +b/www
logdir +b/logs
gnupghome +b/gpg

Then, you need to create the GPG key to sign the repository:

$ GNUPGHOME=gpg gpg --gen-key
Please select what kind of key you want:
   (1) RSA and RSA (default)
   (2) DSA and Elgamal
   (3) DSA (sign only)
   (4) RSA (sign only)
Your selection? 1
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0) 10y
Key expires at mer. 08 nov. 2023 22:30:58 CET
Is this correct? (y/N) y
Real name: Dailymotion Archive Automatic Signing Key
Email address: the-it-operations@dailymotion.com
Comment: 
[...]

By setting an empty password, you allow reprepro to run unattended. You will have to distribute the public key of your new repository to let APT check the archive signature. An easy way is to ship it in some package.

Local mirror of an official distribution Let s start by mirroring a distribution. We want a local mirror of Ubuntu Precise. For this, we need to do two things:

1. Setup a new distribution in conf/distributions.
2. Configure the update sources in conf/updates.

Let s add this block to conf/distributions:

# Ubuntu Precise
Origin: Ubuntu
Label: Ubuntu
Suite: precise
Version: 12.04
Codename: precise
Architectures: i386 amd64
Components: main restricted universe multiverse
UDebComponents: main restricted universe multiverse
Description: Ubuntu Precise 12.04 (with updates and security)
Contents: .gz .bz2
UDebIndices: Packages Release . .gz
Tracking: minimal
Update: - ubuntu-precise ubuntu-precise-updates ubuntu-precise-security
SignWith: yes

This defines the precise distribution in our repository. It contains four components: main, restricted, universe and multiverse (like the regular distribution in official repositories). The Update line starts with a dash. This means reprepro will mark everything as deleted before updating with the provided sources. Old packages will not be kept when they are removed from Ubuntu. In conf/updates, we define the sources:

# Ubuntu Precise
Name: ubuntu-precise
Method: http://fr.archive.ubuntu.com/ubuntu
Fallback: http://de.archive.ubuntu.com/ubuntu
Suite: precise
Components: main main multiverse restricted universe
UDebComponents: main restricted universe multiverse
Architectures: amd64 i386
VerifyRelease: 437D05B5
GetInRelease: no
# Ubuntu Precise Updates
Name: ubuntu-precise-updates
Method: http://fr.archive.ubuntu.com/ubuntu
Fallback: http://de.archive.ubuntu.com/ubuntu
Suite: precise-updates
Components: main restricted universe multiverse
UDebComponents: main restricted universe multiverse
Architectures: amd64 i386
VerifyRelease: 437D05B5
GetInRelease: no
# Ubuntu Precise Security
Name: ubuntu-precise-security
Method: http://fr.archive.ubuntu.com/ubuntu
Fallback: http://de.archive.ubuntu.com/ubuntu
Suite: precise-security
Components: main restricted universe multiverse
UDebComponents: main restricted universe multiverse
Architectures: amd64 i386
VerifyRelease: 437D05B5
GetInRelease: no

The VerifyRelease lines are GPG key fingerprint to use to check the remote repository. The key needs to be imported in the local keyring:

$ gpg --keyring /usr/share/keyrings/ubuntu-archive-keyring.gpg \
>     --export 437D05B5   GNUPGHOME=gpg gpg --import

Another important point is that we merge three distributions (precise, precise-updates and precise-security) into a single distribution (precise) in our local repository. This may cause some difficulties with tools expecting the three distributions to be available (like the Debian Installer²). Next, you can run reprepro and ask it to update your local mirror:

$ reprepro update

This will take some time on the first run. You can execute this command every night. reprepro is not the fastest mirror solution but it is easy to setup, flexible and reliable.

Repository for local packages Let s configure the repository to accept local packages. For each official distribution (like precise), we will configure two distributions:

• precise-staging contains packages that have not been fully tested and not ready to go to production.
• precise-prod contains production packages copied from precise-staging.

In our workflow, packages are introduced in precise-staging where they can be tested and will be copied to precise-prod when we want them to be available for production. You can adopt a more complex workflow if you need. The reprepro part is quite easy. We add the following blocks into conf/distributions:

# Dailymotion Precise packages (staging)
Origin: Dailymotion #  
Label: dm-staging   #  
Suite: precise-staging
Codename: precise-staging
Architectures: i386 amd64 source
Components: main role/dns role/database role/web #  
Description: Dailymotion Precise staging repository
Contents: .gz .bz2
Tracking: keep
SignWith: yes
NotAutomatic: yes #  
Log: packages.dm-precise-staging.log
 --type=dsc email-changes
# Dailymotion Precise packages (prod)
Origin: Dailymotion #  
Label: dm-prod      #  
Suite: precise-prod
Codename: precise-prod
Architectures: i386 amd64 source
Components: main role/dns role/database role/web #  
Description: Dailymotion Precise prod repository
Contents: .gz .bz2
Tracking: keep
SignWith: yes
Log: packages.dm-precise-prod.log

First notice we use several components (in ):

• main will contain packages that are not specific to a subset of the platform. If you put a package in main, it should work correctly on any host.
• role/* are components dedicated to a subset of the platform. For example, in role/dns, we ship a custom version of BIND.

The staging distribution has the NotAutomatic flag (in ) which disallows the package manager to install those packages except if the user explicitely requests it. Just below, when a new dsc file is uploaded, the hook email-changes will be executed. It should be in the conf/ directory. The Origin and Label lines (in ) are quite important to be able to define an explicit policy of which packages should be installed. Let s say we use the following /etc/apt/sources.list file:

# Ubuntu packages
deb http://packages.dm.gg/dailymotion precise main restricted universe multiverse
# Dailymotion packages
deb http://packages.dm.gg/dailymotion precise-prod    main role/dns
deb http://packages.dm.gg/dailymotion precise-staging main role/dns

All servers have the precise-staging distribution. We must ensure we won t install those packages by mistake. The NotAutomatic flag is one possible safe-guard. We also use a tailored /etc/apt/preferences:

Explanation: Dailymotion packages of a specific component should be more preferred
Package: *
Pin: release o=Dailymotion, l=dm-prod, c=role/*
Pin-Priority: 950
Explanation: Dailymotion packages should be preferred
Package: *
Pin: release o=Dailymotion, l=dm-prod
Pin-Priority: 900
Explanation: staging should never be preferred
Package: *
Pin: release o=Dailymotion, l=dm-staging
Pin-Priority: -100

By default, packages will have a priority of 500. By setting a priority of -100 to the staging distribution, we ensure the packages cannot be installed at all. This is stronger than NotAutomatic which sets the priority to 1. When a package exists in Ubuntu and in our local repository, we ensure that, if this is a production package, we will use ours by using a priority of 900 (or 950 if we match a specific role component). Have a look at the How APT Interprets Priorities section of apt_preferences(5) manual page for additional information. Keep in mind that version matters only when the priority is the same. To check if everything works as you expect, use apt-cache policy:

$ apt-cache policy php5-memcache
  Installed: 3.0.8-1~precise2~dm1
  Candidate: 3.0.8-1~precise2~dm1
  Version table:
 *** 3.0.8-1~precise2~dm1 0
        950 http://packages.dm.gg/dailymotion/ precise-prod/role/web amd64 Packages
        100 /var/lib/dpkg/status
     3.0.8-1~precise1~dm4 0
        900 http://packages.dm.gg/dailymotion/ precise-prod/main amd64 Packages
       -100 http://packages.dm.gg/dailymotion/ precise-staging/main amd64 Packages
     3.0.6-1 0
        500 http://packages.dm.gg/dailymotion/ precise/universe amd64 Packages

If we want to install a package from the staging distribution, we can use apt-get with the -t precise-staging option to raise the priority of this distribution to 990. Once you have tested your package, you can copy it from the staging distribution to the production distribution:

$ reprepro -C main copysrc precise-prod precise-staging wackadoodle

Local mirror of third-party repositories Sometimes, you want a software published on some third-party repository without to repackage it yourself. A common example is the repositories edited by hardware vendors. Like for an Ubuntu mirror, there are two steps: defining the distribution and defining the source. We chose to put such mirrors into the same distributions as our local packages but with a dedicated component for each mirror. This way, those third-party packages will share the same workflow as our local packages: they will appear in the staging distribution, we validate them and copy them to the production distribution. The first step is to add the components and an appropriate Update line to conf/distributions:

Origin: Dailymotion
Label: dm-staging
Suite: precise-staging
Components: main role/dns role/database role/web vendor/hp
Update: hp
# [...]
Origin: Dailymotion
Label: dm-prod
Suite: precise-prod
Components: main role/dns role/database role/web vendor/hp
# [...]

We added the vendor/hp component to both the staging and the production distributions. However, only the staging distribution gets an Update line (remember, packages will be copied manually into the production distribution). We declare the source in conf/updates:

# HP repository
Name: hp
Method: http://downloads.linux.hp.com/SDR/downloads/ManagementComponentPack/
Suite: precise/current
Components: non-free>vendor/hp
Architectures: i386 amd64
VerifyRelease: 2689B887
GetInRelease: no

Don t forget to add the GPG key to your local keyring. Notice an interesting feature of reprepro: we copy the remote non-free component to our local vendor/hp component. Then, you can synchronize the mirror with reprepro update. Once the packages have been tested, you will have to copy them in the production distribution.

Building Debian packages Our reprepro setup seems complete, but how do we put packages into the staging distribution? You have several options to build Debian packages for your local repository. It really depends on how much time you want to invest in this activity:

1. Build packages from source by adding a debian/ directory. This is the classic way of building Debian packages. You can start from scratch or use an existing package as a base. In the latest case, the package can be from the official archive but for a more recent distribution or a backport or from an unofficial repository.
2. Use a tool that will create a binary package from a directory, like fpm. Such a tool will try to guess a lot of things to minimize your work. It can even download everything for you.

There is no universal solution. If you don t have the time budget for building packages from source, have a look at fpm. I would advise you to use the first approach when possible because you will get those perks for free:

• You keep the sources in your repository. Whenever you need to rebuild something to fix an emergency bug, you won t have to hunt the sources which may be unavailable when you need them the most. Of course, this only works if you build packages that don t download stuff directly from the Internet.
• You also keep the recipe³ to build the package in your repository. If someone enables some option and rebuild the package, you won t accidently drop this option on the next build. Those changes can be documented in debian/changelog. Moreover, you can use a version control software for the whole debian/ directory.
• You can propose your package for inclusion into Debian. This will help many people once the package hits the archive.

Builders We chose pbuilder as a builder⁴. Its setup is quite straightforward. Here is our /etc/pbuilderrc:

DISTRIBUTION=$DIST
NAME="$DIST-$ARCH"
MIRRORSITE=http://packages.dm.gg/dailymotion
COMPONENTS=("main" "restricted" "universe" "multiverse")
OTHERMIRROR="deb http://packages.dm.gg/dailymotion $ DIST -staging main"
HOOKDIR=/etc/pbuilder/hooks.d
BASE=/var/cache/pbuilder/dailymotion
BASETGZ=$BASE/$NAME/base.tgz
BUILDRESULT=$BASE/$NAME/results/
APTCACHE=$BASE/$NAME/aptcache/
DEBBUILDOPTS="-sa"
KEYRING="/usr/share/keyrings/dailymotion-archive.keyring.gpg"
DEBOOTSTRAPOPTS=("--arch" "$ARCH" "--variant=buildd" "$ DEBOOTSTRAPOPTS[@] " "--keyring=$KEYRING")
APTKEYRINGS=("$KEYRING")
EXTRAPACKAGES=("dailymotion-archive-keyring")

pbuilder is expected to be invoked with DIST, ARCH and optionally ROLE environment variables. Building the initial bases can be done like this:

for ARCH in i386 amd64; do
  for DIST in precise; do
    export ARCH
    export DIST
    pbuilder --create
  done
done

We don t create a base for each role. Instead, we use a D hook to add the appropriate source:

#!/bin/bash
[ -z "$ROLE" ]    
  cat >> /etc/apt/sources.list <<EOF
deb http://packages.dm.gg/dailymotion $ DIST -staging role/$ ROLE 
EOF
 
apt-get update

We ensure packages from our staging distribution are preferred over other packages by adding an /etc/apt/preferences file in a E hook:

#!/bin/bash
cat > /etc/apt/preferences <<EOF
Explanation: Dailymotion packages are of higher priority
Package: *
Pin: release o=Dailymotion
Pin-Priority: 900
EOF

We also use a C hook to get a shell in case there is an error. This is convenient to debug a problem:

#!/bin/bash
apt-get install -y --force-yes vim less
cd /tmp/buildd/*/debian/..
/bin/bash < /dev/tty > /dev/tty 2> /dev/tty

A manual build can be run with:

$ ARCH=amd64 DIST=precise ROLE=web pbuilder \
>         --build somepackage.dsc

Version numbering To avoid to apply complex rules to chose a version number for a package, we chose to treat everything as a backport, even in-house software. We use the following scheme: X-Y~preciseZ+dmW.

• X is the upstream version⁵.
• Y is the Debian version. If there is no Debian version, use 0.
• Z is the Ubuntu backport version. Again, if such a version doesn t exist, use 0.
• W is our version of the package. We increment it when we make a change to the packaging. This is the only number we are allowed to control. All the others are set by an upstream entity, unless it doesn t exist and in this case, you use 0.

Let s suppose you need to backport wackadoodle. It is available in a more recent version of Ubuntu as 1.4-3. Your first backport will be 1.4-3~precise0+dm1. After a change to the packaging, the version will be 1.4-3~precise0+dm2. A new upstream version 1.5 is available and you need it. You will use 1.5-0~precise0+dm1. Later, this new upstream version will be available in some version of Ubuntu as 1.5-3ubuntu1. You will rebase your changes on this version and get 1.5-3ubuntu1~precise0+dm1. When using Debian instead of Ubuntu, a compatible convention could be : X-Y~bpo70+Z~dm+W.

Uploading To upload a package, a common setup is the following workflow:

1. Upload the source package to an incoming directory.
2. reprepro will notice the source package, check its correctness (signature, distribution) and put it in the archive.
3. The builder will notice a new package needs to be built and build it.
4. Once the package is built, the builder will upload the result to the incoming directory.
5. reprepro will notice again the new binary package and integrate it in the archive.

This workflow has the disadvantage to have many moving pieces and to leave the user in the dark while the compilation is in progress. As an alternative, a simple script can be used to execute each step synchronously. The user can follow on their terminal that everything works as expected. Once we have the .changes file, the build script just issues the appropriate command to include the result in the archive:

$ reprepro -C main include precise-staging \
>      wackadoodle_1.4-3~precise0+dm4_amd64.changes

Happy hacking!

---

1. The gpg/ directory could be shared by several repositories.
2. We teached Debian Installer to work with our setup with an appropriate preseed file.
3. fpm-cookery is a convenient tool to write recipes for fpm, similar to Homebrew or a BSD port tree. It could be used to achieve the same goal.
4. sbuild is an alternative to pbuilder and is the official builder for both Debian and Ubuntu. Historically, pbuilder was more focused on developers needs.
5. For a Git snapshot, we use something like 1.4-git20130905+1-ae42dc1 which is a snapshot made after version 1.4 (use 0.0 if no version has ever been released) at the given date. The following 1 is to be able to package different snapshots at the same date while the hash is here in case you need to retrieve the exact snapshot.

Current Glamor Performance I finally managed to get a Gigabyte Brix set up running Debian so that I could do some more reasonable performance characterization of Glamor in its current state. I wanted to use this particular machine because it has enough cooling to keep from thermally throttling the CPU/GPU package. This is running my glamor-server branch of the X server, which completes the core operation rework and then has some core X server performance improvements as well for filled and outlined arcs. Changes in X11perf First off, I did some analysis of what x11perf was doing and found that it wasn t quite measuring what we thought. I m not interested in competing on x11perf numbers absolutely, I m only interested in doing relative measurements of useful operations, so fixing the tool to measure what I want seems reasonable to me. When x11perf was first written, it drew 100x100 rectangles tight against one another without any gap. And, it filled the window with them, drawing a 6x6 grid of 100x100 rectangles in a 600x600 window. To better exercise the rectangle code and check edge conditions better, we added a one pixel gap between the rectangles. However, we didn t reduce the number of rectangles drawn, so we ended up drawing 11 of the 36 rectangles on top of the first set of 25. Simple region computations would allow the X server to draw only 25 most of the time, skipping the redundant rectangles. The vertical and horizontal line tests were added a while after the first set of tests, and were done without regard to how an X server might optimize for them. x11perf draws these lines packed tightly together, creating a single square of pixels for the result. EXA, UXA and SNA all take vertical and horizontal lines and convert them to rectangles, then take the rectangles and clip them against the window clip list by computing a region from them and intersecting that with the GC composite clip. It s a completely reasonable plan, however, when you take what x11perf was drawing and run it through this code, you end up with a single solid rectangle. Which is surprisingly fast, compared with drawing individual lines. I fixed the overlapping rectangle case by reducing the number of boxes drawn from 36 to 25, and I fixed the vertical and horizontal line case by spacing the lines a pixel apart. I ve pushed out these changes to my x11perf repository on freedesktop.org. What s Fast Things that match GL s capabilities are fast, things which don t are slow. No surprises there. What s interesting is precisely what matches GL Patterns For Free Because GL makes it easy to program fill patterns into the GPU, there are essentially no performance differences between solid and patterned operations. GL Lines Glamor uses GL lines, which can be programmed to match X semantics, to quite good effect. The only trick required was to deal with cap styles. GL never draws the final pixel in a line, while X does unless the cap style is CapNotLast. The solution was to draw an extra line segment containing a single pixel at the end of every joined set of lines for this case. The other implicit requirement is that all zero width lines look the same. Right now, I ve solved that for fill styles and raster ops as they re all drawn with the same GL operations. However, for plane masks, we re currently falling back to software, which may draw something different. Fixing that isn t impossible, it s just tedious. Text Pushing all of the work of drawing core text into glamor wasn t terribly difficult, and the results are pretty spectacular. What s Slow We ve still got room for improvement in Glamor, but there aren t any obvious show-stoppers to getting great performance for reasonable X applications anymore. Wide Lines and Arcs One of the speed-ups I ve made in my glamor branch is to merge all of drawing of multiple filled and zero-width arcs into a single giant GL request. That turned out to both improve performance and save a bit of code. Right now, drawing wide lines and wide arcs doesn t do this, and so we suffer from submitting many smaller requests to GL. It s hard to get excited about speeding any of this up as all of the wide primitives are essentially unused these days. Filled Polygons Because X only lets applications draw a single polygon in each request, Glamor can t really gain any efficiency from batching work unless we start looking ahead in the X protocol stream to see if the next request is another polygon. Alternatively, we could leave the span operation pending to see if more spans were coming before the X serve went idle. Neither of these is all that exciting though; X polygons just aren t that useful. Render Operations These are still not structured to best fit modern GL; some work here would help a bunch. We ve got a gsoc student ready to go at this though, so I expect we ll have much better numbers in a few months. Window Operations You wouldn t think that moving and resizing windows would be so limited by drawing performance, but x11perf tests these with tiny little windows, and each operation draws or copies only a couple of little rectangles, which makes GL quite expensive. Working on speeding up GL for small numbers of operations would help a bunch here. Unexpected Results Solid rectangles are actually running slower than patterned rectangles, and I really have no idea why. The CPU is close to idle during the 500x500 solid rectangle test (as you d expect, given the workload), the vertex and fragment shaders look correct out of the compiler, and yet solid rectangles run at only 0.80 of the performance of the patterned rectangles. GL semantics for copying data essentially preclude overlapping blts of any form. There s the NVtexturebarrier extension which at least lets us do blts within the same object, but even that doesn t define how overlapping blts work. So, we have to create a temporary copy for this operation to make it work right. I was worried that this would slow things down, but the Iris Pro 3D engine is enough faster than the 2D engine that even with the extra copy, large scrolls and copies within the same object are actually faster. Results Here s a giant image showing the ratio of Glamor to both UXA and SNA running on the same machine, with all of the same software; the only change between runs was to switch the configured acceleration architecture.

Today I released AppStream and libappstream 0.6.1, which feature mostly bugfixes, so nothing incredibly exciting to see there (but this also means no API/ABI breaks). The release clarifies some paragraphs in the spec which people found confusing, and fixes a few issues (like one example in the docs not being valid AppStream metadata). As only spec extension, we introduce a priority property in distro metadata to allow metadata from one repository to override data shipped by another one. This is used (although with a similar syntax) in Fedora already to have placeholder data for non-free stuff, which gets overridden by the real metadata if a new application was added. In general, the property tag was added to make the answer to the question which data is preferred much less magic. The libappstream library got some new API to query component data in different ways, and I also brought back support for Vala (so if you missed the Vapi file: It s back now, although you have to manually enable this feature). The CLI tool also got some extensions to query AppStream data. Here is a brief introduction: First of all, we need to make sure the database is up-to-date, which should be the case already (it is rebuilt automatically): The database will only be rebuilt when necessary, if you want to force a rebuild anyway, use the force parameter. Now imagine we want to search for an app containing the word media (in description, keywords, summary, ): which will return: If we already know the name if a .desktop-file or the ID of a component, we can have the tool print out information about the application, including which package it was installed from: If we want to see more details, including e.g. a screenshot URL and a longer description, we can pass details to the tool: (I truncated the results slightly ) Okay, so far so good. But now it gets really exciting (and this is a feature added with 0.6.1): We can now query a component by the items it provides. For example, I want to know which software provides the library libfoo.2: This also works with binaries, or Python modules: This stuff works distribution-agnostic, and as long as software ships upstream metadata with a valid <provides/> field, or the distributor adds it while generating AppStream distro metadata.
This means that software can as soon as we have sufficient metadata of this kind declare it s dependencies upstream in form of a simple text file, referencing the needed components to build and run it on any Linux distribution. Users can simply install missing stuff by passing that file to their package manager, which can look up the components->packaging mapping and versions and do the right thing in installing the dependencies. So basically, this allows things pip -r does for Python, but for any application (not only Python stuff), and based on the distributors package database. With the provides-items, we can also scan software to detect it s dependencies automatically (and have it in a distro-agnostic form directly). We can also easily search for missing mimetype-handlers, missing kernel-modules, missing firmware etc. to install it on-demand, making the system much smarter in handling it s dependencies. And users don t need to do search orgies to find the right component for a given task. Also on my todo list for the future, based on this feature: A small tool telling upstream authors which distribution has their application in which version, using just one command (and AppStream data from multiple distros). Also planned: A cross-distro information page showing which distros ship which library versions, Python modules and application versions (and also the support status of the distro), so developers know which library versions (or GCC versions etc.) they should at least support to make their application easily available on most distributions. As always, you can get the releases on Freedesktop, as well es the AppStream specification.

Here are two separate ideas about programming, Internet security, Internet architecture, and free software. Both of them are fundamental to everything those of us who work on free software are doing.

1. Writing secure and reliable code is a highly complex and demanding task. It's something that one has to learn, like any other skilled profession. It's not something we're very good at teaching via any mechanism other than apprenticeship and experimentation. The field is changing quickly; if you took ten years off from writing security-critical code, you would expect to have to learn multiple new tools (static analysis, testing techniques, security features), possibly new programming languages, and not infrequently new ways of thinking about threat models and vulnerabilities.
2. Nearly every computer user trusts tens of thousands of other programmers and millions of lines of code with their day-to-day computer security and reliability, without auditing that code themselves. Even if you have the skill required to audit the code (and very, very few people have the skill to audit all of the code that they use), you do not have the time. Therefore, our computer security is built on trust. We're trusting other programmers not to be malicious, which is obvious, but we're also trusting other programmers to be highly skilled, careful, cautious (but still fast and productive, since we quickly abandon software that isn't "actively developed"), and constantly adopting new techniques and treat models.

I think both of those principles are very widely understood and widely acknowledged. And, as we all know, both of those principles have failure modes, and those failures mean that our computers are nowhere near as secure as we would like them to be. There has been quite a lot of technical discussion about both of these principles in recent days and months, ranging from better code analysis and testing through the flaws in particular programming languages to serious questions about the trust model we use for verifying the code that we're running. I'm going to switch gears away from that discussion for a moment to talk about a social aspect. When a piece of code that one is using has a security vulnerability, it is not unusual to treat that as a trust failure. In other words, technical flaws are very quickly escalated to social flaws. This is less likely among practitioners, since we're all painfully aware of how many bugs we have in our own code, but it's still not unheard of, particularly if the authors of the code do not immediately and publicly show a socially-acceptable level of humility and contrition (in the middle of what is often a horrifically stressful experience). I think this reaction comes largely from fear. Anyone who has given it a moment's thought is painfully aware of just how much code they are running on trust, and just how many ways it could be compromised, accidentally or maliciously. And how frequently that code is compromised. The less control one has over a situation, the more terrifying it is, and the cold reality is that we have very little control over our computer security. There is a natural tendency, when afraid, to look for targets for that fear. Now, go back and think about the first principle. Anyone who has ever worked on or near free software is painfully aware that we have far more good ideas about how to improve computing and security than we have skilled resources to execute on those ideas. My own backlog of things that I've already thought about, know would be good ideas, and simply have to implement is probably longer than my remaining lifespan. I suspect that's the case for nearly all of us. In other words, we have a severe shortage of programmers who care and who have skill. We desperately need more skilled programmers who can write secure and reliable code. We may (and I think do) also need better tools, techniques, languages, protocols, and all the other machinery that we, as technical people, spend most of our time thinking about. But none of that changes the fact that we need more skilled people. In fact, it makes that need more acute: in addition to skilled people to write the code we use, we need skilled people to write the tools. Skilled people are not born. They're made. And in professions where training techniques are still in their infancy and where we don't have a good formal grasp on which techniques work, those skilled people are made primarily through apprenticeship, experimentation, and learning from failure. Worse, people who were skilled do not remain skilled without continually participating in that learning process. See the above point about a fast-changing field with evolving best practices. It's not enough to know how to write secure code to the best known practices today, or even enough to retrofit all of your existing code to current knowledge (which is often so large of an effort as to be practically impossible). You have to constantly, continually learn more, for which there is no reliable formal training. We have to try, fail, try again, and fail better. But failure that leads to a security vulnerability is treated as a loss of trust. We trusted that person to write secure code that we could use. They failed. Now we can't trust them. Based on the trust model of security, we should revoke their ability to try again and instead rely on people who have not failed, since that will make us more secure. Except now we just broke the learning process. And there's no such thing as a programmer who can stop learning. So what does that do to our resource pool? It's sadly ironic, but I believe the free software community writ large has a very serious collaboration problem: we do not tolerate each other's learning processes. This leads to a wide variety of social failures around hostile communities and the failures of meritocracy that other people have talked about at much greater length. But even if you set that all aside, it threatens our security. We need secure code, a lot of it, a lot more than we have right now. To get that code, we need people who can write it. We need to grow, encourage, and support those people and enable their learning processes. Code is written by people. If we rip people apart when they write bad, insecure code, we don't get better, secure code. We get fewer people writing security code. We get far fewer people writing security code in public, since some of the people who haven't been ripped apart will look at that experience and say, "No way am I going near that problem area. It's too scary. I don't want to end up like those programmers." Fewer people writing security code means fewer people learning how to write better security code. Fewer people capable of writing good, secure code is not a solution to any of our problems. If we do not tolerate, support, and encourage the learning process required to become a skilled programmer, or maintain one's skill in programming, we are destroying our future as a community. When you find code that is broken and badly written, you have found a problem that should be reported, analyzed, and corrected. You have also found a programmer who is about to have one of two very different types of experiences. Either they are about to learn how to become a better programmer, or they are about to be publicly shamed, humiliated, and treated as untrustworthy. Which branch they take is partly up to them, but it's also heavily influenced by how all of us react socially to the discovery of bad code. One of those branches leads to more good, secure code being written in the future. The other does not.

The Google Summer of Code 2014 programme reached another milestone recently: the accepted proposals were published, and over a thousand students were selected by nearly two hundred mentoring organisations, among them the syslog-ng project. We will be working with four students this year (twice we worked with last year), with more mentors, on a wider selection of projects. It was a tough decision to select the proposals, we received some very strong work this year.We would like to express our gratitude to both Debian, for giving us an extra slot (so we could accept four students instead of three), and Google and Carol Smith in particular, for allowing it, and putting up with our fumbling during the process.The accepted projects and students are:

• Tibor Benke: Integration with configuration management systems.

• L szl M sz ros: ZMQ transport, source and destination.

• Gy rgy Demarcsek: TLS support for the MongoDB destination.

• M ZES d m Istv n: AMQP source driver.

Good luck to all students, we're looking forward to working with you throughout the summer! Happy hacking!

Zoe slept solidly all night. She woke up a little before 6am, wanting a cuddle, but it was still dark, so she went back to sleep for another half an hour or so. It was actually nice to get that extra 30 minutes to have a leisurely wake up myself. Sarah wanted to pick up Zoe at 7:45am to get away and get a camp site, so when Zoe finally woke up for the day, we didn't muck around too much. She announced she wanted banana and oat pancakes, but we were out of bananas. I offered her the opportunity to scooter down to the Hawthorne Garage to get some if she went and got dressed. She decided that'd be good. I had a 10am appointment in the city with an intellectual property lawyer to talk about patents, so I had this grand plan of trying to squeeze in a run after Zoe was picked up and before I'd have to head into the city, so I got into my running gear, and we headed down to acquire some bananas. We whipped up the pancakes, and then after a couple of mouthfuls of hers, Zoe declared she didn't like it and wanted Cheerios instead. Oh well. It was nice to get out of the house early in the morning, and it helped get Zoe moving. Sarah ended up getting here closer to 8:30am, which made it a little too tight to go for a run, have a shower and get into the city, so I scrapped the run and pottered around at home instead for a bit, before driving into the city. My goodness casual parking in the city is exorbitant. It cost me $35 for under an hour. I got some good advice from the lawyer, so I know where to proceed from here. Next I headed down to the Valley to get my orientation at River City Labs, but had I read my email before leaving the city, I'd have discovered that the lady giving me the orientation had to leave early because her daughter was ill. It cost me $6 drive into the car park in the Valley, take the elevator down, read my email on my phone and pay my ticket and leave again. Lesson learned. I decided to do the grocery shopping that I hadn't done yesterday while I waited for Anshu to come over.

Unless you've been living under a rock, or in a tent (which would make me slightly jealous) you'll have heard about the recent heartbleed attack many times by now. The upshot of that attack is that lots of noise was made about hardening things, and there is now a new fork of openssl being developed. Many people have commented about "hardening Debian" in particular, as well as random musing on hardening software. One or two brave souls have even made noises about auditing code. Once upon a time I tried to setup a project to audit Debian software. You can still see the Debian Security Audit Project webpages if you look hard enough for them. What did I learn? There are tons of easy security bugs, but finding the hard ones is hard. (If you get bored some time just pick your favourite Editor, which will be emacs, and look how /tmp is abused during the build-process or in random libraries such as tramp [ tramp-uudecode].) These days I still poke at source code, and I still report bugs, but my enthusiasm has waned considerably. I tend to only commit to auditing a package if it is a new one I install in production, which limits my efforts considerably, but makes me feel like I'm not taking steps into the dark. It looks like I reported only three security isseus this year, and before that you have to go down to 2011 to find something I bothered to document. What would I do if I had copious free time? I wouldn't audit code. Instead I'd write test-cases for code. Many many large projects have rudimentary test-cases at best, and zero coverage at worse. I appreciate writing test-cases is hard, because lots of times it is hard to test things "for real". For example I once wrote a filesystem, using FUSE, there are some built-in unit-tests (I was pretty pleased with that, you could lauch the filesystem with a --test argument and it would invoke the unit-tests on itself. No separate steps, or source code required. If it was installed you could use it and you could test it in-situ). Beyond that I also put together a simple filesystem-stress script, which read/wrote/found random files, computes MD5 hashes of contents, etc. I've since seen similar random-filesystem-stresstest projects, and if they existed then I'd have used them. Testing filesystems is hard. I've written kernel modules that have only a single implicit test case: It compiles. (OK that's harsh, I'd usually ensure the kernel didn't die when they were inserted, and that a new node in /dev appeared ;) I've written a mail client, and beyond some trivial test-cases to prove my MIME-handling wasn't horrifically bad there are zero tests. How do you simulate all the mail that people will get, and the funky things they'll do with it? But that said I'd suggest if you're keen, if you're eager, if you want internet-points, writing test-cases/test-harnesses would be more useful than randomly auditing source code. Still what would I know, I don't even have a beard..

While researching some security-related stuff for a post I am currently writing, I found some interesting bits here and there that I though I should share, as they were new to me, and probably for many others.

DNSSEC The first thing is DNSSEC. I knew about it, of course, but never bothered to dig much into it. While reading about many interesting applications of DNS for key distribution, and thinking of ways to use them, it is clear that DNSSEC is a precondition for any of that to work. In case you don't know about it, it is an extension for the DNS service to make it safer, for example, to avoid the bad guys having you think that google.com points to sniffer.nsa.gov. Apart from these ber-cool applications I was thinking about, avoiding DNS-based attacks becomes more and more relevant these days. And I think Debian and the rest of the Free Software world should work on making this available to all end-users as easily as possible. While adoption still looks pretty low, there are some good news. First, Google claims its public DNS supports DNSSEC. Of course, you need to trust Google servers, and the path between your machine and them. But if your resolver supports DNSSEC, you can use their servers and validate the answers. On the other side, I am not too sure about their implementation, as half of the time, it would return a valid answer to a query for an invalid record: dig +dnssec sigfail.verteiltesysteme.net @8.8.8.8). Also, they have not published DNSSEC records for google.com, which seems crazy. Some packages included in Debian already take advantage of DNSSEC, if available (more on that later), but more importantly, there are a couple of DNSSEC-enabled recursive servers, including bind, unbound, and the more commonly-used dnsmasq (there is a wiki page summarising Debian's status). Sadly, the default configuration for dnsmasq does not enable DNSSEC, and most people will not use it, even if it installed, because DHCP-provided servers are usually preferred. It seems to me that it would be wise to have a package that would install dnsmasq with DNSSEC enabled, and make it the only valid resolver for the system. If you want to check if your resolver is correctly validating DNSSEC, you can use this test web page. Another good news is that many top-level domains already support DNSSEC, and in my case, Gandi.net has support in place to set it up. So I am going to look into enabling it for my own domain.

SSHFP One useful and simple advantage of using DNSSEC, is that you can store information there, and then trust it to be correct. One new DNS RR (resource record) that is useful in this context is the SSHFP RR, which allows the sysadmin of a host to publish the host SSH key fingerprint in the DNS zone. The ssh client, when enabling the VerifyHostKeyDNS option, will use that information to trust unknown hosts. One downside to this, is that either if you set the option to ask, or if your resolver does not support DNSSEC, you get the same message, which does not warn you about the extra risk. To help you create your DNS records, you can just run this command:

$ ssh-keygen -r brie.tincho.org
brie.tincho.org IN SSHFP 1 1 6ac93c63379828b5b75847bc37d8ab2b48983343
brie.tincho.org IN SSHFP 2 1 cf0d11515367e3aa7eeb37056688f11b53c8ef23

DANE, S/MIME and GPG Recently, while at FOSDEM, I attended talks that mentioned DANE. This proposed IETF standard introduces a mechanism to use DNS as a secure key distribution system, which could completely override the CA oligopoly, a very attractive proposition for many people. In short, it is very similar to the SSHFP mechanism, but it is not restricted to SSH host keys: it can be used to distribute public key information for any TLS-enabled service. So, instead (or in addition to) of having a CA sign your certificate, and relying on the chain of trust by means of having a local copy of all root CA certificates, you use the chain of trust embedded in DNSSEC to make sure that the DNS RRs you publish are valid. Then, the client application can trust the fingerprint published for the relevant service to verify that it is talking to the right server. This is a very exciting development, and I hope it gets widespread adoption. It is already supported in Postfix, there seem to be some work going on in Mozilla, as well as in Prosody which is a great start. Another exciting development of this, is the generalisation of DANE for other entities, like email addresses. There are two draft RFCs being worked on right now to deploy S/MIME and OpenPGP key material using DNSSEC. This could also change completely the way we manage the Web of Trust.

White Trash The above poster was on a bridge pylon in Flinders St in 2012. It s interesting to see what the Fringe Festival people consider to be associated with white trash . They claim homophobia is a white trash thing however lower class people have little political power and the fact that we still don t have marriage equality in Australia is clear evidence that homophobia is prevalent among powerful people. Toys vs Fairies I took the above photo at Costco in 2012. I think it s worth noting the way that the Disney Fairies (all female and marketed to a female audience) are standing around looking pretty while the Toy Story characters (mostly male and marketed to a male audience) are running out to do things. Having those items side by side on the shelf was a clear example of a trend in toys towards girls being encouraged to be passive while boys are doing things. The Toy Story pack has one female character, so it could be interpreted as being aimed at both boys and girls. But even that interpretation doesn t remove the clear gender difference. It seems ironic to me that the descriptions on the boxes are Read, Play, and Listen for the Toy Story pack and Read, Play, and Colour on the Fairies pack. Colouring is more active than listening so the pictures don t match the contents. Make Up vs Tools I took the above photo in an Aldi store in early 2013, today I was in Aldi and noticed that the same chocolate is still on sale. A clear and pointless gender difference. Rumor has it that some of the gender difference in kids clothing is so that a child can t wear the clothes of an older sibling of different gender, but chocolate only gets eaten once so there is no reason for this. Oath The above poster was inside the male toilet at Melbourne University in 2013. It would probably be good to have something like that on display all the time instead of just for one event. Locks I took the above picture early this year, it shows hundreds of padlocks attached to a bridge across the Yarra River in Melbourne. Each padlock has a message written or inscribed in it, mostly declarations of love. I first noticed this last year, I m not sure how long it s been up. There was nothing formal about this (no signs about it), people just see it and decide that they want to add to it. I guess that the council cuts some of them off periodically as the number of locks doesn t seem to be increasing much in recent times. It would be interesting to do some research into how many locks are needed to start one of these. It would also be interesting to discover whether the nature of the inscriptions determines the speed at which it takes off, would a bunch of padlocks with messages like I Love Linux inspire others as well as messages declaring love for random people? All that is required is some old locks and an engraving tool. I wonder what the social norm might be regarding messing with those locks. If I was to use those padlocks to practice the sport of lock-picking (which I learned when in Amsterdam) I wonder whether random bystanders would try to discourage me. It seems likely that picking the locks and taking them away would get a negative reaction but I wonder whether picking them one at a time and replacing them (or maybe moving them to another wire) would get a reaction. Blackface for Schoolkids A craft shop at the Highpoint shopping center in Melbourne is selling Teacher s Choice brand Multicultural Face Masks . Multicultural is a well regarded term in education, teaching children about other cultures is a good concept but can be implemented really badly. When I was in high school the subject Social Studies seemed to have an approach of look how weird people are in other places instead of teaching the kids anything useful. Sociological Images has an informative article on the Australian Hey Hey it s Saturday blackface incident in 2009 [1]. The idea of these masks seems to involve students dressing up as caricatures of other races. The mask which looks like someone s idea of a Geisha is an even bigger WTF, mixing what the package calls culture (really race) with sex work. When I visited Tokyo I got the impression that French maids fill a similar niche to Geisha for younger Japanese men and the maid cafe thing is really popular there. I think it s interesting to consider the way that a French maid costume is regarded differently to a Geisha costume. I expect that Teacher s Choice doesn t sell French maid costumes. Delicious Cow Usually meat is advertised in a way that minimises the connection to living animals. Often adverts just show cuts of meat and don t make any mention of animals and when animals are shown they are in the distance. The above picture was on the wall at a Grill d burger restaurant in Point Cook. It shows a bovine (looks like a bull even though I believe that cows are the ones that are usually eaten) with a name-tag identifying it as Delicious . The name tag personalises the animal which is an uncommon thing to do when parts of an animal are going to be eaten. Of the animals that are commonly eaten it seems that the general trend is to only show fish as complete live animals, presumably because people can identify with mammals such as cattle in a way that they can t identify with fish. Fish are also the only complete animals that are shown dead, adverts for fish that are sold as parts (EG salmon and tuna) often show complete dead fish. But I ve never seen a meat advert that shows a complete dead cow or sheep.

• [1] http://tinyurl.com/jvvamgb

Instant classic

Trusted:
NO, there were errors:
The certificate does not apply to the given host
The certificate authority's certificate is invalid
The root certificate authority's certificate is not trusted for this purpose
The certificate cannot be verified for internal reasons
Signature Algorithm: md5WithRSAEncryption
    Issuer: C=XY, ST=Snake Desert, L=Snake Town, O=Snake Oil, Ltd, OU=Certificate Authority, CN=Snake Oil CA/emailAddress=ca@snakeoil.dom
    Validity
        Not Before: Oct 21 18:21:51 1999 GMT
        Not After : Oct 20 18:21:51 2001 GMT
    Subject: C=XY, ST=Snake Desert, L=Snake Town, O=Snake Oil, Ltd, OU=Webserver Team, CN=www.snakeoil.dom/emailAddress=www@snakeoil.dom
...
            X509v3 Subject Alternative Name: 
            email:www@snakeoil.dom

For your own pleasure:

openssl s_client -connect www.walton.com.tw:443 -showcerts

or just run

echo '
-----BEGIN CERTIFICATE-----
MIIDNjCCAp+gAwIBAgIBATANBgkqhkiG9w0BAQQFADCBqTELMAkGA1UEBhMCWFkx
FTATBgNVBAgTDFNuYWtlIERlc2VydDETMBEGA1UEBxMKU25ha2UgVG93bjEXMBUG
A1UEChMOU25ha2UgT2lsLCBMdGQxHjAcBgNVBAsTFUNlcnRpZmljYXRlIEF1dGhv
cml0eTEVMBMGA1UEAxMMU25ha2UgT2lsIENBMR4wHAYJKoZIhvcNAQkBFg9jYUBz
bmFrZW9pbC5kb20wHhcNOTkxMDIxMTgyMTUxWhcNMDExMDIwMTgyMTUxWjCBpzEL
MAkGA1UEBhMCWFkxFTATBgNVBAgTDFNuYWtlIERlc2VydDETMBEGA1UEBxMKU25h
a2UgVG93bjEXMBUGA1UEChMOU25ha2UgT2lsLCBMdGQxFzAVBgNVBAsTDldlYnNl
cnZlciBUZWFtMRkwFwYDVQQDExB3d3cuc25ha2VvaWwuZG9tMR8wHQYJKoZIhvcN
AQkBFhB3d3dAc25ha2VvaWwuZG9tMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKB
gQC554Ro+VH0dJONqljPBW+C72MDNGNy9eXnzejXrczsHs3Pc92Vaat6CpIEEGue
yG29xagb1o7Gj2KRgpVYcmdx6tHd2JkFW5BcFVfWXL42PV4rf9ziYon8jWsbK2aE
+L6hCtcbxdbHOGZdSIWZJwc/1Vs70S/7ImW+Zds8YEFiAwIDAQABo24wbDAbBgNV
HREEFDASgRB3d3dAc25ha2VvaWwuZG9tMDoGCWCGSAGG+EIBDQQtFittb2Rfc3Ns
IGdlbmVyYXRlZCBjdXN0b20gc2VydmVyIGNlcnRpZmljYXRlMBEGCWCGSAGG+EIB
AQQEAwIGQDANBgkqhkiG9w0BAQQFAAOBgQB6MRsYGTXUR53/nTkRDQlBdgCcnhy3
hErfmPNl/Or5jWOmuufeIXqCvM6dK7kW/KBboui4pffIKUVafLUMdARVV6BpIGMI
5LmVFK3sgwuJ01v/90hCt4kTWoT8YHbBLtQh7PzWgJoBAY7MJmjSguYCRt91sU4K
s0dfWsdItkw4uQ==
-----END CERTIFICATE-----
'   openssl x509 -noout -text

At least they're secure against heartbleed.

I've taken over "maintaining" DVswitch from Ben Hutchings a few years ago, since Ben realized he didn't have the time anymore to work on it well. After a number of years, I have to admit that I haven't done a very good job. Not becase I didn't want to work on it, but mainly because I don't have enough time to fix DVswitch against the numerous moving targets that it uses; the APIs of libav and of liblivemedia are fluent enough that just making sure everything remains compilable and in working order is quite a job. DVswitch is used by many people; DebConf, FOSDEM, and the CCC are just a few examples, but I know of at least three more. Most of these (apart from DebConf and FOSDEM) maintain local patches which I've been wanting to merge into the upstream version of dvswitch. However, my time is limited, and over the past few years I've not been able to get dvswitch into a state where I confidently felt I could upload it into Debian unstable for a release. One step we took in order to get that closer was to remove the liblivemedia dependency (which implied removing the support for RTSP sources). Unfortunately, the resulting situation wasn't good enough yet, since libav had changed API enough that current versions of DVswitch compiled against current versions of libav will segfault if you try to do anything useful. I must admit to myself that I don't have the time and/or skill set to maintain DVswitch on an acceptable level all by myself. So, this is a call for help: If you're using DVswitch for your conference and want to continue doing so, please talk to us. The first things we'll need to do:

• Massage the code back into working order (when compiled against current libav)
• Fix my buildbot instance so that my grand plan of having nightly build/test runs against libav master actually works.
• Merge patches from the suse and CCC people that look nice
• Properly release dvswitch 0.9 (or maybe 1.0?)
• Party!

See you there?

After the whole Heartbleed fiasco, I ve decided to continue my march towards improving my online security. I d already begun the process of using LastPass to store my passwords and generate random passwords for each site, but I hadn t completed the process, with some sites still using the same passwords, and some having less than ideal strength passwords, so I spent some time today improving my password position. Here s some of the bad examples of password policy I ve discovered today. First up we have Live.com. A maximum of 16 characters from the Microsoft auth service. Seems to accept any character though. This excellent example is from creditexpert.co.uk, one of the credit agencies here in the UK. They not only restrict to 20 characters, they restrict you to @, ., _ or . So much for teaching people how to protect themselves online. Here s Tesco.com after attempting to change my password to QvHn#9#kDD%cdPAQ4&b&ACb4x%48#b . If you can figure out how this violates their rules, I d love to know. And before you ask, I tried without numbers and that still failed so it can t be the three and only three thing. The only other idea might be that they meant i.e. rather than e.g. , but I didn t test that. Edit: Here is a response from Tesco on Twitter: Here s a poor choice from ft.com, refusing to accept non-alphanumeric characters. On the plus side they did allow the full 30 characters in the password. The finest example of a poor security policy is a company who will remain nameless due to their utter lack of security. Not only did they not use HTTPS, they accepted a 30 character password and silently truncated it to 20 characters. The reason I know this is because when I logged out and tried to log in again and then used the forgot my password option, they emailed me the password in plain text. I have also been setting up two-factor authentication where possible. Most sites use the Google Authenticator application on your mobile to give you a 6 digit code to type in in addition to your password. I highly recommend you set it up too. There s a useful list of sites that implement 2FA and links to their documentation at http://twofactorauth.org/. I realise that my choice LastPass requires me to trust them, but I think the advantages outweigh the disadvantages of having many sites using the same passwords and/or low strength passwords. I know various people cleverer than me have looked into their system and failed to find any obvious flaws. Remember people, when you implement a password, allow the following things:

• Any length of password. You don t have to worry about length in your database, because when you hash the password, it will be a fixed length. You are hashing your passwords aren t you?
• Any character. The more possible characters that can be in your passwords, the harder it will be to brute force, as you are increasing the number of permutations a hacker needs to try.

If you are going to place restrictions, please make sure the documentation matches the implementation, provide a client-side implementation to match and provide quick feedback to the user, and make sure you explicitly say what is wrong with the password, rather than referring back to the incorrect documentation. There are also many JS password strength meters available to show how secure the inputted passwords are. They are possibly a better way of providing feedback about security than having arbitrary policies that actually harm your security. As someone said to me on twitter, it s not like password is too strong was ever a bad thing. The post Bad Password Policies appeared first on David Pashley.com.

We had some requests to get GHC (the Glasgow Haskell Compiler) up and running on two new Ubuntu architectures: arm64, added in 13.10, and ppc64el, added in 14.04. This has been something of a saga, and has involved rather more late-night hacking than is probably good for me. Book the First: Recalled to a life of strange build systems You might not know it from the sheer bulk of uploads I do sometimes, but I actually don't speak a word of Haskell and it's not very high up my list of things to learn. But I am a pretty experienced build engineer, and I enjoy porting things to new architectures: I'm firmly of the belief that breadth of architecture support is a good way to shake out certain categories of issues in code, that it's worth doing aggressively across an entire distribution, and that, even if you don't think you need something now, new requirements have a habit of coming along when you least expect them and you might as well be prepared in advance. Furthermore, it annoys me when we have excessive noise in our build failure and proposed-migration output and I often put bits and pieces of spare time into gardening miscellaneous problems there, and at one point there was a lot of Haskell stuff on the list and it got a bit annoying to have to keep sending patches rather than just fixing things myself, and ... well, I ended up as probably the only non-Haskell-programmer on the Debian Haskell team and found myself fixing problems there in my free time. Life is a bit weird sometimes. Bootstrapping packages on a new architecture is a bit of a black art that only a fairly small number of relatively bitter and twisted people know very much about. Doing it in Ubuntu is specifically painful because we've always forbidden direct binary uploads: all binaries have to come from a build daemon. Compilers in particular often tend to be written in the language they compile, and it's not uncommon for them to build-depend on themselves: that is, you need a previous version of the compiler to build the compiler, stretching back to the dawn of time where somebody put things together with a big magnet or something. So how do you get started on a new architecture? Well, what we do in this case is we construct a binary somehow (usually involving cross-compilation) and insert it as a build-dependency for a proper build in Launchpad. The ability to do this is restricted to a small group of Canonical employees, partly because it's very easy to make mistakes and partly because things like the classic "Reflections on Trusting Trust" are in the backs of our minds somewhere. We have an iron rule for our own sanity that the injected build-dependencies must themselves have been built from the unmodified source package in Ubuntu, although there can be source modifications further back in the chain. Fortunately, we don't need to do this very often, but it does mean that as somebody who can do it I feel an obligation to try and unblock other people where I can. As far as constructing those build-dependencies goes, sometimes we look for binaries built by other distributions (particularly Debian), and that's pretty straightforward. In this case, though, these two architectures are pretty new and the Debian ports are only just getting going, and as far as I can tell none of the other distributions with active arm64 or ppc64el ports (or trivial name variants) has got as far as porting GHC yet. Well, OK. This was somewhere around the Christmas holidays and I had some time. Muggins here cracks his knuckles and decides to have a go at bootstrapping it from scratch. It can't be that hard, right? Not to mention that it was a blocker for over 600 entries on that build failure list I mentioned, which is definitely enough to make me sit up and take notice; we'd even had the odd customer request for it. Several attempts later and I was starting to doubt my sanity, not least for trying in the first place. We ship GHC 7.6, and upgrading to 7.8 is not a project I'd like to tackle until the much more experienced Haskell folks in Debian have switched to it in unstable. The porting documentation for 7.6 has bitrotted more or less beyond usability, and the corresponding documentation for 7.8 really isn't backportable to 7.6. I tried building 7.8 for ppc64el anyway, picking that on the basis that we had quicker hardware for it and didn't seem likely to be particularly more arduous than arm64 (ho ho), and I even got to the point of having a cross-built stage2 compiler (stage1, in the cross-building case, is a GHC binary that runs on your starting architecture and generates code for your target architecture) that I could copy over to a ppc64el box and try to use as the base for a fully-native build, but it segfaulted incomprehensibly just after spawning any child process. Compilers tend to do rather a lot, especially when they're built to use GCC to generate object code, so this was a pretty serious problem, and it resisted analysis. I poked at it for a while but didn't get anywhere, and I had other things to do so declared it a write-off and gave up. Book the Second: The golden thread of progress In March, another mailing list conversation prodded me into finding a blog entry by Karel Gardas on building GHC for arm64. This was enough to be worth another look, and indeed it turned out that (with some help from Karel in private mail) I was able to cross-build a compiler that actually worked and could be used to run a fully-native build that also worked. Of course this was 7.8, since as I mentioned cross-building 7.6 is unrealistically difficult unless you're considerably more of an expert on GHC's labyrinthine build system than I am. OK, no problem, right? Getting a GHC at all is the hard bit, and 7.8 must be at least as capable as 7.6, so it should be able to build 7.6 easily enough ... Not so much. What I'd missed here was that compiler engineers generally only care very much about building the compiler with older versions of itself, and if the language in question has any kind of deprecation cycle then the compiler itself is likely to be behind on various things compared to more typical code since it has to be buildable with older versions. This means that the removal of some deprecated interfaces from 7.8 posed a problem, as did some changes in certain primops that had gained an associated compatibility layer in 7.8 but nobody had gone back to put the corresponding compatibility layer into 7.6. GHC supports running Haskell code through the C preprocessor, and there's a __GLASGOW_HASKELL__ definition with the compiler's version number, so this was just a slog tracking down changes in git and adding #ifdef-guarded code that coped with the newer compiler (remembering that stage1 will be built with 7.8 and stage2 with stage1, i.e. 7.6, from the same source tree). More inscrutably, GHC has its own packaging system called Cabal which is also used by the compiler build process to determine which subpackages to build and how to link them against each other, and some crucial subpackages weren't being built: it looked like it was stuck on picking versions from "stage0" (i.e. the initial compiler used as an input to the whole process) when it should have been building its own. Eventually I figured out that this was because GHC's use of its packaging system hadn't anticipated this case, and was selecting the higher version of the ghc package itself from stage0 rather than the version it was about to build for itself, and thus never actually tried to build most of the compiler. Editing ghc_stage1_DEPS in ghc/stage1/package-data.mk after its initial generation sorted this out. One late night building round and round in circles for a while until I had something stable, and a Debian source upload to add basic support for the architecture name (and other changes which were a bit over the top in retrospect: I didn't need to touch the embedded copy of libffi, as we build with the system one), and I was able to feed this all into Launchpad and watch the builders munch away very satisfyingly at the Haskell library stack for a while. This was all interesting, and finally all that work was actually paying off in terms of getting to watch a slew of several hundred build failures vanish from arm64 (the final count was something like 640, I think). The fly in the ointment was that ppc64el was still blocked, as the problem there wasn't building 7.6, it was getting a working 7.8. But now I really did have other much more urgent things to do, so I figured I just wouldn't get to this by release time and stuck it on the figurative shelf. Book the Third: The track of a bug Then, last Friday, I cleared out my urgent pile and thought I'd have another quick look. (I get a bit obsessive about things like this that smell of "interesting intellectual puzzle".) slyfox on the #ghc IRC channel gave me some general debugging advice and, particularly usefully, a reduced example program that I could use to debug just the process-spawning problem without having to wade through noise from running the rest of the compiler. I reproduced the same problem there, and then found that the program crashed earlier (in stg_ap_0_fast, part of the run-time system) if I compiled it with +RTS -Da -RTS. I nailed it down to a small enough region of assembly that I could see all of the assembly, the source code, and an intermediate representation or two from the compiler, and then started meditating on what makes ppc64el special. You see, the vast majority of porting bugs come down to what I might call gross properties of the architecture. You have things like whether it's 32-bit or 64-bit, big-endian or little-endian, whether char is signed or unsigned, that sort of thing. There's a big table on the Debian wiki that handily summarises most of the important ones. Sometimes you have to deal with distribution-specific things like whether GL or GLES is used; often, especially for new variants of existing architectures, you have to cope with foolish configure scripts that think they can guess certain things from the architecture name and get it wrong (assuming that powerpc* means big-endian, for instance). We often have to update config.guess and config.sub, and on ppc64el we have the additional hassle of updating libtool macros too. But I've done a lot of this stuff and I'd accounted for everything I could think of. ppc64el is actually a lot like amd64 in terms of many of these porting-relevant properties, and not even that far off arm64 which I'd just successfully ported GHC to, so I couldn't be dealing with anything particularly obvious. There was some hand-written assembly which certainly could have been problematic, but I'd carefully checked that this wasn't being used by the "unregisterised" (no specialised machine dependencies, so relatively easy to port but not well-optimised) build I was using. A problem around spawning processes suggested a problem with SIGCHLD handling, but I ruled that out by slowing down the first child process that it spawned and using strace to confirm that SIGSEGV was the first signal received. What on earth was the problem? From some painstaking gdb work, one thing I eventually noticed was that stg_ap_0_fast's local stack appeared to be being corrupted by a function call, specifically a call to the colourfully-named debugBelch. Now, when IBM's toolchain engineers were putting together ppc64el based on ppc64, they took the opportunity to fix a number of problems with their ABI: there's an OpenJDK bug with a handy list of references. One of the things I noticed there was that there were some stack allocation optimisations in the new ABI, which affected functions that don't call any vararg functions and don't call any functions that take enough parameters that some of them have to be passed on the stack rather than in registers. debugBelch takes varargs: hmm. Now, the calling code isn't quite in C as such, but in a related dialect called "Cmm", a variant of C-- (yes, minus), that GHC uses to help bridge the gap between the functional world and its code generation, and which is compiled down to C by GHC. When importing C functions into Cmm, GHC generates prototypes for them, but it doesn't do enough parsing to work out the true prototype; instead, they all just get something like extern StgFunPtr f(void);. In most architectures you can get away with this, because the arguments get passed in the usual calling convention anyway and it all works out, but on ppc64el this means that the caller doesn't generate enough stack space and then the callee tries to save its varargs onto the stack in an area that in fact belongs to the caller, and suddenly everything goes south. Things were starting to make sense. Now, debugBelch is only used in optional debugging code; but runInteractiveProcess (the function associated with the initial round of failures) takes no fewer than twelve arguments, plenty to force some of them onto the stack. I poked around the GCC patch for this ABI change a bit and determined that it only optimised away the stack allocation if it had a full prototype for all the callees, so I guessed that changing those prototypes to extern StgFunPtr f(); might work: it's still technically wrong, not least because omitting the parameter list is an obsolescent feature in C11, but it's at least just omitting information about the parameter list rather than actively lying about it. I tweaked that and ran the cross-build from scratch again. Lo and behold, suddenly I had a working compiler, and I could go through the same build-7.6-using-7.8 procedure as with arm64, much more quickly this time now that I knew what I was doing. One upstream bug, one Debian upload, and several bootstrapping builds later, and GHC was up and running on another architecture in Launchpad. Success! Epilogue There's still more to do. I gather there may be a Google Summer of Code project in Linaro to write proper native code generation for GHC on arm64: this would make things a good deal faster, but also enable GHCi (the interpreter) and Template Haskell, and thus clear quite a few more build failures. Since there's already native code generation for ppc64 in GHC, getting it going for ppc64el would probably only be a couple of days' work at this point. But these are niceties by comparison, and I'm more than happy with what I got working for 14.04. The upshot of all of this is that I may be the first non-Haskell-programmer to ever port GHC to two entirely new architectures. I'm not sure if I gain much from that personally aside from a lot of lost sleep and being considered extremely strange. It has, however, been by far the most challenging set of packages I've ported, and a fascinating trip through some odd corners of build systems and undefined behaviour that I don't normally need to touch.

I'm pondering a rewrite of my console-based mail-client. While it is "popular" it is not popular. I suspect "console-based" is the killer. I like console, and I ssh to a remote server to use it, but having different front-ends would be neat. In the world of mailpipe, etc, is there room for a graphic console client? Possibly. The limiting factor would be the lack of POP3/IMAP. Reworking things such that there is a daemon to which a GUI, or a console client, could connect seems simple. The hard part would obviously be working the IPC and writing the GUI. Any toolkit selected would rule out 40% of the audience. In other news I'm stalling on replying to emails. Irony.