Search Results: "beh"

17 April 2014

Andrew Pollock: [life] Day 79: Magic, flu shots, and play dates and dinner

Zoe slept until 7:45am this morning, which is absolutely unheard of in our house. She did wake up at about 5:15am yelling out for me because she'd kicked her doona off and lost Cowie, but went back to sleep once I sorted that out. She was super grumpy when she woke up, which I mostly attributed to being hungry, so I got breakfast into her as quickly as possible and she perked up afterwards. Today there was a free magic show at the Bulimba Library at 10:30am, so we biked down there. I really need to work on curbing Zoe's procrastination. We started trying to leave the house at 10am, and as it was, we only got there with 2 minutes to spare before the show started. Magic Glen put on a really good show. He was part comedian, part sleight of hand magician, and he did a very entertaining show. There were plenty of gags in it for the adults. Zoe started out sitting in my lap, but part way through just got up and moved closer to the front to sit with the other kids. I think she enjoyed herself. I'd have no hesitation hiring this guy for a future birthday party. Zoe had left her two stuffed toys from the car at Megan's house on Tuesday after our Port of Brisbane tour, and so after the magic show we biked to her place to retrieve them. It was close to lunch by this stage, so we stayed for lunch, and the girls had a bit of a play in the back yard while Megan's little sister napped. It was getting close to time to leave for our flu shots, so I decided to just bike directly to the doctor from Megan's place. I realised after we left that we'd still left the stuffed toys behind, but the plan was to drive back after our flu shots and have another swim their neighbour's pool, so it was all good. We got to the doctor, and waited for Sarah to arrive. Sarah and I weren't existing patients at Zoe's doctor, but we'd decided to get the flu shot as a family to try and ease the experience for Zoe. We both had to do new patient intake stuff before we had a consult with Zoe's doctor and got prescriptions for the flu shot. I popped next door to the adjacent pharmacy get the prescriptions filled, and then the nurse gave us the shots. For the last round of vaccinations that Zoe received, she needed three, and she screamed the building down at the first jab. The poor nurse was very shaken, so we've been working to try and get her to feel more at ease about this one. Zoe went first, and she took a deep breath, and she was winding up to freak out when she had her shot, but then it was all over, and she let the breath go, and looked around with a kind of "is that it?" reaction. She didn't even cry. I was so proud of her. I got my shot, and then Sarah got hers, and we had to sit in the waiting room for 10 minutes to make sure we didn't turn into pumpkins, and we were on our way. We biked home, I grabbed our swim gear, and we drove back to Megan's place. The pool ended up being quite cold. Megan didn't want to get in, and Zoe didn't last long either. Megan's Mum was working back late, so I invited Megan, her Dad and her sister over for dinner, and we headed home so I could prepare it. One of Zoe's stuffed toys had been located. We had a nice dinner of deviled sausages made in the Thermomix, and for a change I didn't have a ton of leftovers. Jason had found the other stuffed toy in his truck, so we'd finally tracked them both down. After Megan and family went home, I got Zoe to bed without much fuss, and pretty much on time. I think she should sleep well tonight.

15 April 2014

Colin Watson: Porting GHC: A Tale of Two Architectures

We had some requests to get GHC (the Glasgow Haskell Compiler) up and running on two new Ubuntu architectures: arm64, added in 13.10, and ppc64el, added in 14.04. This has been something of a saga, and has involved rather more late-night hacking than is probably good for me. Book the First: Recalled to a life of strange build systems You might not know it from the sheer bulk of uploads I do sometimes, but I actually don't speak a word of Haskell and it's not very high up my list of things to learn. But I am a pretty experienced build engineer, and I enjoy porting things to new architectures: I'm firmly of the belief that breadth of architecture support is a good way to shake out certain categories of issues in code, that it's worth doing aggressively across an entire distribution, and that, even if you don't think you need something now, new requirements have a habit of coming along when you least expect them and you might as well be prepared in advance. Furthermore, it annoys me when we have excessive noise in our build failure and proposed-migration output and I often put bits and pieces of spare time into gardening miscellaneous problems there, and at one point there was a lot of Haskell stuff on the list and it got a bit annoying to have to keep sending patches rather than just fixing things myself, and ... well, I ended up as probably the only non-Haskell-programmer on the Debian Haskell team and found myself fixing problems there in my free time. Life is a bit weird sometimes. Bootstrapping packages on a new architecture is a bit of a black art that only a fairly small number of relatively bitter and twisted people know very much about. Doing it in Ubuntu is specifically painful because we've always forbidden direct binary uploads: all binaries have to come from a build daemon. Compilers in particular often tend to be written in the language they compile, and it's not uncommon for them to build-depend on themselves: that is, you need a previous version of the compiler to build the compiler, stretching back to the dawn of time where somebody put things together with a big magnet or something. So how do you get started on a new architecture? Well, what we do in this case is we construct a binary somehow (usually involving cross-compilation) and insert it as a build-dependency for a proper build in Launchpad. The ability to do this is restricted to a small group of Canonical employees, partly because it's very easy to make mistakes and partly because things like the classic "Reflections on Trusting Trust" are in the backs of our minds somewhere. We have an iron rule for our own sanity that the injected build-dependencies must themselves have been built from the unmodified source package in Ubuntu, although there can be source modifications further back in the chain. Fortunately, we don't need to do this very often, but it does mean that as somebody who can do it I feel an obligation to try and unblock other people where I can. As far as constructing those build-dependencies goes, sometimes we look for binaries built by other distributions (particularly Debian), and that's pretty straightforward. In this case, though, these two architectures are pretty new and the Debian ports are only just getting going, and as far as I can tell none of the other distributions with active arm64 or ppc64el ports (or trivial name variants) has got as far as porting GHC yet. Well, OK. This was somewhere around the Christmas holidays and I had some time. Muggins here cracks his knuckles and decides to have a go at bootstrapping it from scratch. It can't be that hard, right? Not to mention that it was a blocker for over 600 entries on that build failure list I mentioned, which is definitely enough to make me sit up and take notice; we'd even had the odd customer request for it. Several attempts later and I was starting to doubt my sanity, not least for trying in the first place. We ship GHC 7.6, and upgrading to 7.8 is not a project I'd like to tackle until the much more experienced Haskell folks in Debian have switched to it in unstable. The porting documentation for 7.6 has bitrotted more or less beyond usability, and the corresponding documentation for 7.8 really isn't backportable to 7.6. I tried building 7.8 for ppc64el anyway, picking that on the basis that we had quicker hardware for it and didn't seem likely to be particularly more arduous than arm64 (ho ho), and I even got to the point of having a cross-built stage2 compiler (stage1, in the cross-building case, is a GHC binary that runs on your starting architecture and generates code for your target architecture) that I could copy over to a ppc64el box and try to use as the base for a fully-native build, but it segfaulted incomprehensibly just after spawning any child process. Compilers tend to do rather a lot, especially when they're built to use GCC to generate object code, so this was a pretty serious problem, and it resisted analysis. I poked at it for a while but didn't get anywhere, and I had other things to do so declared it a write-off and gave up. Book the Second: The golden thread of progress In March, another mailing list conversation prodded me into finding a blog entry by Karel Gardas on building GHC for arm64. This was enough to be worth another look, and indeed it turned out that (with some help from Karel in private mail) I was able to cross-build a compiler that actually worked and could be used to run a fully-native build that also worked. Of course this was 7.8, since as I mentioned cross-building 7.6 is unrealistically difficult unless you're considerably more of an expert on GHC's labyrinthine build system than I am. OK, no problem, right? Getting a GHC at all is the hard bit, and 7.8 must be at least as capable as 7.6, so it should be able to build 7.6 easily enough ... Not so much. What I'd missed here was that compiler engineers generally only care very much about building the compiler with older versions of itself, and if the language in question has any kind of deprecation cycle then the compiler itself is likely to be behind on various things compared to more typical code since it has to be buildable with older versions. This means that the removal of some deprecated interfaces from 7.8 posed a problem, as did some changes in certain primops that had gained an associated compatibility layer in 7.8 but nobody had gone back to put the corresponding compatibility layer into 7.6. GHC supports running Haskell code through the C preprocessor, and there's a __GLASGOW_HASKELL__ definition with the compiler's version number, so this was just a slog tracking down changes in git and adding #ifdef-guarded code that coped with the newer compiler (remembering that stage1 will be built with 7.8 and stage2 with stage1, i.e. 7.6, from the same source tree). More inscrutably, GHC has its own packaging system called Cabal which is also used by the compiler build process to determine which subpackages to build and how to link them against each other, and some crucial subpackages weren't being built: it looked like it was stuck on picking versions from "stage0" (i.e. the initial compiler used as an input to the whole process) when it should have been building its own. Eventually I figured out that this was because GHC's use of its packaging system hadn't anticipated this case, and was selecting the higher version of the ghc package itself from stage0 rather than the version it was about to build for itself, and thus never actually tried to build most of the compiler. Editing ghc_stage1_DEPS in ghc/stage1/package-data.mk after its initial generation sorted this out. One late night building round and round in circles for a while until I had something stable, and a Debian source upload to add basic support for the architecture name (and other changes which were a bit over the top in retrospect: I didn't need to touch the embedded copy of libffi, as we build with the system one), and I was able to feed this all into Launchpad and watch the builders munch away very satisfyingly at the Haskell library stack for a while. This was all interesting, and finally all that work was actually paying off in terms of getting to watch a slew of several hundred build failures vanish from arm64 (the final count was something like 640, I think). The fly in the ointment was that ppc64el was still blocked, as the problem there wasn't building 7.6, it was getting a working 7.8. But now I really did have other much more urgent things to do, so I figured I just wouldn't get to this by release time and stuck it on the figurative shelf. Book the Third: The track of a bug Then, last Friday, I cleared out my urgent pile and thought I'd have another quick look. (I get a bit obsessive about things like this that smell of "interesting intellectual puzzle".) slyfox on the #ghc IRC channel gave me some general debugging advice and, particularly usefully, a reduced example program that I could use to debug just the process-spawning problem without having to wade through noise from running the rest of the compiler. I reproduced the same problem there, and then found that the program crashed earlier (in stg_ap_0_fast, part of the run-time system) if I compiled it with +RTS -Da -RTS. I nailed it down to a small enough region of assembly that I could see all of the assembly, the source code, and an intermediate representation or two from the compiler, and then started meditating on what makes ppc64el special. You see, the vast majority of porting bugs come down to what I might call gross properties of the architecture. You have things like whether it's 32-bit or 64-bit, big-endian or little-endian, whether char is signed or unsigned, that sort of thing. There's a big table on the Debian wiki that handily summarises most of the important ones. Sometimes you have to deal with distribution-specific things like whether GL or GLES is used; often, especially for new variants of existing architectures, you have to cope with foolish configure scripts that think they can guess certain things from the architecture name and get it wrong (assuming that powerpc* means big-endian, for instance). We often have to update config.guess and config.sub, and on ppc64el we have the additional hassle of updating libtool macros too. But I've done a lot of this stuff and I'd accounted for everything I could think of. ppc64el is actually a lot like amd64 in terms of many of these porting-relevant properties, and not even that far off arm64 which I'd just successfully ported GHC to, so I couldn't be dealing with anything particularly obvious. There was some hand-written assembly which certainly could have been problematic, but I'd carefully checked that this wasn't being used by the "unregisterised" (no specialised machine dependencies, so relatively easy to port but not well-optimised) build I was using. A problem around spawning processes suggested a problem with SIGCHLD handling, but I ruled that out by slowing down the first child process that it spawned and using strace to confirm that SIGSEGV was the first signal received. What on earth was the problem? From some painstaking gdb work, one thing I eventually noticed was that stg_ap_0_fast's local stack appeared to be being corrupted by a function call, specifically a call to the colourfully-named debugBelch. Now, when IBM's toolchain engineers were putting together ppc64el based on ppc64, they took the opportunity to fix a number of problems with their ABI: there's an OpenJDK bug with a handy list of references. One of the things I noticed there was that there were some stack allocation optimisations in the new ABI, which affected functions that don't call any vararg functions and don't call any functions that take enough parameters that some of them have to be passed on the stack rather than in registers. debugBelch takes varargs: hmm. Now, the calling code isn't quite in C as such, but in a related dialect called "Cmm", a variant of C-- (yes, minus), that GHC uses to help bridge the gap between the functional world and its code generation, and which is compiled down to C by GHC. When importing C functions into Cmm, GHC generates prototypes for them, but it doesn't do enough parsing to work out the true prototype; instead, they all just get something like extern StgFunPtr f(void);. In most architectures you can get away with this, because the arguments get passed in the usual calling convention anyway and it all works out, but on ppc64el this means that the caller doesn't generate enough stack space and then the callee tries to save its varargs onto the stack in an area that in fact belongs to the caller, and suddenly everything goes south. Things were starting to make sense. Now, debugBelch is only used in optional debugging code; but runInteractiveProcess (the function associated with the initial round of failures) takes no fewer than twelve arguments, plenty to force some of them onto the stack. I poked around the GCC patch for this ABI change a bit and determined that it only optimised away the stack allocation if it had a full prototype for all the callees, so I guessed that changing those prototypes to extern StgFunPtr f(); might work: it's still technically wrong, not least because omitting the parameter list is an obsolescent feature in C11, but it's at least just omitting information about the parameter list rather than actively lying about it. I tweaked that and ran the cross-build from scratch again. Lo and behold, suddenly I had a working compiler, and I could go through the same build-7.6-using-7.8 procedure as with arm64, much more quickly this time now that I knew what I was doing. One upstream bug, one Debian upload, and several bootstrapping builds later, and GHC was up and running on another architecture in Launchpad. Success! Epilogue There's still more to do. I gather there may be a Google Summer of Code project in Linaro to write proper native code generation for GHC on arm64: this would make things a good deal faster, but also enable GHCi (the interpreter) and Template Haskell, and thus clear quite a few more build failures. Since there's already native code generation for ppc64 in GHC, getting it going for ppc64el would probably only be a couple of days' work at this point. But these are niceties by comparison, and I'm more than happy with what I got working for 14.04. The upshot of all of this is that I may be the first non-Haskell-programmer to ever port GHC to two entirely new architectures. I'm not sure if I gain much from that personally aside from a lot of lost sleep and being considered extremely strange. It has, however, been by far the most challenging set of packages I've ported, and a fascinating trip through some odd corners of build systems and undefined behaviour that I don't normally need to touch.

9 April 2014

Thorsten Glaser: Heartbleed vs. Startcom / StartSSL

First of all, good news, MirBSD is not vulnerable to The Heartbleed Bug due to my deliberate choice to stick to an older OpenSSL version. My inquiry (in various places) as to what precisely could leak when a vulnerable client connected to a n n-vulnerable server has yet to be answered, though we can assume private key material is safe. Now the bad news: while the CA I use and a CA I don t use offer free rekeying (in general), a CA I also use occasionally refuses to do that. The ugly: they will not even revoke the certificates, so any attacker who gained your key, for example when you have been using a certificate of theirs on a Debian system, will be able to use it (e.g. to MITM your visitors traffic) unless you shell over lots of unreasonable money per certificate. (Someone wrote they got the fee waived, but others don t, nor do I. (There s also a great Twitter discussion-thingy about this involving Zugschlus, but I won t link Twitter because they are not accessible to Lynx users like me and other Planet Debian authors.)
I ve been using GoDaddy privately for a while, paid for a wildcard certificate for *.mirbsd.org, and later also at work. I ve stopped using it privately due to current lack of money. Occasionally, for n n-wildcard gratis SSL certificates for HTTP servers. Startcom s StartSSL certificates are unusable for real SSL as used in SMTP STARTTLS anyway, so usage isn t much.
Now I ve got a dilemma here. I ve created a CA myself, to use with MirBSD infrastructure and things like that X.509 certificates for my hosts (especially so I can use them for SMTP) and possibly personal friends (whose PGP key I ve signed with maximum trust after the usual verification) but am using a StartSSL certificate for www.mirbsd.org as my GoDaddy wildcard certificate expires in a week or so (due to the aforementioned monetary issues), and I d rather not pay for a limited certificate only supporting a single vhost. There is absolutely no issue with that certificate and key (only ever generated and used on MirBSD, only using it in Apache mod_ssl). Then, there s this soon-to-be tax-exempt non-profit society of public utility I m working with, whose server runs Debian, and which is affected, but has been using a StartSSL certificate for a while. Neither the society nor I can afford to pay for revocation, and we do not see any possible justification for this especially in the face of CVE-2014-0160. I expect a rekey keeping the current validity end date, and would accept a revocation even if I were unable to get a new certificate, since even were we to get a certificate for the society s domain from someplace else, an attacker could still MITM us with the previous one from Startcom. The problem here is: I d really love to see (all of!) Startcom dropped from the global list of trustworthy CAs, but then I d not know from where to get a cert for MirBSD; Globalsign is not an option because I will not limit SSL compatibility to a level needed to pass their quality test possibly GoDaddy, ISTR they offer a free year to Open Source projects no idea about one for the society but it would solve the problem of not getting the certificates revoked. For everyone. I am giving Startcom time until Friday after $dayjob (for me); after that, I ll be kicking them off MirBSD s CA bundle and will be lobbying for Debian and Mozilla to do the same. Any other ideas of how to deal with that? I d probably pay 5 for a usable certificate accepted by people (including old systems, such as MSIE 5.0 on Win2k and the likes) without questioning most of the time, I only serve public content anyway and just use SSL to make the NSA s job more difficult (and even when not I m not dealing with any payment information, just the occasional login protected area). By the way, is there any way to access the information that is behind a current-day link to groups.google.com with Lynx or Pine? I can t help but praise GMane for their NNTP interface. ObFunfact: just when I was finished writing this wlog entry, I got a new eMail Special offer just for you. from GoDaddy. Sadly, no offer for a 5 SSL certificate, just the usual 20-35% off coupon code.

6 April 2014

Julien Danjou: Doing A/B testing with Apache httpd

When I started to write the landing page for The Hacker's Guide to Python, I wanted to try new things at the same time. I read about A/B testing a while ago, and I figured it was a good opportunity to test it out. A/B testing If you do not know what A/B testing is about, take a quick look at the Wikipedia page on that subject. Long story short, the idea is to serve two different version of a page to your visitors and check which one is getting the most success. When you found which version is better, you can definitely switch to it. In the case of my book, I used that technique on the pre-launch page where people were able to subscribe to the newsletter. I didn't have a lot of things I wanted to test out on that page, so I just used that approach on the subtitle, being either "Learn everything you need to build a successful Python project" or "It's time you make the most out of Python". Statistically, each version would be served half of the time, so both would get the same number of view. I then would build statistics about which page was attracting the most subscribers. With the results I would be able to switch definitively to that version of the landing page. Technical design My Web site, this Web site, is entirely static and served by Apache httpd. I didn't want to use any dynamic page, language or whatever. Mainly because I didn't want to have something else to install and maintain just for that on my server. It turns out that Apache httpd is powerful enough to implement such a feature. There are different ways to build it, and I'm going to describe my choices here. The first thing to pick is a way to balance the display of the page. You need to find a way so that if you get 100 visitors, around 50 will see the version A of your page, and around 50 will see the version B of the page. You could use a random number generator, pick a random number for each visitor, and decides which page he's going to see. But it turns out that I didn't find a way to do that with Apache httpd at first sight. My second thought was to pick the client IP address. But it's not such a good idea, because if you got visitors from, for example, people behind a company firewall, they are all going to be served the same page, so that kind of kills the statistics. Finally, I picked time based balancing: if you visit the page on a second that is even, you get version A of the page, and if you visit the page on a second that is odd, you get version B. Simple, and so far nothing proves there are more visitors on even than odd seconds, or vice-versa. The next thing is to always serve the same page to a returning visitor. I mean that if the visitor comes back later and get a different version, that's cheating. I decided the system should always serve the same page once a visitor "picked" a version. To do that, it's simple enough, you just have to use cookies to store the page the visitor has been attributed, and then use that cookie if he comes back. Implementation To do that in Apache httpd, I used the powerful mod_rewrite that is shipped with it. I put 2 files in the books directory, named either "the-hacker-guide-to-python-a.html" and "the-hacker-guide-to-python-b.html" that got served when you requested "/books/the-hacker-guide-to-python".
RewriteEngine On
RewriteBase /books

# If there's a cookie called thgtp-pre-version set,
# use its value and serve the page
RewriteCond % HTTP_COOKIE thgtp-pre-version=([^;])
RewriteRule ^the-hacker-guide-to-python$ % REQUEST_FILENAME -%1.html [L]

# No cookie yet and
RewriteCond % HTTP_COOKIE !thgtp-pre-version=([^;]+)
# the number of seconds of the time right now is even
RewriteCond % TIME_SEC [02468]$
# Then serve the page A and store "a" in a cookie
RewriteRule ^the-hacker-guide-to-python$ % REQUEST_FILENAME -a.html [cookie=thgtp-pre-version:a:julien.danjou.info,L]

# No cookie yet and
RewriteCond % HTTP_COOKIE !thgtp-pre-version=([^;]+)
# the number of seconds of the time right now is odd
RewriteCond % TIME_SEC [13579]$
# Then serve the page B and store "b" in a cookie
RewriteRule ^the-hacker-guide-to-python$ % REQUEST_FILENAME -b.html [cookie=thgtp-pre-version:b:julien.danjou.info,L]

With that few lines, it worked flawlessly. Results The results were very good, as it worked perfectly. Combined with Google Analytics, I was able to follow the score of each page. It turns out that testing this particular little piece of content of the page was, as expected, really useless. The final score didn't allow to pick any winner. Which also kind of proves that the system worked perfectly.
But it still was an interesting challenge!

4 April 2014

Gunnar Wolf: Mozilla: So our communitary echobox *does* resound with social issues

I woke up to the news that, after a very short tenure, Brendan Eich steps down as the Mozilla CEO. Why? Because of the community outcry. Because some years ago, Eich pubilcly supported (and donated funds) the ban of any kind of marriages in California that were not between a man and a woman. The world has advanced enormously in this regard in the last years/decades, and so many individuals and organizations opposed and announced they would boycott Mozilla that either him or Mozilla could not stand the pressure anymore. So, of course, it's sad the person had to resign. Many people talked about freedom of speech, freedom of harbouring his own personal opinion But when it comes to the rights of minorities, particularly of minorities that have suffered such hard prejudice and abuse as the gay, lesbian and all the other non-orthodox sexual- and gender- orientations, righting a wrong is much more important than preserving an individual's freedom of opinion. Besides, it's not just thinking or talking about something The concrete proposition Eich supported (and eventually made him resign) is about bringing the life of thousands of people to a hellish state of uncertainty, and going back to not having a way for the society to legally recognize their way of being, their love, their lifes. But anyway What prompts me into writing this is that, once again, the Free Software (and related denominations) community has shown that a set of core values, seemingly shared by a very large amount of our own people with no coordination or correlation with what conforms us as a community (and thus, being emergent traits), are strong enough to create a critical mass, to achieve cohesion. And that ours is not just a technical community of people writing software at all layers of the stack, but first and foremost is a group of social activists, committed to making the world better. I will quote from Matthew Garrett's post on this topic, clearly more contundent and thorough that what I'm trying to come up with:
The Mozilla Manifesto discusses individual liberty in the context of use of the internet, not in a wider social context. Brendan's appointment was very much in line with the explicit aims of both the Foundation and the Corporation - whatever his views on marriage equality, nobody has seriously argued about his commitment to improving internet freedom. So, from that perspective, he should have been a fine choice. But that ignores the effect on the wider community. People don't attach themselves to communities merely because of explicitly stated goals - they do so because they feel that the community is aligned with their overall aims. The Mozilla community is one of the most diverse in free software, at least in part because Mozilla's stated goals and behaviour are fairly inspirational. People who identify themselves with other movements backing individual liberties are likely to identify with Mozilla. So, unsurprisingly, there's a large number of socially progressive individuals (LGBT or otherwise) in the Mozilla community, both inside and outside the Corporation. A CEO who's donated money to strip rights from a set of humans will not be trusted by many who believe that all humans should have those rights. It's not just limited to individuals directly affected by his actions - if someone's shown that they're willing to strip rights from another minority for political or religious reasons, what's to stop them attempting to do the same to you? Even if you personally feel safe, do you trust someone who's willing to do that to your friends? In a community that's made up of many who are either LGBT or identify themselves as allies, that loss of trust is inevitably going to cause community discomfort.

3 April 2014

Matthew Garrett: Mozilla and leadership

A post I wrote back in 2012 got linked from a couple of the discussions relating to Brendan Eich being appointed Mozilla CEO. The tldr version is "If members of your community doesn't trust their leader socially, the leader's technical competence is irrelevant". That seems to have played out here.

In terms of background[1]: in 2008, Brendan donated money to the campaign for Proposition 8, a Californian constitutional amendment that expressly defined marriage as being between one man and one woman[2]. Both before and after that he had donated money to a variety of politicians who shared many political positions, including the definition of marriage as being between one man and one woman[3].

Mozilla is an interesting organisation. It consists of the for-profit Mozilla Corporation, which is wholly owned by the non-profit Mozilla Foundation. The Corporation's bylaws require it to work to further the Foundation's goals, and any profit is reinvested in Mozilla. Mozilla developers are employed by the Corporation rather than the Foundation, and as such the CEO is responsible for ensuring that those developers are able to achieve those goals.

The Mozilla Manifesto discusses individual liberty in the context of use of the internet, not in a wider social context. Brendan's appointment was very much in line with the explicit aims of both the Foundation and the Corporation - whatever his views on marriage equality, nobody has seriously argued about his commitment to improving internet freedom. So, from that perspective, he should have been a fine choice.

But that ignores the effect on the wider community. People don't attach themselves to communities merely because of explicitly stated goals - they do so because they feel that the community is aligned with their overall aims. The Mozilla community is one of the most diverse in free software, at least in part because Mozilla's stated goals and behaviour are fairly inspirational. People who identify themselves with other movements backing individual liberties are likely to identify with Mozilla. So, unsurprisingly, there's a large number of socially progressive individuals (LGBT or otherwise) in the Mozilla community, both inside and outside the Corporation.

A CEO who's donated money to strip rights[4] from a set of humans will not be trusted by many who believe that all humans should have those rights. It's not just limited to individuals directly affected by his actions - if someone's shown that they're willing to strip rights from another minority for political or religious reasons, what's to stop them attempting to do the same to you? Even if you personally feel safe, do you trust someone who's willing to do that to your friends? In a community that's made up of many who are either LGBT or identify themselves as allies, that loss of trust is inevitably going to cause community discomfort.

The first role of a leader should be to manage that. Instead, in the first few days of Brendan's leadership, we heard nothing of substance - at best, an apology for pain being caused rather than an apology for the act that caused the pain. And then there was an interview which demonstrated remarkable tone deafness. He made no attempt to alleviate the concerns of the community. There were repeated non-sequiturs about Indonesia. It sounded like he had no idea at all why the community that he was now leading was unhappy.

And, today, he resigned. It's easy to get into hypotheticals - could he have compromised his principles for the sake of Mozilla? Would an initial discussion of the distinction between the goals of members of the Mozilla community and the goals of Mozilla itself have made this more palatable? If the board had known this would happen, would they have made the same choice - and if they didn't know, why not?

But that's not the real point. The point is that the community didn't trust Brendan, and Brendan chose to leave rather than do further harm to the community. Trustworthy leadership is important. Communities should reflect on whether their leadership reflects not only their beliefs, but the beliefs of those that they would like to join the community. Fail to do so and you'll drive them away instead.

[1] For people who've been living under a rock
[2] Proposition 8 itself was a response to an ongoing court case that, at the point of Proposition 8 being proposed, appeared likely to support the overturning of Proposition 22, an earlier Californian ballot measure that legally (rather than constitutionally) defined marriage as being between one man and one woman. Proposition 22 was overturned, and for a few months before Proposition 8 passed, gay marriage was legal in California.
[3] http://www.theguardian.com/technology/2014/apr/02/controversial-mozilla-ceo-made-donations-right-wing-candidates-brendan-eich
[4] Brendan made a donation on October 25th, 2008. This postdates the overturning of Proposition 22, and as such gay marriage was legal in California at the time of this donation. Donating to Proposition 8 at that point was not about supporting the status quo, it was about changing the constitution to forbid something that courts had found was protected by the state constitution.

comment count unavailable comments

1 April 2014

Petter Reinholdtsen: ReactOS Windows clone - nice free software

Microsoft have announced that Windows XP reaches its end of life 2014-04-08, in 7 days. But there are heaps of machines still running Windows XP, and depending on Windows XP to run their applications, and upgrading will be expensive, both when it comes to money and when it comes to the amount of effort needed to migrate from Windows XP to a new operating system. Some obvious options (buy new a Windows machine, buy a MacOSX machine, install Linux on the existing machine) are already well known and covered elsewhere. Most of them involve leaving the user applications installed on Windows XP behind and trying out replacements or updated versions. In this blog post I want to mention one strange bird that allow people to keep the hardware and the existing Windows XP applications and run them on a free software operating system that is Windows XP compatible. ReactOS is a free software operating system (GNU GPL licensed) working on providing a operating system that is binary compatible with Windows, able to run windows programs directly and to use Windows drivers for hardware directly. The project goal is for Windows user to keep their existing machines, drivers and software, and gain the advantages from user a operating system without usage limitations caused by non-free licensing. It is a Windows clone running directly on the hardware, so quite different from the approach taken by the Wine project, which make it possible to run Windows binaries on Linux. The ReactOS project share code with the Wine project, so most shared libraries available on Windows are already implemented already. There is also a software manager like the one we are used to on Linux, allowing the user to install free software applications with a simple click directly from the Internet. Check out the screen shots on the project web site for an idea what it look like (it looks just like Windows before metro). I do not use ReactOS myself, preferring Linux and Unix like operating systems. I've tested it, and it work fine in a virt-manager virtual machine. The browser, minesweeper, notepad etc is working fine as far as I can tell. Unfortunately, my main test application is the software included on a CD with the Lego Mindstorms NXT, which seem to install just fine from CD but fail to leave any binaries on the disk after the installation. So no luck with that test software. No idea why, but hope someone else figure out and fix the problem. I've tried the ReactOS Live ISO on a physical machine, and it seemed to work just fine. If you like Windows and want to keep running your old Windows binaries, check it out by downloading the installation CD, the live CD or the preinstalled virtual machine image.

Joey Hess: adding docker support to propellor

Propellor development is churning away! (And leaving no few puns in its wake..) Now it supports secure handling of private data like passwords (only the host that owns it can see it), and fully end-to-end secured deployment via gpg signed and verified commits. And, I've just gotten support for Docker to build. Probably not quite work, but it should only be a few bugs away at this point. Here's how to deploy a dockerized webserver with propellor:
host hostname@"clam.kitenet.net" = Just
    [ Docker.configured
    , File.dirExists "/var/www"
    , Docker.hasContainer hostname "webserver" container
    ]
container _ "webserver" = Just $ Docker.containerFromImage "joeyh/debian-unstable"
        [ Docker.publish "80:80"
        , Docker.volume "/var/www:/var/www"
        , Docker.inside
            [ serviceRunning "apache2"
                 requires  Apt.installed ["apache2"]
            ]
        ]
Docker containers are set up using Properties too, just like regular hosts, but their Properties are run inside the container. That means that, if I change the web server port above, Propellor will notice the container config is out of date, and stop the container, commit an image based on it, and quickly use that to bring up a new container with the new configuration. If I change the web server to say, lighttpd, Propellor will run inside the container, and notice that it needs to install lighttpd to satisfy the new property, and so will update the container without needing to take it down. Adding all this behavior took only 253 lines of code, and none of it impacts the core of Propellor at all; it's all in Propellor.Property.Docker. (Well, I did need another hundred lines to write a daemon that runs inside the container and reads commands to run over a named pipe... Docker makes running ad-hoc commands inside a container a PITA.) So, I think that this vindicates the approach of making the configuration of Propellor be a list of Properties, which can be constructed by abitrarily interesting Haskell code. I didn't design Propellor to support containers, but it was easy to find a way to express them as shown above. Compare that with how Puppet supports Docker: http://docs.docker.io/en/latest/use/puppet/
docker::run   'helloworld':
  image        => 'ubuntu',
  command      => '/bin/sh -c "while true; do echo hello world; sleep 1; done"',
  ports        => ['4444', '4555'],
...
All puppet manages is running the image and a simple static command inside it. All the complexities that puppet provides for configuring servers cannot easily be brought to bear inside the container, and a large reason for that is, I think, that its configuration file is just not expressive enough.

29 March 2014

Dirk Eddelbuettel: R / Finance 2014 Open for Registration

The annoucement below just went to the R-SIG-Finance list. More information is as usual at the R / Finance page:
Now open for registrations: R / Finance 2014: Applied Finance with R
May 16 and 17, 2014
Chicago, IL, USA
The registration for R/Finance 2014 -- which will take place May 16 and 17 in Chicago -- is now open! Building on the success of the previous conferences in 2009, 2010, 2011, 2012 and 2013, we expect around 300 attendees from around the world. R users from industry, academia, and government will joining 30+ presenters covering all areas of finance with R. We are very excited about the four keynotes by Bill Cleveland, Alexios Ghalanos, Bob McDonald and Luke Tierney. The main agenda (currently) includes sixteen full presentations and twenty-one shorter "lightning talks". We are also excited to offer four optional pre-conference seminars on Friday morning. To celebrate the sixth year of the conference in style, the dinner will be returning to The Terrace of the Trump Hotel. Overlooking the Chicago River and skyline, it is a perfect venue to continue conversations while dining and drinking. More details of the agenda are available at:
http://www.RinFinance.com/agenda/
Registration information is available at
http://www.RinFinance.com/register/
and can also be directly accessed by going to
http://www.regonline.com/RFinance2014
We would to thank our 2014 Sponsors for the continued support enabling us to host such an exciting conference:
International Center for Futures and Derivatives at UIC Revolution Analytics
MS-Computational Finance at University of Washington OneMarketData
RStudio
On behalf of the committee and sponsors, we look forward to seeing you in Chicago!
Gib Bassett, Peter Carl, Dirk Eddelbuettel, Brian Peterson,
Dale Rosenthal, Jeffrey Ryan, Joshua Ulrich
See you in Chicago in May!

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

28 March 2014

Matthias Klumpp: Appstream: The next step

appstream-logoWith the release of GNOME-Software 3.12, it s also time for another update about what is going on behind the scenes of Appstream, the project providing all the data needed to make software-centers work. So here is the long overdue update :) If you have attended my FOSDEM talk or seen the slides, you know about the concept of component metadata to describe (almost) all software components which make up a Linux system, as well as their public interfaces they provide for users and other software to access. This metadata specification was originally designed as part of the Listaller project for use with the 3rd-party software installer.
But we already have the Appstream specification, which describes applications available on a distribution. And technically, applications are nothing but specialized components.
When I was asked about extending the Appstream spec with fonts and codecs, I decided to give up the Listaller components concept and merge it with the Appstream project, effectively creating a single source of metadata for all software in distributions. Appstream and components were separate, because Appstream was designed just for application metadata shown in software-centers, while component-metadata was more technical and not user-visible. This separation, however, did not reflect reality. Software centers want to show things components were designed for (codecs, fonts, ), and application meta-information also can contain technical items not shown in a software-center, but useful for other functionality (e.g. for mimehandler finding). So, if you like clear naming of things (like I do), you can now think of the upcoming Appstream release 0.6 as Appstream not being a stream of application metadata, but rather metadata provided for use in applications. ;-) Extending Appstream with components has a number of benefits:
First of all, users will soon get systems which are able to automatically install missing firmware, codecs, fonts and even libraries or Python-modules. This happens in a distro-agnostic way, so any application can request installations of missing components through Appstream and PackageKit without having to care about distribution details. For distributors, you will get more information about the software upstream provides: Which public interfaces does it make available? What is the upstream s homepage? Summary? Short release-descriptions? Exposed library symbols? etc. This data was available before, of course, but we made it machine-readable and standardized now. This will also allow us to match software across distributions quickly, allowing efficient exchange of patches between distributions, and answering questions upstream has like in which distribuions is my software available, and in which version? . Tracking upstream software is also easier now. Last but not least, the metadata format gives upstreams a small say in how their software gets packaged (= how it gets split / if there is a metapackage depending on the split parts). For upstream authors, it is much easier to maintain just a few Appstream XML metafiles, instead of adding a Listaller component file as well (which was not XML). Most data for creating the metainfo files is already available, some can even be filled in automatically by the build-system. With all the advantages this new system has comes a small disadvantage: These changes broke the backwards-compatibility of the Appstream spec, so to use Appstream data in version 0.6 or higher, you need to adjust your parsers. But of course you are using a library to access the data anyway, which already does that for you ;-) GNOME-Software 3.12 has almost complete support for the new (but still not released) Appstream 0.6 spec through it s libappstream-glib library. The libappstream library in Appstream s Git branch will get 0.6 support as well very soon. The only thing you need to do is adjust your code for the new library version, which broke it s API as well due to a rewrite in C (I hope that we can provide language bindings again very soon).
Soon, there will also be a Qt library for accessing Appstream, since some people expressed a demand for it (and working with GObject in Qt C++ code always feels a litle bit odd).
The 0.6 specification is still being fine-tuned to iron out some issues which might come up and mainly to improve the documentation and adjust all tools. It will be released in 2-3 weeks, and bring not only components support, but also lots of other small improvements and additions for better description of applications/components. In some cases, tag names have changed to have less ambiguous names. Thanks to Richard Hughes for helping with the spec our discussions about the Appstream spec are highly productive and in the end lead to a better specification. Also, the new 0.6 specification includes many extensions used in GNOME-Software by default now, which makes it easier for other distributions to provide them for GS. The current draft for the Appstream 0.6 specifications can be found here. Some details might still change before the final release. As soon as the spec is final, there will be a ML announcement for distributors as well (watch the XDG distributions list), so there is enough time for distributors to future-proof their Appstream distro-data generators (I will also describe the changes more detailed then). If you are confused now about the whole components stuff: Fear not! ;-) I will do some more detailed posts in a few weeks explaining upstream projects how to provide metadata properly (after the 0.6 release). I will also finish the wiki page for Appstream AppData in KDE projects, where we can then use 0.6-style data in KDE right away.

27 March 2014

Mario Lang: Demonstrating a common m17n a11y problem with a web audio game

There is a rather infamous multilingualisation bug in almost every screen reader I have ever used. As a german native speaker, I usually set my operating system user interface language to german. However, since I do a lot of technical work, I frequently end up on english websites. But if I temporarily switch my speech synthesis language to english, certain user interface elements and key names are not translated! So the screen reader continues to say "Eingabe" or "Leertaste", but with an english accent! I consider this behaviour hilarious, especially in 21st century. To demonstrate how hilarious it actually is, I have made a little browser audio game. Voice snippets have been contributed by my girlfriend. A natural voice is still a lot better than sampled speech synthesizers, and sounds more fun as we think. The idea is simple: Angela will announce a key name, and you are supposed to hit it on the keyboard as fast as possible. After a round of 50 guesses (time to answer will decrease with every guess) a final score will be announced. I usually don't write any JavaScript. I still managed to hack this up (with a little help from Simon) in roughly 2 hours of browsing Stackoverflow. The game is here. The code is on GitHub. If you have any problems getting this to work, please let me know or send a pull request. I'd like to get it working at least on iOS as well (after all, it is the touch device platform with best accessibility support currently), however, I could not get mobile safari to behave. So patches to enable use on touch devices are welcome. It should be simple, just add a few buttons which call the pressed function with the appropriate character as parameter. However, I tried, and something is fishy regarding window.setTimeout since it does not behave the same way as if you use it with Firefox. Oh, and my favourite key name is "Runt Glamour Zoo" :-) Now, this was funny. But it is also very sad. If you play this game, remember that blind and visually impaired computer users really have to cope with such accented pronounciations on a daily basis. We grew used to it, but if I take a step backward, it's really unacceptable that we still have to cope with this daily. And this is the simple part of the bug, since it is only dealing with key labels, which are internal to the system. It gets much worse if you want your screen reader to automatically switch to the correct synthesis language. Language detection is apparently not good enough to be implemented, at least that's what they tell me. And if the document (HTML) specifies a language, it's wrong in more than 50% of the cases. This is a disaster. Come to think of it, that is probably a reason why I prefer braille over speech. With braille, this problem goes away completely.

Andrew Pollock: [life] Day 58: Playgroup, rain and errands

My yoga teacher was out sick this morning. I had grand plans of instead biking to the pool and going for a swim, but when my alarm went off, and the weather outside was grey and miserable, having an extra half an hour lie in seemed more attractive. I think I made the right choice, because I felt like a million bucks today. I did all of the preparation to bake a batch of carrot and kale muffins before Zoe arrived, and we baked a batch as soon as she arrived and had them out of the oven in time to drive to Playgroup. I was expecting a larger crowd today on account of the wet weather, but it turned out quite the opposite. That said, Zoe still had a good time. There's really no other kids her age though, so it's still winding up as a "play with Dad in a different environment". My ABN came through yesterday, so after Playgroup we walked to the bank to give it to them and sort out a business credit card. Zoe was super well behaved while I did that, so we grabbed a fresh hot cross bun from Brumby's across the street afterwards. I needed some more stamps, so I figured we could just walk down to the post office down the other end of Oxford Street while we were there. That was slow going, but Zoe was enjoying walking with her umbrella in the rain. We got to the post office, and I discovered that postage is going up to 70 cents next week, and they couldn't sell be 70 cent stamps yet, and buying 60 cent stamps would be pointless after Monday, so I left empty handed. Zoe was eyeing off the umbrellas they had for sale in the post office. No sooner had we walked out of the post office and Zoe managed to walk all over her umbrella and totally destroy it. If I hadn't seen it happen, I'd have said she did it deliberately to get a new umbrella, but it really was an accident, so we had to turn around and buy one of the umbrellas from the post office. We made our way back down Oxford Street, and stopped in the boutique toy shop there. Zoe was particularly enthralled by the musical jewelery boxes, and really wanted one. I negotiated with her for it to be a birthday present, and got it gift wrapped. She seemed fine with the idea of not being able to have it until her birthday. We then made our way back to the car and drove home for a rather late lunch. I'm glad Zoe had the hot cross bun after the bank to keep her going, as she didn't seem to mind the late lunch at all. I made fritters with some left over corned beef, and then after we'd had lunch I thought we might as well get out of the house. Zoe's Kindergarten has a working bee on Saturday (I'm expecting it'll get canceled due to the wet weather, though). Incidentally, I have no idea where the "bee" in "working bee" comes from. Megan's Dad is Welsh, and I was lost for words when it came to explaining what a working bee was. Anyway, I wanted to get some gardening gloves, so we went to Bunnings. I managed to find some cute little kids ones as well, so Zoe can help. After Bunnings, we went to the pet shop to get some more cat litter. The pet shop seems to be a great source of entertainment for Zoe. She absolutely loved playing around with the hutches they had on display, and checking out all the fish, and the aquarium accessories. We eventually made it out of the pet shop, and we went around to the adjacent shopping mall in search of some craft supplies at some of the cheap shops there. It turns out the cheap shops are also a great source of entertainment. By the time we were done there, it was time to get home so Sarah could pick Zoe up, so we headed home. There was enough time for Zoe to watch a little bit of TV before Sarah arrived. So despite the weather, and no real plan for the day, we managed to completely fill the day, and Zoe had a great time. She seemed no worse for wear for powering through without a nap. There was a few small tantrums at Bunnings, but that mostly revolved around her tipping over her miniature shopping trolley.

26 March 2014

Wouter Verhelst: LDAP and Kerberos authentication

Kerberos is a great protocol for single sign-on authentication. It's supported by many protocols, allowing you to not have to enter a password to each and every one of them; instead, the protocols behind the scenes (not "just" Kerberos, but also the things that embed tickets, such as GSSAPI, or the things that embed GSSAPI, such as SASL or SPNEGO) use your ticket-granting-ticket to ask for security credentials for the service you're trying to use, magic happens, and you're authenticated. I blogged about kerberos before (even if it was ages ago); since then, I've not only used it on my own systems, but also on the systems of various customers. One thing I've learned in that time, however, is that most web application developers have a bad case of NIHilism when it comes to authentication. Most webservers that I've seen have a wide range of methods to do authentication in the webserver through various means, including things like certificate-based authentication, one-time password modules, and, yes, kerberos. Yet almost no webapp out there will look at the magic variables that those webservers set to explain we're authenticated, instead reinventing the wheel through webforms and other various stupid means. Sigh. So, that means, no kerberos authentication for webapps. Worse, if the application has no way to pass on authentication to something external, that means users will now have to learn another password: one for Kerberos, one for the webapp. And, probably, one for this other webapp, too -- because once you add one webapp, people expect you to add more of them. Well, mostly. In some cases, webapps do have ways to externalize authentication. In most cases this means "store passwords in a database", or "try authenticating against this other service here". When "this other service here" is an IMAP server, then all you need to do is make sure cleartext authentication on the IMAP server eventually ends up trying to authenticate against the Kerberos server, and you're all set. When "this other service here" is an LDAP server, however, you're out of luck. Right? It turns out that no, you're not. I recently learned that OpenLDAP can, in fact, check "simple" bind requests by checking some other service, and that this other service can be a Kerberos realm. Doing so is called "Pass-through authentication" in the OpenLDAP documentation, and this is how you do it:

mech_list: plain
pwcheck_method: saslauthd

This directs the SASL libraries, loaded by slapd, to talk to saslauthd when trying to authenticate some SASL things - Make sure slapd has the correct permissions to access the saslauthd unix domain socket. On Debian, that means you need to add the "openldap" user to the "sasl" group:
 adduser openldap sasl 
(obviously that won't be active until the next slapd restart) And that's it; if you now try a simple bind against the LDAP directory, and enter your Kerberos password, you should be in. If it doesn't work, try running "testsaslauthd"; if that works, it means the error is in your slapd configuration. If it doesn't, then the problem is in saslauthd. Some notes: Oh, and if you're a webapp developer: please please please make it easy for me to use an external authentication mechanism. This isn't hard; all you need is a separate page that will read the magic variables (probably REMOTE_USER if you're using apache), set a session variable or whatever, and then redirect to your normal "we've just logged in" page. ikiwiki gets this right; you can too! (for added bonus points, have some way to declare a mapping from REMOTE_USER values to internal, readable, usernames, but it's not too crucial)

24 March 2014

Elena 'valhalla' Grandi: Moving away from google services, step N out of M

For years, I've used a gmail account to subscribe to public mailing lists because it was convenient, it had excellent spam filtering and since the lists were already available on google-indexed public archive I felt that there was no real privacy issue.
I already have a local copy of every message in the account, in case I lose access to it for some random reason, and my real contacts already know and use my other email address(es), so I'm not really worried about continuing to use gmail for this kind of traffic.

When I started contributing to Debian I also used that email address, because I was already subscribed to a few mailing lists under it, and it was going to end up being used on the BTS and other spam-attracting public places like that, but I was less happy with the choice.

Lately, however, gmail's spam filters have started to behave oddly, with lots of false positives including a number of messages from @debian.org addresses, and this basically removes the main reason why I was using gmail in the first place, so I'm seriously considering dropping it also for publicly indicized communications.

My first step, right before leaving for Barcelona was to create a couple other email addresses, one for mailing lists and the other for debian-related work, and add them to my gpg/OpenPGP key, so that any new signature on them from the conf would also include the new UIDs.

If you've received a badly signed message (thanks heirloom mailix for mangling stuff :) ) followed by an hopefully correctly signed one where I was asking you to sign new UIDs, and you were wondering, this is the reason.

Now that the new UIDs have been signed by a few DDs I'm going to start using the new address in Debian: I believe I have changed it everywhere I needed to, except for the maintainer/uploader field for my packages, where it will change on their next uploads.

As for mailing lists, I still don't know how to proceed: I could proceed in a systematic way, but I'm lazy and I will probably just change a few main subscriptions, use the new address for new subscriptions and then migrate the rest a few at a time, unless some other event happens and I'll have to hurriedly migrate everything or risk losing important messages. Even in the latter case, I don't need any information that is only stored on google's servers, so I won't will panic just a bit :)

Now, where did I put that documentation about xmpp servers?

23 March 2014

Andrew Shadura: ifupdown and command status handling

In 2011, I first tried to fix a Debian bug #547587. The bug was about hook script result codes not being checked, so if a script fails, this isn t detected. Unfortunately, just checking the return code wasn t enough, as lots of scripts didn t care about their return code at all, so I had to unapply the patch. Now, after 2.5 years, I think it s time to try again, as most of the scripts have been fixed since then. At the same time, I ve changed error handling a little bit further: errors in any commands or scripts during interface configuration are considered fatal, but when interface is deconfigured, errors are ignored. However, this may change break configurations which depend on previous behaviour. Please report bugs if you find any.

22 March 2014

Steve Kemp: That is it, I'm going to do it

That's it, I'm going to do it: I have now committed myself to writing a scalable, caching, reverse HTTP proxy. The biggest question right now is implementation language; obviously "threading" of some kind is required so it is a choice between Perl's anyevent, Python's twisted, Rubys event machine, or node.js. I'm absolutely, definitely, not going to use C, or C++. Writing a a reverse proxy in node.js is almost trivial, the hard part will be working out which language to express the caching behaviour, on a per type, and per-resource basis. I will ponder.

21 March 2014

Andrew Pollock: [life] Day 52, Trial Ballet class, rain and paint, Science Friday

I woke up at 2am, and then spent over an hour trying to get back to sleep. That was phenomenally annoying. Then at 5am, Zoe's white noise app on her Nexus 7 decided to go a bit wonky (as it does from time to time), which caused her to wake up. I couldn't convince her tablet to behave, so I just let her get in bed with me, but neither of us went back to sleep. I think at about 6am I told her she could turn on the TV so I could try and squeeze in a bit more rest. I managed to forget to put the oats in the Thermomix when I made porridge this morning. It took me a while to figure out why the consistency and colour didn't look quite right when I dished it up. Yep, it was one of those mornings. Zoe had her trial ballet class this morning, at 9:45am in Coorparoo. I managed to get Zoe organised a bit early, so we were the first ones there. It's in a church hall, and has a nice floor, with a mirrored wall and barres on each long wall. Eventually the other girls arrived, there were 5 regulars in total. A good sized class. I had mixed feelings when I discovered that parents weren't normally able to be present in the dance studio to watch the classes. I was today, because it was a trial, but once Zoe starts regular classes, I'll have to wait outside. I'm disappointed I won't be able to watch, but it'll be nice to have 45 minutes to myself. Zoe did really well in the class today. She picked everything up very quickly, and I don't think she'll be negatively impacted by having entered in Term 2. She seemed very comfortable and looked like she enjoyed herself. Not long after we arrived, the heavens opened in a massive downpour, which continued when we had to leave. It was a fair dash back to the car. I'd had plans of a picnic lunch in New Farm Park, but scrapped them on account of the weather. We got home, and had a late morning tea, and then Zoe wanted to play with her Marble Run. After that, we gave Smudge a good brushing and then Zoe wanted to do some craft and painting. I let Zoe set herself up, and it was fun to see what bits and pieces she chose to dig out of the arts and craft bins. She wanted to make another crown for Mummy's boss, but we don't have any crown making stuff at home (yet), so she ended up starting to decorate a wand and gluing a pom pom on a shoe box, before moving on to painting her cardboard box cubby house and rocket. She wanted to start doing some hand painting right around lunch time, but I managed to convince her to do that after lunch. We had some lunch, and then she did a little bit more painting and stickering before her nap. She passed out for her nap very quickly. After her nap, I asked her if she wanted to paint or do science, and without hesitation, she chose science! I delved into the "Weather" chapter of 365 Science Experiments and we did a couple with ice cubes and salt and then she saw one where you stick a balloon over the neck of a glass bottle and put it in hot water and the balloon "inflates", so we did that one as well. We also popped a balloon to simulate thunder. It was all nice and easy, without a lot of preparation required. After that, we did a bit of a clean up and then Zoe watched some of her DVDs from the library while she waited for Sarah to arrive to pick her up. It was a good day today. No meltdowns.

18 March 2014

Vincent Bernat: EDNS client subnet support for BIND

To provide geolocation-aware answers with BIND, a common solution is to use a patch adding GeoIP support. A client can be directed to the closest (and hopefully fastest) web server:
view "FRANCE"  
     match-clients   geoip_cityDB_country_FR;  ;
     zone "example.com" in  
         type master;
         file "france.example.com.dns";
      ;
 ;
view "GERMANY"  
     match-clients   geoip_cityDB_country_DE;  ;
     zone "example.com" in  
         type master;
         file "germany.example.com.dns";
      ;
 ;
/* [...] */
view "DEFAULT"  
    zone "example.com" in  
        type master;
        file "example.com.dns";
     ;
 ;
However, an end user does not usually talk directly to authoritative servers. They proxy the query to a third-party recursor server which will query the authoritative server on their behalf. The recursor also caches the answer to be able to serve it directly to other clients. On most cases, we can still rely on the recursor GeoIP location to forward the client to the closest web server because it is located in the client s ISP network, as shown on the following schema: Query for www.example.com through an ISP recursor
  1. Juan is located in China and wants to know the IP address of www.example.com. She queries her ISP resolver.
  2. The resolver asks the authoritative server for the answer.
  3. Because the IP address of the resolver is located in China, the authoritative server decides to answer with the IP address of the web server located in Japan which is the closest one.
  4. Juan can now enjoy short round-trips with the web server.
However, this is not the case when using a public recursor as provided by Google or OpenDNS. In this case, the IP address of the end client and the source IP address of the recursor may not share the same locality. For example, in the following schema, the authoritative server now thinks it is in relation with an European customer and answers with the IP address of the web server located in Europe: Query for www.example.com through an open recursor Moreover, caching makes the problem worse. To solve this problem, a new EDNS extension to expose the client subnet has been proposed. When using this extension, the recursor will provide the client subnet to the authoritative server for it to build an optimized reply. The subnet is vague enough to respect client s privacy but precise enough to be able to locate it. A patched version of dig allows one to make queries with this new extension:
$ geoiplookup 138.231.136.0
GeoIP Country Edition: FR, France
$ ./bin/dig/dig @dns-02.dailymotion.com www.dailymotion.com \
>     +client=138.231.136.0/24
; <<>> DiG 9.8.1-P1-geoip-1.3 <<>> @dns-02.dailymotion.com www.dailymotion.com +client=138.231.136.0/24
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23312
;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; CLIENT-SUBNET: 138.231.136.0/24/24
;; QUESTION SECTION:
;www.dailymotion.com.           IN      A
;; ANSWER SECTION:
www.dailymotion.com.    600     IN      A       195.8.215.136
www.dailymotion.com.    600     IN      A       195.8.215.137
;; Query time: 20 msec
;; SERVER: 188.65.127.2#53(188.65.127.2)
;; WHEN: Sun Oct 20 15:44:47 2013
;; MSG SIZE  rcvd: 91
$ geoiplookup 195.8.215.136
GeoIP Country Edition: FR, France
In the above example, a client located in France gets a reply with two IP addresses located in France. If we now are an US client, we will get IP addresses located in the US:
$ geoiplookup 170.149.100.0
GeoIP Country Edition: US, United States
$ ./bin/dig/dig @dns-02.dailymotion.com www.dailymotion.com \
>     +client=170.149.100.0/24
; <<>> DiG 9.8.1-P1-geoip-1.3 <<>> @dns-02.dailymotion.com www.dailymotion.com +client=170.149.100.0/24
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23187
;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; CLIENT-SUBNET: 170.149.100.0/24/24
;; QUESTION SECTION:
;www.dailymotion.com.           IN      A
;; ANSWER SECTION:
www.dailymotion.com.    600     IN      A       188.65.120.135
www.dailymotion.com.    600     IN      A       188.65.120.136
;; Query time: 18 msec
;; SERVER: 188.65.127.2#53(188.65.127.2)
;; WHEN: Sun Oct 20 15:47:22 2013
;; MSG SIZE  rcvd: 91
$ geoiplookup 188.65.120.135
GeoIP Country Edition: US, United States
The recursor is expected to cache the two different answers and only serve them if the client matches the appropriate subnet (the one confirmed in the answer from the authoritative server). With this new extension, the authoritative server knows that Juan is located in China and answers with the appropriate IP address: Query for www.example.com through an open recursor with client subnet Not many authoritative servers support this extension (PowerDNS and gdnsd, as far as I know). At Dailymotion, we have built a patch for BIND. It only works when BIND is configured as an authoritative server and it doesn t expose no configuration knobs. Feel free to use it (at your own risk). Once installed, you need to register yourself to OpenDNS and to Google to receive queries with the extension enabled.

16 March 2014

Andrew Shadura: hgk misbehaviour with Tk 8.4

Strangely enough, running hgk (a port of gitk to Mercurial, also known as hg view) with Tk 8.4 crashes my X server. Updating both Tk and the X doesn t help. I don t feel like I want to debug X now, but probably that s what I have to do :(
Backtrace:
0: /usr/bin/X (xorg_backtrace+0x49) [0xb7712ed9]
1: /usr/bin/X (0xb7572000+0x1a4c64) [0xb7716c64]
2: linux-gate.so.1 (__kernel_rt_sigreturn+0x0) [0xb755040c]
3: /lib/i386-linux-gnu/i686/cmov/libc.so.6 (0xb711d000+0x772f0) [0xb71942f0]
4: /lib/i386-linux-gnu/i686/cmov/libc.so.6 (__libc_calloc+0xab) [0xb7196d5b]
5: /usr/bin/X (0xb7572000+0x55ca9) [0xb75c7ca9]
6: /usr/bin/X (0xb7572000+0x48005) [0xb75ba005]
7: /usr/bin/X (0xb7572000+0x491c3) [0xb75bb1c3]
8: /usr/bin/X (0xb7572000+0x12c621) [0xb769e621]
9: /usr/bin/X (XkbHandleActions+0x20b) [0xb76c8c1b]
10: /usr/bin/X (XkbProcessKeyboardEvent+0xbc) [0xb76c937c]
11: /usr/bin/X (AccessXFilterPressEvent+0xcd) [0xb76c181d]
12: /usr/bin/X (0xb7572000+0x157675) [0xb76c9675]
13: /usr/bin/X (mieqProcessDeviceEvent+0x1e6) [0xb76f3226]
14: /usr/bin/X (mieqProcessInputEvents+0xfd) [0xb76f335d]
15: /usr/bin/X (ProcessInputEvents+0x14) [0xb75ef3e4]
16: /usr/bin/X (0xb7572000+0x3c25d) [0xb75ae25d]
17: /usr/bin/X (0xb7572000+0x2a51a) [0xb759c51a]
18: /lib/i386-linux-gnu/i686/cmov/libc.so.6 (__libc_start_main+0xf5) [0xb71368c5]
19: /usr/bin/X (0xb7572000+0x2a8f8) [0xb759c8f8]

14 March 2014

Andrew Pollock: [life] Day 45, BJJ, Science Friday and baking cookies

We had a pretty busy day today. Sarah dropped Zoe off at around 8:30am. I reminded Zoe that she'd forgotten to give Sarah her crown that she'd made at Playgroup the previous day, so she gave that to her. After Sarah left, I let Zoe watch a little bit of TV while I finished getting myself organised. We drove to Brazilian Jiu Jitsu class today, because I wanted to go directly to the Planetarium afterwards. Zoe's BFF Megan came to BJJ class today for a trial class. Zoe was very excited. Megan picked everything up really quickly, and the girls had a good time. She didn't want to wear her uniform today though. I managed to get out of her later that she felt it made her sweaty. I told her it was probably all the running around that made her sweaty, not the uniform. We went out for a coffee with Megan, her Dad and her little sister afterwards. We found a little take away shop around the corner. Not quite the place that did babyccinos unfortunately, so Megan's Dad very cleverly got a chocolate milk and split it three ways with the girls and called it a "milkshake". I've realised that for Science Friday, I've pretty much been doing solely chemistry so far, so I wanted to throw in a bit of astronomy this week. The Sir Thomas Brisbane Planetarium had just recently reopened after getting a spanky new projector because 1978 called and wanted theirs back. Unfortunately, they weren't showing Perfect Little Planet today, but the slightly more specific Black Holes: Journey into the Unknown. I wanted Zoe to see something in the auditorium, so I thought we'd give it a shot anyway. I think we got to the Planetarium at around 11am, and the next show was at 1:30pm, so we had a bit of a look around the foyer for a bit, which Zoe quite enjoyed. They had a life-size model of Neil Armstrong's spacesuit, and a scale model of a Saturn V rocket. Zoe quite enjoyed both of them. I'll have to dig up some Youtube footage of a Saturn V launch. We then headed over to the cafe in the adjacent Botantic Gardens, and had a reasonable lunch. After lunch we wandered around the Botanic Gardens to kill some time, so you could say that today's Science Friday also included a goodly amount of botany as well. We went into the Tropical Dome and looked at the fish in the lily pond, and toured the fern garden, before heading back. We timed it pretty well, and got back to the Planetarium with about 10 minutes to kill. I'd prepped Zoe for the presentation, and told her that she needed to keep quiet, and if we had to leave we wouldn't be able to go back in again, and she could whisper to me if she needed to. I have to say, she was so well behaved. She sat through the whole 45 minute presentation and the projected tour of the night sky. I guess the presentation was sufficiently visually interesting to hold her attention for the duration. There was certainly a lot of "wow" out of her. I was so proud of how she went that I had to go and buy her an ice cream as a reward. After that, we went for a drive around Mount Coot-tha. I pointed out the TV towers that we can see from our balcony, and then she fell asleep on the way home. She didn't wake up particularly happy from her late, short nap, when we got home, but once she got herself sorted out, we baked the cookies that we'd made yesterday. They came out very nicely indeed. Once they were all baked, we dropped off a dozen to Megan's place, and then came home to have dinner. Bedtime went pretty well. I hope she has a good night's sleep, as we have a big day tomorrow.

Next.

Previous.