While I remember, I have uploaded the slides from my talk about Solr and Perl at the London Perl Workshop. This talk was inspired by having seen and contributed to at least five different sets of Solr search code at my current job, all of which (I now believe) were doing it wrong. I distilled this hard-won knowledge into a 20 minute talk, which - funny story - I actually delivered twice to work around a cock-up in the printed schedule. I don't believe any video was successfully taken, but I may be proved wrong later. I have also uploaded the Parse::Yapp grammar mentioned in the talk. In case you don't have time to read the slides, the right way to present Solr via Perl is to use the 'edismax' parser, and write your code a bit like this:

my $solr = WebService::Solr->new($url);
my $s = $query->param('q');
# WebService::Solr::Query objects are useful for
# 'fq' params, but avoid them for main 'q' param.
my $options =  
 fq => [WebService::Solr::Query->new(...)];
 ;
$solr->search($s, \%options);

The key thing here is not to put any complicated parsing code in between the user and Solr. Avoid Search::QueryParser at all costs.

At the London Perl Workshop last Saturday, one of the lightning talks was about Questhub.io, formerly known as "play-perl.org". It's social gamification for your task list, or something like that. Buzzword-tastic! But most importantly, there seems to be a nice community of programming types to procrastinate with you on your quests. This means I can finally get to work refuting lamby's prediction about gamification of Debian development! Tasks are referred to as "Quests", and are pursued in themed "Realms", for that World of Warcraft feeling. For example, there's a "Perl" realm, and a "Lisp" realm, and a "Haskell" realm, but also non-programming realms like "Fitness" and "Japanese". Of course, part of me now wants to construct a federated version which can be self-hosted. :) Another downside of questhub currently is the lack of SSL support - your session cookies are sent in plain text. I hope this changes soon.

It's my first Father's Day! Sophie was born 2 months ago (3345g or 7lb 6oz), and I've been on a blogging hiatus for quite a bit longer than that. She's very cute. I am getting into the swing of fatherhood - lots of nappy changing. :) I took my two weeks of paternity leave, but spread the second "week" over two weeks by working just afternoons, which gave me lots of time with mummy and baby. We watched a DVD called "The Happiest Baby on the Block", and mastered the techniques therein (mainly swaddling and white noise). So all things considered, we're getting quite a bit of sleep. Sophie is very curious about my typing, and leans towards anything she's interested in... so she's currently suspended at an angle besides me. Maybe she'll be interested in what her parents do, when she grows up. :) But for now, we're enjoying that she's learned to smile.

Another year. 2012 was busy - I got moved house twice, changed jobs, and got married. In 2013, I should become a father, fingers crossed (due mid-April). Change is a familiar friend now. I just listened to Tom Armitage speaking about coding on Radio 4 - I /think/ the podcast mp3 link will work for people outside the UK, but the iPlayer probably won't. If you can get hold of it, it's worth the 20 minutes of your time. If I had to make a New Year's resolution, it would be to listen to more Radio 4 - there's such a lot of it, though. I'm going to try subscribing to some of their podcasts and listening to them on my commute - timeshifting some of the best bits. Might work.

I have been dealing with an interesting forking issue at work. It happens to involve Perl, but don't let that put you off. So, suppose you need to perform an I/O-bound task that is eminently parallelizable (in our case, generating and sending lots of emails). You have learnt from previous such attempts, and broken out Parallel::Iterator from CPAN to give you easy fork()ing goodness. Forking can be very memory-efficient, at least under the Linux kernel, because pages are shared between the parent and the children via a copy-on-write system. Further suppose that you want to generate and share a large data structure between the children, so that you can iterate over it. Copy-on-write pages, should be cheap, right?

my $large_array_ref = get_data();
my $iter = iterate( sub  
    my $i = $_[1];
    my $element = $large_array_ref->[$i];
    ...
 , [0..1000000] );

Sadly, when you run your program, it gobbles up memory until the OOM killer steps in. Our first problem was that the system malloc implementation was less good for this particular task than Perl's built-in malloc. Not a problem, we were using perlbrew anyway, so a quick few experimental rebuilds later and this was solved. More interesting was the slow, 60MB/s leak that we saw after that. There were no circular references, and everything was going out of scope at the end of the function, so what was happening? Recall that Perl uses reference counting to track memory allocation. In the children, because we took a reference to an element of the large shared data structure, we were effectively writing to the relevant page in memory, so it would get copied. Over time, as we iterated through the entire structure, the children would end up copying almost every page! This would double our memory costs. (We confirmed the diagnosis using 'smem', incidentally. Very useful.) The copy-on-write semantics of fork() do not play well with reference-counted interpreted languages such as Perl or CPython. Apparently a similar issue occurs with some mark-and-sweep garbage-collection implementations - but Ruby 2.0 is reputed to be COW-friendly. All was not lost, however - we just needed to avoid taking any references! Implement a deep copy that does not involve saving any intermediate variables along the way. This can be a bit long-winded, but it works.

my $large_array_ref = get_data();
my $iter = iterate( sub  
    my $i = $_[1];
    my %clone;
    $clone id   = $large_array_ref->[$i] id ;
    $clone foo  = $large_array_ref->[$i] foo ;
    ...
 , [0..1000000] );

This could be improved if we wrote an XS CPAN module that cloned data structures without incrementing any reference counts - I presume this is possible. We tried the most common deep-copy modules from CPAN, but have not yet found one that avoids reference counting. This same problem almost certainly shows up when using the Apache prefork MPM and mod_perl - even read-only global variables can become unshared. I would be very interested to learn of any other approaches people have found to solve this sort of problem - do email me.

On Monday, I need to start hiring a Perl programmer - or, at least, a programmer willing to write Perl. I work for a website where people post their CVs, which tends to help - although this will mean that my boss wants me to do it without going through recruiters. Which is fine. I just have to use the search interface that recruiters normally use. And looking through all these CVs, it dawned on me that I don't have a clue whether any of the people are suitable for the job. I have to search for keywords that we think might be relevant - "Perl", I guess - and then sort through the hundreds of people who come back from the search. It's very painful, because you can't really judge a CV without reading it - and even that won't necessarily tell you the important things about that person. Do they actually write good code? Do they work well in a team? When searching for a piece of information, you probably need just one website to answer your question; when searching for job candidates, I guess you need to see a range of CVs. And then you need to interview them; this could take weeks. Sucks to be me.

Today, Kate and I got married! Thank you to everyone who sent best wishes. A big wedding party will follow in the next 18 months or so (when we've saved some money!), to which many more people will be invited. This was the minimum viable subset of wedding - we got the product to market early, and both stakeholders are very satisfied. We had dinner at the Caribbean restaurant in town which is always busy - turns out there's a reason for that. They do these one-pot meals in enamel dishes, which tasted amazing. It is a strange new feeling to be a husband. :)

I just read the blog post from Tim Retout about the computer science book collection available in his local library, and just wanted to share my comment on his theory about computer books becoming obsolete so soon. That is part of the reason why the selection is so sad in almost any local library (it is in mine too), but I believe the major contributing factor is that the people buying books to the library have no way to know a good and future computer classic from trash. And they need to know which one will become a classic in the future, as they would normally buy one of the recently published books. During my university years, I worked for a while at the university library, and even there the person in charge of buying computer related books (and in fact any natural science related book), did not know enough about computers to make a good educated guess. Once, just before Christmas, they had some leftover money on the book budget and I was asked if I could pick out a lot of computer books in the university book store, for the library to buy for their collection. I had a great time picking all the books I dreamt of buying and reading, and the books I knew were classics (like most of the Stevens collection). I picked several of the generic O'Reilly books (ie documenting protocols, formats and systems, not specific versions of products) and stayed away from the 'teach yourself X in N days' class. I had a great time, and probably picked out more than a hundred books for the library that evening. The sad fact is that there is no way a overworked librarian is going to know that for example The Practice of Programming is a must-have in any computer library, and they will most of the time end up picking the wrong books to buy. Perhaps you can help your local library make better choices by giving the suggestions for books to get? I know they would love to hear from you, even if their budget might block them from getting your favourite book right away.

I visited my local library at the weekend, on a whim. (The weekend before, I'd been to the British Library for the first time, so I guess this inspired me.) The computing sections at public libraries do not tend to inspire me, on the whole. Southampton's is actually relatively good - there are four tall bookcases assigned to computing, although one is introductory IT (Word and Excel), and another seems to be assigned to graphic design (Photoshop). Still, the two in the middle do have some programming texts - you just have to work around "Red Hat Linux 7 for Dummies" and the like. I think the trouble is, computing books become obsolete so quickly; you don't get point releases of Shakespeare every six months. I also suspect the low demand for serious computing texts creates a vicious circle, where the type of people who might want that sort of thing know better than to look in the library for it. It got me thinking - could this be changed? Could I change it?

The other day, I released GNU Enscript 1.6.6. You should all go and send me bug reports. It's basically the same as the 1.6.5.90 release, but more official. (I'm bored of the long version numbers - maybe I ought to knock a decimal point off.)

My email's been out of control for a while now. I've noticed a correlation between the state of my email and my state of mind - I don't know which way the causation flows, if any. At the weekend (after assembling the shelves), I archived my entire inbox. Again. But I find the hard part with email bankruptcy is preventing the entire cycle recurring. This time, I was more drastic. I deleted all my labels, and all but three of my filters. I removed the Smart Labels, the chat widget, the calendar integration, the Google+ circles (more on that soon). There are no distractions in my inbox any more. I'm forcing myself to process every non-spam email that comes to my address. (Yes, I'm embarassed to say that I use Gmail - it seemed like a good idea at the time, because of the spam filtering. I occasionally notice bugs with ancient emails becoming unarchived or unread, and wonder if Google uses a probabilistic data structure to access all those attributes quickly enough. Can't investigate, though, because it's Gmail. Consider this the first step towards moving away.) So rather than hiding the deluge in the background, I'm unsubscribing from a lot of mailing lists, and informing marketers that I don't want their communications. It turns out that I get a lot less email than I feared - and processing them this way means that high-traffic sources get removed first. My remaining problem is what to do with email from mailing lists like debian-private - I don't really care enough to read everyone's vacation plans, but unsubscribing doesn't seem like the right thing either - there's (obviously) no online archive for this type of mailing list. For the moment, I've added a filter back to hide just vacation messages, but there's a more general problem lurking here.

Bookshelves are a wonderful thing. Kate and I have been living without enough book space since we moved in together - all our shelves have been double-stacked. Finding anything is a pain, because it's impossible to tell if you still own the book you're looking for. No longer. One short visit to Ikea (plus delivery and assembly), and we have a huge 5x5 Expedit bookshelf dividing our living area. It has comfortably absorbed our entire collection (plus boxes) - we have double-stacked it, but can access both sides. Books are grouped by subject matter, so can actually be found. (We have a huge O'Reilly collection, but gave up sorting by publisher when we discovered various non-computing books in there...) Now I can sit on the sofa, and have (paper) reading material within arm's reach. It's good to have something more to our living space than a television and a pair of laptops. As an added bonus, you can't see the washing up from the living room any more. Screw e-books.

Today, as an experiment, I attempted to fix a Debian bug while on the train to work. I use a 3G card from Three.co.uk in my Lenovo Thinkpad x121e, and my commute is from Southampton Central to Fleet (changing at Winchester) - just under an hour. 3G coverage is not 100%, but tends to be better around the major stops.

• First, I found a bug. I used udd.debian.org to browse for a relatively simple RC bug, and found bug #674992 in actionaz. The fix was outlined in the report already, so there was very little thinking required.
• Next, I confirmed the FTBFS using cowbuilder. Unfortunately, this required downloading roughly 120MB of dependencies - I have 1GB of data per month, but I couldn't afford to do this every day. I was lucky in that I was near Basingstoke at the time, so had a good HSDPA signal to get the bulk of this. The build had failed before I reached Fleet.
• In the background, I updated debian/control and debian/changelog with the fix. I was able to set off the build, but had to suspend the laptop until lunchtime before it could finish. Cowbuilder needed to download only a few extra build-deps, as the vast majority were cached from the initial run.
• On the train home, I checked over the result, signed it and uploaded. In this instance, the built package was small enough to upload, but I could see this being a problem with others.
• Finally, I sent the nmudiff, although that was delayed briefly by a drop in connectivity before Southampton Airport.

Thoughts: firstly, part of me is amazed that this is possible. Secondly, there could be a case for a local Debian mirror on my laptop. Otherwise, an interesting experimental extension to UDD would be "Required bandwidth" - the sum of the recursive build-dependencies plus the upload size of the diff/binaries.

Following on from the weekend of change, I've got engaged to Kate. :) We now need to organise a combined housewarming/engagement party...

Agile processes harness change for the customer's competitive advantage. Principles behind the Agile Manifesto

After two and a half years at Smoothwall, I'm moving on - Friday is my last day. Since I joined the development team, we have adopted Agile development, set up a pretty nifty Gerrit/Jenkins code review + integration system, and introduced dpkg for package management. Along the way, I helped with a bunch of important features for the business, like a ground-up rewrite of the web filter, and time-based browsing usage quotas. I will be starting at CV-Library on Monday, for a whole new set of challenges. They're based in Fleet, so I'll have an hour-long commute each way on the train. I've been assured by an expert in these matters that facilitating wage slavery is a comparatively more ethical pursuit than facilitating internet censorship. :) To make the commute somewhat more bearable, I'm moving house on Saturday. So far, the packing's going quite well...

I've finally got around to deleting my Facebook account. I'd love to claim that this was a grand gesture against privacy-invading apps, or a bid to recoup vast amounts of my spare time... but it's not, really. I rarely logged in to the site, these days, so Facebook has very little of my personal data.

• Michael Meeks gave awesome talks at FOSDEM, so Kate was inspired to hack on LibreOffice. I was inspired to write this blog entry in a list. She probably wins.
• Building LibreOffice master on Debian stable failed for her with a segmentation fault in GNU Make. A bit of searching threw up Savannah bug #20033, which is hitting everyone on the upstream mailing list.
• Bumped severity and offered to NMU Debian bug #622644.
• Then actually tried building make-dfsg in cowbuilder, and aclocal fails in the clean environment because /usr/share/aclocal does not exist. I think it's related to Debian bug #565663, but I'm still poking it.

Recently, I tried to get 3G working on my Lenovo ThinkPad X121e - it has an Ericsson F5521gw mobile broadband card. This is supported by ModemManager, but all I got were unknown errors (276 and 272). Searching online, there were very few results (hence this short note) - just previous unrelated Linux kernel issues. I found someone with the same problem on Fedora, but no solution, so I started off by filing a bug report with Debian. Of course, then I found the Arch user who had filed the same bug on Launchpad, and had discovered that resetting the BIOS to its default settings fixes the issue. If only that page mentioned the keywords "Ericsson", or "Lenovo"... So after all that, it was just some weird BIOS issue. I hate hardware.

So since my last post about perl tutorials, the Perl Tutorial Hub has leaped from page 2 to be the top result for the relevant Google search. The Leeds tutorial has dropped off the first page. I couldn't figure out how such a dramatic reversal could have happened, until I asked Mithaldu on IRC; the admins of the old Leeds tutorial have added a (delayed) redirect. So, Google has interpreted that as a 302 status, and given perl-tutorial.org all the old inbound links, presumably. Perhaps there is hope for Perl yet. :)

Hello, World! Last year, a bit of a fuss was kicked up in the Perl community about the low quality of search results for the phrase "Perl tutorial". Various ideas for fixing this were proposed, including the handy Perl tutorial hub, but kicking Leeds University off the coveted top spot is going to be a real challenge. The problem is, most Perl tutorials on the internet were written for Perl 4; modern Perl doesn't get a look-in. It's a miracle anyone manages to learn Perl at all... While thinking over this problem, I was reading Mithaldu's original criteria for the "content creation" option. "Community effort"... "github repo"... "exported to HTML regularly"... if only Perl had some central site where you can publish documentation... that all Perl hackers can access and update... like CPAN. So although my documentation-writing skills are pretty weak, I proudly introduce the Perl-Tutorial CPAN dist and github repository. The great thing about writing Perl documentation using POD is that you can link to other CPAN references so easily - as the basics get filled out, they can guide the user towards how to learn more about each topic. Everyone who's anyone knows how to send a pull request on github, and there seems to be far more of a community feel to CPAN these days. Version 0.001 is just "Hello, World!" - but watch this space. :)