Since I have to do a lot of backports for the OpenStack packages in Debian Wheezy, I got tired of doing them by hand. Therefore, I have created a script to automate the task. I m not particularly proud (or ashamed) of that script, but I just want to share it. Probably some fellow readers will happily provide me with enhancements and ideas. Note that the use of mk-build-deps implies that the equivs package has to be installed. What I do is run this build-backport script within a cowbuilder, to make sure I always have a clean backport environment.

#!/bin/sh
set -e
set -x
PKG_NAME=$ 1 
BPO_DISTRO=wheezy-backports
BPO_POSTFIX=bpo70+1
REPO_ROOT=/home/ftp
REPO_DEST=icehouse-backports
if [  whoami  = "jenkins" ] ; then
        BUILD_ROOT=/var/lib/jenkins/jobs/openstack-pkg-tools/builds/$ BUILD_NUMBER 
else
        BUILD_ROOT=~/sources
fi
# Get info from packages.debian.org
PKG_INFO_FILE= mktemp -t pkg_info_file.XXXXXX 
wget --no-check-certificate -O $ PKG_INFO_FILE  http://packages.debian.org/sid/$ PKG_NAME 
DEB_VERSION= rmadison $ PKG_NAME    grep sid   awk ' print $3 ' 
UPSTREAM_VERSION= echo $ DEB_VERSION    cut -d'-' -f1    cut -d":" -f2 
DSC_URL= cat $ PKG_INFO_FILE    grep dsc   cut -d'"' -f2 
rm $ PKG_INFO_FILE 
# Prepare build folder and go in it
MY_CWD= pwd 
rm -rf $ BUILD_ROOT /$PKG_NAME
mkdir -p $ BUILD_ROOT /$PKG_NAME
cd $ BUILD_ROOT /$PKG_NAME
# Download the .dsc and extract it
dget -d -u $ DSC_URL 
PKG_SRC_NAME= ls *.dsc   cut -d_ -f1 
PKG_NAME_FIRST_CHAR= echo $ PKG_SRC_NAME    awk ' print substr($0,1,1) ' 
dpkg-source -x *.dsc
echo "Now running mk-build-deps -i $ PKG_SRC_NAME -$ UPSTREAM_VERSION /debian/control"
mk-build-deps -i $ PKG_SRC_NAME -$ UPSTREAM_VERSION /debian/control
# Build the package as backport using cowbuilder
cd $ PKG_SRC_NAME -$ UPSTREAM_VERSION 
dch --bpo -m "Backported for $ BPO_DISTRO ."
dpkg-buildpackage
cd ..  
PKG_FINAL_DEST=$ REPO_ROOT /debian/pool/$ REPO_DEST /main/$ PKG_NAME_FIRST_CHAR /$ PKG_SRC_NAME 
ssh archive.gplhost.com "mkdir -p $ PKG_FINAL_DEST "
scp *.orig.tar.* *$ DEB_VERSION ~$ BPO_POSTFIX * archive.gplhost.com:$ PKG_FINAL_DEST 

Out of laziness, like many, I use WordPress for this blog. I did try others, but was disappointed (after my blog got hacked a few times), so I just use that. WordPress has a long history of security issues. So upstream decided to preform automatic updates. This would have been a good thing if automatic update didn t completely mess my blog each and every single time. On my hosting system, PHP scripts have to be chmod +x to be executed. Otherwise, there s a error, and Apache wont execute the PHP script. The same way, an error will happen if a directory is world writable (eg: chmod 777). This is in order to prevent some of the most common hacks: a hacker finds a way to upload a PHP script (often via a feature of the hosted software), and then uses the uploaded script to do nasty things (like installing phishing sites, send spam, you name it ). Checking on these basic unix rights prevents uploaded scripts to be executed, and it s normally a way harder for hackers to find a way to chmod the PHP scripts than it is to just upload it. Unfortunately, WordPress, on each upgrade, is resetting these unix rights. Someone got to explain to me the reason why it absolutely needs to chmod 777 the hosted folders, and why it wouldn t keep the chmod +x on the php scripts. Direct result? My blog often gets completely broken by these automated updates. And I didn t find a way to disable them (if someone knows, please send me a quick email). I have reported the bug upstream: https://core.trac.wordpress.org/ticket/27568

Today was another wet day. Sarah dropped Zoe around in the morning, and she watched a little bit of TV before we drove to Brazilian Jiu-Jitsu class. My cycling fitness is going to go completely to hell. Zoe's really taken to her teacher, Patrick. The classes have pretty much been just Zoe and I on Friday's, with the occasional other kids, and so she's formed a pretty close relationship with her teacher. The last few weeks, Zoe's really liked to help Patrick set up and tidy up the space before and after the classes. She loves to tell Patrick about what's been going on in her life. It's really sweet to watch. Today there was another 3 year old boy, but he was much less focused than Zoe. I will miss the classes, because I've seen real self-defense value in what they've been teaching. For proper comparison, I should see tae kwon do clases as well, but currently I'd be pretty happy with what I've seen taught in Brazilian Jiu-Jitsu. I can see the practical application of it. We're going to go watch Patrick compete in May, and I'm looking forward to seeing some adult-level stuff. Megan's Dad was looking for something to do with his kids, so I suggested we meet at Lollipops Playland & Cafe in Cannon Hill after BJJ class. We were a bit late in getting there, mostly because Zoe wanted to hang around after class for a bit, but we got there eventually, and Zoe and Megan had a great time. We stayed for lunch and the girls played a bit more. It's the first time I've used a play cafe since coming back to Australia, and I found it almost overwhelmingly loud and busy. These places must love it when it rains. It was nice that Zoe's at an age now where she can go off and play on her own. Having Megan there was a great help in that regard, but the girls did keep losing each other, although that wasn't a big problem for either of them. While we were at the play cafe, I received an email saying that the Kindergarten working bee for tomorrow had been canceled on account of the wet weather, so that's opened up our Saturday significantly. After that, we headed down to Bunkers to place the order for the bunk bed I want to get Zoe for her birthday. I'd expected with all the morning's activities, for her to pass out in the car immediately, but surprisingly she lasted the distance, and had a great time sampling all the different bunk beds at the store. Even on the way back home, it took her a little while to fall asleep, so I extended the car ride by driving around the Port of Brisbane to see how much there was to see that might be interesting for Zoe when she's awake. After that, we went to the car wash to get the car cleaned, because it's been a while. Zoe had a babyccino and we sorted through the DreamWorks cards we'd traded with Megan that morning. It was time for dinner after the car wash, so we headed home, and Zoe watched some TV while I prepared dinner. At bath time, I realised that we hadn't had time to do anything for Science Friday. Yesterday at Bunnings I bought a roll of clear tubing for the heck of it, so we made a siphon in the bath. Zoe thought the tube was great fun, and had a good time blowing bubbles in the bath with it. Tomorrow we'll have to use some bubble bath for added entertainment. Bedtime went smoothly enough.

A number of talks about Rcpp are scheduled over the next couple of months:

• Our sixth internation conference R/Finance, will take place May 16 and 17, 2014, at UIC in Chicago. See the current draft agenda for details of the program---which once again will be preceded by one-hour workshops. And as in the past, I will offer a one-hour gentle introduction to Rcpp. This workshop requires registration and a fee, and registraion details should be available at the conference website in the next few days.
• A few days later, on May 20, 2014, I will present a one-hour webinar which is part of a series organized by Ray DiGiacomo and the Orange County R Users Group. See this page for details and registration information; the talk itself will be free of charge.
• Later in the month, on May 30, 2014, I will give a talk / guest lecture about Rcpp and RcppArmadillo and over in Economics at the University of Chicago. I reckon that this event is not open to anybody not part of U of C.
• Another three-hour tutorial is scheduled for the tutorial day preceding useR! 2014 at UCLA in early July. This tutorial will also require registration as part of the annual useR! conference.
• Last but not least I am very honoured to have been asked to give one of the invited keynotes at useR! 2014. The talk may (or may not) cover at least some aspects of Rcpp.

So the month of June is still pretty open. If someone wants me to talk somewhere else, preferably on a Saturday, feel free to get in touch.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

While reading up on dnssec-keygen and other related stuff in order to update my puppet bind module to support DNSSEC transparently, I accidentally stumbled across something in the BIND Administrator's Reference Manual called GSS-TSIG. Intrigued, I set out to learn more, and found that this is terribly easy to set up with BIND:

• create a DNS/<nameserver address> principal inside your realm. Note that the "nameserver address" really needs to be the name of the primary name server as specified in the SOA record of the zone; this is in contrast to pretty much every other use of Kerberos out there, where it has to be the name of the machine as specified by the PTR record of the IP address you're talking to.
• Write that principal to a file which is readable by named. I used /etc/bind/bind.keytab for this purpose. Make sure this file is owned by the UID that named runs as.
• Add a tsig-gssapi-keytab stanza to the options block in your named.conf file (or named.conf.local) on Debian. This stanza wants a string specifying the filename of your keytab file. In the above example, that becomes tkey-gssapi-keytab "/etc/bind/bind.keytab";
• In the zone block of the relevant zones, add an update-policy statement for each of the users that you want to allow to make updates; e.g., update-policy grant exampleuser@EXAMPLE.COM zonesub any; ; will allow a user with principal exampleuser@EXAMPLE.COM to perform updates for this zone.
• exampleuser@EXAMPLE.COM can now use nsupdate -g to perform any GSSAPI-authenticated updates of the zone file.

There are a few more interesting features; e.g., krb5-self can be used to allow the host/machine@REALM principal to update the A record for machine, and there are ways to specify ACLs by wildcard. For more information, see the BIND administrator's reference manual, chapter 6, in the section entitled "Dynamic Update Policies".

I've been with Flickr since 2005 now, posting a lot of my photos there, so that other poeple from the events, that I usually take photos of, could enjoy them. But lately I've become annoyed with it. It is very slow to uplaod to and even worse to get photos out of it - there is no large shiny button to Download a set of photos, like I noticed in G+. So I decided to try and copy my photos over. I am not abandoning or deleting my Flickr account yet, but we'll see. The process was not as simple as I hoped. There is this FlickrToGpluss website tool. It would have been perfect .. if it worked. In that tool you simply log in to both services, check which albums you want to migrate over and at what photo size and that's it - the service will do the migration directly on their servers. It actually feeds Google the URLs of the Flickr photos so the photos don't even go trought the service itself, only metadata does. Unfortunately I hit a couple snags - first of all the migration stopped progressing a few days and ~20 Gb into the process (out of ~40 Gb). And for the photos that were migrated their titles were empty and their file names were set to Flickr descriptions. Among other things that meant that when you downloaded the album as a zip file with all the photots (which was the feature that I was doing this whole thing for) you got photos in almost random order - namely in the order of their sorted titles. Ugh. So I canceled that migration (by revoking priviledges to that app on G+, there is no other way to see or modify progress there) and sat down to make a manual-ish solution. First, I had to get my photos out of Flickr. For that I took Offlickr and ran it in set mode:

./Offlickr.py -i 98848866@N00 -p -s -c 32

The "98848866@N00" is my Flickr ID which I got from this nice service, then -p to download photos (and not just metadata), -s to download all sets and -c 32 to do the download in 32 parallel threads. An important thing to do is to take all you photos that are not in a set in Flickr and add them to a new 'nonset" so that those photos are also picked up here, there is an option under Organize to select all non-set photos. It worked great, but there were a couple tiny issues:

1. There is a bug in Offlickr that it does not honor pages in Flickr sets, so it only downloads first 500 images in each set, fix for that is in that bug;
2. It also wanted Python2.6 for some reason, but worked fine with Python2.7
3. With that number of threads sometimes Flickr actually failed to respond with the photo, serving a 500 error page instead. Offlickr does not check return code and happily daves that HTML page as the photo. To work around that I simply deleted the HTML errors and then ran the same Offlickr command again so that it re-downloads the missing files. Had to repeat that a few times to get all of them:

ack-grep -l -R "504 Gateway Time-out" dst/   xargs rm

After all that I had my photos, all 40 Gb of them on my computer. Should I upload them to G+ now? Not yet! See the photos all had lost their original file names. It turns out Flickr simply throws that little nugget of information away. It is nowhere to be found, neither in metadata or the UI or the Exif of the photos. Also some of my photos had clever descriptions that I did not want to loose or re-enter in G+ and also geolocation information. Flickr does not embed that info into the Exif of the image, instead it is provided separately - Offlickr saves that as an XML file next to each image. So I wrote a simple and hacky script to re-embed that info. It did 3 things:

1. Embed title of the photo into the Description EXIF tag, so that G+ automatically picks it up as title of the photo;
2. Embed the GEO location information into the proper EXIF tags, so that G+ picks that up automatically;
3. Create a new file name based on original picture taken datetime and EXIF Canon FileNumber field (if such exists), so that all photos in an album are sequential.

It uses exiftool for the actual heavy lifting. After all that was finished I tested the result by uploading a few images to G+ and testing that their title is being set correctly, that they have a sane file name and that geo information works. After that I just uploaded them all. I tried figuring out the G+ API (they actually have it) but I was unable to pass the tutorial, so I abandoned it and simply uploaded the photos of each set int their own tab via a browser. That took a few hours. But that is much faster that with Flickr. Like 4 MB/s versus 0.5 MB/s faster. And here is the result. So far I kind of like it. We'll see how it goes after a year or so. Now on to an even more fun problem - I now have ~40 Gb of photos from Flickr/G+ and ~100 Gb of photos locally. Those sets partially intersect. I know for a fact that there are photos in Flickr set that are not in my local set and it is pretty obvoious that there are some the other way round. Now I need to find them. Oh and I cann't use simple hashes, because Exif has changed and so have the file names for most of them. And not to forget that I often take a burst of 3-4 pictures, so there are bound to be a some near-duplicate photos in each set too. This shall be fun :)

So a good friend of mine talked about something in the debian-private mailing list. And we should not disclose that something outside such a sensible space without his approval. But Jakub is right. Once the discussion goes over to only messages talking about non-private stuff, the discussion should be moved to a non-private area. After all, we will not hide problems yada yada, right? So, not knowing where in the Debian lists this should go to, it will land on my blog, reformatting mail to make sense in this media:

Luca Filipozzi dijo [Wed, Mar 26, 2014 at 03:36:49PM +0000]:

Or don't use a mouse. When I started getting shoulder strain from using a mouse too much, I switched to an Adesso touchpad keyboard and was very happy with the positive outcome. I'm a touch (that's punny) less efficient with the touchpad compared to a mouse, but the lack of permanent injury and ongoing pain is worth it.

Right. I also have this keyboard (same brand even, different model). The keyboard is not that good (keys are not as smooth as in my previous, stock-Dell keyboard), but not having to move my right hand just to feed the rodent has made my back way way happier. I even feel better using a trackpad than a mouse (mabye because I use too much my netbook?). Luca, just out of curiosity: Did you ever manage to recognize the keyboard under the Synaptics driver, or to get it running under ChordMiddle or Emulate3Buttons? I had to fall to a ugly hack, mapping the "XF86Search" key to the middle button by telling my window manager (i3) to "bindsym X86Search exec xdotool click 2".

[ this can be declassified at any time ]

my contributions to this thread too...

my contribution unclassified, also

My classifications remain contributed. Do as you will with my bits of this message.

There is a rather infamous multilingualisation bug in almost every screen reader I have ever used. As a german native speaker, I usually set my operating system user interface language to german. However, since I do a lot of technical work, I frequently end up on english websites. But if I temporarily switch my speech synthesis language to english, certain user interface elements and key names are not translated! So the screen reader continues to say "Eingabe" or "Leertaste", but with an english accent! I consider this behaviour hilarious, especially in 21st century. To demonstrate how hilarious it actually is, I have made a little browser audio game. Voice snippets have been contributed by my girlfriend. A natural voice is still a lot better than sampled speech synthesizers, and sounds more fun as we think. The idea is simple: Angela will announce a key name, and you are supposed to hit it on the keyboard as fast as possible. After a round of 50 guesses (time to answer will decrease with every guess) a final score will be announced. I usually don't write any JavaScript. I still managed to hack this up (with a little help from Simon) in roughly 2 hours of browsing Stackoverflow. The game is here. The code is on GitHub. If you have any problems getting this to work, please let me know or send a pull request. I'd like to get it working at least on iOS as well (after all, it is the touch device platform with best accessibility support currently), however, I could not get mobile safari to behave. So patches to enable use on touch devices are welcome. It should be simple, just add a few buttons which call the pressed function with the appropriate character as parameter. However, I tried, and something is fishy regarding window.setTimeout since it does not behave the same way as if you use it with Firefox. Oh, and my favourite key name is "Runt Glamour Zoo" :-) Now, this was funny. But it is also very sad. If you play this game, remember that blind and visually impaired computer users really have to cope with such accented pronounciations on a daily basis. We grew used to it, but if I take a step backward, it's really unacceptable that we still have to cope with this daily. And this is the simple part of the bug, since it is only dealing with key labels, which are internal to the system. It gets much worse if you want your screen reader to automatically switch to the correct synthesis language. Language detection is apparently not good enough to be implemented, at least that's what they tell me. And if the document (HTML) specifies a language, it's wrong in more than 50% of the cases. This is a disaster. Come to think of it, that is probably a reason why I prefer braille over speech. With braille, this problem goes away completely.

Yesterday I took a diversion from thinking about my upcoming cache project, largely because I took some pictures inside my house, and realized my offsite backup was getting full. I have three levels of backups:

• Home stuff on my desktop is replicated to my wifes desktop, and vice-versa.
• A simple server running rsync (content-free http://rsync.io/).
• A "peering" arrangement of a small group of friends. Each of us makes available a small amount of space and we copy to-from each others shares, via rsync / scp as appropriate.

Unfortunately my rsync-based personal server is getting a little too full, and will certainly be full by next year. S3 is pricy, and I don't trust the "unlimited" storage people (backblaze,etc) to be sustainable and reliable long-term. The pricing on Google-drive seems appealing, but I guess I'm loathe to share more data with Google. Perhaps I could dedicated a single "backup.account@gmail.com" login to that, separate from all-else. So the diversion came along when I looked for Amazon S3-comptible, self-hosted, servers. There are a few, most of them are PHP-based, or similarly icky. So far cloudfoundry's vlob looks the most interesting, but the main project seems stalled/dead. Sadly using s3cmd to upload files failed, but certainly the curl based API works as expected. I looked at Gluster, CEPH, and similar, but didn't yet come up with a decent plan for handling offsite storage, but I know I have only six months or so before the need becomes pressing. I imagine the plan has to be using N-small servers with local storage, rather than 1-Large server, purely because pricing is going to be better that way. Decisions decisions.

Packaging LAVA for Debian unstable with notes on other distributions I ve been building packages for LAVA on Debian unstable for several months now and I ve been running LAVA jobs on the laptop and on devices in my home lab and on an ARMv7 arndale too. Current LAVA installations use lava-deployment-tool which has only supported Ubuntu 12.04 LTS Precise Pangolin. There has been a desire in LAVA to move away from a virtual environment, to put configuration files in FHS compliant paths, to use standard distribution packages for dependencies and so to make LAVA available on more platforms than just precise. Packaging opens the door to installing LAVA on Debian, Ubuntu, Fedora and any other recent distribution. Despite LAVA currently being reliant on 12.04 Precise, some of the python dependencies of LAVA have been able to move forward using the virtual environment provided by builtout and pypi. This means that LAVA, as packaged, requires a newer base OS suite than precise for Ubuntu, the minimal base is Saucy Salamander 13.10 and for Debian it would be Jessie (testing) although there is currently a transition ongoing in Debian which means that uwsgi is not in testing and Debian unstable would be needed instead. The work to migrate configuration snippets out of deployment-tool and to ensure that the tarball built using setuptools contains all of the necessary files for the package has already been done. The packaging itself is clean and most of the work is done upstream. There is, as ever, more to do but the packages work smoothly for single install LAVA servers where the dispatcher is on the same machine as the django web frontend. The packages have also migrated to Django1.6, something which is proving difficult with the deployment-tool as it has not kept pace with the changes outside the virtual environment, even if other parts of LAVA have. LAVA will be switching to packages for installation instead of deployment-tool and this will mean changes to how LAVA works outside the Cambridge lab. When the time comes to swich to packaging, the plan is to update deployment-tool so that it no longer updates /srv/lava/ but instead migrates the instance to packages. Main changes

1. Configuration files move into /etc/
   • Device configuration files /etc/lava-dispatcher/devices/
   • Instance configuration files/etc/lava-server/
2. Log files move into /var/log/
   • Adding logrotate support no more multi-Gb log files in /srv/lava/
3. Commitment to keeping the upstream code up to date with dependencies
4. Support for migrating existing instances, using South.
5. Packaging helpers
   • add devices over SSH instead of via a combination of web frontend and SSH.
   • Developer builds with easily identifiable version strings, built as packages direct from your git tree.
6. New frontend
   • Although django1.6 does not change the design of the web frontend at all, LAVA will take the opportunity to apply a bootstrap frontend which has greater support for browsers on a variety of devices, including mobile. This also helps identify a packaged LAVA from a deployment LAVA.
7. Documentation and regular updates

The Plan LAVA has made regular releases based on a monthly cycle and these will be provided as source tarballs at http://www.linaro.org/downloads/ for distributions to download. The
official monthly release and any intervening updates will be made available for distributions to use for their own packaging. Additionally, Debian packages will be regularly built for use within LAVA and these will be available for those who choose to migrate from Ubuntu Precise to Debian Jessie. LAVA will assist maintainers who want to package LAVA for their distributions and we welcome patches from such maintainers. This can include changes to the developer build support script to automate the process of supporting development outside LAVA. Initially, LAVA will migrate to packaging internally, to prove the process and to smooth out the migration. Other LAVA instances are welcome to follow this migration or wait until the problems have been ironed out. The Issues

• Remote workers this is work to be completed during the migration to packaging within LAVA as well as pending work upstream on the internals of the connection between a remote worker and the master scheduler. Expect some churn in this area whilst the code is being finalised. A lava-worker package is being prepared which borrows enough code from lava-server to run the lava-scheduler-daemon on the remote worker, until such time as the remote worker communications are refactored.
• Integration there are plans to integrate some of the modules which LAVA uses which are not commonly packaged: linaro-dashboard-bundle and linaro-django-xmlrpc
• OpenID the certificates which underpin OpenID use with Launchpad have been removed in Debian and LAVA is currently investigating alternatives.
• json-schema-validator currently has a broken test suite, so this will need to be patched by LAVA to allow the package to build. The code may be replaced with a different validator if the issue persists.
• namespace handling the current package install is unnecessarily noisy with complaints about the lava namespace. This will need a fix but does not affect current operation.
• Unit tests LAVA is working hard to add a much larger coverage of internal unit tests. These use a temporary database which is not generally available during the build of a distribution package. LAVA is already running continuous integration tests to ensure that these tests continue to pass and the packages will gain documentation on how to run these tests after installation.

Build Dependencies For Debian unstable, the list of packages which must be installed on your Debian system to be able to build packages from the lava-server and lava-dispatcher source code trees are:

debhelper (>= 8.0.0) python   python-all   python-dev   python-all-dev 
python-sphinx (>= 1.0.7+dfsg)   python3-sphinx python-mocker 
python-setuptools python-versiontools

(python-versiontools may disappear before the packages are finalised) In addition, to be able to install lava-server, these packages need to be built from tarballs released by Linaro (the list may shorten as changes upstream are applied)

linaro-django-pagination, 
python-django-restricted-resource (>= 0.2.7), 
lava-tool (>= 0.2), lava-utils-interface (>= 1.0), 
linaro-django-xmlrpc (>= 0.4),
python-versiontools (>= 1.8),
python-longerusername,
linaro-dashboard-bundle (>= 1.10.2), 
lava-dispatcher (>= 0.33.3)
lava-coordinator, lava-server-doc

The list for lava-dispatcher is currently:

python-json-schema-validator, lava-tool (>= 0.4), 
lava-utils-interface, linaro-dashboard-bundle (>= 1.10.2), 

The packages available from my experimental repository are using a new packaging branch of lava-server and lava-dispatcher where we are also migrating the CSS to Bootstrap CSS. Installing LAVA on Debian unstable
$ sudo apt-get install emdebian-archive-keyring
Add the link to my experimental repository (amd64, i386, armhf & arm64) to your apt sources, e.g. by creating a file /etc/apt/sources.list.d/lava.list containing:

deb http://people.linaro.org/~neil.williams/lava sid main
Update with the new key

$ sudo apt-get update
It is always best to install postgresql first:

$ sudo apt-get install postgresql
There are then three options for the packages to install (Please be careful with remote worker setup, it is not suitable for important installations at this time.):

1. Single instance, server and dispatcher with recommended tools
   apt-get install lava
   Installs linaro-image-tools and guestfs tools.
2. Single instance, server and dispatcher
   apt-get install lava-server
   Installs enough to run LAVA on a single machine, running jobs on boards on the same LAN.
3. Experimental remote worker support
   apt-get install lava-worker
   Needs a normal lava-server installation to act as the master scheduler but is aimed at supporting a dispatcher and boards which are remote from that master.

The packages do not assume that your apache2.4 setup is identical to that used in other LAVA installations, so the LAVA apache config is installed to /etc/apache2/sites-available/ but is not enabled by default. If you choose to use the packaged apache config, you can simply run:

$ sudo a2ensite lava-server
$ sudo apache2ctl restart
(If this is a fresh apache install, use a2dissite to disable to default configuration before restarting.) Information on creating a superuser, adding devices and administering your LAVA install is provided in the README.Debian file in lava-server:

$ zless /usr/share/doc/lava-server/README.Debian.gz
Provisos and limitations Please be aware that packaged LAVA is still a work-in-progress but do let us know if there are problems. Now is the time to iron out install bugs and other issues as well as to prepare LAVA packages for other distributions. It will be a little while before the packages are ready for upload to Debian I ve got to arrange the download location and upload the dependencies first and some of that work will wait until more work has gone in upstream to consolidate some of the current dependencies.

Kerberos is a great protocol for single sign-on authentication. It's supported by many protocols, allowing you to not have to enter a password to each and every one of them; instead, the protocols behind the scenes (not "just" Kerberos, but also the things that embed tickets, such as GSSAPI, or the things that embed GSSAPI, such as SASL or SPNEGO) use your ticket-granting-ticket to ask for security credentials for the service you're trying to use, magic happens, and you're authenticated. I blogged about kerberos before (even if it was ages ago); since then, I've not only used it on my own systems, but also on the systems of various customers. One thing I've learned in that time, however, is that most web application developers have a bad case of NIHilism when it comes to authentication. Most webservers that I've seen have a wide range of methods to do authentication in the webserver through various means, including things like certificate-based authentication, one-time password modules, and, yes, kerberos. Yet almost no webapp out there will look at the magic variables that those webservers set to explain we're authenticated, instead reinventing the wheel through webforms and other various stupid means. Sigh. So, that means, no kerberos authentication for webapps. Worse, if the application has no way to pass on authentication to something external, that means users will now have to learn another password: one for Kerberos, one for the webapp. And, probably, one for this other webapp, too -- because once you add one webapp, people expect you to add more of them. Well, mostly. In some cases, webapps do have ways to externalize authentication. In most cases this means "store passwords in a database", or "try authenticating against this other service here". When "this other service here" is an IMAP server, then all you need to do is make sure cleartext authentication on the IMAP server eventually ends up trying to authenticate against the Kerberos server, and you're all set. When "this other service here" is an LDAP server, however, you're out of luck. Right? It turns out that no, you're not. I recently learned that OpenLDAP can, in fact, check "simple" bind requests by checking some other service, and that this other service can be a Kerberos realm. Doing so is called "Pass-through authentication" in the OpenLDAP documentation, and this is how you do it:

• Add a "userPassword" attribute to the user who will authenticate, with a value of the form " SASL principal@KERBEROS.REALM". That is, take the user's principal name, tack " SASL " in front of it, and put that in the userPassword attribute.
• Make sure you have saslauthd installed and running, with the kerberos5 mechanism active. In Debian, that means you have to install sasl2-bin, and edit /etc/default/saslauthd, so it has the variable START set to yes (yuck), and the variable MECHANISMS set to kerberos5.
• Create a file /etc/ldap/sasl/slapd.conf, and add the following contents:

mech_list: plain
pwcheck_method: saslauthd

This directs the SASL libraries, loaded by slapd, to talk to saslauthd when trying to authenticate some SASL things - Make sure slapd has the correct permissions to access the saslauthd unix domain socket. On Debian, that means you need to add the "openldap" user to the "sasl" group:

 adduser openldap sasl 

(obviously that won't be active until the next slapd restart) And that's it; if you now try a simple bind against the LDAP directory, and enter your Kerberos password, you should be in. If it doesn't work, try running "testsaslauthd"; if that works, it means the error is in your slapd configuration. If it doesn't, then the problem is in saslauthd. Some notes:

• this is not the same thing as using Kerberos authentication for LDAP. OpenLDAP has the ability to allow SASL binds for LDAP, which is way more interesting (since it allows authentication to actually be done by Kerberos rather than by "entering a password"); instead, with this method, it is possible to verify a password for a simple bind against a Kerberos password.
• Since this can't use hashed passwords, this method is inherently insecure. Only use it if all else fails, and for the love of $DEITY, at least make sure all the network connections which are going to contain such passwords are encrypted with SSL or TLS or similar. Otherwise, everyone who can sniff anything on your network will learn all the Kerberos passwords, which is a very very bad thing.

Oh, and if you're a webapp developer: please please please make it easy for me to use an external authentication mechanism. This isn't hard; all you need is a separate page that will read the magic variables (probably REMOTE_USER if you're using apache), set a session variable or whatever, and then redirect to your normal "we've just logged in" page. ikiwiki gets this right; you can too! (for added bonus points, have some way to declare a mapping from REMOTE_USER values to internal, readable, usernames, but it's not too crucial)

I used to host my internet infrastructure at Hurricane Electric. It all started in october 1998 with POP3/SMTP, HTTP and DNS. In the coming years, I began to host all services except DNS on my own. But I kept using he.net for its DNS management interface. It was dead simple, and therefore accessible. All they had was a basic textarea with BIND alike configuration in it. I could log in to their admin interface and change my DNS records as desired. A few years ago, they auto-upgraded my account to their new shiny DNS panel, which, surprise surprise, is no longer accessible with a simple text browser. After a bit of bitching with support, they ended up downgrading my account back to the old functionality, so I was happy again. However, as you might guess, last time I needed to change a DNS record, I found that the DNS panel has been ugpraded yet again and is again no longer accessible to me. So it was time to leave the sinking ship. But I needed to find an accessible DNS hosting service. Not an easy task, given that everyone seems to do more or less the same thing these days.

A quick roundup of things I made during the last months, following tutorials here and there. 1: a soft bunny more or less based on this one 2: skelly man! You can find pattern and instructions on Chez Beeper Bebe's blog 3: bibs: the embroidered one is based on this pattern by Charlotte Lyons on SouleMama's blog, the other re-uses the same pattern but add a bird applique instead. 1, 4: my first try with the beautiful Weekender Bag ended with something else entirely: apparently, where euclidean geometry principles apply, cutting a piece of fabric 10 cm smaller will result in a bag 10 cm smaller!
Go figure! Fhtang! I'm happy anyway with the final result, but it resembles more a tote bag than a weekender one. I then made also a successful Weekender, but haven't taken a pic yet. If you try that pattern, you may find this blogpost helpful. 2: This is my try at copycatting this model without a pattern and didn't end very well.
However, a couple lessons learned:

• you don't improvise armholes and sleeves: you have to carefully plan them. I know this now, but at the time I was just a kid with a crazy dream ;)
• a-line dresses with straight necklines are definitely not meant for my body type. no matter how cute they look on someone else.
• I really should hack my mannequin, so to have it of my actual size. Or I will continue sew clothes fitting it, and not me.

3: Blouse with knit-stretchy stuff! (You love me when I speak technical, don't you?)
I did that without pattern, mostly using a similar top I have as inspiration/guide

And today from our ever-growing collection of what the fuck is wrong with you people?!... This is wrong on so many levels, I can't even begin to describe it. Sadly, it seems that this will get funded. And if it does not, technology will only become cheaper over time...

I ve just been given an Armourdillo Hybrid case for the Nexus 5 [1] to review. The above pictures show the back of the case, the front of the case, the stand, and the front of the case with the screen blank. When I first photographed the case the camera focused on a reflection of the window, I include that picture for amusement and to demonstrate how reflective the phone screen is. This case is very hard, the green plastic is the soft inner layer which is still harder than the plastic in a typical gel case . The black part is polycarbonate which is very hard and also a little slippery. The case is designed with lots of bumps for grip (a little like the sole of a running shoe) so it s not likely to slip out of your hand. But the polycarbonate slides easily on plastic surfaces such as the dash of a car. It s fortunate that modern cars have lots of cup holders that can be used for holding a phone. I haven t dropped the phone since getting the new case, but I expect that the combination of a hard outer shell and a slightly softer inner shell (to cushion the impact) will protect it well. All the edges of the case extend above the screen so dropping the phone face down on a hard flat surface shouldn t cause any damage. The black part has a stand for propping the phone on it s side to watch a movie. The stand is very solid and is in the ideal position for use on soft surfaces such as a doona or pillow for watching TV in bed. Appearance This case is mostly designed to protect the phone and the bumps that are used for grip detract from the appearance IMHO. I think that the Ringke Fusion case for my Nexus 4 [2] looks much better, it s a trade-off between appearance and functionality. My main criteria for this case were good protection (better than a gel case) and small size (not one of the heavy waterproof cases). It was a bonus to get a green case for the Enlightened team in Ingress. NB Armourdillo also offers a blue case for the Resistance team in Ingress as well as other colors. MobileZap also have a number of other cases for the Nexus 5 [3].

• [1] http://tinyurl.com/k8sy7ry
• [2] http://etbe.coker.com.au/2013/08/13/nexus-4-ringke-fusion-case/
• [3] http://www.mobilezap.com.au/34606/google/nexus-5/covers.htm

I am happy to announce the release of OASIS v0.4.3. OASIS is a tool to help OCaml developers to integrate configure, build and install systems in their projects. It should help to create standard entry points in the source code build system, allowing external tools to analyse projects easily. This tool is freely inspired by Cabal which is the same kind of tool for Haskell. You can find the new release here and the changelog here. More information about OASIS in general on the OASIS website. Here is a quick summary of the important changes:

• Added -remove switch to the setup-clean subcommand designed to remove unaltered generated files completely, rather than simply emptying their OASIS section.
• Translate path of ocamlfind on Windows to be bash/win32 friendly.
• Description is now parsed in a more structured text (para/verbatim).

Features:

• stdfiles_markdown (alpha): set default extension of StdFiles (AUTHORS, INSTALL, README) tp be '.md'. Use markdown syntax for standard files. Use comments that hides OASIS section and digest. This feature should help direct publishing on GitHub.
• disable_oasis_section (alpha): it allows DisableOASISSection to be specified in the package with a list of expandable filenames given. Any generated file specified in this list doesn't get an OASIS section digest or comment headers and footers and is therefore regenerated each time oasis setup is run (and any changes made are lost). This feature is mainly intended for use with StdFiles so that, for example, INSTALL.txt and AUTHORS.txt (which often won't be modified) can have the extra comment lines removed.
• compiled_setup_ml (alpha): allow to precompile setup.ml to speedup.

This new version closes 4 bugs, mostly related to parsing of _oasis. It also includes a lot of refactoring to improve the overall quality of OASIS code base. The big project for the next release will be to setup a Windows host for regular build and test on this platform. I plan to use WODI for this setup. I would like to thanks again the contributor for this release: David Allsopp, Martin Keegan and Jacques-Pascal Deplaix. Their help is greatly appreciated.

For years, I've used a gmail account to subscribe to public mailing lists because it was convenient, it had excellent spam filtering and since the lists were already available on google-indexed public archive I felt that there was no real privacy issue.
I already have a local copy of every message in the account, in case I lose access to it for some random reason, and my real contacts already know and use my other email address(es), so I'm not really worried about continuing to use gmail for this kind of traffic.

When I started contributing to Debian I also used that email address, because I was already subscribed to a few mailing lists under it, and it was going to end up being used on the BTS and other spam-attracting public places like that, but I was less happy with the choice.

Lately, however, gmail's spam filters have started to behave oddly, with lots of false positives including a number of messages from @debian.org addresses, and this basically removes the main reason why I was using gmail in the first place, so I'm seriously considering dropping it also for publicly indicized communications.

My first step, right before leaving for Barcelona was to create a couple other email addresses, one for mailing lists and the other for debian-related work, and add them to my gpg/OpenPGP key, so that any new signature on them from the conf would also include the new UIDs.

If you've received a badly signed message (thanks heirloom mailix for mangling stuff :) ) followed by an hopefully correctly signed one where I was asking you to sign new UIDs, and you were wondering, this is the reason.

Now that the new UIDs have been signed by a few DDs I'm going to start using the new address in Debian: I believe I have changed it everywhere I needed to, except for the maintainer/uploader field for my packages, where it will change on their next uploads.

As for mailing lists, I still don't know how to proceed: I could proceed in a systematic way, but I'm lazy and I will probably just change a few main subscriptions, use the new address for new subscriptions and then migrate the rest a few at a time, unless some other event happens and I'll have to hurriedly migrate everything or risk losing important messages. Even in the latter case, I don't need any information that is only stored on google's servers, so I won't will panic just a bit :)

Now, where did I put that documentation about xmpp servers?

As usual, I look at the application stats for phpMyAdmin just after student application period of Google Summer of Code is over. First of all we got more proposals than in last years, this time there is way more students from India and discussions on mentors lists shows this is quite similar for other projects. Maybe it's just different timing which works better for students there, but there might be different reasons as well. There is also quite low number of spam or bogus proposals. Same as in past years, people leave the submission to the last moment, even though we encourage them to submit early so that they can adjust the application based on our feedback. Anyway we're just working on evaluation and will finalize it in upcoming days. Of course you will know the results from Google on April 21st.

Filed under: English phpMyAdmin 0 comments Flattr this!

Until this time next month I'll be posting code-based discussions only. Recently I've been wanting to explore creating clustered services, because clusters are definitely things I use professionally. My initial attempt was to write an auto-clustering version of memcached, because that's a useful tool. Writing the core of the service took an hour or so:

• Simple KeyVal.pm implementation.
• Give it the obvious methods get, set, delete.
• Make it more interesting by creating a read-only append-log.
• The logfile will be replayed for clustering.

At the point I was done the following code worked:

use KeyVal;
# Create an object, and set some values
my $obj = KeyVal->new( logfile => "/tmp/foo.log" );
$obj->incr( "steve" );
$obj->incr( "steve" );
print $obj->get( "steve" ) # prints 2.
# Now replay the append-only log
my $replay = KeyVal->new( logfile => "/tmp/foo.log" );
$replay->replay();
print $replay->get( "steve" ) # prints 2.

In the first case we used the primitives to increment a value twice, and then fetch it. In the second case we used the logfile the first object created to replay all prior transactions, then output the value. Neat. The next step was to make it work over a network. Trivial. Finally I wanted to autodetect peers, and deploy replication. Each host would send out regular messages along the lines of "Do you have updates made since $time?". Any that did would replay the logfile from the given unixtime offset. However here I ran into problems. Peer discovery was supposed to be basic, and I figured I'd write something that did leader election by magic. Unfortunately Perls threading code is .. unpleasant:

• I wanted to store all known-peers in a singleton.
• Then I wanted to create threads that would announce and receive updates.

This failed. Majorly. Because you cannot launch the implementation of a class-method as a thread. Equally you cannot make a variable which is "complex" shared across threads. I wrote some demo code which works without packages and a shared singleton:

• https://github.com/skx/peer-discovery

The Ruby version, by contrast, is much more OO and neater. Meh. I've now shelved the project. My next, big, task was to make the network service utterly memcached compatible. That would have been fiddly, but not impossible. Right now I just use a simple line-based network protocol. I suspect I could have got what I wanted using EventMachine, or similar, but that's a path I've not yet explored, and I'm happy enough with that decision.

This is a fairly small Perl module that provides a more convenient interface to the ANSI color escape sequences. The primary change in this release is interesting for me but not so much for anyone else. It's the first of my core Perl modules that I've converted to Module::Build and to the new Perl test infrastructure that's now maintained in rra-c-util. (Yes, I know that Module::Build is apparently going to be dropped from Perl core, but the package also generates a Makefile.PL for backward compatibility.) Starting with this release, all my subsequent package releases will start using the Lancaster Consensus environment variables to control whether to run non-default tests (namely AUTOMATED_TESTING, RELEASE_TESTING, and AUTHOR_TESTING). Hopefully this won't cause me too many problems. I'm currently setting AUTHOR_TESTING unconditionally, since I really want to see the results of those tests for all my code, but it's possible that will cause me too many problems with other people's code. (It would have been nice if the spec for AUTHOR_TESTING would let you set the value of the variable to the identity of the author whose tests you want run.) I like having all my release tests run by automated testing so that I can identify any problems with the code to conditionally skip them, so I enable all the release tests when AUTOMATED_TESTING is set. This is probably peculiar to me. The other changes in this release are all documentation and test suite fixes. There are no code changes in this release. Thanks to Olivier Mengu and David Steinbrunner for various bug reports. You can get the latest release from the Term::ANSIColor distribution page.