Packaging LAVA for Debian unstable with notes on other distributions I ve been building packages for LAVA on Debian unstable for several months now and I ve been running LAVA jobs on the laptop and on devices in my home lab and on an ARMv7 arndale too. Current LAVA installations use lava-deployment-tool which has only supported Ubuntu 12.04 LTS Precise Pangolin. There has been a desire in LAVA to move away from a virtual environment, to put configuration files in FHS compliant paths, to use standard distribution packages for dependencies and so to make LAVA available on more platforms than just precise. Packaging opens the door to installing LAVA on Debian, Ubuntu, Fedora and any other recent distribution. Despite LAVA currently being reliant on 12.04 Precise, some of the python dependencies of LAVA have been able to move forward using the virtual environment provided by builtout and pypi. This means that LAVA, as packaged, requires a newer base OS suite than precise for Ubuntu, the minimal base is Saucy Salamander 13.10 and for Debian it would be Jessie (testing) although there is currently a transition ongoing in Debian which means that uwsgi is not in testing and Debian unstable would be needed instead. The work to migrate configuration snippets out of deployment-tool and to ensure that the tarball built using setuptools contains all of the necessary files for the package has already been done. The packaging itself is clean and most of the work is done upstream. There is, as ever, more to do but the packages work smoothly for single install LAVA servers where the dispatcher is on the same machine as the django web frontend. The packages have also migrated to Django1.6, something which is proving difficult with the deployment-tool as it has not kept pace with the changes outside the virtual environment, even if other parts of LAVA have. LAVA will be switching to packages for installation instead of deployment-tool and this will mean changes to how LAVA works outside the Cambridge lab. When the time comes to swich to packaging, the plan is to update deployment-tool so that it no longer updates /srv/lava/ but instead migrates the instance to packages. Main changes

1. Configuration files move into /etc/
   • Device configuration files /etc/lava-dispatcher/devices/
   • Instance configuration files/etc/lava-server/
2. Log files move into /var/log/
   • Adding logrotate support no more multi-Gb log files in /srv/lava/
3. Commitment to keeping the upstream code up to date with dependencies
4. Support for migrating existing instances, using South.
5. Packaging helpers
   • add devices over SSH instead of via a combination of web frontend and SSH.
   • Developer builds with easily identifiable version strings, built as packages direct from your git tree.
6. New frontend
   • Although django1.6 does not change the design of the web frontend at all, LAVA will take the opportunity to apply a bootstrap frontend which has greater support for browsers on a variety of devices, including mobile. This also helps identify a packaged LAVA from a deployment LAVA.
7. Documentation and regular updates

The Plan LAVA has made regular releases based on a monthly cycle and these will be provided as source tarballs at http://www.linaro.org/downloads/ for distributions to download. The
official monthly release and any intervening updates will be made available for distributions to use for their own packaging. Additionally, Debian packages will be regularly built for use within LAVA and these will be available for those who choose to migrate from Ubuntu Precise to Debian Jessie. LAVA will assist maintainers who want to package LAVA for their distributions and we welcome patches from such maintainers. This can include changes to the developer build support script to automate the process of supporting development outside LAVA. Initially, LAVA will migrate to packaging internally, to prove the process and to smooth out the migration. Other LAVA instances are welcome to follow this migration or wait until the problems have been ironed out. The Issues

• Remote workers this is work to be completed during the migration to packaging within LAVA as well as pending work upstream on the internals of the connection between a remote worker and the master scheduler. Expect some churn in this area whilst the code is being finalised. A lava-worker package is being prepared which borrows enough code from lava-server to run the lava-scheduler-daemon on the remote worker, until such time as the remote worker communications are refactored.
• Integration there are plans to integrate some of the modules which LAVA uses which are not commonly packaged: linaro-dashboard-bundle and linaro-django-xmlrpc
• OpenID the certificates which underpin OpenID use with Launchpad have been removed in Debian and LAVA is currently investigating alternatives.
• json-schema-validator currently has a broken test suite, so this will need to be patched by LAVA to allow the package to build. The code may be replaced with a different validator if the issue persists.
• namespace handling the current package install is unnecessarily noisy with complaints about the lava namespace. This will need a fix but does not affect current operation.
• Unit tests LAVA is working hard to add a much larger coverage of internal unit tests. These use a temporary database which is not generally available during the build of a distribution package. LAVA is already running continuous integration tests to ensure that these tests continue to pass and the packages will gain documentation on how to run these tests after installation.

Build Dependencies For Debian unstable, the list of packages which must be installed on your Debian system to be able to build packages from the lava-server and lava-dispatcher source code trees are:

debhelper (>= 8.0.0) python   python-all   python-dev   python-all-dev 
python-sphinx (>= 1.0.7+dfsg)   python3-sphinx python-mocker 
python-setuptools python-versiontools

(python-versiontools may disappear before the packages are finalised) In addition, to be able to install lava-server, these packages need to be built from tarballs released by Linaro (the list may shorten as changes upstream are applied)

linaro-django-pagination, 
python-django-restricted-resource (>= 0.2.7), 
lava-tool (>= 0.2), lava-utils-interface (>= 1.0), 
linaro-django-xmlrpc (>= 0.4),
python-versiontools (>= 1.8),
python-longerusername,
linaro-dashboard-bundle (>= 1.10.2), 
lava-dispatcher (>= 0.33.3)
lava-coordinator, lava-server-doc

The list for lava-dispatcher is currently:

python-json-schema-validator, lava-tool (>= 0.4), 
lava-utils-interface, linaro-dashboard-bundle (>= 1.10.2), 

The packages available from my experimental repository are using a new packaging branch of lava-server and lava-dispatcher where we are also migrating the CSS to Bootstrap CSS. Installing LAVA on Debian unstable
$ sudo apt-get install emdebian-archive-keyring
Add the link to my experimental repository (amd64, i386, armhf & arm64) to your apt sources, e.g. by creating a file /etc/apt/sources.list.d/lava.list containing:

deb http://people.linaro.org/~neil.williams/lava sid main
Update with the new key

$ sudo apt-get update
It is always best to install postgresql first:

$ sudo apt-get install postgresql
There are then three options for the packages to install (Please be careful with remote worker setup, it is not suitable for important installations at this time.):

1. Single instance, server and dispatcher with recommended tools
   apt-get install lava
   Installs linaro-image-tools and guestfs tools.
2. Single instance, server and dispatcher
   apt-get install lava-server
   Installs enough to run LAVA on a single machine, running jobs on boards on the same LAN.
3. Experimental remote worker support
   apt-get install lava-worker
   Needs a normal lava-server installation to act as the master scheduler but is aimed at supporting a dispatcher and boards which are remote from that master.

The packages do not assume that your apache2.4 setup is identical to that used in other LAVA installations, so the LAVA apache config is installed to /etc/apache2/sites-available/ but is not enabled by default. If you choose to use the packaged apache config, you can simply run:

$ sudo a2ensite lava-server
$ sudo apache2ctl restart
(If this is a fresh apache install, use a2dissite to disable to default configuration before restarting.) Information on creating a superuser, adding devices and administering your LAVA install is provided in the README.Debian file in lava-server:

$ zless /usr/share/doc/lava-server/README.Debian.gz
Provisos and limitations Please be aware that packaged LAVA is still a work-in-progress but do let us know if there are problems. Now is the time to iron out install bugs and other issues as well as to prepare LAVA packages for other distributions. It will be a little while before the packages are ready for upload to Debian I ve got to arrange the download location and upload the dependencies first and some of that work will wait until more work has gone in upstream to consolidate some of the current dependencies.

In last weeks, I've finally managed to create my personal gallery, where I could present selection of my photographs. This task was outstanding on my todo list for pretty long time, the major obstacle being selecting photos which to present there. During my sickness few weeks ago, I finally went through most of my photos and chosen the ones which I think are worth presenting. Still I did not manage to go through photos from half year spent in Japan, so these will be added sometimes later. Also most of the pictures miss descriptions, though I'm (slowly) fixing this. From technical side, the gallery is being generated using lazygal with custom theme. You can check the gallery on photos.cihar.com, I hope you will like it :-).

Filed under: English Photography 1 comments Flattr this!

One of things I missed for quite a long time in Lazygal compared to lot of other galleries is navigation using keys in between the images. Well, it should be easy to implement, shouldn't it? A little bit of googling and reading jQuery documentation and the code is here :-). It seems to work fine in browsers which I use, you can try it yourself.

As I produce quite a lot of photos and want to manage them in some reasonable way, I started to look for some tools which can do commenting/tagging/whatever of taken photos. I really don't want these meta data stored outside picture itself, so most of photo managers currently available are not acceptable. When reading Gnome 2.20, I noticed support for XMP (Extensible Metadata Platform) in eog. Well I didn't know this standard up to now, but it looks interesting. Especially when open source tools can easily adopt library open sourced by Adobe. I would like to add support for this to Lazygal, but I didn't find any reasonable Python module to work with XMP. There is simple extractor, but nothing what would allow me to easily grab meta data out of file. In fact one solution seems to exist, but it costs almost $300 and is not free software in any way. Or did I just miss something when using Google and some such Python library exists?

After searching for new gallery and investigating lazygal, I finally decided it's time to give it some real world usage. My current album from Japan is being processed by this damn fast tool. It needed a bit of hacking, but as the core was already there and Alexandre Rossi is quite cooperative upstream. I managed to implement almost everything I requested in original post. The only thing I'm still not completely happy with is the theme, but I hope I will improve it over time. From original requirements, I completely dropped links to full size images. There is simply no reason to put here crappy pictures which my only camera I currently have here (built in camera in Nokia 6234) produces. Also once I'll buy new camera (what will be most likely Pentax K10D), I probably won't upload huge 10Mpix images on web server as I don't think it would be good for anything else than wasting my bandwidth.

I received number of reactions on my photoalbum post. Several were saying "I'm using some great Perl tool which is not actively maintained anymore.". Well I know such tools. I also used BINS some time ago, but it did break something (I don't recall what exactly right now, it's quite far far away in past) and I was unable to fix it. That was reason I switched to Matew. The other replies were much better. One of them recommended trying lazygal (well it was recommended by it's author). I gave it a try and it seemed to fit my basic needs quite well and extending it's EXIF support was a piece of cake. I will continue to tune it to my needs, but I'm pretty close right now. The hardest thing for me will be to come up with some good looking themes, but I will most likely steal my ones for Matew :-).