Problem description: One of my customers had a problem with their
Selenium tests in the
Jenkins continuous integration system. While Perl s
Test::WebDriver still worked just fine the Selenium tests using Ruby s
selenium-webdriver suddenly reported failures. The problem was caused by Debian wheezy s upgrade of the Iceweasel web browser. Debian originally shipped Iceweasel version 17.0.10esr-1~deb7u1 in wheezy, but during a security-update version 24.3.0esr-1~deb7u1 was brought in through the wheezy-security channel. Because the selenium tests are used in an automated fashion in a quite large and long-running build pipeline we immediately rolled back to Iceweasel version 17.0.10esr-1~deb7u1 so everything can continue as expected. Of course we wanted to get the new Iceweasel version up and running, but we didn t want to break the existing workflow while working on it. This is where on-demand slave selection through labels comes in.
Basics: As soon as you re using Jenkins slaves you can instruct Jenkins to run a specific project on a particular (slave) node. By attaching labels to your slaves you can also use a label instead of a specific node name, providing more flexibility and scalability (to e.g. avoid problems if a specific node is down or you want to scale to more systems). Then Jenkins decides which of the nodes providing the according label should be considered for job execution. In the following screenshot a job uses the selenium label to restrict its execution to the slaves providing selenium and currently there are two nodes available providing this label:
TIP 1: Visiting $JENKINS_SERVER/label/$label/ provides a list of slaves that provide that given $label (as well as list of projects that use $label in their configuration), like:
TIP 2: Execute the following script on $JENKINS_SERVER/script to get a list of available labels of your Jenkins system:
import hudson.model.*
labels = Hudson.instance.getLabels()
labels.each label -> println label.name
Solution: In the according customer setup we re using the
swarm plugin (with automated Debian deployment through
Grml s netscript boot option, grml-debootstrap + Puppet) to automatically connect our Jenkins slaves to Jenkins master without any manual intervention. The swarm plugin allows you to define the labels through the
-labels command line option.
By using the
NodeLabel Parameter plugin we can configure additional parameters in Jenkins jobs: node and label . The label parameter allows us to execute the jobs on the nodes providing the requested label:
This is what we can use to gradually upgrade from the old Iceweasel version to the new one by keeping a given set of slaves at the old Iceweasel version while we re upgrading other nodes to the new Iceweasel version (same for the selenium-server version which we want to also control). We can include the version number of the Iceweasel and selenium-server packages inside the labels we announce through the swarm slaves, with something like:
if [ -r /etc/init.d/selenium-server ] ; then
FLAGS="selenium"
ICEWEASEL_VERSION="$(dpkg-query --show --showformat='$ Version ' iceweasel)"
if [ -n "$ICEWEASEL_VERSION" ] ; then
ICEWEASEL_FLAG="iceweasel-$ ICEWEASEL_VERSION%%.* "
EXTRA_FLAGS="$EXTRA_FLAGS $ICEWEASEL_FLAG"
fi
SELENIUM_VERSION="$(dpkg-query --show --showformat='$ Version ' selenium-server)"
if [ -n "$SELENIUM_VERSION" ] ; then
SELENIUM_FLAG="selenium-$ SELENIUM_VERSION%-* "
EXTRA_FLAGS="$EXTRA_FLAGS $SELENIUM_FLAG"
fi
fi
Then by using -labels $FLAGS EXTRA_FLAGS in the swarm invocation script we end up with labels like
selenium iceweasel-24 selenium-2.40.0 for the slaves providing the Iceweasel v24 and selenium v2.40.0 Debian packages and
selenium iceweasel-17 selenium-2.40.0 for the slaves providing Iceweasel v17 and selenium v2.40.0.
This is perfect for our needs, because instead of using the selenium label (which is still there) we can configure the selenium jobs that should continue to work as usual to default to the slaves with the
iceweasel-17 label now. The development related jobs though can use label
iceweasel-24 and fail as often as needed without interrupting the build pipeline used for production.
To illustrate this here we have slave selenium-client2 providing Iceweasel v17 with selenium-server v2.40. When triggering the production selenium job it will get executed on selenium-client2, because that s the slave providing the requested labels:
Whereas the development selenium job can point to the slaves providing Iceweasel v24, so it will be executed on slave selenium-client1 here:
This setup allowed us to work on the selenium Ruby tests while not conflicting with any production build pipeline. By the time I m writing about this setup we ve already finished the migration to support Iceweasel v24 and the infrastructure is ready for further Iceweasel and selenium-server upgrades.
Registration is now open for DebConf8, which will take place in Mar del Plata from Sunday 10 to Saturday 16 August 2008.
To register follow one of the processes outlined below:
I get all excited when I load up 
This weekend I received my 17th certification (or Statement of
Accomplishment) for MOOCs. In particular, this last MOOC that I took was
Of course, my interest also made me want to streamline the build process and
use the command line tools (with ant, another tool that was "alien" to me,
having used makefiles for everything that I needed).
The use of Eclipse (and the Android development environment in general) in
my Debian unstable computers
I was pretty knackered today. I should have done some Debian stuff, but I just
didn't have it in me, and I had a backlog of household stuff to get done.
Zoe woke up at 4am, and I didn't have the energy to try and get her to go back
to sleep in her bed, so I just let her sleep in my bed. We both slept in until
7am, which was nice.
As a result, we didn't have as much time to have a slow start, and Zoe was a
bit grumpy and uncooperative as a result. I think there was at least one
meltdown before breakfast.
It conveniently rained around the time we were ready to leave, so we drove to
Kindergarten. Drop off was super smooth. She pretty much waved me off as soon
as we got there.
I picked up a couple of packages from the post office on the way home, and then
started hanging out the washing before I had to go back to see the podiatrist
to get my orthotics fitted in my new running shoes. I finished hanging out the
washing and putting away stuff from the Coochie trip and Melbourne trip and had
some lunch.
After lunch I started work on making a little step for Zoe so she can turn the
light off in her bedroom. I had enough material left over from the clothes
lines I made for her to make a really dodgy little "stool".
It rained again around pick up time, so I drove to Kindergarten again, and
picked up Zoe. She'd just woken up from a nap before I got there and was in a
good mood. Megan's Dad was picking up Megan on foot, and they were going to
have a coffee at the local coffee shop, so we joined them.
After that, we went to the post office to check my post office box. I had a
cheque that needed to be banked, so we went to the bank and the supermarket,
and by then it was pretty much time for Sarah to pick up Zoe from me.
I did a bit more carpentry before I lost the light, and went to yoga.
I ve slowly come to the conclusion that I should likely not work as much as I do. Looking back over my GitHub graphs, and feeling my own internal pressure to do more work is getting to the point of insanity. I can t remember the last time I took a whole day off.
As a result, I m going to make a new rule for myself, and put it in a semi-public place to help with peer-pressure enforcing this.
I m not going to use my computer (except in vary rare and urgent circumstances) on Sundays, from here on out. The exact rules are still up in the air (Is the Phone OK? Tablet?)
Let s see how this goes! I ll start next week (or just stop doing things today)
If you ever find yourself doing a bit of technical support for relatives
over the phone, there's nothing like actually seeing what they are doing on
their computer. One of the best tools for such remote desktop sharing is
A new bug-fix / maintenance release of
My hard-working KDE friend Mario asked me to help him to get Debian people to help him with his thesis. Here is what he writes:
Dear Free Software contributor*
I m currently in the process of writing my diploma thesis. I ve worked hard during the last few weeks and months on a 
First step will be getting a NetBSD ISO image for installation
purpose. It can be downloaded from 
So you can see no response header in the above image but there is a
response and below how is response content visible on firebug.
Still weird part was when using wget/curl directly to get the CSS I
was getting correct reply and the file, which is puzzling.
So I went ahead opened the CSS url directly using browser and saved
the resulting file. When I used file command on it, output
suggested the saved file in gzip compressed!. I went ahead
uncompressed it and opened the file and inside it I found response
header and the response body! So what just happened? Did apache some
how manage to compress entire respone not just response body?
The more puzzling question was why did it work when I was using
mod_proxy_uwsgi and failed when I switched to mod_uwsgi. I had no
clue at this point why this behavior is coming up.
So response is still gziped but this time only response body was
gziped and Content-Encoding field was set properly which makes browser
to properly uncompress the response body and use it.
So it seems like there is a difference between mod_uwsgi and
mod_proxy_uwsgi implementation. mod_uwsgi was sending the response
along with response headers which was compressed by mod_deflate there
by rendering browser helpless to interpret the response. But
mod_proxy_uwsgi seems to be only sending the response content without
response body and this content was later compressed by mod_deflate and
proper response headers were set by apache before sending it to
browser.
So now whose fault is this? Is it the bug in mod_uwsgi or, mod_deflate
not being able to exclude response headers from compression? I've no
clue! If you have a clue and you want to share it with me, please
consider writing to me over email. My details are available in
Thanks to bernat on #debian @ irc.debian.org for 
