When you read this,
GNOME Shell 3.2 will
(hopefully!) have finally transitioned to Debian s
testing suite.
Planet GNOME readers might think
Debian now has outdated versions of software even in their development
versions, or the distribution s development marches at glacial pace.
Wheezy GNOME users will finally have a Shell that matches the rest
of their GNOME components, something that works with the
Shell extensions website and much
less problems and limitations compared to 3.0.2.
The reality is that GNOME 3.2 s packaging was quite ready back when it
was released in late September, but a number of not-so-desirable situations
held GNOME Shell from transitioning to testing until today, four months
later. So, what happened?
TL;DR: transitioning from GNOME 2 GNOME 3 is not so easy if you
want to keep testing
in a sane state, and when you need to deal with
dozens of indirectly related packages, for more than 10 architectures but it
shouldn t take nearly a full year
, either
Let s go back to the last months of 2010. Debian
squeeze is in
very deep freeze, and the release team and many Debian developers are
focusing on squashing as many release critical bugs as they can, in order to
make Debian 6.0 the great release it ended up being. The GNOME project has
recently delayed the big launch of
GNOME 3.0 again, until
March 2011; Debian has already settled on GNOME 2.28 for its release,
although it will end up cherry-picking many updates from the 2.30 release
modules.
With most of the stabilization work being done, many Debian GNOME team
members were at that time working on packaging very early versions of what
would end up being GNOME 3.0 technology: GTK+3.0, GNOME Shell, Mutter and
some brave users even tried to use it via the experimental archive.
On February 6th, Debian 6.0 was released, and soon after, on April 6th,
GNOME made a huge step forward with the much anticipated
release of GNOME 3.0.
At that time, Debian developers were busy breaking
unstable as
much as they could, as it s tradition on the weeks following a major
release, and the Debian GNOME team was able to start moving some GNOME 3.0
libraries (those which were parallel-installable with their GTK+2.0
versions) to unstable.
However, moving the bulk of GNOME 3.0 to unstable wasn t so easy. When
you start doing that, you need to be sure you re ready to have all affected
packages in a transitionable state as soon as possible, to minimise the
chances of blocking transitions of unrelated packages via the dependencies
they pick up with rebuilds. All the packages involved in a transition need
to be ready to go in the same testing run , for all supported
architectures. When you re dealing with dozens of GNOME source packages at
the same time, many of which introduce new libraries, or worse, introduce
incompatible APIs that affect many more unrelated packages, things get
hairy, and
you need a plan.
So,
Joss outlined what a sane
approach to this monster transition could look like. The amount of work to
do was what we call fun on
#debian-gnome. In a nutshell, we
had to deal with quite a few transitions, starting with
having a newer version of libnotify in unstable, and a pre-requisite for
that was making sure all the packages using libnotify1 were ready to use the
source-incompatible libnotify4, and this meant preparing patches and NMUs
for many of our packages, as well as many others not under our control.
Before starting a controlled transition like this one, we had to get an
ACK from the release team, who was busy enough handling other huge
transitions like Perl 5.12, so by the time we got our own slot, we were well
into Summer.
With libnotify done in August, it was time to get our hands dirty with
more exciting stuff, like getting Nautilus in testing. This meant bumping a
soname and requiring all packages providing Nautilus extensions to migrate
to GTK+3.0, or drop the extension entirely, as you can t mix GTK+2.0 and
GTK+3.0 symbols in the same process. However, in GNOME 3.0, automounting
code had moved from Nautilus to gnome-settings-daemon, so in order to not
break filesystem automounting in testing for an unreasonable amount of time,
both Nautilus and g-s-d needed to go in at the same time. The fun thing is
that g-s-d dragged glib2.0, gvfs, gnome-control-center, gdm3, gnome-media,
gnome-session and gnome-panel into the equation, so this transition needed
extra planning and a lot more work than initially expected: migrating all
nautilus extensions, plus ensuring all Panel applets had migrated to GTK+3.0
and the new libpanel-applet-4 interface. In short, this was the
monster transition
we were trying to avoid.
By the time all this mess was sorted out, GNOME 3.2 had been released,
and for what users said, it was
a lot better than 3.0. We still had
no more than a few bits and pieces of 3.0 in testing, and we were working
hard to get 3.0 in wheezy. With all the excitement around 3.2, at times it
was difficult to explain outsiders why we were beating a dead 3.0 horse
Going back to our huge transition, it was just a matter of time before all
the packages would be built and be ready to enter, on the same run, in
testing.
A few weeks later, in early November and after several rounds of
mass-bug-filings, fixing unrelated FTBFS, many NMUs, package removal
requests and dealing with any possible problem that could block our
transition, everything seemed to be set, and our release team magicians had
everything in place for the big magic to happen. However, our
first clash
with the rest of Debian happened a few hours before our victory, in the form
of an unannounced ruby-gnome2 upload which resetted the count for everyone.
It was fun to see the release team trying all sorts of black magic in an
attempt to mitigate the damage. Fortunately, after a few tries they managed
to fool britney (the script that handles package transitions from unstable
to testing) somehow, and the hardest part of the job was done with just one
day of delay.
At last, the core of GNOME 3 was in testing, and testing users found soon
after. The rest of the week saw a cascade of hate posts against GNOME 3 in
Planet Debian, and personally I
didn t find that especially motivating to keep on working on the rest of
GNOME bits. With experimental clear of GNOME 3.0 stuff, we finally were able
to focus on packaging whatever GNOME 3.2 components were not already done,
and preparing for what should be a plain simple transition of GNOME 3.0 to
3.2.
After our share of wait for a transition slot, as Perl 5.14, ICU and
OpenSSL were in the line before us, and after dealing with a minor tracker
0.12 transition, we were ready for our next episode:
evolution-data-server.
At first sight, we thought this would be a lot easier, but it still
got a bit hairy
due to evo-data-server massive soname bumps. We were
given our slot
just before Christmas, after a few weeks of wait for others to finish their
migration rounds, and most of the pack entered wheezy a few days before the
new year.
No rejoicing, though, as GNOME Shell 3.2 didn t make it. First, we
discovered it was FTBFS on kFreeBSD architectures, as NetworkManager had
been promoted from optional to required, for apparently no good reason,
leaving the BSD world in the cold, including our exotic GNU/kFreeBSD
architectures. Now, let s clarify that I m a supporter of the Debian
kFreeBSD architectures and was really happy to see it accepted as a
technology preview in squeeze. However, as you know, GNOME Shell currently
requires hardware acceleration to run, a requirement hardly met in kFreeBSD,
unless you re using a DRI1 X driver. We seriously doubted anyone had
ever ran a GNOME 3 session on kfreebsd-*. However, if it didn t
build, it was a blocker bug for GNOME Shell. We considered creating
different meta-packages for kFreeBSD architectures, to conclude it d be a
mess, so our awesome Michael Biebl ended up cooking up a patch that restored
the ability to build the Shell without NetworkManager support.
With this out of our way, we just needed to upload Michael s fix and
watch the buildds do their part of the job. Or maybe not?
Enter Iceweasel 9.
In parallel, and with incredible bad timing, Iceweasel 9.0 was uploaded
to Debian the very same day it was released by Mozilla. Again, it greeted us
with a nasty surprise: yet another mozjs API change, which made gjs FTBFS,
which meant our kFreeBSD fixes would be unusable until someone who knew Gjs
internals well enough bit the bullet and worked around the new API changes.
Again, Michael Biebl tried to be our saviour, but unfortunately wasn t able
to fix all the problems, so we tried to focus on plan B.
Mozilla had released a fork of the mozjs that is included in Firefox, so
that embedders would have a bit less of a hard time with these recurrent API
changes. This was based on Firefox 4, and was already being packaged by
Ubuntu. Gjs would build using this older version just fine, so we just
needed to get it in Debian as soon as possible. We just needed to find a
sucke^Wvolunteer that would be inclined to maintain the beast. Only after a
few weeks we managed to get Chris Coulson, the Ubuntu packager, to maintain
the package directly through the Debian archive via package syncs. However,
his package had only been auto-compiled in the three Ubuntu architectures,
that is amd64, armel and i386. It s late January 2012, and we ve been
fighting this war for 10 months.
After getting some help from Michael to get the new package in shape for
Debian standards, we were excited to sponsor it for Chris. Duh, after a few
days in the NEW fridge, it was rejected by the ftp-masters. The license
statement was missing quite a few details, so I went ahead and sacrificed a
few hours of my copious free time to get this sorted out. A few days later,
mozjs was accepted, but the result was horrible. It was
very red.
mozjs didn t build on half of our targets.
Mike Hommey was quick to
file a bug
and point us to the most obvious fuckups. As he had dealt with this in the
past as the Iceweasel maintainer, all of these issues were fixed and patches
were ready to be applied verbatim or with minimal changes to our sources.
With mozjs finally built successfully (although with
severe problems on ia64),
we were finally able to rebuild Gjs against it, upload GNOME Shell with our
kFreeBSD fixes and wait until today for this mess to be over. Whew.
I can t say I ve enjoyed all the stages of this ride. Some bumps on the
road were clearly there to test our patience, but it has helped me get back
in touch with non-leaf GNOME packaging, which was all I was doing for a
while due to being super-busy lately with studies. It also reminds me of the
privilege of working side by side with some awesome people, not only Joss,
Michael, Sjoerd, Laurent or Gustavo, to name just a few Debian GNOME team
members, but also the receptive release team members like Julien or Cyril,
and NEW-processing record-breaking ftp-master Luca. Without them, we might
be trying to figure out the Nautilus transition since last Summer.
We really hope GNOME 3.4 will be a piece of cake compared to this. ;)
Apart from being somewhat slow, one of the downsides of the
original Raspberry Pi SoC was that it had an old ARM11 core which
implements the ARMv6 architecture. This was particularly
unfortunate as most common distributions (Debian, Ubuntu, Fedora,
etc) standardized on the ARMv7-A architecture as a minimum for
their ARM hardfloat ports. Which is one of the reasons for Raspbian
and the various other RPI specific distributions.
Happily, with the new Raspberry Pi
2 using Cortex-A7 Cores (which implement the ARMv7-A
architecture) this issue is out of the way, which means that a a
standard Debian hardfloat userland will run just fine. So the
obvious first thing to do when an RPI 2 appeared on my desk was to
put together a quick Debian Jessie image for it.
The result of which can be found at: https://images.collabora.co.uk/rpi2/
Login as root with password debian (Obviously do change the
password and create a normal user after booting). The image is 3G,
so should fit on any SD card marketed as 4G or bigger. Using
bmap-tools
for flashing is recommended, otherwise you'll be waiting for 2.5G
of zeros to be written to the card, which tends to be rather
boring. Note that the image is really basic and will just get you
to a login prompt on either serial or hdmi, batteries are very much
not included, but can be apt-getted :).
Technically, this image is simply a Debian Jessie debootstrap
with a extra packages for hardware support. Unlike Raspbian the
first partition (which contains the firmware & kernel files to
boot the system) is mounted on /boot/firmware rather then on /boot.
This is because the VideoCore expects the first partition to be a
FAT filesystem, but mounting FAT on /boot really doesn't work right
on Debian systems as it contains files managed by dpkg (e.g. the
kernel package) which requires a POSIX compatible filesystem.
Essentially the same reason why Debian is using /boot/efi for the
ESP partition on Intel systems rather the mounting it on /boot
directly.
For reference, the RPI2 specific packages in this image are from
https://repositories.collabora.co.uk/debian/
in the jessie distribution and rpi2 component (this repository is
enabled by default on the image). The relevant packages there
are:
There s a lot of discussion / moaning /arguing at this time, so I thought I d post something about how LAVA got into Debian Jessie, the work involved and the lessons I ve learnt. Hopefully, it will help someone avoid the disappointment of having their package missing the migration into a future stable release. This was going to be a talk at the Minidebconf-uk in Cambridge but I decided to put this out as a permanent blog entry in the hope that it will be a useful reference for the future, not just Jessie.
Context
LAVA relies on a number of dependencies which were at the time all this started NEW to Debian as well as many others already in Debian. I d been running LAVA using packages on my own system for a few months before the packages were ready for use on the main servers (I never actually installed LAVA using the old virtualenv method on my own systems, except in a VM). I did do quite a lot of this on my own but I also had a team supporting the effort and valuing the benefits of moving to a packaged system.
At the time, LAVA was based on Ubuntu (12.04 LTS Precise Pangolin) and a new Ubuntu LTS was close (Trusty Tahr 14.04) but I started work on this in 2013. By the time my packages were ready for general usage, it was winter 2013 and much too close to get anything into Ubuntu in time for Trusty. So I started a local repo using space provided by Linaro. At the same time, I started uploading the dependencies to Debian. json-schema-validator, django-testscenarios and others arrived in April and May 2014. (Trusty was released in April). LAVA arrived in NEW in May, being accepted into unstable at the end of June. LAVA arrived in testing for the first time in July 2014.
Upstream development continued apace and a regular monthly upload, with some hotfixes in between, continued until close to the freeze.
At this point, note that although upstream is a medium sized team, the Debian packaging also has a team but all the uploads were made by me. I planned ahead. I knew that I would be going to Macau for Linaro Connect in February a critical stage in the finalisation of the packages and the migration of existing instances from the old methods. I knew that I would be on vacation from August through to the end of September 2014 including at least two weeks with absolutely no connectivity of any kind.
Right at this time, Django1.7 arrived in experimental with the intent to go into unstable and hence into Jessie. This was a headache for me, I initially sought to delay the migration until after Jessie. However, we discussed it upstream, allocated time within the busy schedule and also sought help from within Debian with the RFH tag. Rapha l Hertzog contributed patches for django1.7 support and we worked on those patches upstream, once I was back from vacation. (The final week of my vacation was a work conference, so we had everyone together at one hacking table.)
Still there was more to do, the django1.7 patches allowed the unit tests to complete but broke other parts of the lava-server package and needed subsequent tweaks and fixes.
Even with all this, the auto-removal from testing for packages affected by RC bugs in their dependencies became very important to monitor (it still is). It would be useful if some packages had less complex dependency chains (I m looking at you, uwsgi) as the auto-removal also covers build-depends. This led to some more headaches with libmatheval. I m not good with functional programming languages, I did have some exposure to Scheme when working on Gnucash upstream but it wasn t pleasant. The thought of fixing a scheme problem in the test suite of libmatheval was daunting. Again though, asking for help, I found people in the upstream team who wanted to refresh their use of scheme and were able to help out. The fix migrated into testing in October.
Just for added complications, lava-server gained a few RC bugs of it s own during that time too fixed upstream but awkward nonetheless.
Achievement unlocked
So that s how a complex package like lava-server gets into stable. With a lot of help. The main problem with top-level packages like this is the sheer weight of the dependency chain. Something seemingly unrelated (like libmatheval) can seriously derail the migrations. The package doesn t use the matheval support provided by uwsgi. The bug in matheval wasn t in the parts of matheval used by uwsgi. It wasn t in a language I am at all comfortable in fixing but it s my name on the changelog of the NMU. That happened because I asked for help. OK, when django1.7 was scheduled to arrive in Debian unstable and I knew that lava was not ready, I reacted out of fear and anxiety. However, I sought help, help was provided and that help was enough to get upstream to a point where the side-effects of the required changes could be fixed.
Maintaining a top-level package in Debian is becoming more like maintaining a core package in Debian and that is a good thing. When your package has a lot of dependencies, those dependencies become part of the maintenance workload of your package. It doesn t matter if those are install time dependencies, build dependencies or reverse dependencies. It doesn t actually matter if the issues in those packages are in languages you would personally wish to be expunged from the archive. It becomes your problem but not yours alone.
Debian has a lot of flames right now and Enrico encouraged us to look at what else is actually happening in Debian besides those arguments. Well, on top of all this with lava, I also did what I could to help the arm64 port along and I m very happy that this has been accepted into Jessie as an official release architecture. That s a much bigger story than LAVA yet LAVA was and remains instrumental in how arm64 gained the support in the kernel and various upstreams which allowed patches to be accepted and fixes to be incorporated into Debian packages.
So a roll call of helpers who may otherwise not have been recognised via changelogs, in no particular order:
When I was first approached about this, I thought Collabora was a small
company. But as I looked more into it, I discovered that's not longer the
case, there's many more people than I imagined working there (here!), and
was delighted to see I knew many of them, and many other are well known
members of the major Free Sofware communities. I'll be joining the
sysadmin team to work closely with
What Zappa was working on will likely always remain a
mystery.
The blue impulses is the per-frame size (scaled), the green line
the bitrate average over 10 frames and the red line the target
bitrate. Clearly this isn't an ideal picture, we will need to do
some more tweaks to our default settings to make x264 behave the
way we want. The same will be true for all other codecs and codec
elements, at the moment the GStreamer VP8 element doesn't handle
run-time bitrate changes at all and the Theora element acts it its
very own interesting way (
The screenshot isn't very exciting, which is exactly how it
should be. In the world i would like to live in people don't have
to care about video codecs, whether it is for video calls, playing
online media or whatever else they want to do with video. Looking
at the list of 
A Belorussian translation of this article is available 
I can recommend some feline assistance whenever installing
things on an ExoPC.
Oh well, guess it's time for a nice walk to get some fresh new
inner tubes.
So, I spent an hour or two this afternoon following the iPhone 4 liveblog.
It all looked fairly compelling (the screen!), right up until Steve's 'and
one more thing': video calling.
The screenshot is featuring myself in Collabora's UK office
(top-left), Robert McQueen from a hotelroom in Taipei (top-right),
Will Thompson and Magical trevor also from the Collabora UK office
(bottom-left) and Mike Ruprecht from somewhere in Missouri. So we
were quite spread out over our little planet. Both audio and video
quality was quite good, even though the network in robs hotel was a
bit flaky from time to time.
In case you want to try it out have a look at the 
As you all know by now, exciting moves from Google on the 
One of the biggest problems with the IM experience in Moblin 2.0 was that it was easy to miss when somebody was talking to you. If you were looking away when the notification popped up, whoops, it s gone. With our switch to Mission Control 5 I was able to integrate a Telepathy 
The Debian Python Modules Team is 
And while I was on it, I thought I could do the same for the GNOME and GStreamer teams:
(hint: pox-guest and piotr in python-modules are the same person)
