Steve McIntyre: Bootstrapping arm64 in Debian
I promised to write about this a long time, ooops... :-)
Another ARM port in Debian - yay!
arm64 is officially a release architecture for Jessie, aka Debian
version 8. That's taken a lot of manual porting and development effort
over the last couple of years, and it's also taken a lot of CPU time -
there are ~21,000 source packages in Debian Jessie! As is often the
case for a brand new architecture like arm64 (or AArch64, to use ARM's
own terminology), hardware can be really difficult to get hold of. In
time this will cease to be an issue as hardware becomes more
commoditised, but in Debian we really struggled to get hold of
equipment for a very long time during the early part of the port.
First bring-up in Debian Ports
To start with, we could use ARM's own AArch64 software models to
build the first few packages. This worked, but only very slowly. Then
Chen Baozi and the folks running
the Tianhe-2
supercomputer project in Guangzhou, China contacted us to offer access
to some arm64 hardware, and this is what Wookey used for bootstrapping
the new port in the
unofficial Debian Ports
archive. This has now become the normal way for new architectures to
get into Debian. We got most of the archive built in debian-ports this
way, and we could then use those results to seed the initial core set
of packages in the main Debian archive.
Second bring-up - moving into the main Debian archive
By the time that first Debian bring-up was
done, ARM was starting to produce its
own "Juno" development boards, and with the help of my boss^4 James
McNiven we managed to acquire a couple of those machines for use as
official Debian build machines. The existing machines in China were
faster, but for various reasons quite difficult to maintain as
official Debian machines. So I set up the Junos as buildds just before
going to DebConf in August 2014. They ran very well, and (for dev
boards!) were very fast and stable. They built a large chunk of the
Debian archive, but as the release freeze for Jessie grew close we
weren't quite there. There was a small but persistent backlog of
un-built packages that were causing us issues, plus the Juno machines
are/were not quite suitable as porter boxes for Debian developers all
over the world to use for debugging their packages on the new
architecture.
More horsepower - Linaro machines
This is where Linaro came to
our aid. Linaro's goal is to help improve Free and Open Source
Software on ARM, and one of the more recent projects in Linaro is a
cluster of
servers that are made available for software developers to use to
get early access to ARMv8 (arm64) hardware. It's a great way for
people who are interested in this new architecture to try things out,
port their software or indeed just help with the general porting
effort.
As Debian is seen as such an important part of the FLOSS ecosystem,
we managed to negotiate dedicated access to three of the machines in
that cluster for Debian's use and we set those up in October, shortly
before the freeze for Jessie. Andy Doan spent a lot of his time
getting these machines going for us, and then I set up two of them as
build machines and one as the porter box we were still needing.
With these extra machines available, we quickly caught up with the
ever-busy "Needs-Build" queue and we've got sufficient build power now
to keep things going for the Jessie release. We were officially added
to the list of release architectures at the Cambridge mini-Debconf in
November, and all is looking good now!
And in the future?
I've organised the loan of another arm64 machine from AMD for
Debian to use for further porting and/or building. We're also
expecting that more and more machines will be coming out soon as
vendors move on from prototyping to producing real customer
equipment. Once that's happened, more kit will be available and
everybody will be able to have arm64-powered computers in the server
room, on their desk and even inside their laptop! Mine will be running
Debian Jessie... :-)
Thanks!
There's been a lot of people involved in the Debian arm64
bootstrapping at various stages, so many that I couldn't possibly
credit them all! I'll highlight some, though. :-)
First of all, Wookey's life has revolved around this port for the
last few years, tirelessly porting, fixing and hacking out package
builds to get us going. We've had loads of help from other teams in
Debian, particularly the massive patience of the DSA folks with
getting early machines up and running and the prodding of the
ftpmaster, buildd and release teams when we've been grinding our way
through ever more package builds and dependency loops. We've also had
really good support from toolchain folks in Debian and ARM, fixing
bugs as we've found them by stressing new code and new machines. We've
had a number of other people helping by filing bugs and posting
patches to help us get things built and working. And (last but not
least!) thanks to all the folks who've helped us beg and borrow the
hardware to make the Debian arm64 port a reality.
Rumours of even more ARM ports coming soon are entirely
scurrilous... *grin*