Building packages in Debian seems to be a solved problem. But is it? At the bottom, installing the
dpkg-dev package provides all the basic tools needed. Assuming that you already succeeded with creating the necessary packaging metadata (i.e.,
debian/changelog,
debian/control,
debian/copyright, etc., and there are great helper tools for this such ash
dh-make,
dh-make-golang, etc.,) it should be as simple as invoking the
dpkg-buildpackage tool. So what's the big deal here?
The issue is that
dpkg-buildpackage expects to be called with an appropriately setup build context, that is, it needs to be called in an environment that satisfies all build dependencies on the system. Let's say you are building a package for Debian unstable on your Debian stable system (this is the common scenario for the official Debian build machines), you would need your build to link against libraries in unstable, not stable. So how to tell the package build process where to find its dependencies?
The answer (in Debian and many other Linux distributions) is you do not at all. This is actually a somewhat surprising answer for software developers without a Linux distribution development background
. Instead, chroots "simulate" an environment that has all dependencies that we want to build against at the system locations, that is.
/usr/lib, etc.
Chroots are basically full system installations in a subdirectory that includes system and application libraries. In order to use them, a package build needs to use the
chroot(2) system call, which is a privileged operation. Also creating these system installations is a somewhat finicky process. In Debian, we have tools that make this process easier, the most popular ones are probably
pbuilder(1) and
sbuild(1). Still, they are somewhat clumsy to use, add significant levels of abstraction in the sense that they do quite a bit of magic behind the scenes to work hide the fact that privileged operations (need root to run
chroot(2), etc.) are required. They are also somewhat brittle, for instance, if a build process is aborted (SIGKILL, or system crash), you may end up with temporary directories and files under userids other than your own that again may require root-privileges to cleanup.
What if there was an easy way to do all of the above without any process running as root? Enter rootless buildah.
Modern Linux container technologies allow unprivileged users to "untie" a process from the system (cf. the
unshare(2) system call). This means that a regular user may run a process (and its child processes) in an "environment" where system calls behave differently and provide a configurable amount of isolation levels. This article demonstrates a novel build tool
buildah, which is:
- easy to setup build environment from the command line
- secure, as no process needs to run as root
- simple in architecture, requires no running daemon like for example docker
- convenient to use: you can debug your debian/rules interactively
Architecturally,
buildah is written in golang and compiled as a (mostly) statically linked executable. It builds on top a number of libraries written in golang, including github.com/containers/storage and github.com/containers/image. The overlay functionality is provided by the
fuse-overlayfs(1) utility.