One of my goals for
Consfigurator is to make it
capable of installing Debian to my laptop, so that I can stop booting to GRML
and manually partitioning and debootstrapping a basic system, only to
then
turn to configuration management to set everything else up. My configuration
management should be able to handle the partitioning and debootstrapping, too.
The first stage was to make Consfigurator capable of debootstrapping a basic
system, chrooting into it, and applying other arbitrary configuration, such as
installing packages. That s been in place for some weeks now. It s
sophisticated enough to avoid starting up newly installed services, but I
still need to add some bind mounting.
Another significant piece is teaching Consfigurator how to partition block
devices. That s quite tricky to do in a sufficiently general way I want to
cleanly support various combinations of LUKS, LVM and regular partitions,
including populating /etc/crypttab and /etc/fstab. I have some ideas about
how to do it, but it ll probably take a few tries to get the abstractions
right.
Let s imagine that code is all in place, such that Consfigurator can be
pointed at a block device and it will install a bootable Debian system to it.
Then to install Debian to my laptop I d just need to take my laptop s disk
drive out and plug it into another system, and run Consfigurator on that
system, as root, pointed at the block device representing my laptop s disk
drive. For virtual machines, it would be easy to write code which loop-mounts
an empty disk image, and then Consfigurator could be pointed at the
loop-mounted block device, thereby making the disk image file bootable.
This is adequate for virtual machines, or small single-board computers with
tiny storage devices (not that I actually use any of those, but I want
Consfigurator to be able to make disk images for them!). But it s not much
good for my laptop. I casually referred to taking out my laptop s disk drive
and connecting it to another computer, but this would void my laptop s
warranty. And Consfigurator would not be able to update my laptop s NVRAM, as
is needed on UEFI systems.
What s wanted here is a live system which can run Consfigurator directly on
the laptop, pointed at the block device representing its physical disk drive.
Ideally this live system comes with a chroot with the root filesystem for the
new Debian install already built, so that network access is not required, and
all Consfigurator has to do is partition the drive and copy in the contents of
the chroot. The live system could be set up to automatically start doing that
upon boot, but another option is to just make Consfigurator itself available
to be used interactively. The user boots the live system, starts up Emacs,
starts up Lisp, and executes a Consfigurator deployment, supplying the block
device representing the laptop s disk drive as an argument to the deployment.
Consfigurator goes off and partitions that drive, copies in the contents of
the chroot, and executes grub-install to make the laptop bootable. This is
also much easier to debug than a live system which tries to start partitioning
upon boot. It would look something like this:
;; melete.silentflame.com is a Consfigurator host object representing the
;; laptop, including information about the partitions it should have
(deploy-these :local ...
(chroot:partitioned-and-installed
melete.silentflame.com "/srv/chroot/melete" "/dev/nvme0n1"))
Now, building live systems is a fair bit more involved than installing Debian
to a disk drive and making it bootable, it turns out. While I want
Consfigurator to be able to completely replace the Debian Installer, I decided
that it is not worth trying to reimplement the relevant parts of the Debian
Live tool suite, because I do not need to make arbitrary customisations to any
live systems. I just need to have some packages installed and some files in
place. Nevertheless, it is worth teaching Consfigurator how to invoke Debian
Live, so that the customisation of the chroot which isn t just a matter of
passing options to lb_config(1) can be done with Consfigurator. This is what
I ve ended up with in Consfigurator s source code:
(defpropspec image-built :lisp (config dir properties)
"Build an image under DIR using live-build(7), where the resulting live
system has PROPERTIES, which should contain, at a minimum, a property from
CONSFIGURATOR.PROPERTY.OS setting the Debian suite and architecture. CONFIG
is a list of arguments to pass to lb_config(1), not including the '-a' and
'-d' options, which Consfigurator will supply based on PROPERTIES.
This property runs the lb_config(1), lb_bootstrap(1), lb_chroot(1) and
lb_binary(1) commands to build or rebuild the image. Rebuilding occurs only
when changes to CONFIG or PROPERTIES mean that the image is potentially
out-of-date; e.g. if you just add some new items to PROPERTIES then in most
cases only lb_chroot(1) and lb_binary(1) will be re-run.
Note that lb_chroot(1) and lb_binary(1) both run after applying PROPERTIES,
and might undo some of their effects. For example, to configure
/etc/apt/sources.list, you will need to use CONFIG not PROPERTIES."
(:desc (declare (ignore config properties))
#?"Debian Live image built in $ dir ")
(let* (...)
;; ...
(eseqprops
;; ...
(on-change
(eseqprops
(on-change
(file:has-content ,auto/config ,(auto/config config) :mode #o755)
(file:does-not-exist ,@clean)
(%lbconfig ,dir)
(%lbbootstrap t ,dir))
(%lbbootstrap nil ,dir)
(deploys ((:chroot :into ,chroot)) ,host))
(%lbchroot ,dir)
(%lbbinary ,dir)))))
Here, %lbconfig is a property running lb_config(1), %lbbootstrap one which
runs lb_bootstrap(1), etc. Those properties all just change directory to the
right place and run the command, essentially, with a little extra code to
handle failed debootstraps and the like.
The ON-CHANGE and ESEQPROPS combinators work together to sequence the
interaction of the Debian Live suite and Consfigurator.
- In the innermost ON-CHANGE expression: create the file auto/config and
populate it with the call to lb_config(1) that we need to make, as described
in the Debian Live manual, chapter 6.
- If doing so resulted in a change to the auto/config file e.g. the user
added some more options ensure that lb_config(1) and lb_bootstrap(1)
both get rerun.
- Now in the inner ESEQPROPS expression, use DEPLOYS to configure the chroot,
essentially by forking into the chroot and recursively reinvoking
Consfigurator.
- Finally, if any of the above resulted in a change being made, call
lb_chroot(1) and lb_binary(1).
This way, we only rebuild the chroot if the configuration changed, and we only
rebuild the image if the chroot changed.
Now over in my personal consfig:
(try-register-data-source
:git-snapshot :name "consfig" :repo #P"src/cl/consfig/" ...)
(defproplist hybrid-live-iso-built :lisp ()
"Build a Debian Live system in /srv/live/spw.
Typically this property is not applied in a DEFHOST form, but rather run as
needed at the REPL. The reason for this is that otherwise the whole image will
get rebuilt each time a commit is made to my dotfiles repo or to my consfig."
(:desc "Sean's Debian Live system image built")
(live-build:image-built.
'("--archive-areas" "main contrib non-free" ...)
"/srv/live/spw"
(os:debian-stable "buster" :amd64)
(basic-props)
(apt:installed "whatever" "you" "want")
(git:snapshot-extracted "/etc/skel/src" "dotfiles")
(file:is-copy-of "/etc/skel/.bashrc" "/etc/skel/src/dotfiles/.bashrc")
(git:snapshot-extracted "/root/src/cl" "consfig")))
The first argument to LIVE-BUILD:IMAGE-BUILT. is additional arguments to
lb_config(1). The third argument onwards are the properties for the live
system. The cool thing is GIT:SNAPSHOT-EXTRACTED the calls to this ensure
that a copy of my Emacs configuration and my consfig end up in the live image,
ready to be used interactively to install Debian, as described above. I ll
need to add something like
(chroot:host-chroot-bootstrapped
melete.silentflame.com "/srv/chroot/melete") too.
As with everything Consfigurator-related, Joey Hess s
Propellor
is the giant upon whose shoulders I m standing.