The gritty details
Start by netbooting into your rescue image. For OVH, you'd go to the control
panel, in the Services/Netboot section and select "rescue pro". Then reboot
your server. OVH will mail you a temporary password when it finishes rebooting.
Connect to it, without saving the temporary SSH key:
$ ssh -oUserKnownHostsFile=/dev/null -oStrictHostKeyChecking=no root@$ IP
For the rest of the text, I am assuming you have one hard drive called
/dev/sda. We start by partitioning it:
# fdisk /dev/sda
Start a new partition table with o, and then create two primary partitions: a
small one for /boot at the beginning (100 to 300 MB would do), and a second
one with the remaining space. Set both as type 83 (Linux), and don't forget
to activate the first one, as this servers refuse to boot from the hard drive
without that.
Create the file system for /boot, and the encrypted device:
# mkfs.ext4 /dev/sda1
# cryptsetup -s 512 -c aes-xts-plain64 luksFormat /dev/sda2
The encryption parameters are the same as the ones used by the Debian Installer
by default, so don't change them unless you really know what you are doing. You
will need to type a passphrase for the encrypted device, be sure not to forget
it! This passphrase can later be changed (or secondary passphrases added) with
the cryptsetup tool.
Look up the crypt device's UUID, and save it for later:
# cryptsetup luksDump /dev/sda2 grep UUID:
UUID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Open the encrypted device (type the passphrase again), and set up the LVM
volume group:
# cryptsetup luksOpen /dev/sda2 sda2_crypt
# pvcreate /dev/mapper/sda2_crypt
# vgcreate vg0 /dev/mapper/sda2_crypt
Create the logical volumes, this is of course a matter of personal taste and
there are many possible variations. This is my current layout, note that I put
most of the "big data" in /srv.
# lvcreate -L 500m -n root vg0
# lvcreate -L 1.5g -n usr vg0
# lvcreate -L 3g -n var vg0
# lvcreate -L 1g -n home vg0
# lvcreate -L 10g -n srv vg0
# lvcreate -L 500m -n swap vg0
# lvcreate -L 100m -n tmp vg0
Some possible variations:
- You can decide to use a ramdisk for
/tmp, so instead of creating a logical
volume, you would add RAMTMP=yes to /etc/default/tmpfs.
- You can merge
/ and /usr in one same partition, as neither of them
change much.
- You can avoid having swap if you prefer.
- You can put
/home in /srv, and bind mount it later.
Now, create the file systems, swap space, and mount them in /target. Note
that I like to use human-readable labels.
# for i in home root srv tmp usr var; do
mkfs.ext4 -L $i /dev/mapper/vg-$i; done
# mkswap -L swap /dev/mapper/vg0-swap
# mkdir /target
# mount /dev/mapper/vg0-root /target
# mkdir /target/ boot,home,srv,tmp,usr,var
# mount /dev/sda1 /target/boot
# for i in home srv tmp usr var; do
mount /dev/mapper/vg-$i /target/$i; done
# swapon /dev/mapper/vg-swap
Don't forget to set the right permissions for /tmp.
# chmod 1777 /target/tmp
If you want to do the /home on /srv, you'll need to do this (and then copy
to /etc/fstab):
# mkdir /target/srv/home
# mount -o bind /target/srv/home /target/home
The disk is ready now. We will use debootstrap to install the base system.
The OVH image carries it, otherwise consult the relevant section in the
Install manual
for details. It is important that at this point you check that you have a good
GPG keyring for debootstrap to verify the installation source, by comparing
it to a good one (for example, the one in your machine):
# gpg /usr/share/keyrings/debian-archive-keyring.gpg
pub 4096R/B98321F9 2010-08-07 Squeeze Stable Release Key <debian-release@lists.debian.org>
pub 4096R/473041FA 2010-08-27 Debian Archive Automatic Signing Key (6.0/squeeze) <ftpmaster@debian.org>
pub 4096R/65FFB764 2012-05-08 Wheezy Stable Release Key <debian-release@lists.debian.org>
pub 4096R/46925553 2012-04-27 Debian Archive Automatic Signing Key (7.0/wheezy) <ftpmaster@debian.org>
Now, for the actual installation. You can use any Debian mirror, OVH has their
own in the local network. In OVH's case it is critical to specify the
architecture, as the rescue image is i386. I didn't notice that and had to
painfully switch architectures in place (which was absolutely not possible a
couple of years ago).
# debootstrap --arch amd64 wheezy /target http://debian.mirrors.ovh.net/debian
After a few minutes downloading and installing stuff, you almost have a Debian system ready to go. Since this is not D-I, we still need to tighten a few screws manually. Let's mount some needed file systems, and enter the brand new system with chroot:
# mount -o bind /dev /target/dev
# mount -t proc proc /target/proc
# mount -t sysfs sys /target/sys
# XTERM=xterm-color LANG=C.UTF-8 chroot /target /bin/bash
The most critical parts now are to correctly save the parameters for the encrypted device, and the partitions and logical volumes. You'll need the UUID saved before:
# echo 'sda2_crypt UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx none luks' \
> /etc/crypttab
Create the file systems table in /etc/fstab. Here I use labels to identify
the devices:
# file system mount point type options dump pass
LABEL=root / ext4 errors=remount-ro 0 1
LABEL=tmp /tmp ext4 rw,nosuid,nodev 0 2
LABEL=var /var ext4 rw 0 2
LABEL=usr /usr ext4 rw,nodev 0 2
LABEL=home /home ext4 rw,nosuid,nodev 0 2
# Alternative home in /srv:
#/srv/home /home auto bind 0 0
LABEL=srv /srv ext4 rw,nosuid,nodev 0 2
LABEL=boot /boot ext4 rw,nosuid,nodev 0 2
LABEL=swap none swap sw 0 0
You can also just use the device mapper's names
(/dev/mapper/<volume_group>-<logical_volume>), be sure not to use the
/dev/<volume_group>/<logical_volume> naming, as there are some initrd-tools
that choked on them.
# file system mount point type options dump pass
/dev/mapper/vg0-root / ext4 errors=remount-ro 0 1
/dev/mapper/vg0-tmp /tmp ext4 rw,nosuid,nodev 0 2
/dev/mapper/vg0-var /var ext4 rw 0 2
/dev/mapper/vg0-usr /usr ext4 rw,nodev 0 2
/dev/mapper/vg0-home /home ext4 rw,nosuid,nodev 0 2
# Alternative home in /srv:
#/srv/home /home auto bind 0 0
/dev/mapper/vg0-srv /srv ext4 rw,nosuid,nodev 0 2
/dev/sda1 /boot ext4 rw,nosuid,nodev 0 2
/dev/mapper/vg0-swap none swap sw 0 0
Some tools depend on /etc/mtab, which now is just a symbolic link:
# ln -sf /proc/mounts /etc/mtab
Now configure the network. You most surely can use DHCP, but you might prefer
static configuration, that's personal choice. For DHCP, it is very
straightforward:
# cat >> /etc/network/interfaces
auto eth0
iface eth0 inet dhcp
For static configuration, first find the current valid addresses and routes as
obtained by DHCP:
# ip address
# ip route
And then store them:
# cat >> /etc/network/interfaces
auto eth0
iface eth0 inet static
address AAA.BBB.CCC.DDD/24
gateway AAA.BBB.CCC.254
pre-up /sbin/ip addr flush dev eth0 true
Note that pre-up command I added; that is to remove the configuration that
is to be done by the kernel during boot (more on that later), otherwise
ifupdown will complain about existing addresses.
If your provider does IPv6, add it too. For OVH, the IPv6 set-up is a bit
weird, so you need to add the routes in post-up. Your default gateway is
going to be your /64 prefix, with the last byte replaced by ff, and then
followed by :ff:ff:ff:ff. As you can see, that gateway is not in your network
segment, so you need to add an explicit route to it. They have some
information, but it is completely unreadable.
If your IPv6 address is 2001:41D0:1234:5678::1/64, you will add:
iface eth0 inet6 static
address 2001:41D0:1234:5678::1/64
post-up /sbin/ip -6 route add 2001:41D0:1234:56ff:ff:ff:ff:ff dev eth0
post-up /sbin/ip -6 route add default via 2001:41D0:1234:56ff:ff:ff:ff:ff
You probably don't want the auto-configured IPv6 addresses, so disable them via
sysctl:
# cat >> /etc/sysctl.conf
# Disable IPv6 autoconf
net.ipv6.conf.all.autoconf = 0
net.ipv6.conf.default.autoconf = 0
net.ipv6.conf.eth0.autoconf = 0
net.ipv6.conf.all.accept_ra = 0
net.ipv6.conf.default.accept_ra = 0
net.ipv6.conf.eth0.accept_ra = 0
To have a working DNS resolver, we can use the local server (OVH in this case):
# cat > /etc/resolv.conf
search $DOMAIN
nameserver 213.186.33.99
The most important part of a new install: choose a host name (and make the
system use it).
# echo $HOSTNAME > /etc/hostname
# hostname $HOSTNAME
# echo "127.0.1.1 $HOSTNAME.$DOMAIN $HOSTNAME" >> /etc/hosts
If we want to speficy the BIOS clock to use UTC:
# echo -e '0.0 0 0.0\n0\nUTC' > /etc/adjtime
Set up your time zone:
# dpkg-reconfigure tzdata
Configure APT with your preferred mirrors. I also prevent APT from installing
recommends by default.
# echo deb http://ftp2.fr.debian.org/debian wheezy main contrib non-free \
>> /etc/apt/sources.list
# echo deb http://ftp2.fr.debian.net/debian wheezy-updates main contrib non-free \
>> /etc/apt/sources.list
# echo deb http://security.debian.org/ wheezy/updates main contrib non-free \
>> /etc/apt/sources.list
# echo 'APT::Install-Recommends "False";' > /etc/apt/apt.conf.d/02recommends
# apt-get update
Before installing any package, let's make sure that the initial ram disk
(initrd) that is going to be created will allow us to connect. There will be
no chance of using the root password during boot. Your public key is usually
found in $HOME/.ssh/id_rsa.pub.
# mkdir -p /etc/initramfs-tools/root/.ssh/
# echo $(YOUR_PUB_RSA_KEY) > /etc/initramfs-tools/root/.ssh/authorized_keys
If you change this, or the host key stored at
/etc/dropbear/dropbear_*_host_key, the /etc/crypttab, or any other
critical piece of information for the booting process, you need to run
update-initramfs -u.
Now we can install the missing pieces:
# apt-get install makedev cryptsetup lvm2 ssh dropbear busybox ssh \
initramfs-tools locales linux-image-amd64 grub-pc kbd console-setup
During the installation you will have to choose where to install grub, I
recommend directly on /dev/sda. Also, the magic initrd will be created. We
want to double check that it has all the important pieces for a successful
boot:
# zcat /boot/initrd.img-3.2.0-4-amd64 cpio -t conf/conf.d/cryptroot \
etc/lvm/lvm.conf etc/dropbear/\* root/.ssh/authorized_keys sbin/dropbear
etc/lvm/lvm.conf
etc/dropbear/dropbear_dss_host_key
etc/dropbear/dropbear_rsa_host_key
sbin/dropbear
root/.ssh/authorized_keys
conf/conf.d/cryptroot
All these files need to be there. Most critically, we need to check that the cryptroot file has the right information to access the root file system:
# zcat /boot/initrd.img-* cpio -i --to-stdout conf/conf.d/cryptroot
target=sda2_crypt,source=UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx,key=none,rootdev,lvm=vg0-root
If all that was correct, now we need to tell the kernel to configure the
network as soon as possible so we can connect to the initrd and unlock the
disks. This is done by passing a command-line option though grub. This should match what was done in /etc/network/interfaces: either DHCP or static configuration. For DHCP, this line should be changed in /etc/default/grub:
GRUB_CMDLINE_LINUX="ip=:::::eth0:dhcp"
For static configuration:
GRUB_CMDLINE_LINUX="ip=MY_IP_ADDR::MY_DEFAULT_GW:MY_NETMASK::eth0:none"
It is also a good idea to disable the quiet boot and graphical boot splash, in case we need to use QEMU to fix some booting issue:
GRUB_CMDLINE_LINUX_DEFAULT=""
GRUB_TERMINAL=console
And make the changes effective:
# update-grub2
Having fsck fix problems automatically can be a life-saver too:
# echo FSCKFIX=yes >> /etc/default/rcS
Get some very useful packages:
# apt-get install vim less ntpdate sudo
Create an user for yourself, possibly make it an administrator:
# adduser tincho
# adduser tincho sudo
# adduser tincho adm
This is mostly done, exit the chroot, and unmount everything.
# exit # the chroot.
# umount /target/ dev,proc,sys,boot,home,srv,tmp,usr,var
# umount /target
# swapoff -a
# lvchange -an /dev/mapper/vg0-*
# cryptsetup luksClose sda2_crypt
Disable the netboot option from your administration panel, reboot, and hope it
all goes well.
If you followed every step carefully, a few minutes later you should be able to ping your server. Use this snippet to enter the password remotely:
$ stty -echo; ssh -o UserKnownHostsFile=$HOME/.ssh/known_hosts.initramfs \
-o BatchMode=yes root@"$HOST" 'cat > /lib/cryptsetup/passfifo'; \
stty echo
It is very important that you close the pipe (with control-D twice) without
typing enter. For my servers, I have a script that reads the passphrase from
a GPG-encrypted file and pipes it directly into the remote server. That way, I
only type the GPG passphrase locally:
$ cat unlock.sh
#!/bin/sh
BASE="$(dirname "$0")"
HOST="$1"
gpg --decrypt "$BASE"/key-"$HOST".gpg \
ssh -o UserKnownHostsFile="$BASE"/known_hosts.initramfs -o BatchMode=yes \
root@"$HOST" 'cat > /lib/cryptsetup/passfifo'
It might be a good idea to create a long, impossible to guess passphrase that
you can use in the GPG-encrypted file, and that you can also print and store in
somewhere safe. See the luksAddKey function in the cryptsetup(8) man page.
Once again, if everything went right, a few seconds later the openSSH server
will replace the tiny dropbear and you will be able to access your server
normally (and with the real SSH host key).
Hope you find this article helpful! I would love to hear your feedback.
Yes, Laura and I are engaged!
There are no words adequate for this kind of occasion, but there is a picture that gets close:
I never imagined a person could find a friend so wonderful, a person that enjoys so much in common, someone that understands me and that I understand so well. And yet, here I am, engaged to that friend. Amazing only begins to describe the feeling.
One of the first things Laura and I talked about was a hymn I was tinkering with, typesetting with GNU LilyPond. That hymn ends like this:
Since Love is Lord of heav n and earth, how can I keep from singing?
It is a wonderful thought, and very true. Even literally true; I often find myself singing, humming, or playing the penny whistle during my day.
One of my good friends once told me, I am completely sure that your happiest days lie ahead of you. And he was right. I have already experienced them. This is a wonderful time, and every day brings plentiful reasons to be thankful. And as Laura and I prepare for a life together, I know that it is true not only that my friend was right, but that he is right my happiest days are yet to come, and our happiest days are yet to come, too.
I have been blessed in many ways, and feel like the luckiest man alive.
To be loved, and to love, a person so wonderful is truly a remarkable gift.
Had an afternoon of steak and science fiction. Elysium is only so-so, but
look what we found in a bookstore that was half religious materials and
half SF, local books, and carefully hidden romance novels:
after a break, here's again a report about my work on RC bugs. in this case,
all bugs either affect packages in the pkg-perl group and/or packages
involved in the Perl 5.18 transition.
So, last year I've switched to an OpenPGP smartcard setup for my
whole personal/Debian PGP usage. When doing so, I've also switched
to subkeys, since it's pretty natural when using a smartcard. I
initially set up an expiration of one year for the subkeys, and
everything seems to be running just fine for now.
The expiration date was set to october 27th, and I though it'd
be a good idea to renew them quite in advance, considering there's
my signing key in there, which is (for example) used to sign
packages. If the Debian archive considers my signature subkey
expired, that means I can't upload packages anymore, which is a bit
of a problem (although I think I could still upload packages signed
by the main key). dak (Debian Archive Kit, the software managing
the Debian archive) uses keys from the keyring provided by Debian
admins, which is usually updated every month or so from the
When starting a new 
Eric (a fellow Fedora board member) has a post describing 
Very long post ahead (sorry for the wall of text), part of a series of posts on some sysadmin topics, see
So, I m back from DebConf, which probably translated to the 10 busiest days of my life, but also to one of the best times of my life. The Le Camp venue clearly contributed to this success: having everybody at the same place, but at the same time many opportunities for quiet chat or just enjoying the view, was really a good idea. Everybody who made DebConf possible deserve a huge thank you , as well as all attendees: it is really a honor to be a part of such a fantastic community.
Now, let s go back to daily life, and to my re-filled Debian TODO list!
As I slowly read my good friends wishing each other a good trip, telling they got home safely, and the IRC channels form thick drops of a bitter-sweet etheral substance, I cannot help feeling DebConf13 is over For me as well, from the distance. Many friends gave me warm greetings, and without being there, gave me that beautiful feeling of real community that Debian has given me for ten years already, since I met in real-life many of its developers at DebConf3 in Oslo. And yes, I have stated this far too many times I have attended every DebConf since (and worked organizing most of them). This year, 
WebRTC: premium voice/video chat in modern browsers, no plugin required
Another member of Debian's GSoC program, 
As the grand finale, we took 
The grand-finale of the 
