The gritty details
Start by netbooting into your rescue image. For OVH, you'd go to the control
panel, in the Services/Netboot section and select "rescue pro". Then reboot
your server. OVH will mail you a temporary password when it finishes rebooting.
Connect to it, without saving the temporary SSH key:
$ ssh -oUserKnownHostsFile=/dev/null -oStrictHostKeyChecking=no root@$ IP
For the rest of the text, I am assuming you have one hard drive called
/dev/sda
. We start by partitioning it:
# fdisk /dev/sda
Start a new partition table with o
, and then create two primary partitions: a
small one for /boot
at the beginning (100 to 300 MB would do), and a second
one with the remaining space. Set both as type 83 (Linux), and don't forget
to activate the first one, as this servers refuse to boot from the hard drive
without that.
Create the file system for /boot
, and the encrypted device:
# mkfs.ext4 /dev/sda1
# cryptsetup -s 512 -c aes-xts-plain64 luksFormat /dev/sda2
The encryption parameters are the same as the ones used by the Debian Installer
by default, so don't change them unless you really know what you are doing. You
will need to type a passphrase for the encrypted device, be sure not to forget
it! This passphrase can later be changed (or secondary passphrases added) with
the cryptsetup tool.
Look up the crypt device's UUID, and save it for later:
# cryptsetup luksDump /dev/sda2 grep UUID:
UUID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Open the encrypted device (type the passphrase again), and set up the LVM
volume group:
# cryptsetup luksOpen /dev/sda2 sda2_crypt
# pvcreate /dev/mapper/sda2_crypt
# vgcreate vg0 /dev/mapper/sda2_crypt
Create the logical volumes, this is of course a matter of personal taste and
there are many possible variations. This is my current layout, note that I put
most of the "big data" in /srv
.
# lvcreate -L 500m -n root vg0
# lvcreate -L 1.5g -n usr vg0
# lvcreate -L 3g -n var vg0
# lvcreate -L 1g -n home vg0
# lvcreate -L 10g -n srv vg0
# lvcreate -L 500m -n swap vg0
# lvcreate -L 100m -n tmp vg0
Some possible variations:
- You can decide to use a ramdisk for
/tmp
, so instead of creating a logical
volume, you would add RAMTMP=yes
to /etc/default/tmpfs
.
- You can merge
/
and /usr
in one same partition, as neither of them
change much.
- You can avoid having swap if you prefer.
- You can put
/home
in /srv
, and bind mount it later.
Now, create the file systems, swap space, and mount them in /target
. Note
that I like to use human-readable labels.
# for i in home root srv tmp usr var; do
mkfs.ext4 -L $i /dev/mapper/vg-$i; done
# mkswap -L swap /dev/mapper/vg0-swap
# mkdir /target
# mount /dev/mapper/vg0-root /target
# mkdir /target/ boot,home,srv,tmp,usr,var
# mount /dev/sda1 /target/boot
# for i in home srv tmp usr var; do
mount /dev/mapper/vg-$i /target/$i; done
# swapon /dev/mapper/vg-swap
Don't forget to set the right permissions for /tmp.
# chmod 1777 /target/tmp
If you want to do the /home
on /srv
, you'll need to do this (and then copy
to /etc/fstab
):
# mkdir /target/srv/home
# mount -o bind /target/srv/home /target/home
The disk is ready now. We will use debootstrap
to install the base system.
The OVH image carries it, otherwise consult the relevant section in the
Install manual
for details. It is important that at this point you check that you have a good
GPG keyring for debootstrap
to verify the installation source, by comparing
it to a good one (for example, the one in your machine):
# gpg /usr/share/keyrings/debian-archive-keyring.gpg
pub 4096R/B98321F9 2010-08-07 Squeeze Stable Release Key <debian-release@lists.debian.org>
pub 4096R/473041FA 2010-08-27 Debian Archive Automatic Signing Key (6.0/squeeze) <ftpmaster@debian.org>
pub 4096R/65FFB764 2012-05-08 Wheezy Stable Release Key <debian-release@lists.debian.org>
pub 4096R/46925553 2012-04-27 Debian Archive Automatic Signing Key (7.0/wheezy) <ftpmaster@debian.org>
Now, for the actual installation. You can use any Debian mirror, OVH has their
own in the local network. In OVH's case it is critical to specify the
architecture, as the rescue image is i386
. I didn't notice that and had to
painfully switch architectures in place (which was absolutely not possible a
couple of years ago).
# debootstrap --arch amd64 wheezy /target http://debian.mirrors.ovh.net/debian
After a few minutes downloading and installing stuff, you almost have a Debian system ready to go. Since this is not D-I, we still need to tighten a few screws manually. Let's mount some needed file systems, and enter the brand new system with chroot
:
# mount -o bind /dev /target/dev
# mount -t proc proc /target/proc
# mount -t sysfs sys /target/sys
# XTERM=xterm-color LANG=C.UTF-8 chroot /target /bin/bash
The most critical parts now are to correctly save the parameters for the encrypted device, and the partitions and logical volumes. You'll need the UUID saved before:
# echo 'sda2_crypt UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx none luks' \
> /etc/crypttab
Create the file systems table in /etc/fstab
. Here I use labels to identify
the devices:
# file system mount point type options dump pass
LABEL=root / ext4 errors=remount-ro 0 1
LABEL=tmp /tmp ext4 rw,nosuid,nodev 0 2
LABEL=var /var ext4 rw 0 2
LABEL=usr /usr ext4 rw,nodev 0 2
LABEL=home /home ext4 rw,nosuid,nodev 0 2
# Alternative home in /srv:
#/srv/home /home auto bind 0 0
LABEL=srv /srv ext4 rw,nosuid,nodev 0 2
LABEL=boot /boot ext4 rw,nosuid,nodev 0 2
LABEL=swap none swap sw 0 0
You can also just use the device mapper's names
(/dev/mapper/<volume_group>-<logical_volume>
), be sure not to use the
/dev/<volume_group>/<logical_volume>
naming, as there are some initrd-tools
that choked on them.
# file system mount point type options dump pass
/dev/mapper/vg0-root / ext4 errors=remount-ro 0 1
/dev/mapper/vg0-tmp /tmp ext4 rw,nosuid,nodev 0 2
/dev/mapper/vg0-var /var ext4 rw 0 2
/dev/mapper/vg0-usr /usr ext4 rw,nodev 0 2
/dev/mapper/vg0-home /home ext4 rw,nosuid,nodev 0 2
# Alternative home in /srv:
#/srv/home /home auto bind 0 0
/dev/mapper/vg0-srv /srv ext4 rw,nosuid,nodev 0 2
/dev/sda1 /boot ext4 rw,nosuid,nodev 0 2
/dev/mapper/vg0-swap none swap sw 0 0
Some tools depend on /etc/mtab
, which now is just a symbolic link:
# ln -sf /proc/mounts /etc/mtab
Now configure the network. You most surely can use DHCP, but you might prefer
static configuration, that's personal choice. For DHCP, it is very
straightforward:
# cat >> /etc/network/interfaces
auto eth0
iface eth0 inet dhcp
For static configuration, first find the current valid addresses and routes as
obtained by DHCP:
# ip address
# ip route
And then store them:
# cat >> /etc/network/interfaces
auto eth0
iface eth0 inet static
address AAA.BBB.CCC.DDD/24
gateway AAA.BBB.CCC.254
pre-up /sbin/ip addr flush dev eth0 true
Note that pre-up
command I added; that is to remove the configuration that
is to be done by the kernel during boot (more on that later), otherwise
ifupdown
will complain about existing addresses.
If your provider does IPv6, add it too. For OVH, the IPv6 set-up is a bit
weird, so you need to add the routes in post-up
. Your default gateway is
going to be your /64 prefix, with the last byte replaced by ff
, and then
followed by :ff:ff:ff:ff
. As you can see, that gateway is not in your network
segment, so you need to add an explicit route to it. They have some
information, but it is completely unreadable.
If your IPv6 address is 2001:41D0:1234:5678::1/64, you will add:
iface eth0 inet6 static
address 2001:41D0:1234:5678::1/64
post-up /sbin/ip -6 route add 2001:41D0:1234:56ff:ff:ff:ff:ff dev eth0
post-up /sbin/ip -6 route add default via 2001:41D0:1234:56ff:ff:ff:ff:ff
You probably don't want the auto-configured IPv6 addresses, so disable them via
sysctl
:
# cat >> /etc/sysctl.conf
# Disable IPv6 autoconf
net.ipv6.conf.all.autoconf = 0
net.ipv6.conf.default.autoconf = 0
net.ipv6.conf.eth0.autoconf = 0
net.ipv6.conf.all.accept_ra = 0
net.ipv6.conf.default.accept_ra = 0
net.ipv6.conf.eth0.accept_ra = 0
To have a working DNS resolver, we can use the local server (OVH in this case):
# cat > /etc/resolv.conf
search $DOMAIN
nameserver 213.186.33.99
The most important part of a new install: choose a host name (and make the
system use it).
# echo $HOSTNAME > /etc/hostname
# hostname $HOSTNAME
# echo "127.0.1.1 $HOSTNAME.$DOMAIN $HOSTNAME" >> /etc/hosts
If we want to speficy the BIOS clock to use UTC:
# echo -e '0.0 0 0.0\n0\nUTC' > /etc/adjtime
Set up your time zone:
# dpkg-reconfigure tzdata
Configure APT with your preferred mirrors. I also prevent APT from installing
recommends by default.
# echo deb http://ftp2.fr.debian.org/debian wheezy main contrib non-free \
>> /etc/apt/sources.list
# echo deb http://ftp2.fr.debian.net/debian wheezy-updates main contrib non-free \
>> /etc/apt/sources.list
# echo deb http://security.debian.org/ wheezy/updates main contrib non-free \
>> /etc/apt/sources.list
# echo 'APT::Install-Recommends "False";' > /etc/apt/apt.conf.d/02recommends
# apt-get update
Before installing any package, let's make sure that the initial ram disk
(initrd
) that is going to be created will allow us to connect. There will be
no chance of using the root password during boot. Your public key is usually
found in $HOME/.ssh/id_rsa.pub
.
# mkdir -p /etc/initramfs-tools/root/.ssh/
# echo $(YOUR_PUB_RSA_KEY) > /etc/initramfs-tools/root/.ssh/authorized_keys
If you change this, or the host key stored at
/etc/dropbear/dropbear_*_host_key
, the /etc/crypttab
, or any other
critical piece of information for the booting process, you need to run
update-initramfs -u
.
Now we can install the missing pieces:
# apt-get install makedev cryptsetup lvm2 ssh dropbear busybox ssh \
initramfs-tools locales linux-image-amd64 grub-pc kbd console-setup
During the installation you will have to choose where to install grub
, I
recommend directly on /dev/sda
. Also, the magic initrd
will be created. We
want to double check that it has all the important pieces for a successful
boot:
# zcat /boot/initrd.img-3.2.0-4-amd64 cpio -t conf/conf.d/cryptroot \
etc/lvm/lvm.conf etc/dropbear/\* root/.ssh/authorized_keys sbin/dropbear
etc/lvm/lvm.conf
etc/dropbear/dropbear_dss_host_key
etc/dropbear/dropbear_rsa_host_key
sbin/dropbear
root/.ssh/authorized_keys
conf/conf.d/cryptroot
All these files need to be there. Most critically, we need to check that the cryptroot
file has the right information to access the root file system:
# zcat /boot/initrd.img-* cpio -i --to-stdout conf/conf.d/cryptroot
target=sda2_crypt,source=UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx,key=none,rootdev,lvm=vg0-root
If all that was correct, now we need to tell the kernel to configure the
network as soon as possible so we can connect to the initrd
and unlock the
disks. This is done by passing a command-line option though grub
. This should match what was done in /etc/network/interfaces
: either DHCP or static configuration. For DHCP, this line should be changed in /etc/default/grub
:
GRUB_CMDLINE_LINUX="ip=:::::eth0:dhcp"
For static configuration:
GRUB_CMDLINE_LINUX="ip=MY_IP_ADDR::MY_DEFAULT_GW:MY_NETMASK::eth0:none"
It is also a good idea to disable the quiet boot and graphical boot splash, in case we need to use QEMU to fix some booting issue:
GRUB_CMDLINE_LINUX_DEFAULT=""
GRUB_TERMINAL=console
And make the changes effective:
# update-grub2
Having fsck
fix problems automatically can be a life-saver too:
# echo FSCKFIX=yes >> /etc/default/rcS
Get some very useful packages:
# apt-get install vim less ntpdate sudo
Create an user for yourself, possibly make it an administrator:
# adduser tincho
# adduser tincho sudo
# adduser tincho adm
This is mostly done, exit the chroot, and unmount everything.
# exit # the chroot.
# umount /target/ dev,proc,sys,boot,home,srv,tmp,usr,var
# umount /target
# swapoff -a
# lvchange -an /dev/mapper/vg0-*
# cryptsetup luksClose sda2_crypt
Disable the netboot option from your administration panel, reboot, and hope it
all goes well.
If you followed every step carefully, a few minutes later you should be able to ping
your server. Use this snippet to enter the password remotely:
$ stty -echo; ssh -o UserKnownHostsFile=$HOME/.ssh/known_hosts.initramfs \
-o BatchMode=yes root@"$HOST" 'cat > /lib/cryptsetup/passfifo'; \
stty echo
It is very important that you close the pipe (with control-D twice) without
typing enter. For my servers, I have a script that reads the passphrase from
a GPG-encrypted file and pipes it directly into the remote server. That way, I
only type the GPG passphrase locally:
$ cat unlock.sh
#!/bin/sh
BASE="$(dirname "$0")"
HOST="$1"
gpg --decrypt "$BASE"/key-"$HOST".gpg \
ssh -o UserKnownHostsFile="$BASE"/known_hosts.initramfs -o BatchMode=yes \
root@"$HOST" 'cat > /lib/cryptsetup/passfifo'
It might be a good idea to create a long, impossible to guess passphrase that
you can use in the GPG-encrypted file, and that you can also print and store in
somewhere safe. See the luksAddKey
function in the cryptsetup(8)
man page.
Once again, if everything went right, a few seconds later the openSSH server
will replace the tiny dropbear
and you will be able to access your server
normally (and with the real SSH host key).
Hope you find this article helpful! I would love to hear your feedback.