I have several backup servers that run the excellent
rsnapshot software, which uses
Secure Shell (SSH) for remote access. The SSH private key of the backup server can be a weak link in the overall security. To see how it can be a problem, consider if someone breaks into your backup server and manages to copy your SSH private key, they will now have the ability to login to all machines that you take backups off (and that should be all of your machines, right?).
The traditional way to mitigate SSH private key theft is by password protecting the private key. This works poorly in an unattended server environment because either the decryption password needs to be stored in disk (where the attacker can read it) or the decrypted private key has to be available in decrypted form in memory (where attacker can read it).
A better way to deal with the problem is to move the SSH private key to a smartcard. The idea is that the private key cannot be copied by an attacker who roots your backup server. (Careful readers may have spotted a flaw here, and I need to explain one weakness with my solution: an attacker will still be able to login to all your systems by going through your backup server, however it will require an open inbound network connection to your backup server and the attacker will never know what your private key is. What this does is to allow you to more easily do damage control by removing the smartcard from the backup server.)
In this writeup, I ll explain how to accomplish all this on a
Debian/
Ubuntu-system using a
OpenPGP smartcard, a
Gemalto USB Shell Token v2 with gpg-agent/scdaemon from
GnuPG together with
OpenSSH.
First we need to install some packages. The goal is to configure
OpenSSH to talk to the
gpg-agent which will start and talk to
scdaemon which in turn talks to
pcscd which talks to the smart card reader. For some strange reason, the scdaemon binary is shipped with GnuPG s S/MIME interface in the
gpgsm package.
# apt-get install pcscd gnupg-agent gpgsm
The above command should install and start
pcscd and if all works well, you should be able to check the status of the smartcard using GnuPG.
# gpg --card-status
You need to initialize the smartcard and generate a private key on it, again using GnuPG. If you trust GnuPG more than the smartcard to generate a good private key, you may generate the private key using GnuPG and then move it onto the smartcard (hint: use the
keytocard command). Make sure you don t leave a copy of the private key on the same machine!
# gpg --card-edit
gpg: detected reader Gemalto GemPC Key 00 00'
...
gpg/card> admin
Admin commands are allowed
gpg/card> name
Cardholder's surname:
Cardholder's given name: host.example.org
gpg: 3 Admin PIN attempts remaining before card is permanently locked
Please enter the Admin PIN
gpg: gpg-agent is not available in this session
gpg/card> lang
Language preferences: en
gpg/card> generate
Make off-card backup of encryption key? (Y/n) n
Please enter the PIN
What keysize do you want for the Signature key? (2048)
What keysize do you want for the Encryption key? (2048)
What keysize do you want for the Authentication key? (2048)
Please specify how long the key should be valid.
0 = key does not expire
<n> = key expires in n days
<n>w = key expires in n weeks
<n>m = key expires in n months
<n>y = key expires in n years
Key is valid for? (0)
Key does not expire at all
Is this correct? (y/N) y
You need a user ID to identify your key; the software constructs the user ID
from the Real Name, Comment and Email Address in this form:
"Heinrich Heine (Der Dichter) <heinrichh@duesseldorf.de>"
Real name: host.example.org
Email address:
Comment:
You selected this USER-ID:
"host.example.org"
Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o
gpg: existing key will be replaced
gpg: please wait while key is being generated ...
gpg: key generation completed (33 seconds)
gpg: signatures created so far: 0
gpg: existing key will be replaced
gpg: please wait while key is being generated ...
gpg: key generation completed (18 seconds)
gpg: signatures created so far: 1
gpg: signatures created so far: 2
gpg: existing key will be replaced
gpg: please wait while key is being generated ...
gpg: key generation completed (23 seconds)
gpg: signatures created so far: 3
gpg: signatures created so far: 4
gpg: key 12345678 marked as ultimately trusted
public and secret key created and signed.
gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u
pub 2048R/12345678 2011-09-19
Key fingerprint = 1234 5678 1234 5678 1234 5678 1234 5678 1234 5678
uid host.example.org
sub 2048R/23456789 2011-09-19
sub 2048R/34567890 2011-09-19
gpg/card> quit
#
Now for the interesting part. OpenSSH talks to an agent for private key handling, and GnuPG s gpg-agent supports this protocol when the
--enable-ssh-support parameter is given. During startup, gpg-agent will print some environment variables that needs to be set when
ssh is run. Normally gpg-agent is invoked by the Xsession.d login scripts, so that the environment variables are inherited by all your processes. However, for an unattended machine without any normal login process, we need to write a script to start gpg-agent. First do these manual steps, to confirm that everything works.
# gpg-agent --daemon --enable-ssh-support > /var/run/gpg-agent-info.env
# . /var/run/gpg-agent-info.env
# ssh-add -L
ssh-rsa AAAAB3N... cardno:000500000BD8
#
If the final step printed a SSH public id, the (sometimes) tricky part in getting the hardware to work is (hopefully) complete. What remains is to script things so that
gpg-agent is started on boot and to make sure that your backup scripts has the proper environment variables before launching whatever processes will launch
ssh. Further, since we will be running unattended, we need a mechanism to unlock the smartcard using a PIN interactively once on each boot of the machine. I prefer manually entering the PIN on every boot over having the PIN stored in a file on the disk.
I will use the
/etc/rc.local mechanism to start gpg-agent, like this:
# cat> /etc/rc.local
#!/bin/sh -e
exec gpg-agent --daemon --enable-ssh-support \
--pinentry-program /usr/local/sbin/pinentry-unattended \
--write-env-file /var/run/gpg-agent-info.env
^D
The astute reader will now ask what
/usr/local/sbin/pinentry-unattended is and why it is needed. Now here is the situation. scdaemon will normally query the user for a PIN using a tool called
pinentry which reads and write to the user s TTY directly. This won t work in unattended mode, so we want the scdaemon to signal failure here unless we are actually unlocking the smartcard manually. Here is the entire script:
#!/bin/sh
# /usr/local/sbin/pinentry-unattended -- by Simon Josefsson
if test x"$PINENTRY_USER_DATA" = xinteractive; then
exec pinentry "$@"
fi
exit 1
What remains is a script to unlock the smartcard by providing the PIN. This is typically invoked manually if the server has restarted for some reason. Don t worry, any ssh sessions invoked by cron until you have managed to unlock the smartcard will fail with an authentication error it won t hang waiting for a PIN to be entered.
#!/bin/sh
# /usr/local/sbin/unlock-smartcard -- by Simon Josefsson.
. /var/run/gpg-agent-info.env; export GPG_AGENT_INFO SSH_AUTH_SOCK SSH_AGENT_PID
gpg-connect-agent 'scd killscd' /bye > /dev/null
while ! gpg-connect-agent 'scd serialno' /bye grep -q SERIALNO; do
sleep 1
done
PINENTRY_USER_DATA=interactive
export PINENTRY_USER_DATA
checkpin
And the script
checkpin is as follows:
#!/bin/sh
# /usr/local/sbin/checkpin -- by Simon Josefsson.
id= gpg-connect-agent 'scd serialno' /bye head -1 cut -d\ -f3
gpg-connect-agent "scd checkpin $id" /bye grep -q OK
At this point, you should have everything configured and installed. Don t forget to
chmod +x the scripts. The typical use-pattern is as follows. After the machine has been started, gpg-agent is running but the smartcard is not unlocked with the PIN. You need to manually login to the machine and run unlock-smartcard and enter the PIN. In the script that runs the backup jobs, invoked via cron, make sure that the first line of the scripts reads (assuming Bourne shell script syntax):
. /var/run/gpg-agent-info.env; export GPG_AGENT_INFO SSH_AUTH_SOCK SSH_AGENT_PID
To avoid needlessly attempting ssh connections if the smartcard is not unlocked, your backup script can also call the
checkpin code and abort if it doesn t return true.
checkpin exit 1
Some final words about debugging. A basic command to run to check that the GnuPG side is working is
gpg --card-status, it should print some information about the smartcard if successful. To check that the SSH agent part is working, use
ssh-add -L. If you get error messages, try killing the
scdaemon process by running
killall -9 scdaemon and let
gpg-agent respawn a new
scdaemon process.
That s it! If you like my writeup, please flattr it.
I received recently a mail about my attempt to provide 
This post is made in a personal capacity. While it represents the outcome of multiple discussions on the topic with a variety of people, nothing written below should be construed as Red Hat's corporate position
Rapha l Hertzog 
A month ago I started a new job, at 
I am currently running a 
A couple of times I have written code which basically does this:
The 
Good morning all!
It feels like forever since I last blogged . Which is extremely bad of me *slaps own wrist*. However, I m been horrendously stupidly busy. As some of you may know, I m very much an Android fan. Recently, I ve gotten myself a shiny Galaxy SII.
Given the fact that alongside my day job of being a Web Developer for an Online Retailer, I m also a 
Jean-Yves Lefort s 
Dear Lazyweb .
I ran into a very strange issue with MySQL authentication. The initial situation is as follows:
No, my blog isn't dead, and neither is me. It's just that way too many things happened since 
Poor Sarah. 
I was expecting to write more about the 7th Rugby World Cup, but real
life, Debian work and running activities prevented me to do so..
Still, I watched several games and I can share my feelings now.
First of all, this is a tremendous organization from New Zealand. It
seems that about the entire country is working on making this even a
great success and I really appreciate to see a place I certainly have
to visit some day receiving such wordlwide attention (OK, admitedly,
mostly in the part of the world that understands rugby).
First round already gave several surprises even if the 1/4 finals were
after all kinda expected (at least, the list of countries).
