This blog post is a little recipe to do a Debian migration for a node using Puppet and some other good practices.
We do all the following commands as root, which on of the exceptional situation where you should have a real root session (login through console, or
su -).
I tend to avoid using the X server while doing an upgrade, so my 'best' setup is to have a laptop to take note and check things on the internet and a session to the computer to upgrade (ssh +
su - or login as root into console). In both case, I use
screen during the upgrade so that I can handle disconnection.
Create or update your admin-syscheck script
First of all, a good practice is to have a script that runs various test on the system that checks everything is ok. This is not only for upgrade but in general. But in the case of upgrade, it can be particularly useful. I call this script admin-syscheck. This is a simple bash script.
This script check various aspect of the system and serve me as my external worker to check the most advanced knowledge I have gathered about setting up a service. For example, I know that having
*.bak or
*.dpkg-dist in
/etc/ means that something needs to be merged and a file should be deleted. Another example is about setting up the right alias for
127.0.0.1 and
::1 (which you can differentiate using
getent ahostsv4 and
getent ahostsv6).
I have packaged this script and distributed it using a specific apt-get repository. You can just distribute it using puppet. I recommend to run it daily to track changes (e.g. after an
apt-get dist-upgrade) and to check that my setup is aligned with my most up-to-date knowledge about setting up a service (i.e. this is my external worker).
In our case we are interested in checking presence of old and new configuration files, before and after upgrading. Here is this specific section of my script:
if ! $IS_ROOT; then
warning "Not trying to detect dpkg leftover file in /etc/."
else
LEFTOVER_FILES=( $(find /etc/ \
-name "*.dpkg-dist" -o \
-name "*.dpkg-old" -o \
-name "*.ucf-old" -o \
-name "*.ucf-dist" -o \
-name "*.bak") )
for i in "$ LEFTOVER_FILES[@] "; do
if [ "$i" = "/etc/hosts.deny.purge.bak" ]; then
continue
fi
if $fix; then
BASE=$ i%.*
cond_exec vim -d $BASE $i
read -p "Delete $i (y/N)? " ans
if [ "$ans" = "y" ]; then
cond_exec rm $i
fi
else
report_error "dpkg leftover file: '$i'."
fi
done
fi
(cond_exec allows to do a dry run, you can just remove it).
Setting
$fix to
true will spawn a
vim -d old new command where you can edit and then delete the leftover file. This is extremly handy.
Upgrading to Wheezy
I strongly recommend to read first the
upgrade chapter of the release notes. This gives you a more complete overview of the upgrade procedure. I just go through the basic step here.
1. Update everything on the system:
$> apt-get update
$> apt-get dist-upgrade
2. Check that the current configuration apply cleanly:
$> puppet agent --test
3. Run admin-syscheck:
$> admin-syscheck
And fix all the problems.
4. Disable puppet:
I use a cronjob to run puppet, so I just comment the line for the job (
/etc/cron.d/puppet-custom). You should disable puppet by stopping the daemon and preventing it to run by editing
/etc/default/puppet and set
START=no.
4. Fix your sources and pinning:
Change squeeze to wheezy in
/etc/apt/sources.list and remove useless files in
/etc/apt/sources.lists.d/.
(You may keep certain sources that refers to stable, like google-chrome.list).
$> rm /etc/apt/sources.list.d/* # Check if this ok to do this with your system.
Although, I tend to fully purge
/etc/apt/sources.list expect for the main line (removing backports and security is fine for a short time). The first run of puppet after upgrade will anyway reset this file.
$> rm /etc/apt/preferences.d/* # (at least the ones that do pin some version)
You can although remove all pinning from
/etc/apt/preferences.
5. Now you start the real upgrade:
$> apt-get update
$> apt-get dist-upgrade
6. During the upgrade you will be asked if you want to keep old configuration files or install the newer one from the maintainer.
I have always wondered what to answer to this. But here is the answer after a few major upgrade: always install configuration files from maintainer if the service has no ultra-specific settings that could break during the upgrade.
The only file, that I should not upgrade on my system, is
/etc/sudoers. In this very specific case, you need to make sure before the upgrade that the old and new configuration can coexist. In the squeeze to wheezy case, I have just setup a few extra augeas rules to set the
secure_path before the upgrade and it was fine. This is typically the kind of situation where you are thankful to have a real root session.
7. The upgrade can be long and require various fixing to remove/re-add packages to circumvent problems. At the end you will have a set of file *.dpkg-old and *.ucf-old (and some *.dpkg-dist and *.ucf-dist). The
*-old files are your old version of the file, while the corresponding files match the maintainer version of it. The
*-dist files are the maintainer version of the file and the corresponding files match your old version of it.
Starting from here you have 2 options:
- This is one of the first computer you upgrade, go to 'first upgrade'.
- Your puppet configuration for wheezy is already bullet proof, go to 'further upgrade'.
First upgrade
This is the tricky part, you'll have to spend a little time on it:
1. Go over all *. ucf,dpkg - old,dist and merge them with the corresponding configuration file. Use
admin-syscheck with
fix=true
2. Make a copy of your
/etc directory into
/etc.new:
$> cp -rp /etc /etc.new
3. Run puppet again:
$> puppet agent --test
4. Disable again the automatic run of puppet, if the previous command has re-enabled it.
5. Make a diff between
/etc and
/etc.new. Since you have a run of puppet, you know what has changed and should not have changed.
$> diff -Nurd /etc.new /etc
Everytime you find some files that doesn't match your expectation for the upgrade with puppet, change the corresponding puppet manifest to have what you expect.
For example, if this is a file:
if ($lsbdistcodename == 'wheezy')
file
"/etc/foo":
source => "puppet:///files/foo.wheezy"
else
file
"/etc/foo":
source => "puppet:///files/foo.squeeze"
People working with augeas and puppet, will appreciate the fact that they probably have 0 changes to make for this to work (since it only does a few replacement in configuration files).
6. Once you are happy with the changes, copy back
/etc.new to
/etc and go to step 3 until the difference is almost 0.
7. Re-enable automatic run of puppet.
Do this procedure for a least each computer category you have (e.g. Desktop and Server nodes). Once you are fully confident your new puppet setup works, you will be able to use 'further upgrade' for the other nodes.
Further upgrade.
This one is super easy compare to a first upgrade:
1. Re-enable puppet and have it run at least once:
$> puppet agent --test
2. Merge
*. dpkg,ucf - dist,old with corresponding files (you can run
admin-syscheck with
fix=true). This is mostly a sanity check since you should have already solved most problem with the 'first upgrade' procedure.
That's it.
Enjoy your upgrade to Wheezy with puppet.
This weekend I'm mostly alternating between reading, writing, and trying to avoid death by the plague.[*]
I've switched Lumail to using a UTF-8 aware string library, which means we can now handle the obvious case:
I write a fair number of Unix command line programs, and I want to
have test suites for them. A year and a half ago I wrote a tool to
make that easier: 
Due to the extremely r
RichiH used flint and steel to light the bonfire. We carefully fed it up
from those sparks to a blaze. Put on the biggest logs we could find
to make it last. Now I'm sitting on the hill above it watching folks
gathered around. A poem is being read, in Hindi, then
translated for us to English. Others have shared songs and poems in a dozen
languages, both classics and their own. Out below the darkness of
the lake. This is Debian at 20.
At the start of this DebConf, I gave a talk on
"
This DebConf is obviously quite special for me, being the DPL. It has really been great so far to talk to meet so many people, and especially to meet so many new Debian contributors or Developers. I m really happy to see that the next generation is ready! :)
On Sunday, I delivered my 
OpenMAMA was developed by 
Commercial background
Reuters and Bloomberg are well known as two of the main vendors in the market data industry. Both of them (and other competitors) provide a range of data services and related software. Buying a data feed doesn't automatically give access to the software from the same franchise: the software and APIs are promoted as a valuable product in their own right. It is actually quite possible (and not unheard of) for an institution to run a 
On Sunday I arrived to DebConf 13. It has been so much fun that I didn't have the time to post anything about it!
As usual, I really enjoy meeting old friends and putting faces to nick names. Last night the Cheese and Wine party was once again great.
Not everything has been partying, though. I've been discussing with Enrico ideas for recognising Debian Contributors, as he presented on his talk on Sunday. We still have to discuss further, and obviously, sit down and write a lot of code 
Yesterday we also met with Luk, and discussed what to do with the ancient net-tools package. We had had the idea of writing compatibility wrappers using iproute2, but that turned out to be too complicated and brittle. After looking a the current state of net-tools, and its reverse dependencies, we decided that the best way to go is to deprecate it by asking rdepends to migrate to iproute2 (for most of them it should be trivial), and then downgrade net-tools to optional. It won't be removed from the archive, as people will still want it, but it will not be required by any core functionality any more.
In the next few days, we will be sending an email to debian-devel, and filling about 80 bugs to get rid of the dependency on net-tools, many with patches.
Since becoming interested in triathlon in October of last year, my goal in 2013 was not to just finish a "70.3" (or "Half-Ironman"), but to complete one in under 5 hours.
At the time, each discipline was woefully inadequate; I had not swum in over a decade, my cycling was average and I had only begun running again after years away from exercise.
The somewhat arbitrary target time would dictate my training. For example, the high proportion of time spent on the bike combined with the injury-sensitive nature of running meant that cycling would have a highest return on investment of the three disciplines. Despite that, I would need to take the run seriously but carefully - I decided to enter a series of monthly 10K races where I could steadily reduce my times over the year.
Races themselves formed a crucial part of my preparation and a carefully balanced race calendar was surprisingly important, balancing upcoming milestones that would provide an achievable goal as well as uncovering weaknesses in enough time so they could be worked on.
Whilst I did post about 
(
