If you are following this blog, you probably remember
per-process namespaces. Today, I m going to tell you how I did use them in the process of preparing this server to be upgraded to Lenny.
I must say I was not using the most recent stuff on this server, and this is why I needed such preparation. First, this server is still running with php4. And well, the following line in the
apt-get dist-upgrade output got my full attention (emphasis mine):
The following packages will be REMOVED:
cacti libapache2-mod-php4 libarchive-tar-perl libcurl3-openssl-dev
libgssapi2 libpci2 librpm4 libssp0 linux-kernel-headers modutils php4-mysql
While this server is not important enough not to be broken for a couple hours, I do like to test procedures that could help on servers that are.
The first thing I needed to do on this server was obviously to upgrade php. But we all know how php applications are not fully compatible with all versions of php, so I also needed to test the upgrade was not breaking anything.
On a server you don t care much about, you can just upgrade, test if all is okay, and be done with it. Obviously, if all is not okay, your visitors will see it, and you may also have a bad time downgrading.
Another way to perform the upgrade is to have a similarly installed server on the side, test and validate the upgrade there, and replicate the upgrade on the production server if everything is fine. However, that does require additional resources, and possibly to setup them if they don t already exist.
A cheaper way to do the above is to do it on the production server, both in-place and on the side (you ll see what I mean), using unionfs and per-process namespaces. Full containers could be used instead of per-process namespaces (such as openvz, vserver or lxc), but they still require much more setup, especially when you don t use them in the first place. Chroots could work just as well as per-process namespaces, but one of the ideas here is to expose the per-process namespaces feature, and allow for improvement of this procedure with pid and network namespaces, which are not available in Etch (where I m starting from), but are in Lenny.
Unionfs allows to merge several directories into a single one, accessing some read-only, and others read-write. Installing unionfs is as easy as running the following command:
apt-get install unionfs-modules- uname -r
I won t describe all the kinds of setups that are possible with unionfs, but only one typical use case, which is what we will be using here:
# mkdir /tmp/root-cow
# mount -t unionfs -o dirs=/tmp/root-cow:/=ro none /mnt
The first thing we do here is to create an empty directory. Next, we merge it with the root filesystem (
/), that we will keep read-only (meaning unionfs won t allow itself to write there), and mount the merged filesystem under
/mnt.
none could just be anything, as there is no device to be mounted.
The result, in
/mnt, is just something that looks like the root filesystem:
# ls /
bin boot dev etc home initrd lib media mnt opt proc root sbin srv sys tmp usr var
# ls /mnt
bin boot dev etc home initrd lib media mnt opt proc root sbin srv sys tmp usr var
But creating or modifying a file will do so in
/tmp/root-cow:
# echo a > /mnt/a
# cat /a
cat: /a: No such file or directory
# cat /tmp/root-cow/a
a
# echo foo.com > /mnt/etc/mailname
# cat /etc/mailname
glandium.org
# cat /tmp/root-cow/etc/mailname
foo.com
# find /tmp/root-cow
/tmp/root-cow
/tmp/root-cow/etc
/tmp/root-cow/etc/mailname
/tmp/root-cow/a
Keep in mind that it doesn t include submounts, though:
# ls /var
backups cache lib local lock log mail opt run spool tmp www
# ls /mnt/var
This means we ll have to also mount
/var,
/proc,
/dev, and
/sys.
That s enough testing for now, and we ll first do some cleanup before going after the real job:
# umount /mnt
# rm -rf /tmp/root-cow
Using the
newns utility from my
first post on per-process namespaces, let s create a new namespace to keep our testing private, and populate it with the necessary mount points:
# newns
# umount /tmp
# mount -t tmpfs tmpfs /tmp
# mkdir /tmp/root-cow /tmp/var-cow
# mount -t unionfs -o dirs=/tmp/root-cow:/=ro none /mnt
# mount -t unionfs -o dirs=/tmp/var-cow:/var=ro none /mnt/var
# cd /mnt
# pivot_root . mnt
# mount --move /mnt/proc /proc
# mount --move /mnt/sys /sys
# mount --move /mnt/lib/init/rw /lib/init/rw
# mount --move /mnt/tmp /tmp
# mount --move /mnt/dev /dev
The second and third statements are useful to avoid sharing
/tmp with the main namespace, which means the directories we create in the fourth statement won t be visible in
/tmp outside this namespace.
The fifth and sixth statements put the union filesystems in place. As my system only has a separate
/var filesystem (no
/usr, and I don t care about
/home here), I only need to setup these two. Feel free to add more union filesystems as necessary.
The pivot_root call allows to switch to the unionfs ed root: everything under
/mnt (
/mnt and
/mnt/var in our case) will be remounted under
/, while what was under
/ is remounter under
/mnt under the new root.
This means, in our case, that we have
/ and
/var as union filesystems, while the old
/ and
/var are respectively in
/mnt and
/mnt/var.
It also means
/dev,
/proc,
/sys and other filesystems are remounted as
/mnt/dev,
/mnt/proc,
/mnt/sys, etc., which is why we next
mount --move all of them to a better place in our namespace.
Once all this setup is done, we are ready to do our php upgrade test. As Etch doesn t have support for neither PID namespaces nor network namespaces, we ll still have some conflicts with the main namespace for TCP port binding and process handling, so we need to be a little careful:
# rm /var/run/apache2.pid
# echo Listen 8080 > /etc/apache2/ports.conf
With these changes, we can now safely start apache2 in this namespace, at the same time the main apache2 runs in the main namespace:
# /etc/init.d/apache2 start
Now, there are actually 2 problems showing up in apache in this setup.
The first one is that displaying static files doesn t work. At all. It appears unionfs doesn t support sendfile(). Which is fine. But apache doesn t check for sendfile() s error cases and doesn t fallback to a working solution when sendfile() doesn t work. So we have to manually disable it:
# echo EnableSendfile off > /etc/apache2/conf.d/sendfile.conf
The second one is that access to the mysql socket doesn t work properly under the unionfs. I didn t want to investigate further, so I only worked around the issue by forcing to use the socket outside the unionfs:
# mount --bind /mnt/var/run/mysqld /var/run/mysqld
Finally, we can test and validate our php upgrade:
# apt-get install php5-mysql php5-cli php5-snmp libapache2-mod-php5
# /etc/init.d/apache2 restart
# apt-get remove --purge php4-common
Note that installing libapache2-mod-php5 will remove libapache2-mod-php4, and apache2 gets restarted, taking into account this change. But the php modules (php5-mysql and php5-snmp) install is only going to happen after that, and no apache2 restart is triggered then, which leaves a half working php setup
Also note that the cacti setup in etch supposes php4 is installed, using a
IfModule statement against mod_php4.c and none for mod_php5.c, which means a part of its setup doesn t work out of the box (most notably, the
DirectoryIndex).
Once all is validated, all we need to do is to stop apache2 and exit from the namespace. The union filesystems and temporary filesystems are then automagically cleaned-up and the namespace disappears as well as all modifications we just did, since all processes using the namespace ended. We are now free to upgrade on the production server, as we know all the side effects.
I ll now be able to upgrade to Lenny without php getting removed.
I ve been using git this last few days and I am still working on a workflow for my projects. Unfortunately, as others have noticed, git violates POLS is so many ways, it ends up being hard to get.
Creating a remote repository seems to be the first thing to bite a developer switching to git (mainly if coming from a centralized SCM). I have not decided which is the best way of doing it, but I ve been using git-daemon via inetd and a path in my remote hosting holding all my repositories for public pulling, and ssh for pushing. Here is how to create it:
All of this being said, it s time to revisit this question is all of this needed for a smart , better by design next-generation SSD such as Intel s? Aligning your file system on an erase block boundary is critical on first generation SSD s, but the Intel X25-M is supposed to have smarter algorithms that allow it to reduce the effect of write-amplification. The details are a little bit vague, but presumably there is a mapping table which maps sectors (at some internal sector size we don t know for sure whether it s 512 bytes or some larger size) to individual erase blocks. If the file system sends a series of 4k writes for file system blocks 10, 12, 13, 32, 33, 34, 35, 64, 65, 66, 67, 96, 97, 98, 99, followed by a barrier operation, a traditional SSD might do read/modify/write on four 128k erase blocks one covering the blocks 0-31, another for the blocks 32-63, and so on. However, the Intel SSD will simply write a single 128k block that indicates where the latest versions of blocks 10, 12, 13, 32, 33, 34, 35, 64, 65, 66, 67, 96, 97, 98, and 99 can be found.
This technique tends to work very well. However, over time, the table will get terribly fragmented, and depending on whether the internal block sector size is 512 or 4k (or something in between), there could be a situation where all but one or two of the internal sectors on the disks have been mapped away to other erase blocks, leading to fragmentation of the erase blocks. This is not just a theoretical problem; there are reports from the field that this happens relatively easy. For example, see Allyn Malventano s 
Got myself a new 
It's time to drive some buzz this way ;-) Although this post will only be a pointer towards the Spanish post I made on 
So, I've had this plan for ages to follow up with what I did
with my 
Alright, Debian GNU/Linux 5.0 AKA as Lenny 
What does the new stable release bring for system administrators? I ll give an overview what news you might expect when upgrading from Debian GNU/Linux 4.0, codename Etch (released on 8th April 2007) to the current version Debian GNU/Linux 5.0, codename