There were quite a lot of small changes in my personal e-mail, web server and DNS infrastructure during the last months. While for more than ten years all my mails came together on a Linux account at the Students Representatives of Computer Science (German: Fachschaftsrat Informatik, short: FS Info) at the University of Saarland and all but one domain was hosted at my former employer, now nearly everything (including this blog) has moved to a Hetzner root server run by me and some friends and running Debian Sarge for AMD64. Most of these moves happend divided in small steps during January and February this year. Only one move happend a little earlier than expected: Without any notice all external mail to my FS Info account got lost in a black hole one hop in front of the FS Info mail server. It looks like the RBG of the Dept. of CS at the University of Saarland didn’t like my face —eh— mails anymore and just dropped them. No bounces, no notices. Not that I knew them as friendly and kind, but blackholing active e-mail accounts without notice leads to dismissal at other places. They should really go and hear alphascorpii’s BOFH talk. So my long-time e-mail address abe@fsinfo.cs.uni-sb.de is no more valid and any mail there won’t get read or answered. But I don’t mind if you send any junk there, it’s not my job anymore. For luck the hoster of my deuxchevaux.org domain, Internett, which I used for all non-university e-mail, changed the forward within a good hour, so I only lost incoming mails of about one and a half day (which was the time until I noticed the blackholing), namely most mails written to all of my addresses at 30th and 31st of January 2007. If you wrote me around that time and miss the answers, you now know why. The few people I expected mail from, already have been contacted. So for the future, write emails only to abe@ either deuxchevaux.org or — if you can’t remember how to spell that (no, you’re not the only one ;-) — noone.org (“no one” as in “nobody”, not “no. one” as in “the best” ;-) And so since then none of my coworkers can make fun of me because I don’t have root on the box where my mail is received. ;-) One of the moves also gave me the possibility to get “my own” Jabber server, so the time with those two jabber addresses (abe@jabber.noxa.de and abe@amessage.de), where usually at least one had some problems with some gateways are also over. If you want to contact me using Jabber, use abe@noone.org, equal to my easier to remember e-mail address. ;-) So there’s still left to move: web, mail and DNS of deuxchevaux.org and beckert.at (domain of my parents) and my old web pages on fsinfo.cs.uni-sb.de as well as Planet Symlink. And of course the redesign of my web pages, which I’ve planned since more than a half decade… ;-) The infrastructure in my real-life home also changed: My gateway to the world is now a 266 MHz MIPS based ASUS WL500g Premium (which I named “pluriel” after the multifunctional Citroën C3 Pluriel) running FreeWRT 1.0.1 which replaced tryane, an as noisy (power supply fan) as compact Unisys Aquanta CP mini-desktop with a passively cooled 200 MHz Pentium MMX.

This is what I just wrote on Mark’s blog about Ian’s move to Sun:

“To some extent I m quite excited by what this might mean for OpenSolaris going forward, but Nexenta have been pushing the OS/Debian (or Ubuntu, more accurately) integration kick for some while without actually seeming to get any (public) traction within Sun
I ve also been disappointed by how little Ian seems to be in touch with how linux development works these days, but that s mostly from what he s been writing in public, rather than any particularly interaction with him, so hopefully that s not a fair summary.
I really hope that Sun can actually make this work.”

I thought I’d expand on this a bit, especially in light of my past moaning about Solaris and the installer and package management in the installer specifically.
What I really, really want, is a modern OS, which has an easily extensible and controllable installer, with good visibility and debugging infrastructure, which is very easy to manage on a grand scale - by which I mean hundreds or thousands of machines up to date, secure and consistent. At present, Ubuntu comes closest:

• d-i is a superb installer that is very easy to drive in an automated fashion – far easier than either kickstart or jumpstart in my opinion, even though both have been around far longer!
• People say apt-get, but that rather misses the point – or rather, it’s the icing on the cake. As most Debian or Ubuntu developers will tell you, the real strength of packages on the platform is in the underlying metadata, and the well maintained and enforced packaging Policy.
• Every packager is a specialist – more or less, if you’re packaging something in Ubuntu or Debian it’s because you use it, either professionally or personally, and have an interest in, and knowledge of, making it work as well as possible.
• The FHS – until my $PATH on Solaris is shorter than the next Harry Potter tome, Ubuntu has this won hands down.

However, there are some definite areas where Ubuntu or Debian (or Linux in general) struggle compared to Solaris – the sheer engineering resources that Sun can throw at a problem, and the talent they have available to them do result in fantastic results when they correctly identify a problem space. They also “own” Solaris – there’s no need for them to try and build awareness of a problem, and the correct solution, over a number of disparate communities.
ZFS and DTrace are the hackneyed and obvious projects here, but from a sysadmin perspective I think FMA, while far less sexy, is one of the best things Solaris10 has. And this is what I mean when I say operating system visibility.
The integration of Zones is also far better than Zen on Linux can offer currently, although both Red Hat and skx are working hard to fix this.
I’m really looking forward to the day when I get an OS that solves all these problems…

[UPDATE: I'm closing comments on this post, because the spammers have discovered it -- dnb]

Lars says:

The uncomfortability of Synaptic may be partly due to the fact that it is
a different model from Windows: you get all your programs from one place,
instead of anywhere on the Internet, partly due to just not using Synaptic
much at all, and partly due to Synaptic not being perhaps as easy to
understand as possible. I personally find Synaptic to be very nice, and
use it regularly, but I am biased by having used Debian package managers
for over a decade.

Now, synaptic is a lot nicer for users than scary console frontends like aptitude. But it's really a tool for "power users". Here's a quick list just of what I see when I start it (bearing in mind that my UI design skills suck):

• The "quick introduction" screen that pops up is way too long. The fact that it needs to be so long is already bad.
  
• There's a huge, and I mean HUGE list of sections down the left. What are these? Which one do I look in? For that matter, what is "gnome", and are "shells" programs for the seashore?
  If I want to find software for processing video, do I look in "graphics" or "multimedia"? (and what are these funny "(contrib)" markers?) Which of the huge number of packages in each section do I want? From the point of view of a new user (especially), this categorization is useless, not to mention often wrong or misleading.
  
• The four buttons in the lower-left, which change the display, are a good idea, but they don't say what they do. A label saying something like "Group By" would go a long way here.
  
• I think the version numbers in the main display probably provide too much information for a novice user. They won't really know what to make of them, and if they need the information it can be provided on demand; as it is, it's just one more distraction to sort through while looking for what they really want. The extra space could, e.g., be used to provide more room for the status column (currently labeled "S").
  

So much for superficial observations. There are a few more things that I see that are worth more discussion.

Mechanics of selecting packages

To change the state of a package, you click on the unlabeled little box to its left, then select the correct option from a list containing "Unmark", "Mark for Installation", "Mark for Upgrade", "Mark for Reinstall"...you get the idea. Even if only one option is valid, the menu still appears with all but one option grayed out, and you have to select the one you want.

This is complicated, cumbersome, and surprising. It also introduces terminological confusion, since "Unmark" isn't really what you think of when you go to cancel an installation or whatever.

The best option I can think of here is a pulldown menu (which I believe the GTK+ list widget (god, I almost wrote "control") supports). The menu should default to the current state, e.g., "not installed", and should have options that describe what will be done with the package, e.g., "install", and so on. Only valid actions should appear on the menu. The Package menu should probably adapt in the same way as you select packages. I don't recall offhand how GTK+ menus work, but if possible commonly used commands like "remove" should be closer to the mouse when the menu appears than uncommon commands like "reinstall".

Another choice I can see, if you want to avoid menus, would be something like this: have two checkboxes, one for "install" and one for "upgrade". "install" would be checked for installed packages and not checked for not-installed packages. Checking or clearing it would mark packages for installation or deletion. "upgrade" would mark or unmark packages that were installed for upgrade. Of course, this introduces a redundant column for not-installed packages, which I don't like. It also seems to me that it would be less obvious what's going on for new users. But it would be more efficient for power users (who should all be using aptitude anyway ;-) ).

Progress bar madness

I haven't extensively used synaptic, but several core misfeatures of apt and our archive pretty much guarantee that if it displays progress bars for key operations like loading the cache or updating the package lists, the progress bars either run multiple times (i.e., don't actually tell you how close to finishing you are) or actually go BACKWARDS -- yes, BACKWARDS.

The great package hunt

Maybe the biggest problem is that unless you know what package you're looking for, basically all our package interfaces are useless. There are lots of fancy searching features you could add, but actually a quick test in synaptic suggests to me that this would be going in the wrong direction. Unlike aptitude, synaptic includes descriptions, and searches for stuff users might actually want (like, say, "burn CD") seem to turn up relevant results (i.e., software that burns CDs).

I'm sure the search algorithm could be improved (with inexact matching, better indexing (a package description suffix tree?), etc), but the big problem I see is that you tend to get back several dozen results, many of which are libraries, development packages, command-line software or software that doesn't work; some of the other software is software that can burn CDs but whose purpose is not to burn CDs (e.g., backup software).

How can we solve this? The first thing that springs to mind for me is to use popularity-contest data to weight the list. I know it's not terribly accurate, and it has some skewing problems (stuff in the default install will be favored), but I suspect it would greatly bias the results in terms of software that's useful to many people (using the "recently used" tallies might especially help here).

I don't have many other good ideas on the searching problem; I've seen vague suggestions of counting reverse dependencies, but this doesn't seem like a good metric of useful software to me; trying to extract more information from the short and sometimes inaccurate package descriptions seems like a really hard problem to me. (Google can do cool statistical analysis partly because it has so much data to work with AIUI; even a single Web page typically has orders of magnitude more information than a package description)

Another issue is the problem of finding related software once you've installed one piece. We have "suggests" and "enhances", but they aren't exactly displayed in a prominent or easy-to-understand way by any frontend. I think it might be good to display a clickable list of suggested packages somewhere visible (e.g., at the top of the description box) when the user has selected a package for installation. We could even display a little bar, like on (eg) Amazon: "Debian suggests installing these packages..." ;-)

It might also be interesting (if this is considered acceptable; since the data is anonymous I don't see a problem) to analyze the popcon data to find out what packages are being installed in tandem. I don't know if we have enough information to do it, but it could turn out to be really useful to be able to click on a package and see other packages that are popular with users who installed that package. Going back to online stores again, those "other people who bought 'Types and Programming Languages' also bought 'Introduction to Category Theory'" links are amazingly useful, and something similar would probably be a big help in Debian.

Agi gave me the link on IRC to this rap song for and about nerds. It s like the Beastie Boys rhyming to the BOFH script. I am a fan now.
There is a video on YouTube for raw footage of a performance that CS rapper Monzy gave last summer outside Stanford University s Computer Science building. This footage was shot for the documentary, Nerdcore For Life.

Lyircs and MP3 for Kill Dash Nine by Monzy.

Monzy s blog

We used to be able to introduce new features distribution-wide. Many years ago the debian-policy mailing list was used to discuss new features, and every policy violation had to be fixed by maintainers. Nowadays the developers who can still be bothered to post to debian-policy can only fight about which current practices are uncontroversial enough that they can be documented. Does anybody remember the old times? When did the policy process change silently? How did it happen?

In about half an hour I'll leave for Chemnitz in order to attend the Chemnitzer Linux-Tage 2007. I am very much looking forward to going there. Ever since the LinuxTag left Karlsruhe, the CLT are my favourite Linux event within Germany. Because of this I registered even two talks this year, one in the beginners section and of course the inevitable social talk. ;-) They're both new, so this CLT will feature two premieres. Giving new talks basically means that anything is possible. Very high rock and roll factor! New talks always have the highest risc for "accidents" but also they're usually most fun, because there's still some room (or need) for improvisation left. This is probably also why I tend to lose interest in a talk once I've given it a few times. Though, the BOFH one is of course a golden oldie! PS: Tolimar is a copycat!

I had a full day yesterday. I'm not sure how this happened, but for some reason both Montreal Wiki Wednesday and Yulblog First Wednesdays fall on (ta-da!) the same day of each month. So we get a whole lot of Internet social-software craziness all in one intense burst during the day. Wiki Wednesday last night was a good time. We had it up at La bande passante, the informal meeting space that Robin Millette, Marco and roommates have up near Metro Cr mazie. It's a really nice space for a meeting of <20 people in an informal setting. They have lots of space, equipment for presentations, WiFi, a big table and comfy chairs. Hugh McGuire, Anne Goldenberg and Antoine Beaupr , Marc LaPorte, Seb Paquet and Robin Millette were all there (with various bande-passistes passing in and out as needed). I talked about experiences with RecentChangesCamp 2007, especially the "what we should have done" session at the end of the event. It really informs what we're going to do with RoCoCo in May of this year. In particular, we've made a commitment to try to get invitations and sponsorship requests out in February -- about 3 months before the event proper. We hope this is close enough to the event itself that people won't put it off or forget about it, but far enough away that people have time to make plans. Hugh talked about his new project Visible Politics. It's kind of a Wikipedia-meets-They Work for You. It's a wiki project for sharing objective information about the voting records, positions, and affiliations of Canadian federal politicians. It's still in its infancy, but I think the project can lend a really great hand to making public information about civic activities possible. On top of that, Robin demoed a really cool project for measuring activity on parts of a wiki site, Hot Off the Wiki. His demo versions use the BarCamp wiki, but I think the plan is to expand the tool to work for other wiki engines and sites. It's a good idea. I also demoed the MediaWiki OpenID extension, which I think was pretty interesting. I didn't say it at the time, but I mostly did the demo for Marc, who I think could benefit from one of the US$5000 Open Source OpenID bounty. I believe that TikiWiki meets the popularity requirements and the OpenIDEnabled.com libraries should make it easy to implement. Anyways, good Wiki Wednesday all around. I'm looking forward to the next one in March. tags: wiki wiki wednesday recentchangescamp rcc07 visible politics montreal openid tikiwiki

YULblog when I tell you to blog After WW, Hugh and I went down to the YULblog event at the Quincaillerie on rue Rachel. We actually stopped in and had a poutine at La Banquise really quickly, which definitely put me in a good mood. When we got to the bar around 8:30 -- in my mind, fashionably late -- the place was already jam-packed. There were easily over 50 bloggers there, and maybe closer to 75. My friend Nicolas Ritoux was already there, sitting down at a table eating papadum with a lot of nice folks. I got a chance to talk to Patrick Tanguay about his upcoming trip to San Francisco, and Niko and I chatted in French about how we never chat in French. I also managed to get a whole stack of Moo Cards from the likes of Hugh and Vanou. All in all a good time, although I left a little early to get some baby quality time in. I really love YULblog; it's a great event that makes me feel connected to the local community. tags: yulblog fevrier

Personal note Thanks for the blogroll link from the anonymous author of The Love Shack. My guess is that it's Alex Malinovich, but I'm not sure. Anyways, let me know who you are so I can add you to my roll. tags: rcc07 recentchangescamp the love shack alex malinovich

These are the current translation ratios for French and Czech in unstable, after 17 days of "blitz l10n NMU". As expected, we are going down because of an uncoordinated upload of viewvc. I even went a little bit more annoyed because the maintainer did some changes to the debconf templates, updated *his* Spanish translation...but didn't care to warn other translators. Next time, I'll use my rubber cluebat, damog..:-) We will be hit again becacuse the sing package was re-uploaded by its maintainer iummediately after I NMU'ed it. Unfortunately, he did then correct some typographical errors in the templates and then fuzzied 9 formerly complete translations. Sad. (update as of 09:32 UTC: this will be corrected by the maintainer) Yesterday, I continued working on Thomas Huriaux dynamic status page:

• bindgraph to be uploaded today
• openssl097 due out Feb 6th
• prc-tools due out Feb 6th
• ipsec-tools due out Feb 7th
• libpam-ldap dues out Feb 7th (with a call for updates running)
• php4-rrdtool due out Feb 7th
• glide still waiting for news
• ifhp needs the removal of debian/po

After last post about a FHS amendment to address the structure of user’s home folders, I received a lot of comments and there is one very significant thing that can be changed in the proposal - instead of having $HOME/ .data .cache .config /appname structure, to change that to a mandatory $HOME/.library/appname/ cache config … . This version still has all the benefits of the first solution (configuration for an application can be easily identified and erase, and all cache can easily be excluded from backups using “$HOME/.library/*/cache” regexp) and also has additional benefits, main of which is the ability to later introduce the concept of user installed packages. The idea is that it would be possible to support having /bin, /lib and /share subdirectories in these application directories thus making an ability for the whole application to e packed in a single directory and allowing the application to be installed simply by unpacking this directory. I admit that much of this is glanced over from MacOS X world, but I do not think that it diminishes the idea itself.
Some problems appear there - support of these distributed bin folders, support of separate lib folders, handling of application plugins, handling of dependencies, handling of the application menu, upgrading notifications for the user software vs. system software. But nothing there that can not be solved. I feel that this can bring together FHS and LSB by providing something of an API for software being installed by users. Having no registry of the software in this solution allows for some interesting things, for example having multiple versions of one program just by renaming the application folder.
A lot of specification work is required here, therefore I proposed a workshop on this topic in Debconf7. I hope to have something that everyone can agree on and maybe even some code by then, so that after Debconf7 there can be a formal policy amendment proposal. So there are two questions:

1. Is there something better in $HOME/.cache/appname vs. $HOME/.library/appname/cache ? Any other trade-offs that I have forgot?
2. What do you think about the further idea of users having the ability to install software in their home folders or remove such software in a very simple fashion?

If you want another example, one from a company who any developer would agree is an outstanding engineering organization, here's one: "Sun has maintained binary compatibility between operating system releases for nearly a decade, enabling existing Solaris applications to run unmodified on Solaris 10. This means that Solaris applications developed 10 years ago will run on Solaris 10 unchanged, taking full advantage of new and advanced Solaris features."

Yes, it's true that there is outstanding backward binary compatibility in Solaris. However, if you're going to try use that as an argument in favour of Solaris, try again. Have you ever actually used a Solaris system?

• The ugliness on a Solaris system starts with $PATH, which is all over the place: /usr/ucb/bin for the compiler, /usr/gnu/bin (IIRC) for bash, /usr/bin for sed. The FHS is nonexistent.
• Everyone who writes code for the Berkeley socket API knows that Solaris has to be different:
  

  #ifndef sun
  int yes=1;
  #else
  char yes='1';
  #endif
  

• Even if you do it that way, on Solaris you have to say -lsocket.
• I've heard claims that Solaris, partially because of the backward compatibility, is rather slow; that Linux beats the shit out of Solaris on the hardware (SPARC) that was more or less specifically designed for Solaris (and vice versa). Having used it for a short while once, I cannot think of a counterargument for that one.

If you're going to say that backwards compatibility is good and that we should incorporate backwards compatibility into Linux somehow, then please do not point people towards the two ugliest operating systems on the earth with a lot of garbage baggage for the sake of backwards compatibility...

Erich wonders about a sane way to reorganize the configuration file mess we have, going in fact farther than Aigars' previous rant on the topic: Not only it would be really desirable to do away with the dotfile-as-everything-but-more mess, it would be really, really nice to do it in a more or less standard way. Erich suggests four base configuration syntaxes - I won't reveal too much, as his message deserves being read as well ;-) But hey, I have a question here:
I know XML is well-established and well-supported, and I'm told there are a couple several hundred thousand people who think it is really cool and even human-editable. Still, I hate XML. Configuration files are often, yes, written by the programs themselves - But one of my most beloved features of Unix-like systems is that I am free to poke in them, as they are meant to be human-editable.
XML is not human-editable. I'm sorry, say whatever you want, but keeping XML valid and happy is... Just not for me. Why not pushing instead something prettier, and with almost the same feature set of XML, plus a much-enhanced readability/modifyability? Why not promoting my dear and beloved YAML? (Yes, the YAML project home page _is_ valid YAML)
Note: No, this only looks like a rant, but it is really a question. Honest!

Aigars Mahinovs mentioned that it would be nice if applications would not clutter the home directory with their config files as much. I agree a lot. Dotfiles for configuration suck badly. If you start a new application (or if you are in an early stage and have a small userbase), please follow the Freedesktop.org XDG base directory specification. I.e. use .config/applicationname instead of .applicationname for configuration files (unless an overriding environment variable is set), and .local/share/applicationname for data files. And for cache files, use .cache/applicationname (anyone writing a patch for firefox?) What I'd also like to propose is to reduce the number of different configuration file syntaxes. It sucks having to find out which syntax an application uses again and again, because every application is different. We probably won't be able to convince everybody on one format, but we could maybe cut it down to, like 3 formats, please? There is no reason that one application needs to start comments with "#" and another with ";" while a third application uses "//" but doesn't allow "/* comment */"... A small set of syntaxes (preferrably with standardized parsers) should be good enough for just about every application:

• key=value pairs, quotations optional, comments match "^\s*#"
• ini-style configuration files with limited nesting
• JSON syntax (nesting, all strings quoted)
• XML files (arbitrary nesting, mixed contents, repeated elements)

If we had just these few syntaxes and an appropriate schema description language, we could write generic configuration file editors. That would be very nice. Also we avoid having config file parser bugs by using libraries, and if someone prefers using e.g. gconf or dconf, this could be handled by the library, transparently for the application. (Note that many application lack error handling when writing their configuration files, especially when the disc is full... - they really should be using a library. IIRC xchat also has some bugs of this kind, I remember having tried to patch some of them once when we were preparing for the sarge release, just to find out that there might be many more of them...)

Justification. Currently there is a huge mess of files and folders that start with a “.” in any users home folder. There is no structure or policy on how applications should choose file and folder names for data that needs to be stored in users home directory. Additionally there is no established consistency between Gnome, KDE and most other applications. Gnome application have part of their configuration information in gconf folder and other part in a gnome subfolder. KDE applications have a complex structure under .kde/. And most other applications either have one file directly in users home folder or have their own dot-folder there. Problems. There are two major problems with current ad-hoc approach:

1. Namespace pollution of users home space. In any desktop system “ls -al” in the home folder would be a very lengthy affair. Moreeven this namespace is folded - KDE and Gnome apps live in their own subfolders.
2. Lack of structure in folders makes it much harder to properly understand their purpose. For example, if a user has misconfigured an application and would like to reset it to default settings, it currently could be hard to figure out how to do that. Especially without losing his data in that application.

Proposal. I propose to implement a policy on where an application can store data and configuration information in users home folder. Such specification would be proposed as an extension for FHS, first implemented in Debian and then promoted for inclusion upstream and in other distributions. Structure. I propose a following structure for a users home folder:

• data/appname/ - each application, that stores user data in application-specific format, will have to store this data into this folder. Only configuration information that is essential to load this data would be permitted into this folder (in hidden dot files). For example, Evolution could store emails and email access information here.
• .conf/appname/ - each application that stores any configuration information whould have to store all of it in this folder. GConf will have to also store applications configuration info here. If this folder would be deleted, application must revert to default configuration, but still be able to fully access user data in data/appname/ or other, generic folders (below). For example, if .conf/evolution/ is erased, then on next start-up Evoution should still be able to access all email that is stored in data/evolution/ without any configuration wizards.
• .tmp/appname/ - each application that would need to create temporary files that survive reboot, would put them here. These folders can be safely deleted at any time with no data or configuration loss. For example, Firefox cache information and Nautilus thumbnail cache would have to be here.
• Desktop/ - the desktop folder, just like it is now
• Media folders - (Documents/, Photos/, Videos/, Music/, …) a suggestion of basic document structure for user to use or ignore. Applications dealing with particular media types would default to use files from such folders, but should also cope with working with any other folders on users choice. For example, a screensaver with slideshow function would by default use photos from Photos/, but will have to have a configuration option for user to change that setting.
• Additionally I would propose having a .mounts/ folder where applications could mount user specific media and other mounts, such as FUSE. For example, FUSE could be used for automatic archive introspection.

Benefits The main benefit for such system for me is twofold:

• SBackup can be configured to exclude .tmp/ and .mounts/ by default thus saving significant space in backups.
• A tool can be made to restore settings and/or data for specific applications in an uniform fashion.

So, any suggestions on the idea and on where this should be best discussed?

For nearly two years, I have been pondering about a good and failure-resilient DNS setup to implement for my own domains. In the last days, I have set the first prototype into use. No, I haven’t dared to touch zugschlus.de, my most important domain, yet. This is planned for the weekend. So, if you experience difficulties in accessing any of my Internet services, please inform me and allow me to fix the issue. However, zugschl.us, marc-haber.de and a number of less important test domains have already been moved to the new, svn-driven Scheme and delegation has been changed to the new DNS servers, dns1.nosuid.net, dns2.notwork.de and dns3.mxonly.de. q.bofh.de, my old main DNS server, continues to run as a slave for a transition period, and it continues to feed the slave servers run by various commercial and non-commercial entities that have kindly been serving the zones in the last years. The new servers are all run by myself. dns1 and dns2 are dedicated servers (“real” machines that both have other duties as well) located at two different ISPs/hosters located in different parts of Germany, while dns3 is a vServer located in the US (which only does DNS due to its rather severely limited amount of memory). The three domains the servers are located in are in two different top level domains, are registered through two different registrars and are not used for anything besides DNS (and providing the host name for ivanova.notwork.de, one of the DNS servers and the machine hosting this blog). The two .de-Domains mxonly.de and notwork.de are MX only domains, which means that they are not delegated outside of the .de zone but their resource records are directly written to the .de TLD zone. This reduces the chance of the domains becoming unavailable due to failures in the delegated servers (because there are none), and we depend on .de being available anyway. A downside of MX only domains is that the number of registrars that support them through their automatisms is rather limited. The .net zone is served by the registrar’s name servers since .de is the only TLD I know of that has reasonable prices and offers MX only domains. One can argument that the .net zone is a weak point here, but I suspect that I’m reasonably safe with the MX only .de domains that the .net domain is only a paranoid safety catch anyway.

Experiences with a Trac Server In the last few days after Xmas I set up http://trac.cs.unibo.it, a server for my CS department which will host a bunch of Trac instances, initially for academic projects only then it will be opened to student projects as well. The server set-up has been "powered-by" the following technologies:

1. Debian Etch
2. Apache 2.2 with FastCGI, SuExec, and WebDAV over TLS (for accessing SVN repositories without forcibly passing through Trac)
3. Trac instances backed by Sqlite databases and Subversion repositories
4. Some bits of Genshi templating for generating the projects index
5. Some shell scripts for automating creation and removal of projects

I'm quite happy with the result. From the overall experience I drew some observations though:

• installing Etch is more a dream than ever, in 15 minutes I re-installed the server (which previously was running Ubuntu) with a decent partitioning scheme and all that I needed to work remotely. The sysadm of the department saw me coming out from the server room and asked, unbelievingly, did you already finish?
• apache2.2 in Debian is now annoyingly missing the nice script for creating easily TLS certificates (i.e. apache2-ssl-certificate). I followed-up on #395823 attaching the same script took from Sarge and the needed configuration file. Worked for me
• SuExec is stupid. You have to put stuff under /var/www for it to work at all, the only way to change the path is rebuild apache .... or, you can mount --bind /srv/trac /var/www/trac!
• BTW I like /srv. According to the FHS it's the place where I want to put my Trac projects. Unfortunately the Debian installer neglects that path and per default you end up with a system where /srv is in the same partition as /, and hence with basically no free space. Hence I also bind-mounted /srv/trac from /home/trac
• In Etch we have no available implementation of FastCGI (which really improves Trac performances BTW). libapache2-mod-fastcgi is non-free and for some reason it was not autobuild everywhere and hence is missing from testing. I pinged aba on IRC about that, so the package should probably enter testing again in a few days. Go RMs!

Happy new year birthday trac.cs.unibo.it.

During the past month, I’ve been working on migrating some servers onto a VMWare ESX Server. I did test VMWare Workstation a long time ago, and have had a VMWare Server on my work laptop since it has been freed (as in beer), but ESX Server is something else. It seems to me, though, that except it can run unmodified OSes without VT/Pacifica technology, it is not technically superior to what you can do with Xen and other free software. For instance, it may be possible to do better virtual switching setups with “standard” linux bridges and ebtables. But from the administrator perspective ESX Server has the advantage of its administration console. I’m still waiting to see a nice and featureful free (as in speech) configuration software for Xen (or maybe, dear lazyweb, could you show me some good urls I missed). We don’t have the full VMWare Infrastructure, though, so I can’t speak of VMotion or VMHA, but that sounds neat, on paper. I never tested Xen migration either, so, I won’t say it’s better :) Anyways, VMWare ESX Server is a pretty good product, but there are quite a few quirks, or even really painful misfeatures:

• Sometimes, the console shows more items than what the user is supposed to see. Though you still can’t act on these, you wouldn’t expect them to show up (for example, I sometimes see items that are only supposed to appear if you have a Virtual Center, which we don’t have).
• If you rename a virtual switch, you have to go through all the VMs that were connected to it to change their network configuration accordingly.
• You now have to edit the settings to connect/disconnect the CD drive or the network. That used to be less annoying with the console software in version 2.5.
• You can’t display more than an hour of performance graphs without a Virtual Center. Pretty painful when you only have one ESX server. (also known as the “buy more of my products business plan”)
• VMFS doesn’t maintain coherency between readdir() and stat(). The d_ino readdir() returns in its struct dirent and st_ino in stat()’s struct stat don’t match. This is especially annoying with Legato Networker, which checks that coherency and doesn’t save files that “changed inode”. To circumvent this misfeature, I installed fuse and slightly modified libfuse and the fusexmp_fh example so that I can mount a mirror of /vmfs with coherent inodes. Now VM disks can be safely saved.
• It’s impossible to create a loopback device on a file residing on VMFS. The filesystem doesn’t accept the LOOP_SET_FD request. This means that, while the VM disks files are basically raw disk images, you can’t directly mount the filesystems on the service console with a loopback device. Again, with the modified fusexmp_fh program, this is now possible.
• While there was a (quite broken, as in kernel freeze) way to mount filesystems from VM disks with ESX server 2.5 (which we also tested before upgrading to version 2) with vmware-mount, the only “official” way I found to do this with version 3 is to use vcbMount, which requires a VMWare Virtual Consolidated Backup server (not really free neither as in speech nor as in beer ; seems to be another instance of the “buy more of my products business plan”), and an extra server connected to the SAN.
• …

As said above, we can now setup a loopback device on the VM disk files (which needs a little trickery with offsets to get the partitions positions right, but that’s not very hard). While it’s possible to mount filesystems from an offline disk this way, it’s not a good idea to mount an ext3 filesystem from a running VM, because the filesystem is flagged for recovery, and the service console kernel would want to replay the journal, which may have nasty side effects. I don’t know for NTFS yet. There may be a solution to “cleanly” mount filesystems from an online disk, though (not yet tested):

• Snapshot the VM which the disk is on. After that, the -flat.vmdk file is frozen (only if it’s the first snapshot).
• Use an unionfs or funionfs (over fuse) to keep the real -flat.vmdk file readonly but still can write on it, so that the service console kernel can replay the journal on the writeable part of the unionfs.
• Loopback mount partitions from the image in the unionfs.

That would be pretty pervert (ext3 over loopback over unionfs over vmfs), but should just work. I’ll post a detailed procedure.

Almost one year ago I got a great present from a great man, a Linksys Wireless router with a serial port that he had soldered on it himself, running OpenWrt (White Russian). Sexy!

This toy has made me happy ever since, even though this great man insisted on me using WPA, which basically means most of my friends can t use wireless at my place (blame crappy firmware), and I still, from time to time, have some wire mess around my living room, but that is also fun, specially for the cats, so no complaints here :)

I recently upgraded to White Russian RC6, and this great man pointed me to a wonderful web interface to manage the system with: X-Wrt.
This web interface is very helpful for me, even though I can ssh into the router, I am used to my aliases, longer history, my way of working on the command line, which makes my ssh sessions there a bit frustrating, as I am getting old and a bit reluctant to changes. The web interface also has nice graphs and stats, a beautiful CSS layout, and it makes my life 100% easy. This stuff just works^Wrocks!

The installation system was so easy I was amazed. Just fill in a form with your Linksys IP address, click on install and you are done!

I miss https for the config interface, and I would love to find a Linksys model that supports OpenWrt and does DSL at the same time, so I could get rid of the crappy 3com doing the ATM part in bridge mode. Less devices, less power wasted, less points of failure, happier me.

The amount of packages available for OpenWrt is just amazing, and it has a package manager that slightly reminds me of apt-get. It can even do asterisk!

So if you own any of the supported devices, I encourage you to get rid of the crappy firmware they usually ship with, and have some White Russian fun with it, which also happens to be one of my favourite drinks :)

Enjoy!

... or "why breaking your SONAME is a bad idea". If you ever had any questions on why it is a bad idea to break your SONAME (i.e. not changing it despite of ABI and/or API changes in your library), please take a look at the current Debian bug list of gtkpod. Oh, and it also shows why it is a specially bad idea to prepare a package for a library broken in that regard without at least changing the package name... So, to all the Debian gtkpod users out there: Please don't use libgpod from debian-multimedia.org together with gtkpod 0.99.4 from Debian. I will try to make sane versions of libgpod 0.4.0 and gtkpod 0.99.8 available as soon as possible. If you want to help with this (the RFH for gtkpod is still open), please don't hesitate and drop me a mail.

A great idea from ij: why don't we outsource the release process to third world countries? African developers could work on the release for the price of a few glass beads. People complaining that 6000$ is too much for a month of work by a computer professional are showing that they have no clue about how a single-person company works financially.