I tried the installation of KDE 4.1 beta 1 on Debian, as described here. I ended up undoing this about an hour later. The environment isn't really usable yet, so my advice to the public is: don't do it, on a system you plan to actually use.

Here is a random assortment of problems I encountered:

• You can't configure size and appearance of the panel. This is a long-standing problem, but I thought it was fixed.
• In general, there don't appear to be all that many configuration options available to replicate various behaviors that I have become accustomed to.
• Rendering problems in the system tray; icons appear on top of each other.
• Various kicker widgets don't appear to be available for KDE 4 (yet?).
• Most applications don't copy their old settings, but it appears that some do?
• KMail doesn't interact well with encrypted IMAP accounts. I didn't try much beyond that, because I have no accounts that are not affected by that.
  
• Clicking on an RSS link in Konqueror doesn't link to Akregator anymore. Probably, various MIME type associations are missing.
• I couldn't get the environment to remember to open terminal windows maximized. Very annoying.
  
• I couldn't add any application icons to the panel. I'm not sure where on is supposed to store one's favorites now.
  

So it was quite obvious that that various pieces don't really fit together all that well yet. On the upside, I noticed many really good ideas that I would look forward to using if the overall environment worked.

The good news is that the packaging appears to be really robust (I found one packaging bug with a file conflict) and the upgrade and downgrade works well (you have to use aptitude, as described in Ana's post; apt craps out completely).
So when KDE 4 is actually usable, Debian will be ready. But not for lenny; that would be suicide in my opinion.

Today a client asked me to advise him on how to dramatically reduce the number of servers for his business. He needs to go from 18 active servers to 4. Some of the machines in the network are redundant servers. By reducing some of the redundancy I can remove four servers, so now it’s a need to go from 14 to 4. To determine the hardware requirements I analyzed the sar output from all machines. The last 10 days of data were available, so I took the highest daily average numbers from each machine for user and system CPU load and added them up, the result was 221%. So for the average daily CPU use three servers would have enough power to run the entire network. Then I looked at the highest 5 minute averages for user and system CPU load from each machine which add up to 582%. So if all machines were to have their peak usage times simultaneously (which doesn’t happen) then the CPU power of six machines would be needed. I conclude that the CPU power requirements are somewhere between 3 and 6 machines, so 4 machines may do an OK job. The next issue is IO capacity. The current network has 2G of RAM in each machine and I plan to run it all on 4G Xen servers, so it’s a total of 16G of RAM instead of 36G. While some machines currently have unused memory I expect that the end result of this decrease in total RAM will be more cache misses and more swapping so the total IO capacity use will increase slightly. Now four of the servers (which will eventually become Xen Dom0’s) have significant IO capacity (large RAIDs - they appear to have 10*72G disks in a RAID-5) and the rest have a smaller IO capacity (they appear to have 4*72G disks in a RAID-10). The other 14 machines have the highest daily averages for iowait adding up to 9% and the highest 5 minute averages adding up to 105%. I hope that spreading that 105% of the IO capacity of a 4 DISK RAID-10 across four sets of 10 disk RAID-5’s won’t give overly bad performance. I am concerned that there may be some flaw in the methodology that I am using to estimate capacity. One issue is that I’m very doubtful about the utility of measuring iowait, one issue is that iowait is the amount of IDLE CPU time when there are processes blocked on IO. So if for example you have 100% CPU time being used then iowait will be zero regardless of how much disk IO is in progress! One check that I performed was to add the maximum CPU time used, the maximum iowait, and the minimum IDLE time. Most machines gave totals that were very close to 100% when those columns were added, so it seems that if the maximum iowait for a 5 minute period plus the maximum CPU use plus the minimum idle time add up to 100% and the minimum idle time was not very low then it seems unlikely that there was any significant overlap between disk IO and CPU use to hide iowait. One machine had a total of 147% for those fields in the 5 minute average which suggests that the IO load may be higher than the 66% iowait number may indicate. But if I put that in a DomU on the machine with the most unused IO capacity then it should be OK. I will be interested to read any suggestions for how to proceed with this. But unfortunately it will probably be impossible to consider any suggestion which involves extra hardware or abandoning the plan due to excessive risk… I will write about the results.

Share This

The way .ssh/authorized_keys is typically used is not secure. Because using it securely is hard, and dumping in passwordless ssh keys is easy. I spent about 5 hours today locking down my authorized_keys. rsync The only easy case is a rsync of a single directory. Follow these good instructions. If you need to rsync multiple separate directories, it's easy to find several documents involving a validate-rsync.sh. Do not use, it is insecure -- it allows rsync to be run with any parameters. Including parameters that allow the remote system to rsync in a new ~/.ssh/authorized_keys. Oops. (You can probably also trick validate-rsync.sh into running other arbitrary commands.) To be secure, you have to check the rsync parameters against some form of whitelist. git Locking down git should be easy. Use git-shell. Unless you also want to be able to log in to an interactive shell on the same account. To do that, you'll need to generate a separate ssh key for git. Both git servers I use are named git.*, so I set it up like this in ~/.ssh/config on the client:

Host git.*
    IdentityFile ~/.ssh/id_rsa.git

Then on the server, I added the key to authorized_keys, prefixed with this:

command="perl -e 'exec qw(git-shell -c), $ENV SSH_ORIGINAL_COMMAND '"

(I also tried the simpler command="git-shell -c $SSH_ORIGINAL_COMMAND"; but it didn't work with alioth's old version of openssh, and I didn't want to worry about exposing SSHORIGINALCOMMAND to the shell.) Before arriving at that, I took a detour to using gitosis. It can make it impressively easy to manage locked down git over ssh, but you have to fit how it does things. Including repository layout, and a dedicated account. So I spent several hours changing my git repo layout. In the end, I decided gitosis was too complicated and limiting for what I needed, but I do recommend checking it out. svn Like git, this is fairly easy, since svn provides svnserve. The svnbook documents how to use it in .authroized_keys. Basically, just use

command="svnserve -t"

A similar approach as used for git can be used here if you want to have a dedicated ssh key that causes svnserve to be run. d-i daily builds d-i has a d-i-unpack-helper that can be put in authorized_keys. unison Can probably be locked down similarly to rsync, but I haven't tried yet. duplicity The only way seems to not use duplicity over ssh at all, and instead use a different transport. too hard What if you want to allow locked down git, and a rsync, and maybe unison too, all to the same account? You'll end up with some nasty mismash of all of the above, and there are plenty more things using ssh as a transport that need other techniques to lock them down. Until this becomes easier, a majority will just dump passwordless ssh keys in ~/.ssh/authorized_keys, creating exploitable trust paths that don't need to exist.

Tomorrow (Thu 29th) at 7.30pm is the Kewstoke annual village meeting. That should be my first meeting as a councillor, if I get back from London in time. (I hope I do - councillors who miss meetings get this sort of criticism but at least village councillors aren't paid that much: I think expenses for all 12 or so councillors totalled under 60 last year.) Apart from the two reports, the agenda is a good summary of the challenges affecting the village this year: Cygnet Liaison, Sand Bay Management and the Toll Road.

I’m currently deciding where to get a Xen DomU hosted. It will be used for a new project that I’m about to start which will take more bandwidth than my current ISP is prepared to offer (or at least they would want me to start paying and serious bandwidth is expensive in Australia). Below is a table of the options I’ve seriously considered so far (I rejected Dreamhost based on their reputation and some other virtual hosts were obviously not able to compare with the prices of the ones in the table). For each ISP I listed the two cheapest options, as I want to save money I’ll probably go for the cheapest option at the ISP I choose but want the option of upgrading if I need more. I’m not sure how much storage I need, I think that 4.5G is probably not enough and even 6G might get tight. Also of course it depends on how many friends I share the server with. Quantact has a reasonable cheap option for $15, but the $25 option is expensive and has little RAM. Probably 192M of RAM would be the minimum if I’m going to share the machine with two or more friends (to share the costs). VPSland would have rated well if it wasn’t for the fact that they once unexpectedly deleted a DomU belonging to a client (they claimed that the bill wasn’t paid) and had no backups. Disabling a service when a bill is not paid is fair, charging extra for the “service” of reenabling it is acceptable, but just deleting it with no backups is unacceptable. But as I’m planning on serving mostly static data this won’t necessarily rule them out of consideration. It seems that linode and slicehost are the best options (Slicehost seems the most clueful and Linode might be the second most). Does anyone have suggestions about other Xen servers that I haven’t considered? XenEurope seems interesting. One benefit that they have is being based in the Netherlands which has a strong rule of law (unlike the increasingly corrupt US). A disadvantage is that the Euro is a strong currency and is expected to get even stronger. Services paid in Euros should be expected to cost more in future when paid in Australian dollars, while services paid in US dollars should be expected to cost less. Gandi.net has an interesting approach, they divide a server into 64 “shares” and then you can buy as many as you want (up to 16 shares for 1/4 of a server) for your server. If at any time you run out of bandwidth then you just buy more shares. They also limit bandwidth by guaranteed transfer rate (in multiples of 3Mb/s) instead of limiting the overall data transferred on a per-monthly basis (as most providers do). They don’t mention whether you can burst above that 3Mb/s limit - while 3Mb/s for 24*7 is a significant amount of data transfer it isn’t that much if you have a 200MB file that will be downloaded a few times a day while interactive tasks are also in progress (something that may be typical usage for my server). Of course other providers generally don’t provide any information on how fast data can be transferred and will often be smaller than 3Mb/s. Also if anyone who I know wants to share access to a server then please contact me via private mail.

ISP	RAM	Disk	Bandwidth (per month)	Price $US
Linode	360M	10G	200GB	$20
Linode	540M	15G	300GB	$30
Slicehost	256M	10G	100GB	$20
Slicehost	512M	20G	200GB	$38
VPSLand	192M	6G	150GB	$16
VPSLand	288M	8G	200GB	$22
Quantact	96M	4.5G	96GB	$15
Quantact	128M	6G	128GB	$25
rimuhosting	96M	4G	30G	$20
XenEurope	128M	10G	100G	$16 (E10)
XenEurope	256M	20G	150G	$28 (E17.50)
Gandi.net	256M	5G	3Mb/s	$7.50 or E6

Share This

Catching up: some OCaml in Debian news Hei, it looks like I stopped giving updates about OCaml news in Debian, time to catch up. 2 hot news basically:

1. OCaml 3.10.2 transition is ongoing. Release managers did not forget about our old request of transitioning OCaml to 3.10.2 in time for Lenny, and as soon as the last Perl transition was other they pinged us about the transition. Together with Luk we are now working on it. After some initial issues on arm which was unable to run docbook2html on our poor little policy document without segfaulting(!), it now looks like is not going too bad ... get ready for OCaml 3.10.2 in Debian Lenny!
2. the "core" OCaml library by Jane St Capital (which I consider a really good replacement for the legacy standard library FWIW) has entered testing a few days ago. It will be released with Debian Lenny, no matter if with 3.10.1 or 3.10.2

Stay tuned!

I finally sent out a flood of mails to those people that applied for Travel Sponsorship for this years Debian Conference in Argentina. We have 58 people who got a position in our sponsorship queue, asking for nearly 60.000 USD in total. We are not yet sure if we are able to reimburse all of them, but we will at least try to. We also have 24 people rejected from this queue and possibly some people haven’t been able to fill out their settings in our conference management system correctly/in time. While we are sorry for them, we can’t do much here.

foosel tagged me, whatever that means. Perhaps it’s the English word for “Stöckchen” (German for “small stick”) of which I always wondered how the English blogging part of the blogosphere is calling that kind of coercing blog posts… ;-) So these are the rules:

1. blog a list with your favorite desktop Linux software (as many or few you want)
2. add links to the software project’s websites
3. post these rules
4. tag three other Linux using bloggers

Interestingly splitbrain, who started the thing just calls it “Meme”, but to me memes are the same thing just without duress. ;-) So you want to know about what Linux desktop software I like and use, hmm? Desktop means GUI, doesn’t it? There are only a few GUI application I really use often since, as you probably know, X is primarily a terminal multiplexer and screen resolutions are compared by how many 80×25 xterms with fixed font you can get on one screen without overlapping. ;-) But to be honest: Although I’m more the command line guy hacking cryptic lines into windows with small fonts, there are a few thing where I don’t want to miss X and the GUI applications: For all things web – that means web browser, feed reader, etc. But then there is also a bunch of GUI software I use occasionally or as alternative tool to some text mode or command line software. Web

• Liferea – My favuorite feed reader although it takes ages to start and since a few days also starts crashing, probably since I have configured it to cache up to 1000 items per feed and have subscribed to several hundred feeds. I do not read them all though, but I use them togther with Liferea’s “search all feeds” feature as a Google News replacement. ;-) I though read a lot of feeds in it, since I use it for news, blogs, webcomics and to read missed tweets on Twitter. It organizes the feeds in a tree structure so I can easily group different types of content together.
• Opera – I’m back using Opera as my primary web browser since they offer alpha versions for 64-bit Linux. Initally I started using Opera with version 3.60 on Windows 95 somewhere about 10 years ago and I’ve always come back to it when no current free browser fits my needs. Although it hasn’t an AddOn possibility as Firefox has, I still prefer it over the bloaty and leaky and quite unstable Firefox 2, since it offers nearly every functionality I need (mainly mouse gestures and a flexible tab management), is fast, needs less RAM and is quite stable for an alpha version. And Firefox only offers those features I need via Addons which are often the cause for leaking or crashing. Haven’t tested Firefox 3 yet, but it’s said to be be less bloaty…
• Kazehakase – Formerly I used kazehakase as my primary web browser since I really like its user interface, but the version in Etch is quite slow and seems to have memory leaks. It’s currently the second browser I have always open. But since my browsers always have uptimes in terms of months I don’t need web browsers that are leaking, so I’m thinking about replacing it with something more stable.
• Conkeror – A Gecko 1.9 (i.e. Firefox 3) based web browser completely controllable with the keyboard. And the key bindings are those from Emacs and partially also from the classic text-mode browser Lynx. Will be available in Debian Experimental soon.
• Netsurf looks very promising as it’s a simple and fast browser with it’s own rendering engine and originating on RISC OS. But since I’m a heavy tab user (60 tabs in one window are not really seldom), a browser (yet) without tabs isn’t really that useful for me. But I hope it will get tabs soon.
• Midori – The other upcoming new browser in the Linux world is using Apple’s WebKit (which itself is based on KDE’s KHTML) underneath. Only in Experimental yet (form a Debian point of view :-). Use it on my Debian Sid machine to play around with it.
• Twitux – A simple GTK Twitter client which doesn’t clutter the screen with unnecessary icons or buttons. Just a small menu bar, status bar and the tweets.
• Azureus – In the seldom case where I need to download files via Bittorrent I either use Opera’s builtin client or Azureus. The nice thing about Azureus is that you can get nice graphical as well as textual statistics about all aspects of your downloads.

X / Desktop Environment

• FVWM – My favourite window manager for normal, big or multiple screens. I use it since more than 10 years (twm and tvtwm were its predecessors) and its configuration has evolved since then quite a bit to tinted transparent window frames and title bars, etc. I tried other window managers in between (e.g. Sawfish and GNOME’s own Metacity, each for a month or so and both together GNOME, also played around with KDE on one machine) and I always came back to FVWM. No other window manager is so fast and configurable in regards of keybindings. Handles multiple screen very well and out of the box, too.
• ratpoison – My favourite window manager for small screens (less than about 1024×768, e.g. on my EeePC, on the 8” touchscreen connected to my MicroClient Jr. or on my 1996 ThinkPad 760ED with 133 MHz Pentium 1) since it doesn’t waste screen space for window borders or title bars. It just maximizes all windows by default to screen resolution. You then can manage (split, resize, switch, close, kill) windows as you are used to manage shells and text-mode applications with screen(1). Doesn’t work that well with multiple xrandr managed screens though if they don’t have the same size.
• FLWM – The Fast and Light Window Manager. My favourite low-end but still DAU compatible window manager. Use that on demo and guest accounts, especially on low end machines.
• Synergy – connects displays of other computers (not only X but also even Mac or Windows) with your mouse and keyboard similar to a KVM switch. I use it at work to add my laptop as fourth monitor. ;-)
• trayer – A desktop environmen independend system tray developed by the FVWM Crystal Project. Since I changed from manually editing /etc/network/interface on my laptop each time I came into a new wireless LAN to using GNOME’s Network Manager, I needed a system tray for the nm-applet. Trayer is quite easy to configure using command line options and can handle tinted transparency as I use with FVWM and ATerms. So it fits in perfectly.
• ratmenu and dmenu – For showing generated menus together with ratpoison, I use ratmenu (e.g. as replacement for ratpoison’s non-interactive window list) and dmenu (e.g. as application menu using my own wrapper which generates the menu from some config file). Probably will publish that code once it proved itself stable.

xtrlock – the simplest tool to lock you desktop: The mouse turns into a lock and it only goes away if you enter the right password. No screen saver included though and everyone can see what’s on your desk. I like it though. Use it on low-end machines.

XScreenSaver and Really Slick Screensavers (GLX Port) – Configurable and command controllable screen saver daemon. Favourite modes: GLMatrix and Substrate from XScreenSaver and Lattice Sky Rocket and Hufo’s Smoke from RSS GLX.

xosview – my favourite system monitor since more than a decade.

Terminals

• xterm – there is no better X terminal emulator than the original xterm. I found no other terminal which is so fast, has no problems with text-mode applications (aterms break aptitude’s display), no problems with character set encodings, which can be embedded into other applications and which has a fully working classic Unix cut & paste.
• aterm – When I need a fancy transparent terminal for showing a fancy desktop, I use the AfterStep Terminal Emulator aterm. In that case, the system tray, the window borders, the window’s title bar and the terminal on my desktop have the same fancy tinted transparency.
• yeahconsole – A wrapper around xterm which works like the pulldown console in quake. Good for the short shell usage inbetween. ;-) The other similar pull down consoles I know (KDEish yakuake and GNOMEish tilda) had some issues with focus and keybindings while yeahconsole works just out of the box and showed no problems until now.

Audio and Video

• XMMS and Audacious – If I want to play a single list of files of the same file format or single stream, I usually use the command line tools mpg123 and ogg123. But if I need anything more fancy or more flexible, I prefer the WinAMP clones. Formerly XMMS, nowadays Audacious. Both with some old skin which I use since more than a decade and which I initially used with WinAMP 2 on Windows 95.
• mplayer – no fancy GUI, easily controllable with the keyboard, plays most video file formats I can remember. ;-)

Editing and Developing

• GNU Emacs – I’ve been raised with GNU Emacs and Lisp at university, so I’m quite sticked to that. I usually only start one Emacs instance and connect to it using emacsclient. I also like TRAMP for editing remote files. but I don’t need it that often. On machines, where I don’t want a full blown Emacs installation or under root I prefer GNU Emacs’ little brother GNU Zile (Zile Is a Lossy Emacs), but that’s text-mode and no GUI software.
• OpenOffice.org – I think it’s a really great software, but I use it quite seldom, usually only when I have to open some file in a Microsoft file format. For writing letters, articles, presentations and so I have LaTeX.
• Gnumeric – My preferred spreadsheet application. Although for some purposes I use the OpenOffice.org spreadsheet, usually when Gnumeric has not all necessary features.

Graphics

• xv – Yet another tool I use since more than a decade: No other image viewer is so fast and yet so easy to use with both keyboard and mouse. Open source, but unfortunately not (yet?) free software.
• keyjnote – fancy PDF presenter with a lot of interactive features.
• pdfcube – PDF presenter turning pages as a cube as compiz or Macs do with the desktop.

Chat

• Pidgin – I usually use irssi inside a screen for IRC as well as Jabber and ICQ (via Bitlbee), but I also often have a local Jabber client running which then is Pidgin (formerly known as GAIM).

Other Tools

• Unison – I use it to synchonise the cache and state of my feed reader between laptop and workstation. And I do indeed prefer the GUI version over the text-mode version. I use the text-mode only if I use it from some remote location.
• XKeyCaps – The ideal tool to wreck you keyboard layout. ;-)
• XGnokii – Used it to backup my former Nokia mobile phones, the 6130, the 6210i and the 6310i. Doesn’t work anymore with my new E51, though.
• Sunbird / Iceowl – Not really using it yet, but I plan to use it as my primary calendar tool.
• QEMU / KVM / KQEMU – My favourite desktop hardware emulator. (For servers, I prefer Xen for virtualization.)

Games

• ScummVM
• Planet Penguin Racer / Extreme Tux Racer (all forks of Tux Racer)
• Frozen Bubble
• Battle for Wesnoth
• cuyo, xpuyopuyo, flobopuyo
• Sauerbraten
• PrBoom / Freedoom
• Neverball / Neverputt
• Briquolo
• xfrisk
• Icebreaker
• XBomb

Non-Desktop Applications In case someone wonders about my mail client, Jabber client, IRC client, ICQ client, file manager, notes taking application, shell and versioning system – they’re all command line or text-mode applications:

• E-Mail: mutt
• Chat, Instant Messaging: irssi + Bitlbee + GNU Screen
• File management: coreutils (and sometimes busybox ;-)
• Notes: hnb
• Shell: zsh
• Version control: Subversion (svn), Mercurial (hg)

Who’s next? That’s difficult:

• maol would be interesting, but since a while he just blogs in Jeopardy style, so he would need pack all those programs into the subject of his blog post… No, not a good idea.
• Venty! No, has no active blog anymore.
• Dieter! No, no Linux user.

Hmmm, I think I have to look in a different corner of my circle of friends. Hmm. Ah, now I know:

• dyfa – not really a Linux user, but I guess FreeBSD is ok, too. :-)
• nion – this will be really interesting. He even uses more strange software than I do. ;-)
• alphascorpii – no idea what she prefers (except that it will be available as Debian package ;-)

And no, I don’t expect posts as comprehensive as mine. :-)

Someone recently asked me whether it was difficult to fill up my syllabus for my hacker course. I wish. The hard part is actually deciding what to put on as there is too much. This is what I have so far but it will likely change over the summer. I have read a lot of the material but what I most excited about is teaching/reading Richard Sennet’s new book on Craft, which was recently reviewed in depth and in relationship to open source (which Sennet does discuss briefly) by Siva Vaidyanathan in the Chronicle of Higher Education (an article that I co-authored also got some props in the review, which is always nice). The question of what type of activity programming is a complex and deeply interesting one. Its craft-like roots, in part, have to do with the UNIX tradition, something written about humorously by Neal Stephenson and more seriously by other folks like Peter Salus and Chris Kelty in his wonderful rich chapter on UNIX. But craft is not enough to understand coding either. The aesthetics of coding also is a literary affair and the two pieces that capture the aesthetics of code in this light are the following two, also on my syllabus: Black, Maurice
2002 At the Edge of Language: The Art of Code. (a PhD Dissertation from the Department of English at UPenn) Chopra, Samir and Scott Dexter
2007 Free Software and the Aesthetics of Code. In Decoding Liberation. I am excited to review this material, as I need to work through my own chapter on software coding, which is less about the aesthetics of code and more about the tension between collaboration and individualism in production (which obviously maps onto questions of craft and aesthetics but is not quite the same thing).

Someone recently asked me whether it was difficult to fill up my syllabus for my hacker course. I wish. The hard part is actually deciding what to put on as there is too much. This is what I have so far but it will likely change over the summer. I have read a lot of the material but what I most excited about is teaching/reading Richard Sennet’s new book on Craft, which was recently reviewed in depth and in relationship to open source (which Sennet does discuss briefly) by Siva Vaidyanathan in the Chronicle of Higher Education (an article that I co-authored also got some props in the review, which is always nice). The question of what type of activity programming is a complex and deeply interesting one. Its craft-like roots, in part, have to do with the UNIX tradition, something written about humorously by Neal Stephenson and more seriously by other folks like Peter Salus and Chris Kelty in his wonderful rich chapter on UNIX. But craft is not enough to understand coding either. The aesthetics of coding also is a literary affair and the two pieces that capture the aesthetics of code in this light are the following two, also on my syllabus: Black, Maurice
2002 At the Edge of Language: The Art of Code. (a PhD Dissertation from the Department of English at UPenn) Chopra, Samir and Scott Dexter
2007 Free Software and the Aesthetics of Code. In Decoding Liberation. I am excited to review this material, as I need to work through my own chapter on software coding, which is less about the aesthetics of code and more about the tension between collaboration and individualism in production (which obviously maps onto questions of craft and aesthetics but is not quite the same thing).

The first step is to copy /images/xen/vmlinuz and /images/xen/initrd.img from the Fedora (or RHEL or CentOS) DVD somewhere convenient, I use /boot/OS/ (where OS is the name of the image) but other locations will do. Now choose a suitable Ethernet MAC address for the interface (see my previous post on how I choose them [1]). Create a temporary block device for the install, I use /dev/VG0/OS-install (where OS is replaced by the name of the distribution, “f8” or “cent5“), it’s a logical volume in an LVM volume group named VG0. The device should be at least 2G in size for a basic Fedora install (512M for swap, 1G for files, and 512M free after the install). It is of course possible to use DOS partitions for the Xen block devices, but this would be unreasonably difficult to manage. An option for people who don’t like LVM would be to use files on an XFS filesystem (Ext3 performs poorly when creating and removing large files). When configuring Xen on Debian systems I generally use /dev/hda type device names. The device name seems quite arbitrary and /dev/hda is a familiar name for hard drives that many people have been used to for 15+ years. But the Fedora install process doesn’t like it and I’m forced to use /dev/xvda etc. I often install Fedora on a machine that only has 256M of RAM spare for the DomU. For recent versions of Fedora 256M of RAM is about the minimum for an install at the best of times, and a HTTP install takes even more because the root filesystem used for the install is copied via HTTP and stored in a RAM disk. It might be possible to use less RAM with a CD or DVD install or even a NFS install, but I couldn’t get CD/DVD installation working and I generally don’t give Xen DomU’s NFS access if I can avoid it. So I had to create a swap space (an attempt to do an install with 256M of RAM and no swap aborted when installing the kernel package). I expect that most serious use of Xen will have 256M of RAM or less for the DomU, part of the problem here is that Xen allocates RAM not virtual memory. VMWare allocates virtual memory so the total memory for virtual machines can be greater than physical RAM and thus this problem will be less common with VMWare. I believe that the best way of configuring virtual machine images is to have the virtual machine manager (Xen in this case) provide block devices to the virtual machine and have the virtual machine implement no partitioning (no LVM or anything equivalent). The main reason is that DOS partition tables and LVM configuration on a block device used by Xen can not be used easily in the host environment (the Dom0 for Xen). I am not aware of how to access DOS partition tables (although I’m sure it’s possible somehow) and while LVM can be used it’s a bad idea due to the fact that there is no way to deactivate a LVM volume group that is active, and the fact that there is no support for having multiple volume groups of the same name. The lack of support for multiple volume groups of the same name is a reasonable limitation, but an insurmountable problem when using a virtual machine environment. It’s quite reasonable to create several cloned instances of a virtual machine and renaming an LVM volume group would require more changes inside the virtual machine than you would want. Also using snap-shots of old versions of the virtual machine data is difficult if the same volume group name is used. So for ease of management I want to have filesystems on block devices (such as /dev/xvda) instead of partitions (such as /dev/xvda1). Unfortunately Anaconda (the Fedora installer) doesn’t support this. So I had to do the initial install with DOS partitions and then fix it afterwards. So use the manual option and create a primary partition for the root filesystem and then create a non-primary partition for swap (when using small amounts of RAM such as 256M) so that swap can be used during the install. The root filesystem needs to be at the start of the disk to make it easier to sort this out later. After installing Fedora and shutting the virtual machine down the next step is to copy the block device to the desired configuration (filesystem on an unpartitioned device). If the root filesystem is the first partition then the first 63 sectors will be the partition table and reserved space so dd can be used to copy the data with the following commands: dd if=/dev/VG0/OS-install of=/dev/VG0/OS bs=512 skip=63
e2fsck -f /dev/VG0/OS
resize2fs /dev/VG0/OS The next step is to mount the device /dev/VG0/OS in the Dom0 to change /etc/fstab, I use /dev/xvda for the root device and /dev/xvdb for swap. Now to remove the cruft:
Avahi is a network service discovery system, mainly used for laptops and isn’t needed on a server, it is installed by default on all recent Fedora, RHEL and CentOS releases but it is not useful in a DomU (any unused network service is a security risk if you don’t disable or remove it). Smartmontools is for detecting impending failure of a hard drive and does not do any good when using a virtual block device (you run it on the Dom0). It might be considered a bug that smartd doesn’t exit on startup when it sees a device such as /dev/xvda. The pcsc-lite package for managing smart cards is of no use to me and all the other people who don’t own readers for smart-cards, and it can therefore be removed. Bluetooth networking support (in the package bluez-utils) is also only usable in a Dom0 (AFAIK), and the only bluetooth device I own is my mobile phone so I can’t use it on my computer. The command “yum remove avahi smartmontools pcsc-lite bluez-utils” removes them. For almost all of my DomU’s I don’t use NFS or do any printing, so I remove the packages related to them. Also autofs is in most cases only useful for servers when mounting NFS filesystems. I remove them with the command “yum remove nfs-utils portmap cups autofs“. The GPM daemon (which supports cut/paste operations with a mouse on virtual consoles) is of no use on a Xen DomU, unfortunately the vim-enhanced package depends on it. I could just disable the daemon, but as I like to run small images I remove it with “yum remove gpm“. I may have to reinstall it on some images as some of my clients like the extra VIM functionality. It’s unfortunate that debootstrap doesn’t work on CentOS (and presumably Fedora) so installing a Debian DomU on a CentOS/Fedora Dom0 requires creating an image on a Debian machine or downloading an image from www.jailtime.org . Sample Xen Config for the install: kernel = “/boot/OS/vmlinuz”
ramdisk = “/boot/OS/initrd.img”
memory = 256
name = “OS”
vif = [ 'mac=00:16:3e:66:66:68, bridge=xenbr0' ]
disk = [ 'phy:/dev/VG0/OS-install,xvda,w' ]
extra=”askmethod text” Sample Xen Config for operation: kernel = “/boot/cent5/vmlinuz-2.6.18-53.el5xen”
ramdisk = “/boot/cent5/initrd-2.6.18-53.el5xen.img”
memory = 256
name = “cent5″
vif = [ 'mac=00:16:3e:66:66:68, bridge=xenbr0' ]
disk = [ 'phy:/dev/VG0/cent5,xvda,w', 'phy:/dev/VG0/cent5-swap,xvdb,w' ]
root = “/dev/xvda ro”

• [1] http://etbe.coker.com.au/2007/03/27/xen-and-eth-device-renaming/

Share This

The CIO and the Office For Technology of New York State (USA) just released a new report on electronic records strategy, titled A Strategy for Openness: Enhancing E-Records Access in New York State. I submitted my own comments during the call for public comments earlier this year -- apparently there were only 114 comments submitted by the public (you can see the call for comments and an overview/analysis of the responses). I wish there'd been more, since it looks like the 59 individual commenters were overwhelmingly supportive of a requirement for the Gov't to use ODF. The rest of the comments were from governments, NGOs, and businesses, and they don't seem as unanimous as the individual commenters. Looking at the metadata for the executive summary shows just how far we have to go:

[0 dkg@squeak doc]$ pdfinfo PartIerecordsStudy.pdf    egrep '^Title Producer Creator'
Title:          Microsoft Word - PartIerecordsStudy.doc
Creator:        PScript5.dll Version 5.2.2
Producer:       Acrobat Distiller 8.1.0 (Windows)
[0 dkg@squeak doc]$ 

And in a message sent to commenters announcing the report, they seem to be aware that they're not quite doing their best at interoperability:

(Please note this is the first time any of NYS government's agencies has ever published a document in ODF format. We ran into problems this morning not caused by the format, but rather with some controls on our systems which were not prepared to encounter such documents. So the website links to ODF versions are not working at the moment, but will be repaired soon).

What web server are they running that is having trouble "encountering" ODF? No surprise:

[0 dkg@squeak doc]$ wget -S -O/dev/null 'http://www.oft.state.ny.us/policy/esra/erecords-study.htm' 2>&1   grep Server
  Server: Microsoft-IIS/6.0
[0 dkg@squeak doc]$ 

Sigh. My initial reaction to the report (i've only read the executive summary, not the supporting documentation) is disappointment. While they claim that "openness" is an important feature, they fall far short of taking a strong stand for free and open formats. The main thrust of the executive summary seems to be (paraphrasing here, i welcome corrections):

• it's a bad idea to mandate a single particular technology because technologies will change faster than law.
• we need an Electronic Records Committee (ERC) to provide regular, executive guidance on electronic record storage and maintenance
• "openness" is good, but needs to be weighed against (ill-specified, at least in the executive summary) "other features"
• we're not going to make any concrete recommendations about what to do next

While these are unsurprising conclusions, they don't take the strong stand i'd like them to take. They seem aware of the "chicken and egg" problem of wide adoption and vendor support, and aware of the anti-democratic effects of proprietary formats. But they don't seem to be willing to act as a catalyst to push free formats explicitly. If they did, they could break the chicken/egg vendor lock-in cycle and facilitate better open government practices while simultaneously encouraging a healthy information ecosystem. At this point, i suppose my biggest hope from the process is that the proposed ERC forms pilot groups with long-term goals modeled after Munich, but i'm not holding my breath. If there are any advocates or participants in the Munich process who want to share what's been working for you (and what hasn't), i'd be interested in hearing about it.Tags: government, odf, policy

The way Xen works is that the RAM used by a virtual machine is not swappable, so the only swapping that happens is to the swap device used by the virtual machine. I wondered whether I could improve swap performance by using a tmpfs for that swap space. The idea is that as only one out of several virtual machines might be using swap space, a tmpfs storage could cache the most recently used data and result in the virtual machine which is swapping heavily taking less time to complete the memory-hungry job. I decided to test this on Debian/Etch (both DomU and Dom0). Above is a graph of the time taken in seconds (on the Y axis) to complete the command “apt-get -y install psmisc openssh-client openssh-server linux-modules-2.6.18-5-xen-686 file less binutils strace ltrace bzip2 make m4 gcc g++“, while the X axis has the amount of RAM assigned to the DomU in megabytes. The four graphs are for using a real disk (in this case an LVM logical volume) and for using tmpfs with different amounts of RAM backing it. The numbers 128000, 196000, and 256000 and the numbers of kilobytes of RAM assigned to the Dom0 (which manages the tmpfs). As you can see it’s only below about 20M of RAM that tmpfs provides a benefit. I don’t know why it didn’t provide a benefit with larger amounts of RAM, below 48M the amount of time taken started increasing exponentially and I expected that there was the potential for a significant performance boost. After finding that the benefits for a single active DomU were not that great I did some tests with three DomU’s running the same APT command. With 16M of RAM and swap on the hard drive it took an average of 408 seconds, but with swap on the tmpfs it took an average of 373 seconds - an improvement of 8.5%. With 32M of RAM the times were 225 and 221 seconds - a 1.8% improvement. Incidentally to make the DomU boot successfully with less than 30M of RAM I had to use “MODULES=dep” in /etc/initramfs-tools/initramfs.conf. To get it to boot with less than 14M I had to manually hack the initramfs to remove LVM support (I create my initramfs in the Dom0 so it gets drivers that aren’t needed in the DomU). I was unable to get a DomU with 12M of RAM to boot with any reasonable amount of effort (I expect that compiling the kernel without an initramfs would have worked but couldn’t be bothered). Future tests will have to be on another machine as the machine used for these tests caught on fire - this is really annoying, if someone suggests an extra test I can’t run it. To plot the data I put the following in a file named “command” and then ran “gnuplot command“:
unset autoscale x
set autoscale xmax
unset autoscale y
set autoscale ymax
set xlabel “MB”
set ylabel “seconds”
plot “128000″
replot “196000″
replot “256000″
set terminal png
set output “xen-cache.png”
replot “disk” In future when doing such tests I will use “time -p ” (for POSIX format) which means that it displays a count of seconds rather than minutes and seconds (and saves me from running sed and bc to fix things up). I am idly considering writing a program to exercise virtual memory for the purpose of benchmarking swap on virtual machines. My raw data is below:

time apt-get -y install psmisc openssh-client openssh-server linux-modules-2.6.18-5-xen-686 file less binutils strace ltrace bzip2 make m4 gcc g++ 96M: 0m37.416s
64M: 0m47.607s
48M: 0m55.050s
32M: 1m8.200s
30M: 1m15.010s
28M: OOM process 2 with default initrd, needed MODULES=dep to make it boot
28M: 1m18.694s
24M: 1m25.517s
20M: 1m38.376s
16M: 2m15.594s
14M: 3m14.777s
13M: OOM when the initrd is not hacked to remove LVM stuff and drivers for Dom0 stuff
13M: 4m15.930s
12M: really won’t boot 128000K of RAM in Dom0 for caching (18M cache and 70M free), 256M swap per DomU on tmpfs. 96M: 0m40.121s
64M: 0m48.207s
32M: 1m12.394s
24M: 1m21.929s
16M: 1m59.042s
14M: 2m48.517s
13M: 3m29.391s 196000K of RAM in Dom0 for caching: 64M: 0m47.871s
32M: 1m12.078s
24M: 1m22.360s
16M: 2m0.336s
14M: 2m36.459s 256000K of RAM in Dom0 for caching: 64M: 0m47.949s
32M: 1m11.435s
24M: 1m21.402s
16M: 1m58.884s
14M: 2m39.081s 256000: of RAM in Dom0, three DomU’s: 32M: 3m41.583s 3m40.737s 3m41.071s
16M: 6m16.430s 6m14.832s 6m7.569s hard drive, three DomU’s: 32M: 3m42.100s 3m47.516s 3m46.333s
16M: 6m41.587s 6m50.837s 6m51.795s

Share This

When recording changes in some software there are basically three approaches, with their different advantages and disadvantages.

• One approach is just record modifications chronologically. Each modification is just a little diff (or some glorified variant of diff, or if you insist something that is no diff but just has some similarity). Advantages are:
  • It keeps itself out of the way. You just do modifications and store them somehow one after the other.
  • You can reconstruct all past points, so can regain missing documentation by looking at the evolution in time.
  So it is no surprise that essential every tool (i.e. all the VCSes) to handle non-packaging work on software essentially uses this way. If you have an canonical other source for the software (like an upstream for an package), it has quite some severe drawbacks:
  • Reviewablity does not scale. While every single diff can easily be reviewed, the mass is not. Most changes are relative to past versions of the software. If something introduces a problem or and upstream change introduces a problem with that, the fix might only show up years later, separated by tons of other changes in between. (The effect especially with changing upstream always depends a bit on the later merging, but the worst case is still the same).
  • For the same reason changes cannot be easily extracted. Single commits usually are fine, but what of those past changes is needed to make some specific change in the functionality is easily lost after some time, some improvements and updates to new upstream versions. The whole approach degenerates to having a single large .diff.gz file of the package. (Except one has a bit of history recorded, so when sorting out the stuff in that diff one can better guess what effect a single unattributable line has).
• The next possibility is to store modifications sorted by topic and against upstream (either as .diff files relative to the upstream source, as topic branches, or or or). Advantages:
  • As you have to send the patch upstream anyway, you have to create such an patch initially, anyway. Also each is in a format that if upstreams wants to apply a single one, each is in the correct format for this.
  • Branches relative to a (relatively) fix point are usually use to store in VCSes.
  • When other people want to pick a subset of those patches, they only have to do the merging, and not dismerging with the missing ones first.
  On the other hand, there are some disadvantages with this:
  • As the differences are relative to the upstream code, multiple patches are in general not applicable without additional merging.
  • Because of this, those patches cannot be used to generate the source to build the Debian package.
  • Thus those patches by itself are never directly used, unless the maintainer regularly tests them, they are essentially untested and will see bitrot.
  • As those are not the changes to the package because of the missing merge, they are nice for a cursory review, but if you want the review the actual changes, you need to review the merged variants.
• Last but not least, there are stacked patches, i.e. patches that are supposed to be applied in a specific sequence. Advantages:
  • The patches itself are actually used. Thus they get wide testing and bitrot cannot happen that easily.
  • Equally, reviewing the actual changes can be separated into the different patches, as connected changes are next to each other and the files describe the changes without pitfalls in merges.
  • If someone else wants to use multiple patches from the start of the sequence, no merging is necessary. They can directly be applied.
  Given these advantages, it is no big surprise that all Debian specific patch management tools (like dbs) and other programs to deal deal with multiple patches against other people's code (like quilt) use this approach. Though there are also some disadvantages:
  • Ordering patches is not important. Only unrelated patches can be skipped to have later patches applied cleanly, so ordering needs to predict what patches are more likely to be used by other people.
  • Testing only happens for the whole stack. If some match modifies parts that a later patch removes, bitrot can still happen.
  • Extracting those patches from the currently available VCSes is not trivial.

So the format most suitable to Debian packages (stacked patches) is the total opposite of a format most suitable when you are upstream yourself and nothing is suitable for everything. There are many different thinkable ways to combine the different things to get more of the advantages, though many are a bit lacking (like storing quilt series in a VCS as text files), not yet possible or non-trivial with the current tools. Hopefully the future will improve that.

I just wrote about the system administration issues related to the recent Debian SSL/SSH security flaw [1]. The next thing we need to consider is how we can change things to reduce the incidence of such problems. The problem we just had was due to the most important part of the entropy supply for the random number generator not being used due to a mistake in commenting out some code. The only entropy that was used was the PID number of the process which uses the SSL library code which gives us 15 bits of entropy. It seems to me that if we had zero bits of entropy the problem would have been discovered a lot sooner (almost certainly before the code was released in a stable version). Therefore it seems that using a second-rate source of entropy (which was never required) masked the problem that the primary source of entropy was not working. Would it make sense to have a practice of not using such second-rate sources of entropy to reduce the risk of such problems being undetected for any length of time? Is this a general issue or just a corner case? Joss makes some suggestions for process improvements [2]. He suggests that having a single .diff.gz file (the traditional method for maintaining Debian packages) that directly contains all patches can obscure some patches. The other extreme is when you have a patch management system with several dozen small patches and the reader has to try and discover what each of them does. For an example of this see the 43 patches which are included in the Debian PAM package for Etch, also note that the PAM system is comprised of many separate shared objects (modules), this means that the patching system lends itself to having one patch per module and thus 43 patches for PAM isn’t as difficult to manage as 43 patches for a complex package which is not comprised of multiple separate components might be. That said I think that there is some potential for separating out patches. Having a distinction between different types of patches might help. For example we could have a patch for Makefiles etc (including autoconf etc), a patch for adding features, and a patch for fixing bugs. Then people reviewing the source for potential bugs could pay a lot of attention to bug fixes, a moderate amount of attention to new features, and casually skim the Makefile stuff. The problem began with this mailing list discussion [3]. Kurt’s first message starts with “When debbuging applications” and ends with “What do you people think about removing those 2 lines of code?“. The reply he received from Ulf (a member of the OpenSSL development team) is “If it helps with debugging, I’m in favor of removing them“. It seems to me that there might have been a miscommunication there, Ulf may have believed that the discussion only concerned a debugging built and not a build that would eventually end up on millions of machines. It seems possible that the reaction would have been different if Kurt had mentioned that he wanted to have a single source tree for both debugging and for regular use. It also seems likely that his proposed change may have received more inspection if he had clearly stated that he was doing to include it in Debian where it would be used by millions of people. When I am doing Debian development I generally don’t mention all the time “this code will be used by millions of people so it’s important that we get it right“, although I do sometimes make such statements if I feel that my questions are not getting the amount of consideration from upstream that a binary package destined for use by millions of people deserves. Maybe it would be a good practice to clarify such things in the case of important packages. For a package that is relevant to the security of the entire distribution (and possibly to other machines around the net - as demonstrated in this case) it doesn’t seem unreasonable to include a post-script mentioning the scope of the code use (it could be done with an alternate SIG if using a MUA that allows selecting from multiple SIGs in a convenient manner). In the response from the OpenSSL upstream [4] it is claimed that the mailing list used was not the correct one. Branden points out that the openssl-team mailing list address seems not to be documented anywhere [5]. One thing to be learned from this is that distribution developers need to be proactive in making contact with upstream developers. You might think that building packages for a major distribution and asking questions about it on the mailing list would result in someone from the team noticing and mentioning any other things that you might need to do. But maybe it would make sense to send private mail to one of the core developers, introduce yourself, and ask for advice on the best way to manage communication to avoid this type of confusion. I think that it is ideal for distribution developers to have the phone numbers of some of the upstream developers. If the upstream work is sponsored by the employer of one of the upstream developers then it seems reasonable to ask for their office phone number. Sometimes it’s easier to sort things out by phone than by email. Gunnar Wolf describes how the way this bug was discovered and handled shows that the Debian processes work [6]. A similar bug in proprietary software would probably not be discovered nearly as quickly and would almost certainly not be fixed in such a responsible manner. Update: According to the OpenSSL project about page [7], Ulf is actually not a “core” member, just a team member. I had used the term “core” in a slang manner based on the fact that Ulf has an official @openssl.org email address.

• [1] http://etbe.coker.com.au/2008/05/18/debian-ssh-problems/


• [2] http://np237.livejournal.com/17981.html


• [3] http://marc.info/?t=114651088900003&r=1&w=2


• [4] http://www.links.org/?p=328


• [5] http://advogato.org/person/branden/diary/5.html


• [6] http://gwolf.org/node/1743


• [7] http://www.openssl.org/about/

Share This

I must admit that the "no complicated configuration file" was an advantage for most users of awesome, but well, it was not possible to continue on this path. As I explain in a mail on the awesome mailing-list, integrating a high level language for handling configuration was becoming necessary. So I did it, I wrote a 10K lines patch (about +3K/-6K SLOC) that just drop all the libconfuse configuration handling and now export a Lua API. It took me about two weeks to complete this task, so it was not really hard. And I am so fucking happy that the code base is now even lighter (3K SLOC less for doing more!). The Lua API is sometimes a PITA to understand, because there seems to be a lot of functions more or less connected or doing the same thing, and the separation between the core and the aux library is not always clear to beginners. But in the end, it's a language really easy to integrate, and very easy to learn. Lovely. Now it's possible to do almost everything possible in awesome, since you can now have conditional configuration and hooks to do stuff on events (i.e. change window border size on focus, toggle title bars when switching to floating layout, ). The configuration file format and syntax is I guess still clear. You can also execute Lua code on the fly and manipulate objects, like I did in this small video demonstration recorded a few days ago.

An interesting thread on openssh-unix-dev points out a way that a compromised remote ssh account or host could potentially trick users into divulging the passphrase for their local secret key. Given that we're all dealing with the potential for compromised ssh hosts lately, i think this is an important consideration. (this has apparently been raised on openssh-unix-dev as far back as 8 years ago). Unfortunately, a couple messages with mangled headers seem to have broken the thread into two chunks in the archive, severing the start of the thread from the part where it starts getting really good: In particular, Damien Miller's suggestion is to forbid in-terminal key-based authentication entirely, and rely instead on ssh-agent. My impression of this strategy is that it's analogous to forcing an out-of-band verification process -- since the in-channel communication is happening in the tty, and the remote host will have some level of control over the tty once authentication succeeds, it's important that any locally-divulged secrets (e.g. the passphrase for the local secret key) are not transmitted over the tty in question. Of course, disabling in-terminal key-based authentication creates something of a usability problem for users who aren't using X11 (and who don't have some other method for an out-of-band ssh-askpass). And it also points out the additional problems with X11 forwarding (e.g. a compromised host allowed to forward X connections could trigger a mimicry of a standard ssh-askpass password prompt). This is something that debian might want to consider, as we've diverged from upstream on the default settings for ForwardX11Trusted. I'm uncomfortable with how Damien's suggestion raises the bar to key-based authentication in general, since users will now have to understand both keys and agents in order to effectively authenticate this way. But maybe that's what's needed, since we desperately need to phase out password-based authentication (and all the security pitfalls associated with it) in this brave new networked world. At any rate, i've adopted the suggestion for now:

[0 dkg@squeak ~]$ tail -n1 .ssh/config
IdentityFile none
[0 dkg@squeak ~]$ 

I'm curious to know if other people have adopted this strategy, or have other mitigating techniques.Tags: security, ssh

While I'm mentioning changes, I stopped using Evil Spying Charter Communicatons yesterday and switched over to BTES fiber. I'm using my thecus as an internet gateway since it has two ethernet ports. This is the first time since college that my internet connection has been a simple ethernet jack with no "modem". Quite nice. :-) Trying to resist the urge to put a packet analizer on the house's other ethernet uplink jack, which is used for the new power meter they installed.