Search Results: "fh"

7 March 2009

John Goerzen: My Week

It s been quite the week. Stomach Flu Last Friday, my stomach was just starting to feel a little odd. I didn t think much off it a little food that didn t go over well or stress, I thought. Saturday I got out of bed and almost immediately felt like throwing up. Ugh. I probably caught some sort of stomach flu. I was nauseous all day and had some terrible diarrhea to boot. I spent parts of Saturday, Saturday night, Sunday, and Sunday night supervising some emergency downloads as the BOFH would say. By Sunday afternoon, I thought I was doing good enough to attend a practice of the Kansas Mennonite Men s Choir. I made it through but it wasn t quite as up to it as I thought. Monday morning I woke up and thought the worst was behind me, so I went to work. By evening, the worst clearly was not behind me. I was extremely cold, and then got very hot a few hours later. Tuesday I left work a little early because of not feeling well. Servers Wednesday a colleague called me at home before I left to say that the ERP database had a major hiccup. That s never good. The database is this creaky old dinosaur thing that has a habit of inventing novel ways to fail (favorite pastime: exceeding some arbitrary limit to the size of files that no OS has cared about for 5 years, then hanging without telling anybody why). My coworkers had been working on it since 5. I went into the office and did what I could to help out, though they had mostly taken care of it. Then we went to reboot the server. It didn t come back. I/O error on sda just after init started, and it hung. Puzzled, as it just used that disk to boot from. Try rebooting again. This time, I/O error as the fibre channel controller driver loads. Again, puzzled as it just used that controller to load grub. Power cycle this time. And now the server doesn t see the fibre channel link at all. Eep. Check our fiber optic cables, and power cycle again. And THIS time, the server doesn t power back up. Fans whir for about a second, then an ominous red light I never knew was there shows up. Eeep! So I call HP. They want me to remove one CPU. Yes, remove one CPU. I tried, and long story short, they dispatch a local guy with a replacement motherboard. Can you send along a FC controller, in case it s dead too? Nope, not until we diagnose a problem with it. Local guy comes out. He s a sharp guy and I really like him. But the motherboard wasn t in stock at the local HP warehouse, so he had to have it driven in from Oklahoma City. He gets here with it by about 4:30. At this point the single most important server to the company s business has been down almost 12 hours. He replaces the motherboard. The server now powers up yay! And it POSTs, and it . doesn t see the disks. !#$!#$ He orders the FC controller, which is so very much not in stock that they can t get it to us until 8:30AM the next morning (keep in mind this thing is on a 4-hour 24/7 contract). Next morning rolls around. Outage now more than 24 hours. He pops the FC controller in, we tweak the SAN settings appropriately, we power up the machine, and . still doesn t see any disks, and the SAN switch still doesn t see any link. EEP! Even the BIOS firmware tool built into the controller doesn t see a link, so we KNOW it s not a software issue. We try plugging and unplugging cables, trying different ports, everything. Nothing makes a difference. At this point, while he ponders what else he can replace while we start migrating the server to a different blade. We get ERP back up on its temporary home an hour later, and he basically orders us every part he can think of while we ve bought him some room. Several additional trips later, he s replaced just about everything at least once, some things 2 or 3 times, and still no FC link. Meanwhile, I ve asked my colleague to submit a new ticket to HP s SAN team so we can try checking of the switch has an issue. They take their sweet time answering until he informs them this morning that it s been *48 HOURS* since we first reported the outage. All of a sudden half a dozen people at HP take a keen interest in our case. As if they could smell this blog post coming So they advise us to upgrade the firmware in the SAN switch, but they also say we really should send this to the blade group; the problem can t be with the SAN and of course the blade people are saying the problem s GOT to be with the SAN . We try to plan the firmware upgrade. In theory, we can lose a switch and nobody ever notices due to multipathing redundancy. In practice, we haven t tested that in 2 years. None of this equipment had even been rebooted in 390 days. While investigating this, we discovered that one of the blade servers could only see one path to its disks, not two. Strange. Fortunately, THAT blade wasn t mission-critical on a Friday, so I power cycled it. And it powered back up. And it promptly lost connection to its disks entirely, causing the SAN switches to display the same mysterious error they did with the first blade the one that nobody at HP had heard of, could find in their documentation, or even on Google. Yes, that s right. Apparently power cycling a server means it loses access to its disks. Faced with the prospect of our network coming to a halt if anything else rebooted (or worse, if the problem started happening without a reboot), we decided we d power cycle one switch now and see what would happen. If it worked out, our problems would be fixed. If not, at least things would go down in our and HP s presence. And that worked? What? Yes. Power cycling the switch fixed every problem over the course of about 2 minutes, without us having to do anything. Meanwhile, HP calls back to say, Uhm, that firmware upgrade we told you to do? DON T DO IT! We power cycle the other switch, and have a normal SAN life again. I let out a WOOHOO! My colleague, however, had the opposite reaction. Now we ll never be able to reproduce this problem to get it fixed! Fair point, I suppose. Then began the fairly quick job of migrating ERP back to its rightful home it s all on Xen already, designed to be nimble for just these circumstances. Full speed restored 4:55PM today. So, to cap it all off, within the space of four hours, we had fail: Murphy, I hate you. The one fun moment out of this was this conversation: Me to HP guy: So yeah, that machine you ve got open wasn t rebooted in 392 days until today. HP guy: WOW! That s INCRED oh wait, are you running Linux on it? Me: Yep. HP: Figures. No WAY you d get that kind of uptime from Windows. And here he was going to be all impressed.

3 March 2009

Marco d'Itri: The files in /etc/modprobe.d/

The module-init-tools upstream maintainers decided that in the future the files in /etc/modprobe.d/ will be processed only if they have a .conf suffix. The latest module-init-tool release complains loudly for each one and still processes them, but this will change.

Please update your packages and remember that the files in this directory are opened and parsed every time modprobe is run (and it is run very often at boot time!), so try to look at the big picture and install a file there only if it is really needed. Do not install a file if it only contains comments, if it is only useful for some unusual environment or if it not needed by the kernel currently in the archive (nowadays most drivers have their own aliases built in, check /lib/modules/$KVER/modules.alias and submit a kernel patch if they are missing).

22 February 2009

Theodore Ts'o: Should Filesystems Be Optimized for SSD s?

In one of the comments to my last blog entry, an anonymous commenter writes:
You seem to be taking a different perspective to linus on the adapting to the the disk technology front (Linus seems to against having to have the OS know about disk boundaries and having to do levelling itself)
That s an interesting question, and I figure it s worth its own top-level entry, as opposed to a reply in the comment stream. One of the interesting design questions in any OS or Computer Architecture is where the abstraction boundaries should be drawn and which side of an abstraction boundary should various operations be pushed. Linus s arguments is that there a flash controller can do a better job of wear leveling, including detecting how worn a particular flash cell might be (for example, perhaps by looking at the charge levels at an analog level and knowing when the last time the cell was programmed), and so it doesn t make sense to try to do wear leveling in a flash file system. Some responsibilities of flash management, such as coalescing newly written blocks into erase blocks to avoid write amplification can be done either on the SSD or in the file system for example, by using a log-structured file system, or some other copy-on-write file system, instead of a rewrite-in-place style file system, you can essentially solve the write amplification problem. In some cases, it s necessary let additional information leak across the abstraction for example, the ATA TRIM command is a way for the file system to let the disk know that certain blocks no longer need to be used. If too much information needs to be pushed across the abstraction, one way or another, then maybe we need to rethink whether the abstraction barrier is in the right place. In addition, if the abstraction has been around for a long time, changing it also has costs, which has to be taken into account. The 512 byte sector LBA abstraction has been around long time, and therefore dislodging it is difficult and costly. For example, the same argument which says that because the underlying hardware details are changing between different generations of SSD is all of these details should be hidden in hardware, was also used to justify something that has been a complete commercial failure for years if not decades: Object Based Disks. One of the arguments of OBD s was that the hard drive has the best knowledge of how and where to store an contiguous stream of bytes, and so perhaps filesystems should not be trying to decide where on disk an inode should be stored, but instead tell the hard drive, I have this object, which is 134 kilobytes long; please store it somewhere on the disk . At least in theory the HDD or SSD could handle all of the details of knowing the best place to store the object on the spinning magnetic media or flash media, taking into account how worn the flash is and automatically move the object around in the case of an SSD, and in the case of the HDD, the drive could know about (real) cylinder and track boundaries, and store the object in the most efficient way possible, since the drive has intimate knowledge about the low-level details of how data is stored on the disk. This theory makes a huge amount of sense; but there s only one problem. Object Based Disks have been proposed in academia and advanced R&D shops of companies like Seagate et.al. have been proposing them for over a decade, with absolutely nothing to show for it. Why? There have been two reasons proposed. One is that OBD vendors were too greedy, and tried to charge too much money for OBD s. Another explanation is that the interface abstraction for OBD s was too different, and so there wasn t enough software or file systems or OS s that could take advantage of OBD s. Both explanations undoubtedly contributed to the commercial failure of OBD s, but the question is which is the bigger reason. And the reason why it is particularly important here is because at least as far as Intel s SSD strategy is concerned, its advantage is that (modulo implementation shortcomings such as the reported internal LBA remapping table fragmentation problem and the lack of ATA TRIM support) filesystems don t need to change (much) in order to take advantage of the Intel SSD and get at least decent performance. However, if the price delta is a stronger reason for its failure, then the X25-M may be in trouble. Currently the 80GB Intel X25-M has a street price of $400, so it costs roughly $5 per gigabyte. Dumb MLC SATA SSD s are available for roughly half the cost/gigabyte (64 GB for $164). So what does the market look like 12-18 months from now? If dumb SSD s are still available at 50% of the cost of smart SSD s, it would probably be worth it to make a copy-on-write style filesystem that attempts to do the wear leveling and write amplification reduction in software. Sure, it s probably more efficient to do it in hardware, but a 2x price differential might cause people will settle for a cheaper solution even if isn t the absolutely best technical choice. On the hand, if prices drop significantly, and/or dumb SSD s completely disappear from the market, then time spent now optimizing for dumb SSD s will be completely wasted. So for Linus to make the proclamation that it s completely stupid to optimize for dumb SSD s seems to be a bit premature. Market externalities for example, does Intel have patents that will prevent competing smart SSD s from entering the market and thus forcing price drops? could radically change the picture. It s not just a pure technological choice, which is what makes projections and prognostications difficult. As another example, I don t know whether or not Intel will issue a firmware update that adds ATA TRIM support to the X25-M, or how long it will take before such SSD s become available. Until ATA TRIM support becomes available, it will be advantageous to add support in ext4 for a block allocator option that aggressively reuses blocks above all else, and avoids using blocks that have never been allocated or used before, even if it causes more in-file system fragmentation and deeper extent allocation trees. The reason for this is at the moment, once a block is used by the file system, at least today, the X25-M has absolutely no idea whether we still care about the contents of that block, or whether the block has since been released when the file was deleted. However, if 20% of the SSD s blocks have never been used, the X25-M can use 20% of the flash for better garbage collection and defragmentation algorithms. And if Intel never releases a firmware update to add ATA TRIM support, then I will be out $400 out of my own pocket for an SSD that lacks this capability, and so adding a block allocator which works around limitations of the X25-M probably makes sense. If it turns out that it takes two years before disks that have ATA TRIM support show up, then it will definitely make sense to add such an optimization. (Hard drive vendors have been historically S-L-O-W to finish standardizing new features and then letting such features enter the market place, so I m not necessarily holding my breath; after all, the Linux block device layer and and file systems have been ready to send ATA TRIM support for about six months; what s taking the ATA committees and SSD vendors so long? <grin> On the other hand, if Intel releases ATA TRIM support next month, then it might not be worth my effort to add such a mount option to ext4. Or maybe Sandisk will make an ATA TRIM capable SSD available soon, and which is otherwise competitive with Intel, and I get a free sample, but it turns out another optimization on Sandisk SSD s will give me an extra 10% performance gain under some workloads. Is it worth it in that case? Hard to tell, unless I know whether such a tweak addresses an optimization problem which is fundamental, and whether or not such a tweak will either be unnecessary, or perhaps actively unhelpful in the next generation. As long as SSD manufacturers force us treat these devices as black boxes, there may be a certain amount of cargo cult science which may be forced upon us file system designers or I guess I should say, in order to be more academically respectable, we will be forced to rely more on empirical measurements leading to educated engineering estimations about what the SSD is doing inside the black box . Heh. Related posts (automatically generated):
  1. Aligning filesystems to an SSD s erase block size I recently purchased a new toy, an Intel X25-M SSD,...

21 February 2009

Ren&#233; Mayorga: It is already too late to say this, but welcome to Lenny \o/

I start to move some servers when lenny was freeze, and did not have any complain, the update was really smooth; but I still let some servers to be migrated just at the end of the release cycle, and sure, they fail I actually have some fun migrating freeradius and cacti, and note that I actually will have more fun migrating asterisk and openser servers, but for the sake of this release, is the fun that every BOFH should expect, and also, cheers and thanks to everyone who work on this release :)

15 February 2009

Michael Prokop: Debian GNU/Linux 5.0 codename Lenny - News for sysadmins

Alright, Debian GNU/Linux 5.0 AKA as Lenny has been released. Time for a Debian unstable unfreeze party! 8-) What does the new stable release bring for system administrators? I ll give an overview what news you might expect when upgrading from Debian GNU/Linux 4.0, codename Etch (released on 8th April 2007) to the current version Debian GNU/Linux 5.0, codename Lenny (released on 14th February 2009). I try to avoid duplicated information so make sure to read the release announcement and the official release notes for Lenny beforehand. Noteworthy Changes Virtualisation Virtualisation related new tools: Desktop oriented packages like virtualbox and qemu are available as well of course. Noteworthy Updates This is a (selective) list of some noteworthy updates: New packages Lenny ships over 7000 new packages. Lists of new/removed/replaced packages are available online. I ll name 238 sysadmin related packages that might be worth a look. (Note: I don t list addon stuff like optional server-modules, docs-only and kernel-source related packages. I plan to present some of the following packages in more detail in separate blog entries.) Further Ressources

9 February 2009

Biella Coleman: Electric Sun

Welcome, Congress, to our generation s electric sun. Earlier, I had posted the Wikileaks link to these congressional reports with comments of my own but I thought I would pen down a few thoughts as I finally electronically leafed through some of them. These reports remind me a little bit of another set of reports that are publicly available, which are the Congressional Quarterly reports, which are an excellent resource for research. They are a bit dry but provide a wealth of information and perhaps more important, citations to law cases, journalist articles, and academic pieces (everything that journalistic pieces, in other words, do not do). It does 1/2 the research for you, as I like to think. The few reports I have scanned from the leaks remind me, in fact, of the CQ reports, in content, style and tone. And while I thought CQ reports published on a wide range of topics, these semi-private reports are far more extensive in terms of topics and far more specific as well:
CRS: Humane Treatment of Farm Animals: Overview and Issues, December 10, 2008
CRS: FDA s Authority to Ensure That Drugs Prescribed to Children Are Safe and Effective, December 2, 2008
CRS: African American Members of the United States Congress: 1870-2008, December 4, 2008
CRS: The Pigford Case: USDA Settlement of a Discrimination Suit by Black Farmers, January 13, 2009
CRS: Selected Legal and Policy Issues Related to Coalbed Methane Development, March 9, 2004
I look forward to hearing/learning more about how they are used (I can t imagine that each report is read by many) and why exactly they were hidden away as they don t seem to be the type of information that should be kept classified.

31 January 2009

Dirk Eddelbuettel: State-of-the-art in parallel computing with R: New paper

A few weeks ago, we finished a paper that surveys the current state of parallel computing with R. The paper was lead by Markus Schmidberger and written while he was visiting the Fred Hutchinson Cancer Research Center in Seattle. The co-authors are Martin Morgan, myself, Hao Yu, Luke Tierney and Ulrich Mansmann. The paper is now available as a technical report from LMU Munich via open access, and also from my papers page.

26 January 2009

Marco d'Itri: A Linux-based alarm clock

That is, my laptop. Since at night I keep it in the same room where I sleep, I figured that I could as well use it as an highly customizable alarm clock. My alarm script uses rtcwake(8) from util-linux-ng to set the RTC alarm clock, suspends the computer and then plays some music at a set volume when it wakes up (if it never wakes up, check your BIOS settings). Since it uses the date(2) command it can parse the alarm time in many formats. 
#!/bin/sh -e
# for a random tune, list multiple space-separated files
MUSIC='/home/mp3/The_Velvet_Underground/The_Velvet_Underground_&_Nico/07_Heroin.mp3'
# some days I like different music
DOW="$(date +%u)"
if [ $DOW -eq 6 -o $DOW -eq 7 ]; then
  MUSIC='/home/mp3/The_Velvet_Underground/The_Velvet_Underground_&_Nico/01_Sunday_Morning.mp3'
fi
# choose your favourite music player
player()   mpg123 --random --quiet --control --title "$@"   true;  
#player()   mplayer -loop 0 "$*";  
# play the music at this volume
VOLUME='50%'
# ALSA mixer control used to set and restore the master volume
VOLUMECTL='iface=MIXER,name="Master Playback Volume"'
##############################################################################
# this is more elegant, but it needs rtcwake from util-linux-ng >= 2.14.2
rtcwake_set_alarm()  
  local when="$1"
  local rtc="$2"
  if [ "$rtc" ]; then rtc="--device $rtc"; fi
  local epochtime=$(date --date "$when" +%s)
  [ "$epochtime" ]   return 1
  sudo rtcwake $rtc --mode no --time $epochtime
 
# you can use this function instead if your system lacks a working rtcwake
set_alarm()  
  local when="$1"
  local rtc="$2"
  [ "$rtc" ]   rtc='rtc0'
  local epochtime=$(date --date "$when" +%s)
  [ "$epochtime" ]   return 1
  local alarmfile="/sys/class/rtc/$rtc/wakealarm"
  sudo sh -c "echo 0 > $alarmfile && echo $epochtime > $alarmfile"
 
##############################################################################
if [ "$1" ]; then
  WHEN="$1"
else
  echo "Usage: $0 WHEN"
  exit 1
fi
set_alarm "$WHEN"
# save the volume
oldvolume="$(amixer cget "$VOLUMECTL"   sed -nre '/ : /s/.*=//p')"
amixer -q cset "$VOLUMECTL" $VOLUME
# actually here I run a script which also deals with network interfaces,
# IM and IRC clients and so on
sudo pm-suspend
player $MUSIC
# restore the volume
amixer -q cset "$VOLUMECTL" "$oldvolume"
exit 0

19 January 2009

Stefano Zacchiroli: worst than bofh

One of the least useful error messages I've ever seen 
# dar -N -x backup -K <censor> -g some/where -v
Extracting contents of the archive...
FATAL error, aborting operation
elastic buffer incoherent structure
#
Thanks dar, now it's clear.

Marco d'Itri: automatic backups on a removable device

These scripts allow automatically running rsnapshot when the right USB hard disk is plugged in. If you do not want so much automation then just let gnome-volume-manager do its work and then run the last script from the command line. /etc/udev/rules.d/68-automount.rules: 
ACTION=="add", SUBSYSTEM=="block", \
        ENV ID_FS_USAGE =="filesystem", ENV ID_FS_LABEL =="red", \
        RUN+="backup.agent $env ID_FS_LABEL "
/lib/udev/backup.agent: 
#!/bin/sh -e
##############################################################################
wait_for_fs()  
  local fs_path="$1"
  local count="$2"
  if [ -z "$count" ]; then count=60; fi
  while ! mountpoint -q $fs_path; do
    sleep 1
    count=$(($count - 1))
    if [ $count -eq 0 ]; then
      echo "$fs_path has not been mounted after $count seconds!" 1>&2
      return 1
    fi
  done
  return 0
 
##############################################################################
do_stuff()  
  if [ -e /tmp/nobackup ]; then exit 0; fi
  wait_for_fs $FSPATH   exit 1
  nice -n 19 ionice -c 3 sh $FSPATH/doit
  rsnapshot -v diff daily.0 daily.1   mail -s "rsnapshot diff" md
  if [ -e /tmp/noumount ]; then exit 0; fi
  pumount $FSPATH
 
##############################################################################
if [ ! -t 0 ]; then
  exec >> /var/log/backup.log 2>&1
fi
if [ "$1" ]; then
  FSNAME="$1"
else
  echo "Usage: $0 name"
  exit 1
fi
FSPATH="/media/$FSNAME"
# the script may be called multiple times due to multiple change events
# being fired by the kernel
if mountpoint -q $FSPATH; then exit 0; fi
# daemonize
do_stuff &
A while loop which waits for the file system to appear is not elegant, but you cannot do this the right way with only a shell script because it requires watching /proc/mounts using inotify(3). /media/red/doit: 
#!/bin/sh -e
##############################################################################
file_older_than_days()  
  local filename="$1"
  local days="$2"
  # consider a day passed if it will end in at most 6 hours
  local more=$((60 * 60 * 6))
  local now=$(date +%s)
  local filetime=$(stat --format=%Y "$filename" 2> /dev/null   echo 0)
  local age=$(($now - $filetime))
  if [ "$age" -gt $((60 * 60 * 24 * $days - $more)) ]; then
    return 0
  else
    return 1
  fi
 
##############################################################################
RSNAPSHOT_ROOT=$(awk '/^snapshot_root[\t ]/   print $2  ' /etc/rsnapshot.conf)
if [ -z "$RSNAPSHOT_ROOT" ]; then
  echo "Cannot determine snapshot_root!"
  exit 1
elif [ ! -d "$RSNAPSHOT_ROOT" ]; then
  echo "snapshot_root is not a directory: $RSNAPSHOT_ROOT"
  exit 1
fi
if file_older_than_days "$RSNAPSHOT_ROOT/daily.0/" 1; then
  if file_older_than_days "$RSNAPSHOT_ROOT/weekly.0/" 14; then
    if file_older_than_days "$RSNAPSHOT_ROOT/monthly.0/" 60; then
      /usr/bin/rsnapshot monthly
    fi
    /usr/bin/rsnapshot weekly
  fi
  /usr/bin/rsnapshot daily
fi

16 January 2009

Christoph Berg: RFH: mutt

From: Christoph Berg <myon@debian.org>
To: Debian Bugs Submit <submit@bugs.debian.org>
Subject: Bug#512072: RFH: mutt -- text-based mailreader supporting MIME, GPG, PGP and threading
Package: wnpp
Hi, the Debian Mutt package needs more maintainers. There are almost 200 open bugs. Some of these are already forwarded upstream and might just need some triaging/poking. Some need forwarding. Some might be fixed with a trivial patch. Others are Debian specific, mostly for the extra patches we include. There's duplicates and sub-wishlist items. We are using bts-link to link to dev.mutt.org's trac, but that also needs more tweaking. There's a new upstream version 1.5.19 pending, we need people to check which bugs still apply. The (in?)famous mutt-patched package needs love to get the sidebar patch updated to the new version, and to get some of mutt-ng's extra gimmicks and add-ons integrated. I've moved packaging to git since that seems to be the least scary choice nowadays: http://git.debian.org/?p=pkg-mutt/mutt.git The repository has been updated to 1.5.19. The mutt-patched build is disabled for now, as the patch doesn't apply. Testers welcome :) If you are interested in helping out, please get in contact with me. To get started, subscribe to mutt's PTS feed. There's no extra maintainer mailinglist, mailing mutt@packages.debian.org will work for PTS subscribers. If you prefer IRC, join #mutt on irc.freenode.net. Thanks,
Christoph

10 January 2009

Marco d'Itri: ssh-agent for unattended processes

This code allows unattended ssh(1) processes, like the ones started by cron(8), to access the SSH keys stored by a running ssh-agent(1) (which will probably have been started in a screen(1) instance). Be sure to understand the security implications of persistent ssh-agent processes and passwordless keys. 
#!/bin/sh -e
import_ssh_agent()  
  local pid
  for pid in $(pgrep -u "$LOGNAME" -x -f /bin/bash); do
    local var="$(grep -z '^SSH_AUTH_SOCK=' /proc/$pid/environ   true)"
    [ "$var" ]   continue
    eval "$var"
    export SSH_AUTH_SOCK
    break
  done
 
[ "$SSH_AUTH_SOCK" ]   import_ssh_agent
# now the script can run commands like:
ssh -o BatchMode=yes hostname command

8 January 2009

Wouter Verhelst: On DebGem

I feel I should comment on the whole DebGem thing. About a week ago, the guys from that project mailed me, stating that they were about to go public, and if I would please have a look at it before they would do so and perhaps give some constructive criticism. So I did. I must say I wasn't very impressed, initially. What you get there is what the name suggests: a Ruby Gem, but then just repackaged as a Debian package. As in, files laid out exactly the same way. As in, still not FHS compliant. As in, many other problems. I came up with a list of problems that was quite long, and they've fixed most of them; but the fundamental problem that RubyGems are peculiar beasts, making it hard, if not impossible, for Debian people to package them so that they adhere to Debian's strict quality requirements remains, and this service does not (or cannot) solve that. To be sure, it's a step in the right direction, which will ease the maintenance of Ruby software on many a Debian system, and for that they deserve praise. However, it is not a solution; much more work needs to be done and it needs to be done within the Ruby community.

3 January 2009

Marco d'Itri: I love quagga...

gw-uffici.fro# sh ipv6 ospf6 nei
Neighbor ID     Pri    DeadTime  State/IfState         Duration I/F[State]
192.0.2.130     200 -341952:-44:-21   Full/BDR             00:00:08 eth0[DROther]
192.0.2.131     200 -341952:-44:-18 Twoway/DROther         00:00:05 eth0[DROther]
[...]

22 December 2008

Steve Kemp: They look like big, good, strong hands, don't they?

Russ Allbery recently commented that it is really nice to receive patches for trivial scripts posted online. I agree. More than once I've posted a trivial script and had it be improved by people, or later included elsewhere. So in the spirit of sharing here is my latest toy script: This is a trivial script which searches a Maildir hierarchy and outputs a list of each email address which you've ever sent mail to. Why would you want that? In my case my (personal) spam filtering makes use of whitelisting, and the assumption is that if I've ever mailed you in the past then I want to see your replies, and you get a break. These days my (personal) mail filtering has a couple of broad rules: After that then I see your message only if CRM119 decides I should. 
#
# remove potentially spoofed header
#
:0 fhw
* ^X-whitelist:
  $FORMAIL -I "X-whitelist"
#
#  GPG-signed messages are OK and will be whitelisted
#
:0fW
* < 1024000
 /home/steve/bin/isgpged
:0e
  $FORMAIL -A "X-whitelist: yes" -A "X-GPG-Signed: Yes"
#
#  Get the sender of the message.
#
FROM= formail -x From:  sed 's/^\([^@]*[ <]\)//'   sed 's/\([ >]\).*$//' 
#
# Add a whitelist tag if appropriate
#
:0 fhw
* !^X-whitelist: yes
* ? test -s $HOME/.procmail_whitelist
* ? echo $FROM  fgrep -qisf $HOME/.procmail_whitelist
  $FORMAIL -A "X-whitelist: yes" -A "X-Whitelist-Test: $FROM"
The net result of these tests is that I can now run the spam filter on non-whitelisted mails: 
#
# Run CRM114 mailreaver
#
:0fw: .msgid.lock
* !^X-whitelist: yes
  /usr/bin/crm -u /home/steve/.crm /usr/share/crm114/mailreaver.crm
#
#  Spam.
#
:0:
* ^X-CRM114-Status: SPAM.*
* !^X-whitelist: yes
.CRM.Spam/
#
#  Unsure.
#
:0
* ^X-CRM114-Status:.*UNSURE
* !^X-whitelist: yes
.CRM.Unsure/
There is more to my setup than that, but that's the minimum you'd need to see. Of course this is a reminder, once more, that the kind of filtering that you carry out for yourself is different from that that other people will do. ObFilm: The NeverEnding Story

9 December 2008

Josselin Mouette: RubyGem is from Mars, apt-get is from Earth

There is again some ongoing debate about Ruy gems and how to integrate them cleanly into the operating system. Let s put aside the fact such tools are useful on Windows and MacOS which don t have decent packaging systems. This is not a reason to dictate stupid requirements for other operating systems. Still, even without that, gems, eggs and the like are great tools for development; tools that developers can t work without today. They allow to easily setup a development environment with everything needed, at the latest versions. However they were not meant for integration and production environments, and will never be suitable for them. We re living in another world indeed: a world where servers must be up 24/7, where security issues flood your inbox every day, where you need dozens of servers running the very same software. This world is the Earth. Python eggs and Ruby gems were designed for the same target audience and with similar requirements; as such, they share the same deficiencies when it comes to other audiences:
  1. They replace the operating system s packaging tool. This is very nice to not wait for the packagers when you want the latest development version, but you lose track of what is actually installed. In production, you need to know exactly what s installed and at what version. You also need to know what you will need to update in case of a security issue.
  2. They install packages automatically on the system. This is really the biggest WTF; there are some reasons to do that to begin with, but they are all wrong.
  3. They use a specific file format. This requirement comes mostly from the Windows world, and it really makes no sense on Unix. It makes it much harder to integrate complex applications using components in different languages.
  4. They ship all files at the same place, happily violating the FHS and making it even harder to integrate with other things.
Python tools (setuptools and easy_install) have shifted their philosophy and brought a few small improvements which allow us to easily work around issues 1-3. These tools now include all you need to build a development environment in any directory that will not have an impact on the system packaging, making a clear distinction between the two targets. Furthermore, the metadata is shipped separately and can safely be included in a distribution package. Similarly, modules will not be installed automatically on your system. I have yet to see such a move from Ruby developers. However, what we are still missing is something along the lines of dh_make_perl: starting from a CPAN module, it will generate a clean and working Debian package in a few minutes. There have been some similar attempts for Python, but nothing as efficient. And I am certainly guilty of not having done something in this area, as one of the most knowledgeable of issues we encounter while packaging Python modules. As a note, I should add the underlying issue behind all of this is still the same: the NIH syndrome. Developers are reinventing the wheel, engine and transmission. Which is not that bad per se, but by not looking at existing solutions for the problem of making a car move, they are inventing a square wheel, a steam-powered engine and a superconductor-powered magnetic transmission. For example, while trying to fix the 4th of items listed earlier and they are really willing to fix it and several more general issues, setuptools developers forgot to look at the existing solutions: Certainly, the existing tools are everything but perfect, and not necessarily suitable for Ruby and Python, but some of the design ideas behind them are clearly the good solutions to some of the issues of software development and integration. Adapting these ideas to radically different languages is going to take more time than just throwing other layers of symlink farms.

8 December 2008

MJ Ray: 95% of British Residential ISPs Censor - but I think I m OK

The Internet Watch Foundation has objected to the cover of a 1976 Scorpions album (which is, as far as I know, still legally on sale here, with that cover) and their collaborator ISPs have blocked access to Wikipedia. They claim that it covers 95% of British Residential ISPs but I could get through. Can you? Read more formal reports from AP via Google (claims 95% of users, which I think might be an error), BBC, Wikimedia Foundation News Release or The Guardian report (which chickens out of showing the whole cover image). Inspired by a short message from the owner of Cockspiracy, I’ve asked the Phone Coop election candidates for their views on it (note: I expect the question URL to change soon, so beware if you link to it), comparing it to our phorm promise. Did this affect you? Do you care? Have IWF just scuttled their ship?

9 November 2008

Simon McVittie: Spec-writing On A Plane

(Written on Friday 16th, posted on Monday)

As I write this, I'm on a plane off the coast of the Netherlands, on the way back from a couple of days designing APIs with Rob McQueen and the RTCom people at NRC Helsinki (who we work with on the chat and VoIP functionality for Nokia internet tablets).

We've had a very useful couple of days, mainly designing the next-generation channel requesting and dispatch API for Telepathy (the same APIs I've been doing preparatory work for on the Telepathy mailing list in the last couple of weeks). API sketches for review and comment are either on are on Merge Monkey or on the way there.

Before even reaching Nokia, Rob and I did a lot of design on the plane over to Helsinki, and I took some notes: here they are, by way of a glimpse at how the Telepathy design process works when we don't have a whiteboard[1] :-)

  • Geoloc looks good in principle, needs some more work
    • add more docstrings
    • add a way to ask people for uncached info (RequestLocations)
    • move access control to Co.I.Presence, call it RichPresenceAccessControl?
  • DeliveryReporting looks good, ish
    • improve wording of delivery-echo and rationale
    • when sending a delivery report, add a reason for the status (using Channel_Text_Send_Error?)
  • Uniqueness requirement for (channel, handle_type != 0, handle) is in the way - let's drop it
  • MSN should use "authentic" 1-1 chats, and use Ch.I.UpgradeableToRoom (name tbd) to migrate the underlying switchboard to be a room
  • XMPP should use Ch.I.UpgradeableToRoom too, but the 1-1 channel may also stay open
  • Mutable handle => handle 0 must stay (for requestotron's benefit)
  • bundles ftw
  • renaming: we want SelfHandleChanged in the core
  • TpC* - add ability to subscribe to ifaces ay construct time (blocks Ready) or later
  • Dispatching: when registering a c'handler, specify everything it can handle
  • If a new bundle can be handled in its entirety by the same handler (call it 'the bundle handler'), do it. Otherwise split it up completely (there is no 'bundle handler').
  • If a channel in an existing bundle can be handled by the bundle handler, send it there, else send to the default handler.
  • Group - add SelfHandleChanged, maybe? Also Connection
  • Add HandleOwners: prop a uu , HandleOwnersChanged(a uu , au)
  • Use TargetHandleType, TargetHandle
  • Stringified initiator handle as a separate property
  • Maybe even a FirstMessage property, although this stretches the definition of "immutable"
  • org.freedesktop.Telepathy.Client interface

Nokia's Mikhail Zabaluev kindly took notes for some further discussion we had with him and Alberto Mardegan about the dispatch API, which he already posted to the mailing list.

We hope this API will make it possible to integrate Telepathy into the desktop better - it should make it a lot easier to share connections between processes in a useful way and hand out incoming channels to the appropriate handlers, which was always a major goal for Telepathy. It should also let us and others implement exciting new notification mechanisms (hi to the people on IRC and the mailing list wanting to patch their media players to integrate with Telepathy! :-)
[1]When we do have a whiteboard, the design process is: "Robot101 scribbles on a whiteboard, smcv writes down <tp:rationale> elements"

7 October 2008

Marco d'Itri: Linux HA firewalls: how?

I am configuring a simple two-hosts redundant linux firewall. What should I use to move the virtual gateway IP between the customer-side interfaces? And why? Apparently the choices are: keepalived, ucarp, vrrpd, heartbeat and maybe even something else.

6 October 2008

Patrick Schoenfeld: Where do find help for commands?

Today an discussion rised up in #debian-devel on the OFTC IRC Network. It started because someone noted the bug report #501318 which is, to summarize it, just a user mistake, because someone obviously read the man page (time(1)) for the /usr/bin/time command, while time is also a shell builtin (which does not accept the same arguments as the time command. Certainly this bug reports appears to be funny at the first sight, but on the other hand, there is a suboptimal situation that leads to this.

  1. There are some builtin commands that also have binary equivalents (like time, printf, echo). Its quiet easy to tell weither the one or the other is used, by using the which command, but thats not really a realistic workflow, so its better to know this. The difference between these commands is often causing problems, which we have to cope with. For example bashs builtin echo behaves different as /bin/echo and people who use the bash as their default shell tend to use what bash provides, which in turn causes problems if other people who use a different default shell try to work with these scripts. But this is another problem, because...
  2. ... every program, utility and function in Debian has to provide a manpage, as said by our policy: "Each program, utility, and function should have an associated manual page included in the same package." I guess that the rationale behind that is that 'man' is a very common tool in the Un*x world, which is wide-spread and which usage is much recommended to find out how specific tools behave. It is always referred to in Documentation, weither it is Debian specific or not, e.g. in books.

    The time package, which is of priority Standard and which includes /usr/bin/time, does conform to that policy by providing a manpage for the time command. The various shells don't do that, because they usually don't have a man page for each and every command (usually they have a more generic manpage which includes the builtins or in some seldom cases a special manpage for such and similar things (as for zsh, which has zshmisc(1)) and because its not that easy, because the man command cannot (AFAIK) distinguish between the user joey_average calling 'man time' in a bash, while the user schoenfeld calls the command 'man time' in a zsh, or if joey_foobar calls 'man read' in a shell which does not have a builtin time command and uses /usr/bin/time instead.
So whats wrong about this situation? Would you say that users that expect 'man time' (or similar examples) to do the right thing are making wrong assumptions? I disagree. Its what they've always been told to do. And if it does not show anything eventually run info, or look at HTML documentation or what else. But they haven't been prepared for the case where the manpages does show something, something wrong. The good thing is that the manpage for time includes a sentence:

"Users of the bash shell need to use an explicit path in order to run the external time command and not the shell builtin variant."

The bad thing about it is, that it is at approx. 70-80% of the manpage.

Clint, the maintainer of zsh, mentioned run-help which seems to be a part of the zsh, but not of any other shell, and does more or less the right thing (at least for the builtins) but not for external commands and not even for itself (it opens the code of the function instead of something user-readable like a manpage). I guess "one tool for a specific need" is a good maxime, but is it really a good maxime for finding documentation?

But how could the situation be bettered? I could think of a wrapper for man, similar to run-help but as a more generic solution. Any ideas for it? Is it the right way at all to better the users experience? Any other ideas? Other opinions?

Next.

Previous.