Search Results: "tar"

21 February 2012

Russ Allbery: Debian packaging updates

Now is the point in the Debian release cycle where I usually try to go through my packages and bring them generally up to date. In case other people are doing the same thing, here are a few new facilities or techniques that I'm rolling out across my packages. (This is apart from the obvious stuff, like multiarch where appropriate and debhelper 9, and the older stuff, like using dh.) It's also worth mentioning that Ubuntu was responsible for breaking a lot of ground here. Due to bug reports and patches submitted from Ubuntu, several of my packages already had hardening build flags and --as-needed issues fixed before this round of packaging refresh, which made adding these features much easier than it would be otherwise.

20 February 2012

Tanguy Ortolo: Opportunistic SSH agent

To use an SSH agent, one usually has to:
  1. launch the agent;
  2. add his key to it.
The first step can be automated in the desktop or shell startup script (this is a typical use case for login shell-only startup scripts, by the way), but the second one cannot if your private key is protected by a passphrase.Opportunistic key loading GNOME Keyring includes an SSH agent which has an interesting feature I call opportunistic key loading : it asks for the passphrase to decrypt the private key the first time it is needed. This feature mimics the behaviour of PGP agents, and eliminates the second step I previously mentioned. Question Do you know if there exists another SSH agent able to do opportunistic key loading?

Richard Hartmann: Lenovo hardware quality

This is half rant, half warning to others. Relative merrits of the IBM ThinkPad X31 vs the Lenovo ThinkPad X220... Old IBM ThinkPad X31 which I abused for years: I upgraded the RAM, swapped the battery once and the HDD twice. If not for a loose power socket, I would still use it. Contrast my new Lenovo ThinkPad X220 after a few months of pampering: Google tells me there are several people with random issues with their X220, especially when the mini-PCI slot is being used in a machine with USB 3.0. Removing the UMTS/GPS module did help, but still, this means no SSD for me as that's the slot I would have to use. Recently, things have been degrading very quickly to the point where I refuse to do even simple things, like write and push blog posts, on this machine. I am not sure if I am simply unlucky and received a dud, if the X220 is a model with issues, or if this is indicative of a general trend within Lenovo, but it's time to contact Lenovo support. If I had known all of this in advance, I would simply have soldered the power socket back into my X31 and held out for the X221. Or something else, entirely.

Michal Čihař: Prague carnivals

During these days, there are various carnivals in Prague. On Saturday I've visited Malostransk masopust, which is probably not the biggest one, but definitely the one in most attractive location, starting at Prague castle and going down to Kampa. This was actually my first visit of this carnival and I quite enjoyed it. Bellow you can find some photos taken during the event to catch the atmosphere:

Filed under: English Photography 0 comments Flattr this!

19 February 2012

Richard Hartmann: RAID sucks

Intro RAID sucks, and so do all other Free alternatives in the Linux world. Having been in exactly the same situation several times in the past, I have been following Russell Coker's posts regarding data integrity with interest. FWIW, the correct answer to "BTRFS (sic) and Xen" is: subvolumes. Now that Fedora has, once again, decided to postpone their switch to Btrfs as default file system, I decided to write up my own take on this topic. Any and all my considerations are made under the assumption that important data is backed up while semi-important data is at least living on two different machines. Beware, this is a long post, but I do like to think it's well worth reading. RAID Disk failures Let's recap the current situation for "traditional" file systems like xfs, ext 2..4 , etc. In case you need to freshen up on the technical details go here. Silent corruption All the RAID levels with redundancy are fine and dandy if you lose a whole disk and put in a new one. Of course, that does not help you the tiniest bit once you get read errors. Controllers which manage RAID 1 or 10 may or may not compare the data while it's being read and may or may not tell you about discrepancies. They can then toss a coin and give you either result. There is no way to determine which is correct. RAID 2, 3, 4, 5, and 6 could, in theory, verify the data as it's being read, but that would limit your read rate to that of a single disk so no one does that. We actually had a massive web presence fail over night, once. No one knew what the cause was as there hadn't been any kind of access logged. As that particular deployment is done directly from a VCS, we simply ran a diff and found one change. We traced the failure down to a syntax error; a one character change made everything fail. Looking at the ASCII table, it was clear that one single bit had flipped. This sucked. A lot. Mitigation So you end up scrubbing your RAID sets on a weekly basis, recovering from silent corruption with the help of your RAID 6's two parity stripes. And even though you schedule the scrubbings with the least priority, you still take a performance hit when randomly seeking a lot. ZFS Some smart people at Sun set out to fix these problems and more, and fix it they did. The "Z" in ZFS is meant to imply that this is the last file system you will ever need and if not for its license, this might have worked. Of course, "last" and "forever" mean "not now" and "10 years" respectively, in computer terms. The limits within ZFS have been chosen so that the entropy needed to create a disk able to max out a ZFS volume would, literally, require our oceans to boil. Let's just say that you won't encounter these limits any time soon. While that is a nice thing to know, it does not say anything about data integrity. There are several mechanisms ensuring data integrity built in ZFS, the most fundamental being extensive checksumming. Checksums are vastly superior to live comparisions which RAID would have to perform as it does not need to sacrifice any significant i/o reading checksum data. With HDD access, you are i/o-bound, and not CPU-bound, anyway, so performing the checksum calculations is laughably cheap when factoring in the added safety. Everything, be data, metadata, inodes, you name it, is checksummed. Every read operation must go through the checksumming functions, ensuring that all data which reaches your user space is correct. If ZFS can't verify the checksum, it will not deliver any data at all, ensuring noticeable, as opposed to silent, corruption. So how does ZFS recover when the checksum does not match the data? copies= 1..3 The easy way is to store several copies of your data within your volume. Mount a ZFS volume or subvolume with copies=2 or 3 and all data that is being written from then on will be copied twice or three times. Trivial in principle, but powerful when built into a file system. RAIDz 1..3 This is where things start to get interesting. RAIDz1, 2, and 3 will allow you to lose 1, 2, or 3 disks respectively while still retaining your data; at the obvious expense of the storage capacity of as many disks. Basically, RAIDz2 is ZFS' variant of RAID 6, but it's so much better than mere RAID 6. Moo! RAIDz will not assign fixed slices and fill them up like RAID does. It will look at the current write speeds and use appropriate slice sizes dynamically. As a direct result, data which has been written at roughly the same time will always be near other data written at the same time. This is very nice for functionality like snapshots, writeable subvolumes and other things and is called COW, ZFS' variant of super cow powers. This also enables ZFS to write data, read it back and verify the checksum and only then point to the new data. Atomic commits done right. Keep rollin' As ZFS has to keep track of what data is still in use in which volumes and subvolumes anyway, it knows which regions are free to be reclaimed. Instead of filling up random places, ZFS will roll over your disks, overwriting unused data on the fly. This bears repeating: Contrary to RAID 5 and 6, ZFS will never need to read old data in order to write new data. Write performance galore and anyone still stuck on RAID 10 can finally enjoy the increased data security. Btrfs To be completely honest, I do not know Btrfs as well as I know ZFS. If I get anything wrong, correct me. If I offend anyone with my outsider's interpretation, that is not my intention. Btrfs has been initiated by Oracle way before it ever even thought about buying Sun or history might have run a different course, for worse or a lot worse. The closing of Java, Solaris, ZFS, Hudson, OpenOffice (since then "gifted" aka thrown away), and others makes me think the latter. As there are few technological developments which caused me as much stress, overtime and pain as OCFS2, I am naturally wary of file systems sponsored by Oracle and their data focus integrity/availability. The fact that there's still no way to fsck a Btrfs volume could be birthing pain or yet another facet of Oracle's stance on this topic, I honestly don't know. Either way, it's a good idea to be wary for now. I can't say too much on Btrfs' technical underpinnings, but I know it uses COW and that it has its own variant of RAID. Toss in snapshots (writeable?), subvolumes and integrated block device management, and you have the building blocks of a decent file system. Still, Btrfs is not ready for prime time. Btrfs has been "one or two years in the future" for a few years now so I will not be holding my breath. And once the first distributions are starting to use Btrfs by default, people will lose data. A year or three after that, I will feel comfortable to use it myself and in production. Commercial solutions You can either hand a lot of money to proprietary vendors who will not tell you a thing about how things work internally (and no way to fix things directly) or you can buy solutions that employ ZFS in the back-end. I prefer my storage hardware to be relatively dumb while keeping the intelligent bits where I can see and poke them so this is not really part of these considerations. So what can you do? Test your disks before deploying them Easy, but vitally important:
disk=/dev/foo
smartctl -a $disk
smartctl -t long $disk
badblocks -swo $vendor_$model_$serial_$timestamp.badblocks.swo $disk
smartctl -a $disk

If you are lazy, get a copy of disktest, a tool I wrote to do exactly this. It should wait for the long self-test to finish, can not read out all vendor names, and does not have a way to document who ran the test, but it's a start. Patches and feedback are, obviously, welcome. And yes, I will package this soonish. Using your disks This list is surprisingly short. You can use And that's it. At work, I prefer small RAID 6 volume sets. It sucks, but there is nothing better. For personal use, I have a machine with a FUSE-based RAIDz2 mounted with copies=2. Three disks can fail and my data is still secure. While this setup is slow as molasses due to FUSE, speed is not a consideration for me here; data safety is. A migration to Debian/kFreeBSD in the medium term is still likely. If you have things to add or disagree with me, I would love to hear from you. Update: Jan Christian Kaessens pointed out that, contrary to my experience, zfsonlinux did not bring down his sytem in flames. As zfsonlinux is a kernel module, potential performance is a lot better. Also, it supports ZFS Pool Version 28 as opposed to FUSE-ZFS' 23. While the really nice features are in v 29 and 30 (which are closed thanks to Oracle) v 28 still has some nice changes.

18 February 2012

Asheesh Laroia: Help a BSD developer bike across the US, and give hope to cancer communities

<style type="text/css"> @import "/pub/special-css/venk.css"; </style>
'Cancer' is a cluster of diseases, a betrayal by the majesty and power of the development program that constructed and heals us.
Support Venkatesh's bike ride, and alleviate the toll of cancer.
My friend Venkatesh, pictured above, is going to bike four thousand miles, all the way across the continental US, from Baltimore to Portland. He's doing it to raise money for the Ulman Cancer Fund for Young Adults. I'm writing this because I want you to donate money to his cause. He's a DragonFly BSD developer, loves bikes, and your donation could make a big difference. I first met Venkatesh through the Johns Hopkins computer club, an ACM chapter. I was the head of the club, and he had just started his career at Hopkins. He was looking for advice on running Brickwiki, the LEGO encyclopedia. Quickly, I became his friend; in that time, I've learned the following things about him. In the years since I graduated from Hopkins, I've been impressed by Venkatesh's ongoing curiosity and contributions to open source projects like DragonFly. I'm honored to have this chance to help him bike across the country for a good cause. Here is a quick word about the 4K for cancer effort:
Since 2002, groups of college students have undertaken a 70 day, 4000+ mile summer bike ride across the United States with the goal of offering hope, inspiration and support to cancer communities along the way. This past summer was our 10th year of cycling across the country as 76 volunteers rode along three different routes: Baltimore to San Francisco, Baltimore to Portland, and Baltimore to Seattle. Our riders raised a combined $476,000 to support organizations and individuals in the fight against cancer.
His fundraising goal is $5,000. Anything from $5 to $500 is a donation to an organization that helps young adult cancer surviers and their families get access to information and support resources. Can you help?

Biella Coleman: Montreal, first impressions

frozenlights When I told people of my plans to move to Montreal, it usually prompted one of two reactions: one was some version of joyful envy, many people exclaiming breathlessly Montreal is one of my favorite cities, one person once even clutching my arm and told me as he looked me straight in the eye: you are so lucky, there is no city quite like Montreal in North America. The second reaction came off as a thinly veiled mixture of disdain and disbelief usually peppered with many reallys: oh wow, really, really you would really leave NYC? I am pretty certain they really were thinking something along the lines of what a fool, how dare she leave a great job, a great university, a world class city (the only city to live) for some Canadian mid-sized city, which is like tundra for a good chunk of the year? The decision to move weighed heavily on me, if for no other reason I had a choice to stay or to go and I honestly have not had a choice, a decision to make about where to go since I got accepted to graduate school (and even then the choice was more obvious than this one). So over a month into my move what is my verdict? In a word, win. I don t miss NYC at all though I get why some people cannot leave the place and know that despite some oddities and difficulties of living in Montreal, it fits my tastes and needs much better than NYC which dwarfed me in so many ways. I never felt I could enjoy it, I grew tired of the cramped living quarters, the noise ate at my soul, and I simply felt more overwhelmed by the fact that I could not even get a handle on the neighborhoods in my vicinity, much less all the other hoods in the area. Instead of dogging NYC anymore, I think I will spend a little time on first impressions, as they will soon be lost to the familiarity borne with time and experience. In essence Montreal is chock full of life but rather intimate, a quirky city with lots of charm but some grit and lacking the way over designed and done feel of cities like Portland. Here are some of quirks: 1. The Hawt Metro: I fell in love with the Metro when I first rode it a few years ago. I just love the powder blue color of the cars and the super sweet 1960s aesthetic of many stations. Even better and unlike NYC, they are just clean and quiet. The downside? The temperatures approximates a sauna during the winter and while you would think this is a good thing, when you are layered with the clothing necessary to survive outside (re: long underwear along with Canada-coat, gloves, hat, and scarf), it is hooooooot down there and you feel like you need to pass out. 2. Spend money, get free stuff: In many establishments you get free stuff (like blueberries or some like sports bar) when you spend over a certain amount of money, like 70 bucks. Quirky local tradition. 3. The culture of negotiation and the kick ass standard lease: Housing is amazing here. There is plenty of it, there are many different styles, and it is rather fun exploring all the different hoods that make up the city. Problem is too that lots of apartments have weird problems and issues and I had to steer clear of anything that could even possibly have mold. I spent weeks day in day out looking for a place, desperate to move out of my corporate apartment very nicely provided by the university but still not my ideal living situation. Finally found a place that fit all my needs in the perfect location and I took it to only find out that places are priced to negotiate and I was faced with the decision to negotiate or not. Sort of did, was not thrilled about it (thinking that I might lose the place) but it sort of worked and I scored the place. If price is up to haggling, the lease on the other hand, is standard (you can buy one at your local bodega ). It is the law to use it and it is like a no nonsense, straightforward lease, which is very protective of renters. 4. Montreal is known for its exceptional food but you know Poutine is just plain gross: Food here is good and I can tell that I will get a handle of restaurants in a way that felt impossible in NYC. The gluten free religion seems to be spreading, thankfully. There are many little Fruiterias! to get your fruits veggies which I am still exploring and right around my house is a crazy supermarket that is so cheap, which is weird because consumer goods generally ain t cheap in Montreal like they are in the states but this place is a gem and everyone agrees (and almost impossible to notice from the street!). Now Poutine is disgusting. Ok it did not help that the first night I went out to eat it, I was still under the clutches of Noro virus, and I think it re-activated the nausea that had been zapped by some strong medicine and the hospital, the day before, which brings me to the next point, the health care system. 5. Healthcare, I had to use it way too early: So last week I came down with the Noro virus and you usually wait it out as it runs through you quickly but I was dehydrated before I even started to vomit to for 8 hours straight, at which point severely tired and so nauseous (I was yelling to make it stop), I went to the hospital. Now I had talked to lot of people about the healthcare system in Montreal as it seems good, really great but a bit over taxed, especially compared to where I had lived in Edmonton where it was like a magic fairy tale dream. I had heard of two things: the care is excellent but the facilities are shocking and the wait times unless dire can be atrociously long. So facilities, yea they are kinda shocking, somewhat shabby but who cares, so long as the care is good, no? Packaging is irrelevant so long as the goods are derived. Before going I was scared of the long wait times (and also the taxi took me by mistake to Montreal General Hospital was looked too much like a HUGE version of the creepy buildings in The Shining for me, and I was supposed to go to Jewish General so left for there). The wait time: nearly none, somewhat as shocking as the facilities first looked to me. I think it was a combination of the time I arrived, with the fact that my lounge was parched and yellowish-gross (sorry, it was gross), indicating I was dehydrated, oh and I was crying bit hysterically, for despite my high threshold for pain, nausea terrorizes me. I was covered by insurance and since I did not yet have my McGill health cards (it takes three months to qualify for the local stuff), I did pay and the price was laughably cheap compared to what I would pay in NYC for the same treatment. 6. Now for my favorite, snow so nice, ice oh Christ: Well this winter has been, by all accounts, weak and warm, the spirit of winter barely making its way from the underworld to the outerworld. But even though it is has been more idle than full throttle, I still got a pretty accurate taste of what winter is like, with a few days of 20- c temps, and having to walk a number of times on a layer of frozen ice that makes it feel like a very dangerous mini-ice age in the making. IMAG0028-1 I do rather love the quiet snow falling and just love love sprinting through the snow with my dog, Roscoe, who has taken a liking for prancing in the white stuff and looks awful cute with his winter man s ice beard. After a sizable snowfall, it is clear they city does not toy with the snow removal although the sidewalk snow plows do look somewhat like very large and dangerous but kinda cute toys. But let s be frank, winters are hard, so hard that I think I would go mad if I had to stay through the entire thing, being born and raised in the tropics So the fact that I am writing this from the southern hemisphere in the height of summer gives me the assurance I can handle the rest of the snow, ice, ice and snow upon my return.

Dirk Eddelbuettel: Using Rcout with Rcpp / RcppArmadillo to coordinate output with R

The new RcppArmadillo release 0.2.35 now supports the Rcpp::Rcout output stream device. Based on a contributed Rcpp patch by Jelper Ypma, the Rcpp::Rcout output stream gets redirected to R's buffered output. In other words, R's own output and that eminating from C++ code using Rcpp::Rcout are now both in sync. This avoids a stern warning from Section 5.6 in the Writing R Extensions manual:
Using C++ iostreams, as in this example, is best avoided. There is no guarantee that the output will appear in the R console, and indeed it will not on the R for Windows console. Use R code or the C entry points (*note Printing) for all I/O if at all possible.
and does in fact provide exactly what is recommended: the same entry points R itself uses. Below is a sample program, once again using the wonderful inline package to compile, load and link C++ code into R from a simple text variable submitted to the cxxfunction. What is shown in R code to load the package, the definition of the C++ code as assigned to a variable src and the creation of the dynamically-loadaded R function called fun which contains the code from we compiled, link and load via a single call to cxxfunction() given src.
library
library(inline)
src <- '
  Rcpp::Rcout << "Armadillo version: " << arma::arma_version::as_string() << std::endl;
  // directly specify the matrix size (elements are uninitialised)
  arma::mat A(2,3);
  // .n_rows = number of rows    (read only)
  // .n_cols = number of columns (read only)
  Rcpp::Rcout << "A.n_rows = " << A.n_rows << std::endl;
  Rcpp::Rcout << "A.n_cols = " << A.n_cols << std::endl;
  // directly access an element (indexing starts at 0)
  A(1,2) = 456.0;
  A.print("A:");
  // scalars are treated as a 1x1 matrix,
  // hence the code below will set A to have a size of 1x1
  A = 5.0;
  A.print("A:");
  // if you want a matrix with all elements set to a particular value
  // the .fill() member function can be used
  A.set_size(3,3);
  A.fill(5.0);
  A.print("A:");
  arma::mat B;
  // endr indicates "end of row"
  B << 0.555950 << 0.274690 << 0.540605 << 0.798938 << arma::endr
    << 0.108929 << 0.830123 << 0.891726 << 0.895283 << arma::endr
    << 0.948014 << 0.973234 << 0.216504 << 0.883152 << arma::endr
    << 0.023787 << 0.675382 << 0.231751 << 0.450332 << arma::endr;
  // print to the cout stream
  // with an optional string before the contents of the matrix
  B.print("B:");
  // the << operator can also be used to print the matrix
  // to an arbitrary stream (cout in this case)
  Rcpp::Rcout << "B:" << std::endl << B << std::endl;
  // save to disk
  B.save("B.txt", arma::raw_ascii);
  // load from disk
  arma::mat C;
  C.load("B.txt");
  C += 2.0 * B;
  C.print("C:");
  // submatrix types:
  //
  // .submat(first_row, first_column, last_row, last_column)
  // .row(row_number)
  // .col(column_number)
  // .cols(first_column, last_column)
  // .rows(first_row, last_row)
  Rcpp::Rcout << "C.submat(0,0,3,1) =" << std::endl;
  Rcpp::Rcout << C.submat(0,0,3,1) << std::endl;
  // generate the identity matrix
  arma::mat D = arma::eye<arma::mat>(4,4);
  D.submat(0,0,3,1) = C.cols(1,2);
  D.print("D:");
  // transpose
  Rcpp::Rcout << "trans(B) =" << std::endl;
  Rcpp::Rcout << trans(B) << std::endl;
  // maximum from each column (traverse along rows)
  Rcpp::Rcout << "max(B) =" << std::endl;
  Rcpp::Rcout << max(B) << std::endl;
  // maximum from each row (traverse along columns)
  Rcpp::Rcout << "max(B,1) =" << std::endl;
  Rcpp::Rcout << max(B,1) << std::endl;
  // maximum value in B
  Rcpp::Rcout << "max(max(B)) = " << max(max(B)) << std::endl;
  // sum of each column (traverse along rows)
  Rcpp::Rcout << "sum(B) =" << std::endl;
  Rcpp::Rcout << sum(B) << std::endl;
  // sum of each row (traverse along columns)
  Rcpp::Rcout << "sum(B,1) =" << std::endl;
  Rcpp::Rcout << sum(B,1) << std::endl;
  // sum of all elements
  Rcpp::Rcout << "sum(sum(B)) = " << sum(sum(B)) << std::endl;
  Rcpp::Rcout << "accu(B)     = " << accu(B) << std::endl;
  // trace = sum along diagonal
  Rcpp::Rcout << "trace(B)    = " << trace(B) << std::endl;
  Rcpp::Rcout << std::endl;
'
fun <- cxxfunction(signature(), body=src, plugin="RcppArmadillo")
setwd("/tmp")                           # adjust on other OSs
fun()                                   # output to stdout
sink("rcpparma.log.txt")                # start 'sink' to output to file
fun()                                   # no output to screen
sink()                                  # stop 'sink'
We then switch to a temporary directory (as the example code, taken from one of the two examples in Conrad's Armadillo sources, creates a temporary file) and run the new function. To demontrate how it does in fact now mesh perfectly with R, we create an output 'sink' (which catches all output) and re-run. This simple example demonstrated how we can use the new Rcout output stream from Rcpp to have dynamically-loaded C++ code cooperate more cleanly with the (buffered) R output. It also demontrated some of the nice features in Armadillo which we bring to R via RcppArmadillo.

Neil Williams: Introducing ladder

We needed a new package at work and once I'd written it, I realised that it could well be useful for others, so I did the ITP and the package is now in Debian unstable.

Ladder - Stepwise repository upgrade tool

Ladder creates a local repository of packages required
to upgrade a tarball of a chroot to a new milestone or
software release.
.
The repository can be signed and includes all specified
target packages and dependencies. The repository can then
be distributed and used to upgrade multiple devices in
sequence without needing network access.


The only dependencies are some simple perl modules, reprepro for the repository, apt and gnupg for the signing. None of those need to be particularly recent versions, so once ladder has had a little time in testing, I will be doing a backport to Squeeze. Lenny is a little less likely but eminently possible if someone specifically asks for it. Not that it particularly needs a backport mind, if you want to run ladder on some other Debian-based system, it's simply a case of installing the version in unstable. reprepro was in Lenny at a suitable version for what ladder requires, so any system remotely recent should be able to run ladder without changes.

Normally, whenever you need to upgrade a device, you need to get that device onto a network, get access over the serial port or some form of pre-configured network connection, run the commands and clean up afterwards. OK, now do that again... oh and here are another 40 or 100 devices which all need precisely the same thing done and they all have to be shipped to paying customers today. What does any geek do? Script it.

Ladder is part of that process. There would need to be some scripting / programming support on the devices concerned but ladder makes it easier to put that support in place at the design stage and then automate the actual update without the device needing to get onto a network and, potentially, allowing you to decide how to offer the upgrade (automatic, user involvement, engineers only etc.) Once that is implemented, the device is simply pointed at a locally mounted filesystem which contains the ladder step for the migration. Each step is a SecureApt signed repository which is available at a deterministic location which can be easily scripted and which contains only the packages necessary (with dependencies) to upgrade any device installed at Milestone A to Milestone B. Nice and small, only the stuff you need, whatever architecture you like and if you need to migrate from Milestone A to Milestone D, then you create a few steps on the ladder and automate the migration from A to B to C to D. All you need is enough space on whatever local media you need to use to plug into the devices. Then copy the media enough times, start the interface on each device and do something useful whilst the upgrade happens. Then, when the next lot need to be upgraded, you already have the media... Oh and ladder does not need to run on the same architecture as the devices - all package handling is entirely architecture-neutral. So create the ladder steps and media on your fast x86 machines and then let the slower SSD devices take their own sweet time via automation. Simple - maybe, hopefully.

I'm expressly talking about devices here because this is where the original requirement arose - embedded devices which don't have direct / automatic / accessible network ports or even a working network configuration preset. These aren't new images to flash onto the device either, these are apt repositories, so only the stuff which needs to be changed is changed which saves a lot of time writing to slow SSD storage.

If the mechanism sounds familiar, yes, it's using the same principles as xapt, multistrap and emdebian-grip but I thought it could be useful, so I generalised the interface (a bit) and uploaded.

Let me know via the BTS if you need tweaks to the support. Ladder isn't particularly about getting desktop machines from Debian Potato to Debian Wheezy because you need to identify a target package to be the top point of the dependency chain which gets into the step repository and all devices should start with the same packages at the same versions. Normally, the kind of devices I use at work have a single top level application which is the sole user interface and which directly or indirectly depends on all the other software required for that device. Whilst the target package value could take a few carefully selected packages instead of just one and there is support for an extra packages field, it's not intended / expected to be useful for desktops. This is for use with multiple devices which all have the same package selection, are at the same milestone at the start and which all need to migrate smoothly to whatever milestone is intended - one step at a time. Ladder could be used to migrate between Debian releases and I've included example config files to support that, but the principle usage is with internal / proprietary repositories based on a common Debian stable platform where the user interface software is managed via identifiable milestones and where users have no direct control over installing packages. It's a production / manufacturing support tool more than a user / admin support tool. If you want to manage servers and desktops which allow users to install (or request to be installed) arbitrary packages, there are plenty of tools which already support this and are used by DSA in Debian for exactly these tasks. Ladder is not puppet but Linp didn't sound like a good name...

Ladder step repositories only include the binary packages, not sources - so if the migration is not going to be done in-house and you're expecting to distribute these steps to users somehow, ensure that the original source repository is available online for the Debian / free software packages concerned. If you can't do that (because the source is proprietary), you possibly shouldn't be distributing the binaries as a user update mechanism in the first place because each step will include packages from the core Debian system which need to have the sources available when distributed.

Inevitably, the ladder source package already contains two POT files for translations of the runtime messages and the POD based manpage. I expect to need to expand / clarify the manpage in due course, so don't rush to translate it yet as it's likely to change.

Craig Small: VMware Player on Debian

For various reasons, having vmware running on my desktop would be kind of useful. VMware provide a Free (as in beer) version of their software called VMware Player. I downloaded the file VMware-Player-4.0.2-591240.x86_64.bundle off their website and tried to build it. It failed to build. Given my previous lack of success with VMware server, I wasn t too surprised. What was surprising was it wasn t too hard to fix it. The problem was that the vmnet module would not compile and that was due to three things: The patch is only a few lines and means I can compile vmware on my Debian sid computer running kernel 3.2.0-1 vmnet.patch To use it, you will need to find where the modules are built, for me it is /usr/lib/vmware/modules/source
  1. mv vmnet.tar vmnet2.tar
  2. tar xf vmnet2.tar
  3. patch -p1 < vmnet.patch
  4. tar cf vmnet.tar vmnet-only
With that you can run the player which will try to build the modules and you re done!

Benjamin Mako Hill: Advice for Prospective Doctoral Students

There is tons of advice on the Internet (e.g., on the academic blogs I read) for prospective doctoral students. I am very happy with my own graduate school choices but I feel that I basically got lucky. Few people are saying the two things I really wish someone had told me before I made the decision to get a PhD: Most People Getting Doctorates Probably Shouldn't In most fields, the only thing you need a PhD for is to become a professor -- and even this requirement can be flexible. You can have almost any job in any company or non-profit without a PhD. You can teach without a PhD. You can write books without a PhD. You can do research and work in thinktanks without a PhD. You don't even always need a PhD to grant PhDs to other people: two of my advisors at the Media Lab supervised PhD work but did not have doctorates themselves! Becoming a tenured professor is more difficult without a doctorate, but it is not impossible. There are grants and jobs outside of universities that require doctorates, but not nearly as many as most people applying for PhDs programs think. Getting a doctorate can even hurt: If you want to work in a company or non-profit, you are usually better off with 4-6 years of experience doing the kind of work you want to do than with the doctorate and the less relevant experience of getting one. Starting salaries for people with doctorates are often higher than for people with masters degrees. But salaries for people with masters degrees and 5 years of experience are even higher -- and that's before you take into account the opportunity costs of working for relatively low graduate student wages for half a decade. PhD take an enormous amount of time and, in most programs, you spend a huge amount of this time doing academic busy work, teaching, applying for grants or fellowships, and writing academic papers that very few people read. These are skills you'll need to be a successful professor. They are useful skills for other jobs too, but not as useful as the experience of actually doing those other jobs for the time it takes to get the degree. Evaluating Graduate Programs If you are still convinced you need a doctorate, or any graduate degree for that matter, you will need to pick a program. Plenty of people will offer advice on how to pick the right program and trying to balance all the complicated and contradictory advice can be difficult. Although I love my program and advisors, I've known many less happy students. Toward that end, there are two pieces of meta-advice that I wish everybody was told before they applied:
  1. Find recent graduates of the program you are considering, and the faculty advisor(s) you are planning on working with, and look at where they are now. Are these ex-students doing the kind of work that you want to do? Are they at great programs at great universities? Chances are good that a PhD program and its faculty will prepare future students to be like, and do work like, the students they have trained in the past. Programs that consistently make good placements are preparing their students well, supporting them, making sure they have the resources necessary to do good work, and helping their students when they are on the job market. A program whose students do poorly, or just end doing work that isn't like the kind you want to do, will probably fail you too.

  2. If recent graduates seem to be generally successful and doing the kind of work you want to do, find one who looks most like the kind of academic you want to become and talk to them about their experience. Chances are, your faculty advisors will overlap with theirs and your experience will be similar. Ex-students can tell you the strengths of weaknesses of the program you are considering and what to watch out for. If they had a horrible experience, there's a decent chance you will too, and they will tell you so.

Doing these two things means you don't have to worry about trying to think of all the axes on which you want to evaluate a program or pour through admissions material which is only tangentially connected to the reality you'll live for a long time. What matters most is the outcomes, of course, because you're be living the rest of your life for a lot longer than you'll be in the PhD program.

17 February 2012

Andreas Barth: Sch fflertanz

Every 7 years with the Sch fflertanz the end of the Black Death in 1517 is celebrated. After lots of people died, everyone was too scared to go out in the streets again, even after the Black Death was gone. The Sch ffler (cooper = people who traditionally build barrels) started to cheer the people in Munich up with their dance, and made them go on the streets and start their normal lifes again. This is one of the few local traditions that even survived the modern times. The barrel is signed with "Sch fflertanz 2012" and "Zur Erinnerung an das Pestjahr 1517" = "To remind of the Black Death-year 1517", and should remind us that even in the darkest times, there is still hope and life goes on.

16 February 2012

Eddy Petri&#537;or: HOWTO: Git - reauthor/fix author and committer email and author name after a git cvsimport

You might find yourself at some moment when your git repository imported from CVS does not contain all the correct names and email addresses of the commits which were once in CVS but are now part of your project history in your git repo. Or you might do a cvsimport which missed a few authors.

Let's suppose you first import the cvs repo into git, but then you realise you missed some authors.

Before being able to do a git cvsimport, you need a checkout of the module or cvs subdir that you want to turn into its own git repo.

For ease of use I defined CVSCMD as
cvs -z9 -d :pserver:my_cvs_id@cvs.server.com:/root_dir
You will need to replace the items written in italics according to you situation, more exactly, you need to define 'my_cvs_id', 'cvs.server.com' and 'root_dir'. If your acces method to the server is not pserver, you should change that accordingly. This information should be available from your project admin or pages.


Check out the desired module or even subdir of a module

$CVSCMD checkout -d localdirname MODULE/path/to/subdir

git cvsimport -A ../authors -m -z 600 -C ../new-git-repo -R

How to find out the commits which do need rewriting

The way to limit yourself only to the commits that had no cvs-git author and commit information on git-cvsimport time is to use a filter like this:
git log -E --author='^[^@]*$' --pretty=format:%h
This tells git log to print only the abbreviated hashes (%h) for the commits that have NO '@' sign in the 'Author:', which happens if no cvs user id to git author and email was provided in the authors file and git cvsimport time.

We will use this command's output to tell later git filter-branch which commits need rewriting. *

But before that...

How do we find if our authors file is complete?

For this task we'll use a slighly modified form of the previous command and some shell script magic.
git log -E --author='^[^@]*$' --pretty=format:%an sort -u > all-leftout-cvs-authors
And now in all-leftout-cvs-authors we'll have a sorted list of all cvs id's which were not handled in the original git-cvsimport. In my case there are only 19 such ids:
$ wc -l all-leftout-cvs-authors
19 all-leftout-cvs-authors

Nice, that will be easy to fix. Now edit your all-leftout-cvs-authors file to add the relevant information in a format similar to this:
john = John van Code <john@code.temple.tld>
jimmy = Jimmy O'Document <jimmy@documenting.com>
In case you can't make a complete cvs-user-to-name-and-email map, you might want to use stubs of the following form in order to be able to easily identify later such commits, if you prefer (or you could let them unaltered at al ;-):
cvsid=cvsid <cvsid@cvs.server.com>

How to actually do the filtering to fix history (using git-filter-branch and a script)

After this is done, we'll need just one more piece, the command to do the altering itself which reads as follow (note that my final authors file is called new-authors and that I placed this in a script in order to be able to easily run it without trying to escape all spaces and such madness):

[ "$authors_file" ]   export authors_file=$HOME/new-authors

#git filter-branch -f --remap-cvs --env-filter '
git filter-branch -f --env-filter '

get_name ()
grep "^$1=" "$authors_file" sed "s/^.*=\(.*\)\ .*$/\1/"


get_email ()
grep "^$1=" "$authors_file" sed "s/^.*\ <\(.*\)>$/\1/"


if grep -q "^$GIT_COMMITTER_NAME" "$authors_file" ; then
GIT_AUTHOR_NAME=$(get_name "$GIT_COMMITTER_NAME") &&
GIT_AUTHOR_EMAIL=$(get_email "$GIT_COMMITTER_NAME") &&
GIT_COMMITTER_NAME="$GIT_AUTHOR_NAME" &&
GIT_COMMITTER_EMAIL="$GIT_AUTHOR_EMAIL" &&
export GIT_AUTHOR_NAME GIT_AUTHOR_EMAIL &&
export GIT_COMMITTER_NAME GIT_COMMITTER_EMAIL
fi
' -- --all
You might wonder what's up with the commented git filter-branch line with the --remap-cvs option. This script will NOT work for you as long as you have the stock git-filter-branch script and keep the option --remap-cvs while not patching your git-filter-branch script (/usr/lib/git-core/git-filter-branch), but that option will provide a file with the mappings from the old to the new commit ids. If you want that function, too, you'll want to apply this patch to git-filter-branch:

diff --git a/git-filter-branch b/git-filter-branch
old mode 100644
new mode 100755
index ae602e3..d1f7ef6
--- a/git-filter-branch
+++ b/git-filter-branch
@@ -149,6 +149,11 @@ do
prune_empty=t
continue
;;
+ --remap-cvs)
+ shift
+ remap_cvs=t
+ continue
+ ;;
-*)
;;
*)
@@ -368,6 +373,33 @@ while read commit parents; do
die "could not write rewritten commit"
done <../revs

+# Rewrite the cvs-revisions file, if requested and the file exists
+
+ORIG_CVS_REVS_FILE="$ GIT_DIR /cvs-revisions"
+if [ -f "$ORIG_CVS_REVS_FILE" ]; then
+ if [ "$remap_cvs" ]; then
+ printf "CVS remapping requested\n"
+
+ CVS_REVS_FILE="$tempdir/cvs-revisions"
+ cp "$ORIG_CVS_REVS_FILE" "$CVS_REVS_FILE"
+ printf "\nFound $ORIG_CVS_REVS_FILE; will copy and alter it as $CVS_REVS_FILE\n"
+ cvs_remap__commit_count=0
+ newcommits="$(ls ../map/ wc -l)"
+ for commit in ../map/* ; do
+ cvs_remap__commit_count=$(($cvs_remap__commit_count+1))
+ printf "\rRemap CVS commit $commit ($cvs_remap__commit_count/$newcommits)"
+
+ oldsha1="$(basename $commit)"
+ read newsha1 < $commit
+ sed -i "s@$oldsha1\$@$newsha1@" "$CVS_REVS_FILE"
+ done
+ else
+ warn "\nNo CVS remapping requested, but cvs-revisions file found. All CVS mappings will be lost.\n"
+ fi
+elif [ "$remap_cvs" ]; then
+ warn "\nWARNING: CVS remap was ignored, since no original cvs-revisions file was found\n"
+fi
+
# If we are filtering for paths, as in the case of a subdirectory
# filter, it is possible that a specified head is not in the set of
# rewritten commits, because it was pruned by the revision walker.
@@ -491,6 +523,11 @@ if [ "$filter_tag_name" ]; then
done
fi

+if [ "$remap_cvs" -a -f "$CVS_REVS_FILE" ]; then
+ mv "$ORIG_CVS_REVS_FILE" "$ORIG_CVS_REVS_FILE.original"
+ cp "$CVS_REVS_FILE" "$ORIG_CVS_REVS_FILE"
+fi
+
cd ../..
rm -rf "$tempdir"


Then, after running this script, let's call it filter, you should have a brand new git repo with the appropriate authors and their emails set.


P.S.: I have started writing this post some time ago but stopped just before the last part, the one with the filter script. I realise I might be missing something in the explanation, but if you have problems, please comment so I can help you fixing them.

P.P.S.: * I realised in the filter script at some point I wanted to do something like:
for R in $(git log -E --author='^[^@]*$' --pretty=format:%H   head -n 2) ; do
[the same git filter branch command above but ending in ...]
' $R
done
But I think I remember that $R didn't work on the whole history, but only on that revision, or some other weird of that sort. I know I ended up not filtering explicitly those revisions, but the entire history. I hope this helps.

Jan Hauke Rahm: Debugging getopt weirdness

I ve been working on enum again recently and got completely stuck in getopt_long() return values. I don t want to bore you with details, so let me cut to the chase: I needed a way to look into what getopt_long() is actually doing. Now, there are a few ways to figure out the doings of a function. If it s your own code, you could just go ahead and add a few printf()s to see if your variables fill up as intended. Then you could use gdb to walk through your running code. But what if you want to do so in functions outside the scope of your own code? In my case: getopt, i.e. libc? Obviously, you can download the library, add your printf() stuff, compile and install it, and see it all happening. But libc isn t exactly small, fast-compiling, and you really don t want to mess it up and have it installed on your system. So how about just taking what you actually care for and push it in? This is nothing experimental or particularly hacky . I just don t want to forget about it and maybe someone can make use of it. So here we go Download your libc sources to a new directory and copy the files you need. Try something like
mkdir -p /tmp/getopt/mygetopt
cd /tmp/getopt
apt-get source eglibc
cp eglibc-*/posix/getopt* mygetopt/
cd mygetopt
Now write up a primitive Makefile:
all: libmygetopt.so
libmygetopt.so: getopt.c getopt1.c getopt_int.h getopt.h
	gcc -shared -Wl,-soname,libmygetopt.so.1 -fPIC -o libmygetopt.so.1.0.0 getopt1.c getopt.c
	ln -s libmygetopt.so.1.0.0 libmygetopt.so.1
	ln -s libmygetopt.so.1 libmygetopt.so
clean:
	$(RM) libmygetopt*
In order for this to actually work, I fiddled around with a few #ifdef s in getopt.c. But that s something you ll probably find out yourselves. You should at least be able to compile it ( make it). Now put a printf statement somewhere in the function you want to inspect. Recompile the lib and then go back to your actual program. Running the program ist easy.
src/enum -p1 1 3
1.0
2.0
3.0
Surprisingly, running it using your own getopt version is just as easy. Just push your library in:
LD_PRELOAD=libmygetopt.so LD_LIBRARY_PATH=/tmp/getopt/mygetopt:$LD_LIBRARY_PATH src/enum -p1 1 3
Yo, Ulrich, my getopt version is waaaay cooler than yours!
Yo, Ulrich, my getopt version is waaaay cooler than yours!
1.0
2.0
3.0
If you re disappointed now, that s your own fault. I said it in the beginning, this is nothing special. It s not my idea. It s something that developers all use all the time or something like that. In case you didn t know about it, good for you. Otherwise, sorry for reading my memory dump.

15 February 2012

Mike (stew) O'Connor: Udev and multiple keyboards

I've started using my kinesis keyboard with my laptop a lot during the workday, but not the rest of the time, which means my USB keyboard is being plugged and unplugged often. I prefer to have a lot of keys remapped on the kinesys, So I was having an annoying time having to run xmodmap (possibly without the benefit of having Mod4 to get the window manager to give me a shell) to set these keys. I survived about two weeks before this bugged me enough to figure out what could be done to solve this. I had solved it once before by dropping a magically crafted file into /etc/hal/fdi/policy, but that system is long gone. Luckily it looks like thanks (at least in part) to Martin Pitt, udev has a keymap facility for handling this type of scenario. /usr/share/doc/udev/README.keymap.txt.gz has a bunch of good hints that got me started.
$ /lib/udev/findkeyboards
USB keyboard: input/event2
AT keyboard: input/event0
Finds my keyboard devices. I'm interested in the USB connected keyboard. So I run:
/lib/udev/keymap -i input/event2
To discover scan codes my keyboard generates for the keys I want to remap. here #654928 got me, the trick is to issue the above command from the laptop keyboard, so that the terminal you launched keymap in doesn't get wonky because it never got the key-release event from the enter key. When I hit keys on my external keyboard now, I get stuff like:
scan code: 0x7004A   key code: home
scan code: 0x700E7   key code: rightctrl
They key codes coming out match what is printed on the keycaps. From here I generate a file matching files in /lib/udev/keymaps, but change the keycodes to the keycodes I want to remap to, ending up with:
0x7004A leftmeta 
0x700E7 rightalt
This file goes into /lib/udev/keymaps/kinesis-stew. It appears the things like leftmeta/rightalt which are valid come from /usr/include/linux/input.h. It also seems to be valid to put hex there, but I didn't find a good way to figure out what hex codes go with which key codes. Then I look for the VendorID/DeviceID by looking in the output of "udevadm info --export-db" for input/event2 and find:
ID_VENDOR_ID=05f3
ID_MODEL_ID=0007
So now I'm ready to create my udev rule (which I put into /etc/udev/rules.d/99-stew-keymap.rules
ATTRS idVendor =="05f3", ATTRS idProduct =="0007", RUN+="keymap $name kinesis-stew"
And now everything seems to be good. I can plug and unplug my external keyboard without having to run xmodmap to get the keybindings I want, and the keybindings for the built-in laptop keyboard are not disturbed.

Dirk Eddelbuettel: Kurt Elling, Regina Carter and Anat Cohen at the CSO

Yesterday was Valentine's Day, and Kurt Elling came back to Chicago for a show at Symphony Center. He brought along guest stars Regina Carter (violin) and Anat Cohen (clarinet), as well as a small band led by Elling's long-time collaborator Laurence Hobgood on piano. The program was titled Passion World and featured songs from around the world. Quite impressively, Elling actually sang in French, Spanish, Portuguese, German and of cause English. The program lacked a little bit of cohesion at times, but it was still a most enjoyable and solid two-hour set. A similar concert will take place in Minneapolis on Saturday, so for those near the Twin Cities I would recommended looking for tickets.

Bartosz Fe&#324;ski: Start from scratch

That seems to be the correct approach when you haven t been updating your package for 5 years. Just start from scratch and package it again. It s easier than fixing all lintian warnings and apply its suggestions.

Russell Coker: Links February 2012

Sociological Images has an interesting article about the attempts to apply the word Camping to OWS and framing the issues [1]. Lester Macgurdy wrote an insightful article about the snake , a new technique for OWS protesters to beat riot police [2]. Ron Barassi suggests that Australia Day be celebrated on the 27th of May to commemorate the day in 1967 when the Australian constitution was amended to not be racist [3]. The current Australia Day is often referred to as Invasion Day . IMHO Ron deserves another Best and Fairest award. Stefon Harris gave an entertaining TED talk about improv Jazz music titled There Are No Mistakes on the Bandstand [4]. It seems that his concepts can apply to some extent to many collaborative projects. John Robb wrote an interesting article about the future of drone (UAV) warfare [5]. He suggests that having one person control each drone is a temporary thing and that the future is to have a cloud of cheap autonomous drones taking strategic control from one person. His comparison of Starcraft players to future drone fighters is interesting. The OWS movement is branching out into other related areas, OccupyYourHomes.org is one of the latest ones [6]. When banks try to forclose on homes without good cause the OWS people are protesting. Cory Doctorow wrote an important article for The Guardian about corporations using the Youtube ContentID system to pirate works that other people have uploaded [7]. Matt Taibbi s description of Goldman Sachs as a great vampire squid wrapped around the face of humanity, relentlessly jamming its blood funnel into anything that smells like money will never die [8]. It has spawned many other creative descriptions of the evil and greed of Goldman Sachs and even Lloyd Blankfein of Goldman Sachs describes his company as having burned down the Reichstag, shot the Archduke Ferdinand and fired on Fort Sumter he was trying to use satire, but I don t think that Goldman Sachs people would act differently to Fritz Thyssen. Keith Packard wrote an interesting article about the Calypso CalDAV system which he uses with Android [9]. He makes lots of good points about how to improve calendaring and contacts on Android, unfortunately I lack time to fiddle with such things at the moment so I ll stick with Google in spite of the risks. Asheesh Laroia wrote a great article about the problems with short (32bit) GPG keys [10]. It seems that creating keys with matching ID numbers isn t particularly difficult and that GPG doesn t handle them as well as we would like giving the possibility of at best annoying DoS attacks and at worse security problems due to using the wrong key. Sociological Images has an interesting article about when game show audiences are trustworthy [11]. It seems that French people don t want an undeserving person to win so they will intentionally advocate the wrong answer if the contestant should know it. Paul Wayper gave a great lecture titled SE Linux for Everyone [12]. He covers the basics of SE Linux in a user-friendly way and explains some simple solutions to common problems which don t involve compromising system security. Paul Tassi wrote an insightful article for Forbes about piracy [13]. His conclusion is that the media companies should make it cheaper and easier to be a customer and not spend insane amounts of money on low quality products. The Reid Report has an interesting article about Ron Paul s racism [14]. Ron Paul is generally well regarded outside the US because he wants the US government to stop meddling in the affairs of other countries, but while he s less bad than other US politicians in terms of foreign policy that doesn t make him a good person. Anonymous hacked some mailboxes belonging to a neo-Nazi group and found links to Ron Paul [15]. I ve always been suspicious of the way Ron Paul wanted to avoid anti-racism legislation on supposed Libertarian principles. The Reid Report has an interesting summary of Ron Paul news plus some criticism of Glenn Greenwald and others who associate with him [16]. Related posts:
  1. Links February 2011 Australia s Department of Finance has mandated that the MS-Office document...
  2. Links January 2012 Cops in Tennessee routinely steal cash from citizens [1]. They...
  3. Links February 2009 Michael Anissimov writes about the theft of computers from the...

MJ Ray: Food Co-ops in Bristol

A previous conferenceLast week I went along to the food co-ops networking event at the Southville Centre in Bristol. It was a useful event and very inspiring and informative to meet people from so many other co-ops, as well as attend some useful workshops: the two I went to were Good meetings and communication and Starting and developing a food co-op, while there were also ones on funding and Simply Legal available. There was some time for networking, as well as a relaxed end to the day which let me catch up with a few more people. I would have preferred a little more time for the workshops and a little less on case studies (every food co-op is different and I don t think any of the featured ones were quite what I was looking for), but that s a very minor thing and didn t really reduce the usefulness of the whole day. Our co-op is a tech worker co-op and not a food co-op, so I didn t know that much about how to start one before the event. Now I ve got a much better idea of what I need to do when I eventually move back out to what may be a co-op desert in King s Lynn. Are you a member of a food co-op or buying group? If so, what would you say about it? Were you involved in its start-up?

14 February 2012

Petter Reinholdtsen: How to figure out which RAID disk to replace when it fail

Once in a while my home server have disk problems. Thanks to Linux Software RAID, I have not lost data yet (but I was close this summer :). But once a disk is starting to behave funny, a practical problem present itself. How to get from the Linux device name (like /dev/sdd) to something that can be used to identify the disk when the computer is turned off? In my case I have SATA disks with a unique ID printed on the label. All I need is a way to figure out how to query the disk to get the ID out. After fumbling a bit, I found that hdparm -I will report the disk serial number, which is printed on the disk label. The following (almost) one-liner can be used to look up the ID of all the failed disks:
for d in $(cat /proc/mdstat  grep '(F)' tr ' ' "\n" grep '(F)' cut -d\[ -f1 sort -u);
do
    printf "Failed disk $d: "
    hdparm -I /dev/$d  grep 'Serial Num'
done
Putting it here to make sure I do not have to search for it the next time, and in case other find it useful. At the moment I have two failing disk. :(
Failed disk sdd1:       Serial Number:      WD-WCASJ1860823
Failed disk sdd2:       Serial Number:      WD-WCASJ1860823
Failed disk sde2:       Serial Number:      WD-WCASJ1840589
The last time I had failing disks, I added the serial number on labels I printed and stuck on the short sides of each disk, to be able to figure out which disk to take out of the box without having to remove each disk to look at the physical vendor label. The vendor label is at the top of the disk, which is hidden when the disks are mounted inside my box. I really wish the check_linux_raid Nagios plugin for checking Linux Software RAID in the nagios-plugins-standard debian package would look up this value automatically, as it would make the plugin a lot more useful when my disks fail. At the moment it only report a failure when there are no more spares left (it really should warn as soon as a disk is failing), and it do not tell me which disk(s) is failing when the RAID is running short on disks.

Next.

Previous.