I recently purchased a Seagate FreeAgent GoFlex 1 TB hard drive. It is a 2.5" compact disk that you connect over USB. It draws its power from the USB controller. My previous Extern HDDs were all 2.5" laptop drives, for which I bought a USB enclosure. Those devices always worked perfect, as in:

• They were automatically detected
• Partitions/File Systems understood
• Desktop systems would prompt for actions

The typical kernel messages I got for these devices were:

[56063.268107] usb 1-1: new high speed USB device number 13 using ehci_hcd

[56063.401635] usb 1-1: New USB device found, idVendor=14cd, idProduct=6116

[56063.401645] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=2

[56063.401652] usb 1-1: Product: USB 2.0 SATA BRIDGE

[56063.401658] usb 1-1: Manufacturer: Super Top

[56063.401663] usb 1-1: SerialNumber: M6116018VF16

[56063.402857] scsi8 : usb-storage 1-1:1.0

[56064.400896] scsi 8:0:0:0: Direct-Access WDC WD50 00BEVT-24A0RT0 PQ: 0 ANSI: 0

[56064.401576] sd 8:0:0:0: Attached scsi generic sg2 type 0

[56064.402102] sd 8:0:0:0: [sdb] 976773168 512-byte logical blocks: (500 GB/465 GiB)

[56064.402615] sd 8:0:0:0: [sdb] Write Protect is off

[56064.402618] sd 8:0:0:0: [sdb] Mode Sense: 03 00 00 00

[56064.403101] sd 8:0:0:0: [sdb] No Caching mode page present

[56064.403105] sd 8:0:0:0: [sdb] Assuming drive cache: write through

[56064.404861] sd 8:0:0:0: [sdb] No Caching mode page present

[56064.404864] sd 8:0:0:0: [sdb] Assuming drive cache: write through

[56064.419657] sdb: sdb1

[56064.421850] sd 8:0:0:0: [sdb] No Caching mode page present

[56064.421854] sd 8:0:0:0: [sdb] Assuming drive cache: write through

[56064.421857] sd 8:0:0:0: [sdb] Attached SCSI disk

But for my Seagate FreeAgent GoFlex HDD, things were different. The kernel would detect but then it would not mount. The error reported is that the device is busy. My FreeAgent's kernel messages look similar to what the regular one has:

[ 168.520140] usb 1-1: new high speed USB device number 5 using ehci_hcd

[ 168.657424] usb 1-1: New USB device found, idVendor=0bc2, idProduct=5021

[ 168.657433] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3

[ 168.657439] usb 1-1: Product: FreeAgent GoFlex

[ 168.657444] usb 1-1: Manufacturer: Seagate

[ 168.657449] usb 1-1: SerialNumber: NA0C1BML

[ 168.659079] scsi5 : usb-storage 1-1:1.0

[ 169.657136] scsi 5:0:0:0: Direct-Access Seagate FreeAgent GoFlex 0148 PQ: 0 ANSI: 4

[ 169.708786] sd 5:0:0:0: Attached scsi generic sg3 type 0

[ 169.709079] sd 5:0:0:0: [sdc] 1953525167 512-byte logical blocks: (1.00 TB/931 GiB)

[ 169.709954] sd 5:0:0:0: [sdc] Write Protect is off

[ 169.709963] sd 5:0:0:0: [sdc] Mode Sense: 1c 00 00 00

[ 169.710567] sd 5:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

[ 169.759942] sdc: sdc1

[ 169.762050] sd 5:0:0:0: [sdc] Attached SCSI disk

As the Device Mapper Multipath Debian Maintainer, I have multipath-tools installed on my laptop. Turns out, for some reason, the device is consumer by the device mapper multipath stack.

20:12:58 rrs@champaran:~$ ls /dev/mapper/

1Seagate@ 1Seagate FreeAgent GoFlex N 1Seagate FreeAgent GoFlex Np1 control LocalDisk-ROOT@ LocalDisk-SWAP@

Also the block device ID listing is enumerated as type SCSI.

19:56:44 rrs@champaran:~$ ls /dev/disk/by-id/

ata-HITACHI_HTS723216L9SA60_091220FC1220NCJASEVG@ scsi-SATA_HITACHI_HTS7232091220FC1220NCJASEVG@

ata-HITACHI_HTS723216L9SA60_091220FC1220NCJASEVG-part1@ scsi-SATA_HITACHI_HTS7232091220FC1220NCJASEVG-part1@

ata-HITACHI_HTS723216L9SA60_091220FC1220NCJASEVG-part2@ scsi-SATA_HITACHI_HTS7232091220FC1220NCJASEVG-part2@

ata-HITACHI_HTS723216L9SA60_091220FC1220NCJASEVG-part3@ scsi-SATA_HITACHI_HTS7232091220FC1220NCJASEVG-part3@

ata-HL-DT-ST_DVDRAM_GU10N_M189CNI1127@ scsi-SSeagate_FreeAgent_GoFle_NA0C1BML@

ata-ST1000LM010-9YH146_W1000ZD8@ usb-WDC_WD12_00BEVE-11UYT0_ST_Killer-0:0@

dm-name-1Seagate@ usb-WDC_WD12_00BEVE-11UYT0_ST_Killer-0:0-part1@

dm-name-LocalDisk-ROOT@ wwn-0x5000c5003d19c8c2@

dm-name-LocalDisk-SWAP@ wwn-0x5000cca586e112bc@

dm-uuid-LVM-buywwzKkpfKG2RegankA2nPkmFBBPFe3D5DepV8w8nLrHfoAjIIQVnakOQJZEqJX@ wwn-0x5000cca586e112bc-part1@

dm-uuid-LVM-buywwzKkpfKG2RegankA2nPkmFBBPFe3k9aJfc9B7wRmVIwfoagffHUZjuN6c4cM@ wwn-0x5000cca586e112bc-part2@

dm-uuid-mpath-1Seagate@ wwn-0x5000cca586e112bc-part3@

raid-1Seagate@

But I'm not sure why the same doesn't happend for my external laptop drive. It gets properly tagged as device type USB. So Laptop or Server, if you have a FreeAgent that you want to connect to your machine, and see the device busy error when accessing the device['s] directly, do the following:First, the ID of the device.

20:17:43 rrs@champaran:~$ /lib/udev/scsi_id --whitelisted --page=0x83 --device=/dev/sdc

1Seagate FreeAgent GoFlex N

Add the ID to /etc/multipath.conf under blacklist section

blacklist

wwid "1Seagate FreeAgent GoFlex*"

wwid "1Seagate FA GoFlex Desk*"

# devnode "^(ram raw loop fd md dm- sr scd st)[0-9]*"

# devnode "^hd[a-z][[0-9]*]"

# device

# vendor DEC.*

# product MSA[15]00

#

Run mulitpath -F to flush the unused maps. Run multipath -v3 to ensure that now the device is blacklisted.

Nov 03 20:19:02 sdc: (Seagate:FreeAgent GoFlex) wwid blacklisted

The front cause for the mis-behavior is:

# Coalesce multipath devices before multipathd is running (initramfs, early

# boot)

ACTION=="add change", SUBSYSTEM=="block", RUN+="/sbin/multipath -v0 /dev/$name"

But something else, maybe the default blacklist table, needs the actual fix.

The Seattle R User Group was kind enough to invite me to give a talk about R, C++ and Rcpp. So if you can make it to the Thomas building of the Fred Hutchinson Cancer Research Center in Seattle, WA, on December 7, I would love to see you there. I have some ideas about freshening up the presentation(s) based on material Romain and I have used in the past. This should make the why as well as how a little clearer; now I just have to find some to put this together. And if there are particular aspects you would like to see covered, please do get in touch with me.

A couple of times I have written code which basically does this:

1. accept() an incoming connection
2. do some magic to get another socket
3. start passing bit between the two sockets
4. perform a correct close down sequence

An example use case of this is a proxy implementing the HTTP CONNECT method, but in for some known hostnames it will log a message and mangle the hostname before proceeding. This has been used as a legacy fall back solution while changing a network setup. But my uses are not restricted to HTTP proxies I have done the same for a few legacy protocols where the magic has been of different complexities. The two final steps are quite general and it would be nice to have a module doing just that. Take two sockets and make it easy (or even automatic) to pass bytes between them. The na ve non-blocking solution would use a scalar string buffer for each direction and perform a select loop while maintaining the write vector depending on which buffers contain data. I have written this code multiple times. In development this is usually quite successful, in production less so. While Perl might be quite suited for the magic in step 2, the na ve way of passing bytes have quite an overhead for the buffer management. A less na ve way would use a array of strings for buffers, but I m not quite sure if this would be a win in all cases. You might be able to get away with some string operations on the read side of the buffer, but it might be more expensive on the write side. I have not benchmarked this. Most of the time I don t care about Perl level IO handles. I know that there is a real C level file descriptor beneath. So an even better POSIX compliant solution might be to use XS to have plain C strings and use readv()/writev() and a iovec structure as buffer. Can we do even better? At least on Linux we can. With the Linux splice() specific system call it is possible to us a pipe as buffer and never to have to copy data from and to user space. I have not been able to find any off the shelf solution on CPAN. So I think I need to write it myself, but what would the nice and general API be? I guess the basic interface would be something like:

    my $chain = IO::Splice->new($fh1, $fh2);
    $chain->pump(); # read and write from both handles if possible and needed
    $chain->read($fh1);  # read to buffer from one specific handle
    $chain->write($fh2); # write from buffer to one specific handle
    $chain->can_write(); # returns the handles it needs to write to

but it might be simpler to have two callbacks for setting a file handle in write or no-write state:

    my $readset  = IO::Select->new( $fh1, $fh2);
    my $writeset = IO::Select->new();
    my $chain = IO::Splice->new( $fh1, $fh2,
        writable => sub    $writeset->add( shift )  ,
        unwritable => sub   $writeset->remove( shift )  
    );
    while ( ... select ... )  
        $chain->pump();
     

As said, I think I have plenty of implementations of the na ve way but before releasing some code it would be nice to get some input on the API. But the best feedback would be a module that already have a usable API but might not implement the Linux specific way. That would allow me to steal the interface

No I don t really know any SQL. In fact, even at vocational school, where we focussed on database normalisation anyway, I tried hard to avoid the topic. Feel free to access here my entire knowledge about SQL (I did use Amaya, Arena and Arachne though. Liked only Arachne out of these three, and then, only under DOS, not its Unix version. Maybe the WWW could be named AAA instead? But then, lynx(1) is the one true browser ) Ah, well. While at it the entirety of my Perl knowledge is here: perltoc(1) with quick links to perlfunc(1). The entirety of my (X)HTML and ECMAscript knowledge, DE: SELFHTML; although, the spec and DTD helped; and to write my notes on JSON, I took a peek at the formal ECMAscript spec as well propos, does anyone know a (good enough) indent(1) equivalent for ECMAscript, as I am trying to strip down some, inherited (GPL, yes) code for a hobby project, but Geo-people seem to produce illegible code?

So, this story begins in January 2001, when I applied in the Debian New Maintainer Process. I wanted to maintain Geneweb, which I was using to publish my genealogy research results. And I wanted to keep it up-to-date while upstream was doing a quite fast development. Moreover, I quickly noticed that Geneweb had big trouble in respecting the way data is organized on a Debian system, and respect the FHS. How to have weveral users able to publish their data on the same server without compromizing the overall system security, etc. Upstream development didn't really care about that. Daniel de Rauglaudre is an excellent genealogist and developer but he was not interested in making Geneweb clean "the Debian way". For instance, at that time, it wasn't easy to setup a server, where genealogy databases could be published without having to manually launch a daemon in user mode at each reboot. Arranging this was indeed my first contribution. I contributed a few patches to make it easier to turn Geneweb into something FHS-compliant...and I developed init scripts and an organization allowing one to have shared databases after system reboots. That was rapidly a great introduction to Debian maintainer scripts and even security-related challenges. And all this....because I needed it. Basically, in 2001, the geneweb package adopted the organization it still has in Debian and Ubuntu, 10 years later:

• daemon optionnally launched at boot time, listening on a configurable port
• daemon running with an unprivileged account
• local group of users allowed to "publish" their data by putting files in /var/lib/geneweb (should now be moved to /srv, I guess)
• related maintenance of this with debconf (including keeping local modifications made *out* of debconf, which is sometimes neglected by maintainers)

While doing so, I was going through the New Maintainer Process. The Debian developer who had signed my key was Sam Tardieu, of of those longstanding "Freenix" dudes I was sometimes hanging around. I don't remember who did write my advocacy. Michael Mattice was my first Application Manager. Interestingly, here are the three first questions he asked my as part of the Policy and Procedures check :

• Firstly, can you explain the key points of the Social Contract and DFSG?
• Donald Knuth, author of TeX, insists that no-one has the right to modify the source code of TeX, and that any changes must be made using "change files" (a sort of patch file). Furthermore, any modification of TeX which produces something which fails the "trip test" (a regression test suite) may not be called TeX. So how come TeX (in the guise of tetex-*) is in main?
• What is Debian's (current) approach to non-free software? Why?
• Debian was offered a Debian-specific license to package a certain piece of software (I forget which). Would we put it in main?

Mike actually asked the questions on February 17th and I probably answered immediately. Unfortunately it seems that I lost my answers. Maybe they are in some archive somewhere. Would be interesting to see how I ended giving an answer about the TeX exception.:-) Immediately, Mike moved to the Tasks and Skills step and checked my geneweb and lifelines packages (I already prepared a package for that other genealogy software). From my records, he found a few lintian warnings, which I probably made my best to fix, then later on, on March 12th, he answered me that thigns were OK and he was marking the T&S step as passed...handing things over to the NM database, asking me what username I wanted on debian.org (obviously, you know what I answered to this!). Then nothing happened..:-) I was not that impatient, but finally, around June 27th, I politely mailed NM Frontdesk (I guess) to get news...and I got a very quick answer by Martin Michlmayr. Martin was indeed worried about the situation (he told me I was on his radar....fun as, many years later, as many DDs, Martin probably went on mine because of pending localization work..:-)) Finally, Martin decided he was not entirely happy with the (maybe too short) processing for P&P and proposed me (very politely, you know how Martin is!) to do it again. So, some more questions...:-) Here, I have the answers. Really fun ro read. Hang on, pure bubulle style, very verbose!

• Q: Do you know (and can you explain) the difference between free speech and free beer? Is Debian mainly about free speech or free beer? My answer: Free speech means that one is able to say what one wants, without any constraint. No one can tell you what you should say. Mozilla is free software. A free beer, on the other hand, is a beer that you don't be charged for. You may drink a free beer but maybe cannot be free to choose the beer you want to drink...:-). Internet Explorer is a free beer. By chance, we french-speaking people have two different words for this (libre and gratuit). Some people have even adopted "libre software " for being used instead of "free software".
• Q: The e-mail client pine is in non-free. Can you tell me the difference between main, contrib and non-free? My answer (typos and horrible English not corrected...): Packages in main are OK with Debian Free Software Guidelines (which defin what is considered "free" by Debian) and all other requirements of the Debian Policy. Packages in contrib are OK witgh DFSG bu may break some other constraints. For instance, they may be wrappers for non-free packages (the old netscape wrappers were good examples. The setiathome package is also another example as it requires the download of the setiathome client which is not free, because source code is not available). Very buggy packages may also be placed into contrib instead of main (a QA requirement). Pakages in non-free are not conform to DFSG. Simply speaking they are not "free software" as defined by DFSG. Some of them because their licence prohibits redistribution, or redistribution of derived work (the french package of Bernard Gaulle fot Latex could fall there).
• Q: Do you agree to uphold the Social Contract and the DFSG? Answer: Yes I do (wow! I'm back 16 years down to my wedding day...:-)) (Doh, already kidding with fellow DDs)
• Q: If you are accepted as a Debian developer, you will get accounts on the Debian machines. Have you read the Debian Machine Usage Policies (DMUP) at http://www.debian.org/devel/dmup ? Do you accept them? My answer: I did read the DMUP and I fully accept them.
• Q: What are Non-Maintainer Uploads (NMUs) and when would you do an NMU? Answer: This is an update for a package which is done by someone who is *not* the official package maintainer (I guess I give the correct english definition : I read the translated dev. reference....). First of all, a NMU should be done after trying to get in contact with the official package maintainer. I don't know if this is the awaited answer, but this point looks important to me. This is the only way for verifying that (s)he is not working on the same problem. NMU's are the most welcome during freeze phases when they correct bugs rated "severe" of higher. They are also welcome when they come from Debian Security Manager(s) for correcting security-related bugs. Another NMU source are those which are needed for ports to work. As a recent, and probably not very experienced package manager, I would probably be very prident with NMU's anyway...:-)
• Q: Tell me 3 different methods to close a bug in the BTS. Answer: Well, If I have properly understood Dev. Reference, there are two recommended methods and a third which is discouraged :
  1. properly use the debian/changelog file (by adding "Closes: Bug#xxxxx" statements in it). The bug will automagically be closed by the archive maintenance software
  2. by mail by sending the .changes file to xxxxx-done@bugs.debian.org, xxxxx being the bug number
  3. by sending a "close" command to control@bugs.debian.org
  The latter should never be used, says Dev. Reference. I must admit that I don't know why it is still mentioned. There is certainly a good reason. I'm not aware of another method...
• Q: What would you do if a bug was reported against your package and you are not able to fix it yourself? Answer: First, I would double check with upstream author(s) if this cannot be fixed by him/her/them.... Then, I would probably ask for help and/or advice in debian-qa if the bug is critical, grave or serious. If it is normal, I would ask on debian-devel or maybe think more about possible solutions.... If the bug still cannot be fixed and is "severe" or higher, I would either consider orphaning it (not being able to fix it may mean that I do not have the skills for maintaining it properly) or having it adopted by someone else. I don't think there is a unique answer there. The key point for me is certainly "ask for help in debian-qa". (OK, not the best answer ever...)
• Q: You've just heard about this great program and would like to package > it, what would you do? Answer: First, check on the WNPP page. The purpose here is avoiding having two people working on the same package. Then, assuming no on,e works on it, open a bug against wnpp speudo-package with a copy to debian-devel. The "bug" should include a short description of the package and what I intend to do for packaging. The URL for downloading sources for the program should be given. The subject of the bug report is normalized. The bug report should be of the "wishlist" class. When the new package is ready and uploaded for the first time, it should include the appropriate statement for closing the wnpp bug.
• Q: Do you know what 'lintian' is? Why is it useful? Answer: It is a set of tools for making automatic checks of packages. It may detect some bugs, common errors, Debian Policy violations and so on. This is a very useful tools for polishing packages. For instance, in my "geneweb" unofficial package, the last remaining lintian report is about the creation of a /var/geneweb directory which violates Debian Policy.
• Q: What does version 3.4-2.1 mean? What Debian control file would you put this in? (Hint: NMU). Answer: This is the Debian package of version 3.4 of program "foo". The current package version in the distribution was -2 and someone made a NMU of this package. This would be put in debian/changelog
• Q: You have a package in contrib, why would it have to go there? What could you do (in theory at least) to get it into main? Answer: The package in consistent with DFSG but other constraints are not respected. The most probable reason is being dependent of "non-free" packages (there may be other reasons but none comes to my mind at this time....). In that case, I should try to make it independent of these packages. For instance, trying to replace a "Depends: netscape-base" line by a "Depends: www-browser" line...if it may be done, of course!
• Q: If you had a file in your package which usualy gets changed by a user for local settings, how doyou make sure your next version of the package doesn't overwrite it? Answer: By putting its name into the debian/conffiles line. By the way, I guess you're meaning "changed by a local system administrator for local settings" here, don't you? (hey, answering a question with another one, good trick, man...)
• Q: What would you do if you wanted to retire from the project and let other developers maintainer your packages? Answer: The correct term is "orphan" the package. Make a bug report against wnpp (with a normalised subject starting with O, see Dev. Ref). The bug severity is "normal". If this is an important package for the distribution, the bug should be of "important" severity and the bug report should have a subject beginning with RFA (Request For Azdoption). In that case, an announce in debian-devel is also asked. In both cases, a new package should be uploaded with Debian QA Group as maintainer.

What a bunch of questions..:-). Reading my answers ten years after, I see that some answers are somehow naive but, after all, they're not so bad. I really remember spending hours in writing them (my English was somehow more shaky at that time) The last question was: "Please tell me about about yourself and what you intend to do for Debian". Here, Martin opened a big can of worms..:-) So, you'll learn about this in the next episode..:-)

As I wrote in part 1, my family and I were in Mexico recently. Today I ll write about the places we stayed. We spent most of the time in a room we rented in a private home in Guadalajara. My friend Jonathan had found it for us, and it was not too far from his home. The owner was a grandmother, and across the courtyard was more family, including a granddaughter close in age to our boys. They enjoyed playing together. It was really a perfect arrangement for us. There s no better way to be a part of local life when traveling than to stay in someone s house. Here s our bedroom: Notice the glass slats in the window it s a nifty, though not airtight, alternative to our regular windows. More on that later. More of the inside: We had a bit of a language barrier while in Mexico, though never anything significant. My Spanish vocabulary started with almost nothing and I reached maybe a few dozen words by the time we left. Terah knew some Spanish from high school and college, and my friend was fluent. Our hostess also knew a little English. But we all communicated well enough. Terah or Jonathan would help translate when needed. As I d seen before, the children would say things to each other, but never seemed to be bothered that their playmates didn t understand what was being said. They just had a great time anyhow. On Sunday afternoon, when we came back from our activities, there was a buzz of activity. Children everywhere outside, running and playing. Adults too, chatting. We didn t know exactly what was happening, but sent Jacob and Oliver out to play anyhow (which they were eager to do). The yard was enclosed by a wall, so children could pretty much run around without lots of supervision. Eventually they were invited to have some birthday cake (ah ha!) it was one of the children s birthday. Jacob and Oliver actually were served the first two pieces of cake (as the amigos ). Everyone seemed so friendly, warm, and welcoming. Each morning started with breakfast at the house, followed by a scene like this: That s Jacob and Oliver, looking to see if Jonathan had arrived for the day yet. I often noticed in Mexico that I was unsure if I was inside or outside. Here in Kansas, we can have a string of summer days that each exceed 110F (44C) or a few weeks in winter that never get above 15F (-9C), even in daytime. And then we have some pleasant days like right now, too or rain blowing sideways at 60MPH. In general, we spend a lot of effort keeping the outside, well, out. It is quite clear that this isn t a problem in the Guadalajara area. Some restaurants could have been described as buildings with large, open windows so you feel a lot of breeze while inside. Or perhaps as a simple shade roof with a few supports on the edges, so you re never really inside at all (sort of like going under a small shade tent outdoors). To the extent that windows could close, many of them couldn t be made airtight. It was clear that in that area, people spend more energy finding ways to invite the outdoors in rather than to keep it out, thanks to the year-round moderate climate. Perhaps the best example of this surprised me one evening. We were arriving in Guanajuato, an old silver mining town in the mountains, and were going to spend the night at Hotel Socav n, which had been recommended to us by a local friend of Jonathan s. From the street, the hotel looked tiny. But walk in, and you get in this old-looking (and feeling) entry tunnel: That s the front desk, apparently cut out of it there on the right. We asked to see some of the rooms before buying apparently a normal request around there. The innkeeper agreed, and gave us keys and directions to find them on the third floor. It included going up 2 flights of stairs, passing through a courtyard, and going up another flight. It was after dark, and the hotel was dimly lit something I was fine with. I thought we were stepping out into a beautiful atrium with some potted plants in the center of the building something fairly common in some nicer hotels. Until I felt rain on my head. Then I realized that the courtyard, which began two floors up from the street, was open to the sky. Beautiful! Here was the view from out room door: And down the hall : After getting home, a Google happened to turn up some reviews of this hotel. I was so annoyed at what some people wrote! One person gave them only 3 stars because they didn t have air conditioning, had poor water pressure, and lots of steps . Someone else complained of the dark entry tunnel something I couldn t help but smiling about the moment I entered. My review, which should hopefully get posted soon, is certainly different. I gave them 5 stars, because if I wanted a Super 8 with generic fluorescent lighting and the same layout as thousands of other hotels, I would have gone to Nebraska instead of Mexico. Most homes and local hotels in the region don t have air conditioning because they don t need it, and that s just how water pressure is in Mexico (due to needing to pump it from municipal supplies to private storage tanks overhead). And who doesn t appreciate entering a hotel through a brick tunnel? Ah, sigh This should give you some idea of the kind of travel we like: part of the point of traveling is enjoying the differences from home, and I think it is a huge mistake to be annoyed at everything that is different. Enjoy the differences! Finally, here s a photo of the staircase in the home we stayed in, which I thought was fascinating:

Finally I managed to write up a transition statement for my not so new, but stronger GnuPG key. See below:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1,SHA256
I am transitioning my GPG key from an old 1024-bit key to a new 4096-bit key.
The old key will continue to be valid for some time, but I prefer all new
correspondence to be encrypted for the new key, and will be making all
signatures going forward with the new key.
If you have signed my old key, I would appreciate signatures on my new key as
well, provided that your signing policy permits that without reauthenticating
me.
The old key, which I am transitioning away from, is:
pub   1024D/99E141B4 2004-02-10
      Key fingerprint = 25FE 4741 4770 0558 949D  1DB1 58DD 3FE2 99E1 41B4
The new key, to which I am transitioning, is:
pub   4096R/51B85139 2009-06-18
      Key fingerprint = A3EB B41F C5AB D675 CEE4  1C45 EA6C A6B9 51B8 5139
Thanks in advance.
Cheers,
Micha Lenk
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
iEYEARECAAYFAk5aOnkACgkQWN0/4pnhQbTxPgCgzRhREZlQiKJyI9UIdJLLs3Zq
bH4AnA1myFxgDWM7aUMHXgvvsujLTjiWiQIcBAEBCAAGBQJOWjp5AAoJEOpsprlR
uFE5gCsP/0dtCUPl9aQHV1MbQl7+bMofpsC2ikkpdZmrzi68jTG16We49BuzY+PV
S8FhXqg17/YxhKYDnDNTowfztUOyjAOJxy5vrqm3X5xiLwTqN3js9mra+vb4s35k
EVbKMzLLDhj3i0FjeargWEmJmm9cVhaZWKvOvQUhDJAilqvEQ0/50P7B8I+1YvtV
UHoQKbweTljVSlK5R1YPPy9i6r2/oZBYxK4nrknWwS+qPQ5luqelJd+mZdgQ6tow
7HIvtmPgCblJ+hYZWFpoZK6vxs8RaBbuCQcKwYArNhZT/v4TeD/LAaUmIkbQMyKV
J2TKuEHya4+5GMbtg6BGKeiZpleEHPnAq1AfvGpz6opkxjxCLG3RO3X8D3EuM3RW
mkq60mWM8+Zwu1yKbb62iHplp6jpyiQgdjJlB6eHjX7SdY7CvHgYZxGDx4kclP0/
HAdig2U1T+nG6Nn5XflmmKwvNLuKlQlIwJ5NeXyCONRnYvdomQn2hgvkjwMLCdFh
ulIhxa4UvDY7/aQNPZeOrvDHb2XYpiV3TwA9hLgQXXWd0FPmUMVVPpKpRjilaWth
Mtq/QiMGP5Mq/YxgLInRZHGyajDtE67RD4+RgHYOP50cP3UGPoB4ncc2EEtM2kfe
BzJrXPHmdtyGEA6Korl0YwUTRnYsqkqkY1VqDsO0UkOLlV7RAzb9
=9vw1
-----END PGP SIGNATURE-----

I already got mails like What a pity that your nice blog posting series ended . No, it didn t end. As announced, I knew that I won t be able to keep up a daily schedule. It worked as long as I had already written the postings in advanced. But in the end the last postings were already written just in time and then I ran out of leisure and muse for a time. But as I said: It didn t end, it will be continued. And this is the next such posting. Oh, and for those who tell me further tools, I should blog about: I appreciate that, especially because that way I also hear about tools I didn t know about. But why just telling me and not blogging yourself about it? :-) At least those whose blog is part of Planet Debian or Planet Symlink anyway really should do this themselves. I d really like to see also others writing about cool tools. I neither have a right on the idea nor on the name of this series (call it meme if you want :-), so please go on and publish your favourite tools in a blog posting, too. :-) And for all those who want to join me and Myon blogging about cool Unix tools, independent if listed on Planet Debian or Planet Symlink, I encourage you to offer a separate feed for this kind of postings and join us on Planet Commandline. Anyway, here s the next such posting: As system administrator you often have the case that you have to kill all processes of one user, e.g. if a daemon didn t properly shut down itself or amok running leftovers of a GUI session. Many use pkill -SIGNAL -u user from the procps package or killall -SIGNAL -u user from the psmisc package for it. But that s a) quite cumbersome to type and b) is there a chance to forget about the -u and then bad things may happen, especially with pkill s default substring match, so I prefer another tool with a more explicit name:

slay slay has an easy to remember name (at least for BOFHs ;-) which is even quicker to type (alternating one character with the left and the right hand, at least on US layout keyboards) than pkill (all characters to type with the right hand), and has the same easy to remember commandline syntax like kill itself:

slay -SIGNAL user [user  ]

But beware, slay is

not only for BOFHs, but also from a BOFH It has a mean mode which is activated by default. With mean mode on, it won t kill the given user but the user who called the program if it is invoked as an ordinary user without root rights. *g* Interestingly I never ran into this issue despite I use this program often and for many years now. But some Ubuntu users did, probably because adding a sudo in front of some command is easier to forget than doing an ssh root@localhost or su - beforehand. They even seem to be so desperate about it that they forwarded the issue from Launchpad to the Debian Bug Tracking System. ;-) But to be honest even if I was very amused about those bug reports isn t this issue grave , as it causes very likely (unexpected) data loss?

Now playing: Monzy kill dash nine ( and your process is mine ;-)

Todays addition to the tips series has a funny history. A friend of mine was stuck in an envrionment, where she couldn't install Perl or CLive but PHP with some PECL modules was installed. She needed to download a few videos for a presentation, but didn't know how to fetch the videos from YouTube. So I was asked to help. ;-) After some pondering of the problem, I came up with the following script:

<?php
/*
 * Copyright (C) 2011  Kai Wasserb ch <kai@dev.carbon-project.org>
 *
 * Small little script to help out in the absence of Perl/Clive but with
 * PHP present.
 *
 * License: GPLv3 <http://www.gnu.org/licenses/gpl-3.0.txt>
 *
 */
if(php_sapi_name() !== 'cli')
  die("This script needs to be run on the CLI!\n");
if($argc !== 2 )
  die("No URL given on CLI or too much parameters!\n");
// Script expects a URL like http://www.youtube.com/watch?v=[ID]
if(filter_var($argv[1], FILTER_VALIDATE_URL, FILTER_FLAG_QUERY_REQUIRED) === false)
  die("Parameter given as URL does not seem to be an URL or contain a query!\n");
if(!$ch = curl_init($argv[1]))
  die("Couldn't init cURL handle for  ".$argv[1]." \n");
curl_setopt($ch, CURLOPT_HEADER,          false );
curl_setopt($ch, CURLOPT_RETURNTRANSFER,  true  );
$htmlsource = curl_exec($ch);
$error = curl_error($ch);
curl_close($ch);
if($htmlsource === false)
  die('cURL returned an error: '.$error."\n");
unset($error);
if(!preg_match('/url_encoded_fmt_stream_map=([a-zA-Z0-9%\.,\-_]+)&/',
               $htmlsource, $matches))
  die("No matches found for url_encoded_fmt_stream_map!\n");
$final = NULL;
foreach(explode(',', urldecode($matches[1])) as $decoded_url_map)  
  parse_str($decoded_url_map, $tmp);
  if($tmp['quality'] === 'hd1080')  
    $final = $tmp;
    $final['url']   = urldecode($tmp['url']);
    // $final['type']  = urldecode($tmp['type']); // Not used
    break;  // Not really needed, but should save us some
            // iterations, since 1080p seems to come always first.
   
 
if(!is_array($final)   is_null($final))
  die("No 1080p version of the video was found!\n");
if(!preg_match('/<meta property="og:title" content="(.*)"/', $htmlsource, $title))
  die("Couldn't determine title!\n");
$ftitle = preg_replace('/[^\w\d]*/', '', $title[1]);
printf("Found hd1080-quality video for  %s .\n", $argv[1]);
printf("Downloading to %s.mp4 ...", $ftitle);
if(!$ch=curl_init($final['url']))
  die("\nCouldn't create cURL handle for  ".$final['url']." \n");
if(!$fh = fopen(getcwd().DIRECTORY_SEPARATOR.$ftitle.'.mp4', 'wb'))  
  curl_close($ch);
  die("\nCouldn't create  ".$ftitle.".mp4 .\n");
 
curl_setopt($ch, CURLOPT_HEADER,         false );
curl_setopt($ch, CURLOPT_FILE,           $fh   );
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true  );
if(!curl_exec($ch))  
  $error = curl_error($ch);
  fclose($fh);
  curl_close($ch);
  die("\nCouldn't download video. cURL error: ".$error."\n");
 
curl_close($ch);
fclose($fh);
printf(" done.\n");
?>

You can download it from my website or you can just copy the code given above into a file. However you get the code, the execution will look like: php -f /path/to/script.php http://www.youtube.com/watch?v=[ID]. Now, before I get a lot of e-mails telling me, what could be improved on the script or its other limitations (like only being able to download 1080p videos), please remember that it was hacked together for a simple use case with no real intent on my part to use it for anything else. It only later occurred to me, that others might be in a similar pinch sometime. Thus I'm releasing the above code "as is" under the GNU GPLv3. Therefore you're free to adapt it to your needs. Please also note, that I'm not recommending using the above script instead of CLive. If you can install CLive, do so and don't bother with the above.

For the past few years, the Linux Containers project has been developing some kernel technologies which provide isolation, accounting and management for kernel resources. When completed, together they will allow implementing lightweight virtualization of the operating system like is currently possible with OpenVZ. It is very important to understand that not all the pieces are ready yet, so currently a LXC container does not provide the same security of other virtualization technologies. This example shows how the root user of a container can easily run code as root in the host system, since sysfs does not support namespaces:

lxc$ cat <<END > /tmp/evil-helper
#!/bin/sh
echo 'hi!' >> /tmp/evil-helper.log
END
lxc$ chmod +x /tmp/evil-helper
lxc# mkdir /sys
lxc# mount -t sysfs sysfs /sys
lxc# echo /var/lib/lxc/test/rootfs/tmp/evil-helper > /sys/kernel/uevent_helper
lxc# echo change > /sys/class/mem/null/uevent

The last command will trigger a uevent, which will be processed by running in the host our script created inside the container. Or else:

inside# echo /sbin/poweroff > /sys/kernel/uevent_helper

Because of this, LXC containers are vulnerable to a malicious root user and are not really a replacement for OpenVZ. This will change when support for labelling files with user namespaces will be completed.

Many people still install Linux from CDs. But a growing number install from USB. In an ideal world you'd be able to download one image that would let you do either, but it turns out that that's quite difficult. Shockingly enough, it's another situation where the system firmware exists to make your life difficult.

Booting a hard drive is pretty easy. The BIOS reads the first 512 bytes off the drive, copies them to RAM and executes them. That code is then responsible for either starting your bootloader or identifying the currently active partition and jumping to its boot sector, but before too long you're in a happy place where you're executing whatever you want to. Life is good. So you'd think that CDs would work in a similar way. The ISO 9660 format even leaves a whole 32KB at the start of a filesystem, which is enough space for a pretty awesome bootloader. But no. This is not how CDs work. That would be far too easy.

Let's imagine we're back in the 90s. People want to be able to boot off CD without needing a boot floppy to do so. And you're a PC vendor with a BIOS that's been lovingly[1] forced into a tiny piece of flash and which has to execute out of an almost as tiny piece of RAM if you want your users to be able to play any games. Letting boot code read arbitrary content off the CD would mean adding a new set of interrupt hooks, and that's going to be even more complicated because CDs have a sector size of 2K while hard drives are 512 bytes[2] and who's going to pay to implement this and for the extra flash and RAM and look surely there has to be another way?

So, of course, another way was found. The El Torito specification defines a way for shoving a reference to some linear blocks into the ISO 9660 header. The BIOS reads those blocks into memory and then redirects either the floppy or hard drive access interrupts (depending on the El Torito type) to that region. The boot code can then proceed as if it had been read off a floppy without all the trouble of actually putting a floppy in the machine, and the extra code required in the system BIOS is minimal.

USB sticks, however, are treated as hard drives. The BIOS won't look for El Torito images on them. Instead, it'll try to execute a boot sector. That isn't there on a CD image. Sigh.

A few years ago a piece of software called isohybrid popped up and solved this problem nicely. isohybrid is a companion to isolinux, which itself is a bootloader that fits into an El Torito image and can then load your kernel and installer from CD. isohybrid takes an ISO image, adds an x86 boot sector and partition table and does some more fiddling to turn a valid ISO image into one that can be copied directly onto a USB stick and booted. The world suddenly becomes a better place.

But that's BIOS. EFI makes this easier, right? Right?

No. EFI does not make this easier.

Despite EFI being a modern firmware for the modern world[3], EFI implementations are not required to be able to understand ISO 9660. In fact, I've never seen one that does. FAT is all the spec requires, and FAT is typically all you get. Nor will EFI just execute some arbitrary boot code from the start of the CD. So, how does EFI boot off CD?

El Torito. Obviously.

It's not quite as bad as it sounds, merely almost as bad as it sounds. While the typical way of using El Torito for a long time was to use floppy or hard drive emulation, it also supports a "No emulation" mode. It also supports setting a type flag for your media, which means you can distinguish between images intended for BIOS booting and EFI booting. But the fact remains that your CD has to include an embedded FAT partition that then contains a bootloader that's able to read ISO 9660 because your firmware is too inept to handle that itself[4].

How about USB sticks? Thankfully, booting these on EFI doesn't require any boot sectors at all. Instead you just have to have a partition table, a FAT partition and a bootloader in a well known location in that FAT partition. The required partition is, in fact, identical to the one you need in an El Torito image. And so this is where we start introducing some extra hacks.

Like I said earlier, isohybrid fakes up an MBR and adds some boot code that points at the actual bootloader. It needs to do a little more on EFI. The first problem is that the isohybrid MBR partition has to cover the entire ISO 9660 filesystem on the USB stick so that the operating system can access it later, but the El Torito FAT image is inside that partition. A lot of MBR-based code becomes very unhappy if you try to set up a partition that's a subset of another partition. So we can't really use MBR. On to GPT.

GPT, or the GUID Partition Table, is the EFI era's replacement for MBR partitions. It has two main advantages over MBR - firstly it can cover partitions larger than 2TB without having to increase sector size, and secondly it doesn't have the primary/logical partition horror that still makes MBR more difficult than it has any right to be. The format is pretty simple - you have a header block 1 logical block into the media (so 512 bytes on a typical USB stick), and then a pointer to a list of partitions. There's then a secondary table one block from the end of the disk, which points at another list of partitions. Both blocks have multiple CRCs that guarantee that neither the header nor the partition list have been corrupted. It turns out to be a relatively straightforward modification of isohybrid to get it to look for a secondary EFI image and construct a GPT entry pointing at it. This works surprisingly well, and media prepared this way will boot EFI machines if burned to a CD or written to a USB stick.

There's a few quirks. Macs will show two boot icons for these CDs[6], one marked "EFI Boot" and one helpfully marked "Windows"[7], with the latter booting the BIOS El Torito image. That's a little irritating, but not insurmountable. The other issue is that older Macs won't look for boot loaders in the legacy locations. This is where things start getting horrible.

Back in the old days, Apple boot media used to have a special "blessed" folder. Attempting to boot would involve the firmware looking for such a folder and then using that to start itself up. Any folder in the filesystem could be blessed. Modern hardware doesn't use boot folders, but does use boot files. For an HFS+ filesystem, the inode of the bootloader is written to a specific offset in the filesystem superblock and the firmware simply finds that inode and executes it. And this appears to be all that older Macs support.

So, having written a small tool to bless an HFS+ partition, I tried the obvious first step of burning a CD with three El Torito images (one BIOS, one FAT, one HFS+). It failed. While Refit could see the bootloader in the HFS+ image, the firmware appeared to have no interest at all in booting off it. Yet Apple install media would boot. What was the difference?

The difference, obviously, was that these earlier Macs don't appear to support El Torito booting. The Apple install media contained an Apple partition map.

The Apple partition map (APM) is Apple's legacy partition table format. Apple mostly dropped it when they went to x86, where it's retained for two purposes. The first is for drives that need to be shared between Intel Macs and PPC ones. The second seems to be for their install DVDs. Some further playing revealed that burning a CD with an APM entry pointing at the HFS+ filesystem on the CD gave me a boot icon. Problem solved?

Not really. Remember how I earlier mentioned that ISO 9660 leaves 32KB at the start of the image, and that an isohybrid image then writes an MBR and boot sector in the first 512 bytes of that, and the GPT header starts 512 bytes into a drive? That means that it's easy to produce an ISO that has both a boot sector, MBR partition table and GPT. None of them overlap. APM, on the other hand, has a header that's located at byte 0 of the media, overlapping with the boot sector. And it has a partition listing that's located at sector 1, overlapping with the GPT. Is all lost?

No. Merely sanity.

The first thing to remember is that the boot sector is just raw assembler. It's a byte stream that's executed by the CPU. And there's a lot of things you can tell a CPU to do that result in nothing happening. Peter Jones pointed out that the only bits of the AFP header you actually need are the letters "ER", followed by the sector size as a two byte big endian integer. These disassemble to harmless instructions, so we can simply move the boot code down a little and stick these at the beginning. A PC that executes it will read straight through the bizarre (but harmless) Apple bytes and then execute the real boot code.

The second thing that's important here is that we were just given the opportunity to specify the sector size. The GPT is only relevant when the image is written to a USB stick, so assumes a sector size of 512 bytes. So when the GPT starts one sector into the drive, it's actually starting 512 bytes into the drive. APM also starts one sector into the drive, but we can simply put a different sector size into the header and suddenly we're able to choose where that's going to be. 2K seems like a good choice, and so the firmware will now look for the header at byte 2048.

That's still in them middle of the GPT partition listing, though. Except we can avoid that as well. GPT lets you specify where the partition listing starts and doesn't require it to be immediately after the header. So we can offset the partition listing to, say, byte 8192 and leave a hole for the Apple partition map.

And, shockingly, this works. Setting up a CD this way gives a boot icon on old Macs. On new Macs, it gives three - one for legacy boot, one for EFI boot via FAT and one for EFI boot via HFS. Less than ideal, but eh. The one remaining problem is that this doesn't work for USB sticks (the firmware sees the GPT and ignores the APM), so we also need to add a GPT entry for the HFS+ partition. Job done.

So, it is possible to produce install media that will work if burned to CD or written to a USB stick. It's even possible to produce a version that will work on Macs, as long as you're willing to put up with three partition tables and an x86 boot sector that doubles as an APM header. And patches to isohybrid to do all of this will be turning up as soon as I tidy the code to the point where it works without having to hack in offsets by hand.

[1] Insert some other adverb here if you feel like it
[2] Why yes, 15 years later BIOSes still tend to assume 512 bytes. Which is why your 4K sector disk is much harder to work with than you'd like it to be.
[3] Ever noticed how the modern world involves a great deal of suffering, misery and death? EFI fits into that world perfectly.
[4] Obviously if you want your media to be bootable via both BIOS and EFI you need to produce a CD with two El Torito images. BIOS systems should ignore the image that says it's for EFI, and EFI systems should ignore the BIOS one. Some especially creative BIOS authors[5] have decided that users shouldn't have their choices limited in such a way, and so pop up a screen that says:

1.
2.
Select CD-ROM boot type:

and wait for the user to press a key. The lack of labels after the numbers is not a typographical error on my part.
[5] Older (pre-2009, and some 2009 models) Apple hardware has this bug if a dual-El Torito CD is booted via the BIOS compatibility layer. This is especially unfortunate because said machines often fail to provide a working keyboard emulation at this stage, resulting in you being stuck forever at an impressively unhelpful screen. This isn't a Linux bug, since it's happening before we've run any of our code at all. It's not even limited to Linux. 64-bit install media for Vista SP1, Windows 7 and Server 2008 all have similar El Torito layout and all trigger the same bug on Apple hardware. Apple's aware of this, and has resolved the issue by declaring that these machines don't support 64 bit Windows.
[6] Even further investigation reveals that Apple will show you as many icons as there are El Torito images, which is a rare example of Apple giving the user the freedom to brutally butcher their extremities if they so desire
[7] "Windows" is Apple code for "Booting via BIOS compatibility". The Apple boot menu will call any filesystem with a BIOS boot sector Windows.

It is half-time for me at DebCamp/DebConf11 in Banja Luka, Bosnia-Herzegovina, and that is a good time to write about my work so far. I started with some finger exercises, such as the HTML copying for gnome-terminal (as blogged previously) and cosmetic changes to the buildd.debian.org/status website. After that, I began my main project: A SAT-based solver for the testing transition problem. In Debian, new software first enters a repository called unstable, where it is tested. Once some requirements are fulfilled (package has been in unstable for usually at least 10 days and has no new release critical bugs), it is entitled to enter the repository called testing, which will eventually form a new stable Debian release. But some packages need to migrate together, usually because either requires the new version of the other. Also, no packages in testing ought to become uninstallable by some seemingly unrelated change. The software that decides these things is called britney. It is sufficiently good in making sure nothing bad happens, but not smart enough to figure out what to do in case more than two packages need to migrate simultaneously. My goal is to improve this. Now, the the various requirements can all be expressed as predicative formulas, and a lot of research has been going into writing good solvers for such problems, called SAT-solvers. Hence my plan was to only specify what we expect from a testing migration, but leave the search for a solution to such a general purpose and highly optimized program. My initial progress was good and I had some result after two days of hacking, and the approach is promising. Because the data sets are pretty large (1.5GB of input data, the final SAT problem has 1.8 million clauses with 250.000 variables), so I learned quite a bit about profiling and optimizing Haskell programs, and that parsec is slower than working with BS.lines and BS.split to parse simple input. I also used the FFI to use dpkg code directly to compare Debian version numbers. At some point I noticed that I actually want to solve PMAX-SAT problems: Given two set of clauses, hard clauses and soft clauses, find a variable assignment that fulfills the hard clauses and as many of the soft clauses as possible. Unfortunately, there are no fast Free PMAX-SAT solvers around. The ones that I found and that were fast enough that I could use them, msuncore by Joao Marques-Silva at the University College Dublin and MiniMaxSat by Federico Heras at the Universitat Polit cnica de Catalunya, are only available as statically linked binaries. It is a shame that this is acceptable by the academic community; imagine mathematicians would stop including the proofs in their papers, and only share their theorems. So if you happen to have written a PMAX-SAT solver and can solve this instance (in weighted DIMACS format) in less than five minutes, and want to brag that the Debian project is using your code, then please release it under a Free license (e.g. GPL or BSD) and tell me about it! The code of my SAT-based solver is, of course, available, though slightly unpolished. Parts of it (notably the interface to picosat and the PMAX-SAT solvers) might be of general interest and I plan to put them on Hackage as a separate license. Update: I have contacted the authors of the SAT solvers mentioned above, and they have reconfirmed that they have no intention of releasing the source. Now I put my hope in maxsatz2009 by Chumin LI, which is GPL but was not able to cope with my large instance directly; I think I need to change the memory management.

Flattr this post

Jacob has been eagerly awaiting this day for at least, well, 2 days now. This morning he started asking questions. All day long he has been wondering how long until the big event. And that event is the arrival of the UPS truck carrying a special $8 package for him. And in that $8 package is not a DVD, or a book, or a toy, but rather 12 rolls of adding machine tape (totaling 1/3 of a mile). Oh yes, our four-year-old can hardly contain his excitement over office supplies. Let me back up a bit to explain. I am always on the lookout for a certain kind of toy. The sort of thing that you don t find in the toy section of a store. The sort of thing that is interesting and engages the curiosity of our boys, that most people would consider would toss, and that usually costs less than $5. Along those lines, Jacob has an old manual typewriter he can use, he and I built a switchbox full of switches and light bulbs that the boys love to play with, we built him a command-line-only PC out of spare parts, the boys have some radios they can use, and one day I salvaged a decades-old adding machine that was going to be thrown out. Jacob liked it he enjoys number and letter games and had some fun, but didn t zealously engage with it until just a few days ago. He sat down at his table and started making cards . He will print out a bunch of numbers, making a strip a couple of feet long, and then say, Dad, I made another card for you! We will then happily accept the card and he will continue making more. Here he is, holding up one of his cards: Jacob had a ton of fun making them, and Oliver wanted in on the action too. Oliver was somewhat interested in the adding machine, but not as much as Jacob, and Jacob didn t want to share. So somehow they decided that Oliver would run and get cards from Jacob and then take them to whatever parent Jacob said. Today Jacob said, Dad, the Oliver delivery service will bring you a card! Saturday morning we had some time to hang around and play, so we played some adding machine tape games. I tucked some long cards under my waistband, and the boys happily chased me all over the house trying to get them back. We also decorated: And here is what the room looked like after some cleanup: That orange bucket on the desk is my card bucket where I can put all the cards he gives me. He even helpfully empties it out into the recycle bin when there are too many old cards in it! But then tragedy struck it was out of paper! I ordered a 12-roll pack off Amazon, and boy was it a long couple of days until it arrived. Today before the UPS truck got here, I showed him how we could feed old paper through and print on the back side an incredible innovation, it seemed! Terah had a conversation with him earlier that went like this: Terah: Jacob, we need to clean up the house. Jacob: Why? Terah: Because some people are coming over. Jacob: Who is OH YES! The UPS delivery man! Yes, let s clean! Terah: Ah, well yes he will be here too, but your great uncle and aunt are coming. Jacob: Oh yay! And the UPS man! It is hard to describe Jacob s excitement when the package arrived. I quickly opened it up, and at his great uncle s prompting, he counted to see if he really did get 12 rolls of paper. IT IS TWELVE ROLLS! YAY! And then we quickly installed it in the machine and off he went. At his great uncle s prompting, he started writing a J on each card . This meant Jacob , or maybe he d put two J s, meaning To John From Jacob , or perhaps the two J s meant To Terah From Jacob he d always have to clue us in. Anyhow, for an $8 investment in paper this has brought hours of joy to our boys. Even though Jacob isn t yet really using it to perform arithmetic, it is setting him up to enjoy that sort of thing and encouraging his curiosity. Both good things.

If a single man can display so much hatred
think only of how much love we all can display together.

Stine Renate H heim

I wrote a Norwegian post explaining my experience at Ut ya. I had taken this blog for dead, and had entirely forgotten that it was syndicated on Planet Debian. I don t want to let Google Translate make this disaster any worse than it is the translation of bullets into balls being particularly bad so the international attention the massacre has garnered in consideration, I am writing an English translation of my experiences. I feel somehow duty-bound to make people aware of what happened, but I don t want to get into anything else but a sober description of the events and some very brief reflections. There are many details I have chosen to omit. Others have written their experiences of the events at Ut ya. I wanted to write mine down as well, and get it out there . Partly, I want to write this down because I m unsure if I will remember all the details at a later point in time, although I think I d prefer it if I couldn t. I m also writing this because people are asking about my experiences and it s much better to have an URL to give them, lest I have to keep going through the same spiel over and over again. Our former Prime Minister and current labour movement demigod Gro Harlem Brundtland had recently left the island. I had been the cameraman for a video interview of her talking about Ut ya, and I was in the media group room encoding the video into a file suitable for YouTube, when someone else in the room startled and said that Twitter was full of messages about a loud explosion in Oslo. As the newspapers brought us information about the extent of the damages, a consensus arose that an informational meeting was in order. As soon as the current round of talks finished, we were gathered into the main hall. The meeting was duly held, and after the statement was made that a TV feed would be made available, I took it upon myself as the local alpha geek to make it happen. Of course, the situation caused both the wireless network and the GPRS networks to become totally unusable. As I was waiting for someone to set up a password, I took the opportunity to face the consequences of having eaten two bits of a microwavable dish called Hold-It the local equivalent of a Hot Pocket and went to the toilet. As I was in there, I first heard agitated shouting, then screams, then gunshots coming from just outside the toilets. More than anything else, it sounded like a toy gun. I was convinced that someone was making a joke in incredibly bad taste and I stormed out of the booth with the intent of halting it. As I tore the door open, I saw two of my comrades hiding in a recessed corner. Their facial expressions left absolutely no doubt that this was no toy. They signalled for me to get back in the booth. I closed the door, did a mental double-take in utter, complete confusion, and opened it again. They were still signalling. Had they not stood there, I would have run straight into the gunman; they saved my life. I looked out into the hallway, and I made eye contact with a young boy lying in a pool of blood. He was motioning for me to help him. I heard more gunshots from inside the building and retreated back inside. As I was trying to think through my next move, I realized that the decidedly insubstantial wood-fiber door would not resist any kind of bullets. I made my way out into the hallway, with the intent of escaping outside. At that point, I was of course not aware that there was an intention to kill as many as possible, so I thought that the open spaces outside would be a place of relative safety. Of course, this proved to be wrong and my life was probably saved a second time by one of the caf volunteers taking me into a hard-to-spot employee s bathroom. We sat there for ninety minutes. Always ready to make a run for it, ready for just about anything. A peculiar group dynamic arose with these two people with whom I had barely previously spoken. We came to share a strange sense of common destiny and gallows humour. One of them had seen the shooter and described the police uniform. I perceived it to be realistic that we were the only ones aware of the wounded outside the toilet. I tried to reach the emergency services, but all their lines were busy; the terror attack in Oslo had probably clogged their lines. I finally got through to the fire services, who could inform me that the police did know about the situation and were on their way. This was to take 90 minutes and by the time we evacuated, the young boy outside my door had perished. The despair I first saw in his eyes as I passed him, fleeing from one room to the other and the empty, blank stare as we left, are burned into me and they are images I will never in my life forget. Finally, the real police arrived. We walked out. I chose the path through the minor conference hall something I now regret. The sight was simply beyond my capacity to describe fully, and so terrifying that I barely remember the sight only the terror it struck in me. There were several people bunched up in a corner, a big amorphous heap of bodies. Some were conscious and yelled at me not to do anything that could startle the police, others lay still. Their bodies were all covered in blood, and a thick pool of blood extended at least a half-metre in all directions around them. The policeman across the hall was screaming orders at me, but he was screaming so loudly that I couldn t make out his words at first. We were first moved into the camp newspaper s offices. There were about eight of us there, I think, in addition to one girl who lay wounded. Towards the end she was drifting in and out of consciousness. We covered her with sweaters to keep her warm and one of us tried to at least temper her bleeding. The bullet had missed her heart, but by the entry wound it was clear that it was not by far. I do not know who this girl was or how she is now. I sat behind and never saw her face. The wounded were evacuated first. I don t remember how long we remained; I had lost all concept of time. In spite of protests from the group who knew him, one kid was put in handcuffs. At the time I didn t understand why, and the policeman seemed to say something almost to the effect that there was no reason for it at all. I didn t see when they undid his cuffs, but I remember thinking that this treatment made a terrible experience even worse for him. I tried my best to comfort him but knew it would be little help. Later, when things stabilised a little, we were told that he was handcuffed because he had come from an unsecured area. The police was extremely good at carefully explaining what was happening and why; this was a big help and I am grateful for it. Eventually we were moved out into the main corridor of the building, where we joined up into a group of about fifty. Only when I saw the two people who saved my life did any emotion other than mild confusion arise. I broke down shivering in tears in one of their arms. After a few seconds, I came back to my senses and realised that this was not the time. I quickly gathered myself, got the shaking under control, and sat down. We were given some chocolate and soda from the kiosk. I remember making an offhand remark that an inability to find joy in free candy was a sure sign of a bad situation. We all laughed out loud. Gallows humour is a coping mechanism, but in retrospect one almost feels guilty for it. We were shown out in a single file with hands above our heads. I remember an intense concern that someone would slip in the steep, muddy slope and create a misunderstanding. Outside, there were more bodies. Some under improvised covers a tarpaulin from the waffle stand, the deflated bouncy castle but some simply lay there. Everyone I met displayed a courage, a mental discipline and unity of purpose far beyond anything one would ever wish to expect from people this young. Everyone conducted themselves with an attitude that could almost be described as stiff upper lip . Safely across the fjord we were offered blankets. I was asked if I was aware of any injuries, and asked to lift my shirt and show my abdominal region. We were shown into the bus which took us to the hotel used by the survivors and their family. I simply cannot describe in any words the relief I felt when I was able to embrace my living comrades. It was completely unlike anything I had ever felt before in my life. The euphoric feeling was tempered only by the realisation that there would be many I could never see again, comrades whom I had taken great pride in calling my friends, with futures in the service of all mankind, futures I had previously found such great joy in pondering and guessing about. The feeling which continues to upset me the most, is the feeling that so many of my comrades left behind grieving families and friends. Torn away senselessly. I do not know how much more than this relatively sober account of the events on Ut ya I can muster. I would, however, like to offer some reflections. First of all, from the bottom of my heart, I want to thank the police who saved the lives of so many still on the island, the holidaymakers who took aboard swimmers into their boats and the rescue services staffed primarily by volunteers who have spared no effort in trying to soften the blow as much as they can. The opportunity to spend time with those comrades who underwent the same experience as myself has also been an immeasurable aid. I was also so relieved to find my very closest friend among the survivors, which has also been an indescribable help. If I can name a single positive in this tragedy: Had he arrived with his automatic weapon fifteen or twenty minutes prior, he would have arrived during the informational meeting, at a time when the major hall was absolutely jam-packed the death toll would be many times what it ended up being. I am agonisingly aware of the meager comfort this provides to those who have been bereft of their closest, but I do find some solace in this. We cannot sweep under a rug that this was without question! a political attack on the labour movement. But it is thankfully also an attack which has been perceived by everyone as an attack on the Norwegian society, and on a symbol of the wide recruitment to the participatory democracy which lies at our very national soul. I cannot thank the Norwegian people, and indeed the people in other nations who have offered their condolences, enough for their shows of support and shared grief. It really has been a tremendous help to me knowing that so many people feel with us. I also want to thank from the bottom of my heart the rock-steadiness of everyone in both the national and local wings of the Labour Party and Labour Youth in supporting us, and the political milieu in general for their resolute steadfastness saving me from losing yet more that I cherish; our freedoms in a participatory democracy. Our Party has lost many of its very brightest youngsters. Personally I feel an angry spite, a deep restless urge to get the wheels of society going again. I want to show his kind that we will not be broken. We re stronger than that. I will not be frightened into silence and passivity. I want to remember the dead, and honor them by carrying on our common work. I want to end this with a request to everyone who reads this, echoing a statement I read by one of my good friends and comrades: Please, don t let me see any messages of hatred, wishes for the death penalty, anything like that. If anyone should be of the belief that anything will improve by murdering this sad little person, they would be profoundly wrong. All attention now should be plowed into caring for those victims and their relatives who did not share my luck, and not giving an audience to a perpetrator who wants one. Tore Sinding Bekkedal

At last DebConf, with the help of Gregor, I've tried to summarize some BoF organization best practices in a sort of HOWTO. As promised, although with 1 year of delay, we have just moved the DebConf BoF HOWTO to a more stable place on the DebConf wiki. In doing so, we have also integrated some of the feedback received last year. If you've proposed a BoF at DebConf11, please take a minute to have a look at it and let us know your comments. As usual, your feedback and contributions to improve the text are more than welcome!

It s not very often that I watch a movie anymore. It s been a few years since I ve actually purchased one (normally I see them from Netflix). But yesterday I saw one that may change that. The Lives of Others is an incredible film set in the former East Germany (GDR/DDR) mostly in 1984. The authenticity of it is incredible and so is the story. It s subtitled, but if you re an American wary of subtitled European films, don t be wary of this one. It is easy to watch and worth every minute. The story revolves around the Stasi, the GDR Ministry for State Security ( secret police ). It is an incredible picture of what living in a police state was like, and how many of the informants were victims of the regime too. My breath caught near the beginning of the film, showing the inside of a Stasi building. A prisoner was being interrogated for helping someone attempt to escape to the west. But the reason my breath caught was this incredible feeling of I was there . Last year, Terah and I were in Leipzig and visited the Stasi museum there, Museum in der Runden Ecke . I always have an incredible sense of history when being in a preserved place, and this building was literally the Stasi headquarters for Leipzig. Much of it was preserved intact, and seeing it in the film brought home even more vividly the terrible things that happened in that building, and others like it, not so very long ago. We watched the special features on the Blu-Ray disc, and one of them was an interview with director Florian Henckel von Donnersmarck. He described how he spent a lot of time interviewing both victims of the Stasi, as well as ex-Stasi officers. One of the most disturbing things to me was his almost offhand comment that most of the former Stasi officers still had some pride in performing their jobs well. Even now, freed of the state s ideology, they were proud of the work they did which could be put most charitably as ruining people s lives. What leads a person to view life that way? How can we try to make sure it doesn t happen again elsewhere? I am happy to say that most of us have never experienced anything like the Stasi. And yet, small reflections of that mindset can be seen almost everywhere. Societies at wartime or feeling under threat, even Western democracies, can drum up those feelings. In the USA, for instance, the McCarthyism era saw people s careers ruined for alleged anti-state behavior. Contemporary examples include the indefinite detention (I hate that word; shouldn t we say imprisonment ?) of terrorism suspects at Guantanamo Bay, and the terrible treatment of Bradley Manning, who revealed some true but embarrassing things about the US military which really needed to be revealed. Even tobacco farmers and companies are selling a product they know ruins lives, but somehow keep doing it. And there are still members of the public that try to make life difficult for people that don t think like they do. From organizing campaigns of telephone harassment of colleges that don t perform the American national anthem before sporting events, to tossing about the term un-American (a loaded McCarthyist one, which many may not even be aware) at an inflated rate, we are not immune from attempts at forcing conformity or silence in others, and blind loyalty to state. I am never in a particularly celebratory mood on July 4, the biggest day for American boasting, faux patriotism, militarism, and general flag-waving. We do have a lot to be proud of and thankful for, but it seems that we celebrate all the wrong things on July 4, and see it as an occasion to proclaim American exceptionalism rather than as one to see how far we ve come and bolster hope for how far we can, and should, yet go. No, I don t think that the land of the free ought to have operated secret prisons in Europe (nor the Europeans to have been complicit in it), or that the American military was defending our freedom 100% of the time they were deployed, or that it is right for governments to mandate daily recitation of an untrue document (the pledge of allegiance) in schools. And yet, I am mindful that I have a lot to be thankful for stability, lack of much internal violent conflict, etc. And this particular day I am happy that a post like this is not something that gets the attention of some government agency and mostly that I will have a handful of angry emails to delete.

There are two different uses (within Debian/Ubuntu) of the initialism UDD: Ubuntu Distributed Development and the Ultimate Debian Database. This article talks about the latter. At UDS-O in Budapest a few weeks ago we had a session (video) on DEX: an initiative to improve collaboration between Debian and its derivatives. There's an Ubuntu DEX subproject, and this is what the session was about, thinking up ideas. I thought of something that could possibly yield some results --- my action from the session was:

[laney] measure relative uploads between ubuntu and debian and see if anything interesting pops up: TODO

It seems to me that if we could tell which packages were uploaded with the most frequency to Ubuntu with respect to Debian, then we may be able to figure out why this is and if there's any way we can do the work in Debian instead. Clearly for this to happen we are going to need data! And where better to house a load of data than the Ultimate Debian Database itself? So I wrote some scripts to download the Ubuntu changes, handily made available as mbox archives, chop them around a bit and eventually split them out in a format that UDD can understand. Lucas Nussbaum then kindly integrated the output into UDD itself, and the end result is that an as-near-as-possible complete history of Ubuntu uploads is now available for your creative querying. For example, Wednesday sees the most uploads and Sunday the fewest:

udd=> SELECT EXTRACT (DOW FROM date) AS dow, COUNT(EXTRACT (DOW
FROM date)) AS count FROM ubuntu_upload_history GROUP BY dow ORDER
BY count DESC;
 dow   count 
-----+-------
   3   26739
   2   26699
   1   25361
   5   23497
   4   22154
   6    9687
   0    8752
(7 rows)

I'll lovingly craft a script to generate the data we need for the DEX project in the next few days. But I hope that the data proves useful for other things too. The three tables you want to look at (with \d+ at a psql prompt) are ubuntu_upload_history, ubuntu_upload_history_launchpad_closes (LP bugs closed by uploads) and ubuntu_upload_history_closes (Debian bugs closed, probably not so interesting). Information on how to use UDD yourself is on this wiki page.

In the graphics side, the major differences between ARM and X86 systems is that on ARM 3D acceleration is done with OpenGL ES. It is mostly a subset of modern OpenGL used on X86 desktop machines. From QEMU point of view this could mostly be ignored, as OpenGL was mostly used on games and specialist applications. This has now been changing, as desktops and user interfaces have started OpenGL to render graphics. Without acceleration, these user interfaces become slower than slugs crossing a tarpit.

For this reason, MeeGo introduced OpenGL ES acceleration support to QEMU. With the lack of easily available MeeGo QEMU images and test setup, I've created a test setup for Linaro 11.05 image.

Running torus opengl es1 demo from mesa-demos inside QEMU.

I finally got around to book my (train) trip back from Zagreb. The hotline of Deutsche Bahn was interesting. The booking system crashed and I was called back. But in the end it worked and I can print it the next time I get near a vending machine.

I'm worried how I manage to get there from Banja Luka on my own. And the travel to Zagreb together with Joachim Breitner won't be as comfortable as it should be. But meh, I'm going to DebConf again! \o/

So I got my hands on that new machine, somewhere, somehow, and I had to make myself comfortable on it. The biggest difference in that machine and the ones I used before is the RAID controller, an Adaptec 5405, instead of the 3ware ones I used to have before. That means getting used to new tools, messages and all these things which I ll try to document here.Step 1: does vendor deliver?Well, yes, mostly. There is (mainline) kernel support for the controller, Squeeze s kernel will boot up just fine and I just throw my junk into /dev/sda and it will handle the whole raidy thingy itself. But what is when I want to check the health of the disks, to add a spare or modify the caching? Ask the mighty vendor and it will answer Yes, you can! .So I downloaded that asm_debian_x86_x64_v6_50_18570.tgz, unpacked and faced two debs, one i368 and one amd64. Yes, you could have saved half of the bandwidth if someone would have asked for the arch before, but who cares for some additional 60MiB today? So let s see what it does. Oh no, wait, there is a README ASM_IUG_for_Debian_and_Ubuntu.txt:

1. Make sure to be in root level access.
2. Install the ASM storman .deb package. For example, if the package is storman_6.50-18570_i386.deb,
   then install it using the following command dpkg -i storman_6.50-18570_i386.deb
3. Adaptec Storage Manager will get installed at /usr/StorMan
4. Add the line /usr/StorMan to /etc/ld.so.conf. Then run "ldconfig". This is needed for the arcconf
   utility to find the path of the libstdc++.so.5 library

Well, 1 is obvious, so is 2.
For 3: hum, /usr/StorMan? Ever heard of FHS? Don t think so.
For 4: WTF?! I prefer to get libstdc++5 from my local Debian mirror instead.But at least they provide a .deb, right? Uhm, no, it s an RPM converted via alien :( And don t dare to look at preinst, postinst and postrm (it extracts an own copy of Sun JRE 1.6u16, chmods around wildly on install and does rm -rf something on uninstall, did I see that some days ago on the webs?). Let s get rid of that package as soon as possible, I decided, and just keep arcconf (which will be described later), esp because StorMan itself seems to include some call-home functionality which I d like not to have:

cp -a /usr/StorMan/arcconf /usr/local/sbin/
apt-get remove --purge storman

Step 2: does community deliver?You can, by the way, get (better) packages at http://hwraid.le-vert.net/wiki/DebianPackages from Adam C cile (Le_Vert). I didn t test those much, but I currently use the arcconf package from there, which seems to be exactly what I need: proper dependencies and just the arcconf binary, no Java sh*t.Step 3: monitor the controllerAs this post is about monitoring, not ranting, let s continue. As mentioned before, I only have arcconf installed, which is just a cli to do stuff with the controller. If I understood the docs right, the actual StorMan is able to send notification mails when a disk dies or the controller gets eaten by aliens, but as I don t have StorMan, I have to do things myself.Adam has a aacraid-status package in his repository, which includes some minimalistic shell-daemon to monitor the output of arcconf. It didn t perfectly fit my needs (and had some hickups with my disks which refuse to report a vendor, whyever), so I decided to write something myself (slightly based on what Adam has done in aacraid-status, though).The result is on GitHub: https://github.com/evgeni/aacraidAs you can see the code contains (at the time of writing) exactly ZERO comments and there is no README either, but it s actually quite usable already.

• aacraid.py is the main magic inspired by Adam s aacraid-status, basically just parsing arcconf output
• aacraid-status prints all useful info about all controllers, arrays and disks (the output is pretty much like Adam s)
• aacraidd is a small daemon which will monitor all controllers every (per default) 5 minutes and mail you in case of status-change

I am now running aacraidd and get happy mails when something breaks :) [running = running from /etc/rc.local, no init script yet :)]Step 4: monitor the disks via smartmontoolsThere is one little problem I have with arcconf: it s closed source and does magic to the controller.With the old 3ware controller I was able to monitor the disks with smartmontools/smartd using something like this:

/dev/twa0 -d 3ware,0 -a -s L/../../2/01 -m root
/dev/twa0 -d 3ware,1 -a -s L/../../2/03

This is possible with Adaptect controllers too (not through /dev/twaX obviously). You have to have the sg module loaded, which will result in /dev/sg[012] on my machine, with sg0 being the controller itself and sg1/sg2 the both disks attached to it. Having sg loaded, one can now use smartd with:

/dev/sg1 -d sat -a -s L/../../7/01 -m root
/dev/sg2 -d sat -a -s L/../../7/02 -m root

(-d sat is the important option here!)Thanks to Thomas Krenn Wiki: Smartmontools mit Adaptec RAID Controller for the basic info and ostlogd.spenneberg.net: Adaptec-Raid-Controller S.M.A.R.T.-Healthstatus monitoren for the pointer to the sg module!Step 5: relaxYupp, 4 steps are enough, go relax until the next disk dies (you will be doing something REALLY important that moment according to Murphy).

0 comment(s) this blog is flattr enabled