Search Results: "will"

21 October 2008

Matthew Palmer: Load Balancing FTP Servers

While it's a protocol we all love to hate, and wish it would go away and die a quiet death, sometimes someone has a good reason for supporting FTP. (In my case, it's usually because that reason has a lot of money to provide to my employer). But what if you want your FTP servers load balanced (because downtime is for suckers)? Obviously, you need to have some sort of shared filesystem behind all your load balanced FTP servers, but that's a rant for another day (but let me just say: if you're running GFS, especially on a busy site, may $DEITY have mercy on your soul). Because of the silly random port games that FTP likes to play, it's not a trivial job to load balance an FTP service amongst multiple hosts. You can't just throw a round-robin on port 21 and hope to get anything useful going. Ironically, while "active" FTP is a pain in the arse for most people, it's a lot easier to load balance, because the initial connection comes in on a well-known port which can be load-balanced, while all the other "random" connections are server-initiated and your standard routing/NATing/whatever you're doing for outgoing connectivity should be able to handle it. However, passive FTP is a bigger pain. The problem is pretty simple to understand -- the FTP client will want to connect to a high port on the "virtual" (load-balanced) IP, and without serious traffic sniffing, your load balancer won't be able to map that incoming connection on port 54321 to the real server that actually wanted the connection. The solution, thankfully, is fairly straightforward. You allocate a range of high ports (maybe a thousand or so) to each real server, with a different port range for each real server. You then tell the load balancer to send all traffic for each range of ports to the corresponding real server, and then configure that real server to put all it's passive connections into that port range (I'm fairly sure most FTP servers can support that bit of config). So, unfortunately, you can no longer disallow FTP support on the basis that you can't load balance it. No shortage of other reasons, though.

Russell Coker: Kernel issues with Debian Xen and CentOS Kernels

Last time I tried using a Debian 64bit Xen kernel for Dom0 I was unable to get it to work correctly, it continually gave kernel panics when doing any serious disk IO. I’ve just tried to reproduce that problem on a test machine with a single SATA disk and it seems to be working correctly so I guess that it might be related to using software RAID and LVM (LVM is really needed for Xen and RAID is necessary for every serious server IMHO). To solve this I am now experimenting with using a CentOS kernel on Debian systems. There are some differences between the kernels that are relevant, the most significant one is the choice of which modules are linked in to the kernel and which ones have to be loaded with modprobe. The Debian choice is to have the drivers blktap blkbk and netbk linked in while the Red Hat / CentOS choice was to have them as modules. Therefore the Debian Xen utilities don’t try and load those modules and therefore when you use the CentOS kernel without them loaded Xen simply doesn’t work. Error: Device 0 (vif) could not be connected. Hotplug scripts not working. You will get the above error (after a significant delay) from the command “xm create -c name” if you try and start a DomU that has networking when the driver netbk is not loaded. XENBUS: Timeout connecting to device: device/vbd/768 (state 3) You will get the above error (or something similar with a different device number) for every block device from the kernel of the DomU if using one of the Debian 2.6.18 kernels, if using a 2.6.26 kernel then you get “XENBUS: Waiting for devices to initialise“. Also one issue to note is that when you use a file: block device (IE a regular file) then Xen will use a loopback device (internally it seems to only like block devices). If you are having this problem and you destroy the DomU (or have it abort after trying for 300 seconds) then it will leave the loopback device enabled (it seems that the code for freeing resources in the error path is buggy). I have filed Debian bug report #503044 [1] requesting that the Xen packages change the kernel configuration to allow more loopback devices and Debian bug report #503046 [2] requesting that the resources be freed correctly. Finally the following messages appear in /var/log/daemon.log if you don’t have the driver blktap installed:
BLKTAPCTRL[2150]: couldn’t find device number for ‘blktap0′
BLKTAPCTRL[2150]: Unable to start blktapctrl It doesn’t seem to cause a problem (in my tests I can’t find something I want to do with Xen that required blktap), but I have loaded the driver - even removing error messages is enough of a benefit. Another issue is that the CentOS kernel packages include a copy of the Xen kernel, so you have a Linux kernel matching the Xen kernel. So of course it is tempting to try and run that CentOS Xen kernel on a Debian system. Unfortunately the Xen utilities in Debian/Lenny don’t match the Xen kernel used for CentOS 5 and you get messages such as the following in /var/log/xen/xend-debug.log: sysctl operation failed — need to rebuild the user-space tool set?
Exception starting xend: (13, ‘Permission denied’) Update: Added a reference to another Debian bug report.

Ben Hutchings: For those who care about firmware

Some drivers in Linux include "firmware" - code that runs on the device the driver deals with, not the main processors in the system. Most of this is in a binary or equivalent form, which is probably not the form that it was originally written in. This makes it non-free by Debian's standards, an issue which has led to much argument around previous releases and has blown up once again. The good news is that kernel developers are making a continuing effort to separate this from the driver code. I've collected together the patches that do this and written 2 of my own, which appear to cover all the remaining sourceless firmware. But I was only able to test one of these drivers (radeon) - the rest is up to you. Affected drivers
DriverHardware which requires firmwareBug reportChange made
cassiniSun "Cassini" Ethernet controllers with "Neptune" PHY498631removed
dabusbBayCom DAB-USB radio receivers502663moved to firmware-dabusb
e100Intel PRO/100 Ethernet controllers with PCI revision 8, 9, 15 or 16 or built into ICH6 or ICH7 (LAN-on-motherboard)494308moved to firmware-e100
kawethKawasaki LSI USB Ethernet controllers502665removed
mgaMatrox G200, G400, G450(?) and G550 graphics controllers, 3D acceleration only502666moved to firmware-matrox
qla1280QLogic QLA1XXX SCSI controllers502667moved to firmware-qlogic
r128ATI Rage 128 graphics controllers, 3D acceleration only494007moved to firmware-ati
starfireAdaptec DuraLAN aka Starfire Ethernet controllers501152moved to firmware-adaptec
tehutiTehuti Ethernet controllers501153moved to firmware-tehuti
typhoon3Com Typhoon (3CR990) Ethernet controllers502669moved to firmware-3com
whiteheatConnectTech WhiteHEAT USB serial adapters502668removed
How to test
  1. Download and install the linux-source package from http://people.debian.org/~benh/firmware-removal/. There is a signed .changes file there that you can use to verify the package.
  2. Build and install a kernel from that source (you should probably use make-kpkg).
  3. Boot the kernel. Check that the driver fails cleanly: it should not crash, and there should be a useful error message in /var/log/messages
  4. Download and install the relevant firmware package, if available. These are available from the same place as the linux-source package.
  5. Reboot or reload the driver. Check that the driver is now working.
  6. Add your results to the bug report as shown above.
Firmware that is removed Do you depend on the drivers for which I have removed the firmware? Do you have contacts with the vendors who might be able to help relicence it? Please send mail to the bug report. Update I checked my email and found a response from ConnectTech; the WhiteHEAT firmware will probably be clear to package in firmware-nonfree. There is no need for anyone else to contact them.

Joey Hess: debhelper option parsing overhaul

I've been meaning to fix debhelper's option parsing code for a while (7 years?!), and finally got around to it with version 7.1.0, in experimental. Now debhelper commands can define their own command-line options. This is especially useful for third-party commands that couldn't add new options before, without pestering or bribing me. (Bribes still happily accepted however.) It's in experimental because I removed global support for a large number of lesser-used options. Some of these might be used by third-party debhelper commands -- if so, they will be broken by this change, and need to define those options themselves. Also, it used to be possible to pass nonsensical options to a command, like "dh_strip --include-conffiles --no-restart-on-upgrade". To avoid breaking unknown numbers of packages that might do things like this, I've made debhelper start warning about such things. This warning will be converted to an error as soon as I dare.

David Welton: Android is out!

As most people probably know by now, the Android source code is available, as promised: http://source.android.com/ The next part is the interesting part, though, in my opinion. Source code is wonderful, and a huge gift from Google, but the real "secret sauce" for the most successful open source projects is the community. It will be interesting to see how Google manages this aspect of their project. It could be a situation where they create, and the public consumes, and it's fairly one way, with the odd patch here and there going back. Or it could be that they'll really let people get involved. Running a community for something this big, with this much code, and this much interest to the world at large is a daunting task, but Google certainly has the open source expertise to figure it out, so what we end up seeing will depend on exactly how Google wants to manage things. All in all though, a big step for open source - congratulations to all the people at Google who made it happen!

Biella Coleman: Measuring a Courses Success: xkcd

So we are about 1/2 way done with my hacker class and I have to say it has been a pretty fun ride, especially since many students thought hacking was all about pimply kids engaging in malicious acts of computer violence and many had no knowledge of basic technological concepts like UNIX or source code. We had to first sweep away the cobwebs of misrepresentation and replace with a more solid foundation of facts (however messy the world of hacking still is!) But after building a foundation, you still wonder whether students are learning. Traditionally we gauge progress with exams or essays, which can be effective but let’s face it, at times a little tedious. But today a student sent me the following very short email, which made me realize I had another barometer at my disposal to gauge their progress xkcd:
http://xkcd.com/344/ I wouldn’t have gotten the joke if it wasn’t for you. Thanks.
The email not only made me smile (for it is always nice to know your students are leaning something) but it gave me the idea that in the future I might include a comic based exams composed of 5-10 comic strips (many from xkcd) and ask for an exegesis of them. Why not? Seems like fun to me. And it would be great to include the following as one of my course objectives: “By the end of this class you will be able to read xkcd and actually understand (most) of it.” If they could do that, well, they must have learned at least something :-) update: One of my readers, Florian, sent this nice bit, which I have not read before: “I read your blog via Planet Debian and immediately felt reminded of Jane
Goodall’s Foreword to Gary Larson’s The Far Side Gallery 5 which I’d
like to share with you:”
Recently I was talking with one of the best researcher know, Tim
O’Halloran. He has been able to inspire generations of middle and high
school students to care for the natural world. Tim told me that Gary
Larson has had a major impact on his teaching. Tim uses Far Side
cartoons to introduce topics, to illustrate points, and to “reinforce
the notion that the more we investigate the universe, the richer is our
experience.” When designing exam papers Tim finds the cartoons “ease the
tension and spark the memory.” It all began when, in the fall of 1985,
he was given the task of teaching science to 162 Tulsa ninth-graders who
were convinced that it was absolutely irrelevant to their futures. Tim
put one hundred Far Side cartoons on a large bulleting board, and told
the students to study them. The consensus was that they didn’t
understand the humor - The Far Side was “too weird”. However, Tim wrote
me, “Each time we completed a unit and the students approached the
bulletin board with newly acquired wisdom, I smiled quietly and thanked
the cosmos for Gary’s perspectives as the kids roared with the confident
laughter of the enlightened.” Having taught myself for a while, I can fully and happily relate.
Later in the semester, and for my hacker course, we will be reading Understanding Comics, which will give us a bit of a meta perspective on why comics are so good at conveying a certain type of message. I can’t wait to read it as I have heard it is fantastic.

Biella Coleman: Measuring a Courses Success: xkcd

So we are about 1/2 way done with my hacker class and I have to say it has been a pretty fun ride, especially since many students thought hacking was all about pimply kids engaging in malicious acts of computer violence and many had no knowledge of basic technological concepts like UNIX or source code. We had to first sweep away the cobwebs of misrepresentation and replace with a more solid foundation of facts (however messy the world of hacking still is!) But after building a foundation, you still wonder whether students are learning. Traditionally we gauge progress with exams or essays, which can be effective but let’s face it, at times a little tedious. But today a student sent me the following very short email, which made me realize I had another barometer at my disposal to gauge their progress xkcd:
http://xkcd.com/344/ I wouldn’t have gotten the joke if it wasn’t for you. Thanks.
The email not only made me smile (for it is always nice to know your students are leaning something) but it gave me the idea that in the future I might include a comic based exams composed of 5-10 comic strips (many from xkcd) and ask for an exegesis of them. Why not? Seems like fun to me. And it would be great to include the following as one of my course objectives: “By the end of this class you will be able to read xkcd and actually understand (most) of it.” If they could do that, well, they must have learned at least something :-) update: One of my readers, Florian, sent this nice bit, which I have not read before: “I read your blog via Planet Debian and immediately felt reminded of Jane
Goodall’s Foreword to Gary Larson’s The Far Side Gallery 5 which I’d
like to share with you:”
Recently I was talking with one of the best researcher know, Tim
O’Halloran. He has been able to inspire generations of middle and high
school students to care for the natural world. Tim told me that Gary
Larson has had a major impact on his teaching. Tim uses Far Side
cartoons to introduce topics, to illustrate points, and to “reinforce
the notion that the more we investigate the universe, the richer is our
experience.” When designing exam papers Tim finds the cartoons “ease the
tension and spark the memory.” It all began when, in the fall of 1985,
he was given the task of teaching science to 162 Tulsa ninth-graders who
were convinced that it was absolutely irrelevant to their futures. Tim
put one hundred Far Side cartoons on a large bulleting board, and told
the students to study them. The consensus was that they didn’t
understand the humor - The Far Side was “too weird”. However, Tim wrote
me, “Each time we completed a unit and the students approached the
bulletin board with newly acquired wisdom, I smiled quietly and thanked
the cosmos for Gary’s perspectives as the kids roared with the confident
laughter of the enlightened.” Having taught myself for a while, I can fully and happily relate.
Later in the semester, and for my hacker course, we will be reading Understanding Comics, which will give us a bit of a meta perspective on why comics are so good at conveying a certain type of message. I can’t wait to read it as I have heard it is fantastic.

MJ Ray: Monday Machine-Mangling Madness, or: Server2 on fire!

//www.flickr.com/photos/dominicspics/1901310469">dominicspics</a> (cc-by)
Bonfire Onlooker by dominicspics (cc-by) Of all the things I wanted to do yesterday, testing TTLLP’s disaster recovery plan was not one of them. I often suggest customers make disaster recovery plans, but I don’t like having to use them. One of our server farms suffered a serious hardware failure and slowly died during the day. (It didn’t catch fire - the title is a silly reference to lp0 on fire.) The current stinging electricity bills means that there’s an emphasis on power-saving now that wasn’t there 4 or so years ago, when we still built servers ourselves, before WEEE and cheap big-brand systems. Today’s servers aren’t quite as resilient, because each extra piece of hardware in use takes extra power. There are some much cleverer tactics now, but they don’t catch everything before it goes “bang” and we can’t just bring the “spare” hardware into action if it’s not installed yet. Replacement hardware will be installed as soon as possible, as well as some extra hardware that will help protect against a similar failure in future. The slow failure of the server also means that we lost some emails, voicemails and instant messages from 10am to 2pm yesterday. If one was yours, please resend. Most services have been rerouted or temporarily replaced with help from friendly associates. Three important services that haven’t yet are: 1. our financial system (which gets updated in batches anyway), 2. our task tracking (which is newer than our recovery plan and I need reminding about its backups, but at worst all the current data exists in other places) and 3. our newsletter server. I’ve copied the customer newsletter subscriptions to an older listserver, to send out news of our recovery, but the member newsletters (with GPG integration and so on) will wait for the replacement hardware. I’m hoping we can recover some things from the old server filesystem, else I’ll recreate it from the last release (that’ll teach me to hack it on the live server!). In the meantime, I’ve still most of yesterday’s work to do…

20 October 2008

Kapil Paranjape: Going for a spin

When a sphere (globe or ball) is turned about its centre a number of times, there is at least one point that starts and ends at the same place. This a consequence of the algebraic statment that a 3x3 orthogonal matrix with determinant has 1 as an eigenvector with eigenvalue 1. One of my students recently asked me for a geometric proof and I came up with the following construction. (Of which I am evidently quite proud!) We begin by making a more precise statement of the above proposition. Let a, b, c, ..., z be a succession of points on the sphere and A, B, C, ..., Z be a sucession of rotations; where A is a rotation about the axis through a, B a rotation about the axis through b and so on. We will show how to find a point on the sphere so that the combined effect of these rotations is a rotation about the axis through that point. It is enough to show how this can be done for two successive rotations since the rest of the argument/construction follows by induction/recursion. So we restrict ourselves to the simpler case of two points a and b on the sphere and two rotations A and B respectively about the axes through these points. Let me introduce some elementary notions and results. The term "great circle" is used for the circle obtained by cutting a sphere with a plane passing through the centre of the sphere; any two points on the sphere lie on a (common) great circle. The term "antipode of a point p" denotes a point q that is diametrically opposite to the given point p; the axis through a point also passes through its antipode. Two points on the sphere that are not antipodal lie on a unique great circle, whereas the antipode of p lies on every great circle through p. Consider the point c which goes to the point b under the rotation A and cb be the great circle containing these two points. Let d denote a point on bc that lies halfway between b and c. There are two such points which are antipodes; pick any one. Let ad be the great circle containing a and d. Similarly, let e be the point which a goes to under the rotation B and ae be the great circle containing these two points. Let f denote a point on ae that lies halfway between a and e; let bf denote the great circle which contains b and f. A point g (and its antipode) where the great circle ad meets the great circle bf is fixed under the combination of the two rotations. In fact, the result of the rotation A followed by the rotation B is a rotation G about the axis through the point g. The proof that this construction works relies on the following description of a rotation X of a sphere about the axis through a point x on the sphere. Let y be any other point and z be its image under the rotation. As above let t be the point halfway between y and z on the great circle containing these two points. The rotation X is obtained as a succession of two reflections; first a reflection in the plane containing the x, t and the centre of the sphere and second a reflection in the plane containing x, z and the centre of the sphere. One can see this by noting that the composite of two reflections is indeed a rotation; moreover, a rotation that fixes x is uniquely determined by what it does to y. We use the above description to write the composite of A and B as succession of four reflections through planes as follows: the plane containing d, a and the centre of the sphere; the plane containing b, a and the centre of the sphere; the plane containing a, b and the centre of the sphere; the plane containing f, b and the centre of the sphere The two reflections in the middle cancel each other out. We are then left with the composite of two reflections. Clearly, the points that lie on the fixed plane of each of these reflections are fixed by this composite rotation. This intersection is an axis of the sphere through the point g.

Decklin Foster: And yet I swear this oath

There was a lot of political talk last week in the US about this guy "Joe the Plumber". I'm really quite mystified by the (lack of) Democratic response to this one; it seems like a perfect opportunity for the Dems to explain what their ideals actually mean, after years of complete inarticulacy. The Republican party, and conservatives in general, have somehow successfully sold their platform of, as Robert Reich brilliantly put it on the Daily Show last week, "socialism for the rich and capitalism for everyone else", to poor but socially-conservative citizens for quite some time. They've convinced people to vote against their economic self-interest, and cut taxes on the rich instead of themselves. Is voting for policies that directly benefit you less necessarily bad? I don't think so. In many cases, selflessness and altruism are better in the long run. Conservative economic policies speak to a sense of right and wrong that give people a feeling of purpose, and so an incentive to create: this is mine, I made it, you can't take it away. In a theoretical sense, leaving more wealth in the hands of the rich should "trickle down" to society in general, by providing more capital, and thus creating jobs, etc. However, I do not believe that this is why Americans, specifically, tend to support conservative economic policy. The USA has this idea of the "American Dream", pulling yourself up by your bootstraps, if you will, and striking it rich through hard work and character. It's advantageous for conservatives to sell this idea because it makes people think "tax the rich? What if I get rich?", which is a sort of enlightened self-interest, arguably. And it gets poor people to vote for them. What makes this "plumber" thing interesting to me is that is should demonstrate where this approach ought to fall down, logically and emotionally. But it does not. Here is the problem. A working stiff who earns "five figures", such as myself, generally makes that money by going to an employer every day, performing some services for them, and then drawing a paycheck. Where this money comes from is abstracted away (this is the marvelous thing we call "business"). Let's say you want to sell the "American Dream" to this person. They get their paycheck every week, and there's a chunk taken out of it by Uncle Sam. Maybe they're saving up to start a business, or go to college, etc. They sure could do it faster if taxes weren't so high. So you remind them this: when you do strike it rich (since this is America, where anyone can strike it rich), do you want those taxes to be even greater? This story contains a lie of omission, probably the greatest lie ever told in US politics, and one that has almost universally succeeded as far back as I can remember. And that is: when you do make that jump to where our hypothetical Joe is, making just over a quarter-million dollars, that it's not going to look any different. You'll get your paycheck from some abstracted source, just like before, but more will be taken out unless we stop raising taxes on those higher income brackets. There will be some node higher up in the tree for that wealth to "trickle down" from; you could say it's implied, even, that wealth is created at the root and destroyed at the leaves. What's wrong here? Simple: a self-employed plumber does not have a paycheck, except maybe to him/herself for accounting purposes. They collect their income in the form of payments from customers. They don't have an abstraction: they hire more labor and become the abstraction. If their customers have no money, the abstraction leaks, their employees have to be laid off (and then can no longer afford plumbing services themselves). The success of the plumber depends as much on how many people are able to pay them as it does on his costs (such as taxes). We want to make sure those customers can still pay. This is, in theory terms, the idea of demand-side or Keynesian economics. (Not really exactly. I took about two econ courses. But bear with me for a moment.) Now, you can argue demand-side vs. what we would call supply-side economics, which is the idea that no, first you have to stop taxing Joe so he can hire people so they have money to go buy things, for ages. People have made whole careers out of it. I am not here to do that. What I am here to wonder out loud is: why is this argument not happening, now, in our public sphere? Even if only as a small plot point in one Presidential campaign? There are very good arguments for both sides (I agree with the Keynesians, myself), and they should be what this debate is about, not digging up dirt on some guy whose real name, horrors, might not be "Joe" or protests about all those you-know-what poor people getting "government giveaways" that are disturbingly close to racism in my opinion. When I turn on talk radio I hear zombies, unable to string together any thought more complicated than "Democrat == higher taxes on everyone" (which would be depressing even if the claim were true). It begins to get unsettling when you understand your opponent's argument, and yet watch them completely unable to articulate it. When they deal with this by appealing to fear and thought-killing sound bites, it gets alarming. Obama has backpedaled whenever the whole issue has come up. In the last debate, he was saved by merely shutting up and letting out enough rope for McCain to hang himself. Not very inspiring. As a result I hear repetitions of Obama's phrase "spread the wealth around" constantly met on the airwaves with... silence. He won't touch it, and honestly, I can't blame him for not wanting to fan the flames of class warfare with only two more weeks left to stick it out. But I don't understand why, and why a single person on the left hasn't had the faith that this idea can be discussed in our society. I think that, whichever side you are rooting for (and yeah, I am for the Democrats), the Democrats should have won this point readily. Not to belabor the point, but it really amazes me that they have not. It wouldn't have taken much. Here's what you do: go to Wherever, OH with your crack PR team and find several of "Joe the Plumber"s customers. Interview them all. A few of them are, in this economy, going to be really hurting. If you're lucky: Remember that Joe owes taxes to the IRS. Very possibly, someone owes Joe money! That would be a coup. You just put them in front of the media and the whole thing sells itself. Mary needs to make these repairs... if only her tax credit were larger... etc. This would not settle the argument, by any means, especially not for political junkies. But it would begin it. We could talk about policy, whether one way of "spreading" things will result in more wealth to spread than another, instead of red-baiting. I feel a bit ripped off that we haven't even gotten that much. The theory part is pretty much irrelevant to most people; that's fine. But they deserve context. I can only assume that my ("my") party is still either terrified or stupid. I hope winning the Presidency changes that somewhat. That whole right and wrong, purpose, "noble cause" thing, right? Right, guys?

Michal &#268;iha&#345;: Gammu test version 1.21.90

I just published new Gammu testing version. Major changes are in handling some special Nokia responses, handling of Siemens phonebook entries and introduction of automatic testing using CDash on http://cdash.cihar.com/index.php?project=Gammu. Full list of changes: You can download from usual place: http://cihar.com/gammu/, Debian users will find packages in experimental soon.

Steve Kemp: May you live forever.

This will be my last post on the topic of tscreen for at least a couple of days. People who want to follow tscreen work know where to get it. Anyway this release exposes the notion of layouts - they allow you to save and restore window positions. i.e. If you split your window, detach and then reattach it will be back in the layout you had before! Simply add "layout new default" to your ~/.tscreenrc file, and have fun. The layout primitive allows: Experimentation is fun, people! I've added a couple of patches from other people which I found on the screen-devel mailing list: Anyway sources and Debian packages for Etch, Lenny, & Sid are available in both i386 and AMD64 flavours. Update: Fixed links to patches, and added if primitive. ObFilm: 300.

Kapil Paranjape: Xen on lenny x86_64

Kudos to the Debian Xen Team for getting Xen ready for lenny in time. As I wrote earlier, my desktop runs a Xen dom0 under Debian etch. The various domU's run Debian lenny among other things. While doing a (much delayed) upgrade of these domU's I noticed that there was a xen-linux-2.6.26-1-amd64 package available under lenny. This made me exclaim, "Three cheers for the Debian Xen Team! Time to upgrade and test things out." The upgrade to lenny went smoothly --- almost. Here are some sticky points which are at least partly due my own goof-ups and perhaps due to some aspect of the xen package upgrades. Since I am not clear about where the "blame" lies, I haven't yet filed bugs! The bottom line is that to upgrade an etch dom0 to a lenny dom0 (on amd64) you should make sure that you install the following packages:
xen-linux-system-2.6.26-1-xen-amd64, linux-image-2.6.26-1-xen-amd64, 
linux-modules-2.6.26-1-xen-amd64, xen-utils-common, xenstore-utils,
xen-hypervisor-3.2-1-amd64, xen-utils-3.2-1

If you install all recommended packages, then just installing the first package above will pull in all the others; however, like me you may be a no-one-recommends-packages-to-me kind of person! :-( Secondly, you should add the "xencons=tty" kernel parameter to those domU's which will boot the lenny 2.6.26 xen kernel. These are minor issues considering that anyone who runs a Xen machine circa 2008 should be knowledgable enough to be able to fix them. However, they could cause unpleasant hiccups for some people so I'm documenting them here. Update: One thing which is required to run the Xserver when your graphics card uses shared memory is to limit the memory used by dom0. To do this you pass an option like "dom0_mem=128M" to the command line for the xen hypervisor in grub's configuration file menu.lst .

John Goerzen: Tabor's Centennial

You know you've got a bunch of Mennonites together when the pastor spontaneously asks the church to sing "Jesus Loves Me" while the children walk up for their children's story. And, without music, everybody spontaneously breaks into beautiful 4-part harmony. It must be genetic or something.

It was a day of beautiful singing of hymns a week ago for Tabor Mennonite Church's centennial. It's hard to write about that weekend, in fact -- it was such a nice experience.

Sunday ended with opening the box sealed in the church's cornerstone back in 1965 when the current building was built. Rudy Schmidt, the person that sealed it up back then, was on hand to open it back up. They took out the stone and removed the box. Before the box was opened, Rudy told a story:

Back in 1965, I was the one to solder this box shut. While I was doing this, all of a sudden I smelled burning from inside. I figured I must have caught something inside on fire. I asked Jake Koehn (another person on the building committee) what to do. Jake said, "Nobody will ever see it again, just put it in." So let's open it!


I think Rudy has been wondering all these years whether he set it on fire or not.

The box was opened, and other than a few scorch marks on an envelope, was completely intact and in good shape. There was a copy of a local newspaper from 1965 with a headline about Goldwater, copies of programs of dedication for the new building and parsonage, a Bible, copies of the church constitution, and all sorts of other interesting things. To that we will add a copy of our centennial book (which I helped develop), CDs with recordings of our centennial activities, programs from the centennial celebration, and photos of the weekend.

Earlier on Sunday, we had a long but good worship service with sermons from two former pastors. The choir sang, including a song commissioned for the occasion by Larry Nickel. We got applause after that one (which happens occasionally, but not regularly, in our church). All the living former pastors, and the widow of one of them, served communion as well.

The service ended with the congregation singing the "Mennonite Anthem". I think it was the most beautiful singing of that song I've ever heard. We had 352 people in church that day, and it seemed that not one person was going to let the moment go by without singing.

Saturday had started off with outdoor activities in the church park. There was old-fashioned soap making, remembering how church members used to store fat all year, then get together to make soap for use in their homes and to donate to the needy. There was rope making, wool spinning, and rides for the children. Under the tent, we had some great music the local Greenhorns band, and the former pastors all shared some memories and stories about their time here.

On Saturday evening, we had a meal and a drama -- and managed to somehow fit 321 people into the church basement for the meal. We think that's a record! The drama was written for the occasion by one of our members. John Gaeddert, a pastor here in the early 1970s, has become an expert wood carver in his spare time. He had carved a piece for us to celebrate the occasion, and presented it during the drama. That fit right in, because the drama was called "Bring Your Own Hedgepost" -- back in the early days of the church, each member was supposed to do just that so they would have a place to tie their horses.

Friday night was a lecture by James Juhnke about our first paster, P. H. Richert, who was pastor for nearly 40 years. Quite entertaining and interesting.

I think the highlight of the weekend for me was getting to talk to some of the former pastors I never knew well. I had a few minutes to chat with John Gaeddert. I introduced myself, and he said, "Oh! I don't think I've met you, but I've heard your name a few times this weekend. You're on the centennial committee, right?" Yes indeed. I was born after he left this church, but he knew my grandparents -- which means he also knows exactly where we live. John and his wife Mary are both such warm and friendly people that it feels like I've known them much longer than a few days!

I got to chat with Jim Schrag too. He was pastor at Tabor until I was about 5 or 6. He didn't remember me specifically, but he was also interesting to talk with. He was one of the people that worked on a detailed history book in 1983 for the church's 75th anniversary. He told me that he processed most of the black and white photos for that book in a darkroom in his basement. I hadn't known that -- and mentioned I had found all those photos in the church archives and had scanned them all in. Jim requested a copy.

Terah found Lenore Waltner, wife of former pastor James Waltner, who passed away about a year ago. James had started his career as a pastor at Tabor, and ended it at College Mennonite Church in Goshen, IN, where Terah grew up. So she knew him, but I didn't. Terah introduced herself to Lenore, who seemed quite excited to make this sort of family connection!

And Brenda Martin Hurst, another former pastor, found Terah and introduced herself. Terah said that Brenda said something like, "Hi, I'm Brenda, and I don't think I know you!" These pastors all want to get to know everybody, I think!

Josselin Mouette: A little rant on the Linux kernel development model

Sometimes, I m becoming really pestered by the number of issues, especially hardware issues, that remain prominent in the Linux system after so many years. And I m convinced that one of the root causes is the kernel development model. While far to be the only one, you understand a lot when you have a look at it. The stable API nonsense To explain the development model, the kernel documentation contains a document written by Greg Kroah-Hartman called stable_api_nonsense.txt. I think this document gets one thing very right: advertising it is complete nonsense. Many things in this document can get you a good laugh if you are used to software development, unfortunately they are not funny given how much they affect us as users and developers. Let s start with the executive summary:
You think you want a stable kernel interface, but you really do not, and you don't even know it. What you want is a stable running driver, and you get that only if your driver is in the main kernel tree. You also get lots of other good benefits if your driver is in the main kernel tree, all of which has made Linux into such a strong, stable, and mature operating system which is the reason you are using it in the first place.
What it should say instead is:
You know you want a stable kernel interface, but you don't have it, and we will never provide it. You want stable interfaces to focus on fixing bugs in your driver instead of updating it for interface changes, and to make integration of your driver in the main kernel tree easy. You would get a lot of good benefits if your driver was in the main kernel tree, but it won't make it unless you adapt to our bizarre processes, all of which have made the Linux kernel into a constantly moving target which is the reason many hardware vendors don't want to support it in the first place.
Our compiler does not have the same ABI as yours This is what happens when the only thing that interests people in charge is to code. Writing this means that these developers have no interest in making the system usable. This becomes more unbearable when such technically sharp people make up false technical arguments:
Depending on the version of the C compiler you use, different kernel data structures will contain different alignment of structures, and possibly include different functions in different ways (putting functions inline or not.)
Really, I wonder how the Microsoft developers, the GNOME guys, the glibc guys, and basically all library developers who know what they are talking about, manage to have stable ABIs despite the compiler changing all the time. One of the incredibly complicated techniques they use is to only rely on guaranteed functionality of the C specification, instead of doing many clever and funky hacks. For things more related to development itself, there are also incredibly complicated techniques like opaque structures, which save a lot of ABI breakage. Actually the only thing you need, if you design your ABI properly, is to be rigorous.
Depending on what kernel build options you select, a wide range of different things can be assumed by the kernel
This statement of a problem also contains the solution: stop making available build options no one cares about (remember, people install binary packages from their distributions) to guarantee stability instead. Instead of implementing the solution, kernel developers deliberately choose to deal with this insanity in the worst possible way: by encouraging it. Only lame developers need stable APIs But the nonsense doesn't stop to the ABI, which after all is merely a problem for distributors. To be more annoying to developers themselves, let s make up reasons to break the API all the time:
Linux kernel development is continuous and at a rapid pace, never stopping to slow down. As such, the kernel developers find bugs in current interfaces, or figure out a better way to do things.
Explained this way, it looks like a good thing. If you've done serious development or integration, you know that actually it is a problem. If you never slow down to look at what you've done, you're only going to add new bugs while you're trying to fix the others.
When they do so, function names may change, structures may grow or shrink, and function parameters may be reworked. If this happens, all of the instances of where this interface is used within the kernel are fixed up at the same time, ensuring that everything continues to work properly.
Everything is said. In the Linux kernel, changes are not done in a continuous way. Every non-minor change is going to have an impact on hundreds of different modules. The amount of work needed to accomplish these changes is absolutely insane. When a project like GNOME decides to phase-out an API, it takes several years before being actually replaced. In the kernel, it can simply happen between two minor releases, together with a rewrite of the whole ATA stack. The lolution Of course, Greg KH has a simple solution for you, in the What to do section.
So, if you have a Linux kernel driver that is not in the main kernel tree, what are you, a developer, supposed to do? Releasing a binary driver for every different kernel version for every distribution is a nightmare, and trying to keep up with an ever changing kernel interface is also a rough job.
No shit !
Simple, get your kernel driver into the main kernel tree.
Simple, isn't it? Unfortunately this is not going to happen while you are too busy adapting it for the constantly moving interfaces. This is quite feasible if you are writing a driver for an Ethernet network adapter, but for a video card this is another story, and for a virtualization layer? Well, the Xen developers have been struggling for years to integrate their technology in the kernel, and it is still far from done. No wonder why you get good drivers only for ethernet cards and not for 3D cards or even wifi chips. This doesn t go without other, worse consequences for users and distributors. Not everyone can afford to run Ubuntu, Debian unstable or Fedora, especially if they have simple requirements like not changing the whole system every 6 months. Corporate users using Debian stable, RHEL, SLES or Ubuntu LTS need a stable kernel that doesn t move, and that doesn t break everything when upgraded. The introduction of projects like DKMS to distribute drivers in a sourceful way while keeping them easily installable is a step in the good direction, despite being a workaround for a broken situation underneath. However these efforts are ruined by the constantly changing APIs that will go on requiring changes to the sources as well as the binaries. Why it can still work In fact, with such processes, you can wonder how it is possible that the system is actually usable. The answer is simple: while being smaller than many other projects, the Linux kernel has much, much more developers. A rough estimation shows that Linux has 2000 developers, many of which are paid full-time to work on it, to work on a 15 Mloc codebase. You can compare it with X.org, which has a few dozens of developers for 3 Mloc. Or with GNOME, 400 developers for 20 Mloc. With OOo, 150 developers for 10 Mloc. Yes, you need 10 times as many people to maintain the kernel code than for another comparable project. That's of course what happens when you keep them busy with a permanent refactoring of the code. This development method, while certainly having the advantage of bringing lots of innovation and fast integration of new features, is also driving away a very large portion of the open source community to work on rewriting Firewire stacks every 6 months. And as long as they are busy with this, they are not writing correct X drivers nor fixing bugs in the applications.

MJ Ray: your country continues to turn into Big Brother

Just received this comment from Jimmy Kaplowitz over in the US: “yuck, your country continues to turn into Big Brother: BBC News: Giant database plan ‘Orwellian’” Hell, yeah. That’s a reason why my support for the party is so weak at the moment. I’ve been working and writing against this since ST@ND and the Regulation of Investigatory Powers Act, which we were told was essential for crime-fighting, but actually helps gov.uk spy on parents making school place applications. Currently, I’m telling people Gov.UK Consults on Forced ISP Snooping: Please Say NO. If anyone has any doubt that the database state will be abused, look at some examples of Labour’s “terrorists” (at least, they had supposedly anti-terror measures used against them) since 2001:-
  1. Wolfie, an 82-year-old peace-campaigning heckler at a Labour conference
  2. Indymedia
  3. a Frenchman in Suffolk who downloaded the Anarchists’ Cookbook
  4. an Algerian pilot
  5. the country of Iceland
and that’s ignoring gov.uk’s frequent data leaks. Noodles replied: “There’s a quote somewhere that goes along the lines of “If the Tory policy is looking sane compared to what you’re doing, you’re doing it wrong” that’s been applied to the Labour fascist state stuff.” Our terrorism measures are a joke, with loopholes that gov.uk can drive a truck through. They should be repealed and clearly-limited ones enacted, shouldn’t they?

19 October 2008

Uwe Hermann: Falling on your back for fun and profit -- human airbag device

Fun stuff I just stumbled over: a personal/human airbag from Japan, supposedly meant for elderly people who might fall and injure themselves. Watch a video of the airbag in action on Youtube (no need for crappy Flash player, you can use youtube-dl for instance). Such a device could be a lot of fun I imagine; make it cheap enough and lots of people will buy it just for fun falling-on-your-back experiments :-)

Julien Blache: Digi AccelePort drivers updated to 1.3-14

Digi released a new beta release of the dgap drivers a couple of days ago, notably fixing the build issue (TTY flip buffer) with newer kernels. They now ship two versions of the driver, one suitable for 2.4 and 2.6 kernels up to and including 2.6.26, and one for kernels 2.6.27+. The appropriate version will be picked up automatically when the module is built, of course. APT source line, unchanged: deb http://debian.technologeek.org/ etch non-free The packages are still built for Etch; Sarge users should stay with 1.3.6 (unless running a newer kernel, but then a rebuild of the userspace tools is required). Feedback at the usual address.

Anand Kumria: How to get more developers.

Christian worries about how to grow the desktop share in the free desktop. Some recent events (Chromium, Geode) have led me to believe that … We've won … already. For example let's look at the reactions to Chromium (which I believe was released way too early): I've just picked three links but the trend is clear. The digirati want to use Linux and Mac OS X to evaluate new technologies NOT Windows. Sure, some of them still have those boxes around for “client work”. But that is not what they use day-to-day. Now let's take a look at Mozillas' Geode. Here is the Introduction to Geode. Look at the comments: In fact the outcry was enough for lack of Linux support that is merited a follow-up Common Geode Q&As:
Geode is meant as a temporary solution to allow websites to experiment with geolocation today. … Skyhook is built in. A side effect is that Linux isn't supported for the simple reason that Skyhook hasn't implemented Linux drivers. Although not ideal. …
A year ago this would not have happened. This amount of push-back from the digirati that the new technology they are supposed to look at and evaluate is not available on their platform would have been accepted as a matter of course. Now, though, they all expect it on their platform of choice. We've won, already. Whilst the largest user-base remains the Windows market, if you want the digirati to look at and evaluate what you have to offer it needs to work on Linux & Mac OS X. Another example, while I remember, is Adobe releasing Flash 10 for Linux on the same day as other platforms. Unheard of. So what does mean for Christian's question?
  1. Within 12 months it will be inconcievable for software to NOT be released on Windows + Linux (and maybe Mac OS X)
  2. Within 24 months we will have power users asking for Free Software (probably not Linux but more likely OpenOffice 3.x & Firefox) routinely
  3. Within 36 months Linux will be 10% of the desktop market. Bigger than Mac OS X. That is a lot of people you can not afford to ignore if you are a company, it will mean that ‘web-based’ will be more and more common — integration between desktop and the web will be key.

18 October 2008

Jeff Bailey: And we're back...

Umm, wow. So we extended the trip a week longer than expected to accommodate the hospitalisation, and wound up being able to finalise the house inspection, some banking, and get started on the insurance.

All of which means that the three of us have possession of our new place on December 1st!! Yay! We have to figure out if, tax-wise, a December move is the right thing for us, but I suspect it will be. The old timeframes aren't so important now, I guess. Moving still feels like the right thing to do.

The hospital bills came to about $10k for 3 days in the hospital, and one follow-up emergency visit. Our US insurance will cover this with no problem.

Thanks for all the well-wishes and support. I don't know how to reply to it in a meaningful way. Words suck and I wish I could hug you all in person. It's such a rollercoaster to have bought a place at the same time and be so excited by it.

When we have our move date, we'll probably have a wine and fruit party or something. =)

Next.

Previous.