Search Results: "ron"

23 October 2008

Joey Hess: anatomy of an atrocious announcement

Here are some tricks that you can use to ensure that your announcement to debian-devel-announce has maximal negative utility. hide who's talking Make sure that your announcement does not start off by making clear who's speaking. The announcement will have a From line with your name on it, and that's good enough. Everyone knows who you are, what groups you're a member of, and can guess what group you're speaking for ... or whether you're just speaking for yourself. Throughout the annoucement, be use to use terms like "We plan", "let us describe", "we are considering", "This is where our proposal comes in.", etc. Maybe in a footnote include a clue such as a mention of "the NM-Committee", but be sure to not sign the announcement with the name of any group or team or set of people, so that the reader is left guessing about who this "we" is all the way to the end and beyond. (And hey, "cabal" is a fun word.) weasel words Scatter throughout the announcement some vaguely worded digs at things you disagree with. The key here is to make clear what you really mean, while providing enough plausible deniability that you can deny having said it later. So you might say ...
Some time ago a few Developers thus went and pushed forward the "Debian Maintainer" status.
... Thus implying that this was a few malcontents who should be ignored. Sure, the whole project voted for this, and so you should be sure to allude to that vote somewhere else. Maybe in a different paragraph entirely, you could refer to "Debian Maintainers (DM) [GR-DM]". (It would be excessive to have that footnote refer to a bogus url like http://vote.debian.org/something.) the big lie Make sure that your announcement starts off all bright and cheery on the surface, with just a hint of deadly steel jaws underneath. Something like:
If you are an existing Debian Developer or Debian Maintainer, don't be afraid, we are not going to take anything away from you.
Natter on for many, many pages of trivilities, good ideas, bad ideas, half-baked ideas, conflicting acronyms, and useless footnotes. Then spring the trap:
In future this will be a list maintained by the NM-committee. At the time of migrating from the old to the new way, Ftpmaster will convert the existing DM-Upload-Allowed fields into that list, so there should be no interruption in your ability to upload.
And ability to upload is all that matters. Who cares about the regression back to the tyranny of unix permissions? Who cares that DDs won't be able to decide when they trust a DM to upload their package, or that DMs won't be able to sign up quickly and easily, or that DMs will have to petition a disinterested committe to change anything? Nothing will have changed, nothing will be taken away, nothing to see here, move along: The big lie.

Adam Rosi-Kessel: Republic National Committee Helps Obama Fundraising

The only plausible result from this RNC mailing, I think, is increased fundraising for Obama over the next few days:
Terrorists Dont Care Who They Hurt

"Terrorists Don't Care Who They Hurt"

For the nearsighted, the front of the mailing says “Terrorists Don’t Care Who They Hurt” and inside: “Why Should We Care What They Have to Say? … Barack Obama. Not Who You Think He Is.” I would be surprised if the Obama campaign doesn’t get a measurable bump in donations for every mailing like this. What I really wonder is how intelligent Republican party members tolerate this sort of thing. Aside from Colin Powell, are other high-IQ Republicans voting against their party because of these tactics? Or do they just believe any means justify the ends? My pet theory as to why these attacks are useless (at best) to the GOP is that they don’t resonate with anything the voters are seeing. The GOP implies Obama is a muslim terrorist who is inexperienced and risky and also an extreme liberal/socialist left-winger (I haven’t heard “communist” yet from the mainstream), while what the electorate sees in the debates, ads, and campaign appearances is an uncannily steady, balanced, articulate individual who comes across as a moderate on most issues (note Obama’s theme of returning to Reagan-era taxation levels!) At least with the “flip-flopper” attacks (and maybe even the Swift-Boat attacks) on John Kerry, there was something people could see that somewhat corresponded with the attacks, even if the underlying facts didn’t actually pan out. Here, the chasm between the nature of the attacks and the visible target is so great, they could only help energize those who are already hard-core partisans. The lesson here is that if you want to launch misleading, baseless attacks against your opponent, at least find some kernel of truth to base them on. On a similar note, I agree with Aaron Swartz that ACORN deserves our financial support. Finally, I predict a bunch of astroturf comments to this blog entry. I won’t bother to respond if the comments fail to say anything intelligible. Technorati Tags: , , , , ,

Adam Rosi-Kessel: Republic National Committee Helps Obama Fundraising

The only plausible result from this RNC mailing, I think, is increased fundraising for Obama over the next few days:
Terrorists Dont Care Who They Hurt

"Terrorists Don't Care Who They Hurt"

For the nearsighted, the front of the mailing says “Terrorists Don’t Care Who They Hurt” and inside: “Why Should We Care What They Have to Say? … Barack Obama. Not Who You Think He Is.” I would be surprised if the Obama campaign doesn’t get a measurable bump in donations for every mailing like this. What I really wonder is how intelligent Republican party members tolerate this sort of thing. Aside from Colin Powell, are other high-IQ Republicans voting against their party because of these tactics? Or do they just believe any means justify the ends? My pet theory as to why these attacks are useless (at best) to the GOP is that they don’t resonate with anything the voters are seeing. The GOP implies Obama is a muslim terrorist who is inexperienced and risky and also an extreme liberal/socialist left-winger (I haven’t heard “communist” yet from the mainstream), while what the electorate sees in the debates, ads, and campaign appearances is an uncannily steady, balanced, articulate individual who comes across as a moderate on most issues (note Obama’s theme of returning to Reagan-era taxation levels!) At least with the “flip-flopper” attacks (and maybe even the Swift-Boat attacks) on John Kerry, there was something people could see that somewhat corresponded with the attacks, even if the underlying facts didn’t actually pan out. Here, the chasm between the nature of the attacks and the visible target is so great, they could only help energize those who are already hard-core partisans. The lesson here is that if you want to launch misleading, baseless attacks against your opponent, at least find some kernel of truth to base them on. On a similar note, I agree with Aaron Swartz that ACORN deserves our financial support. Finally, I predict a bunch of astroturf comments to this blog entry. I won’t bother to respond if the comments fail to say anything intelligible. Technorati Tags: , , , , ,

Matthew Palmer: Porting debian-installer to the DNS-323

I've just started a new job, so I decided to take a few days holiday in between and work on something that's been on my todo list for way too long now -- making the debian-installer work on the D-Link DNS-323. After a couple of days of work, I've just finished off the install of Debian on my DNS-323, which I initiated by loading a firmware image into the web interface of the stock firmware, and then SSHing into the machine at the appropriate moment. What this means is that anyone with a DNS-323 can install Debian on it. I think that's worthy of a small, quiet "woo hoo!"... Looking back now, it seems to have been a fairly simple and straightforward operation, but I can remember that it didn't look so simple when I started a week ago. So, in the interests of perhaps short-cutting someone else's adventure into the world of d-i hacking, I'm writing everything up while it's fresh in my mind (the benefits of being on holidays -- plenty of time to write long blog posts). Alternately, if anyone has a random device they'd like to see d-i on but doesn't have the inclination to do it themselves, feel free to send me one and I'll do it for you. Before I begin, I've got a few "shout outs" (I think that's what the cool kids are calling it these days) to people who've helped me along the way. First off, Martin Michlmayr has answered innumerable dumb questions from me, and I think wrote most of the parts of d-i that I've extended -- it's fair to say that I'd have had no hope of doing this without him. Thanks also to the people in #debian-boot and #debian-arm for answering questions and putting up with my frequent bouts of self-congratulation whenever I passed the most miniscule milestone. Finally, everyone who has put work into d-i has my thanks, too -- it's not perfect, but it's a damn sight better (both to use and also to hack on) than any other installer I've ever met. So, without further ado, off we go... Work out how to get a console Unless you are a guru of unbelievable skill, you will undoubtedly not do everything quite perfectly the first time, and you'll want to see the kernel/init boot messages, and maybe in some cases try some investigation and diagnostic work. For that, you're going to have to have some sort of console access. If the device doesn't have a full screen/keyboard thing going on, the chances are you'll almost certainly end up with a serial console. A few devices have a network management thingo that will give you a console, usually over VNC or some sort of horrid Java applet, but I've only ever encountered that in rackmount servers, which have a keyboard/video interface anyway... In theory, I suppose, the kernel could provide a network console, but I've never heard of such a thing being attempted, and Google is not helping me out. I might add that to the todo list as a future hacking project... On the DNS-323, it was a serial console that was needed. Like a lot of the embedded devices I've seen, it didn't quite talk RS-232, so I needed an adapter (I got a TTL-232R-3V3 USB-to-3.3v-serial adapter from a local electronics supply place, which worked perfectly with Linux as soon as I plugged it in). I needed to solder a few wires to a connecter inside, and connect that up to the USB doohickey, but it all came good in the end. The chances are someone's done a similar job for your device of choice, so a search for things like "devicename serial console" will almost certainly pay off. Work out how to load and boot a kernel/initrd Your development work will involve booting weird kernels and initial ramdisks to make everything work; you need some way of stuffing those into the device and booting from them. Everything has a boot loader of one sort or another; working out what you've got and how to make it sing and dance is the ultimate quest here. How this is done will vary greatly from device to device; the WRT-54GL, for example, waits for a tftp connection for a couple of seconds on startup, which you can use to send a new firmware; many other devices have a boot loader that you have to explicitly control (via the console), but it will make a tftp connection out to another machine if you ask it to. Some devices you just need to load the data in via serial cable (at 115200bps it's bearable; at the 9600bps I've heard some devices work at, it would be tres painful). The DNS-323 runs uBoot as the boot loader, which will do a tftp request by default. However, D-link, or Marvell, or someone in the decision-making chain for this thing decided to knobble it and removed most of the useful commands, so sending the thing via serial port is the only option. Since writing to flash takes a pile of extra time, I found (eventually) that you can just load the data into RAM somewhere and boot from there, without needing to write it to flash (also handy to avoid wasting precious write cycles), since you can tell uBoot to boot a kernel located anywhere in the address space. Obviously you need to reload the kernel/initrd on every boot if you don't write it to flash, so if you've got a "stable" kernel or initrd that you think you'll be using for a while (when you're testing the other item) you should flash it then. One note on uBoot, that took me far too long to work out -- you can't just stuff a kernel fresh off the Makefile in and assume it'll work -- uBoot needs a special header prepended that tells it all about what you're loading (architecture, file type, compression method, etc). You "wrap" a file in this uBoot header with the "mkimage" command (in the uboot-mkimage package in Debian). Incidentally, if you need to remove the uBoot bits from a file, just strip the first 64 bytes off -- that's all the uBoot header is, so something like this will do nicely:
 dd if=file.uboot of=file bs=64 skip=1
A final uBoot warning: it's fairly important to specify a "load address" for the files; if you get them wrong, your kernel and initrd may or may not boot (I never quite got the hang of why some wrong values worked and some didn't). If your device uses uBoot, and you have the ability to extract the uBoot-enabled kernel and initrd from a vendor-provided firmware image, you can use mkimage -l to inspect what all the values need to be for your own kernels and initrds. Just stick with them. Does Debian support your hardware? Obviously, if you want to run Debian on something, Debian has to support that hardware. The easiest way to test this out is to try an install. For this, you'll need to retrieve an appropriate kernel and initrd pair for the installer. Some arches, like i386 and amd64, only have one kernel and initrd for the installer, so you just have to choose if you're going to netboot, or do a full CD install, or whatever. Other arches, though, like arm, have subtly different images for different machines. There are two reasons they vary: firstly, they often need to split into subarches, where they all share a general CPU architecture but there are enough differences in the hardware that they're not all supportable by the same kernel. At a more fine-grained level some different machines of the same sub-architecture need differently formatted firmware images to boot. On ARM, in particular, it's pretty common to need to prepend a "device ID" to the kernel image so that the kernel knows what sort of machine it's booting on, which means that every machine typically needs it's own kernel image, regardless of any other issues with firmware formats. Choosing the sub-architecture should be relatively straightforward, if you know much about the machine you're targetting. Picking a machine type can be a bit harder, but again, if you know what your boot loader needs, it shouldn't be too hard to find a kernel/initrd from the existing machines in your subarch that you can mangle to fit. Load up your kernel/initrd into the device, boot it, and see how far it gets. You might get lucky, and the kernel inits, your hard drives are detected, your NIC is initialised, and you're off and running without further ado. Or you might not be so lucky. The worst case is that the kernel doesn't even get as far as loading the initrd -- it might not print anything at all, or the kernel might hang part way through the boot. This is likely to require a fair amount of debugging fun, as initial boot failures get a bit tricky. A better situation is where the kernel boots OK, and d-i starts, but something in the install process falls over. This can either be a kernel/driver problem, or a d-i problem. Either way, since d-i has started, you're likely to be able to get a shell and do some debugging. In order to make d-i work, you need to have somewhere to install to (disks, usually) and a source of packages to install from (CD/DVD if you're on a suitably large device, or the network otherwise). These are the two key ingredients you need; most everything else is stuff that ranges between "nice to have" and "critical" for operational purposes, but which has absolutely zero impact on your ability to do the install, so we'll ignore it for now. For the DNS-323, I had a few kernel problems (which I'd been warned about by Martin before I started) -- the NIC wasn't initialised with it's MAC address, which meant it was running around my Ethernet screaming "I am 00:00:00:00:00:00! Ph33r m3!", and the SATA controller just didn't seem to exist at all. These were kinda critical to doing an install, so it was time to get hacking to make those work. There were also a couple of parts of d-i that needed tweaking -- the preseeding code didn't support the device, and I needed to package a tool to build a "complete" uploadable firmware image and teach d-i how to use it. Apart from that, though, things were looking pretty good. Make the Debian kernel support your hardware Once you've worked out what hardware you need to make work, you need to dig into the kernel and fix it. Sometimes, if you're really lucky, a kernel version newer than the one currently in Debian has better support for something, and you can backport the fix / feature improvement. Other times, there's something similar already in the kernel for a different device and you can crib off that (I did this extensively for my little fixes). Then there's the "ask an expert" school, where you find someone who knows what they're on about and pick their brain (I did a bit of that, too). Finally, there's plain old "research and development", for which there are piles of documentation already out there. My changes for the DNS-323 came to 121 added lines and 16 deleted lines, most of which was cribbed-and-tweaked from other parts of the kernel. I might have written 10-20 lines of code "from scratch". It's not always scary-hard. Cross-compiling your own kernel One thing that I think needs mentioning in the kernel hacking department is cross-compiling a modified kernel for testing purposes. Chances are your main workstation has a lot more grunt (and build tools) than whatever you're trying to get Debian onto, so cross-compilation is the way to go. You need to first grab a copy of the source you want to build (from apt-get source linux-image-blahdiblah, or a release tarball, or Linus' git repo, it doesn't really matter) and a cross-compilation toolchain. For my cross-compilation toolchain, I just added this to my sources.list:
 deb http://www.emdebian.org/debian/ unstable main
then I ran apt-get update, and installed these packages:
	binutils-arm-linux-gnueabi gcc-4.1-arm-linux-gnueabi
	gcc-4.3-arm-linux-gnueabi-base libc6-dev-armel-cross
Kudos to the emdebian team for making that such an utterly painless exercise, by the way -- the days of needing to faff round with gcc sources to get a cross-compilation toolchain are apparently over. Once you've got a source tree and a cross-compiler available, you can run any of the usual kernel make commands (like make zImage or make oldconfig), as long as you've got a couple of make variables defined. In my case, I used:
 ARCH=arm CROSS_COMPILE=arm-linux-gnueabi-
ARCH tells the kernel what architecture to build for (useful in all make commands because not all arches have the same config options in make menuconfig, for example) and CROSS_COMPILE tells the build system what to prefix the calls to gcc, ld and such with (if you installed the same cross-compiler packages as I did, you'll find it installs files like /usr/bin/arm-linux-gnueabi-gcc, so you can ls /usr/bin/*gcc to find out what you'll need for the CROSS_COMPILE setting). Hack d-i to make things work In addition to tweaking the kernel, it's quite possible that the internals of d-i might need some tuning. It's possible that d-i might need to be "taught" to recognise the hardware in your device, or you'll need to extend some aspect of the installation process to complete the installation. In order to hack d-i, you need to know a bit about how it works and how it is structured, so let's run through that first. The entireity of d-i is based around udebs, which are packages of the same format as "real" debs, but which have been built differently (against a small libc, for instance) and have had everything that isn't absolutely necessary (docs, non-essential scripts, extra functionality, etc) stripped out of them to create the smallest possible packages. There are a variety of different udebs -- most are derived from otherwise normal, law-abiding citizens of the archive, like busybox (basic unix utilities like cp and ls), the shell that's used in the installer, parted, various libraries, and so on. Others are "special", and have no big brother in the archive. These are mostly "task" udebs, that define what actually happens in the installer. When they're "installed", they have special fields in the package metadata that define where in the install sequence they go, and they end up on the menu you see if your installation goes a bit pear-shaped. The tricky thing about udebs is that, as small as they are, there's too many of them to be able to have them all available when the system first boots, so even the "setup" of the installation gets done in several stages. First, the kernel and initrd boot. The initrd contains just enough stuff to get access to the rest of the udebs, and nothing else. Even this early stage is built from udebs -- there's lists of what goes into the initrd for various architectures and flavours of installer under installer/build/pkg-list in the d-i SVN. If you need an extra udeb in your initrd, that's the place to specify it. Once the initrd has gotten access to a source of packages (off a CD, over the network, telepathy, whatever) it can drag the rest of the packages it needs from there. Space is still an issue at this point, though -- there's no diskspace yet to put anything onto, so everything needs to fit into a ramdisk. Don't go loading anything you don't need. The packages loaded at this stage are what defines most of the steps that will be done for the rest of the install, so once all those extra udebs are loaded, the remainder of the install kicks off and does it's thing. Different architectures and install methods have different requirements, too, which complicates matters. For example, when you're installing on a regular PC with screen/keyboard, you can guide the install using that. However, if you've got no screen, how can you control the install? There's serial consoles, but requiring everyone to void the warranty (more) on their little embedded doodad by installing a serial console cable to install Debian isn't so hot. So, lacking a terminal, how do you control the install? Well, on install flavours for devices that might be in this situation, you can SSH into the installer and complete it that way. That feature is, like everything else, provided in a udeb -- called network-console, as it happens. But what IP address do you connect to, and what network settings does the device use? Again, without a console you can't configure your network, and not everyone has a DHCP server (and even fewer people have a DHCP server that will hand out an address that they'll know in advance, or be able to find out). So, there's a udeb (called oldsys-preseed) that does it's best to configure your network based on the settings already in the nvram for your device. In short, there's a lot of tricky, well-thought-out shit going on inside d-i, and you might have to deal with a little bit of it. The nice thing is that by now I expect most of the really hard problems have been solved, so with a bit of luck you'll only ever have to do minor tweaking. In general, all of the packages that are "d-i specific" live in packages in the d-i SVN. If you need to modify a udeb, you need to rebuild it and put it in installer/build/localudebs/ (creating it first if need be) and every time you change it, you need to delete installer/build/apt.udeb/cache/. To give a concrete example (from the ever-present DNS-323), I had to modify oldsys-preseed so that it would get the details from the nvram that the stock firmware uses to configure itself. This, in turn, required that the minix-modules udeb be included in the initrd, because the DNS-323's nvram is actually a small minix partition stored in flash. Painfully, this required some more hacking, because the armel d-i kernel wasn't configured to generate minix-modules. That might need some explanation... Since building the kernel is a fairly massive undertaking, CPU-wise, instead of building all the udebs from scratch there's a package called kernel-wedge to split a real linux-image package into lots of little udebs. There's a separate source package for each architecture, linux-kernel-di-$ARCH-2.6, that contains the configuration data for kernel-wedge to specify what udebs need to be built, and what goes into each of them. A small tweak to the armel package and voila! I had a minix-modules udeb to include in the initrd. With the config stuff I'd done in the d-i build itself and my changes to oldsys-preseed, we were done with d-i. Tell d-i how to build your bootable artifacts If your device needs a "different" install image from other similar devices, you need to write some makefile rules to build those files. Look under the installer/build/config/$ARCH/ directory to see the various rules files available. I had to package a new tool to create uploadable firmware images for the DNS-323, along with adding a rule to use that tool to build images. None of it was particularly difficult. Polish the diamond Getting d-i working is only part of the battle; it's quite likely that there will be other hardware in the machine that needs to be supported. This could involve kernel-level stuff to interface to the hardware, or userland tools to manage the hardware. There's several bits in the DNS-323 that still need work; I can't control the LEDs on the front panel, nor the fan, and I can't use the on-board temperature sensors yet. But, I do have Debian running on the thing, which is what I was shooting for. Examples If you're interested in exactly what I did, there are a number of bugs in the BTS that contain the patches I made:

Stefano Zacchiroli: from Vim to Emacs - part 1

One month with Emacs and counting - Part 1 Straight to the point: since mid September I've been using Emacs, trying to evaluate whether I was willing to switch from Vim to it. Yup, that's true, me (user of Vim since the day I've started using GNU/Linux 10 years ago, (not so) active maintainer in Debian of vim and related packages, author of some popular Vim extensions and of vim-addon-manager) it's considering switching to Emacs. What is worse is that I've de facto already switched. May jamessan forgive me. OK, that was the hardest part to write, now ... This choice will have an impact on me: partly because I'm quite sure several fellow DDs will start making fun of this story :-) , and partly because as many geeks I've always had strong opinions on the editor war. Switching side ain't easy. Hence, I've decided to write a small (2-3) series of blog posts on the issue, to future memory. The post you are reading is about why, since a few months ago, I wasn't willing to give Emacs a try, and how I've changed my mind. Why I wasn't willing to give Emacs a try, but then changed my mind I love knowing tools which can improve my workflow, no matter the task. Editors happen to be at the intersection of many tasks, hence knowing well his own editor and feeling satisfied about it is quite important. This is even more so for free software hackers which, for a reason or another, happen to use the very same editor for a lot of different tasks; in my case: programming, conf file hacking (i.e., playing the sysadm role), mail writing, and everything in between. I've started using vi on Solaris around 1998, then switched to Vim starting from version 4.0, and then followed all its releases up to 7.0. I consider my "Vim karma" to be quite hight and I've delivered several talks about using Vim for LUGs and other free software-related events. Still, I've always been hit bit several minor nuisances and glitches of Vim which couldn't be fixed by simple configuration tuning or add-on implementations (more on this in a future post). In the quest for the perfect workflow, those glitches have made me try several times other editors hoping for more satisfying work environments. Of course Emacs was one of them, which I've tried repeatedly during the years; the last time was circa 2006. For one reason or another, after a few days of experiments, I've always decided to give up with it. Why? Various reasons, listed below, together with an explanation of what (I think) has changed since then.
  1. Poor Unicode/multibyte support. Support for whatever Unicode encoding was close to null. It was not available in the legacy editor, and you had to use something called MULE, to be specifically enabled. That was so 60's, and rather hard to use. Now (apparently starting from Emacs 22.1, June 2007) finally you have built-in Unicode support and the needed two variables for setting the input encoding and the file encoding, which is basically all you need. Looks easy once you have it ...
  2. Lack of "modern" UI toolkit support. Yes, I do live in a console and invoke my editor from it, but nevertheless for long coding sessions I do also love having a nice GTK+ window containing my editor using settings which smoothly integrated with the graphical settings of my desktop environment. Maybe I'm getting old, maybe GNOME has changed my perceptions, but why the heck I had to tolerate different monospace fonts in my gnome-terminal wrt my GUI editor I never understood. Firing up Emacs was like getting back in time of 15 years, and if a geek having a maximized terminal on most of its workspaces had this feeling, then something was really wrong. This has changed: starting again from Emacs 22.1, the editor has frown support for that "bleeding edge" technology called GTK+. Finally you can use Pango font faces and avoid the time travel feeling when switching from a terminal to your GUI.
  3. "Dead upstream" syndrome. As we all know by experience, choosing a free software product is not only a matter of evaluating bugs and features, there are other environmental factors. A notable example is how active upstream is. A few of us will use a product affected by the "dead upstream" syndrome. That's exactly what I was feeling about Emacs. The first argument I can offer to back that feeling is again the Emacs release history (noted that "small" hole between 2001 and 2007? Rest In Peace release early, release often ...). The second argument is the bug tracking system: there was none, bug discussions happened only on the development mailing list and the bug tracker was a text file on CVS! (Yes, Vim doesn't have a proper BTS either, but I was looking for reasons to switch from it and hence looking for something better in several respects.) That has changed completely, as I discovered only at this year DebConf thanks to gismo. 2007 has enjoyed 1 Emacs release and 2008 already two. They do have a BTS now and guess what? is dear old debbugs. According to my first experiences as Emacs bug reported the maintainers are very reactive and also friendly in dealing with bug reporters. Finally, also keeping up with snapshot releases in Debian is entirely trivial, thanks to Romain's repository.
  4. Poor performances & memory consumption. This goes in the direction of probably the most popular joke used by Vim zealots in the editor war: Emacs is a nice operating system, but lacks a good editor, or something such. In the past I had the impression that starting Emacs was really slow and that generally the memory footprint was too high to be acceptable. You know what: starting a full-blown gvim with a good deals of add-ons (and I used quite a lot of them, that's why vim-addon-manager was born) is not that faster. And on the contrary, firing up an emacsclient is way faster than starting up a new gvim instance. Regarding memory consumption Emacs is still "gaining" the battle. Still, one has to keep in mind the different workflow: with Vim you are encouraged to enter and exit the editor (not without drawbacks, as I'll discuss in a future post), while with Emacs you always use the same one. It turns out that via emacsclient the resulting feeling is the same as having multiple editors (as you can fire them up as separate windows or in separate consoles), still sharing all user memory. A final comment on memory consumption while looking at my top sorted by decreasing RSS memory: the top process is xulrunner (from the browser) with 356Mb, then Xorg (159M), pidgin (128Mb !!!!), trackerd, gnome-terminal, gnome-panel, ..., Emacs shows up with 51 Mb at the 7th position. Yes, it is not the thinnest editor in the world, but on average machine it doesn't really matter at all.
In the next post: how to switch from Vim to Emacs, without loosing your mental sanity. (Whether I did succeed wrt sanity is up to the reader to judge.)

22 October 2008

Clint Adams: MDE, KDE, ODE, CDE, DSA, goose, badger, snake

In the olden days, things were a bit simpler. Oh, things were far from perfect; we didn't all have the same levels of access. We all had access to the machine with the ftp archive master, but Only a few people had access to the mailing list server, and only a few people had root (though not all of them were German). I actually had root on a couple of buildds until some guy named Ryan Murray appeared out of nowhere and disabled my accounts. I remember wondering, at the time, who he was and how he had gotten root on everything. As the years went by, the disparity grew. Like the lie told by the illuminati of post-9/11 thinking, things need to be kept safe, so access started to be less of an entitlement and more of a needs-only privilege. It just so happens that you don't need to do anything. However, the people who actually deserve the access can provide alternate services for you in case you want to try something you don't deserve access to. Of course it won't be as good, and if it breaks you may be called an impatient ingrate if you complain. Then if you want something else, you are asked to justify it. It is extremely condescending for a power-hungry, power-hoarding person to demand to know why someone should have access to something. The two main factors in gaining power are such a craving and cronyism, and if you remember that power is relative, you can see why a power-hungry person would not wish to participate in an egalitarian society, and why anarchy is unstable and falls easily to syndicalism. Back on the bus, we now have more layers of access, and thus we end up with more classes of people. As people and machines multiple, there are more opportunities to deny people access to machines, and more instances in which one could inquire why someone needed access to something. In this new Enlightenment, it is not just the power-mad asking the question, but also some hangers-on and other people who do nothing useful. There was some overlap of the two groups. Like everything else that doesn't get struck down violently and immediately, these attitudes become the standard, and people insist that any other choice would lead to instantaneous destruction of the universe. Look at how Anthony Towns redefined the meaning of experimental to work around a technical shortcoming. Now, instead of acknowledging the fact that there's a major deficiency in the release process that makes it inconvenient to upload packages to unstable during a freeze, and trying to fix it, we all mostly misuse experimental. Now we couldn't log into ftp-master, and we couldn't log into half the other machines either, but we could always upload. Even when the release team was giving us the bad advice not to upload, we could still do it. This vexed them, so the privileged ftp-team granted the privileged release team additional privileges. Can you guess what they were? I don't think I could. It's practically unfathomable to me. The release team can block a developer's right to upload. This is the fundamental building block of the whole kit and caboodle. Everything is predicated on this basic action, and their new privilege is the power to take that away from us. Yet this was greeted with very little objection, probably because of the people involved and vague promises of well-meaning and non-misuse. However, there is a simple axiom which applies to all of this:
If you impede me doing something I want to do, you are an asshole.
So now we can still upload (except when a bribe-loving ftpmaster is being petty or when the release team is expediting a transition) and we can still vote, and we can log into a machine or two. We're running low on powers to take away. Maybe we could create an even lower class of citizen, some kind of undermaintainer without any voting rights. Whee. DM was born. Instead of fixing the problems of class inequality, we created another class. Fantastic. Why stop there? Why not create more? Clearly nobody wants to work toward an egalitarian culture, so we might as well make it like a game where you can hop from level to level. Then you can go to society parties and brag that you are a DVMRP-Q White Belt Green Stripe with a concentration in Taiwanese Bug Reporting, and that after a 6-month wait you can make a lateral move to second-chair cantor of Der Process under the wavy waves. As always, making the constructive suggestion to take things in the exact opposite direction will be called out as unconstructive.

Jeff Licquia: HDTV Still Not Ready Yet

So you put off buying a high-def TV for years, because you weren’t sure they had gotten all the standards right. You recently gave in, thinking that the coming shut-off of analog broadcast TV in February meant that they had to have their technology figured out by now. Of course, you were wrong:
CableCARD devices have generally supported only one-way access to cable systems, but their long, winding journey toward full two-way communications is finally coming to an end. Panasonic has announced that it is at last shipping new HDTVs enabled with tru2way technology to the two US markets where they can actually be used.
So what’s the main thing you’re supposed to get with tru2way?
This means that you can walk out of a retail store with a tru2way-enabled HDTV, plug it in at home, and have immediate access to basic features like an on-screen guide and on-demand content.
In other words, we are just now starting to see HDTVs that can just plug into the cable jack and work, without an add-on cable box and all the limitations that implies, right? Well, not really.
All tru2way-compatible devices will have a CableCARD slot built into them to facilitate the decryption of protected content, though details are still sketchy as to how this system will work with devices like PVRs. Physical CableCARDs will apparently not be needed to access basic two-way services and non-encrypted channels.
Meaning that, in order to get anything you can’t get already with broadcast TV (”non-encrypted”), you still need a cable company tech to come out and install the CableCARD. And they don’t know how all of this will integrate with the new video recorders like TiVo. Why is this so hard? It’s producer paranoia. If they don’t play these games, you might watch some show for free, or share it so others can watch it for free, instead of… well, watching it for free live. And you might cut the commercials out, instead of… cutting the commercials out by getting up for more chips during the commercial breaks. (But that’s stealing, so you shouldn’t do that either.) Our family keeps edging closer to deciding to get a HDTV. But then I see stuff like this, and notice that the old tube TV still works fine…

Russell Coker: Upgrading a server to 64bit Xen

I have access to a server in Germany that was running Debian/Etch i386 but needed to be running Xen with the AMD64 version of Debian/Lenny (well it didn’t really need to be Lenny but we might as well get two upgrades done at the same time). Most people would probably do a complete reinstall, but I knew that I could do the upgrade while the machine is in a server room without any manual intervention. I didn’t achieve all my goals (I wanted to do it without having to boot the recovery system - we ended up having to boot it twice) but no dealings with the ISP staff were required. The first thing to do is to get a 64bit kernel running. Based on past bad experiences I’m not going to use the Debian Xen kernel on a 64bit system (in all my tests it has had kernel panics in the Dom0 when doing any serious disk IO). So I chose the CentOS 5 kernel. To get the kernel running I copied the kernel files (/boot/vmlinuz-2.6.18-92.1.13.el5xen /boot/System.map-2.6.18-92.1.13.el5xen /boot/config-2.6.18-92.1.13.el5xen) and the modules (/lib/modules/2.6.18-92.1.13.el5xen) from a CentOS machine. I just copied a .tgz archive as I didn’t want to bother installing alien or doing anything else that took time. Then I ran the Debian mkinitramfs program to create the initrd (the 32bit tools for creating an initrd work well with a 64bit kernel). Then I created the GRUB configuration entry (just copied the one from the CentOS box and changed the root= kernel parameter and the root GRUB parameter), crossed my fingers and rebooted. I tested this on a machine in my own computer room to make sure it worked before deploying it in Germany, but there was still some risk. After rebooting it the command arch reported x86_64 - so it had a 64bit Xen kernel running correctly. The next thing was to create a 64bit Lenny image. I got the Lenny Beta 2 image and used debootstrap to create the image (I consulted my blog post about creating Xen images for the syntax [1] - one of the benefits of blogging about how you solve technical problems). Then I used scp to copy a .tgz file of that to the server in Germany. Unfortunately the people who had set up that server had used all the disk space in two partitions, one for root and one for swap. While I can use regular files for Xen images (with performance that will probably suck a bit - Ext3 is not a great filesystem for big files) I can’t use them for a new root filesystem. So I formatted the swap space as ext3. Then to get it working I merely had to update the /etc/fstab, /etc/network/interfaces, and /etc/resolv.conf files to make it basically functional. Of course ssh access is necessary to do anything with the server once it boots, so I chrooted into the environment and ran “apt-get update ; apt-get install openssh-server udev ; apt-get dist-upgrade“. I stuffed this up and didn’t allow myself ssh access the first time, so the thing to do is to start sshd in the chroot environment and make sure that you can really login. Without having udev running a ssh login will probably result in the message “stdin: is not a tty“, that is not a problem. Getting that to work by the commands ‘ssh root@server “mkdir /dev/pts”‘ and ‘ssh root@server “mount -t devpts devpts /dev/pts”‘ is not a challenge. But installing udev first is a better idea. Then after that I added a new grub entry as the default which used the CentOS kernel and /dev/sda1 (the device formerly used for swap space) as root. I initially used the CentOS Xen kernel (all Red Hat based distributions bundle the Xen kernel with the Linux kernel - which makes some sense). But the Debian Xen utilities didn’t like that so I changed to the Debian Xen kernel. Once I had this basically working I copied the 64bit installation to the original device and put the 32bit files in a subdirectory named “old” (so configuration can be copied). When I changed the configuration and rebooted it worked until I installed SE Linux. It seems that the Debian init scripts will in many situations quietly work when the root device is incorectly specified in /etc/fstab. This however requires creating a device node somewhere else for fsck and the SE Linux policy version 2:0.0.20080702-12 was not permitting this. I have since uploaded policy 2:0.0.20080702-13 to fix this bug and requested that the release team allow it in Lenny - I think that a bug which can make a server fail to boot is worthy of inclusion! Finally to get the CentOS kernel working with Debian you need to load the following modules in the Dom0 (as discussed in my previous post about kernel issues [2]):
blktap
blkbk
netbk It seems that the Debian Xen kernel has those modules linked in and the Debian Xen utilities expect that. Currently I’m using Debian kernels 2.6.18 and 2.6.26 for the DomUs. I have considered using the CentOS kernel but they decided that /dev/console is not good enough for the console of a DomU and decided to use something else. Gratuitous differences are annoying (every other machine both real and virtual has /dev/console). If I find problems with the Debian kernels in DomUs I will change to the CentOS kernel. Incidentally one problem I have had with a CentOS kernel for a DomU (when running on a CentOS Dom0) was that the CentOS initrd seems to have some strange expectations of the root filesystem, when they are not met things go wrong - a common symptom is that the nash process will go in a loop and use 100% CPU time. One of the problems I had was converting the configuration for the primary network device from eth0 to xenbr0. In my first attempt I had not installed the bridge-utils package and the machine booted up without network access. In future I will setup xenbr1 (a device for private networking that is not connected to an Ethernet device) first and test it, if it works then there’s a good chance that the xenbr0 device (which is connected to the main Ethernet port of the machine) will work. After getting the machine going I found a number of things that needed to be fixed with the Xen SE Linux policy. Hopefully the release team will let me get another version of the policy into Lenny (the current one doesn’t work).

21 October 2008

Matthew Palmer: Load Balancing FTP Servers

While it's a protocol we all love to hate, and wish it would go away and die a quiet death, sometimes someone has a good reason for supporting FTP. (In my case, it's usually because that reason has a lot of money to provide to my employer). But what if you want your FTP servers load balanced (because downtime is for suckers)? Obviously, you need to have some sort of shared filesystem behind all your load balanced FTP servers, but that's a rant for another day (but let me just say: if you're running GFS, especially on a busy site, may $DEITY have mercy on your soul). Because of the silly random port games that FTP likes to play, it's not a trivial job to load balance an FTP service amongst multiple hosts. You can't just throw a round-robin on port 21 and hope to get anything useful going. Ironically, while "active" FTP is a pain in the arse for most people, it's a lot easier to load balance, because the initial connection comes in on a well-known port which can be load-balanced, while all the other "random" connections are server-initiated and your standard routing/NATing/whatever you're doing for outgoing connectivity should be able to handle it. However, passive FTP is a bigger pain. The problem is pretty simple to understand -- the FTP client will want to connect to a high port on the "virtual" (load-balanced) IP, and without serious traffic sniffing, your load balancer won't be able to map that incoming connection on port 54321 to the real server that actually wanted the connection. The solution, thankfully, is fairly straightforward. You allocate a range of high ports (maybe a thousand or so) to each real server, with a different port range for each real server. You then tell the load balancer to send all traffic for each range of ports to the corresponding real server, and then configure that real server to put all it's passive connections into that port range (I'm fairly sure most FTP servers can support that bit of config). So, unfortunately, you can no longer disallow FTP support on the basis that you can't load balance it. No shortage of other reasons, though.

Rob Taylor: Reversible debugging one step closer

I just stumbled upon Chronical Recorder. I’ve been waiting for something like this a long while. How did I miss it getting released?! Thanks Novell!

MJ Ray: Monday Machine-Mangling Madness, or: Server2 on fire!

//www.flickr.com/photos/dominicspics/1901310469">dominicspics</a> (cc-by)
Bonfire Onlooker by dominicspics (cc-by) Of all the things I wanted to do yesterday, testing TTLLP’s disaster recovery plan was not one of them. I often suggest customers make disaster recovery plans, but I don’t like having to use them. One of our server farms suffered a serious hardware failure and slowly died during the day. (It didn’t catch fire - the title is a silly reference to lp0 on fire.) The current stinging electricity bills means that there’s an emphasis on power-saving now that wasn’t there 4 or so years ago, when we still built servers ourselves, before WEEE and cheap big-brand systems. Today’s servers aren’t quite as resilient, because each extra piece of hardware in use takes extra power. There are some much cleverer tactics now, but they don’t catch everything before it goes “bang” and we can’t just bring the “spare” hardware into action if it’s not installed yet. Replacement hardware will be installed as soon as possible, as well as some extra hardware that will help protect against a similar failure in future. The slow failure of the server also means that we lost some emails, voicemails and instant messages from 10am to 2pm yesterday. If one was yours, please resend. Most services have been rerouted or temporarily replaced with help from friendly associates. Three important services that haven’t yet are: 1. our financial system (which gets updated in batches anyway), 2. our task tracking (which is newer than our recovery plan and I need reminding about its backups, but at worst all the current data exists in other places) and 3. our newsletter server. I’ve copied the customer newsletter subscriptions to an older listserver, to send out news of our recovery, but the member newsletters (with GPG integration and so on) will wait for the replacement hardware. I’m hoping we can recover some things from the old server filesystem, else I’ll recreate it from the last release (that’ll teach me to hack it on the live server!). In the meantime, I’ve still most of yesterday’s work to do…

20 October 2008

Decklin Foster: And yet I swear this oath

There was a lot of political talk last week in the US about this guy "Joe the Plumber". I'm really quite mystified by the (lack of) Democratic response to this one; it seems like a perfect opportunity for the Dems to explain what their ideals actually mean, after years of complete inarticulacy. The Republican party, and conservatives in general, have somehow successfully sold their platform of, as Robert Reich brilliantly put it on the Daily Show last week, "socialism for the rich and capitalism for everyone else", to poor but socially-conservative citizens for quite some time. They've convinced people to vote against their economic self-interest, and cut taxes on the rich instead of themselves. Is voting for policies that directly benefit you less necessarily bad? I don't think so. In many cases, selflessness and altruism are better in the long run. Conservative economic policies speak to a sense of right and wrong that give people a feeling of purpose, and so an incentive to create: this is mine, I made it, you can't take it away. In a theoretical sense, leaving more wealth in the hands of the rich should "trickle down" to society in general, by providing more capital, and thus creating jobs, etc. However, I do not believe that this is why Americans, specifically, tend to support conservative economic policy. The USA has this idea of the "American Dream", pulling yourself up by your bootstraps, if you will, and striking it rich through hard work and character. It's advantageous for conservatives to sell this idea because it makes people think "tax the rich? What if I get rich?", which is a sort of enlightened self-interest, arguably. And it gets poor people to vote for them. What makes this "plumber" thing interesting to me is that is should demonstrate where this approach ought to fall down, logically and emotionally. But it does not. Here is the problem. A working stiff who earns "five figures", such as myself, generally makes that money by going to an employer every day, performing some services for them, and then drawing a paycheck. Where this money comes from is abstracted away (this is the marvelous thing we call "business"). Let's say you want to sell the "American Dream" to this person. They get their paycheck every week, and there's a chunk taken out of it by Uncle Sam. Maybe they're saving up to start a business, or go to college, etc. They sure could do it faster if taxes weren't so high. So you remind them this: when you do strike it rich (since this is America, where anyone can strike it rich), do you want those taxes to be even greater? This story contains a lie of omission, probably the greatest lie ever told in US politics, and one that has almost universally succeeded as far back as I can remember. And that is: when you do make that jump to where our hypothetical Joe is, making just over a quarter-million dollars, that it's not going to look any different. You'll get your paycheck from some abstracted source, just like before, but more will be taken out unless we stop raising taxes on those higher income brackets. There will be some node higher up in the tree for that wealth to "trickle down" from; you could say it's implied, even, that wealth is created at the root and destroyed at the leaves. What's wrong here? Simple: a self-employed plumber does not have a paycheck, except maybe to him/herself for accounting purposes. They collect their income in the form of payments from customers. They don't have an abstraction: they hire more labor and become the abstraction. If their customers have no money, the abstraction leaks, their employees have to be laid off (and then can no longer afford plumbing services themselves). The success of the plumber depends as much on how many people are able to pay them as it does on his costs (such as taxes). We want to make sure those customers can still pay. This is, in theory terms, the idea of demand-side or Keynesian economics. (Not really exactly. I took about two econ courses. But bear with me for a moment.) Now, you can argue demand-side vs. what we would call supply-side economics, which is the idea that no, first you have to stop taxing Joe so he can hire people so they have money to go buy things, for ages. People have made whole careers out of it. I am not here to do that. What I am here to wonder out loud is: why is this argument not happening, now, in our public sphere? Even if only as a small plot point in one Presidential campaign? There are very good arguments for both sides (I agree with the Keynesians, myself), and they should be what this debate is about, not digging up dirt on some guy whose real name, horrors, might not be "Joe" or protests about all those you-know-what poor people getting "government giveaways" that are disturbingly close to racism in my opinion. When I turn on talk radio I hear zombies, unable to string together any thought more complicated than "Democrat == higher taxes on everyone" (which would be depressing even if the claim were true). It begins to get unsettling when you understand your opponent's argument, and yet watch them completely unable to articulate it. When they deal with this by appealing to fear and thought-killing sound bites, it gets alarming. Obama has backpedaled whenever the whole issue has come up. In the last debate, he was saved by merely shutting up and letting out enough rope for McCain to hang himself. Not very inspiring. As a result I hear repetitions of Obama's phrase "spread the wealth around" constantly met on the airwaves with... silence. He won't touch it, and honestly, I can't blame him for not wanting to fan the flames of class warfare with only two more weeks left to stick it out. But I don't understand why, and why a single person on the left hasn't had the faith that this idea can be discussed in our society. I think that, whichever side you are rooting for (and yeah, I am for the Democrats), the Democrats should have won this point readily. Not to belabor the point, but it really amazes me that they have not. It wouldn't have taken much. Here's what you do: go to Wherever, OH with your crack PR team and find several of "Joe the Plumber"s customers. Interview them all. A few of them are, in this economy, going to be really hurting. If you're lucky: Remember that Joe owes taxes to the IRS. Very possibly, someone owes Joe money! That would be a coup. You just put them in front of the media and the whole thing sells itself. Mary needs to make these repairs... if only her tax credit were larger... etc. This would not settle the argument, by any means, especially not for political junkies. But it would begin it. We could talk about policy, whether one way of "spreading" things will result in more wealth to spread than another, instead of red-baiting. I feel a bit ripped off that we haven't even gotten that much. The theory part is pretty much irrelevant to most people; that's fine. But they deserve context. I can only assume that my ("my") party is still either terrified or stupid. I hope winning the Presidency changes that somewhat. That whole right and wrong, purpose, "noble cause" thing, right? Right, guys?

Jacobo Tarr&#237;o Barreiro: Godawful Enormous Book

Am I the only person who couldn’t stand reading G del, Escher, Bach? I am reading so much praise about it, I sometimes wonder if they sold me the wrong book…

Josselin Mouette: A little rant on the Linux kernel development model

Sometimes, I m becoming really pestered by the number of issues, especially hardware issues, that remain prominent in the Linux system after so many years. And I m convinced that one of the root causes is the kernel development model. While far to be the only one, you understand a lot when you have a look at it. The stable API nonsense To explain the development model, the kernel documentation contains a document written by Greg Kroah-Hartman called stable_api_nonsense.txt. I think this document gets one thing very right: advertising it is complete nonsense. Many things in this document can get you a good laugh if you are used to software development, unfortunately they are not funny given how much they affect us as users and developers. Let s start with the executive summary:
You think you want a stable kernel interface, but you really do not, and you don't even know it. What you want is a stable running driver, and you get that only if your driver is in the main kernel tree. You also get lots of other good benefits if your driver is in the main kernel tree, all of which has made Linux into such a strong, stable, and mature operating system which is the reason you are using it in the first place.
What it should say instead is:
You know you want a stable kernel interface, but you don't have it, and we will never provide it. You want stable interfaces to focus on fixing bugs in your driver instead of updating it for interface changes, and to make integration of your driver in the main kernel tree easy. You would get a lot of good benefits if your driver was in the main kernel tree, but it won't make it unless you adapt to our bizarre processes, all of which have made the Linux kernel into a constantly moving target which is the reason many hardware vendors don't want to support it in the first place.
Our compiler does not have the same ABI as yours This is what happens when the only thing that interests people in charge is to code. Writing this means that these developers have no interest in making the system usable. This becomes more unbearable when such technically sharp people make up false technical arguments:
Depending on the version of the C compiler you use, different kernel data structures will contain different alignment of structures, and possibly include different functions in different ways (putting functions inline or not.)
Really, I wonder how the Microsoft developers, the GNOME guys, the glibc guys, and basically all library developers who know what they are talking about, manage to have stable ABIs despite the compiler changing all the time. One of the incredibly complicated techniques they use is to only rely on guaranteed functionality of the C specification, instead of doing many clever and funky hacks. For things more related to development itself, there are also incredibly complicated techniques like opaque structures, which save a lot of ABI breakage. Actually the only thing you need, if you design your ABI properly, is to be rigorous.
Depending on what kernel build options you select, a wide range of different things can be assumed by the kernel
This statement of a problem also contains the solution: stop making available build options no one cares about (remember, people install binary packages from their distributions) to guarantee stability instead. Instead of implementing the solution, kernel developers deliberately choose to deal with this insanity in the worst possible way: by encouraging it. Only lame developers need stable APIs But the nonsense doesn't stop to the ABI, which after all is merely a problem for distributors. To be more annoying to developers themselves, let s make up reasons to break the API all the time:
Linux kernel development is continuous and at a rapid pace, never stopping to slow down. As such, the kernel developers find bugs in current interfaces, or figure out a better way to do things.
Explained this way, it looks like a good thing. If you've done serious development or integration, you know that actually it is a problem. If you never slow down to look at what you've done, you're only going to add new bugs while you're trying to fix the others.
When they do so, function names may change, structures may grow or shrink, and function parameters may be reworked. If this happens, all of the instances of where this interface is used within the kernel are fixed up at the same time, ensuring that everything continues to work properly.
Everything is said. In the Linux kernel, changes are not done in a continuous way. Every non-minor change is going to have an impact on hundreds of different modules. The amount of work needed to accomplish these changes is absolutely insane. When a project like GNOME decides to phase-out an API, it takes several years before being actually replaced. In the kernel, it can simply happen between two minor releases, together with a rewrite of the whole ATA stack. The lolution Of course, Greg KH has a simple solution for you, in the What to do section.
So, if you have a Linux kernel driver that is not in the main kernel tree, what are you, a developer, supposed to do? Releasing a binary driver for every different kernel version for every distribution is a nightmare, and trying to keep up with an ever changing kernel interface is also a rough job.
No shit !
Simple, get your kernel driver into the main kernel tree.
Simple, isn't it? Unfortunately this is not going to happen while you are too busy adapting it for the constantly moving interfaces. This is quite feasible if you are writing a driver for an Ethernet network adapter, but for a video card this is another story, and for a virtualization layer? Well, the Xen developers have been struggling for years to integrate their technology in the kernel, and it is still far from done. No wonder why you get good drivers only for ethernet cards and not for 3D cards or even wifi chips. This doesn t go without other, worse consequences for users and distributors. Not everyone can afford to run Ubuntu, Debian unstable or Fedora, especially if they have simple requirements like not changing the whole system every 6 months. Corporate users using Debian stable, RHEL, SLES or Ubuntu LTS need a stable kernel that doesn t move, and that doesn t break everything when upgraded. The introduction of projects like DKMS to distribute drivers in a sourceful way while keeping them easily installable is a step in the good direction, despite being a workaround for a broken situation underneath. However these efforts are ruined by the constantly changing APIs that will go on requiring changes to the sources as well as the binaries. Why it can still work In fact, with such processes, you can wonder how it is possible that the system is actually usable. The answer is simple: while being smaller than many other projects, the Linux kernel has much, much more developers. A rough estimation shows that Linux has 2000 developers, many of which are paid full-time to work on it, to work on a 15 Mloc codebase. You can compare it with X.org, which has a few dozens of developers for 3 Mloc. Or with GNOME, 400 developers for 20 Mloc. With OOo, 150 developers for 10 Mloc. Yes, you need 10 times as many people to maintain the kernel code than for another comparable project. That's of course what happens when you keep them busy with a permanent refactoring of the code. This development method, while certainly having the advantage of bringing lots of innovation and fast integration of new features, is also driving away a very large portion of the open source community to work on rewriting Firewire stacks every 6 months. And as long as they are busy with this, they are not writing correct X drivers nor fixing bugs in the applications.

MJ Ray: your country continues to turn into Big Brother

Just received this comment from Jimmy Kaplowitz over in the US: “yuck, your country continues to turn into Big Brother: BBC News: Giant database plan ‘Orwellian’” Hell, yeah. That’s a reason why my support for the party is so weak at the moment. I’ve been working and writing against this since ST@ND and the Regulation of Investigatory Powers Act, which we were told was essential for crime-fighting, but actually helps gov.uk spy on parents making school place applications. Currently, I’m telling people Gov.UK Consults on Forced ISP Snooping: Please Say NO. If anyone has any doubt that the database state will be abused, look at some examples of Labour’s “terrorists” (at least, they had supposedly anti-terror measures used against them) since 2001:-
  1. Wolfie, an 82-year-old peace-campaigning heckler at a Labour conference
  2. Indymedia
  3. a Frenchman in Suffolk who downloaded the Anarchists’ Cookbook
  4. an Algerian pilot
  5. the country of Iceland
and that’s ignoring gov.uk’s frequent data leaks. Noodles replied: “There’s a quote somewhere that goes along the lines of “If the Tory policy is looking sane compared to what you’re doing, you’re doing it wrong” that’s been applied to the Labour fascist state stuff.” Our terrorism measures are a joke, with loopholes that gov.uk can drive a truck through. They should be repealed and clearly-limited ones enacted, shouldn’t they?

19 October 2008

Decklin Foster: Njiiri 0.2.0

Seems like it's been ages since I've been able to do this (probably because it has!): I've released a brand-new piece of software. Njiiri is a client for MPD, using GTK+, written in Ruby. I don't see a lot of desktop apps out there using Ruby, so this was a new and fun experience for me. I've tried to make it as "boring" as possible, in the sense of looking vaguely like a proper, HIG-compliant GNOME app, with (appropriately) no pointless configuration knobs. The browser, in particular, is a mostly pixel-for-pixel clone of the standard file-open dialog. And it's accessed by a big toolbar button with a stock "Open" icon. Basically, if desktop software makes me think ("oh, I'm in iTunes, all the controls work differently, let's adjust our approach"), it's failed. For me, anyway. So, I'm pretty happy with the direction I'm going (even though I'm very much not there yet. I haven't hooked up search, for example). One of the things I like about MPD is that it gives me the freedom to remove things if I want to. And I like removing things; small is beautiful. For example, I originally had no place to stick a crossfading control in, so I was going to just dump it. If you need it, "mpc crossfade" is a terminal away. (It turned out that it did eventually fit nicely into my layout. I'm a bit obsessive about aesthetics.) In a similar way, I've avoided duplicating anything I do with homegrown scripts or mpdtoys; there's just no need. What I want out of a GUI frontend is a sort-of tangible way to manipulate things when I feel like looking at them visually for a bit. Then I close it and move on. MPD certainly isn't the most wonderful implementation of a system which allows that, but the client ecosystem is quite diverse and I'd say it deserves a look if any of this sounds interesting to you.

Joerg Jaspert: packages.debian.org

Today we finally switched packages.debian.org over to it’s new host, which was long overdue (and about most of that time it was waiting on me :( ). It did run on it’s old machine for some time, but thanks to the load it really needed an upgrade. Thankfully 1&1 agreed to sponsor a bigger machine, and here we are with it. Its really nice - cron is running and updating, but you don’t notice it at all, it responds hell fast via the web. Yay. Finally. :) Now, while I am at it, a few statistics. Everyone loves statistics, right? :) This year packages.debian.org had 5777672 unique visitors creating a total of 122314763 hits, and this excludes bots. Double the numbers if you want to know what bots do (like crawlers from search engines). Appearently the most active times are between 0800 and 2100 UTC. Limiting our view to this month, the most active bot in terms of hits is MSNBot, most traffic goes to GoogleBot (appears that 30% less hits means 66% more traffic. Bad Google, what do they miss? :) ). Only 49.9% of packages.debian.org visitors do use (or tell their browser to identify with) Linux, 36.2% have Windows, followed by unknown at 10.5% and Macintosh at 2.9%. 41.9% tell us they use Firefox, while only 19.9% admit to use IE, another 19.9% goes to Mozilla, while Konqueror only has 4.9%, Opera 4.5%, “Debian Apt” 2.2% and Galeon 0.2%. It seems that most of our users directly get to packages.debian.org, 47.7% of the accesses do not have a referer header. Search engines aren’t that great, only 23.5% of the accesses seem to come via one of them, while 28.6% are links from various sites. Mostly www.debian.org and some otherwise very related ones. And finally, mail. As packages.debian.org also does forward mail, lets see what we had there. In the last 12 months we rejected 5367162 mails and forwarded 48784 ones (which means some 97k outgoing mails, as they always get forwarded to two addresses). Comments: 2

17 October 2008

Raphael Geissert: adjusting the clock "iznogoud", technet says


From blogs.t e chnet.com/latam/archive/2008/10/16/argentinadst-2008-2009.aspx:

Please note that if you adjust the machine s clock manually, you may have adverse effects on your environment. This procedure is not supported by Microsoft.


lmao, give me a break.

Their proposed "fix" for end users, vista, and server 2008 users is to "Move machine time zone to (GMT -2:00) Mid Atlantic". IOW what they mean is "we suck at this, you are on your own".

Oh, and yes: I just reminded the iznogoud character; nothing further :).

Mart&iacute;n Ferrari: A season in Plan 9 land

I mostly stayed under a rock during the last weeks: exams season, some assingments plus one week of much needed [VAC]; ironically in the DebConf 8 secondary hotel. I'm in a very interesting course on operating systems' programming, it's a new optative course, and has generated a lot of expectatives: most of the systems-oriented undergrads in the computer science career are attending. The downside is that it's *intensive*: we're expected to build from scratch a minimal operating system, starting from the bootloader. So far my group has already done a bootloader that understands FAT, and the entry point for the OS, in which the basic stuff is setup: gate A20, interruptions, protected mode, GDT, etc. Plan 9 mascot: GlendaAlso, each group is required to do a presentation of some systems research topic, and we choose Plan 9 from Bell Labs, that mostly unknown OS that was meant to be a better UNIX, but never gained popularity. So I spent many days playing with it, trying to grok quickly what it was about, and creating a small network of virtual hosts (in Qemu) that could show how it all fit togheter. Plan 9 has all the credentials: created in the same lab as UNIX, designed by the same people, which already learned from past mistakes. It was started in the late eighties, and it really feels like what the future was meant to be. Sadly, we're just starting to see a few of it's revolutionary ideas being applied to other systems: UTF-8, /proc, unionfs, fuse, clone(2), separate namespaces; all of these follow concepts first created in Plan 9, UTF-8 was designed by Ken Thompson "in front of my eyes, on a placemat in a New Jersey diner one night in September or so 1992", in Rob Pike's words. Not happy enough with reworking all the core concepts, they also recreated the visual environment. I have to admit, the user-facing ideas are so foreign that I still don't feel very comfortable using it. Surprisingly, the desktop is very mice-centric, it cannot be used at all without the mouse, and it's bothersome with less than three buttons. Anyway, the underlying workings are so attractive, that I cannot stop from wondering if a hybrid system could be built (Debian/kPlan9 anyone?). To illustrate, I'll reproduce here the same example that I showed yesterday and left some jaws hanging. I had booted a CPU server and a diskless terminal, called cpu-0 and term-0 respectively. This is a screenshot taken in the terminal, which is basically a telnet to my Debian box:
Plan 9 screenshot: first telnet Nothing spectacular here, except for the weird syntax, it was seen like this in the Debian box:
martin@abraxas:~$ netstat -nt   grep :22
tcp        0      0 10.0.2.1:22             10.0.2.30:39879         ESTABLISHED
But here comes the fun, I ran an import command, which is more or less a remote filesystem mount, with which I mount the remote machine's /net in the local namespace. It has to be noted that the /net path is by convention the entry point to anything network related, including all the networking API.
Plan 9 screenshot: second telnet And the wornderful thing is that this works as it should (note the remote address):
martin@abraxas:~$ netstat -nt   grep :22
tcp        0      0 10.0.2.1:22             10.0.2.10:38158         ESTABLISHED
So, what happened? telnet wanted to create a TCP connection, and that is accomplished opening /net/tcp/clone, but then it found the imported filesystem. The open(2) request was then transferred to the remote machine, and there the networking code did the actual TCP/IP dance. Things like this make you realise that maybe the BSD sockets API is not that nice an abstraction; in Plan 9 there are no BSD sockets, it's only implemented in the POSIX compatibility libraries, which aren't used by any of the core tools. Neither there exists any threads support, you just call rfork(2) setting the appropiate flags, exactly what glibc does under the covers nowadays. They also decided to live without select(2) or poll(2), there are no non-blocking calls. If you want non-blocking, you rfork(). The only syncronisation primitive is rendezvous(2), and everything else is built on that. At this point, it should not surprise you that to kill a process you can type echo kill > /proc/$pid/ctl; and that there's no need for an equivalent to Xnest, as rio (the graphical interface) provides a modified /dev, you just type rio inside a window and there you go. Yes, I'm hooked. If you're interested, I compiled many links here (some Spanish there, but all the content is in English), as one of the weakest points in Plan 9 is the unorganised and missing documentation. Tags: Plan9, Planet Lugfi, Planet Debian, Facu

Russell Coker: The Next Miserable Failure?

Until very recently I thought that it would be almost impossible to get someone worse than George W Bush as the leader of any significant country. Unfortunately it seems that I was wrong and John McCain and Sarah Palin promise more of the economic, regulatory, and military disasters that are the trademarks of the US Republican party (or at least the dominant Neo-Con branch). Here are some links about John McCain: Here’s a good summary of the racial issues in the current US presidential campaign (This is Your Nation on White Privilege) [1]. The Obama campaign is highlighting the connection between John McCain and Charles Keating [2]. McCain was one of the senators helping Keating while his bank (the Lincoln Savings and Loan Association) was going under. In the end 20,000 people lost their savings and the US taxpayers ended up losing $120,000,000,000. Frank Rich has written an article for the New York Times about the racist attacks on Barack Obama [5]. The current actions of the McCain campaign only barely stop short of calling for an assassination. The South Florida Times has an interesting article about the McCain family’s history of slave ownership [7]. Now John McCain is not responsible for the actions of his great-great-grandfather in owning slaves, and there’s nothing wrong with having black relatives who are the descendants of some of those slaves (even though there is doubt about whether the female slaves were legally adults or even consented to the sex acts in question). But he should be honest about it. Denying having non-white relatives in the face of the facts seems to be strong evidence of racism. It is however understandable that John doesn’t want to discuss the fact that some of his relatives have announced plans to vote against him. Rolling Stone magazine published an interesting article about John McCain’s history as a spoiled brat in the navy [10]. It seems that if your father is an admiral you can ignore orders, crash planes, and basically do whatever you like. It also reveals that John was broken by the Viet Cong torturers and provided the name of his ship, the number of raids he had flown, his squadron number and the target of his final raid. I’m not going to criticise John for breaking under torture - I think that the assessment of wing commander John Dramesi (who was tortured by the same Viet Cong torturers but didn’t break) should be accepted. John Dramesi says that McCain “wasn’t exceptional one way or the other” while in captivity. However McCain’s use of his former POW status in propaganda is quite dishonest. John McCain is also documented as having described his wife as a “cunt” and a “trollop“. Here are some links about Sarah Palin: Former US Army Brigadier General (retired) Janis L. Karpinski writes about Sarah Palin [3], it’s interesting to hear what an intelligent female soldier has to say about her. One thing that I found noteworthy was the repeated references to “murdering” wild animals, shooting at a defenseless animal is of course quite different from shooting at a person who can shoot back (and different again from commanding an army). Janis also makes reference to Sarah setting the feminist cause back decades - I think that is what Sarah desires though. Also Janis points out the emotional problems for which pit bull terriers are known. There are many claims that Sarah is a “Maverick” and has a record of opposing corruption. This article in the Village Voice documents some of her corrupt activities - including having her home built for free in exchange for assigning the contract to build the Wasilia ice-hocky rink [4]. Thomas L. Friedman has written an article about Palin’s Kind of Patriotism [6]. According to Sarah it’s not patriotic to pay taxes, it seems to me that encouraging citizens to disobey the law should disqualify her from being elected without all the other issues. Thomas notes that Sarah is promoting the interests of Saudi Arabia by prolonging the US dependence on oil imports. The Huffington Post has an interesting article about Sarah Palin’s church [8]. It’s strange how little notice has been taken of Sarah’s former pastor who stated that people who didn’t vote for Bush were likely to go to hell. The Times has an article about “Troopergate”, some of Sarah Palin’s other corrupt practices, and the role of her husband as a shadow governor [9]. Update: Corrected URL [6].

Next.

Previous.