Though I only use Debian on all my machines, I like playing with new Ubuntu releases, just in case I have to set it up on someone else’s machine. I use VirtualBox for testing. When trying the upcoming new Ubuntu 8.10 release, and after installing the VirtualBox clients on the guest, I had the following issues: First, the video driver refused to give me more than a 800×600 resolution. This can be worked around by adding the following lines in the xorg.conf file: A second issue was the mouse grabbing which did not work at all. Adding the following lines in the xorg.conf file has fixed the issue: You need to restart Xorg after amending the file so that the changes are taken into account. Both issues are due to the fact Xorg 7.4 is supposed to automagically recognize the hardware - but it seems like this is not always the case (or it refuses to use it, not sure what’s happening in that particular case).

I voted early because Sullivan County, TN is phasing in new voting machines this election. On election day, they will use the old touch screen system, while early voters get to test the "new" paper system. The new system is sure to please fans of paper. However, it has two flaws that were obvious to me. I have no way of knowing that my vote will be counted as I filled it out, and I realized half way through that election officials can determine how I voted. Here's why... You have two pieces of paper during the vote. A registration sheet is signed by the voter. A paper ballot has little rectangles that the votor fills in with a ballpoint pen in the privacy of the voting booth. The instructions say to fill in choices completely. Since the voting booths have a hard top, it's difficult to write on them with a ballpoint, let alone fill in little squares completely. The natural thing to do? Slide your second piece of paper under the ballot, so the pen can dig in better. If you don't think about it, you'll probably do this unconciously. Half-way through, I realized that I was leaving imprints on the registration paper, with my name on it, that corresponded to my choices on the ballot. Due to the two-column layout, and the distinctive spacing between different choices, it's possible to look at those imprints and determine exactly how I voted. The registration paper has to be handed in at the end, so the official who sits there all day collecting these has plenty of time to work this out and determine the votes of anyone. (I moved my ballot around and re-filled in squares to try to confuse things.) (Relatedly, one has to carry the paper ballot around the room to take it to the voting machine. It's difficult to keep the ballot obscured while doing this, since you can't fold it, and since they take the registration paper before the ballot, so you can't keep the ballot covered with it either.) The second obvious flaw is that once I fed my ballot into the voting machine, which presumably scanned it and put it in a safe, there was only a confimation that "you've voted!". There was no way to verify that it had counted my vote as I'd marked it.

---

Most voting system work seems to be in the direction of advocating paper ballots for paper ballots' sake. This doesn't seem like the right approach to me. Is no-one working on writing down the characteristics that an ideal voting system would have, and trying to make them all requirements? My take on requirements of an ideal voting system are:

1. Only living people in the set of registered voters may vote.
2. Each votor can only vote once.
3. The votor should be able to verify that his vote was counted correctly.
4. There must be a way for the votor to prove if his vote was not counted.
5. The votor must not be able to prove which way he voted (to avoid payoffs, intimidation, etc).
6. Others must not be able to determine how a votor voted.
7. Individual votes must be retained to allow recounts.
8. The entire vote data should be published so that the results can be verified by third parties.
9. The system must scale to many millions of voters.
10. The correctness of the entire voting system should be formally provable.

The paper ballot system I experienced today fails on at least points #2 [A] , #3 (see above), #4 (I walked away with nothing I can use to prove anything), #5 (I can video my vote), #6 (see above), and #8 [B]. At least it scales well though. The old touch screen system failed most of the same points, though it made at least a small attempt to satisfy #3 (by confirming the vote onscreen) -- and it failed #7, and probably scaled worse. Outside of the American election system, I've participated in voting systems that seem able meet requirements #1 [C], #2, #3, #4, #7, #8, and #10. To me, #10 is the most important point, but it seems to be the one that is neglected most. Some of the points may be worth weakening if they block other points. Is it really worth avoiding giving me something to prove how I've voted, and banning taking cameras into the polling place, in order to meet point #5, if this defeats point #4? I'm curious what requirments I may have left out too. My list is just a rough stab at it. The real point is to analyze voting systems, and reject ones that don't provably meet criteria. Until it becomes accepted practice to do that, our voting systems will not get appreciably less flawed.

---

[A] If you can get two copies of the ballot, you can vote twice ... fairly easy with a friend on the inside to arrange for the two copies to get "stuck together", and another one at the vote scanner to allow you to scan both. [B] The ballots could be published, but there's no way to prove all of them, or the true ballots would be. And in reality, they're either shredded, or locked away, or tossed in a dumpster, or variously all three. [C] Mostly. Perfectly accomplishing #1 is harder than most of the other points, and may be worth leaving out of scope.

Following Ganneff’s post to debian-devel-announce, several discussions have again started on the topic of Debian’s membership and several proposals have been made. Unfortunately none of these proposals try to resolve the underlying trust problem that has been growing over the years. Despite the NM process (or maybe due to it), we managed to give DD status to people who are motivated but whose technical skills are doubtful (at that point people ask for an example, and as much as I hate fingerpointing, here’s an example with #499201. The same maintainer created troubles with libpng during the etch release cycle and tried to take over a base package like mawk recently). With our current model, all DD can sponsor, NMU, introduce/adopt/hijack packages without review. This is fine as long as we trust the body of DD to contain only skilled and reasonable people. I believe that premise to be somewhat broken since Debian has become too big for people to know everybody and since the NM process had no way to grant partial rights to volunteers who were motivated but that clearly had not shown their ability to handle more complex stuff than what they had packaged during their NM period (like some trivial perl modules for example). Thus I strongly believe that any membership reform must provide a convincing answer to that trust problem before being implemented. I took several hours to draft a proposal last Friday and I’ve been somewhat disappointed that nobody commented on it. I hope to draw some attention on it with this blog post. The proposal builds on the idea that we should not have classes of contributors but simply two: a short-term contributor and a long-term contributor (those are called Debian Developers and have the right to vote). But all contributors can be granted privileges as they need them for their work and each privilege requires the contributor to fulfill some conditions. The set of privileges and the conditions associated all need discussions (but I have personal opinions here, see below). There’s however one privilege that is somewhat particular: it’s the right to grant privileges to other contributors. Handling it as a privilege like another is on purpose: it makes it clear that anyone can try to get that privilege and the procedure is clear. In practice, imagine that set of people as a big team encompassing the responsibilities split over DAM/AM/FD/DM-team and where all members can do all the steps required to grant/retire a privilege provided that 2 or 3 members agrees and that nobody opposes (in case of opposition a specific procedure is probably needed). I called that set of people the Debian Community Managers. It should contain only skilled and dedicated developers. One of their main duties would be to retain the trust that the project as a whole must have in all its members. They would have the powers to retire privileges if they discover someone that has not acted according to the (high) expectations of the project. Among the privileges would be limited upload rights (like DM have currently), full upload rights (like DD have currently although it might be that we want to split that privilege further in right to sponsor, right to package new software, right to maintain a package of priority > standard, etc.) and developer status (email + right to vote, once you can prove 6 months of contribution). There’s lots of stuff to discuss in such a proposal (like how to decide who gets what privileges among existing DD) but I think it’s a good basis and need some serious consideration by all the project members. The NM process is there only so that we can collectively trust that new members are as good as we expect them to be and trust can only be built over time so it’s good that we can grant privileges progressively. Some people believe that I’m reinventing a new NM process that will end up to be very similar to the current one. My answer is that the conditions associated to each privilege should be based on the work done by the contributor and the advocations that he managed to collect. It should not be a questionnaire like Task and Skills . This, together with the distribution of the power/work on many people, would render this system very different from today’s NM process. Some people believe that I’m copying Ubuntu when designing this since it’s somewhat similar to the process to become MOTU and/or get upload right to Ubuntu’s main component. Let me say that I’m not copying deliberately at least, I simply took the problem from the most important side. But remember that many aspects of Ubuntu have been designed by Debian developers that tried to avoid known pitfalls of Debian, and maybe they got some things right (or better at least) while doing this. Partagez cet article / Share This

I’ve been remiss in pointing my readers recently to Zachary Schrag’s Institutional Review Blog, which patiently documents the overreach and misapplication of federal regulations regarding the protection of human subjects to the social and behavioral sciences, including research that by federal regulation is exempt from review by IRBs. While Schrag is cautiously optimistic that the new head of the Office of Human Research Protections will be an improvement, the continued domination of the process at most levels by biomedical researchers—along with the general sense that, as Schrag notes, “researchers cannot be trusted to apply the exemptions themselves”—is still troubling to those of us who want to conduct human subjects research, particularly secondary data analysis. Technically speaking (even though I dare say most social scientists observe this requirement in the breach), even the analysis of secondary data collected by others and fully anonymized before we see it (e.g. use of data such as the General Social Survey, American National Election Studies series, the Eurobarometer series, etc.) requires IRB oversight and approval beforehand.

I've been a bit remiss about keeping this space up to date with my upcoming talks over the last month or so. Here's me playing catchup. On Monday October 20th, I gave a talk on Selectricity for the IEEE Boston Section's Society on Social Implications of Technology. It covered more or less the same ground I coverd in my OSCON talk on the same subject. Then next day, Tuesday October 21st, I gave a short talk on Revealing Errors as part of the MIT-Harvard-Yale Cyberscholars meeting. There was nothing new or ground-breaking in either but it was good to spread the word on the projects -- work continues on both. I also have one more coming up: another Revealing Errors talk in Amherst, Massachusetts at my alma mater on October 31st (Halloween). If you are in Western Massachusetts and would like to meet up or attend the talk, let me know.

Preface: legal stuff Depending on where you live, use of random open wifi networks might be considered illegal. Although I know of no law in my country against doing this, where you live, things may be different, so educate yourself before following my example. Wifi roaming on the move For the past year I have used my Eee PC on the metro transit bus, as it is the perfect size for this. When I can, I take advantage of open networks to connect back home. In this article I will show how you can actually use such access, will recommend which software works best and will explain how to set it up. So, what can I really do with the net while in motion? How about irc? No kidding! This shell one-liner does the job:

while ! ssh -t 10.9.8.7 'screen -UDr' 2>/dev/null ; do echo -n "." ; sleep .1 ; done

We’re concerned about privacy and security on other people’s networks, so we use ssh. Inside that screen session, keep irssi running for a continuous irc presence. Every 10th of a second, the script tries to resume your session. Once a the net passes by, typing “[Enter]~.” drops you back out to retry again. The row of dots serves as a progress bar. If you want to get fancy, you could replace that with an ASCII ‘spinner’. I leave that as an exercise for the reader. :) Come on, is that really usable? Certainly. As the good folks on #eeepc at irc.freenode.net and #debian-eeepc at irc.oftc.net can attest, even a half dozen brief connections can be enough to provide tech support, rough out some development ideas, or just carry on some friendly chat. If your spouse or housemate is online too, it can even help you make some last-minute grocery-list changes. :) While the script is working, enjoy some offline things such as blogging, reading a cached Planet Debian feed or a PDF book. OK, so what else do I need to make it work? Well, you’ve seen so far we need on a server:

• openssh-server
• screen
• irssi

On the Eee, you should also have:

• openssh-client
• wpasupplicant
• gnome-terminal

But what do I need wpasupplicant for if I only use open networks? Well, wpasupplicant is really handy for fast control of networks in Roaming Mode (as per /usr/share/doc/wpasupplicant/README.modes.gz). A GUI is just too slow. For example, you may need to filter out ‘junk’ open wifi nets: pay-per-use hotspots or nets that hide their name. You’ll not only want a “catch-all” stanza for any open network, but also a number of the most commonly found default network names, e.g.

# Catch-all; associates with any open network:
network= 
     key_mgmt=NONE
 
# Commonly found default network names:
network= 
     ssid="default" 
     key_mgmt=NONE
 
network= 
     ssid="linksys" 
     key_mgmt=NONE
 
# etc. by watching scan_results closely you'll find several more

While the connect script is running, keep a separate tab open in gnome-terminal for wpa_cli. Use ‘disable_network 0’ to drop the catch-all and let the other stanzas kick in and ‘enable_network 0’ after filtering is no longer needed. Isn’t it really slow to get connected? Without some tweaks, yes. But here are a few things I have found by trial and error that help. One issue is that dhclient takes too long to retry, so you drop the delay between retries back to 1 second in /etc/dhcp3/dhclient.conf:

backoff-cutoff 1;
initial-interval 1;

If the signal is very weak, you’ll want the transmit power turned up to the maximum value for your hardware. For the Eee PC 4G this is 50mW, so you set this in /etc/network/interfaces for ath0, your wireless interface:

wireless-txpower 50mW

Finally, while you’re in motion you are constantly leaving no-longer-usable access points behind. The driver needs to discard them as quickly as possible and scan for new ones. So again in /etc/network/interfaces for ath0, you bring the default scanvalid 60 seconds down to 10 with:

pre-up /sbin/iwpriv ath0 scanvalid 10

Conclusion The netbook has changed how we use the net. Now that we can take it everywhere, we either pay through the nose for expensive cell network access or else use the patchy wifi coverage blanketing most urban areas. A few lucky cities have such things as municipal wifi nets, but the rest of us have to scrape what access we can wherever we go. You are now equipped to try this out on the bus. If you find any other tips that can help, I’d love to hear from you.

As I wrote previously, I started looking around for a new hosting provider, as Layered Tech is jacking up their prices too much. This isn't something I consider 'fun', as it will involve a lot of time swapping things over, with the potential for problems, but the amount of money disappearing every month is too much to ignore at this point. After some dithering, I finally went ahead and spent some money for a minimal slice at Slicehost. Linode seems pretty competitive too, but I more or less flipped a mental coin. I was actually ready to go ahead the other day, but I stopped to think some more after seeing that Slicehost has been acquired by Rackspace. In terms of seeing hard working people rewarded for their efforts, I am of course happy for the guys at Slicehost, but it also leaves a few questions:

• Sometimes, perhaps not immediately, but after a while, there is less incentive to bust your butt if you cashed in in a big way.
• Rackspace is not my kind of hosting outfit: they don't even list their prices where people can see them, like Slicehost, Layered Tech, and most everyone else does. I've heard they're good, but quite expensive. I don't need 'managed' servers - I can do that myself.
• The FAQ says that Slicehost will continue as a going concern in its own right, but there's always the chance that they were acquired for their tech, and that the FAQ just says that they will continue to operate to keep their current clients from bolting.

Of course, good things could come out of the deal too, so I decided to put my paranoia aside and test the waters. There are a lot of things to like about a system like Slicehost's: it's relatively quick and easy to set up new slices, fiddle with the resources that each slice has, and so on, compared with a physical server. Furthermore, they have a reputation for not 'overbooking' their servers, which is why I wanted to get away from a VPS in the first place, many years ago. Setting up a slice was very quick and easy, but unfortunately got off to a rough start: their "fraud detection" system had a problem with me being in a different place than my billing address. Luckily, they were very fast at correcting the problem. As an aside, I think that fraud detection is probably best left to the credit card companies, although of course I don't really know their business, so maybe that particular check has saved them some money/frustration in the past. Upgrading the Slicehost Ubuntu to Intrepid In any case, once I got it up and running, I wanted to test out the upgrade to Intrepid, as that's going to be out in less than a week, and I might as well upgrade while I have 'fresh' servers to play with, and prior to installing a bunch of stuff. It went very well. I did a do-release-upgrade -d, hit 'yes' to everything, rebooted, and everything was fine! Admittedly, I don't have anything running on the server yet, so that kept things short and sweet, and didn't require the upgrade of various systems which might have had troubles, but so far so good If you'd like to sign up for Slicehost, it looks like they give you a referral code. Mine can be utilized by going here.

I have uploaded barry package to Debian some hours ago. It is still waiting for NEW processing, so if you want it you will have to download sources from Debian git repo, and compile the package yourself. It has just been accepted into the archive. Barry is a GPL program that will allow you to charge your Blackberry through the USB port, to make backup copies of your device databases, manage contacts and use your BB as a modem. So if you have a BlackBerry device, you can also enjoy it in Debian. Update 26/10/2008: Barry has just been accepted into the archive, so you can get it in every Sid repository mirror. Thanks!

Things are evolving under the hood for reportbug, and we are preparing a version targetted for squeeze.

The version 3.99.0 has just been released to experimental; the main goals for this release are:

• separate python module (python-reportbug) to support the main programs (reportbug & querybts), and to allow even other tools using reportbug stuff to do it more clearly;
  
• GTK2 ui, to be more user friendly; many thanks to Luca Bruno (lethalman88-gmail-com): please test it a lot, and Luca will be pleased to fix any bugs :D

Other goals we'd like to archive for the 4.0 version (for which 3.99.x are the pre-releases) are:

• SOAP bts quering;
• switch to python-apt for packages information retrival, but python-apt needs better doc, please :) ;
  
• (a lot of) bugs fixing and enhancements;
  
• code refactoring.

Since this is a development version, you can experience some bugs/regressions, so please try it, stress it and "reportbug reportbug" if needed :) You're help is really appreciated! (and we have cookies ;) )

I bought my first Apple hardware a number of years ago -- a 400MHz titanium Powerbook. The thing came with Mac OS X preloaded, but it was so slow that it was unusable. I put Debian on it, and was mostly happy with it -- until the titanium case started coming apart. Oh, and the small detail that the built-in wifi reception was so poor that I had to use a PCMCIA card to get a signal in some parts of my house.

But ever since then, we've always had some piece of Apple hardware in active use in our house.

About 7 years ago, my main workstation was a rapidly-aging 600MHz Alpha with -- gasp -- PCI video. It was OK for programming but really awfully slow at anything graphical. I was wanting to get into video editing, and we bought an 800 MHz iMac. The kind with the inverted-bowl base. A nice machine at the time. This was going to be my main workstation.

When it arrived at home, I unpacked it. Terah walked in and said, "Wow, my new computer is cute!" So it became *her* computer, and has been ever since.

It's rather old and is showing its age. Its wifi card died a couple of years ago. The 800MHz CPU is slow with today's websites.

So yesterday, we clicked on the apple and shutdown for the last time. It left her desk, replaced with a Linux desktop with a 20" 1600x1200 LCD (replacing the 15" 1024x768). I had assembled this Linux desktop out of spare parts in our basement, with a 4-year-old Athlon XP CPU. It's still faster than her Core 2 Duo work laptop running Vista, but that's another story.

I put Debian lenny on it. And I've got to say -- wow -- the utility of Debian as a desktop OS for the non-geeky has taken an incredible turn for the better in the last few years. Terah was fine with switching to Linux. I set it up for her -- Firefox for web, Thunderbird for mail, J-Pilot for syncing to her Palm, OpenOffice, gajim, etc.

Her concerns were: having Thunderbird make a nice "whoosh" sound when it successfully finished transmitting an email, having desktop gadgets (I installed screenlets, which I think she likes a lot), and.... I don't think I remember any others.

So now we both run Linux on our desktops.

Oh, and on her laptop, I poked around in the Vista control panels (I must have "authorized it to display this panel" a dozen times, stupid thing), and eventually found a box where I could optimize for visual effects or optimize for performance. I hit optimize for performance, and it transformed its interface into approximately Windows 2000 after a couple of seconds. I think that change made it a lot faster. It's now about as fast as the 25%-as-fast Linux box. I guess that's good for Vista.

It occurred to me this morning, as I was pondering our forthcoming ads for two political science positions and a conversation I had yesterday with some other social scientists, that had I accepted either of the tenure-track jobs I was offered before coming to TAMIU, I’d never have even applied for this job even though on most dimensions, at least in my personal judgment, it’s a better position than either of those were/are. I’m not really sure what this all means, but I figured letting my readers engage in something a step above dream analysis might be more interesting than not posting anything today.

So I usually don’t announce my publications on my blog and for various reasons. The main one is that I am more than a little embarrassed that some of them are behind walled gardens (but being I am a junior prof, I am not in a position to negotiate otherwise). Otherwise, some of my pieces are semi-jargony (in the academic sense) and may not be of interest to readers who I think are more geek than academic. Finally, the process of publishing is so slow, so long, and so painful, I try not to think about it, much less write about it, if I can help it. But I decided to announce this
piece , which is on hackers (surprise surprise) as it might be of interest to some readers and because people can download it for free here for the rest of the month (but registration is required ) Co-written with Alex Rex Golub, I am pretty happy to see in print although it is far more academic in its tone, argument, and language than most of everything else I have written. This is in part because the article is more theoretical than ethnographic (hence the journal, Anthropological Theory) and tackles the question of liberalism alongside hacking. It seeks to demonstrate that these can be talked about 1) together and in cultural terms 2) that we we can identify some cohesive elements to hacking and liberalism, in part by placing them in conversation with each other 3) and yet we can also locate plurality and diversity within liberalism and hacking as well. This is a lot to tackle and cover in one piece under 35 pages and I am sure it could have been pulled off better but I think it is is a decent start to thinking about these questions. If you are interested but are allergic to academicalese, sticking to the Introduction, the Hacker Ethical Practice: Three Examples section, and Conclusion, will give you a taste of the arguments while avoiding most of the jargon. The irony of this article is that even if it hits at some pretty theoretical issues, it was provoked by a mundane conversation and disagreement I had with one hacker, Karl Fogel, over another hacker, Kevin Mitnick. After returning from the hacker conference, HOPE, I had dinner with Karl and told him about Kevin Mitnick’s keynote speech, which I found particularity enjoyable and entertaining. After calling Mitnick a hacker, Karl responded with the following: Kevin is not hacker. He is a cracker. Though I think I convinced him that cracker may not be the best word for him (and he convinced me there are differences between hacking, noting perceptively that his primary motivation seemed to be getting access to something he wasn’t allowed to have access to that is, it was more about breaking the rules and the thrill of crossing a social line, than about learning a technical system. ), I decided that I wanted to write a piece that squarely addressed tensions and differences among hackers instead of whitewashing them away as most authors, journalist, and even some hackers do.
Although most of the time, it seems like it is the Karl-type hackers who accuse the Mitnick-type hackers of not being true and authentic, recently I have been in a few situations where the tables were turned. For example, last year I was having coffee with a 2600-type who insisted that hacking on Linux was not hacking at all (not innovative enough, according to his world view). Another example I came across was in a recent Phrack issue where the prophiled hacker, the Unix Terrorist, takes a swipe at (well honestly at everyone and everythin) but when he is talking trash about who is and who is not a hacker, he singles out F/OSS developers:

Linus Torvalds isn’t a hacker! Richard Stallman isn’t a hacker! Niels Provos
isn’t a hacker! Fat/ugly, maybe! Hackers, no! And what is up with
the use of the term “cracker”? As far as I’m concerned, that term
applies to people that bypass copyright protection mechanisms.
Vladimir Levin? HACKER. phiber optik? HACKER. Kevin Mitnick? OK,
maybe a gay/bad one, but still WAS a “hacker.” Hope that’s clear.

Of course, one can play the game of defining the authentic hacking (and it makes sense for many to do this!) but my interest as an anthropologist has never been to draw a bright and clear boundary between good hackers and bad hackers. or “real and fake hackers,” but instead to describe and grapple with the tension points and internal ambiguities among hackers (my whole course is designed around this theme, in fact). This is not to say that anything goes in the world of hacking (that is, I don’t consider any computer break in a hack; I often just call it a crime) but if there is a group of people calling themselves hackers and thinking to some degree of the ethical implications of their actions, as an anthropologist, this is enough “social evidence” to start asking some questions about the political and cultural significance of their actions.

Following the recent announce about an intent to extend debian membership to various levels of contribution, I would like to add my two cents.. Some bloggers mentioned how they disagree on the notion of various level of rights, claiming that it was yet another class inequality. Some other bloggers mentioned their support to the proposal, and other felt that the announcement was badly done. I, for myself, don't have an objection to the idea that they should be various level of rights. I do like the notion of anarchism, but I have some doubts it can extend to a wide group of persons. Besides, I personally don't want to have rights on everything, since I don't want to be responsible for them. As for real life, I believe the less broken system to rule a big group of people is democracy. And, as such, I don't have objection to a system establishing different levels of rights, provided it is transparent and responsible to the majority of members. In particular, voting right and vote outcomes should rule everything. According to the debian constitution, it is already the rule that most major decisions in the project can only be taken by votes. That's a good thing for actions and resolutions concerning the project. Now, concerning members, the proposition would establish new rule to accept and give rights to new coming users. Fine. But, I wouldn't think it is fair until the same process apply also to core teams, and until the basis of the project are the voters, which would choose or dismiss members of those teams. Inequality class, just like democracy, can be seen as a necessary evil. But inequality is balanced, to me, when there is a democratic process of electing and dismissing people. Then, we can hope that there won't be any sort of aristocracy.

I've just started a new job, so I decided to take a few days holiday in between and work on something that's been on my todo list for way too long now -- making the debian-installer work on the D-Link DNS-323. After a couple of days of work, I've just finished off the install of Debian on my DNS-323, which I initiated by loading a firmware image into the web interface of the stock firmware, and then SSHing into the machine at the appropriate moment. What this means is that anyone with a DNS-323 can install Debian on it. I think that's worthy of a small, quiet "woo hoo!"... Looking back now, it seems to have been a fairly simple and straightforward operation, but I can remember that it didn't look so simple when I started a week ago. So, in the interests of perhaps short-cutting someone else's adventure into the world of d-i hacking, I'm writing everything up while it's fresh in my mind (the benefits of being on holidays -- plenty of time to write long blog posts). Alternately, if anyone has a random device they'd like to see d-i on but doesn't have the inclination to do it themselves, feel free to send me one and I'll do it for you. Before I begin, I've got a few "shout outs" (I think that's what the cool kids are calling it these days) to people who've helped me along the way. First off, Martin Michlmayr has answered innumerable dumb questions from me, and I think wrote most of the parts of d-i that I've extended -- it's fair to say that I'd have had no hope of doing this without him. Thanks also to the people in #debian-boot and #debian-arm for answering questions and putting up with my frequent bouts of self-congratulation whenever I passed the most miniscule milestone. Finally, everyone who has put work into d-i has my thanks, too -- it's not perfect, but it's a damn sight better (both to use and also to hack on) than any other installer I've ever met. So, without further ado, off we go... Work out how to get a console Unless you are a guru of unbelievable skill, you will undoubtedly not do everything quite perfectly the first time, and you'll want to see the kernel/init boot messages, and maybe in some cases try some investigation and diagnostic work. For that, you're going to have to have some sort of console access. If the device doesn't have a full screen/keyboard thing going on, the chances are you'll almost certainly end up with a serial console. A few devices have a network management thingo that will give you a console, usually over VNC or some sort of horrid Java applet, but I've only ever encountered that in rackmount servers, which have a keyboard/video interface anyway... In theory, I suppose, the kernel could provide a network console, but I've never heard of such a thing being attempted, and Google is not helping me out. I might add that to the todo list as a future hacking project... On the DNS-323, it was a serial console that was needed. Like a lot of the embedded devices I've seen, it didn't quite talk RS-232, so I needed an adapter (I got a TTL-232R-3V3 USB-to-3.3v-serial adapter from a local electronics supply place, which worked perfectly with Linux as soon as I plugged it in). I needed to solder a few wires to a connecter inside, and connect that up to the USB doohickey, but it all came good in the end. The chances are someone's done a similar job for your device of choice, so a search for things like "devicename serial console" will almost certainly pay off. Work out how to load and boot a kernel/initrd Your development work will involve booting weird kernels and initial ramdisks to make everything work; you need some way of stuffing those into the device and booting from them. Everything has a boot loader of one sort or another; working out what you've got and how to make it sing and dance is the ultimate quest here. How this is done will vary greatly from device to device; the WRT-54GL, for example, waits for a tftp connection for a couple of seconds on startup, which you can use to send a new firmware; many other devices have a boot loader that you have to explicitly control (via the console), but it will make a tftp connection out to another machine if you ask it to. Some devices you just need to load the data in via serial cable (at 115200bps it's bearable; at the 9600bps I've heard some devices work at, it would be tres painful). The DNS-323 runs uBoot as the boot loader, which will do a tftp request by default. However, D-link, or Marvell, or someone in the decision-making chain for this thing decided to knobble it and removed most of the useful commands, so sending the thing via serial port is the only option. Since writing to flash takes a pile of extra time, I found (eventually) that you can just load the data into RAM somewhere and boot from there, without needing to write it to flash (also handy to avoid wasting precious write cycles), since you can tell uBoot to boot a kernel located anywhere in the address space. Obviously you need to reload the kernel/initrd on every boot if you don't write it to flash, so if you've got a "stable" kernel or initrd that you think you'll be using for a while (when you're testing the other item) you should flash it then. One note on uBoot, that took me far too long to work out -- you can't just stuff a kernel fresh off the Makefile in and assume it'll work -- uBoot needs a special header prepended that tells it all about what you're loading (architecture, file type, compression method, etc). You "wrap" a file in this uBoot header with the "mkimage" command (in the uboot-mkimage package in Debian). Incidentally, if you need to remove the uBoot bits from a file, just strip the first 64 bytes off -- that's all the uBoot header is, so something like this will do nicely:

 dd if=file.uboot of=file bs=64 skip=1

A final uBoot warning: it's fairly important to specify a "load address" for the files; if you get them wrong, your kernel and initrd may or may not boot (I never quite got the hang of why some wrong values worked and some didn't). If your device uses uBoot, and you have the ability to extract the uBoot-enabled kernel and initrd from a vendor-provided firmware image, you can use mkimage -l to inspect what all the values need to be for your own kernels and initrds. Just stick with them. Does Debian support your hardware? Obviously, if you want to run Debian on something, Debian has to support that hardware. The easiest way to test this out is to try an install. For this, you'll need to retrieve an appropriate kernel and initrd pair for the installer. Some arches, like i386 and amd64, only have one kernel and initrd for the installer, so you just have to choose if you're going to netboot, or do a full CD install, or whatever. Other arches, though, like arm, have subtly different images for different machines. There are two reasons they vary: firstly, they often need to split into subarches, where they all share a general CPU architecture but there are enough differences in the hardware that they're not all supportable by the same kernel. At a more fine-grained level some different machines of the same sub-architecture need differently formatted firmware images to boot. On ARM, in particular, it's pretty common to need to prepend a "device ID" to the kernel image so that the kernel knows what sort of machine it's booting on, which means that every machine typically needs it's own kernel image, regardless of any other issues with firmware formats. Choosing the sub-architecture should be relatively straightforward, if you know much about the machine you're targetting. Picking a machine type can be a bit harder, but again, if you know what your boot loader needs, it shouldn't be too hard to find a kernel/initrd from the existing machines in your subarch that you can mangle to fit. Load up your kernel/initrd into the device, boot it, and see how far it gets. You might get lucky, and the kernel inits, your hard drives are detected, your NIC is initialised, and you're off and running without further ado. Or you might not be so lucky. The worst case is that the kernel doesn't even get as far as loading the initrd -- it might not print anything at all, or the kernel might hang part way through the boot. This is likely to require a fair amount of debugging fun, as initial boot failures get a bit tricky. A better situation is where the kernel boots OK, and d-i starts, but something in the install process falls over. This can either be a kernel/driver problem, or a d-i problem. Either way, since d-i has started, you're likely to be able to get a shell and do some debugging. In order to make d-i work, you need to have somewhere to install to (disks, usually) and a source of packages to install from (CD/DVD if you're on a suitably large device, or the network otherwise). These are the two key ingredients you need; most everything else is stuff that ranges between "nice to have" and "critical" for operational purposes, but which has absolutely zero impact on your ability to do the install, so we'll ignore it for now. For the DNS-323, I had a few kernel problems (which I'd been warned about by Martin before I started) -- the NIC wasn't initialised with it's MAC address, which meant it was running around my Ethernet screaming "I am 00:00:00:00:00:00! Ph33r m3!", and the SATA controller just didn't seem to exist at all. These were kinda critical to doing an install, so it was time to get hacking to make those work. There were also a couple of parts of d-i that needed tweaking -- the preseeding code didn't support the device, and I needed to package a tool to build a "complete" uploadable firmware image and teach d-i how to use it. Apart from that, though, things were looking pretty good. Make the Debian kernel support your hardware Once you've worked out what hardware you need to make work, you need to dig into the kernel and fix it. Sometimes, if you're really lucky, a kernel version newer than the one currently in Debian has better support for something, and you can backport the fix / feature improvement. Other times, there's something similar already in the kernel for a different device and you can crib off that (I did this extensively for my little fixes). Then there's the "ask an expert" school, where you find someone who knows what they're on about and pick their brain (I did a bit of that, too). Finally, there's plain old "research and development", for which there are piles of documentation already out there. My changes for the DNS-323 came to 121 added lines and 16 deleted lines, most of which was cribbed-and-tweaked from other parts of the kernel. I might have written 10-20 lines of code "from scratch". It's not always scary-hard. Cross-compiling your own kernel One thing that I think needs mentioning in the kernel hacking department is cross-compiling a modified kernel for testing purposes. Chances are your main workstation has a lot more grunt (and build tools) than whatever you're trying to get Debian onto, so cross-compilation is the way to go. You need to first grab a copy of the source you want to build (from apt-get source linux-image-blahdiblah, or a release tarball, or Linus' git repo, it doesn't really matter) and a cross-compilation toolchain. For my cross-compilation toolchain, I just added this to my sources.list:

 deb http://www.emdebian.org/debian/ unstable main

then I ran apt-get update, and installed these packages:

	binutils-arm-linux-gnueabi gcc-4.1-arm-linux-gnueabi
	gcc-4.3-arm-linux-gnueabi-base libc6-dev-armel-cross

Kudos to the emdebian team for making that such an utterly painless exercise, by the way -- the days of needing to faff round with gcc sources to get a cross-compilation toolchain are apparently over. Once you've got a source tree and a cross-compiler available, you can run any of the usual kernel make commands (like make zImage or make oldconfig), as long as you've got a couple of make variables defined. In my case, I used:

 ARCH=arm CROSS_COMPILE=arm-linux-gnueabi-

ARCH tells the kernel what architecture to build for (useful in all make commands because not all arches have the same config options in make menuconfig, for example) and CROSS_COMPILE tells the build system what to prefix the calls to gcc, ld and such with (if you installed the same cross-compiler packages as I did, you'll find it installs files like /usr/bin/arm-linux-gnueabi-gcc, so you can ls /usr/bin/*gcc to find out what you'll need for the CROSS_COMPILE setting). Hack d-i to make things work In addition to tweaking the kernel, it's quite possible that the internals of d-i might need some tuning. It's possible that d-i might need to be "taught" to recognise the hardware in your device, or you'll need to extend some aspect of the installation process to complete the installation. In order to hack d-i, you need to know a bit about how it works and how it is structured, so let's run through that first. The entireity of d-i is based around udebs, which are packages of the same format as "real" debs, but which have been built differently (against a small libc, for instance) and have had everything that isn't absolutely necessary (docs, non-essential scripts, extra functionality, etc) stripped out of them to create the smallest possible packages. There are a variety of different udebs -- most are derived from otherwise normal, law-abiding citizens of the archive, like busybox (basic unix utilities like cp and ls), the shell that's used in the installer, parted, various libraries, and so on. Others are "special", and have no big brother in the archive. These are mostly "task" udebs, that define what actually happens in the installer. When they're "installed", they have special fields in the package metadata that define where in the install sequence they go, and they end up on the menu you see if your installation goes a bit pear-shaped. The tricky thing about udebs is that, as small as they are, there's too many of them to be able to have them all available when the system first boots, so even the "setup" of the installation gets done in several stages. First, the kernel and initrd boot. The initrd contains just enough stuff to get access to the rest of the udebs, and nothing else. Even this early stage is built from udebs -- there's lists of what goes into the initrd for various architectures and flavours of installer under installer/build/pkg-list in the d-i SVN. If you need an extra udeb in your initrd, that's the place to specify it. Once the initrd has gotten access to a source of packages (off a CD, over the network, telepathy, whatever) it can drag the rest of the packages it needs from there. Space is still an issue at this point, though -- there's no diskspace yet to put anything onto, so everything needs to fit into a ramdisk. Don't go loading anything you don't need. The packages loaded at this stage are what defines most of the steps that will be done for the rest of the install, so once all those extra udebs are loaded, the remainder of the install kicks off and does it's thing. Different architectures and install methods have different requirements, too, which complicates matters. For example, when you're installing on a regular PC with screen/keyboard, you can guide the install using that. However, if you've got no screen, how can you control the install? There's serial consoles, but requiring everyone to void the warranty (more) on their little embedded doodad by installing a serial console cable to install Debian isn't so hot. So, lacking a terminal, how do you control the install? Well, on install flavours for devices that might be in this situation, you can SSH into the installer and complete it that way. That feature is, like everything else, provided in a udeb -- called network-console, as it happens. But what IP address do you connect to, and what network settings does the device use? Again, without a console you can't configure your network, and not everyone has a DHCP server (and even fewer people have a DHCP server that will hand out an address that they'll know in advance, or be able to find out). So, there's a udeb (called oldsys-preseed) that does it's best to configure your network based on the settings already in the nvram for your device. In short, there's a lot of tricky, well-thought-out shit going on inside d-i, and you might have to deal with a little bit of it. The nice thing is that by now I expect most of the really hard problems have been solved, so with a bit of luck you'll only ever have to do minor tweaking. In general, all of the packages that are "d-i specific" live in packages in the d-i SVN. If you need to modify a udeb, you need to rebuild it and put it in installer/build/localudebs/ (creating it first if need be) and every time you change it, you need to delete installer/build/apt.udeb/cache/. To give a concrete example (from the ever-present DNS-323), I had to modify oldsys-preseed so that it would get the details from the nvram that the stock firmware uses to configure itself. This, in turn, required that the minix-modules udeb be included in the initrd, because the DNS-323's nvram is actually a small minix partition stored in flash. Painfully, this required some more hacking, because the armel d-i kernel wasn't configured to generate minix-modules. That might need some explanation... Since building the kernel is a fairly massive undertaking, CPU-wise, instead of building all the udebs from scratch there's a package called kernel-wedge to split a real linux-image package into lots of little udebs. There's a separate source package for each architecture, linux-kernel-di-$ARCH-2.6, that contains the configuration data for kernel-wedge to specify what udebs need to be built, and what goes into each of them. A small tweak to the armel package and voila! I had a minix-modules udeb to include in the initrd. With the config stuff I'd done in the d-i build itself and my changes to oldsys-preseed, we were done with d-i. Tell d-i how to build your bootable artifacts If your device needs a "different" install image from other similar devices, you need to write some makefile rules to build those files. Look under the installer/build/config/$ARCH/ directory to see the various rules files available. I had to package a new tool to create uploadable firmware images for the DNS-323, along with adding a rule to use that tool to build images. None of it was particularly difficult. Polish the diamond Getting d-i working is only part of the battle; it's quite likely that there will be other hardware in the machine that needs to be supported. This could involve kernel-level stuff to interface to the hardware, or userland tools to manage the hardware. There's several bits in the DNS-323 that still need work; I can't control the LEDs on the front panel, nor the fan, and I can't use the on-board temperature sensors yet. But, I do have Debian running on the thing, which is what I was shooting for. Examples If you're interested in exactly what I did, there are a number of bugs in the BTS that contain the patches I made:

• #502821: oldsys-preseed changes to take existing network config from NVRAM;
• #502936: build a minix modules package for the armel d-i kernel;
• #503040: d-i core changes to build firmware images for the DNS-323;
• #503172: kernel patches to support the SATA controller and NIC better.

One month with Emacs and counting - Part 1 Straight to the point: since mid September I've been using Emacs, trying to evaluate whether I was willing to switch from Vim to it. Yup, that's true, me (user of Vim since the day I've started using GNU/Linux 10 years ago, (not so) active maintainer in Debian of vim and related packages, author of some popular Vim extensions and of vim-addon-manager) it's considering switching to Emacs. What is worse is that I've de facto already switched. May jamessan forgive me. OK, that was the hardest part to write, now ... This choice will have an impact on me: partly because I'm quite sure several fellow DDs will start making fun of this story , and partly because as many geeks I've always had strong opinions on the editor war. Switching side ain't easy. Hence, I've decided to write a small (2-3) series of blog posts on the issue, to future memory. The post you are reading is about why, since a few months ago, I wasn't willing to give Emacs a try, and how I've changed my mind. Why I wasn't willing to give Emacs a try, but then changed my mind I love knowing tools which can improve my workflow, no matter the task. Editors happen to be at the intersection of many tasks, hence knowing well his own editor and feeling satisfied about it is quite important. This is even more so for free software hackers which, for a reason or another, happen to use the very same editor for a lot of different tasks; in my case: programming, conf file hacking (i.e., playing the sysadm role), mail writing, and everything in between. I've started using vi on Solaris around 1998, then switched to Vim starting from version 4.0, and then followed all its releases up to 7.0. I consider my "Vim karma" to be quite hight and I've delivered several talks about using Vim for LUGs and other free software-related events. Still, I've always been hit bit several minor nuisances and glitches of Vim which couldn't be fixed by simple configuration tuning or add-on implementations (more on this in a future post). In the quest for the perfect workflow, those glitches have made me try several times other editors hoping for more satisfying work environments. Of course Emacs was one of them, which I've tried repeatedly during the years; the last time was circa 2006. For one reason or another, after a few days of experiments, I've always decided to give up with it. Why? Various reasons, listed below, together with an explanation of what (I think) has changed since then.

1. Poor Unicode/multibyte support. Support for whatever Unicode encoding was close to null. It was not available in the legacy editor, and you had to use something called MULE, to be specifically enabled. That was so 60's, and rather hard to use. Now (apparently starting from Emacs 22.1, June 2007) finally you have built-in Unicode support and the needed two variables for setting the input encoding and the file encoding, which is basically all you need. Looks easy once you have it ...
2. Lack of "modern" UI toolkit support. Yes, I do live in a console and invoke my editor from it, but nevertheless for long coding sessions I do also love having a nice GTK+ window containing my editor using settings which smoothly integrated with the graphical settings of my desktop environment. Maybe I'm getting old, maybe GNOME has changed my perceptions, but why the heck I had to tolerate different monospace fonts in my gnome-terminal wrt my GUI editor I never understood. Firing up Emacs was like getting back in time of 15 years, and if a geek having a maximized terminal on most of its workspaces had this feeling, then something was really wrong. This has changed: starting again from Emacs 22.1, the editor has frown support for that "bleeding edge" technology called GTK+. Finally you can use Pango font faces and avoid the time travel feeling when switching from a terminal to your GUI.
3. "Dead upstream" syndrome. As we all know by experience, choosing a free software product is not only a matter of evaluating bugs and features, there are other environmental factors. A notable example is how active upstream is. A few of us will use a product affected by the "dead upstream" syndrome. That's exactly what I was feeling about Emacs. The first argument I can offer to back that feeling is again the Emacs release history (noted that "small" hole between 2001 and 2007? Rest In Peace release early, release often ...). The second argument is the bug tracking system: there was none, bug discussions happened only on the development mailing list and the bug tracker was a text file on CVS! (Yes, Vim doesn't have a proper BTS either, but I was looking for reasons to switch from it and hence looking for something better in several respects.) That has changed completely, as I discovered only at this year DebConf thanks to gismo. 2007 has enjoyed 1 Emacs release and 2008 already two. They do have a BTS now and guess what? is dear old debbugs. According to my first experiences as Emacs bug reported the maintainers are very reactive and also friendly in dealing with bug reporters. Finally, also keeping up with snapshot releases in Debian is entirely trivial, thanks to Romain's repository.
4. Poor performances & memory consumption. This goes in the direction of probably the most popular joke used by Vim zealots in the editor war: Emacs is a nice operating system, but lacks a good editor, or something such. In the past I had the impression that starting Emacs was really slow and that generally the memory footprint was too high to be acceptable. You know what: starting a full-blown gvim with a good deals of add-ons (and I used quite a lot of them, that's why vim-addon-manager was born) is not that faster. And on the contrary, firing up an emacsclient is way faster than starting up a new gvim instance. Regarding memory consumption Emacs is still "gaining" the battle. Still, one has to keep in mind the different workflow: with Vim you are encouraged to enter and exit the editor (not without drawbacks, as I'll discuss in a future post), while with Emacs you always use the same one. It turns out that via emacsclient the resulting feeling is the same as having multiple editors (as you can fire them up as separate windows or in separate consoles), still sharing all user memory. A final comment on memory consumption while looking at my top sorted by decreasing RSS memory: the top process is xulrunner (from the browser) with 356Mb, then Xorg (159M), pidgin (128Mb !!!!), trackerd, gnome-terminal, gnome-panel, ..., Emacs shows up with 51 Mb at the 7th position. Yes, it is not the thinnest editor in the world, but on average machine it doesn't really matter at all.

In the next post: how to switch from Vim to Emacs, without loosing your mental sanity. (Whether I did succeed wrt sanity is up to the reader to judge.)

I have access to a server in Germany that was running Debian/Etch i386 but needed to be running Xen with the AMD64 version of Debian/Lenny (well it didn’t really need to be Lenny but we might as well get two upgrades done at the same time). Most people would probably do a complete reinstall, but I knew that I could do the upgrade while the machine is in a server room without any manual intervention. I didn’t achieve all my goals (I wanted to do it without having to boot the recovery system - we ended up having to boot it twice) but no dealings with the ISP staff were required. The first thing to do is to get a 64bit kernel running. Based on past bad experiences I’m not going to use the Debian Xen kernel on a 64bit system (in all my tests it has had kernel panics in the Dom0 when doing any serious disk IO). So I chose the CentOS 5 kernel. To get the kernel running I copied the kernel files (/boot/vmlinuz-2.6.18-92.1.13.el5xen /boot/System.map-2.6.18-92.1.13.el5xen /boot/config-2.6.18-92.1.13.el5xen) and the modules (/lib/modules/2.6.18-92.1.13.el5xen) from a CentOS machine. I just copied a .tgz archive as I didn’t want to bother installing alien or doing anything else that took time. Then I ran the Debian mkinitramfs program to create the initrd (the 32bit tools for creating an initrd work well with a 64bit kernel). Then I created the GRUB configuration entry (just copied the one from the CentOS box and changed the root= kernel parameter and the root GRUB parameter), crossed my fingers and rebooted. I tested this on a machine in my own computer room to make sure it worked before deploying it in Germany, but there was still some risk. After rebooting it the command arch reported x86_64 - so it had a 64bit Xen kernel running correctly. The next thing was to create a 64bit Lenny image. I got the Lenny Beta 2 image and used debootstrap to create the image (I consulted my blog post about creating Xen images for the syntax [1] - one of the benefits of blogging about how you solve technical problems). Then I used scp to copy a .tgz file of that to the server in Germany. Unfortunately the people who had set up that server had used all the disk space in two partitions, one for root and one for swap. While I can use regular files for Xen images (with performance that will probably suck a bit - Ext3 is not a great filesystem for big files) I can’t use them for a new root filesystem. So I formatted the swap space as ext3. Then to get it working I merely had to update the /etc/fstab, /etc/network/interfaces, and /etc/resolv.conf files to make it basically functional. Of course ssh access is necessary to do anything with the server once it boots, so I chrooted into the environment and ran “apt-get update ; apt-get install openssh-server udev ; apt-get dist-upgrade“. I stuffed this up and didn’t allow myself ssh access the first time, so the thing to do is to start sshd in the chroot environment and make sure that you can really login. Without having udev running a ssh login will probably result in the message “stdin: is not a tty“, that is not a problem. Getting that to work by the commands ‘ssh root@server “mkdir /dev/pts”‘ and ‘ssh root@server “mount -t devpts devpts /dev/pts”‘ is not a challenge. But installing udev first is a better idea. Then after that I added a new grub entry as the default which used the CentOS kernel and /dev/sda1 (the device formerly used for swap space) as root. I initially used the CentOS Xen kernel (all Red Hat based distributions bundle the Xen kernel with the Linux kernel - which makes some sense). But the Debian Xen utilities didn’t like that so I changed to the Debian Xen kernel. Once I had this basically working I copied the 64bit installation to the original device and put the 32bit files in a subdirectory named “old” (so configuration can be copied). When I changed the configuration and rebooted it worked until I installed SE Linux. It seems that the Debian init scripts will in many situations quietly work when the root device is incorectly specified in /etc/fstab. This however requires creating a device node somewhere else for fsck and the SE Linux policy version 2:0.0.20080702-12 was not permitting this. I have since uploaded policy 2:0.0.20080702-13 to fix this bug and requested that the release team allow it in Lenny - I think that a bug which can make a server fail to boot is worthy of inclusion! Finally to get the CentOS kernel working with Debian you need to load the following modules in the Dom0 (as discussed in my previous post about kernel issues [2]):
blktap
blkbk
netbk It seems that the Debian Xen kernel has those modules linked in and the Debian Xen utilities expect that. Currently I’m using Debian kernels 2.6.18 and 2.6.26 for the DomUs. I have considered using the CentOS kernel but they decided that /dev/console is not good enough for the console of a DomU and decided to use something else. Gratuitous differences are annoying (every other machine both real and virtual has /dev/console). If I find problems with the Debian kernels in DomUs I will change to the CentOS kernel. Incidentally one problem I have had with a CentOS kernel for a DomU (when running on a CentOS Dom0) was that the CentOS initrd seems to have some strange expectations of the root filesystem, when they are not met things go wrong - a common symptom is that the nash process will go in a loop and use 100% CPU time. One of the problems I had was converting the configuration for the primary network device from eth0 to xenbr0. In my first attempt I had not installed the bridge-utils package and the machine booted up without network access. In future I will setup xenbr1 (a device for private networking that is not connected to an Ethernet device) first and test it, if it works then there’s a good chance that the xenbr0 device (which is connected to the main Ethernet port of the machine) will work. After getting the machine going I found a number of things that needed to be fixed with the Xen SE Linux policy. Hopefully the release team will let me get another version of the policy into Lenny (the current one doesn’t work).

• [1] http://etbe.coker.com.au/2007/01/01/installing-xen-domu-on-debian-etch/


• [2] http://etbe.coker.com.au/2008/10/22/kernel-issues-with-debian-xen-and-centos-kernels/

Share This

Last time I tried using a Debian 64bit Xen kernel for Dom0 I was unable to get it to work correctly, it continually gave kernel panics when doing any serious disk IO. I’ve just tried to reproduce that problem on a test machine with a single SATA disk and it seems to be working correctly so I guess that it might be related to using software RAID and LVM (LVM is really needed for Xen and RAID is necessary for every serious server IMHO). To solve this I am now experimenting with using a CentOS kernel on Debian systems. There are some differences between the kernels that are relevant, the most significant one is the choice of which modules are linked in to the kernel and which ones have to be loaded with modprobe. The Debian choice is to have the drivers blktap blkbk and netbk linked in while the Red Hat / CentOS choice was to have them as modules. Therefore the Debian Xen utilities don’t try and load those modules and therefore when you use the CentOS kernel without them loaded Xen simply doesn’t work. Error: Device 0 (vif) could not be connected. Hotplug scripts not working. You will get the above error (after a significant delay) from the command “xm create -c name” if you try and start a DomU that has networking when the driver netbk is not loaded. XENBUS: Timeout connecting to device: device/vbd/768 (state 3) You will get the above error (or something similar with a different device number) for every block device from the kernel of the DomU if using one of the Debian 2.6.18 kernels, if using a 2.6.26 kernel then you get “XENBUS: Waiting for devices to initialise“. Also one issue to note is that when you use a file: block device (IE a regular file) then Xen will use a loopback device (internally it seems to only like block devices). If you are having this problem and you destroy the DomU (or have it abort after trying for 300 seconds) then it will leave the loopback device enabled (it seems that the code for freeing resources in the error path is buggy). I have filed Debian bug report #503044 [1] requesting that the Xen packages change the kernel configuration to allow more loopback devices and Debian bug report #503046 [2] requesting that the resources be freed correctly. Finally the following messages appear in /var/log/daemon.log if you don’t have the driver blktap installed:
BLKTAPCTRL[2150]: couldn’t find device number for ‘blktap0′
BLKTAPCTRL[2150]: Unable to start blktapctrl It doesn’t seem to cause a problem (in my tests I can’t find something I want to do with Xen that required blktap), but I have loaded the driver - even removing error messages is enough of a benefit. Another issue is that the CentOS kernel packages include a copy of the Xen kernel, so you have a Linux kernel matching the Xen kernel. So of course it is tempting to try and run that CentOS Xen kernel on a Debian system. Unfortunately the Xen utilities in Debian/Lenny don’t match the Xen kernel used for CentOS 5 and you get messages such as the following in /var/log/xen/xend-debug.log: sysctl operation failed — need to rebuild the user-space tool set?
Exception starting xend: (13, ‘Permission denied’) Update: Added a reference to another Debian bug report.

• [1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=503044


• [2] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=503046

Share This

So we are about 1/2 way done with my hacker class and I have to say it has been a pretty fun ride, especially since many students thought hacking was all about pimply kids engaging in malicious acts of computer violence and many had no knowledge of basic technological concepts like UNIX or source code. We had to first sweep away the cobwebs of misrepresentation and replace with a more solid foundation of facts (however messy the world of hacking still is!) But after building a foundation, you still wonder whether students are learning. Traditionally we gauge progress with exams or essays, which can be effective but let’s face it, at times a little tedious. But today a student sent me the following very short email, which made me realize I had another barometer at my disposal to gauge their progress xkcd:

http://xkcd.com/344/ I wouldn’t have gotten the joke if it wasn’t for you. Thanks.

The email not only made me smile (for it is always nice to know your students are leaning something) but it gave me the idea that in the future I might include a comic based exams composed of 5-10 comic strips (many from xkcd) and ask for an exegesis of them. Why not? Seems like fun to me. And it would be great to include the following as one of my course objectives: “By the end of this class you will be able to read xkcd and actually understand (most) of it.” If they could do that, well, they must have learned at least something update: One of my readers, Florian, sent this nice bit, which I have not read before: “I read your blog via Planet Debian and immediately felt reminded of Jane
Goodall’s Foreword to Gary Larson’s The Far Side Gallery 5 which I’d
like to share with you:”

Recently I was talking with one of the best researcher know, Tim
O’Halloran. He has been able to inspire generations of middle and high
school students to care for the natural world. Tim told me that Gary
Larson has had a major impact on his teaching. Tim uses Far Side
cartoons to introduce topics, to illustrate points, and to “reinforce
the notion that the more we investigate the universe, the richer is our
experience.” When designing exam papers Tim finds the cartoons “ease the
tension and spark the memory.” It all began when, in the fall of 1985,
he was given the task of teaching science to 162 Tulsa ninth-graders who
were convinced that it was absolutely irrelevant to their futures. Tim
put one hundred Far Side cartoons on a large bulleting board, and told
the students to study them. The consensus was that they didn’t
understand the humor - The Far Side was “too weird”. However, Tim wrote
me, “Each time we completed a unit and the students approached the
bulletin board with newly acquired wisdom, I smiled quietly and thanked
the cosmos for Gary’s perspectives as the kids roared with the confident
laughter of the enlightened.” Having taught myself for a while, I can fully and happily relate.

Later in the semester, and for my hacker course, we will be reading Understanding Comics, which will give us a bit of a meta perspective on why comics are so good at conveying a certain type of message. I can’t wait to read it as I have heard it is fantastic.