Show local configuration changes Following a train of thought, I got to the idea that it would be nice to have a way to ask an installed package to show a diff between the configuration that it would autogenerate and the configuration that is on the system. For example:

# /var/lib/dpkg/info/postfix.postinst config-diff
--- a/aliases
    +++ b/aliases
    @@ -2,3 +2,7 @@
     postmaster:    root
     clamav: root
     logcheck: root
    +
    +root: enrico
    --- a/postfix/main.cf
    +++ b/postfix/main.cf
    @@ -25,14 +25,35 @@ smtp_tls_session_cache_database = btree:$ queue_directory /smtp_scache
     alias_database = hash:/etc/aliases
     myorigin = /etc/mailname
     mydestination = enricozini.org, localhost.localdomain, localhost
    +virtual_alias_domains = enricozini.com
    +virtual_alias_maps = hash:/etc/postfix/aliases-virtual
     relayhost = 
     mynetworks = 127.0.0.0/8
     mailbox_command = procmail -a "$EXTENSION"
     mailbox_size_limit = 0
     recipient_delimiter = +
     inet_interfaces = all
    +message_size_limit = 20480000

This would be quite useful (if you're not using a tool such as etckeeper): partly for backups, but more interestingly for inspecting a Debian box, to see what is stock configuration and what has been manually changed. Moreover, this could be used to automately create an installation CD that would install a system with the same configuration, by saving the set of debconf preseeds and the set of /etc/ patches. In theory, it would not be too difficult to generate such a diff: for packages that don't change config files in postinst, it's just a matter to diff with the original files as found in the .deb. For the other packages, the code to generate the configuration can be recycled outputting the results to a temporary directory, then a diff can be made with the corresponding files in /etc. I can see this being easily doable for 90% of the packages, and I am wondering about the rest: is it thinkable to pursue this way, assuming that what wouldn't support it today could be made to support it in the future, or are there some configuration cases that just cannot be made to work in this way, no matter what? Note

• There is already a dpkg proposal to do this

Yesterday I wrote that I'd ordered a new ASUS EEE PC, the 901 model, and today it arrived. The machine is gorgeous though I suspect in the long term I'll regret ordering the white model. This entry was written on the device, slowly due to my fat fingers, with its new Debian Lenny operating system. To save time I didn't even use the default system, I just immediately rebooted it into the installer via a 2Gb USB stick. Unfortunately I had to run through the installation twice, because I made some bad partitioning decisions and decided to fix them rather than live with them. Happily everything on the device appears to work perfectly, although there were a few hiccups along the way. The only niggle is that suspend to RAM seems a little flaky; 50% of the time I try to resume and just get a blank window not my X.org desktop. Happily suspend-to-disk works perfectly, and the bootup/restoration process looks very pretty with the splashy package installed. There are times when I really love using Debian, and this is definitely one of them. Together we've produced an operating system which just works on an amazing array of devices and systems and gets better and better as time goes on! For example in the past I'd always regarded Network Manager as "that thing we remove to stop breaking our system" - but now I see it working correctly with no effort on my part at all. Amazing! I guess I'll be returning my Nokia internet tablet in the near future. This device is bigger, but much more capable and versatile. ObFilm: St Trinians (the recent remake; not too bad. Bonus points for the Shampoo cover).

It is often argued that companies have to work with the FOSS community and there are good reasons for doing so. I've tried to collect a number of case stories of good and bad community interaction that may help as a starting point for further exploration of this topic:

• NVidia is probably the best example of a company that is not interested in providing open source drivers for their graphics chips. There are lots of folks who won't buy hardware with NVidia chips for this reason. Intel is the number 1 choice at the moment because they work closely with the community. Related to this, the Linux Foundation and kernel community released a statement regarding open drivers.
• When Sun finally made Java available as open source, they argued that it would help inclusion and distribution of Java (e.g. inclusion into Debian). Sun also felt that the GPL provides the strongest protection "against misbehaviour by monopolists in the Java platform".
• The Novell/Microsoft deal was not perceived well by the community, which led to web sites like http://boycottnovell.com
• Nokia recently had an insightful blog posting about the problems when nobody knows about the open source contributions you make: for example, you cannot influence decisions in a project as easily.
• Sun is often criticized for dominating all open source projects they run. They can be used as a good example that you have to give up some control in order to get outside participation.
• The fact that Xen is not in the mainline kernel is a key reason why a number of companies (e.g. IBM, Red Hat) are looking at KVM (which is in the mainline kernel and actively developed in the community). In fact, "looking" is probably not the best word. IBM is actively working on KVM to make it a viable solution and Red Hat recently acquired Qumranet, the company behind KVM.
• Oracle got fairly bad press for copying the Red Hat distribution and changing Red Hat to Oracle, without engaging in the Red Hat community.
• There's a paper that looks at "empirical evidence on the incentives of firms that engage in OS activities." They look at 146 Italian companies supplying open source solutions, but unfortunately the study doesn't include any major company.

Can you think of other examples? (Originally published on FOSSBazaar where comments are possible)

I just read the news that PC Magazine is being canceled. It’s not exactly a shock, given the state of technical magazines right now. I haven’t read one of those in years, since they turned to be more of a consumer than a technical publication. But I hope I am not the only one out there that remembers PC Magazine from the mid to late 1980s. I had two favorite parts in each issue: the programming example, and the “Abort, Retry, Fail” page at the back of the magazine. The programming example was usually some sort of DOS (or, on occasion, OS/2) utility. It was usually written in assembly, and would be accompanied by a BASIC program you could type in to get the resulting binary, as assemblers weren’t readily available. The BASIC program was line after line of decimal numbers that would decode them and write out the resulting binary — sort of a primitive uuencode for paper. Trying to type those in gave me some serious eyestrain on more than one occasion. By now, I forget what most of those utilities did, but I remember one: BatchMan. It was a collection of tools for use in DOS batch files, and could do things like display output in color or even — yes — play monophonic music. It came with an example that displayed some lyrics about batch programming on-screen, set to what I later realized was the Batman theme. Geek nirvana, right? But Batchman was too big to publish the source code, or the BASIC decoder, in print. It might have been one of those things that eventually led me to a CompuServe account. PC Magazine had some deal with CompuServe that you could get their utilities for free, or reduced cost — I forget. CompuServe was probably where I sent my first email, from my account which was 71510,1421 — comma and all. In later years, you could pay a small fee to send email to the Internet, and I had the amazingly attractive email address of 71510.1421@cis.compuserve.com. Take that, gmail. PC Magazine eventually stopped running utilities that taught people about assembly or batch programming and shifted more to the genre of Windows screensavers. They stopped their articles about how hard disks work and what SCSI is all about, and instead have cover stories like “Vista made easy!” I am, sadly, not making this up. Gone are the days of investigating alternative operating systems like OS/2. It appears that “Abort, Retry, Fail” is gone, too. It was a one-page thing at the back of each magazine that featured braindead error messages and funny stories about people that did things like FAX an image of a floppy disk to a remote office — before such stories were cliche. Sort of like DailyWTF these days. The sad truth is that the people that would FAX an image of a floppy are probably the ones that are reading PC Magazine today. I still have a bunch of PC Magazine issues — the good ones — in my parents’ basement. I also still have my floppies with the utilities on them somewhere. One day, when I get some time — I’m estimating this will be about when Jacob goes to college — I’ll go back and take another look at them.

My current desktop PC has an RS790-based Radeon on-board graphics controller. It also has a Radeon X1900 plugged in. Playing with my Watts Up, I found that the system was (at idle!) drawing around 35W more power with the X1900 than with the on-board graphics.

This is clearly less than ideal.

Recent Radeons all support dynamic clock gating, a technology where the clocks to various bits of the chip are turned off when not in use. Unfortunately it seems that this is generally already enabled by the BIOS on most hardware, so playing with that didn't give me any power savings. Next I looked at Powerplay, the AMD technology for reducing clocks and voltages. It turns out that my desktop hardware doesn't provide any Powerplay tables, so no joy there either. What next?

Radeons all carry a ROM containing a bunch of tables and scripts written in a straightforward bytecode language called Atom. The idea is that OS-specific drivers can call the Atom tables to perform tasks that are hardware dependent, even without knowledge of the specific low-level nature of the hardware they're driving. You can use Atom to do several things, from card initialisation through mode setting to (crucially) setting the clock frequencies. Jerome Glisse wrote a small utility called Atomtools that lets you execute Atom scripts and set the core and RAM frequencies. Playing with this showed that it was possible to save the best part of 5W by underclocking the graphics core, and about the same again by reducing the memory clock. A total saving of 9-10W was pretty significant.

The main problem with reducing the memory clock was that doing it while the screen is being scanned out results in memory corruption, showing up as big ugly graphical artifacts on the screen. I'm a fan of doing power management as aggressively as possible, which means reclocking the memory whenever the system is idle. Turning the screen off to reclock the memory would avoid the graphical corruption but introduce irritating flicker, so that wasn't really an option. The next plan was to synchronise the memory reclocking to the vertical refresh interval, the period of time between the bottom of a frame and the top of the next frame being drawn. Unfortunately setting the memory frequency took somewhere between 2 and 20 milliseconds, far too long to finish inside that time period.

So. Just using Atom was clearly not going to be possible. The next step was to try writing the registers directly. Looking at the R500 register documentation showed that the MPLL_FUNC_CNTL register contained the PLL dividers for the memory clock. Simply smacking a new value in here would allow changing the frequency of the memory clock with a single register write. It even worked. Almost. I could change the frequency within small ranges, but going any further resulted in increasingly severe graphical corruption. Unlike the sort I got with the Atom approach to changing the frequency, this corruption manifested itself as a range of effects from shimmering on the screen down to blocks of image gradually disappearing in an impressively trippy (though somewhat disturbing) way.

Next step was to perform a register dump before and after changing the frequencies via Atom, and compare them to the registers I was programming. MC_ARB_RATIO_CLK_SEQ was consistently different, which is where things got interesting. The AMD docs helpfully describe this register as "Magic field, please use the excel programming guide. Sets the hclk/sclk ratio in the arbiter", about as helpful as being told that the register contents are defined by careful examination of a series of butterflies kept somewhere in Taiwan. Now what?

Back to Atomtools. Enabling debugging let me watch a dump of the Atom script as it ran. The relevant part of the dump is here. The most significant point was:

MOVE_REG @ 0xBC09
src: ID[0x0000+B39E].[31:0] -> 0xFF7FFF7F
dst: REG[0xFE16].[31:0] <- 0xFF7FFF7F

, showing that the value in question was being read out of a table in the video BIOS (ID[0x0000+B39E] indicating the base of the ROM plus 0xB39E). Looking further back showed that WS[0x40] contained a number that was used as an index into the table. Grepping the header files gave 0x40 as ATOM_WS_QUOTIENT, containing the quotient of a division operation immediately beforehand. Working back from there showed that the value was derived from a formula involving the divider frequencies of the memory PLL and the source PLL. Reimplementing that was trivial, and now I could program the same register values. Hurrah!

It didn't work, of course. These things never do. It looked like modifying this value didn't actually do anything unless the memory controller was reinitialised. Looking through the Atom dump showed that this was achieved by calling the MemoryDeviceInit script. Reimplementing this from scratch was one option, but it had a bunch of branches and frankly I'm lazy and that's why I work on this Linux stuff rather than getting a proper job. This particular script was fast, so there was no real reason to do it by hand instead of just using the interpreter. Timing showed that doing so could easily be done within the vblank interval. This time, it even worked.

I've done a proof of concept that involved wedging this into the Radeon DRM code with extreme prejudice, but it needs some rework. However, it demonstrates that it's possible to downclock the memory whenever the screen is idle without there being any observable screen flicker. Combine that with GPU downclocking and we can save about 10W without any noticable degradation in performance or output. Victory!

I gave the code to someone with an X1300 and it promptly corrupted their screen and locked their machine up. Oh well. Turns out that they have a different memory controller or some such madness.

So, obviously, there's more work to be done on this. I've put some test code here. It's a small program that should be run as root. It should reprogram an Atom-based discrete graphics card[1] to half its memory clock. Running it again will halve it again. I don't recommend doing that. You'll need to reboot to get the full clock back. This isn't vblank synced, so it may introduce some graphical corruption. If the corruption is static (ie, isn't moving or flickering) then that's fine. If it's moving then I (and/or the docs) suck and there's still work to be done. If your machine hangs then I'm interested in knowing what hardware you have and may have some further debugging code to be run. Unless you have an X1300, in which case it's known to break and what were you thinking running this code you crazy mad fool.

Once this is stable it shouldn't take long to integrate it into the DRM and X layers. I'm also trying to get hold of some mobile AMD hardware to test what impact we can have on laptops.

[1] Shockingly enough, it's somewhat harder to underclock graphics memory on a shared memory system

Some time ago I spoke to Craig Keating about his plans for a new secondary school in the center of Melbourne. His plan was to focus on the core academic areas and cater to academically gifted students. He had some interesting ideas for his business, one of which was to pay teachers rates that are typical for private schools (higher rates than government schools) but not have any sport programs in the evenings or weekends (private schools typically require teachers to work every Saturday and one evening every week in coaching a sport). This would therefore give an hourly pay rate that was significantly higher than most private schools offered and would thus allow recruiting some of the most skilled teachers. One of his ideas was to intentionally keep the school small so that every teacher could know every student. One of the problems with most schools is that they take no feedback from the students. It seems that this serious deficiency would be largely addressed if the teachers knew the students and talked to them. He pointed out that in the history of our school system (which largely derived from the UK system) the private schools had a lot of sporting activities as a way of filling time for boarding students, given that few schools accept boarders (and those that do have only a small portion of the students boarding) the sports are just a distraction from study. This is not to say that sports are inherently bad or should be avoided. He encouraged parents to take their children to sporting activities that suit the interests of the child and the beliefs of the parents instead of having the child be drafted into a school sport and the parents being forced to take an unwilling child to sporting activities that they detest (which I believe is a common school experience). My own observation of school sport is that it is the epicentre of school bullying. There is an inherent risk of getting hurt when engaging in a sport. Some children get hurt every lesson, an intelligent person who ran a school with an intensive sports program might statistically analyse the injuries incurred and look for patterns. Children who are not good at sport are targeted for attack, for example when I was in year 7 (the first year of high school) one of my friends was assigned to the “cork bobbing” team in the swimming contest - this involved a contest to collect corks floating in the toddler pool for the students who were really bad at swimming. At that moment I knew that my friend would leave the school as the teachers had set him up for more intensive bullying than he could handle. Yet somehow the government still seems to believe that school sports are good! This is not to say that physical activity is bad, the PE 4 Life program [1] (which is given a positive review in the movie Supersize Me [2]) seems useful. It has a focus on fitness for everyone rather than pointless competition for the most skilled. I have just seen a sad announcement on the Keating College web site [3] that they will not be opening next year (and probably not opening at all). The Victorian Registration and Qualifications Authority (VCRA) announced in November that the application to be registered as a school (which was submitted in March) was rejected. The first reason for the rejection was the lack of facilities for teaching woodwork and metalwork. As the VCRA apparently has no problems registering girls’ schools that don’t teach hard maths (a teacher at one such school told me that not enough girls wanted to study maths) it seems unreasonable to deny registration to a school that doesn’t teach some crafts subjects and caters to students who aren’t interested in those areas. The second reason was the lack of facilities for sport and PE. Given the number of gyms in the city area it seems most likely that if specific objections were provided eight months earlier then something could have been easily arranged to cover the health and fitness issues. When I spoke to Craig he had specific plans for using the city baths, gyms, and parks for sporting activities, I expect that most parents who aren’t sports fanatics would find that his plans for PE were quite acceptable. The third reason is the claim that 600 square meters of office space is only enough to teach one class of 24 students. That would mean that 25 square meters is needed for each student! I wonder if students are expected to bring their own binoculars to see the teacher or whether the school is expected to provide them. :-# The government has a list of schools that work with the Australian Institute of Sport [4]. These schools provide additional flexibility in studies for athletes and probably some other benefits that aren’t mentioned in the brief web page. I don’t object to such special facilities being made available for the small number of students who might end up representing Australia in the Olympics at some future time. But I think that a greater benefit could be provided to a greater number of students if there were a number of schools opened to focus on the needs of students who are academically gifted. This doesn’t require that the government spend any money (they spend hundreds of millions of dollars on the AIS), merely that they not oppose schools that want to focus on teaching. Currently the government is trying to force Internet censorship upon us with the claim that it will “protect children” [5]. It seems obvious to me that encouraging the establishment of schools such as Keating College will protect children from bullying (which is a very real threat and is the direct cause of some suicides). While so far no-one has shown any evidence that censoring the net will protect any child.

• [1] http://www.pe4life.org/


• [2] http://etbe.coker.com.au/2006/09/22/supersize-me/


• [3] http://www.keatingcollege.com.au/


• [4] https://secure.ausport.gov.au/asc_internet/ais/scholarships/ais_athlete_career_and_education/ais_ace_-_what_we_do/educational_guidance


• [5] http://etbe.coker.com.au/2008/11/18/other-reasons-for-not-censoring/

Australia has slow Internet access when compared to other first-world countries. The costs of hosting servers are larger and the cost of residential access is greater with smaller limits. I read news reports with people in other countries complaining about having their home net connection restricted after they transfer 300G in one month, I have two net connections at the moment and the big (expensive) one allows me 25G of downloads per month. I use Internode, here are their current prices [1] (which are typical for Australia - they weren’t the cheapest last time I compared but they offer a good service and I am quite happy with them). Most people in Australia don’t want to pay $70 per month for net access, I believe that the plans which have limits of 10G of download or less are considerably more popular. Last time I investigated hosting servers in Australia I found that it would be totally impractical. The prices offered for limits such as 10G per month (for a server!) were comparable to prices offered by Linode [2] (and other ISPs in the US) for hundreds of gigs of transfer per month. I have recently configured a DomU at Linode for a client, Linode conveniently offers a choice of server rooms around the US so I chose a server room that was in the same region as my client’s other servers - giving 7 hops according to traceroute and a ping time as low as 2.5ms! Currently I am hosting www.coker.com.au and my blog in Germany thanks to the generosity of a German friend. An amount of bandwidth that would be rather expensive for hosting in Australia is by German standards unused capacity in a standard hosting plan. So I get to host my blog in Germany with higher speeds than my previous Australian hosting (which was bottlenecked due to overuse of it’s capacity) and no bandwidth quotas that I am likely to hit in the near future. This also allows me to do new and bigger things, for example one of my future plans is to assemble a collection of Xen images of SE Linux installations - that will be a set of archives that are about 100MB in size. Even when using bittorrent transferring 100MB files from a server in Australia becomes unusable. Most Australians who access my blog and have reasonably fast net connections (cable or ADSL2+) will notice a performance improvement. Australians who use modems might notice a performance drop due to longer latencies of connections to Germany (an increase of about 350ms in ping times). But if I could have had a fast cheap server in Australia then all Australians would have benefited. People who access my blog and my web site from Europe (and to a slightly lesser extent from the US) should notice a massive performance increase, particularly when I start hosting big files. It seems to me that the disadvantages of hosting in Australia due to bandwidth costs are hurting the country in many ways. For example I run servers in the US (both physical and Xen DomUs) for clients. My clients pay the US companies for managing the servers, these companies employ skilled staff in the US (who pay US income tax). It seems that the career opportunities for system administrators in the US and Europe are better than for Australia - which is why so many Australians choose to work in the US and Europe. Not only does this cost the country the tax money that they might pay if employed here, but it also costs the training of other people. It is impossible to estimate the cost of having some of the most skilled and dedicated people (the ones who desire the career opportunities that they can’t get at home) working in another country, contributing to users’ groups and professional societies, and sharing their skills with citizens of the country where they work. Companies based in Europe and the US have an advantage in that they can pay for hosting in their own currency and not be subject to currency variations. People who run Australian based companies that rent servers in the US get anxious whenever the US dollar goes up in value. To quickly investigate the hosting options chosen for various blogs I used the command “traceroute -T -p80” to do SYN traces to port 80 for some of the blogs syndicated on Planet Linux Australia [3]. Of the blogs I checked there were 13 hosted in Australia, 11 hosted independently in the US, and 5 hosted with major US based blog hosting services (Wordpress.com, Blogspot, and LiveJournal). While this is a small fraction of the blogs syndicated on that Planet, and blog hosting is also a small fraction of the overall Internet traffic, I think it does give an indication of what choices people are making in terms of hosting. Currently the Australian government is planning to censor the Internet with the aim of stopping child porn. Their general plan is to spend huge amounts of money filtering HTTP traffic in the hope that pedophiles don’t realise that they can use encrypted email, HTTPS, or even a VPN to transfer files without them getting blocked. If someone wanted to bring serious amounts of data to Australia, getting a tourist to bring back a few terabyte hard disks in their luggage would probably be the easiest and cheapest way to do it. Posting DVDs is also a viable option. Given that the Internet censorship plan is doomed to failure, it would be best if they could spend the money on something useful. Getting a better Internet infrastructure in the country would be one option to consider. The cost of Internet connection to other countries is determined by the cost of the international cables - which can not be upgraded quickly or cheaply. But even within Australia bandwidth is not as cheap as it could be. If the Telstra monopoly on the local loop was broken and the highest possible ADSL speeds were offered to everyone then it would be a good start towards improving Australia’s Internet access. Australia and NZ seem to have a unique position on the Internet in terms of being first-world countries that are a long way from the nearest net connections and which therefore have slow net access to the rest of the world. It seems that the development of Content Delivery Network [4] technology could potentially provide more benefits for Australia than for most countries. CDN enabling some common applications (such as Wordpress) would not require a huge investment but has the potential to decrease international data transfer while improving the performance for everyone. For example if I could have a Wordpress slave server in Australia which directed all writes to my server in Germany and have my DNS server return an IP address for the server which matches the region where the request came from then I could give better performance to the 7% of my blog readers who appear to reside in Australia while decreasing International data transfer by about 300MB per month.

• [1] http://www.internode.on.net/residential/internet/home_adsl/nakedextreme/pricing/


• [2] http://www.linode.com/


• [3] http://planet.linux.org.au/


• [4] http://en.wikipedia.org/wiki/Content_Delivery_Network

I played with an Eee for a bit last time I was in Boston, culminating in a patch to make the eeepc-laptop driver use standard interfaces rather than just having random files in /sys that people need to write custom scripts to use. The world became a better place.

However. Asus implemented the rfkill control on the Eee in a slightly odd way. Disabling the wifi actually causes the entire card to drop off the bus, similar to how Bluetooth is normally handled. The difference is that the Bluetooth dongles are almost exclusively USB, while the Eee's wifi is PCI. Linux supports hotplugging of PCI devices, but nothing seemed to work out of the box on the Eee. Another case of this was the SD reader in the Acer Aspire One. Unless a card was present in the slot during boot, it simply wouldn't appear on the PCI bus. It turned out that Acer have implemented things in such a way that removing the card results in the entire chip being unplugged. This was when I started looking more closely into how this functionality is implemented.

The two common cases of PCI hotplug are native PCIe hotplug and ACPI mediated hotplug. In the former case, the chipset generates an interrupt when a hotplug event occurs and the OS then rescans the bus. This is a mildly complicated operation, requiring enabling the slot, checking whether there's a card there, powering the card and all its functions up, waiting for the PCIe link to settle and then announcing the new PCI device to the rest of the OS. ACPI-mediated hotplugging puts more of the load on the firmware rather than the OS - the hotplug event generates a notify message that is caught by the ACPI interpreter in the OS, allowing the OS to check for device presence by calling another ACPI method. If the device is present it's then a simple matter of telling the PCI layer about it.

Native PCIe hotplug has the advantage that there's much less vendor code involved. ACPI is still involved to an extent - an _OSC method on the PCIe bridge is called to allow the OS to tell the firmware that it supports handling hotplug events. This allows the firmware to stop sending any ACPI notifications. ACPI hotplugging requires more support in the firmware, but can work for PCI as well as PCIe.

The general approach taken to getting the Eee's wifi hotplugging to work has been to load the pciehp driver with the pciehp_force=1 argument. This tells the driver to listen for hotplugging events even when there's no _OSC method to tell the firmware that the OS is handling things now. Since the hardware will generate the event anyway, things work. However, this is non-ideal. Some hardware exists where ACPI hotplugging will work, but due to quirks in the hardware design native PCIe hotplugging control will fail. This has been handled in their firmware by having the _OSC method fail, signalling to the pciehp driver that it shouldn't bind to the port. Using pciehp_force overrides that, leading to a situation where hardware could potentially be removed from a port that's powered up. Unfortunate.

My first approach was to add a new argument to pciehp called pciehp_passive. This would indicate to the pciehp driver that it should only listen for notifications from the hardware. User-triggered events would not be supported, avoiding the situation where anyone could remove the card by accident. This worked on my test machine (an Eee 901 somewhere in Ottawa, since I don't actually have one myself...) but was reported to work less well on a 700. Since the 700 didn't claim to have any support for power control, the code was forced to wait a second on every operation to see whether the link powered up or not. This resulted in long pauses during boot and suspend/resume operations.

The final issue that convinced me that this was the wrong approach was reading a document on Microsoft's site on how PCIe hotplugging is implemented in Windows. It turns out that XP doesn't support native PCIe hotplugging at all - that feature was added in Vista. Both the Eee and the Aspire One are available with XP, but things work there. So PCIe native hotplugging was clearly not the right answer. Time to look further.

Armed with a disassembly of the Aspire One's DSDT, I figured out why the ACPI hotplug driver didn't work on it. The first thing the driver does is walk the list of ACPI devices, looking for any that are removable. That was being implemented by looking for an _EJ0 method. _EJ0 indicates that the device can be ejected under the control of the OS. The Aspire One doesn't have an _EJ0 method on its SD readers. However, it did have an _RMV method. This can be used to indicate that a device is removable but not ejectable - that is, the device can be removed (by physically pulling it out or by the hardware taking it away itself), but there's no standard way to ask the OS to logically disconnect it. A quick patch to acpiphp later and the Aspire One now worked without any forcing or spec contravention. This also has the nice side effect of making expresscard hotplug work on a bunch of machines where it otherwise wouldn't.

But back to the Eee. acpiphp still wasn't binding, and a closer examination revealed why. There's nothing to indicate that the Eee's ports are hotpluggable, and there's no topological data in the ACPI tables that ties the wifi function to the PCIe root bridges. However, the Eee firmware was sending an ACPI notification on wifi hotplug. But it was only sending this to the PCIe root bridges, and there's no way to then tell which device had potentially appeared or vanished.

In the end, I gave up on trying to solve this generically. Instead I've got a patch that implements the hotplugging entirely in eeepc-laptop. In an ideal world nobody else will have implemented this in the same way as Asus and we can all be happy.

One month with Emacs and counting - Part 2 A while ago I've blogged about my switch from Vim to Emacs, promising a blog post series, quite a mouthful nevertheless, it's time to continue the series. The first part was about why I think that nowadays Emacs is ready to be switching to. This second part is about flawed Vim design choices which substantially contributed to my choice. (Of course all this post is under a IMO-/rant-disclaimer, it can't be otherwise, so take it cum grano salis) Flawed Vim design choices Vim started as a wonderful editor, following the path of Vi, which in turn has got right a good deal of design choices. One is the often debated modality. In spite of being a HCI PITA in general, modality is not a problem at all for software you are using every single day (and the editor is the 2nd program I use the most, with the terminal being the 1st), because you learn it as a habit. In the case of Vim, modality is good because it let you use powerful yet simple motions in command mode, which can be easily combined with operator-pending commands. This can be considered yet another incarnation of the UNIX philosophy, casted inside the editor itself. ... which brings us to the good old joke which describes Emacs as a great operating system, lacking only a decent editor. In fact, Vim is no longer the nice Unix philosophy player which it used to be, and Emacs plays much more nicely with external tools than Vim. As arguments, let's consider a few (not so) recent Vim evolutions:

• Spell checking. Starting with version 7, Vim has got support for on-the-fly spell-checking of text buffers. It is a super-nice feature, which I've advertised in the past, because spell checking is needed in many tasks we daily do with editors (mail/doc writing, code commenting), and having a single interface to do it comes handy. Unfortunately, how that is implemented in Vim is definitely not along the lines of the UNIX philosophy. We have countless different FOOspell implementations whereas Vim implemented yet another one, without relying on any pre-existing implementation. Even more so, the file format of spell files is basically ad-hoc, and not sharable with other tools. This is already a PITA for packaging (let's add to Debian another 20 binary packages vim-spellfiles-FOO), but is even worst than that: the spell files must be generated in ad-hoc ways which are seriously prone to failures (guess why in Debian we don't have yet the vim-spellfiles-FOO packages which used to exist in experimental a long time ago?). What would have been the UNIX-friendly way of achieving the same result? Well, obviously spawning FOOspell and interact with its process from the editor. Keep that in mind.
• Internal grep. A feature of Vim I've always loved is :grep (yet another instance of quick-fixing, which is also used for jumping to compile-time errors). In its original incarnation it was as simple as invoking grep and parse its output. KISS. Then, Vim started to have a problem: grep -r can take ages and in the meantime the editor was stuck. Hence, starting with Vim 7, vimgrep has born: an internal re-implementation of grep. OK, it is more portable (but how many of us do care? on how many system you use you are missing grep? damn, I used to install it also on Windoze!), and the editor retain control. Control that is exploited just to show a nice progress bar of what is being scanned ..., while the editor is still stuck. Well, to me it looks like the problem would have been solved much more nicely by adding the ability to interact with an external process (deja vu?). That process can then be grep -r. Full stop.
• Lack of debugging support (on purpose). Long standing missing feature of Vim: you can not "easily" drive a debugger from Vim, because a firm choice in Vim is to ... (OK, I confess, I'm starting to beat a dead horse) ... not have support for interaction with external processes. I found such a choice quite dumb: to me it is evident that it induces a trade-off between being feature-complete wrt programmer needs (as available external tools get available to face those needs) and not "re-implementing the wheel" inside the editor. Going back to the lack of debugging support in Vim, that has spawned projects like GdbVim, Clewn, Bram's Agide, and my own tiny teeny incarnation specific to ocamldebug called WOWcamldebug. Every single project of that list acts as an intermediary between Vim and an external process (a debugger of some sort), whereas that could have been implemented by an editor plugin, provided that the editor offer, guess what, support for interaction with external processes.
• Top-level. Similar to the debugging support issue, there is the interpreter issue, namely how to evaluate "phrases" of your interpreted language of choice directly from the editor. Nowadays a lot of languages have the so called top-levels which evaluate language phrases interactively (Python, Ruby, OCaml, many Lisps, ...) and show evaluation results. That ability can speed up testing considerably, but can become cumbersome to use for large code snippets without editor support (good luck with copy and paste). Yes, Vim has support for that, but the implementation choice is close to be ridicolous: only if you have linked Vim together the interpreted for a given language, then you will be able to interactive evaluate phrases of that language. What if you program in Python, Ruby, and Perl? Well, you need to link all of them. What if you also program in OCaml? Then you're screwed, because Vim currently does not support linking with it. (Yes, I know that there are other reasons for doing that linking, they are discussed below.) Do I need to tell you which feature will be enough to address top-level support? interaction with extern.. OK, I'll drop it, you know that already.

Well, I acknowledge that Emacs got this choice right, and that it's an important one: Emacs supports interaction with external processes, and plugin authors have been happy to exploit it to create really cool stuff (sticking to the UNIX philosophy). Please welcome: Flyspell, M-x grep (shameless smartass plug: same number of keystroke of :grep), Grand Unified Debugger, a sampling of language-specific major modes with top-level support. (But I recommend having a look at least once at the coolness of other stuff built-on top of external process support, like Flymake, mentioned by dancer not a long ago.)

• Its own scripting language. I've bored you enough with the external process topic, hence here is the second one: the saga of Vim and its scripting language, i.e., the programming language using which you can customize the editor. Note that the choice of scripting language impacts on 2 different targets: final users in need of fire and forget automations (when they start to get to complex to be implemented on top of, say, macros), and developer of editor extensions such as addons. The impact of scripting language on extension developer is crucial for the evolution of an editor aiming at being powerful. If the scripting language and the API towards the editor is flexible enough, then a lot of cool extensions and features will be developed by third party authors, otherwise the burden stays on editor upstream author.
  1. In its phase 1 (my term, Vim < 7.0) Vim pretended that it needed only a minimal scripting language (Vim script) and that everything else can be developed using external programming languages, linked in the editor. As a consequence Vim script was just a tiny teeny procedural language with editor interfacing capabilities, most notably lacking any "decent" data structures (no lists, no dictionaries, ...). In that phase, if you were a user it was fair enough. You just had to chose your external programming language (at compile time ...), and you were able to write your quick hacks with it. But if you were willing to develop an extension you were screwed, because you had either to impose your external programming language to the final user (that is what gave born to absurdities like vim-full, look at the deps!) or to enjoy a good deal of masochism to explode your 30 lines of Python extension into 200 lines of Vim script to make your Vim extension more "portable".
  2. In its phase 2 (Vim >= 7.0) it has been realized that something were wrong in the initial choices about Vim script, and data structures (lists, dictionaries, function references!) have been added to it, relieving a bit the pain of programming portable extensions. Nevertheless, and probably due to how it started, Vim script is far from being a programming language one can enjoy programming in. Consider for example the standard library of the language. It started as a random collection of unrelated functions being added on a "as-needed" basis, ... and continued along that path! Now the API reference lists together totally unrelated functions, organized inconsistently, and making really painful finding what you need (assuming it exists).

Even in this respect, I acknowledge that Emacs got this right. It has chosen a single, now full-fledged, programming language (Elisp), which is offered both to the final user (fact: the average Emacs user knows much better how to program her own editor than the average Vim user) and to extensions developer. The standard library of the language is organized with a consistent naming (even though not always intuitive) and documented in an organized manner. You might not like the language, but at least is has been taken seriously and managed as such. Finally, being a Lisp dialect, you might need what you learn with Elisp elsewhere. Vim script is too committing: it is useful only inside Vim. I believe that contributed significantly in its scarce diffusion among Vim users. The fact that Elisp was born together with Emacs and that it is also used internally for the editor implementation is not relevant here: I do care about what is offered to me as an user and as an extension developer. What I see on the Emacs side in this respect is much better that what I see on the Vim side. Moreover, from the point of view of the users, is the difference between what is inside the editor and what is outside (i.e., the extensions) that relevant? I think it is not, and that brings me to the final Vim choice which I consider flawed and which has the potential of seriously limit its future evolutions:

• Editor as a distribution. Emacs is managed as a intermediate software distribution layer, similarly to what happens in other softwares where many third party authors cooperates (e.g., TeXLive). The distribution architecture has advantages and contributes to the longevity and potentialities of a software project. What does this mean concretely in the Vim vs Emacs dispute? For example it means that Emacs get more feature-complete each passing release, still preserving uniformity in documentation and keybindings. On the contrary Vim sports many addons, which are very likely to step on each other feet when enabled simultaneously. Actually, that is one of the reason which brought me to the creation of vim-addons: you cannot enable all the content of vim-scripts together due to various kind of conflicts. As we know well in Debian, distributions have a fundamental role in blending together lightly-coupled software components, that's role is missing in Vim evolutionary path and I find that worrisome.

Where to from here in this blog post series? For sure some tips on how to migrate for hard-core Vim users, then we'll see ... If you did enjoy the read please let me know commenting in the discussion page (as you did in the last post, thanks!).

Hey, Xfce 4.6 Beta 2 Hopper has just been released and, guess what, packages are already available. Still not in unstable (or even experimental), that will wait post-lenny. You can install it from my repository, and, as usual, there's no Debian support on them. The known_issue page is currently quite empty, and this beta looks really quite polished. Packages currently available are i386 and amd64, powerpc will be available when I have time to put the mac at build. This beta features mostly bug fixing, and some features too:

• power management stuff in xfce4-session. Based on a patch included long ago in Ubuntu, it means that when you click the logout button it'll ask you if you want to suspend/hibernate the box. That works through hal and it may need some tuning on the machine. Reports appreciated here or there. 
• soname bump in xfconf. That means package renaming from libxconf-0-0 to libxfconf-0-2. No standard upgrade path (Replaces:) is provided because I don't want to clutter the packages before upload to Debian. This will continue for following changes (libxfconf-0-3 for rc for example). For the final release the library name will be changed to libxfconf-1 and the soname will be reset to 0, which means the package name will be libxfconf-1-0. There again, no upgrade path will be provided for people using my repository. The only upgraded path supported will be 4.4 →4.6. (but you won't be totally alone in the dark, I use 4.6 beta on my laptop so I'll pass through hell before you)
• keyboard shortcut can be configured (both window manager ones and applications) and there is now conflict handling
• session management has been greatly improved, you can now save the session immediately (a long-wanted feature) and it manages stuff like SMRestartImmediately etc.

The official announcement can be found there.
Have fnu!

Raphael cheers for the addition of an --as-needed option to a linker command line. While I think the general idea of not linking libraries you don't use directly is a good one, --as-needed is not a proper solution since it tells the linker to second-guess the compiler. A proper solution would be to fix the build systems so they don't pass the names of unneeded libraries in the first place. There are valid reasons for linking against objects that provide no symbols to the object being built, for example if the plugin ABI of a program has these symbols exported to plugins (in order to have all plugins agree on a common set of allocation functions).

I've had my share of old bugs, but #234448 holds the distinct record of being the oldest bug that I ever found a fix for. It's so old, I don't even know how old exactly. Don't be betrayed by the number 234448. While the Debian bug was filed in 2004, the bug itself is older than that. Take a look at this FreeBSD commit, for example; the original import of the 4.4BSDLite code of the 'trek' program already has the bug. And that was over 14 years ago. The first NetBSD commit is even older: 15 years, 7 months ago. BSD itself, of course, is much older than that, and so is trek. The copyright header contains 'Copyright(c) 1980, 1993', which suggests that this bug might be as much as 28 years old. That's almost as old as I am... Like I said, a record.

I’ve been a member of April, the french association for promotion and defense of Free Software, for a bit more than a year, and I often regret not becoming a member earlier. (I was feeling so guilty and shameful about not being a member that I actually postponed becoming a member.) Stop feeling guilty and shameful, become an April member today! Why Is becoming an April member so important?

• Clearly, April doesn’t address the same problems as your local LUG. April is a country-wide organization, and it works on country-wide problems. It’s the only group able to work on such problems at this scale (I’m not sure of the situation in other countries, but I think CCC shares a similar role in Germany for example).
• Each time I talk to people really involved in April (which I’m not), I’m amazed by how powerful they have become. They are able to talk to french or european deputies or ministers’ cabinets, and are considered important. They are doing a fantastic job spreading what matters to us to legislative and executive powers in France and Europe.

Some of the things they worked on recently (from the top of my head):

• Lobbying on :
  • OOXML
  • General announcements about politics (Plan France Num rique 2012, aka Plan Besson).
  • European telecom package and HADOPI law (french graduated response) law, through Quadrature du net. (OK, it doesn’t have anything to do with April, but most of the people involved in Quadrature du Net are also involved in April :-)
  • vente li e : the fact that it’s not possible to buy a computer without a Windows license. It’s illegal in French law, but still the de facto situation almost everywhere.
• Organization of a campaign where candidates to elections in France where asked questions, or asked to sign a declaration about Free Software. In 2007, 8 out of the 12 candidates of the french presidential election answered April’s questions.

So, really, become a member today. It’s only 10 EUR, and you already know they will be well used. April is trying to reach 5000 members by the end of 2008. (Apparently, if you use that address, April will now that you came from me. No benefit for me at all.)

when not being distracted by other uses of the system, i've managed to spend some more time on packaging some more stuff that's useful on a debian ps3. namely:

• ps3-utils: necessary utilities (boot-game-os, ps3-video-mode, etc)
• speutils: threading and message passing library
• spu-medialib: spu-accellerated audio/video library
• xserver-xorg-video-spu: spu-accellerated framebuffer driver for xorg

the last of which i think will be of the most interest to some folks, as it provides an spu-based xvideo extension, which brings back the possibility of playing HD videos etc on the ps3. now if you're interested in checking out these packages, read on, starting with this: !=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!= ZOMG WARNING!!!1!!ONE!! THESE ARE EXPERIMENTAL BLEEDING EDGE PACKAGES. I MAKE NO GUARANTEE WHETHER THESE PACKAGES ARE PROPERLY BUILT, WILL INSTALL, WILL WORK, WILL NOT DESTROY YOUR SYSTEM, WILL NOT CRAWL OUT OF YOUR SYSTEM AND MAKE LONG DISTANCE PHONE CALLS WHILE EATING THAT SNACK YOU WERE SAVING IN THE FRIDGE, ETC. !=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!= okay, so now that we've disposed with that formality, i'm glad if you're interested and still here so... where were we? oh yes. put this in your sources.list:

deb http://people.debian.org/~seanius/ps3-experimental-packages sid main

source packages are also available if you want to poke around or backport the packages to etch or ubuntu or something else wierd like that:

deb-src http://people.debian.org/~seanius/ps3-experimental-packages sid main

now, if you have a properly configured system, you should get a warning:

Reading package lists... Done
W: GPG error: http://people.debian.org sid Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY CA78CB3E6E76D81D
W: You may want to run apt-get update to correct these problems

but i promise this key is valid, and mine. but hey, you don't have to take my word for it

sudo apt-get install debian-keyring
gpg --no-default-keyring --keyring /usr/share/keyrings/debian-keyring.gpg --export --armor CA78CB3E6E76D81D   sudo apt-key add -

then you can install the packages... assuming you want the xorg video driver:

sudo apt-get install xserver-xorg-video-spu

and update the appropriate Device entry in your xorg.conf, so that the normal fbdev is commented out and instead you use the spufbdev driver. example:

Section "Device"
        Identifier      "Generic Video Card"
        Driver          "spufbdev"
        Option "ShadowFB" "false"
EndSection

and then

sudo /etc/init.d/gdm restart

(or the equivalent if you're using something else). that's it! more packages may trickle in here as well, at least until and they find their way into debian proper. i'm also in contact with the upstream authors of this stuff, and they're generally very responsive, so let me know if you have any problems.

I wrote to a site owner last week and I thought I was writing to a webmaster. The site owner complained about some of the jargon and, while explaining who I thought I was writing for, I explained some of it because I think more website owners might benefit from these three explanations:- “Expat-like terms” - made available in a way that is freely sharable, modifiable and redistributable, similar to the Expat software package, whose terms are published at http://www.jclark.com/xml/copying.txt - this is often used as a clear, simple example for encouraging wide distribution of electronic resources (software). “clandestine Google Analytics” - Google Analytics is a service from Google, Inc for tracking users through a website in various ways. I believe the Data Protection Act means that English websites should obtain informed consent from users by publishing a Privacy Policy on their site which discloses what the GA service will be used for and linking through to GA’s own Privacy Policy. Some websites attempt to run Google Analytics on users’ computers without explaining why and without any Privacy Policy. That is what I mean by “clandestine”. “valid xhtml” - validating against the eXtensible HyperText Markup Language standards published by the World Wide Web Consortium (W3C) - the underlying language of the web. There is a test service provided at http://validator.w3.org/ and passing it is a key stepping stone towards making an accessible website. There’s not really such a thing as “invalid xhtml” - if it doesn’t pass validation, it’s not xhtml. So I guess I’m guilty of using a tautology sometimes - sorry about that. Is it worthwhile knowing those three phrases? Are there other key technical phrases which you think site owners should know?

I got this nice message (and some others) in my log today:
smartd: Device: /dev/sdd, 2 Currently unreadable (pending) sectors After running a smartctl -t long /dev/sdd smartctl now reports:

ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  9 Power_On_Hours          0x0032   089   089   000    Old_age   Always       -       8719
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0012   200   200   000    Old_age   Always       -       2
198 Offline_Uncorrectable   0x0010   200   200   000    Old_age   Offline      -       0
[...]
SMART Error Log Version: 1
ATA Error Count: 10 (device log contains only the most recent five errors)
[...]
Error 10 occurred at disk power-on lifetime: 8692 hours (362 days + 4 hours)
  When the command that caused the error occurred, the device was active or idle.
  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 08 78 e5 89 e0  Error: UNC 8 sectors at LBA = 0x0089e578 = 9037176
  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  25 00 08 78 e5 89 2f 00  45d+06:36:27.880  READ DMA EXT
  25 00 08 70 e5 89 2f 00  45d+06:36:27.798  READ DMA EXT
  25 00 08 68 e5 89 2f 00  45d+06:36:27.308  READ DMA EXT
  25 00 08 e0 77 e5 35 00  45d+06:36:27.295  READ DMA EXT
  25 00 08 60 e5 89 2f 00  45d+06:36:26.653  READ DMA EXT
[...]
Error 9 occurred at disk power-on lifetime: 8692 hours (362 days + 4 hours)
  When the command that caused the error occurred, the device was active or idle.
  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 08 60 e5 89 e0  Error: UNC 8 sectors at LBA = 0x0089e560 = 9037152
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed: read failure       90%      8715         797566305
# 2  Extended offline    Completed without error       00%      8198         -

Notice that 197 Current_Pending_Sector indicates that there are 2 sectors with a problem. Note that the SMART error log shows LBA sectors 9037176 (0x89e578) and 9037152 (0x89e560) but it's limited to 0xffffff. The self test logs shows us the proper LBA error at 797566305 (0x2f89e561) which has a 0x2f in front, and also shows us that it's the second sector in the block of 8 we tried to read. The kernel log also shows:

ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
ata4.00: (BMDMA2 stat 0xc0009)
ata4.00: cmd 25/00:08:60:e5:89/00:00:2f:00:00/e0 tag 0 cdb 0x0 data 4096 in
         res 51/40:00:61:e5:89/00:00:2f:00:00/f0 Emask 0x9 (media error)

So I've tried to reproduce the error with:

# dd if=/dev/sdd of=797566305 skip=797566305 bs=512 count=1 iflag=direct
1+0 records in
1+0 records out
512 bytes (512 B) copied, 1.27727 s, 0.4 kB/s

Notice that it can actually read that block, it just seems to take a while. And the error log now contains:

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 01 61 e5 89 e0  Error: UNC 1 sectors at LBA = 0x0089e561 = 9037153

Because it's part of a RAID device I could just let the whole disk resync, but I was a little curious which files were in it. So I found this document that explains on how to find which files has the problem. But it only has examples covering ext2/ext3 on a partition and LVM and nothing about RAID device. So it looks like someone using RAID 0 will have a hard time finding the file that's having a problem. So following the document we start with:

# fdisk -lu /dev/sdd
Disk /dev/sdd: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors
Units = sectors of 1 * 512 = 512 bytes
Disk identifier: 0x000578f9
   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1   *          63      385559      192748+  fd  Linux raid autodetect
/dev/sdd2       972864270   976768064     1951897+  fd  Linux raid autodetect
/dev/sdd3          385560   972864269   486239355   fd  Linux raid autodetect
Partition table entries are not in disk order

In /proc/mdstat we see:

md1 : active raid5 sda3[0] sdd3[3] sdc3[2] sdb3[1]
      1458717696 blocks level 5, 64k chunk, algorithm 2 [4/4] [UUUU]

So sdd3 is part of md1 and I have an ext3 filesytsem on that. Tune2fs -l grep Block gives:

Block count:              364679424
Block size:               4096
Blocks per group:         32768

mdadm tells me:

# mdadm -QE /dev/sdd3
/dev/sdd3:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 747b071a:a70f723b:c3a9ed44:15845cbf
  Creation Time : Mon Nov 12 20:13:51 2007
     Raid Level : raid5
  Used Dev Size : 486239232 (463.71 GiB 497.91 GB)
     Array Size : 1458717696 (1391.14 GiB 1493.73 GB)
   Raid Devices : 4
  Total Devices : 4
Preferred Minor : 1
    Update Time : Mon Nov 10 10:57:28 2008
          State : clean
 Active Devices : 4
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 0
       Checksum : 4b3147f4 - correct
         Events : 78
         Layout : left-symmetric
     Chunk Size : 64K
      Number   Major   Minor   RaidDevice State
this     3       8       51        3      active sync   /dev/sdd3
   0     0       8        3        0      active sync   /dev/sda3
   1     1       8       19        1      active sync   /dev/sdb3
   2     2       8       35        2      active sync   /dev/sdc3
   3     3       8       51        3      active sync   /dev/sdd3

I've been told that 0.90 stores the superblock at the end of the device, so that makes it a little easier. And this seems to agree:

# cat /sys/block/md1/md/dev-sdd3/offset
0

We have a left-symmetric layout, so in raid5_compute_sector() we see

	stripe = chunk_number / data_disks;
	*dd_idx = chunk_number % data_disks;
[...]
	case ALGORITHM_LEFT_SYMMETRIC:
		*pd_idx = data_disks - stripe % raid_disks;
		*dd_idx = (*pd_idx + 1 + *dd_idx) % raid_disks;

So it looks like the chunks are located on the disk like:

stripe	sda	sdb	sdc	sdd
0	D 0	D 1	D 2	P 0-2
1	D 4	D 5	P 3-5	D 3
2	D 8	P 6-8	D 6	D 7
3	P 9-11	D 9	D 10	D 11
4	D 12	D 13	D 14	P 12-14

With each of those 64K. The problem is at 797566305 and the partition started at 385560. So that's at sector 797180745 inside partition sdd3. Each sector is 512 bytes and a chunk is 65536 bytes which gets us 128 sectors per chunk. If we want to know which the stride the sector is on we do: 797180745/128=6227974.5703125 This means that the stripe is number 6227974 and that it's sector 0.5703125*128=73 on sdd3 within that chunk. The parity is on disk: pd_idx = 3 - 6227974 % 4 = 1, being sdb. To get the chunk we need to multiply with the number of data disks (3) and since it's the forth disk and the parity is on the second disk we need to add 1 and get: 6227974*3+1=18683923 So calculating that back to sectors we multiply it with the number of sectors per chunk again and add our offset in the chunk: 18683923*128+73=2391542217 To test that our math is any good we try:

# dd if=/dev/md1 of=797566305.2 skip=2391542217 bs=512 count=1 iflag=direct
1+0 records in
1+0 records out
512 bytes (512 B) copied, 5.99103 s, 0.1 kB/s

We see an error message in the kernel log, there are 2 new errors in the error log, so it seems that everything is calculated correctly. The files are also identical. The file system is in blocks of 4096 bytes, so for the file system this is block 2391542217*512/4096=298942777.125 Then we use debugfs:

# debugfs
debugfs 1.41.2 (02-Oct-2008)
debugfs:  open /dev/md1
debugfs:  icheck 298942777
Block   Inode number
298942777       <block not found>

Which isn't making any sense to me, since I get that error about every 2 hour when a certain cronjob runs. 298942777 / 32768 (Blocks per group) = 9123.00955. So I wonder if ext3 is keeping some meta data there and debugfs doesn't tell that? Anyway, at some point I was running a smartctl -t long /dev/sdd again because I couldn't reproduce the error anymore, the cronjob started, and I then found alot of errors in my log again and now it included:

raid5:md1: read error corrected (8 sectors at 797180744 on sdd3)

It's so much fun that disks always behaves the same trying to read the same sector. Smartctl now reports:

  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0012   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   200   200   000    Old_age   Offline      -       0

I assume the kernel removed the pending sector by writing to it. The other pending error was already gone for some time, I assume new data got written to that one.

Just FYI, I've recently updated the OpenOCD Debian package in unstable. OpenOCD is a Free Software JTAG utility which currently supports quite a large number of JTAG adapters and various CPUs/targets (many ARM and now also some MIPS ones). It's being used by a number of Free Software related projects such as OpenMoko and many others. Here's an example of how you usually use the (new) OpenOCD with a cheapo parallel port JTAG device. First, start the OpenOCD server, providing it an interface config file and a target config file (you can copy/adapt them from /usr/lib/openocd/ interface,target /*.cfg, or use those files directly if they work for your target, of course).

  $ openocd -f parport.cfg -f lpc2148.cfg

Then, in another xterm for example, connect to the now-running OpenOCD telnet server. Here you can now run various commands to probe, control and program the JTAG device(s). Try help for a list of commands. As an example, for flashing a binary onto some LPC2148 eval board you would do something like this:

  $ telnet localhost 4444
  Trying 127.0.0.1...
  Connected to localhost.
  Escape character is '^]'.
  Open On-Chip Debugger
  > reset init
  JTAG device found: 0x4f1f0f0f (Manufacturer: 0x787, Part: 0xf1f0, Version: 0x4)
  srst pulls trst - can not reset into halted mode. Issuing halt after reset.
  target state: halted
  target halted in Thumb state due to debug-request, current mode: Supervisor
  cpsr: 0x800000f3 pc: 0x7fffd2a2
  requesting target halt and executing a soft reset
  target state: halted
  target halted in ARM state due to debug-request, current mode: Supervisor
  cpsr: 0x800000d3 pc: 0x00000000
  > flash write_image /home/foo/program.bin 0
  wrote 1236 byte from file /home/foo/program.bin in 0.533683s (2.261701 kb/s)
  > resume 0

The final resume 0 will start to execute your program on the ARM LPC2148 microcontroller. Check out the openocd info page (info openocd on the command line) for lots more documentation.

I have been meaning to fix this up for a long time, but so far real work keeps getting in the way. The idea is that C-C t brings you to this week's time tracker buffer, and then you use (C-c C-x C-i/C-c C-x C-o) to start and stop timers. The only even slightly clever is stopping the timer and saving on quitting emacs, which I borrowed from the someone on the net. Oh, and one of the things I meant to fix was the dependence on mhc. Sorry. Here is the snippet from my .emacs

(require 'org-timetracker)
(setq   ott-file
  (expand-file-name 
    (let ((now (mhc-date-now)))
      (format "~/.org/%04d/%02d.org" 
         (mhc-date-yy now) (mhc-date-cw now)))))

It needs emacs restarted once a week, in order to pick up the new file name. The main guts of the hack are here. The result might look like the this (works better in emacs org-mode. C-c C-x C-d for a summary)