In reply to a blog post by Russell Coker about air filters for server environments, I just confirm how important it is to take care of a dust free machine.
In most cases the users won't have air conditioning with filters, but are just using plain desktop machines that are collecting dust. In some cases one might argue that a computer with all of its fans is some sort of air filter by itself, but well... ;)

So, keep your machine and yourself happy and remove regularily the dust from the inside of your machines. For example, I just removed the dust from one of the Debian m68k buildds last week. It was running for about a whole year now without any problem or downtime, so removing the dust was necessary.

I removed the case, the expansion cards and other stuff to get most direct access to the machine. I bought some canned air (spray duster) and blew the dust out of the case. Take care to remove the dust from inside of the power supply as well (of course remove the power plug before opening the case anyway for your own security!). After dusting off with the spray cleaner, I'm always using an old tooth brush to clean the dust from the fans itself. All in all, it's some dirty work and maybe you should do it outside of your house whenever possible. ;)

But in the long run, cleaning your machines once or twice a year prolongs the life of your machines. Some of my machines are now running since 1991 in 24/7 mode. Alas, maybe the reason for some people not cleaning their machines is to raise a nice dust puppy. :-)


P.S.: If you can afford it, choose a compressor instead of compressed air in cans.

Serious server rooms have large (and expensive) air-conditioning and filtering systems. Most “server rooms” however are not like that, often it’s just some space in a store-room, sometimes near printers (which are a source of air pollution [1]). The servers that are stored in serious server rooms have air filters as a standard feature. For small server installations it’s often a desktop PC used as a server which has no filters (and often lacking other server features such as ECC RAM). Recently Dell in Australia started selling low-end PowerEdge servers for $450 plus delivery (similar machines to the $800 Dell servers I previously blogged about [2]). Also refurbished machines from HP and IBM can often be purchased at auction for similar prices. Even if you have a proper server with filters on all air inlets it’s still a benefit to have reasonably clean air in the server area. A few years ago I bought four Sunbeam HEPA [3] air filters for my home to alleviate allergy problems. I’ve had one running 24*7 in my main computer area for most of the time since then. As well as wanting to keep my machines free of dust I have the extra issue of machines that I buy at auction which often are filled with dust - the process of cleaning them frees some dust in the air and it’s good to have a filter running to remove it. Excessive dust in the air can prevent cooling fans from operating and cause damage to hardware and loss of data. Of course the really good servers have fan speed sensors that allow the CPU to be throttled or the machine to be halted in case of severe problems. But for desktop machines you often only have the temperature control mechanisms that are built in to the CPU and sometimes machines just start having memory errors when the fan malfunctions and the machine gets hot. As one of the biggest problems facing server rooms is heat dissipation I decided to measure my air filters and see how much electricity they use (it seems reasonably to assume that all their energy eventually gets converted to heat). I’ve now got a blog page about power use of items related to computers [4]. My air filters take 114W when on the highest speed and 13.9W when on the lowest. Initially I was a little surprised at the figure for the high speed, but then I recalled that the energy required to move air is proportional to the speed cubed. I’ll just have to make sure I don’t leave an air filter on overnight in summer… I’m now going to recommend such filters to some of my clients. Spending $400 on an air filter is nothing compared to the amount of money that a server failure costs (when you have expensive down-time and pay people like me to fix it).

• [1] http://www.computeractive.co.uk/computeractive/news/2195474/laser-printer-research-worrying


• [2] http://etbe.coker.com.au/2007/08/25/designing-computers-for-small-business/


• [3] http://en.wikipedia.org/wiki/Hepa


• [4] http://doc.coker.com.au/environment/computer-related-power-use/

Share This

I've used the Quod Libet music player since 2005. Mostly happily, despite the occasional really weird bug. The core of Quod Libet for me, the thing that no other player has, is the album list, just a big list of all available albums, with cover icons, and an easy way to limit the display to ones matching a search term. (I'm not one of those people who does complex, detailed playlists; I tend to play one album at a time, in order.) Quod Libet has lots of other nice features, including a great metadata editor, but the album list is the only thing I use daily. I mostly run Quod Libet on relatively slow machines (with decent speakers), with the display fed to my laptop. Lately it's been feeling a bit slow to scroll around in the album list with hundreds of albums and album icons being displayed over the network and everything running on an old laptop, or an overloaded server. I've known about mpd for a long while, and it's clearly a better fit for how I use servers to play music, but I could never stand to use any of the clients for long. Today I hit the triple cocktail of evil bugs in Quod Libet described in bug #413284. This is a potential last straw type bug, that's been open since March, and which I'd read before and hoped I'd be lucky enough to escape. This motivated me to try the Sonata mpd client again. It has a very clean looking and generally good interface, though like all mpd clients, it annoys me by being based around a playlist, and not making it easy to just select an album from the library and immediately start playing it. But there's a trick that makes it usable for me. I added all my music to the "playlist" (using the mpd command line) and ignore the hard to use library. Since Sontata does have an excellent type-to-limit-display feature, I can just type the name of an artist, album, or track, press enter, and it jumps to it and starts playing.

As a litigator, I’m always interested in novel and more efficient mechanism for resolving disputes, especially where the transaction costs of fighting it out in court can trump the benefits to either party. This process (full article not available online) for resolving property damage claims in China is about as streamlined as it gets:

Near the Lama Temple, as we waited to make the last left turn of the day, we were hit by another car. The driver backed into our side and then pulled away. There wasn’t time to fumble with my crutches, so I hopped out on my good leg. Fortunately, traffic was backed up, and I caught him in about seven hops. I pounded on the window. “You hit my car!” The driver looked up, surprised: a one-legged foreigner, hopping mad and smacking the glass. He stepped out and apologized, saying that he hadn’t felt the impact. Together, we inspected the Jetta–fresh dent above the left rear wheel. The man said, “I’ll give you a hundred.” That was about thirteen dollars. In China, after a minor accident people usually settle the matter on the street, in cash. This routine has become a standard part of life–once, I saw two small children playing a game in which they repeatedly rammed their bikes and shouted, “Pei qian! Pei qian!” — “Compensate! Compensate!” Leslie used her cell phone to call the rental company. Mr. Liu didn’t sound the least bit surprised to hear that we’d had another accident. All he said was “Ask for two hundred.” “That’s too much,” the other driver said. “This is really minor.” “It’s not our decision.” “Well, then, we’ll have to call the police,” he said, but it was clear that he didn’t want to do this. A dozen bystanders had gathered around the cars, which were parked in the middle of the snowy street. With Chinese accidents, the crowd is more like a jury than an audience, and a middle-aged woman bent over to inspect the dent. She stood up and announced, “A hundred is enough.” “What do you have to do with it?” Leslie snapped. “You can’t even drive!” That must have been correct, because the woman shut up. But the driver refused to pay two hundred. “Should we accept one-fifty?” Leslie asked me, in English. Lao-tzu said it best: A man standing on crutches in the snow will not bargain long over a dent to a crappy Jetta rental. Later that day, Leslie returned the car and the one-fifty in cash. Mr. Liu noticed that another light cover had been broken when she hit the brick wall. He said, happily, “What did you kill this time?” When I hit the dog, the same cover was twelve dollars; this time, he asked for only three. It must have been a special price because we did so well at the Lama Temple.

I don’t think this system would work to resolve patent infringement disputes, but it might make sense for some of our domestic fender-benders. Technorati Tags: ADR, China

Share This

I have brought some work with me to be able to catch up during the week-end. Usually, I take a laptop running Windows which allows to connect to our network through a VPN the credentials of which I am not in possession of. This time, I just want to make some analyzes on spreadsheets (created on MS Excel). I have tried to use OOoCalc for the first time in “work conditions” - ie. I expect a little bit more than the basic features, which was source of disappointment. Autofiltering feature does not allow wilcards in the conditions. Even advanced filters do not seem to allow them. I want to select columns which contents end with ‘00?’ (ie ‘00′ followed by any unique character), and I can’t. I have read some posts talking about regular expressions in filters in the old 1.x series, but can’t find anything related in the documentation of the current series - regular expressions seem to be used only for the ‘find & replace’ feature. Moreover, you cannot set more than 3 conditions in autofilter, which is not a lot (I am not sure that previous releases of MS Excel were better). I am fully aware of the advanced filters which do allow up to 8 conditions, but I must say they are not easy to use on a day to day basis. Filters are mostly considered as visibility filters, which means if you apply a filter and use the ‘Fill down’ feature, hidden cells are also filled in. At least it is not the case with copying & pasting: only visible cells are copied. These limitations (bugs?) as well as the old existing bug regarding the decimal separator make it impossible for me to use OOoCalc for my work. I guess I could easily achieve to do my work thanks to OOoBase, but the data I want to extract need to be used and amended by collaboarators who exclusively use MS Excel. Are there any advanced users able to work around these issues?

The current status of the NM process, with 1 NM awaiting FD approval, 7 awaiting DAM approval, and 30 waiting for their accounts to be created, leads people to thinking that the big bottleneck is the account creation stage. However, when you look at what happened since december 2005, it’s not that simple. See the graph linked above, which shows where people are waiting. While it seems that the FD stage hasn’t been blocking people for nearly a year, it’s the DAM stage which has been the biggest blocker. Indeed, on average, 8.3 NMs have been blocked by FD, 16.2 by DAM, and 9.0 by account creation. I understand that processing NM reports in batch mode makes it more efficient. However, I’m wondering if processing them 20 per 20, is really that more efficient than processing them 5 per 5, which would justify such long queues. The NEW queue, which used to be a problem, is now being processed on a very regular basis, and hasn’t grown widely recently, except after Debconf when the ftpmaster processing NEW was on VAC. Couldn’t we have the same thing for DAM, account creations, and while we are at it, removals from unstable? Having clearly defined human-crontab-jobs would certainly make working on Debian less frustrating.

I am making a small project with TurboGears and I am just loving the widgets and the identity framework. There are some rough spots, like the documentation, but luckily for the most pars you can just launch an ipython interpreter and use tab completion to look at what functions are available. But then I got this error:
Traceback (most recent call last):
File "./start-foo.py", line 23, in
from foo.controllers import Root
File “…Foo/foo/controllers.py”, line 22, in
action=”save_settings”
File “/var/lib/python-support/python2.5/turbogears/widgets/meta.py”, line 169, in widget_init
validator = generate_schema(self.validator, widgets)
File “/var/lib/python-support/python2.5/turbogears/widgets/meta.py”, line 277, in generate_schema
if widget.is_named:
AttributeError: type object ‘SettingsFields’ has no attribute ‘is_named’

And I could not find any help on this. Luckily I found Lucas manual and by comparing the code I found that in this part of the code:
settings_form = widgets.TableForm(
fields=SettingsFields,
action="save_settings"
)
Instead of passing the whole SettingsFields class, an instance of it needs to be passed ( as in “fields=SettingsFields(),” ). Cann’t wait to see all the goodies they are making now in TG 2.0, but considering my experience, I will wait until they update the documentation for it before diving in.

Last night I finally got around to having a play with Clutter. Considering I'd never used it before, I'm really impressed with just how easy the API is to use. Within half an hour I had images fading in and out, and this morning I built Eye of Gnome and rewrote my hack as a slideshow plugin. There is no point releasing the code just (patches required, and a nasty bug remaining), but I did make a screencast. NP: Maxinquaye, Tricky

Before the election was called the Howard government (being unethical in every way) started using public money to campaign. Part of this election campaign was two documents sent out to every home (AFAIK) coupled with a media campaign, one was about children and drugs, the other was about children and the Internet. I have to wonder how many people were fooled by this, including a picture of John Howard on the first page after the index made it pretty clear what the purpose was. Before I analyse the content of the documents one thing to keep in mind is the cost of the Iraq war, which the Sydney Morning Herald [1] estimates at $3,000,000,000. Note that this doesn’t include the life-long medical and psychiatric care that some veterans will require or the cost to the economy of such people not being productive tax-payers. It does note that 121 employees of the Australian military have been discharged on medical grounds. If each of those 121 people was disabled in a way that significantly impacted their life then the cost could be as much as $242,000,000. Let’s not assume that everyone who is currently serving and regarded as completely fit will be fine in 10 years time either. The Prime Minister gives a personal introduction to each document, the “NetAlert” document states that $189,000,000 is being spent to protect children on the Internet. The “Talking with your kids about drugs” document states that “more than $1.4 billion” has been spent to combat illicit drug use, I’m not sure what time period that covers - it seems to be his entire 10 years as PM so that would be $140,000,000 per annum. Obviously declaring a war of aggression against Saddam Hussein is something that he considers to be more important than protecting Australian children. Now the content of the NetAlert document. It starts with $84,800,000 to provide access to “the best available internet filtering technology“. I have to wonder, how much would it cost to develop some new Internet filtering software from scratch, or what the opportunity cost might be to re-assign resources from organisations such as CSIRO to the task. I expect that in either case the total would be a lot less than $84,800,000. Developing free software for the task would be a much better use of resources and would also allow cooperation with similar programs in other countries. In the unlikely event that $40,000,000 was required to develop such free software it probably wouldn’t be difficult to get a few other governments to kick in some money and make it $10,000,000 per country. Of course saving $70,000,000 doesn’t sound like much (a mere $3 per person in the country), but it is significantly more than the Free Trade Agreement with the US [2] was supposed to bring in (from memory it was promoted as improving the economic position of the country by $50,000,000 but the best available information is that it made our balance of trade worse). $43,500,000 in additional funding for the police to combat online child sex exploitation is small but OK (it would be better if some of that $84,800,000 had been saved on filtering software and allowed greater expenditure in this area). But it should be noted how child rape in Aboriginal communities is used as a political issue and nothing constructive is done about the problem [3]. I believe that protecting children against rape is far more important than controlling illegal porn (which is where the majority of that $43,500,000 will go). Another possible avenue of research would be on more secure computer systems for Australian homes. It’s all very well to install some filtering software, but if the PC running the software in question is hacked then it’s immediately turned off. Attacks that involve turning web cameras on without the knowledge or consent of the owner are not uncommon either, it’s not unreasonable for a child to get out of bed and then do some homework on their computer before getting dressed - with an insecure computer and a web-cam connected this could be a bad thing… If the Australian government was to mandate that all government data be stored in open formats and that all communication with government computer systems can be done with free software (as a matter of principle you should not have to buy expensive software to communicate with your government) then the incidence of insecure desktop OSs would decrease. The document does recommend some basic OS security mechanisms, but if you use an OS that is insecure by design it’s not going to work. The core message of NetAlert is that children should be supervised when using the net. There is one significant problem with this, full supervision of children requires that at least one parent is not working full-time. In a previous post I compared median house prices and median incomes [4] with the conclusion that even with both parents working it’s difficult to afford a house. It would be difficult for a single income that is below the median to be used to pay the rent on a decent house. So it seems that a significant portion (maybe the vast majority) of Australian families will be forced to do what the government would consider to be a compromise of the Internet safety of their children by having both parents working full-time jobs to pay off a large mortgage (or in some cases merely to pay rent). I believe that making houses more affordable would improve the protection of children and provide other significant benefits for society. If you look at chart 1 and chart 2 on the House Prices research note from the Australian Parliamentary Library [5] you can see a significant increase in the difference between CPI increases and house prices increases since 1996 (when John Howard became Prime Minister) and a significant increase in the ratio of house prices to average income. The charts only include data up to 2006 so don’t show the effects of the Howard government’s “workplace reform” which lowers wages for many people who are below the median income. John Howard is responsible for houses being less affordable and parents being forced to spend more time working and less time supervising their children, I find it offensive that he now tells people to spend more time supervising their children. The document does have some good basic advice about how to supervise children and what level of supervision is necessary. The problem is that children from an early age know more than the typical parents. I doubt the ability of typical Australian parents to implement some of the supervision suggestions on 12yo children. I have been considering writing my own documents on how to filter and monitor child Internet access but the main problem is determining ways of doing so that can be implemented by someone who is not a computer expert. In summary, the Howard government is responsible for reducing the supervision of children and therefore exacerbating the problems that they are claiming to solve. Their NetAlert program provides some information that can’t be implemented by the majority of Australians and entirely skips some of the most sensible steps such as using only secure OSs on machines that will be used on the Internet. I believe that Fedora and CentOS should be recommended by the government in this regard because of the combination of good security by default (of which SE Linux is only one component) and the ease of use. Computers are cheap enough that most families with good Internet connections (the ones at risk in this regard) have multiple computers so no extra cost would be incurred.

• [1] http://www.smh.com.au/news/national/3b-and-rising-rapidly-cost-of-iraq-war/2007/03/20/1174153066804.html


• [2] http://en.wikipedia.org/wiki/Australia-United_States_Free_Trade_Agreement


• [3] http://aboriginalrights.suite101.com/article.cfm/child_rape_dossier


• [4] http://etbe.coker.com.au/2007/08/06/housing-prices/


• [5] http://www.aph.gov.au/library/pubs/rn/2006-07/07rn07.htm

Share This

Been a while since I did so much Debian work in a fairly short time. Today, I finished work on belpic 2.6.0, the support software for the Belgian electronic ID card. My initial builds wouldn't work since I made some stupid (no, really, stupid) errors in properly using that ugly beast, but eventually I saw the light. Working packages are now in incoming, and will soon be on a Debian Mirror Near You(TM). Oh, in case you wonder: the difference between 2.6.0 upstream and 2.5.9 upstream is pretty small, mostly some housekeeping bugfixes; meaning, if 2.5.9 is working for you, there's probably no need to worry about getting the update. I've also added some extra changes to the packaging bits, so that you now no longer have to install the -dev packages just so you can use the browser plugins. Isn't that nice. I did some more work on getting emile, the Early Mac Image LoadEr, to work properly again. Unfortunately, after the initial 0.10-1 upload that I did to Debian, none of my other builds managed to actually work on my test machine. Since I don't want to break existing machines that are using emile, I didn't think to upload those non-working binaries; but now with #451415, I can't ignore that problem any longer. I think I've nailed it now, though, and once barok (my IIci) has finished building (which will take a few hours), the packages should work. Cross your fingers. Finally, I've also prepared an upstream bugfix update of nbd, the userland support tools for the Network Block Device in the Linux kernel, and have uploaded a package from that release into Debian. As if that wasn't enough, and since I genuinely care about the state of my packages in Debian derivatives, I've poked some Ubuntu-people to get it synced there, too. I guess I'm back in shape now.

My blog is syndicated on Planet Debian, Planet GSLUG, and Facebook (through their "notes" system). I occasionally get a comment about one of my posts over on Facebook, and every time I do I remember one of the reasons I avoid that site.

[someone] made a comment about your note "note title".

To see the comment, follow the link below:
[link to Facebook]

What's missing here is of course the actual text of the comment. Blogger does this right: the actual comment is emailed to me. Not only does this impose the annoyance of having to click on the link, but I often read my email offline, and so a message like this is worse than useless for me. (worse as I have to postpone it for later, which means I need to deal with it twice) It would be even better if I didn't have to click to reply, but with a Web service you kind of expect that. (one of the reasons that I'll be setting up my own blog at some point: I can use software that actually allows me to compose replies offline)

I don't know why Facebook does this, but my guess is that either (a) they haven't thought about it, or (b) they want to drive up page views and make it annoying for me to not use their site. If it's (b), then that apparently works for a large swathe of the population, but it makes me want to avoid them like a bad cliche.

The lender for one of my student loans has apparently started using a rather cool system to manage incoming customer service calls. If all the call-takers are busy, then instead of forcing you to sit there listening to a call telling you how important your business is to them, they let you request a call-back. They ask you to enter a phone number where they can reach you, then give you a time window in which they'll be calling (in my case I think it was about two minutes wide). When my turn came up, they called me and I was able to speak with a human immediately.

This is such a huge improvement over the typical call center experience that I'm amazed it hasn't caught on more broadly.

• It's presumably good for them since they don't have to pay for a long incoming toll-free call, particularly when most of the call is just muzak.
  
• It's great for me, because instead of sitting by the phone like a hawk in case I miss the customer service rep, I can go do something useful for a few minutes until the time the call is scheduled for.
  

I'm guessing that this option will be available in more and more places as companies computerize their telephone systems. Another nice twist, if there are any telephony guys reading this, would be to allow me to request the customer service call from a Web page, eliminating the toll-free call altogether and presumably saving some expense for the company.

An interesting question for me is how they generate those estimates of when they'll be able to help you. I'm guessing that they use some queueing theory to estimate the most likely wait time for the next incoming customer, but this is something I have only a fuzzy awareness of. However they did it, the estimate seemed accurate enough to me.

First step of the ultimate organization steps of the trip to India for FOSS.IN, I went today to the Embassy of India in Paris to leave my visa application. I left home at 06:20 to get one of the few trains running because of our strikes. Got two buses relatively easily from the station to the Embassy, then I arrived there 8:30 with already about 40 people before me in the queue. Discussion with fellow people in the queue...and I learned that there was no certain chance that I could actually *enter* the embassy to leave the form, my passport, my photos and the 50 bucks^W euros needed for a passport to India. I entered the embassy at....11:00 when their doors opened (we've been told that strikes in French transport are the reasons for the employees to be late.....I really wonder *how* *we* got there, indeed. Apparently, all people applying for visas live in Paris 16th. Then I got a nice ticket with a number on it (215 for the record) and began waiting in a small romm packed with dozens of people. First number called: 170. Three hours later, I finally got the pink receipt which gives me the right to worry about my passport getting lost and, hopefully, get it back on Monday with a visa for India on it. It's 14:45 while I'm writing this and I'm simply on my way back home after losing a full work day for a stupid administrative piece of paper. Have to be back on Monday 16:00 to get the visa back. Someone was saying in the queue that some days, you can wait up to 23:00 to get your passport back. It is terribly sad that this visa system still exists in some countries (not even blaming India for this: we (European Union) do exactly the same for citizens of India, forcing them to visit the French/spanish/German/whatever Schengen country in Dehli or Mumbai, to get a visa for EU. And I can really imagine what mess it can be as well. It was however interesting to see that all this crap was happening with laughs, good stories and friendly atmosphere with all the other French people waiting. Who says that French people are harsh and rude? Or is it already the influence of India on us? Anyway, Sam Hocevar told me yesterday he plans to come tomorrow for his application. Good luck, Sam and don't forget any of the requirements as well as a good coat, something to read (or your laptop to hack, with some spare batteries....and, no, no hackable WiFi around). Note that the 50 euros *have* to be cash money. So, India, here I am....nearly..:-)

One says the eyes are the mirror of the soul. I made this experience back in easter for the first time. I was over at my brother's place for easter celebration when I got up in the morning, went into the bath and looked into the mirror for morning toilet. I washed my face like always with cold water to refresh myself, and when I removed my hands... I was sure I was looking into a female face. It quite a lot bewildered me; it was the first time this happened. And I wasn't even properly shaved...
An experience like this is something special I guess, and it happened more and more often in the meantime. I guess this is one more proof that what I feel is the right thing. ... even though still some others seem to be immensly ammused by it. When I went to the ceilidh at the debconf in EDI I received some pretty nasty responses to my outfit, which I didn't expect within a project about Freedom and Openness. Though, I give the people the doubt of not knowing what they have done. It's too much in human nature to joke about things they don't understand, not knowingly insulting others. I'd like to dedicate this fine tune from Garbage to them: Bleed Like Me. If you listen closely to the lyrics you might be able to find out why... There has also been a genderfuck night in the club next to the night venue which on the other hand was pretty nice. It was attended by quite some people from Debian, some expectedly, some to my happy surprise. Thank you again guys, for making this evening to something special. I hope you keep it in as nice remembrance as me. My former SO drew a while ago a pretty nice picture about me. I didn't ask for it, or did hint it, which makes me even more happier about it. Thank you Babsi, really. :) I switched my hackergotchi on Planet Debian to it just in case. Today is the Transgender Day of Remembrance. I stumbled upon it in the Venus Envy comic I got notified about earlier this year, you might want to check its contribution, but beware, it might as disturb you as it does to me. I'm thankful that Erin dis survive it, because she gives so much strength with her comic to me and possibly also others...

As correctly pointed out by cmot [1] my previous post about software RAID [2] made no mention of bus bandwidth. I have measured the bus bottlenecks of a couple of desktop machines running IDE disks with my ZCAV [3] benchmark (part of the Bonnie++ suite). The results show that two typical desktop machines had significant bottlenecks when running two disks for contiguous read operations [4]. Here is one of the graphs which shows that when two disks were active (on different IDE cables) the aggregate throughput was just under 80MB/s on a P4 1.5GHz while the disks were capable of delivering up to 120MB/s:
On a system such as the above P4 using software RAID will give a performance hit when compared to a hardware RAID device which is capable of driving both disks at full speed. I did not benchmark the relative speeds of read and write operations (writing is often slightly slower), but if for the sake of discussion we assume that read and write give the same performance then software RAID would only give 2/3 the performance of a theoretical perfect hardware RAID-1 implementation for large contiguous writes. On a RAID-5 array the bandwidth for large contiguous writes is the data size multiplied by N/(N-1) (where N is the number of disks), and on a RAID-6 array it is N/(N-2). For the case of a four disk RAID-6 array that would give the same overhead as writing to a RAID-1 and for the case of a minimal RAID-5 array it would be 50% more writes. So from the perspective of “I need X bandwidth, can my hardware deliver it” if I needed 40MB/s of bandwidth for contiguous writes then a 3 disk RAID-5 might work but a RAID-1 definitely would hit a bottleneck. Given that large contiguous writes to a RAID-1 is a corner case and that minimal sized RAID-5 and RAID-6 arrays are rare in most cases there should not be a significant overhead. As the number of seeks increases the actual amount of data transferred gets quite small. A few years ago I was running some mail servers which had a very intense IO load, four U320 SCSI disks in a hardware RAID-5 array was a system bottleneck - yet the IO was only 600KB/s of reads and 3MB/s of writes. In that case seeks were the bottleneck and write-back caching (which is another problem area for Linux software RAID) was necessary for good performance. For the example of my P4 system, it is quite obvious that with a four disk software RAID array consisting of disks that are reasonably new (anything slightly newer than the machine) there would be some bottlenecks. Another problem with Linux software RAID is that traditionally it has had to check the consistency of the entire RAID array in the case of an unexpected power failure. Such checks are the best way to get all disks in a RAID array fully utilised (Linux software RAID does not support reading from all disks in a mirror and checking that they are consistent for regular reads), so of course the issue of a bus bottleneck becomes an issue. Of course the solution to these problems is to use a server for server tasks and then you will not run out of bus bandwidth so easily. In the days before PCI-X and PCIe there were people running Linux software RAID-0 across multiple 3ware hardware controllers to get better bandwidth. A good server will have multiple PCI buses so getting an aggregate throughput greater than PCI bus bandwidth is possible. Reports of 400MB/s transfer rates using two 64bit PCI buses (each limited to ~266MB/s) were not uncommon. Of course then you run into the same problem, but instead of being limited to the performance of IDE controllers on the motherboard in a desktop system (as in my test machine) you would be limited to the number of PCI buses and the speed of each bus. If you were to install enough disks to even come close to the performance limits of PCIe then I expect that you would find that the CPU utilisation for the XOR operations is something that you want to off-load. But then on such a system you would probably want the other benefits of hardware RAID (dynamic growth, having one RAID that has a number of LUNs exported to different machines, redundant RAID controllers in the same RAID box, etc). I think that probably 12 disks is about the practical limit of Linux software RAID due to these issues and the RAID check speed. But it should be noted that the vast majority of RAID installations have significantly less than 12 disks. One thing that cmot mentioned was a RAID controller that runs on the system bus and takes data from other devices on that bus. Does anyone know of such a device?

• [1] http://fortytwo.ch/blog/archives/2007/11/#e2007-11-16T10_54_22.txt


• [2] http://etbe.coker.com.au/2007/11/16/software-vs-hardware-raid/


• [3] http://www.coker.com.au/bonnie++/zcav/


• [4] http://www.coker.com.au/bonnie++/zcav/results.html

Share This

I've posted an an article on my website called Free Culture Advanced which I wrote for the last edition of the Free Software Foundation Members' Bulletin which went out several months ago. The bulletin is one of things you get when you become an associate member of the FSF. The article makes the case for free culture and a freedom definition in terms that are directed to and I hope will resonate with folks from the free software community. I've posted versions of the article in HTML, PDF, and LaTeX.

CHRISTOPHER MCKINLEY asked:

"I live on the costa blanca spain. I have been told that bbc and itv are to start broadcasting a free to air service which we will be adle to pick up on a small dish. It's scheduled to start in Feb/March 2008. Could you tell me if this is true?"

No, I can't tell whether that's true. The BBC/ITV Freesat service is expected to start in 2008 and I think March is meant to be when Channel 4's Sky encryption contract finishes, so I guess that's where that date comes from. That seems highly probable to me. Then again, Freesat has been advertised as launching in 2006 before. However, I doubt that BBC and ITV will move off of the Astra 2D spot beam that they moved to when they went free-to-air and back to the wider beams needed for small dish reception on the Costa Blanca. So, it will still not work for you. Sorry. (I'm not too cross about that, because the main terrestial Spanish channels seem to be encrypted on satellite.)

The Free Software Foundation sent out a press release today announcing a new addition to the FSF stable of licenses: the Affero General Public License or AGPL. The FSF has also published a set of answers to anticipated questions in the GPL FAQ. The first paragraph of the release explains what the AGPL is:

This is a new license; it is based on version 3 of the GNU General Public License (GNU GPLv3), but has an additional term to ensure that users who interact with the licensed software over a network can receive the source for that program. By publishing this license, the FSF aims to begin fostering user and development communities around free software web services and other network-oriented software.

The GPL is designed to ensure that users of software have access to the source code -- source is prerequisite to freedom and to the type of collaboration that has made free software successful. However, the GPL doesn't say "users" when it talks about who gets freedom; instead, it references people to whom the software is distributed. It doesn't say users for two reasons. The first is that, under copyright, "distribution" is a much more meaningful term and a powerful hook than "use" which is not, in most cases, one of the copyright holder's exclusive rights. The second is that, until very recently, having a copy of software was prerequisite to using it; possession was prerequisite to use. Things have changed. A large part of many people's computing experience involves running web applications. These include email clients (e.g., GMail or other webmails), office applications (e.g., Google Docs), social network systems, and others. These applications all run on servers -- i.e., on other people's computers. The providers of these services, the Googles and the FaceBooks, build upon, modify and improve GPL software without giving back to their users or the community that they took their software from. The AGPL was created several years ago by FSF board member Henri Poole as a way to address this issue. The license took the form of the GPLv2 with one extra clause. It was a first stab at a license and was imperfect. The language and methods were clunky and, most problematically, the license was incompatible with software under the GPL. The new AGPL is based on the GPLv3 and the extra clause has been rethought and rewritten. It has been vetted using the GPLv3 comment process and dozens of insightful comments from dozens of lawyers, hackers, and users of free software have been incorporated. The new license fixes the issues that many folks -- including myself -- had with the first version of the license. More importantly it can now be linked to GPLv3 code which makes the license a whole lot more practical. I am quoted in the release being excited about the license and I really am. I've got 2-3 major development projects (including Selectricity) which I've been waiting to distribute so that I could do so under the AGPLv3. The AGPL isn't a complete answer to the problem faced by disempowered users of web services. Without data or the capacity (in terms of servers, money, and expertise) to run web applications, the state and quality of these users' freedom remains far from clear. Thankfully, there are a whole bunch of folks thinking about what freedom for users of services might be -- it's a conversation that I'm going to push the FSF to participate in and pursue moving forward. The AGPLv3 marks a first solid contribution to the process of answering that question. If you'd like to help supporting or assisting the FSF in this effort, please consider becoming an associate member or donating.

There is an interesting web site WorseThanFailure.com (with the slogan “Curious Perversions in Information Technology”) that documents amusingly failed projects. The name used to be TheDailyWTF.com but changed due to the idea that for some projects success (interpreted to mean limping along in production) is worse than failure (being scrapped and re-written). I’ve created a new category WTF [1] on my blog to document such projects, both ones that I have personally witnessed and ones that friends and colleagues have seen. In the 90’s I spent some time working on an OS/2 GUI program to be the front-end for a mainframe backed system used by call-center workers. The first thing that they did was to develop a file naming scheme, they decided that all source files should be in the same directory (not unreasonable), but that 8.3 should be the limit of file name lengths in case there was a need to port the system to Windows 95. The fact that porting a program which did little other than display a GUI and talk to a S/390 server to a different OS was difficult (given the large amount of platform specific GUI code) was lost on them. Then they decided that the first two letters would be “FE” in case the source was merged with another bank project (of course the bank had many “Front End” systems talking to mainframes - so even this didn’t guarantee that they would be free of name clashes). Characters 3 and 4 were to represent the section of the program, and as the 3 character extension was one of “.h” or “.cpp” that left exactly 4 characters to name the file - we may as well have numbered the source files. Eventually one of the guys started making jokes about the file names by trying to pronounce them as words and that convinced a majority of the developers that the files should have long human-readable names. As the NMAKE expert (they insisted on not using unsupported software such as GNU make) it was my job to fix all the file names. The bank had an internal users’ group for the C++ development environment, but the contractors were not invited to attend. You might think that it would make sense to have the most skillful C++ programmers attend the meetings and share their knowledge, but apparently attending such meetings was a perk of being a permanent employee. What do you do when you have a C++ development project running behind schedule and a permanent employee who has had no prior exposure to C++ and wants to learn? One manager believed that assigning the junior programmer to the project is a solution to both problems - it was good for teaching the programmer C++ in a hurry but not so good for the deadline. There was a lot of excitement related to the back-end development. The mainframe guys assured me that CICS could never handle bad data and it totally wasn’t their fault if my program sent bad data to them and the region crashed. The first CICS region crash occurred when the mainframe guys told me to make the unused space in text fields “empty” - to a C programmer this means filling them with the 0 character but it killed the CICS region (apparently “empty” means filled with spaces to CICS programmers). A later region crash came when they told me to change the field length for the account name everywhere that it occurred - they told me that they wanted the change immediately - so I changed a macro definition, ran “make”, and sent a transaction with the changed size. Apparently “immediately” really meant “after we have spent an hour of software development changing magic numbers throughout our system”. I’m assuming that COBOL has some facility that roughly compares to C macros for defining field lengths and that it was the programmers not the language that was at fault. When the project became seriously behind schedule they hired some expert programmers from another country to help with the development. The experts decided that we needed to redesign the system (not that it had been designed in the first place) to use a “model view controller” architecture. When the real business logic is all on a S/390 and the only logic in the front end is to combine the contents of some screen-scrapes from a virtual 3270 terminal into a single GUI dialogue (with the first screen occasionally getting data from the IVR system) the benefits of MVC seem rather small. The new MVC version of the system was dubbed “phase 2″ and there were a series of deadlines for changing over from “phase 1″ which were missed. So the original team of developers continued on their project without any help from the “experts”. The last report I heard (some time after leaving the project) was that Phase 1 had gone live in production while Phase 2 was still being developed (at that time Phase 2 had a year of development and Phase 1 had about 15 months). The lesson to be learned from this is that management should only hire people that they can control. If you hire a consulting company and then let them do whatever they want (as opposed to solving the problem that they were hired to solve) then at best they will do no good and they may just slow things down. One of the more memorable incidents was when the project manager was talking to his 2IC, he said “I think we’re really breaking ground, [pause] I’m breaking wind and we’re breaking ground”. My desk was a few meters from where the wind (and ground) were being broken, I unfortunately was able to verify that wind was being broken but was not as certain about the ground. I did learn some useful things during this project, one was the fact that sample code from library vendors should not be trusted and ideally should be read from a print-out not pasted into the project source tree. The CTI library vendor had buffer overflows in their sample code. Another useful thing was the coin-tossing method of dispute resolution. If two good programmers have a disagreement about how to implement something tossing a coin is a quick way of resolving the dispute - this only works with two good programmers (in which case either idea will work). The project in question was messed up in many ways. I may write another post in the WTF category about it.

• [1] http://etbe.coker.com.au/category/wtf/

Share This

Three weeks ago I was scanning in all our old analog pictures from the APS cartridge. I've found a nice guy who rents out an APS capable scanner in .nl.

While I was scanning this in on Adele (my iMac) I thought: let's reconfigure Frost (the Debian build machine, backup server, iTunes server and fileserver), so I did a final backup to a FW external HD and inserted a Debian net-install CD. Which failed to install.

A bit confused I tried an older Debian release, which got slightly farther, but it still failed with a write error. So in the end I booted a recent Ubuntu installer, apt-getted the smart utilities and got a report on the disk. The disk claimed it was healthy. However it used 100% of its replacement sectors and gave read errors all over the place. Oh and a write error on sector several sectors.

Lessons learned:

1. Ask for smart support in the Debian installer
2. Install the smart daemon on the next server

So now I'm looking round for a nice Q6600 based server, using an intel motherboard with buildin graphics (maximum Linux compatibility) 4GB ram and 1TB of disk. This turns out to be a problem. None of the big vendors want to sell me a powerful, silent, computer with only Intel buildin graphics, they all insist on throwing in some nVidia or ATI incompatible POS :-(. And the smaller companies all want to sell me 'Gamer' PCs, with the same POS cards. So I have to find somebody who will build to spec but will still give me some warranty...

Not having Frost means that I am not able to do any Debian uploads :-(. Not that I'm not buzzy: I've got 3000 pictures to name, tag and date... (and in some case reimport because iPhoto things that they are raw pictures instead of the 48 bit tiff images that they are)