Search Results: "spa"

16 June 2008

Edd Dumbill: We're all ops people now

Ten years ago, most of us wouldn't have dreamt we'd be managing terabits of storage, tens of megabits of bandwidth, arrays of network-distributed services. The height of a programmer's worry would likely be choice of UI toolkit or finding the right way to indent code, and the height of consumer concern deciding which room to put the new computer in. Now the problems associated with managing large networks are becoming real for everyone, right down to the consumer level. Stupendously large amounts of computing resource are available at an instant. Your household probably has more than a terabyte of storage already. Issues such as single sign-on are going to hit home over the next year, as networked computing and entertainment devices profilerate. Features such as Apple's Time Machine will be increasingly vital — software that makes traditionally gnarly sysadmin tasks consumer-friendly. The rebranding of .Mac into "Mobile Me" is also a step in this direction. The impact on developers As software developers, we also have to cope with the effects of this resource-richness. For small sums of money we can get access to large computing clusters, geographically redundant hosting services. Our programs have left the desktop and found their new home on the web. System administration issues loom large upon us, security concerns lurk auspiciously in the corners of our minds. Although the cost of infrastructure has dropped radically, other costs remain high and are going to stay that way. System administrators are not only grumpy, they demand high wages. Commercial software license fees spiral out of control: traditional per-CPU licensing models make little sense when you can quickly bring up tens of machines. The cost in power is already troubling large companies, and there's no reason to suspect the problems won't ripple down. Help is at hand from a variety of technologies. If they don't yet make massive resource management trivial, they at least make it possible. Some of these also inhabit the weird territory of being both the source of a problem and a solution at the same time: virtualization, for example. Distributed revision control systems Distributed revision control is a technology whose time has finally come in popular circles, thanks in part to Linus Torvald's Git system. DRCS has several important impacts on today's developer: All these trends lower the barrier to entry, increase collaboration and agility of development. You can the value of this as more software tools become free. Selling such tools is rapidly becoming a thing of the past, the advantages of sharing enable the developers at the sharp end to get their jobs done quicker. However, such increased agility and, well, messiness leave other problems to solve, which the next two technologies address. Virtualization Hardware-as-a-service, infrastructure-as-a-service, call it what you will. The ability to create what we used to call entire machines, pick them up and move them around the network is revolutionary, and it's something that will have a real impact on regular developers. The benefits are at several levels. Configuration management Computing is a zero-sum game, and despite our increased ability to create and distribute software, problems still exist. We just pushed them to the next level. In good part, this next level is the problem of configuration management. We now have networks and clusters of (virtual) machines, software so agile we need six decimal places to describe its revision levels, and network and authentication paths that are starting to tangle. How do we manage that? One thing developers crave is repeatability. That's why we love our makefiles, autoconf, Ant, rake and so on. It's the one time even the most imperative-minded programmer writes declarative code. We like to say "let the world be like this." Our new sprawling world lacks this feature, and the best of our old toolkits — .debs, RPMs — address things only at the level of packages in a single environment. So developers must look to the world of operations, a territory we probably thought we needn't enter. In this world the new "make" is called Puppet. You write recipes to describe how things ought to be, and Puppet will make it so. I've been spending some time digging into Puppet, and feel excited by the confidence it's giving me. Now my applications exceed single source trees, and single machines, it gives me the means to tie the whole together. This article was going to be solely about Puppet, but that will have to wait now for another time. It's likely you'll have played with virtual machines and distributed revision control, but have you tried Puppet yet? Give it a spin, and let your mind wander over the benefits for your organization and development approaches. Conclusions
For developers and users alike, our world is changing. Hardware, connectivity and increasingly software is becoming cheap or free. The solidity of the old things we put value on — real things you can touch like disks — is eroding. What really matters is our data, our creations, and their communication. If they don't quite yet exist in a universal "cloud" yet, they're certainly getting frisky. As vendors provide solutions for consumers to manage their new domestic infrastructure, developers must look to network-aware toolkits and operations techniques to manage and get the best from their emergent infrastructures. Also on this topic: Join the conversation about this post

Martin-Éric Racine: updated ALSA driver for early PowerPC hardware

Risto Suominen contacted me recently, after seeing my Ubuntu wiki page where I try to document what works and what doesn't using ALSA on PowerPC hardware. He has developed a patch against the snd-powermac driver that mainly support 603, 604 and G3 PowerMac hardware, to implement better support for several old Mac models. He was wondering where he could find testers for his work, so I offered to blog about it here. You can find his diff and patching instructions on his website. You can contact Risto to report on your testing's result using firstname.lastname@gmail.com

Holger Levsen: 66h offline

This weekend I was offline for 66h, neither touching my computer nor the internet. That was a first this year. It was great! Amaya and me visited a small village in northern spain, which I knew from earlier travels and which is quite isolated from civilisation: no internet connection (3 or 4 valleys and 15km to the next DSL line), no water connection (but a fountain up the mountain), no power connection (but solar power and a generator for heavier machines), no road noise (you can hear the river down in the valley though, and lots of birds), a few happy people and some cats and dogs. Now I have two blisters on my hand and a light sunburn on my back (from digging out bad weeds from a wineyard), my bag hurts from stealing^wtaking stones plates from a church roof and I'm relaxed and happy even though my inbox and todo list is back to normal insanity :-) And I am quite sure it won't take another six years until my next visit there.

But now back to the weeds in my inbox, putting stones in NEW and working together with nice friendly people distributed all over the world.

15 June 2008

Joey Hess: no escape

So my Esc key broke today. Maybe due to high humidity last night -- the laptop was generating a lot of spurious lid open events earlier. New keyboard ordered, etc, nothing too unusual given the crummy nature of laptop keyboards (Fujitsu's in specific). The interesting thing though, is that even though I've remapped Caps Lock to generate Escape for now, this threw my typing off to an amazing extent. I don't just have trouble typing ^]:wq in vim. I have trouble typing arbitrary sentences in programs that don't use escape at all. Why? I don't touch type the way people are taught to in school, with fingers on the home keys. I touch type the way you learn to by typing a lot, ad-hoc, until your hands learn exactly how to type the common things you throw at them. This includes being able to type ^]:wq as fast as I can type "joey". My home keys are, approximatly: 
Tab Q S X Space       Down . ; ' Enter
(Space and Down are covered by my thumbs.) This might seem a bit weird, but as a programmer, I hit tab a lot, and, enter lots of semicolons, periods, and quotes. As a human on the WWW, I scroll down a lot. As a command-line user, Enter is a key I hit a lot. That's my guesses why my fingers go there. This wasn't planned, it evolved. In fact, I have other home key locations, that are used for other activities too. Now that I'm typing in a lot of text here, my WPM has increased a lot, because my hands have moved closer to the center of the keyboard. Something like this. 
      W E F V    J I O
Although every time I've checked, it's been a slightly different set of keys. My hands know where they are, and seem to make pretty good guesses about where I'll want them in the next word and pick a different "home row", depending. Also, if I'm scrolling around a lot, my left hand switches to: 
                      Left Up Right
Point being that this is all very dynamic, and unconcious, and very adapted to this exact keyboard. Learning a new keyboard will take me days of fumbling, that are reminiscent of someone describing switching from QWERTY to DVORAK. And, it turns out that even moving a single key like Esc is close to switching to an entire new keyboard. Needing to reach down to Caps Lock for Esc threw everything else off subtly, even when I wasn't typing that key. Clearly this is not the best way to type. It's just my way. :-/

Russell Coker: The Cost of Owning a Car

There has been a lot of talk recently about the cost of petrol, Colin Charles is one of the few people to consider the issue of wages in this discussion [1]. Unfortunately almost no-one seems to consider the overall cost of running a vehicle. While I can’t get the figures for Malaysia (I expect Colin will do that) I can get them for Australia. First I chose a car that’s cheap to buy, reasonably fuel efficient (small) and common (cheap parts from the wreckers) - the Toyota Corolla seemed like a good option.

A search on www.drive.com.au shows adverts for Corollas made in 1990 and 1991 for prices ranging from $3000 to $5000. Assuming some optimism on behalf of the advertisers I could probably buy one for about $2500. The current term deposit rate from the major Australian banks is just over 8%, so spending $2500 on a car (instead of having the money in a term deposit) means a loss of $200 per annum interest, if instead the car was purchased with a bank loan the rate would be significantly higher (car loans have quite high interest rates). Of course the real cost is whenever you sell a car and buy a newer one that costs more - but this is harder to estimate. The next issue is insurance, for a 1990 Corolla of the smallest engine size (as a rule of thumb the smaller the engine the smaller the insurance bill) comprehensive insurance (sans optional extras such as windscreen protection) would cost me $489 per annum, and third-party fire and theft would cost me $398. Of course those rates are for someone who is more than 25 years old and has a good driving record. The majority of people who “flip burgers” appear to be younger than 25 and would therefore pay more. Car registration starts at about $550 per annum (varies by state). Car service would cost at least $200 a year (for scheduled services). So it seems that the cost of a cheap car is around $1,350 per annum before you even drive it! Now with a car that’s 18 years old you have to expect that some parts are wearing out, including parts that make it unroadworthy and must be replaced. Assuming that $300 per annum is spent on parts and labor to repair parts that wear out seems quite conservative (especially if driving a car that costs less than $3,000 - one of the cheaper cars in that age range). That brings the total cost per year to $1,650 if you are lucky (if unlucky then expensive parts could wear out at financially inconvenient times). The amount of driving that I do is around 5,000 to 7,000km per annum (according to the ABS the average passenger vehicle was driven 14,100Km in 2005 [2]). Assuming 7.5L/100Km (one of the results that has been quoted for fuel efficiency of a car of that age) and 7,000Km of driving that would be 525L of petrol per year, at $1.70 that is $892. So at current fuel prices when driving a small car for minimal distances (either for environmental or financial reasons) the cost of petrol may be about 35% of the vehicle operating cost. If driven for the average distance the fuel cost will be about 52% of the total vehicle operating cost (which is about $3,400) - which seems like a reasonable incentive to save fuel. If someone wants to advocate measures to make cars more affordable for poor people then the first thing to consider would be a reduction in the vehicle registration fees (which are a significant part of the vehicle operating costs). Another would be government incentives for car sharing schemes (see my previous post on car sharing for details on what is on offer in Melbourne at the moment [5]), $3,400 would give 340 hours (or more with the weekend deals) of driving a shared car (including fuel) at current prices. A shared car would not be suitable for driving to work but would be good for most other car use. Also it should be noted that share cars are new and have all the latest safety features - I really wouldn’t want to be involved in a crash while driving a 20yo car. Unlike the costs of registration etc for owning a car, all the costs of a shared car will be reduced if you use it less. The tax rate for someone earning $28,579.20 is $2,850 plus 30c for each $1 over $25,000 [3], which is $3,923.76, which leaves a total after tax income of $24,655.44. Someone working 40 hours a week on the minimum income who drives an average distance will spend about 14% of their after-tax income on their car if they have a small old car and were able to buy it without borrowing money. According to the ABS the median income is actually slightly lower than full-time work at the minimum wage [4]. So it seems that there are many people who are working part-time and earning less money than they would if they worked full-time at the minimum wage. So the question for Colin is, how does this compare to Malaysia? Is someone on a minimal income in Malaysia going to spend ~7% of their after-tax income on paying insurance, tax, and basic maintenance on a car without the additional expense of driving it? How does the cost of the basic car ownership compare to the cost of fuel in Malaysia?

Share This

14 June 2008

Jan Wagner: postfwd 1.10pre7c-1 in experimental

postfwd 1.10pre7c-1 has been accepted and did hit experimental yesterday.
If you using policyd-weight you might want to give it a try, since its active development stoped. If you are annoyed by too much spam, it maybe a good idea as well.
Please have a look on postfwd.cf, there is no default config provided, since the configuration depends heavily on your environment. But maybe you have a look at /usr/share/doc/postfwd/examples, there is what you expect.
If you have any comments, send a mail to the maintainer or if there are bugs I didn’t spotted yet, please fill one!
In case I don’t respond quit fast, maybe I’m in the delivery room or anywhere else in the hospital or something like that, so please give me some extra time.
In case there are no big issues, I plan to upload the package to unstable in a reasonable time.

Enrico Zini: Saving waypoints with Holux M-241

Saving waypoints with Holux M-241 The Holux M-241 is a nice unit, but it looks like it cannot store waypoints while you're recording a track. In fact, if you set it to display latitude and longitude, every time you press enter it stores a track point with the current location. However, after downloading the data with gpsbabel, they are indistinguishable from all the other track points. If you are not taking a track, this is sufficient: your track will be made by all the waypoints you recorded. Together with Riccio, another M-241 user, we noticed that if you are taking a track, with one trackpoint per second, then when you press enter in the lat/lon screen you get two track points in the same second: one from the logger, and one from your keypress. The duplicate timestamp is just enough information to be able to distinguish a waypoint. Here is a little python script that will add a waypoint every time there are two trackpoints with the same timestamp. 
#!/usr/bin/python
import xml.dom.minidom
from xml.dom.minidom import Node
import sys
def get_text(node):
    res = ""
    for n in node.childNodes:
        if n.nodeType == Node.TEXT_NODE:
            res += n.data
    return res.strip()
def get_val(node, name):
    for n in node.childNodes:
        if n.nodeName == name:
            return get_text(n)
    return None
doc = xml.dom.minidom.parse(sys.argv[1])
# Scan the document looking for duplicate timestamps
wpt = []
last_ts = None
for node in doc.getElementsByTagName("trkpt"):
    ts = get_val(node, "time")
    if last_ts != None and last_ts == ts:
        wpt.append(node)
    last_ts = ts
# Add the nodes with duplicate timestamps as waypoints
if len(wpt) > 0:
    for i in wpt:
        node = doc.createElement("wpt")
        for a in i.attributes.values():
            node.setAttribute(a.name, a.value)
        for n in i.childNodes:
            node.appendChild(n.cloneNode(True))
        doc.documentElement.appendChild(node)
doc.writexml(sys.stdout)

Russell Coker: What is Appropriate Advertising?

Colin Charles writes about a woman who is selling advertising space on herself [1]. Like Colin I haven’t bought a t-shirt in about 9 years (apart from some Cafepress ones I designed myself). So it seems that the price for getting some significant advertising at a computer conference is to buy a few hundred t-shirts (they cost $7 each when buying one at a time from Cafepress, I assume that the price gets lower than $3 each when buying truck-loads). I have been given boxer-shorts and socks with company logos on them (which I never wore), I think that very few people will show their underwear to enough people to make boxer-shorts a useful advertising mechanism, socks would probably work well in Japan though. It seems to me that many people regard accepting free t-shirts as being an exception to all the usual conventions regarding advertising. Accepting gifts from companies that you do business with is generally regarded as a bad idea, except of course when t-shirts and other apparel are given out then it’s OK. Being paid to wear a placard advertising a product is regarded as degrading by many people, but accepting a free t-shirt (effectively being paid $7 for wearing advertising) is regarded as OK by almost everyone. I don’t mind being a walking advert for a company such as Google. I use many Google products a lot and I can be described as a satisfied customer. There are some companies that have given me shirts which I only wear in winter under a jumper. The Oracle Unbreakable Linux [2] shirt is one that I wear in winter. Now I would not consider accepting an offer to have advertising on my butt (although I’m pretty sure that it doesn’t get enough attention that anyone would make such an offer). I would however be happy to talk with someone who wants to pay me to wear a t-shirt with advertising when giving a lecture at a conference. I am not aware of any conference which has any real dress requirement for speakers (apart from the basic idea of not offending the audience). The standard practice is that if your employer pays you to give a lecture as part of their marketing operation then they give you a shirt to wear (polo more often than t-shirt). I am currently working on some things which could end up as papers for presentation at Linux conferences. If someone wanted to sponsor my work on one of those free software related projects and then get the recognition of having me wear their shirt while giving a lecture and have me listed as being sponsored by that company in the conference proceedings then that seems like a reasonable deal for everyone. One thing that you need to keep in mind when accepting or soliciting for advertising is the effect it has on your reputation. Being known as someone who wants advertising on their butt probably wouldn’t be fun for very long. On the Internet advertising seems to be almost everywhere. It seems that more than half the content on the net (by the number of pages or by the number of hits) either has an option to donate (as Wikipedia does and some blogs are starting to do), has Google advertising (or a similar type of adverts from another company), is a sales site (IE you can buy online), or is a marketing site (IE provides background information and PR to make you want to buy at some other time). Note that my definition of advertising is quite broad, for example the NSA web site [3] has a lot of content that I regard as advertising/marketing - with the apparent aim of encouraging skilled people to apply for jobs. Not that I’m complaining, I’ve visited the National Cryptologic Museum [4] several times and learned many interesting things! I think that Internet advertising that doesn’t intrude on the content (IE no pop-ups, page diversions, or overly large adverts) is fine. If the advertising money either entirely pays people to produce useful content or simply encourages them to do so (as in the case of all the blogs which earn $10 a month) then I’m happy with that. I have previously written about some of my experience advertising on my blog [5] and how I encourage others to do the same. I don’t think that space on a t-shirt is any more or less appropriate for advertising than space on a web site hosting someone’s blog. Finally there is one thing I disagree with in Colin’s post, that is the use of the word “whore“. It’s not uncommon to hear the term “whoring” used as a slang term for doing unreasonable or unworthy things to make money (where “unreasonable” and “unworthy” often merely means doing something that the speaker wouldn’t be prepared to do). But using the term when talking about a woman is quite likely to cause offense and is quite unlikely to do any good. The Wikipedia page about prostitution [6] has some interesting background information.

Share This

13 June 2008

Gunnar Wolf: Freedom itches

In this Free Software movement we have many mottos - One of which, describing what motivates us to work writing code, is scratch where it itches.
Of course, I could not keep it to myself - Almost a week ago, I took part of the World Naked Bike Ride. What I didn't tell you... Is that it became obvious I cannot reach most of by back - And it's because I'm mostly careless. When the WNBR started, it was still quite cloudy, even starting to rain... so I was mostly careless.
If you opened the newspaper PDF I attached to my previous post, you'll surely remember (not an easy sight to get out of your head, I guess) I had painted on my back "Veh culo libre de emisiones" - Emissions-free vehicle (and yes, it's strictly true: My bike is zero emissions. The animal riding it might not be... But that's a different story). Add incomplete sunblock to the equation, and...

Were it not for the poor lighting conditions under which I took the photo, you'd clearly appreciate the words "libre de" on my back.
And... Well, one week later, my freedom itches.
Badly.

Theodore Ts'o: Learning how to communicate

Pity poor Dr. Ari Jaaksi from Nokia. He gave a talk at the Handsets World conference in Berlin on Tuesday, where according to ZDnet, he lectured Open Source Developers that they needed to learn why DRM and other closed technologies were necessary, because of business issues such as subsidized (device) business models. I suspect he wasn’t prepared for the reaction, which took the form of a major fuss on Slashdot, as well as some declarations from a few people on the maemo-users mailing lists that they would never buy another Nokia device. Dr Jaaski then posted today on his blog an entry entitled, “Some learning to do?”, where he stated that while Nokia needs to learn how the open source world works (not just licenses and legal issues, but also the spirit), that the open source world also needed to learn as well — about WHY things are the way they are. He ended the post with “I’m not a teacher, I’m a learner” — which is fair enough. I actually think that Dr. Jaaksi is right; we all should be interested in learning, and having talked with a number of Nokia employees, I can testify that their hearts are very much in the right place, and there are some strong reasons regarding the demands of carriers, the subsidized business model of handsets in the US, the attitudes of suppliers for components in mobile devices, that very much tie their hands — as well the hands of every other handset company out there. Progress in this space will come slowly, and but I believe there is hope that in the long run things will get better. However, at the same time, I would suggest to Dr. Jaaksi that it’s also important how to communicate with the open source community — or with any community that holds views sometimes with great passion. Many people do not react well to being told that they need to learn something — even if it is true that they do. Admitting that they need to learn is for some people tantamount to admitting ignorance, or even worse admitting stupidity, and so they are loath to do this. Telling them this just makes them defensive and angry. My suggestion for how get people to learn without making them defensive is to use a modified Socratic method. It works especially well with the Open Source crowd, because many of them are engineers, and engineers are by nature problem solvers. So if you give them a problem to solve, they will go to work trying to find potential solutions, and thus start thinking and learning about the problem from a different point of view. Hence, a better approach is to give them a series of business constraints (time to market, the demands of carriers, the fact that most end users prefer to buy subsidized phones with 1-2 year lock-in contracts, the intransigence of suppliers, why a single company can’t afford to create their own chipsets from scratch and have to rely on companies like Broadcom, etc.) and then tell them — we’d love to delight our customers in any way we can, and clearly you are very passionate about DRM, open device drivers, and other OSS issues. We however have to create successful products and sell them at a price such that we don’t go out of business — can you help us work through all of these constraints? Let’s work together so we can make an open source mobile platform that meets the spirit as well as the legal requirements of the open source world! Dr. Jaaksi and his team are struggling with a hard problem, and I am very sympathetic to their issues. And, they have to be commended for investing as much time and effort on the Maemo platform, as well as other Linux and Open Source platforms as they have to date. The strength of the Open Source community is that we work together to solve our mutual problems, while celebrating and giving permission to people to “scratch their own itches”. In order to do that, we need to learn how to better communicate with each other, and that is something that goes in both directions. When I was at MIT as an undergraduate, far too many years ago than I care to admit, I took economics classes and Sloan School of Management classes (even though they were not necessary for a Computer Science degree) for that very reason. I encourage open source advocates to learn how to walk a mile or two in business people’s shoes, since if you want to influence them them to change in the ways that you desire, you need to know where they are coming from — just shouting and throwing rotten tomatoes generally won’t get you very far. On the flip side, people who are on the corporate side of the divide also need to understand that certain ways of engaging the community can be more productive than others. At the end of the day, hopefully we can all agree with Dr. Jaaksi that everyone will benefit if we all commit to learning from each other. Update: I’ve fixed Dr. Jaaksi’s name so it is spelled correctly; my apologies for the error.

Jordi Mallach: Upgrade to PyBlosxom 1.4.3

This week I spent some time upgrading PyBlosxom to version 1.4.3. I was still using 1.2, which probably was insecure and buggy. This is the first step in a bigger plan to replace Apache2 with nginx in this server, but that will come later. I was lucky to find PyBlosxom's author, Will, on IRC at the right time, who kindly answered a few questions and helped solve a few issues with the comments plugin and flavours. So, after a while, I had fixed a few subtle, 4 year old bugs in my XHTML templates and more notably, fixed lots of small bits in the rss feed, which finally makes Liferea and Advogato like my entries. But, the biggest achievement was getting a brand new comments.py plugin from Will, which allows to close comments on entries after an expiration date. So, even if I was happily using Mako's Akismet plugin, I still was getting 5 or 6 spams each day on very old entries (favourites being one about Alonso visiting Val ncia and one remembering the 70th anniversary of the Spanish Civil War). Well, not any longer. My dear spammers, you can now go pester someone else, or pick new entries pretty quickly before they get closed down. It's been a nice fight, but it's a good time to wish you go away and fuck off. With love, Jordi. Thank you, Will!

Russ Allbery: FAI and Ubuntu

My project for the past three days has been to upgrade our etch-based Debian build system to add support for Ubuntu hardy. We're primarily a Debian shop, but we want to use Ubuntu for the public timeshare systems due to its faster stable update cycle. In that environment, the latest and greatest software and active security support are both important simultaneously, and server-level stability isn't as vital. Ubuntu wins over other options because it's basically Debian and therefore shares with Debian the huge package selection, which is important for university timeshare systems. (We also use the external R package repository; we have a lot of heavy R users.) First, FAI has changed a lot since the etch release and hardy has a far more recent version of fai-client, which means that just bootstrapping an Ubuntu NFS root on an etch system and trying to install it with the etch FAI doesn't work due to the mismatch between the server expectations and the fai-client and fai-nfsroot packages. FAI during the etch release boots with a traditional read-only NFS root and works around the fact that its boot device is read-only. Current FAI does essentially the same trick that a live CD does, using unionfs to allow local modifications to the NFS root with a backing tmpfs file system, which is kind of scary but allows FAI to be much simpler. The first thing I tried was debootstrapping Ubuntu (using the debootstrap from backports.org) and then downgrading its fai-client and fai-nfsroot packages to the etch versions, but changes in other part of the system (particularly mount) makes that not very viable. One of the nice things about FAI is that it's basically a collection of shell scripts, which means that it's architecture-independent and one can easily install the unstable packages on an otherwise-etch system. So I upgraded fai-server to the current Debian sid version of FAI and installed the sid versions of fai-nfsroot and fai-client into the Ubuntu NFS root after generating it with make-fai-nfsroot as normal. That works much better. However, there are a few Ubuntu-specific problems to work around (which are patched in the Ubuntu version of FAI, but I don't want to have to track down an Ubuntu system to generate the NFS roots when the rest of our infrastructure is etch): Other than that, most of the work was upgrading the whole system to using the new boot method (changing to boot=live on the PXE kernel options, including live-initramfs in the NFS root, copying the kernel and initrd out of the NFS root into the tftp area instead of using fai-kernels, and adding initrd to the PXE kernel options). I'm now successfully building etch and hardy systems using our FAI setup and will probably upgrade the production service next week. There is one other fairly serious Ubuntu design misfeature that is only tangentially related but that cost me a lot of time. Ubuntu has modified update-grub to make it use ucf, which breaks the traditional update-grub semantics in extremely strange and confusing ways that took hours for me to diagnose. For those who aren't familiar with the details of how update-grub works, it uses comments in /boot/grub/menu.lst as a template to generate the kernel list at the end of menu.lst. It's not the best design, since it uses the same file as both a template and the active configuration file, but it works, and has worked the same way for many years. It's simple and reliable: you only change the defaults in the comments and then update-grub adds in the kernel-specific information and generates the active menu for you. If you want to edit the kernel list yourself for some reason, don't use update-grub; it's completely optional. To see how using ucf breaks this, remember that ucf's goal is to preserve local changes to a configuration file, but its notion of local changes is fundamentally contrary to how menu.lst works. Suppose that you're a large organization with a lot of systems with a variety of kernel revisions, for one reason or another, but with a desire to have uniform kernel parameters like console speed and timeouts. With update-grub, you can just copy your menu.lst template to all of your systems and run update-grub afterwards, and update-grub will apply your new defaults and generate a new kernel list using them. However, with Ubuntu's update-grub, it generates the initial menu.lst and kernel list, and then when you replace that with a fresh template, ucf sees that you've removed the kernel list. It thinks that's a local modification, and when update-grub generates a new kernel list, it "preserves" your modification and leaves the kernel list out of the new menu.lst. Tada, instant unbootable system. Even worse, this bug is sticky: you can re-run update-grub all you want, it succeeds without errors, and it keeps generating an empty menu.lst. This is a completely mystifying error until you dig into the details of what's going on. To recover, you have to run ucf --purge /var/run/grub/menu.lst (and that path, or for that matter any of the ucf stuff, is completely undocumented in the update-grub man page) and then re-run update-grub, at which point ucf will display a debconf prompt (which doesn't support debconf preseeding, but that's another problem). If you then select three-way merge (not the default) from that prompt, you get a correct menu.lst. I'm not sure why Ubuntu made this change, but as near as I can tell, it's fundamentally architecturally broken. The specific problem I had was that we were copying our template over and re-running update-grub in a post-install script, resulting in a system that always had an empty kernel menu and wouldn't boot. Once I finally figured out what's going on, I worked around it by being very careful to put our template in place before the initial system bootstrap and then never changing menu.lst afterwards, but this is a bad limitation for a large site that scales system administrators by automating configuration file updates. But maybe I'm missing some subtlety. I'll go file a Launchpad bug shortly after posting this and we'll see what happens.

12 June 2008

Russell Coker: Wyndham Resorts is a Persistent Spammer

Over the last week I have received five phone calls from Wyndham Resorts asking if I would like to be surveyed. Every time I tell them that I am not going to do their survey, on all but one call I had to repeatedly state that I would not do the survey for more than two minutes before they would go away. The advantage of phone spam over email spam is that the caller pays, I guess that they have a time limit of three minutes when calling a mobile phone to save on calling costs. There have been a number of proposals for making people pay for email to discourage spam. Even a cost of a few cents a message would make spam cease being economically viable for a mass audience (a smaller number of targeted spams would be easier to block or delete). But such plans to entirely change the way email works have of course failed totally. But for phones it could work. I’d like to have a model where anyone who calls me has to pay an extra dollar per minute which gets added on to their phone bill. When people who I want to talk to call me I could reimburse them (or maybe be able to give my phone company a list of numbers not to bill). This could also seamlessly transition to commercial use. I would be happy to accept calls from people asking for advice about Linux and networking issues for $1 per minute. With all the people who call me about such things for free already it would be good to answer some questions for money.

Share This

Enrico Zini: How to autologin X without a display manager

How to autologin X without a display manager Problem: configure a custom Debian box used to drive some industrial machinery. The system should boot directly into the GUI control application, that runs full screen, with root privileges. Everything should respawn if X is killed or the control application dies. In theory, you'd run an X display manager with autologin, then run matchbox-window-manager and the control application as the X session. You wish. At the end of the post is an explanation of why this way failed. So, here is how to get the whole thing to work, without a display manager. Use init to drive the whole thing: 
6:23:respawn:/sbin/getty -L -n -l /usr/local/sbin/autologin
This will respawn everything if it dies, stop respawning if it dies all the time, avoid starting it in single user mode, and not ask for a username. /usr/local/sbin/autologin contains: 
#!/bin/sh
/bin/login -f root MAINAPP=true
This will autologin as root, setting an extra env variable. Then comes root's ~/.bash_profile, that just starts X if we are doing autologin: 
if [ "$MAINAPP" = "true" ]
then
    startx
    logout
fi
If the application was running as a special user, we could have made things simpler and just used startx as the shell for that user; however, we still want root to have bash as the shell, and the above hack does it. Finally, root's ~/.xsession: 
#!/bin/sh
matchbox-window-manager &
# If the touch screen is not calibrated, run the calibration
while [ ! -f /etc/touchscreen-calibration ]
do
        calibrate-touchscreen
done
# Run the main application: if it ends, the session ends
main-application
And there we go, no dependencies at all. Why not using a display manager gdm and kdm seem to do autologin, but their dependency list is not acceptable for something that should just respawn an X server in an industrial system that must be kept simple. xdm on the other hand has a small set of dependencies, but its developers seem to have decided that autologin is a ""glitz", and there is no need for it in such a bare-bones display manager". Dict for "glitz" gives "tasteless showiness". What one has to bear... wdm seems however even more disconcerting, as it "doesn't actually support autologin but you can set the default user and default password in /etc/X11/wdm/wdm-config. Now, [...] you only need to press Enter twice [...] to login". Figure how this would look in the manual: "After powering up the unit, attach a USB keyboard and press enter twice to start the system". And this is why we are not using a display manager.

Jeff Bailey

Dear recruiters #2: There are now *two* standards to follow: not only can you not send me posts from a noreply address, you should also try to convince me that your founders are insane ;)

I seem to have quite accidentally set a new standard in what it takes to hire me. According to Slashdot, Google co-founder Sergey Brin is planning to go to the ISS. The founder of Ubuntu, Mark Shuttleworth did the same.

" You know we're sitting on four million pounds of fuel, one nuclear weapon and a thing that has 270,000 moving parts built by the lowest bidder. Makes you feel good, doesn't it?"

Actually, looking back at almost all the jobs I've ever had, the founders or key management folks were almost *always* insane. I remember a fun conversation with Jane, the COO of Canonical about how I wanted the senior management of any place I work to have such a big vision and to drive relentlessly towards it.

I think she thought *I* was insane. Probably still does ;)

Thinking of which, I'm so happy to be back at work. I'd forgotten what the pace here was like, or maybe it's just coming off of heavy drugs and spinning up to this pace. I feel a bit like a Smart Car on the New Jersey Turnpike. I'll make freeway speeds, but I need a few more days. =)

I reduced the drugs quite rapidly in the last week, and am now down to one Ultram per day, with extra-strength Tylenol supplementing. I also got my second HIV test results back, and it's non-reactive. So it's 99% probable that the blood transfusions were clean. Yay!

Amaya Rodrigo: Strange Findings


Strange Findings
Originally uploaded by Amayita.
I was walking down my street, minding my own bussiness when I found these shoes, abandoned on the sidewalk, waiting for her owner to come back.

Russell Coker: ECC RAM is more useful than RAID

A common myth in the computer industry seems to be that ECC (Error Correcting Code - a Hamming Code [0]) RAM is only a server feature. The difference between a server and a desktop machine (in terms of utility) is that a server performs tasks for many people while a desktop machine only performs tasks for one person. Therefore when purchasing a desktop machine you can decide how much you are willing to spend for the safety and continuity of your work. For a server it’s more difficult as everyone has a different idea of how reliable a server should be in terms of uptime and in terms of data security. When running a server for a business there is the additional issue of customer confidence. If a server goes down occasionally customers start wondering what else might be wrong and considering whether they should trust their credit card details to the online ordering system. So it is obviously apparent that servers need a different degree of reliability - and it’s easy to justify spending the money. Desktop machines also need reliability, more so than most people expect. In a business when a desktop machine crashes it wastes employee time. If a crash wastes an hour (which is not unlikely given that previously saved work may need to be re-checked) then it can easily cost the business $100 (the value of the other work that the employee might have done). Two such crashes per week could cost the business as much as $8000. The price difference between a typical desktop machine and a low-end workstation (or deskside server) is considerably less than that (when I investigated the prices almost a year ago desktop machines with server features ranged in price from $800 to $2400 [1]). Some machines in a home environment need significant reliability. For example when students are completing high-school their assignments have a lot of time invested in them. Losing an assignment due to a computer problem shortly before it’s due in could impact their ability to get a place in the university course that they most desire! Then there is also data which is irreplaceable, one example I heard of was of a woman who’s computer had a factory pre-load of Windows, during a storm the machine rebooted and reinstalled itself to the factory defaults - wiping several years of baby photos… In both cases better backups would mostly solve the problem. For business use the common scenario is to have file servers storing all data and have very little data stored on the PC (ideally have no data on the PC). In this case a disk error would not lose any data (unless the swap space was corrupted and something important was paged out when the disk failed). For home use the backup requirements are quite small. If a student is working on an important assignment then they can back it up to removable media whenever they reach a milestone. Probably the best protection against disk errors destroying assignments would be a bulk purchase of USB flash storage sticks. Disk errors are usually easy to detect. Most errors are in the form of data which can not be read back, when that happens the OS will give an error message to the user explaining what happened. Then if you have good backups you revert to them and hope that you didn’t lose too much work in the mean-time (you also hope that your backups are actually readable - but that’s another issue). The less common errors are lost-writes - where the OS writes data to disk but the disk doesn’t store it. This is a little more difficult to discover as the drive will return bad data (maybe an old version of the file data or maybe data from a different file) and claim it to be good. The general idea nowadays is that a filesystem should check the consistency of the data it returns. Two new filesystems, ZFS from Sun [2] and BTRFS from Oracle [3] implement checksums of data stored on disk. ZFS is apparently production ready while BTRFS is apparently not nearly ready. I expect that from now on whenever anyone designs a filesystem for anything but the smallest machines (EG PDAs and phones) they will include data integrity mechanisms in the design. I believe that one such features become commonly used the need for RAID on low-end systems will dramatically decrease. A combination of good backups and knowing when your live data is corrupted will often be a good substitute for preserving the integrity of the live data. Not that RAID will necessarily protect your data - with most RAID configurations if a hard disk returns bad data and claims it to be good (the case of lost writes) then the system will not read data from other disks for checksum validation and the bad data will be accepted. It’s easy to compute checksums of important files and verify them later. One simple way of doing so is to compress the files, every file compression program that I’ve seen has some degree of error detection. Now the real problem with RAM which lacks ECC is that it can lose data without the user knowing. There is no possibility of software checks because any software which checks for data integrity could itself be mislead by memory errors. I once had a machine which experienced filesystem corruption on occasion, eventually I discovered that it had a memory error (memtest86+ reported a problem). I will never know whether some data was corrupted on disk because of this. Sifting through a large amount of stored data for some files which may have been corrupted due to memory errors is almost impossible. Especially when there was a period of weeks of unreliable operation of the machine in question. Checking the integrity of file data by using the verify option of a file compression utility, fsck on a filesystem that stores checksums on data, or any of the other methods is not difficult. I have a lot of important data on machines that don’t have ECC. One reason is that machines which have ECC cost more and have other trade-offs (more expensive parts, more noise, more electricity use, and the small supply makes it difficult to get good deals). Another is that there appear to be no laptops which support ECC (I use a laptop for most of my work). On the other hand RAID is very cheap and simple to implement, just buy a second hard disk and install software RAID - I think that all modern OSs support RAID as a standard installation option. So in spite of the fact that RAID does less good than a combination of ECC RAM and good backups (which are necessary even if you have RAID), it’s going to remain more popular in high-end desktop systems for a long time. The next development that seems interesting is the large portion of the PC market which is designed not to have the space for more than one hard disk. Such compact machines (known as Small Form Factor or SFF) could easily be designed to support ECC RAM. Hopefully the PC companies will add reliability features in one area while removing them in another.

Share This

11 June 2008

Christian Perrier: [life] Manu en su casa

Yes, Manu Chao was really home yesterday in Paris Bercy. And he was obviously feeling like being home. The concert began at 21:30 and ended up at 00:30. Yes...that makes 3 full hours concert. All that for 25 euros. The crowd support, dance, participation was *amazing*. I often go to concerts in the Bercy arena and never saw the crowd in the upper rows being stood up all concert long. Radio Bemba played their usual tracks first. As usual, the very Manu way to mix songs, themes from all his carreer and mix them up, distord them and make new songs from old ones. This is what always made Mano Negra, then Manu Chao oncerts unique experiences. Just imagine 20,000 people singing Clandestino a capella, most of us not knowing any damn word of Spanish... After about 2 hours of magic without any stop, the very special moments came in. Manu, Gambit, Majiv (what a guitarist when playing rumbas de Barcelona!), Julio (de Barcelona!)...and all others were just feeling like being home with a few friends. So the encores lasted...and lasted...and lasted. They just didn't want to stop (Manu probably said about 3 or 4 times "c'est fini"). Then, they did something I never ever saw in a big 20,000 people concert since my first one back in....long time ago. Manu asked "do you have 5 more minutes?". So, they stopped for 5 minutes to get the guitars re-accorded(sp?)....the crowd just...waited...and the gig re-started again with songs from "Si Sib rie m' tait cont e", the special book-album he made with friends, inspired by the sad winters he spent in Paris in his early years. Soooooo special and, believe me, that *was* special. Seeing all roadies, crew, friends just dancing backstage and partying clearly showed this was not to always happen (this will probably happen again on today's concert). He was just feeling like home. And so were we. Not wanting to leave and shut down the light. Finally, at 00:30, all of us were dead tired and we just hugged around with Manu and left. Just like old friends. I don't know if that will happen everywhere in the upcoming tour (some dates in France, some in Europe, a few in USA....nothing unfortunately in South America as of now)....but if you have the opportunity to go to Radio Bemba concerts, just do it. And (IRC reference), really, Erinn, this was not about "ethnic music for white people" (really wondering where you got this idea...or if you were just kidding me). I dunno what Manu ever did to kill your ears, but just try again and, even more important, go to the gig in your fair city on August 22nd. Radio Bemba concerts will be cheap, about half the price of regular bug size concerts....just use that opportunity.

Adam Rosi-Kessel: No time for blogging, but

I’ve been on the road a lot lately with no time to blog (or sleep). Following are a couple items to keep this space from going completely dead. A wild turkey stopped by our backyard (in Boston): a turkey I called city animal control, and they said these guys are everywhere. Apparently there is no more wild left for the wildlife. In other news, Gears is finally available for Firefox 3! For me, at least, this is a big deal, since I frequently depend on offline mode for Google Reader and have otherwise fully migrated to FF 3 for the speed benefits. Since RC2, it’s mostly stopped crashing as well. Technorati Tags: , , , ,

Share This

Peter Van Eynde: google me: sendfile problems

After upgrading an ubuntu LTS server at work proftpd (not my choice) would not transfer files anymore. It would die with:

error using sendfile(): [75] Value too large for defined data type

After searching a little I found the problem: sendfile does not work if the filesystem on which the files are stored is 'special'. Of course we are serving files from a CIFS directory... Sigh.

Next.

Previous.