As the title says - the Ftpmaster dak repository is now using git instead of bzr. I never did like bzr, so I recently asked what the rest of the team thinks of switching to git. Luckily I got no objections, so here we are, with a VCS thats not dead-slow. If you want to do dak development and submit patches you want to read my post to the debian-dak list and then clone the git repo. Sorry, there is no git-daemon support, you need to use the http way if you do not have access to ries. Unfortunately debian-admin thinks git-daemon isn’t good enough to run on ftp-master and that “http is enough”. Bah. (No, I won’t work around it (git-daemon uses high-port, so could be run without admin help, but NO WAY), and I don’t want to put it on alioth). The actual conversion of the repository was done by Dato, with me commenting and asking for stuff. Like the fixup of the old revisions committer names, so all the old CVS revisions now correctly tell the full name/email of the committer, not just the unix login. Or fixing all the bzr commits done by “ajt@ries”, “joerg@ries” and other broken combinations of “login@machine”. I also use his git hook to send commit mails to get commits mailed to the debian-dak lists. [Update] Nice, with the setup we use we found a bug in git. Ha. (a stupid issue with git shared repositories, and reflogs enabled at the same time that basically means you cant commit as the logs got the wrong permissions set whenever someone else did an action, like git gc.) Comments: 2

Ten years ago, most of us wouldn't have dreamt we'd be managing terabits of storage, tens of megabits of bandwidth, arrays of network-distributed services. The height of a programmer's worry would likely be choice of UI toolkit or finding the right way to indent code, and the height of consumer concern deciding which room to put the new computer in. Now the problems associated with managing large networks are becoming real for everyone, right down to the consumer level. Stupendously large amounts of computing resource are available at an instant. Your household probably has more than a terabyte of storage already. Issues such as single sign-on are going to hit home over the next year, as networked computing and entertainment devices profilerate. Features such as Apple's Time Machine will be increasingly vital — software that makes traditionally gnarly sysadmin tasks consumer-friendly. The rebranding of .Mac into "Mobile Me" is also a step in this direction. The impact on developers As software developers, we also have to cope with the effects of this resource-richness. For small sums of money we can get access to large computing clusters, geographically redundant hosting services. Our programs have left the desktop and found their new home on the web. System administration issues loom large upon us, security concerns lurk auspiciously in the corners of our minds. Although the cost of infrastructure has dropped radically, other costs remain high and are going to stay that way. System administrators are not only grumpy, they demand high wages. Commercial software license fees spiral out of control: traditional per-CPU licensing models make little sense when you can quickly bring up tens of machines. The cost in power is already troubling large companies, and there's no reason to suspect the problems won't ripple down. Help is at hand from a variety of technologies. If they don't yet make massive resource management trivial, they at least make it possible. Some of these also inhabit the weird territory of being both the source of a problem and a solution at the same time: virtualization, for example. Distributed revision control systems Distributed revision control is a technology whose time has finally come in popular circles, thanks in part to Linus Torvald's Git system. DRCS has several important impacts on today's developer:

• Branching and, importantly, merging become much cheaper, allowing agile and flexible iterations of development.
• Loosely connected and geographically diverse development becomes much easier. Even within a single organization it is not uncommon to find teams spanning countries, time zones. Complex multi-site VPN setups aren't necessary when a few SSH keys can do the job.
• Revision control becomes packaging on the cheap. Like it or not, the mere tagging of a source tree has now become a valid option for releasing software.

All these trends lower the barrier to entry, increase collaboration and agility of development. You can the value of this as more software tools become free. Selling such tools is rapidly becoming a thing of the past, the advantages of sharing enable the developers at the sharp end to get their jobs done quicker. However, such increased agility and, well, messiness leave other problems to solve, which the next two technologies address. Virtualization Hardware-as-a-service, infrastructure-as-a-service, call it what you will. The ability to create what we used to call entire machines, pick them up and move them around the network is revolutionary, and it's something that will have a real impact on regular developers. The benefits are at several levels.

• Agile infrastructure — a ready supply of new machines makes it a lot easier and cheaper to test different scenarios, architectures, and to separate concerns. If things go wrong, throw away the image and start over. It's all about cutting the administration load.
• A packaging solution — the new macroeconomics of software distribution mean that distributing entire machine images which communicate exclusively via the network is now a feasible way to distribute your software. We must adjust to the notion of distributing appliances, not code. We may mourn the lost crafts of creating RPMs or installers, but let's face it, it's now a waste of time.
• New business models — your application can now be delivered as a black-box appliance, circumventing compatibility issues, or as a service, with virtualization part of the solution to scaling.

Configuration management Computing is a zero-sum game, and despite our increased ability to create and distribute software, problems still exist. We just pushed them to the next level. In good part, this next level is the problem of configuration management. We now have networks and clusters of (virtual) machines, software so agile we need six decimal places to describe its revision levels, and network and authentication paths that are starting to tangle. How do we manage that? One thing developers crave is repeatability. That's why we love our makefiles, autoconf, Ant, rake and so on. It's the one time even the most imperative-minded programmer writes declarative code. We like to say "let the world be like this." Our new sprawling world lacks this feature, and the best of our old toolkits — .debs, RPMs — address things only at the level of packages in a single environment. So developers must look to the world of operations, a territory we probably thought we needn't enter. In this world the new "make" is called Puppet. You write recipes to describe how things ought to be, and Puppet will make it so. I've been spending some time digging into Puppet, and feel excited by the confidence it's giving me. Now my applications exceed single source trees, and single machines, it gives me the means to tie the whole together. This article was going to be solely about Puppet, but that will have to wait now for another time. It's likely you'll have played with virtual machines and distributed revision control, but have you tried Puppet yet? Give it a spin, and let your mind wander over the benefits for your organization and development approaches. Conclusions
For developers and users alike, our world is changing. Hardware, connectivity and increasingly software is becoming cheap or free. The solidity of the old things we put value on — real things you can touch like disks — is eroding. What really matters is our data, our creations, and their communication. If they don't quite yet exist in a universal "cloud" yet, they're certainly getting frisky. As vendors provide solutions for consumers to manage their new domestic infrastructure, developers must look to network-aware toolkits and operations techniques to manage and get the best from their emergent infrastructures. Also on this topic:

• In search of agile infrastructure for web applications (June 2006)
  

Join the conversation about this post

Yes, another post about the Geode drivers. After much head banging, I finally figured out that the sloppy method used to generate the list of supported PCI ID for each driver in Debian and Ubuntu is what wreaked havoc with the GEODE driver:

1. Support for the GX2 chipset was backported into the old NSC driver, which created a PCI ID conflict with the official support that is present in the GEODE driver.
2. To make matters worse, that damn NSC driver also conflicts with the older CYRIX driver that supports earlier versions of the Geode chipset.

To fix both issues, I have produced new NSC and GEODE packages and uploaded them into my Ubuntu PPA for the Hardy series. Tip: as soon as you have upgraded, purge the -amd transitional package; it only confuses the X server and disrupts operation if you leave it there, especially inside an LTSP chroot. If you were using a static xorg.conf, simply upgrade the Device line to use "geode" rather than "amd" and you're done. After upgrading to these packages, normal operation should be restored for all Geode variants. As a bonus, X should now even be able to work without an xorg.conf (of course, with the wrong keyboard map, unless you live in USA) and in all known LTSP variants. Enjoy!

There has been a lot of talk recently about the cost of petrol, Colin Charles is one of the few people to consider the issue of wages in this discussion [1]. Unfortunately almost no-one seems to consider the overall cost of running a vehicle. While I can’t get the figures for Malaysia (I expect Colin will do that) I can get them for Australia. First I chose a car that’s cheap to buy, reasonably fuel efficient (small) and common (cheap parts from the wreckers) - the Toyota Corolla seemed like a good option.

A search on www.drive.com.au shows adverts for Corollas made in 1990 and 1991 for prices ranging from $3000 to $5000. Assuming some optimism on behalf of the advertisers I could probably buy one for about $2500. The current term deposit rate from the major Australian banks is just over 8%, so spending $2500 on a car (instead of having the money in a term deposit) means a loss of $200 per annum interest, if instead the car was purchased with a bank loan the rate would be significantly higher (car loans have quite high interest rates). Of course the real cost is whenever you sell a car and buy a newer one that costs more - but this is harder to estimate. The next issue is insurance, for a 1990 Corolla of the smallest engine size (as a rule of thumb the smaller the engine the smaller the insurance bill) comprehensive insurance (sans optional extras such as windscreen protection) would cost me $489 per annum, and third-party fire and theft would cost me $398. Of course those rates are for someone who is more than 25 years old and has a good driving record. The majority of people who “flip burgers” appear to be younger than 25 and would therefore pay more. Car registration starts at about $550 per annum (varies by state). Car service would cost at least $200 a year (for scheduled services). So it seems that the cost of a cheap car is around $1,350 per annum before you even drive it! Now with a car that’s 18 years old you have to expect that some parts are wearing out, including parts that make it unroadworthy and must be replaced. Assuming that $300 per annum is spent on parts and labor to repair parts that wear out seems quite conservative (especially if driving a car that costs less than $3,000 - one of the cheaper cars in that age range). That brings the total cost per year to $1,650 if you are lucky (if unlucky then expensive parts could wear out at financially inconvenient times). The amount of driving that I do is around 5,000 to 7,000km per annum (according to the ABS the average passenger vehicle was driven 14,100Km in 2005 [2]). Assuming 7.5L/100Km (one of the results that has been quoted for fuel efficiency of a car of that age) and 7,000Km of driving that would be 525L of petrol per year, at $1.70 that is $892. So at current fuel prices when driving a small car for minimal distances (either for environmental or financial reasons) the cost of petrol may be about 35% of the vehicle operating cost. If driven for the average distance the fuel cost will be about 52% of the total vehicle operating cost (which is about $3,400) - which seems like a reasonable incentive to save fuel. If someone wants to advocate measures to make cars more affordable for poor people then the first thing to consider would be a reduction in the vehicle registration fees (which are a significant part of the vehicle operating costs). Another would be government incentives for car sharing schemes (see my previous post on car sharing for details on what is on offer in Melbourne at the moment [5]), $3,400 would give 340 hours (or more with the weekend deals) of driving a shared car (including fuel) at current prices. A shared car would not be suitable for driving to work but would be good for most other car use. Also it should be noted that share cars are new and have all the latest safety features - I really wouldn’t want to be involved in a crash while driving a 20yo car. Unlike the costs of registration etc for owning a car, all the costs of a shared car will be reduced if you use it less. The tax rate for someone earning $28,579.20 is $2,850 plus 30c for each $1 over $25,000 [3], which is $3,923.76, which leaves a total after tax income of $24,655.44. Someone working 40 hours a week on the minimum income who drives an average distance will spend about 14% of their after-tax income on their car if they have a small old car and were able to buy it without borrowing money. According to the ABS the median income is actually slightly lower than full-time work at the minimum wage [4]. So it seems that there are many people who are working part-time and earning less money than they would if they worked full-time at the minimum wage. So the question for Colin is, how does this compare to Malaysia? Is someone on a minimal income in Malaysia going to spend ~7% of their after-tax income on paying insurance, tax, and basic maintenance on a car without the additional expense of driving it? How does the cost of the basic car ownership compare to the cost of fuel in Malaysia?

• [1] http://www.bytebot.net/blog/archives/2008/06/15/on-fuel-subsidies-and-earningspending-power


• [2] http://www.abs.gov.au/AUSSTATS/abs%40.nsf/mediareleasesbyReleaseDate/66E24E957EB09229CA2571E10077D71D?OpenDocument


• [3] http://www.ato.gov.au/individuals/content.asp?doc=/content/12333.htm&mnu=5053&mfp=001


• [4] http://www.abs.gov.au/AUSSTATS/abs%40.nsf/7d12b0f6763c78caca257061001cc588/96e08af8f4178a45ca257306000d4fd4!OpenDocument


• [5] http://etbe.coker.com.au/2008/04/13/car-sharing-in-melbourne/

Share This

http://www.emdebian.org/buildd/

The scripts are in emdebian-tools 1.1.6 and with a few more fixes and tweaks, that will be the main body of v1.2.0 for Debian unstable.

A few issues with ssh and sudo passphrase caching vs running buildd scripts as root - I'll need to improve the documentation of the process before any other buildds can be set up. The need to have emdebian-tools and subversion installed inside the buildd chroot does make things a little more complicated than a standard wanna-build implementation - as does the need to download, build, install and clean-up cross dependencies.

This autobuilder is the Emdebian target package buildd - Hector Oron has another to build the cross-building toolchains. The toolchain autobuilder has fewer packages but each package is larger and is built for more architectures. Adding more architectures to the target autobuilder is quite difficult - packages that use cache files will need new cache files with different values for certain cache values. See the wiki for info on how to fix cache files. Each solves a different problem within the overall design of any buildd. In each case, instead of listening to email from incoming.debian.org, each buildd picks up the incoming data from the apt cache - a useful little time delay that gives me time to head-off certain problematic updates.

The target package autobuilder runs incrementally - a package is repeatedly rebuilt until it works and is then left alone until the next Debian release. There's lots of work available on the frontend if anyone fancies helping with the PHP. (See Emdebian SVN).

The advantage now is that as more packages fix their cross building bugs, more packages will autobuild without intervention which frees up more of my time to fix the more complicated problems. Equally, I can refer to the buildd logs in bug reports, helping everyone see what is going on.

Dear CNN,

The US presidential election race between senators McCain and Obama is not a "battleground". Battlegrounds are where people get killed. Using the same words for what passes as democracy in this country as for the military massacres in foreign countries is incredibly sad and disgusting.

That, and it's only vaguely less stupid than Toronto's "War on Fire".

postfwd 1.10pre7c-1 has been accepted and did hit experimental yesterday.
If you using policyd-weight you might want to give it a try, since its active development stoped. If you are annoyed by too much spam, it maybe a good idea as well.
Please have a look on postfwd.cf, there is no default config provided, since the configuration depends heavily on your environment. But maybe you have a look at /usr/share/doc/postfwd/examples, there is what you expect.
If you have any comments, send a mail to the maintainer or if there are bugs I didn’t spotted yet, please fill one!
In case I don’t respond quit fast, maybe I’m in the delivery room or anywhere else in the hospital or something like that, so please give me some extra time.
In case there are no big issues, I plan to upload the package to unstable in a reasonable time.

So TV has finally come to the browser. The current incarnation that I'm playing with is Hulu. Hulu has two seasons of babylon 5, so that's what I have been watching. The annoying part of Hulu is that even after it has buffered the video, it will still pause during play back if a network connection is lost. My guess is that they have a set of pings they send back to ensure you are still watching. If a ping has not been acked in so long, then a synchronous ping will be send which blocks playback until the video can continue.

Xserver 1.4.2 for Lenny We originally planned to ship Xorg 7.4 and Xserver 1.5 since they were expected in February. However, Xserver 1.5 (and Mesa 7.1) are not released yet, so we are going to keep some updated X.org 7.3 for Lenny. A new Xserver 1.4 snapshot with some security fixes will enter testing soon. Once this is done, the final Xserver 1.4.2 will be uploaded. It is not perfect but it is the one you will get for Lenny. Many people suffered from Xserver 1.4 bugs in the last months, especially on the input side. Fortunately, many of them have been fixed. Lots of them were also caused by obsolete config files (that have to be manually fixes unfortunately). So we hope this Xserver 1.4.2 will be good enough for Lenny. XaaNoOffscreenPixmaps by default It is worth noting that XaaNoOffscreenPixmaps will now be the default (when XAA is enabled, i.e. by default for all drivers but Intel). It helps Compiz and seems to prevent various rendering problems from happening. Ubuntu and Fedora have been running such a patch for a while, so it looks like we are going to do the same for Lenny. To revert to the old behavior, use Option XaaOffscreenPixmaps in the Device section of your xorg.conf. ATI 6.8.191, Intel 2.3.2, Mesa 7.0.3++ Apart from the server, some big components are getting updated these days: A new Mesa has been uploaded so that it enters testing before the libs are frozen. It contains the latest Mesa 7.0.x git snapshot, including lots of bugfixes, especially for Intel hardware. Mesa 7.0.3 was already pretty solid, this new one (7.0.3-2) will be even better. ATI is still getting a lot of work upstream as usual. 6.8.191 (aka 6.9-rc1) has been released this week. It brings r6xx support, acceleration for r5xx, EXA Composite for r3xx/r4xx/r500, textured video support, ... It also fixes many bugs everywhere. Given the big testing that we had in experimental in the last months, I consider it much better than 6.8.0, so I decided to put it in unstable even if it's only a release candidate. This new ATI does not include the r128 and mach64 drivers anymore. So the new xserver-xorg-video-r128 and -mach64 packages finally entered unstable as well. Due to unexpected upstream version numbers for mach64 and r128 and me messing up with epochs, they have a lower version number (6.8.0-1) than the previous snapshots from experimental (1:6.8.1~git...). There's almost no code difference though, but people might want to install the unstable packages anyway. The old snapshots will be removed from experimental soon. On the Intel side, a new 2.3.2 is expected soon, without many big improvements from what I have seen. X.org post-Lenny Future Xorg 7.4/Xserver 1.5 prereleases should arrive in experimental soon. It requires Mesa 7.1, which still needs some build fix and requires a new yet-to-be-released libdrm. Once all these are properly fixed and released, you should for instance be able to experience some interesting improvement in EXA. It may finally make XAA useless for real. Also, Xorg 7.4 will simplify the Xserver package maintenance since its build won't need the whole Mesa source anymore (the GLcore module that AIGLX uses will be built/shipped within Mesa). Stop build-depending on xutils! Also, we've been trying to cleanup the dependency mess in many X packages for a while. We removed the obsolete dependency from xutils to xutils-dev which was only needed for Sarge->Etch upgrade (xutils-dev was split out of xutils). However, many packages still wrongly build-depend on xutils. Lucas reported that removing this useless dependency in the latest Xorg upload caused 84 FTBFS. So, we're going to revert the change to avoid adding 84 RC bugs. However, everybody build-depending on xutils, please check whether you actually need xutils or xutils-dev so that we can quickly fix this after Lenny.

I have been tracking the Debian release goals progress for a while now, through bug-squashing parties and using various iterations of wiki pages. It's time to write down some thoughts.

First of all, I think the concept of release goals is a fantastic idea. It may be one of the most important conceptual moves in Debian of late. It allows developers to implement distribution-wide changes without having to negotiate with every package maintainer individually and without having to go through the vicious no-change-without-policy/no-policy-without-established-practice maze. It moves the focus back on operating system development instead of package hoarding.

In the future, I think we should come up with some "sexier" release goal descriptions. If you look over the current list, many of these tasks are implementation-level, without any obvious benefit to users: double-build support? no unmet recommends? Or they have incomprehensible descriptions. The pretty much only release goal that has a real user benefit, namely the piuparts cleanness, ensuring clean distribution upgrades, is both the most cryptic and actually the most neglected one in terms of development attention. Don't get me wrong, all these goals are very valid. For better or worse, the press has started to pick up these release goals to put in their various Debian release candidate timeline pre-announcement articles, and next time around we should have something cool in there. If armel support or KDE 4 integration had been release goals, the marketing effect would be better.

The next thing is tracking bugs. If you have some time on your hands, how do you find a release goal bug that you can help with? I had worked up this wiki page to help myself, but it is still too weird. The fact that bts.turmzimmer.net exists shows that the BTS is lacking features or is too hard to use or both. I'm thinking, we should regularly copy the whole BTS into an SQL database and make queries from there. Think aggregation, grouping, full-text search! I'm not familiar with the BTS internals or the LDAP gateway, but I know my SQL. So if anyone knows the other end of this deal, I'd love to chat about setting something like this up.

So then, what happens to the release goals after the release? Has this been thought through?

Obviously, some release goals become technically self-enforcing. Python 2.5 and GCC 4.3 are now the default, and anything that doesn't work with these will have permanent trouble in the future.

Release goals that implement a distribution-wide feature, such as the LSB-based init system enhancements, will really need to become part of policy after the release. Otherwise they would revert to being normal bugs without any NMU powers and the other special attention given to release goals. So over time, the support for the former release goals across packages would again decline. To keep it up, you'd either have to repeat the release goal indefinitely, which would be silly, especially when there are no problems right now, or you'd have to make it a part of policy to enforce it in the future. Essentially, you'd have to consider most release goals trial policy changes, which become permanent if successful.

Near-examples of the above are the eternal release goals of IPv6 support and LFS support. We really have these under control quite well now -- although I still wouldn't dare to call Debian or any general-purpose Linux distribution fully IPv6-ready -- but nowadays these problems are almost all upstream code problems that Debian can't really fix very well on the packaging side. So under the current approach these release goals will exist forever, or worse, will be dropped from release goal status.

And then of course, some release goals fail to be reached. These should reapply for the next release, if interest persists. The dash goal comes to mind.

By the way, we had 13 release goals. 3 are completely done, 5 are pretty much done (less than 10 bugs), 1 more looks like it could get pretty much done before the release, and 4 will most likely fail to be accomplished.

I recently redid my personal web site, at welton.it. Wanting to be quick about it, and make the look and feel a bit more uniform than it has been in the past, I hacked together some pages in Rails. Despite this being sort of a "killing a fly with a bazooka" situation, I've been doing lots with Rails, so it was quick to use. Here's the thing, though: Rails is definitely overkill, as the site is basically static. I don't need to calculate anything or fetch stuff from a database - I just wanted a reasonably good template system, and I am quite comfortable with Rails these days. But the idea of leaving Rails running for a static site was of course no good: I basically need to cache the entire thing, so that Rails is simply not involved. How to do this as quick as possible (in between diaper changing and other baby duties!) ? Ideally, it would be possible to introspect Rails in order to know exactly which pages are present, then cache those, and avoid Rails on the server entirely (just generate them locally and put them in subversion), but that proved to be fairly hacky, so I settled for this code, which simply caches all pages which comes across, when caches_pages is placed in application.rb:

class CacheFileName
  include ActionController::Caching::Pages
  include ActionController::Caching::Pages::ClassMethods
  def cachedname(path)
    page_cache_file(path)
  end
end
def caches_pages()
  return unless perform_caching
  after_filter do  c 
    res = c.cache_page
    cfn = CacheFileName.new
    cf = Cachedfile.new :filename => cfn.cachedname(c.request.path)
    cf.save!
  end
end

It simply caches everything. To be able to easily clear out the cache if there are any changes to the site, we record the changes in the Cachedfile model, which is defined like this:

create_table "cachedfiles", :force => true do  t 
  t.string   "filename"
  t.datetime "created_at"
  t.datetime "updated_at"
end

with this model:

class Cachedfile < ActiveRecord::Base
  def Cachedfile.clean_cache
    Cachedfile.find(:all).each do  cf 
      begin
        fn = ActionController::Base.page_cache_directory + cf.filename
        File.delete fn
      rescue => e
        logger.error "Error deleting # fn : # e.inspect "
      ensure
        cf.destroy
      end
    end
  end
end

which has a class method to go through and clean out all the cached files. I call it manually from ./script/clean_cache:

#!/usr/bin/env ruby
ENV['RAILS_ENV'] = ARGV.first   ENV['RAILS_ENV']   'development'
require File.dirname(__FILE__) + '/../config/boot'
require "# RAILS_ROOT /config/environment"
require 'console_app'
Cachedfile.clean_cache

It's not a beautiful system, but it gets the job done.

Pity poor Dr. Ari Jaaksi from Nokia. He gave a talk at the Handsets World conference in Berlin on Tuesday, where according to ZDnet, he lectured Open Source Developers that they needed to learn why DRM and other closed technologies were necessary, because of business issues such as subsidized (device) business models. I suspect he wasn’t prepared for the reaction, which took the form of a major fuss on Slashdot, as well as some declarations from a few people on the maemo-users mailing lists that they would never buy another Nokia device. Dr Jaaski then posted today on his blog an entry entitled, “Some learning to do?”, where he stated that while Nokia needs to learn how the open source world works (not just licenses and legal issues, but also the spirit), that the open source world also needed to learn as well — about WHY things are the way they are. He ended the post with “I’m not a teacher, I’m a learner” — which is fair enough. I actually think that Dr. Jaaksi is right; we all should be interested in learning, and having talked with a number of Nokia employees, I can testify that their hearts are very much in the right place, and there are some strong reasons regarding the demands of carriers, the subsidized business model of handsets in the US, the attitudes of suppliers for components in mobile devices, that very much tie their hands — as well the hands of every other handset company out there. Progress in this space will come slowly, and but I believe there is hope that in the long run things will get better. However, at the same time, I would suggest to Dr. Jaaksi that it’s also important how to communicate with the open source community — or with any community that holds views sometimes with great passion. Many people do not react well to being told that they need to learn something — even if it is true that they do. Admitting that they need to learn is for some people tantamount to admitting ignorance, or even worse admitting stupidity, and so they are loath to do this. Telling them this just makes them defensive and angry. My suggestion for how get people to learn without making them defensive is to use a modified Socratic method. It works especially well with the Open Source crowd, because many of them are engineers, and engineers are by nature problem solvers. So if you give them a problem to solve, they will go to work trying to find potential solutions, and thus start thinking and learning about the problem from a different point of view. Hence, a better approach is to give them a series of business constraints (time to market, the demands of carriers, the fact that most end users prefer to buy subsidized phones with 1-2 year lock-in contracts, the intransigence of suppliers, why a single company can’t afford to create their own chipsets from scratch and have to rely on companies like Broadcom, etc.) and then tell them — we’d love to delight our customers in any way we can, and clearly you are very passionate about DRM, open device drivers, and other OSS issues. We however have to create successful products and sell them at a price such that we don’t go out of business — can you help us work through all of these constraints? Let’s work together so we can make an open source mobile platform that meets the spirit as well as the legal requirements of the open source world! Dr. Jaaksi and his team are struggling with a hard problem, and I am very sympathetic to their issues. And, they have to be commended for investing as much time and effort on the Maemo platform, as well as other Linux and Open Source platforms as they have to date. The strength of the Open Source community is that we work together to solve our mutual problems, while celebrating and giving permission to people to “scratch their own itches”. In order to do that, we need to learn how to better communicate with each other, and that is something that goes in both directions. When I was at MIT as an undergraduate, far too many years ago than I care to admit, I took economics classes and Sloan School of Management classes (even though they were not necessary for a Computer Science degree) for that very reason. I encourage open source advocates to learn how to walk a mile or two in business people’s shoes, since if you want to influence them them to change in the ways that you desire, you need to know where they are coming from — just shouting and throwing rotten tomatoes generally won’t get you very far. On the flip side, people who are on the corporate side of the divide also need to understand that certain ways of engaging the community can be more productive than others. At the end of the day, hopefully we can all agree with Dr. Jaaksi that everyone will benefit if we all commit to learning from each other. Update: I’ve fixed Dr. Jaaksi’s name so it is spelled correctly; my apologies for the error.

There has been a lot of fear-mongering about nanotech. The idea is that little robots will eat people (or maybe eat things that we depend on such as essential food crops). It’s unfortunate that fear-mongering has replaced thought and there seems to have been little serious discussion about the issues. If (as some people believe) nanotech has the potential to be more destructive than nuclear weapons then it’s an issue that needs to be discussed in debates before elections and government actions to alleviate the threat need to be reported on the news - as suggested in the Accelerating Future blog [0]. I predict that there will be three things which could be called nanotech in the future:

1. Artifical life forms as described by Craig Venter in his talk for ted.com [1]. I believe that these should be considered along with nanotech because the boundary between creatures and machines can get fuzzy when you talk about self-replicating things devised by humans which are based on biological processes.
   I believe that artificial life forms and tweaked versions of current life forms have significant potential for harm. The BBC has an interesting article on health risks of GM food which suggests that such foods should be given the same level of testing as pharmaceuticals [2]. But that’s only the tip of the iceberg, the potential use of Terminator Gene technology [3] in biological warfare seems obvious.

But generally this form of nanotech has the same potential as bio-warfare (which currently has significantly under-performed when compared to other WMDs) and needs to be handled in the same way.

The more commonly discussed robotic nanotech, self-replicating and which can run around to do things (EG work inside a human body). I doubt that tiny robots can ever be as effective at adapting to their environment as animals, I also doubt that they can self-replicate in the wild. Currently we create CPU cores (the most intricate devices created by humans) from very pure materials in “clean rooms”. Making tiny machines in clean-rooms is not easy, making them in dirty environments is going to be almost impossible. Robots as we know them are based around environments that are artificially clean not natural environments. Robots that can self-replicate in a clean-room when provided with pure supplies of the necessary raw materials is a solvable problem. I predict that this will remain in science-fiction.

Tiny robots manufactured in factories to work as parts of larger machines. This is something that we are getting to today. It’s not going to cause any harm as long as the nano-bots can’t be manufactured on their own and can’t survive in the wild.

In summary, I think that the main area that we should be concerned about in regard to nano-bot technology is as a new development on the biological warfare theme. This seems to be a serious threat which deserves the attention of major governments.

• [0] http://www.acceleratingfuture.com/michael/blog/?p=830


• [1] http://www.ted.com/talks/view/id/227


• [2] http://news.bbc.co.uk/1/hi/health/background_briefings/food_safety/278240.stm


• [3] http://en.wikipedia.org/wiki/Terminator_gene

Share This

Russ Nelson announces that the newer version of Gnash has been released. I’ve already created newer packages for it, but I cannot upload them to Debian yet, as they’re blocked for the poppler, c-ares and ffmpeg transitions. I hope to have them uploaded soon. In any case, I’ve put them here, if someone wants to try them. As the backends for the rendering, widgets and media must be decided on compile time, there’s not much to do if you want to use the most uncommon ones. The most stable and conservative option is agg and GStreamer, but I’ve also added the possibility of using OpenGL, which would be better for hardware-accelerated machines. Versions for both GTK and KDE widget systems are compiled. In a week or so they will probably be in Debian, in time for Lenny. Improvements since 0.8.2 release are:

• Native fullscreen support (from AS) implemented.
• Long command-line options properly supported.
• View detailed movie information from GTK gui.
• Added an option in Preferences to initially display a movie as a blank (”Click here to start”) screen.
• Added “dump” GUI to dump a movie to disk.
• Improved Cairo rendering performance.
• Fix support for OpenOffice Impress SWF exporter (malformed..)

My project for the past three days has been to upgrade our etch-based Debian build system to add support for Ubuntu hardy. We're primarily a Debian shop, but we want to use Ubuntu for the public timeshare systems due to its faster stable update cycle. In that environment, the latest and greatest software and active security support are both important simultaneously, and server-level stability isn't as vital. Ubuntu wins over other options because it's basically Debian and therefore shares with Debian the huge package selection, which is important for university timeshare systems. (We also use the external R package repository; we have a lot of heavy R users.) First, FAI has changed a lot since the etch release and hardy has a far more recent version of fai-client, which means that just bootstrapping an Ubuntu NFS root on an etch system and trying to install it with the etch FAI doesn't work due to the mismatch between the server expectations and the fai-client and fai-nfsroot packages. FAI during the etch release boots with a traditional read-only NFS root and works around the fact that its boot device is read-only. Current FAI does essentially the same trick that a live CD does, using unionfs to allow local modifications to the NFS root with a backing tmpfs file system, which is kind of scary but allows FAI to be much simpler. The first thing I tried was debootstrapping Ubuntu (using the debootstrap from backports.org) and then downgrading its fai-client and fai-nfsroot packages to the etch versions, but changes in other part of the system (particularly mount) makes that not very viable. One of the nice things about FAI is that it's basically a collection of shell scripts, which means that it's architecture-independent and one can easily install the unstable packages on an otherwise-etch system. So I upgraded fai-server to the current Debian sid version of FAI and installed the sid versions of fai-nfsroot and fai-client into the Ubuntu NFS root after generating it with make-fai-nfsroot as normal. That works much better. However, there are a few Ubuntu-specific problems to work around (which are patched in the Ubuntu version of FAI, but I don't want to have to track down an Ubuntu system to generate the NFS roots when the rest of our infrastructure is etch):

• Ubuntu hardy ships with a unionfs that can't write to existing files, only create new files. I'm not sure whose bug this is, possibly upstream, but etch unionfs doesn't have this problem. The workaround is to delete the files that FAI wants to overwrite out of the NFS root after generating it with make-fai-nfsroot, which you can do in a hook or by wrappering it and doing the deletions afterwards (we do the latter because we have other local tweaks we make). You want to delete /etc/resolv.conf, /etc/syslog.conf, and auth.log, syslog, daemon.log, kern.log, lpr.log, mail.*, user.log, debug, and messages in /var/log.
• Ubuntu hardy defaults to using upstart, which completely fails in an FAI NFS root environment. I'm not sure why, but first upstart claims the respawn line of every file in /etc/event.d/* is a syntax error (?!) and, even if that's fixed, doesn't actually spawn a getty (at least on a serial console). The implication is that you can't Ctrl-C at the end of the build and work on the system before rebooting, which makes diagnosing problems very difficult. I tried a bunch of different possibilities for an event.d file (and wow is that ever either underdocumented or obscurely documented — I've still never found a man page or other document clearly explaining the possible configuration options) before giving up. The Ubuntu FAI package forces installation of sysvinit in the NFS root, and that works fine (with the normal inittab tweaking to spawn a getty on the serial console). upstart works fine on the newly built system, including serial console, so I'm not sure why the NFS root is broken.

Other than that, most of the work was upgrading the whole system to using the new boot method (changing to boot=live on the PXE kernel options, including live-initramfs in the NFS root, copying the kernel and initrd out of the NFS root into the tftp area instead of using fai-kernels, and adding initrd to the PXE kernel options). I'm now successfully building etch and hardy systems using our FAI setup and will probably upgrade the production service next week. There is one other fairly serious Ubuntu design misfeature that is only tangentially related but that cost me a lot of time. Ubuntu has modified update-grub to make it use ucf, which breaks the traditional update-grub semantics in extremely strange and confusing ways that took hours for me to diagnose. For those who aren't familiar with the details of how update-grub works, it uses comments in /boot/grub/menu.lst as a template to generate the kernel list at the end of menu.lst. It's not the best design, since it uses the same file as both a template and the active configuration file, but it works, and has worked the same way for many years. It's simple and reliable: you only change the defaults in the comments and then update-grub adds in the kernel-specific information and generates the active menu for you. If you want to edit the kernel list yourself for some reason, don't use update-grub; it's completely optional. To see how using ucf breaks this, remember that ucf's goal is to preserve local changes to a configuration file, but its notion of local changes is fundamentally contrary to how menu.lst works. Suppose that you're a large organization with a lot of systems with a variety of kernel revisions, for one reason or another, but with a desire to have uniform kernel parameters like console speed and timeouts. With update-grub, you can just copy your menu.lst template to all of your systems and run update-grub afterwards, and update-grub will apply your new defaults and generate a new kernel list using them. However, with Ubuntu's update-grub, it generates the initial menu.lst and kernel list, and then when you replace that with a fresh template, ucf sees that you've removed the kernel list. It thinks that's a local modification, and when update-grub generates a new kernel list, it "preserves" your modification and leaves the kernel list out of the new menu.lst. Tada, instant unbootable system. Even worse, this bug is sticky: you can re-run update-grub all you want, it succeeds without errors, and it keeps generating an empty menu.lst. This is a completely mystifying error until you dig into the details of what's going on. To recover, you have to run ucf --purge /var/run/grub/menu.lst (and that path, or for that matter any of the ucf stuff, is completely undocumented in the update-grub man page) and then re-run update-grub, at which point ucf will display a debconf prompt (which doesn't support debconf preseeding, but that's another problem). If you then select three-way merge (not the default) from that prompt, you get a correct menu.lst. I'm not sure why Ubuntu made this change, but as near as I can tell, it's fundamentally architecturally broken. The specific problem I had was that we were copying our template over and re-running update-grub in a post-install script, resulting in a system that always had an empty kernel menu and wouldn't boot. Once I finally figured out what's going on, I worked around it by being very careful to put our template in place before the initial system bootstrap and then never changing menu.lst afterwards, but this is a bad limitation for a large site that scales system administrators by automating configuration file updates. But maybe I'm missing some subtlety. I'll go file a Launchpad bug shortly after posting this and we'll see what happens.

Knowing many people in the Debian community, I knew others would be there too. And Gunnar confirmed it yesterday via Planet Debian. Yes, I participated in the World Naked Bike Ride, although at the time I didn't know about the campaign, I thought people were doing it just because we can mostly. So, after being invited to the ride the week before, 5 of our colla were in the old T ria's river bed, near the Fira Alternativa's scenary at the scheduled time. Not having done anything like this before, we were a bit expectant to see how many people would actually do it, before deciding to join them. After a while, more and more people seemed to gather, and it finally took off. Pants off, and there we go! The insane amount of photographing and filming that was going on around us at the beginning was a bit uncomfortable, but after a while we had mostly forgotten we were riding our bikes naked through the commercial arteries of Val ncia. The ride was too long for my taste, covering the whole Fira, Pla a d'Am rica, carrer Colom just in front of El Corte Ingl s (I wonder how many known people saw me there), X tiva, Russafa, back to X tiva, Town Hall, Ciutat Vella, river margin and Blasco Ib ez. Even if it was a sunny day, the chilly wind made me feel really cold, but overall it was quite fun, and an interesting experience I might or might not repeat. I'm certainly not becoming a naturism activist or anything like this. I do think we have way too many taboos, and every time I get rid of one, I feel a lot better. :)

A common myth in the computer industry seems to be that ECC (Error Correcting Code - a Hamming Code [0]) RAM is only a server feature. The difference between a server and a desktop machine (in terms of utility) is that a server performs tasks for many people while a desktop machine only performs tasks for one person. Therefore when purchasing a desktop machine you can decide how much you are willing to spend for the safety and continuity of your work. For a server it’s more difficult as everyone has a different idea of how reliable a server should be in terms of uptime and in terms of data security. When running a server for a business there is the additional issue of customer confidence. If a server goes down occasionally customers start wondering what else might be wrong and considering whether they should trust their credit card details to the online ordering system. So it is obviously apparent that servers need a different degree of reliability - and it’s easy to justify spending the money. Desktop machines also need reliability, more so than most people expect. In a business when a desktop machine crashes it wastes employee time. If a crash wastes an hour (which is not unlikely given that previously saved work may need to be re-checked) then it can easily cost the business $100 (the value of the other work that the employee might have done). Two such crashes per week could cost the business as much as $8000. The price difference between a typical desktop machine and a low-end workstation (or deskside server) is considerably less than that (when I investigated the prices almost a year ago desktop machines with server features ranged in price from $800 to $2400 [1]). Some machines in a home environment need significant reliability. For example when students are completing high-school their assignments have a lot of time invested in them. Losing an assignment due to a computer problem shortly before it’s due in could impact their ability to get a place in the university course that they most desire! Then there is also data which is irreplaceable, one example I heard of was of a woman who’s computer had a factory pre-load of Windows, during a storm the machine rebooted and reinstalled itself to the factory defaults - wiping several years of baby photos… In both cases better backups would mostly solve the problem. For business use the common scenario is to have file servers storing all data and have very little data stored on the PC (ideally have no data on the PC). In this case a disk error would not lose any data (unless the swap space was corrupted and something important was paged out when the disk failed). For home use the backup requirements are quite small. If a student is working on an important assignment then they can back it up to removable media whenever they reach a milestone. Probably the best protection against disk errors destroying assignments would be a bulk purchase of USB flash storage sticks. Disk errors are usually easy to detect. Most errors are in the form of data which can not be read back, when that happens the OS will give an error message to the user explaining what happened. Then if you have good backups you revert to them and hope that you didn’t lose too much work in the mean-time (you also hope that your backups are actually readable - but that’s another issue). The less common errors are lost-writes - where the OS writes data to disk but the disk doesn’t store it. This is a little more difficult to discover as the drive will return bad data (maybe an old version of the file data or maybe data from a different file) and claim it to be good. The general idea nowadays is that a filesystem should check the consistency of the data it returns. Two new filesystems, ZFS from Sun [2] and BTRFS from Oracle [3] implement checksums of data stored on disk. ZFS is apparently production ready while BTRFS is apparently not nearly ready. I expect that from now on whenever anyone designs a filesystem for anything but the smallest machines (EG PDAs and phones) they will include data integrity mechanisms in the design. I believe that one such features become commonly used the need for RAID on low-end systems will dramatically decrease. A combination of good backups and knowing when your live data is corrupted will often be a good substitute for preserving the integrity of the live data. Not that RAID will necessarily protect your data - with most RAID configurations if a hard disk returns bad data and claims it to be good (the case of lost writes) then the system will not read data from other disks for checksum validation and the bad data will be accepted. It’s easy to compute checksums of important files and verify them later. One simple way of doing so is to compress the files, every file compression program that I’ve seen has some degree of error detection. Now the real problem with RAM which lacks ECC is that it can lose data without the user knowing. There is no possibility of software checks because any software which checks for data integrity could itself be mislead by memory errors. I once had a machine which experienced filesystem corruption on occasion, eventually I discovered that it had a memory error (memtest86+ reported a problem). I will never know whether some data was corrupted on disk because of this. Sifting through a large amount of stored data for some files which may have been corrupted due to memory errors is almost impossible. Especially when there was a period of weeks of unreliable operation of the machine in question. Checking the integrity of file data by using the verify option of a file compression utility, fsck on a filesystem that stores checksums on data, or any of the other methods is not difficult. I have a lot of important data on machines that don’t have ECC. One reason is that machines which have ECC cost more and have other trade-offs (more expensive parts, more noise, more electricity use, and the small supply makes it difficult to get good deals). Another is that there appear to be no laptops which support ECC (I use a laptop for most of my work). On the other hand RAID is very cheap and simple to implement, just buy a second hard disk and install software RAID - I think that all modern OSs support RAID as a standard installation option. So in spite of the fact that RAID does less good than a combination of ECC RAM and good backups (which are necessary even if you have RAID), it’s going to remain more popular in high-end desktop systems for a long time. The next development that seems interesting is the large portion of the PC market which is designed not to have the space for more than one hard disk. Such compact machines (known as Small Form Factor or SFF) could easily be designed to support ECC RAM. Hopefully the PC companies will add reliability features in one area while removing them in another.

• [0] http://en.wikipedia.org/wiki/Hamming_code


• [1] http://etbe.coker.com.au/2007/08/01/ecc-ram-in-a-cheap-machine/


• [2] http://en.wikipedia.org/wiki/ZFS


• [3] http://en.wikipedia.org/wiki/Btrfs

Share This

It’s time for recommending five films again. I’ve also decided that I may reserve the last of the five for films I watched prior to starting these series. So here we go:

• Au revoir les enfants (Goodbye, Children): just watch it, really. (France, World War II, children in a school run by priests, but there’s a part I’m not telling you.)
• El bosque animado: a superb film that every Spanish person should watch. Magic comedy & drama set up in rural Galicia. Amazing Tito Valverde.
• eXistenZ: yet another one by David Cronenberg, involving futuristic video games this time. Quite a good see, but predictable ending?
• Billy Elliot: the kid who wants to dance ballet. Predictive and linear, but I just loved it, as I expected — maybe you will too. The scene where Billy says goodbye to Michael just completely and utterly broke my heart, probably because I identified so much with Michael.
• Now, Voyager: Bette Davis plays an ugly duckling, that obviously transforms into an awesome swan. Superb. Oh, and Bette Davis.

I'm a bit slow on the journal-writing front, but belatedly I should mention here that I did upload the new Policy (3.8.0.0) and then a quick bug-fix. If you have a trigger-enabled dpkg, you don't see that you're registering non-existent doc-base files. Lintian should catch that, but doesn't. Yet. I've been trying to find time to kick off the next round of Policy updates. There are several proposals that just need text written for them: #143941 and #241333 are obvious at this point, #416450 should be fairly simple, and I think the due diligence has been done on #470994. There are other fairly straightforward ones too. If anyone beats me to it for any of those (or even better, other bugs), I won't complain. I also closed all of the old Policy bugs that we'd tagged wontfix except for one that I forwarded to the tech-ctte. I also uploaded Lintian 1.24.0 on Saturday. This contains the first steps in a better documented and hopefully faster and cleaner way of extracting information from the lab. Rather than opening and reading files in every check that need data, there are now Lintian::Collect modules that can hold common code and cache the results across multiple different check scripts. Currently, they're not widely used, but they're now available to all checks (hence the version bump). This release also contains some truly fantastic work by Jord Polo Bard s on the web site. Look at lintian.debian.org now. It looks like a real web site! Jord is working on a Google Summer of Code project for Lintian, but it wasn't even the web pages; that was just a bonus. I'm very happy with the outcome. Lintian is still above 100 bugs and I want to start expanding the new infrastructure, so there's more work to be done, but I'm very happy with the progress lately. And it was fun to spend two days solid working on Policy and Lintian for a change, instead of lots of different work projects (or feeling low on energy and barely managing to get a handful of things done).

Lev t t pour tre frais et dispo pour la premi re s ance, je me suis pr par rapidement. La nuit a t assez courte, couch vers minuit pour finaliser le support de cours. Enfin, il est pr t. J'ai juste quelques ajustements y apporter et il partira pour l'impression. Mieux vaut tard que jamais ;) J' tais donc pr t 7h30 pour un petit d jeuner ma foi agr able au bord de la piscine et accompagn de Tisserin et de la superbe Grue Couronn e quelques dizaines de centim tres de moi ;) Un petit tour au coin wifi internet pour envoyer 2 ou 3 mails et me voil attendre. Tout compte fait, j'ai d attendre jusqu' 10h30 pour que l'on vienne me chercher. La salle n' tait pas disponible, les petits tracas administratifs de derni re minute. Une fois sur place l'ENA (Ecole Nationale de l'Administration) je me suis lanc pr senter le syst me 25 personnes. Mais impossible de faire fonctionner loe r troprojecteur mis ma disposition. Pb de configuration Xorg voir ce soir. Donc j'ai projet mes transparent fait avec MagicPoint g n r s en HTML sur une machine sous Windows. Apr s une br ve introduction nous voil partis dans le vif du sujet. Les stagiaires taient tout ou e et donc je n'ai pas m nag les informations dispens es. Repas 14h frugal pour ma part, je n'avais pas trop faim. Poulet en sauve, crudit s, riz, pommes de terre. Me voil reparti 15h pour la deuxi me partie de la journ e. J'ai demand aux stagiaires de se pr senter... Bon, il va falloir que je fasse les bases des syst mes Unix pour tout le monde sans exception. Seance fini vers 18h o je suis rentr l'h tel, seul. Donc au programme de la soir e, remplir mon billet quotidien, r cuper la conf Xorg me permettant d'utiliser le r troprojecteur, manger et ensuite rododo ;)