Since the last time I went looking for backup software, I've still be using rdiff-backup.

It's nice, except for one thing: it always keeps an uncompressed copy of your current state on the disk. This is becoming increasingly annoying.

I did some tests with dar and BackupPC, and both saved considerable disk space over rdiff-backup. The problem with dar, or compressed full/incrementals with tar, is that eventually you have to make a new full backup. You have to do that, *then* delete all your old fulls and incrementals, so there will be times when you have to store a full backup twice.

The hardlinking approach sounds good. It's got a few problems, too. One is that it can lose metadata about, ironically enough, hard links. Another is that few of the hard linking programs offer a compressed on-disk format. Here's what I've been looking at:

BackupPC

Nice on the service. I'm a bit annoyed that it's web-driven rather than commandline-driven, but I can look past that. I can also look past that it won't let me clamp down on ssh access as much as I'd like.

BackupPC writes metadata to disk alongside files, so it can restore hard links, symlinks, device entries, and the like. It also has the nice feature of being able to hard link identical files across machines, so if you're backing up /usr on a bunch of machines and have the same files installed, you save space. Nice.

BackupPC also can compress the files on your disk. It uses pre-compression md5sums for identifying files to hard link, which is nice.

Here's where I get nervous.

BackupPC doesn't just use regular compression, from say gzip or bzip2. It uses its own low-level algorithm centered around the Perl deflate library. And it does it in a nonstandard way owing to a supposed memory issue with zlib. Why they don't just pipe it through gzip or equivalent is beyond me.

This means that, first off, it's using a nonstandard compression format, which makes me nervous to begin with. If that weren't annoying enough, you have to install Perl plus a bunch of modules to extract the thing. This makes me nervous too.

Dirvish

Doesn't support compression.

faubackup

Doesn't support compression.

rdup

Supports compression and encryption. Does not preserve ownership of things unless the destination filesystem does (meaning you must run as root to store your backups.)

Killer lack of feature: it does not preserve knowledge about what was hardlinked on the source system, so when you restore your backup, all hardlinks are lost. Epic fail.

rsnapshot

Doesn't support compression.

StoreBackup

Does support compression, appears to restore metadata in a sane way. Supports backing up to a different machine on the LAN, but only if you set up NFS. Looks inappropriate for doing backups over VPN. Comprehensive, though confusing, manual. Looks like an oddball design with an oddball manual.

So, any suggestions?

A lot has been said about the elections already. I just want to share my personal experiences about it. During the weekend, Tom invited us to his place in lower Harlem (we live upper Harlem) on 123rd St and Adam Clayton Powell Jr Ave, for Tuesday night. As Tuesday arrived, we didn’t see too many people too excited about it. Living in Harlem, we were pretty used to black people supporting Obama, so getting closer and closer to the elections, we didn’t feel people anxious. On Tuesday, we commuted to work and some of our co-workers got to work late because they have been standing in lines for a couple of hours to vote in polls. The whole day was about talking on how much that day was representing for the US and the world, guessing how they results were going to end up, etc. Later at night, Greg, Phil, Raquel and I headed to Tom’s place for his watch-the-results party. His fridge was full of Blue Moon beer, but he also had some Red Stripe Since I like to go light on politics, I preferred some other American variantion, Coors Light Vermont, Connecticut and a few other New England states were claimed by the Democrats. Matty, a friend of Tom attending the party brought US maps with the electoral votes on each one for people to color them as they were called. It was pretty fun. At some point we forgot about the maps since it was pretty clear that Obama was gonna win. But when they called him at around 11 pm (and I was starting to feel nicely drunk), the whole neighborhood and people in the streets just went completely nuts. It’s something I hadn’t experienced at all in my life, it was like if Mexico would have won the f tbol World Cup (which yes, it will never happen anytime soon :P), but nicer. People went out to the Harlem State Office Building square were cars were honking, and a huge amount of people gathered to celebrate, play brass music, cheer “Yes We Can”. Ghetto guys, white hippies, curious latinos, Africans, African Americans, Puerto Ricans, just a lot, a lot of people, all kinds of them, just a clusterfuck. They projected Obama’s speech from Chicago and people went nuts. It was just, fucking amazing. The images they projected from Harlem to the outside world was in that same square we were in. Our Flickr set for the celebration here.

The Open Source Repository and Observatory (OSOR), a new site sponsored by the European Commission to foster the exchange of FOSS related information and software among European public administrations, recently published guidelines on the procurement of open source software. Public administrations in Europe have to follow public tender procedures and the new guidelines give practical and legal advice on how open source software and related services can be incorporated into the procurement process. Rishab Ghosh, who presented the guidelines at the Open Source World Conference in Malaga, argued that the procurement guidelines were needed because of two reasons. First, they studied recent tenders and found that many explicitly mentioned proprietary applications. 16% of 3615 software tenders explicitly asked for products from top 10 software vendors, such as Microsoft, SAP and Oracle. This practice may be illegal because public tenders generally have to describe functional requirements in a general way instead of specifying specific products. Second, many public administrations don't have any experience with the procurement of FOSS. In fact, they often don't know whether or under which circumstances they are allowed to adopt and ask for FOSS solutions. The guidelines are specifically designed in order to clearly and simply explain how public administrations can acquire open source and they don't assume that a country has adopted a specific policy regarding open source. The guidelines include a long section about open standards, open source and how they relate. Both open standards and open source align very well with the needs of public administrations who have an "obligation to support interoperability, transparency and flexibility, as well as economical use of public funds". The guidelines argue that the exit cost, i.e. the cost incurred in moving to another IT system, is also an important consideration but one that is often neglected. The adoption of a proprietary solution without open standards may limit the future choice, thereby increasing the long-time costs and giving the proprietary vendor an unfair advantage in future tenders. The procurement guidelines describe two ways of acquiring FOSS: it is possible to go the usual route of publishing a tender for the supply of software (possibly with related services). However, in the case of FOSS, it is also possible to download the software directly from the Internet. This is possible because the software is not only free of charge but comes with no contractual obligations. If there were any obligations involved with the download (such as fees, the agreement to an EULA or the requirement to purchase services in the future), software download is not an allowed method. What I like about the guidelines is that they explicitly say that downloading software has to be part of the formal procurement process. You have to think about your requirements, look at various alternatives, and so on, and not just blindly download something from the Internet. When it comes to the procurement of FOSS, the guidelines don't suggest that tenders should explicitly ask for FOSS. Instead, they should describe the functional requirements of the software as well as certain properties. For example, a tender could specify that the public administration as well as third parties must have the right to study, distribute and modify the software. In a sense, the guidelines suggest that tenders should include the principles of the Free Software Definition along with justifications for these requirements. Personally, I think there is a great need for these procurement guidelines. There are many public administrations that don't know how to acquire FOSS and these guidelines offer clear advice. Furthermore, I find the guidelines very balanced. They don't recommend that you should always ask for FOSS but incorporate FOSS principles into tenders where it makes sense. It remains to be seen whether the procurement guidelines will have an impact on the FOSS adoption in Europe, but I surely hope so. (Originally published on FOSSBazaar)

Hey Romain, you d be impressed by the number of upstreams whose knowledge of security measures is similarly close to the absolute zero. Since bad guys are always digging through commits for suspicious things, hiding the relevant commits is the best thing you can do to help them. Given things like this exist, It s likely that they have even automated tools to do this search for them. This way, distributors and security engineers have to do the same job, hoping to never miss a change that was actually a security fix. Unfortunately, two small scale projects that we happen to use have exactly the policy Romain described regarding security updates. I m amazed by the fantastic job the distributions security teams manage to achieve in these conditions.

OK, I know I've been very quiet on the blog for weeks now but I have my reasons and all will become clear, eventually. I'm principally waiting for the Lenny release before being able to start full discussions about certain ideas that have been consuming a lot of my time recently. I continue to do what I can to get Lenny out of the door because the wait is driving me nuts. (Fixed another RC bug yesterday, yippee!)

Anyway, something I can talk about is Emdebian Grip. Emdebian has always wanted to support "in-between" devices that don't need the full-on cross-build everything approach of the current Emdebian packages for ARM but don't quite have room for a full Debian install either. Until I had at least a basic package set available via the cross-building system, I didn't want to start working on an intermediate system, for reasons of simplicity mainly, but also because I knew that getting a cross-building package set would involve changes to the tools needed for the intermediary set.

Bah, I'm waffling again. For the background, see Emdebian, Squeeze, Grip and Crush.

The good news is that Emdebian Grip, once running, will take far, far less time (for everyone) than the full-on cross-building malarkey of Emdebian Crush. Grip can be 99.9% automated by breaking Debian into discrete package sets or 'components'.
Componentising Debian for Emdebian
Emdebian Grip is about a smaller Debian, nothing more. No changes in functionality, no changes in dependencies; only changes in final installation size and Packages.gz size. It means using the code from dpkg-cross to unpack packages, remove unwanted files and repack the .deb. Yes, yes, this can be done one package at a time at a trivial level. Emdebian Grip seeks to bring a standard method for this operation that will work for all packages on any architecture for any architecture from any suite, creating and maintaining repositories of "gripped" packages for use in devices like my Acer Aspire1 where you just don't need documentation, manpages and the rest. Squeeze is the obvious target release but any release supported by the current version of dpkg could be "gripped" too, like oldstable.

The componentisation comes in at the level of the Packages.gz file. If the device cannot realistically be expected to run CAD software or has no fancy graphics or other power-hungry hardware, why should packages that need such resources clutter up the apt-cache list? This data isn't just slowing down searching for packages you do want, it slows down every installation process (that bit about 'Reading database . . . . . '), it slows down the process of getting the updated cache data in the first place because there is more of it, it wastes space on your device in several different places when the whole point of Emdebian is that disc space is not cheap and it wastes your precious battery power if you need to do simple package management stuff away from the mains. What is needed is a mini-repository, a mini-mirror that just contains the parts of Debian that your machine needs - a "well-trimmed" mirror with sources and all dependencies satisfied, a mirror that can be automatically updated but which doesn't keep getting NEW packages that you won't use. New dependencies will be added, yes. New packages unrelated to what you currently use? Absolutely not. You could even keep a mini-mirror on your home network, updating and "gripping" while you sleep, because a mini-mirror should be a lot, lot smaller than a standard Debian mirror.

I now have a script that will do the work, what I've lacked is the idea on how to "feed" packages to the Gripper.

So, I'm going to be exploring this method:

1. An arbitrary package set - say the output of 'dpkg -l' from the target machine setup by an expert, or 'dpkg --get-selections' or just the packages from a normal debootstrap - the basic idea is to start with only the packages you absolutely must have for your target device.


2. A very small wrapper for some debootstrap code that downloads those .debs from the Debian mirrors - debootstrap can cope with choosing the mirror, the architecture, the suite etc. (already written, landing in Emdebian SVN soon).


3. Parse the list of packages (.debs) and use 'apt-cache showsrc' to convert into a list of source packages. (Script done).


4. 'apt-get source' to match the source package to the binary. At this point, we have what amounts to a partial mirror but with very well defined limits. There are other ways of doing the same thing so anything that gets you to this point is as equal as the steps above, as long as the .debs exist alongside the .dsc and all associated source package files. What you need is full control over the list of packages at this stage, we'll worry about dependencies later.


5. Simple scripting parses the .dsc to create suitable debian/files content (done).


6. dpkg-genchanges regenerates the .changes file - critical for keeping the Gripped packages with the correct source packages in the final Grip repository


7. Now run 'emgrip' against the .changes file to convert the .debs, update the .changes file with the new, smaller, .deb details and copy everything into a suitable incoming/ directory.


8. point your repository software (tested with reprepro) at the incoming/ directory and generate your Gripped repository - which only contains the packages you listed at the start, just smaller. reprepro has a neat way of allowing foobar-1.2.3-2 to be included even if foobar-1.2.3-1 does not exist in the repository, as long as foobar_1.2.3.orig.tar.gz exists somewhere that reprepro can find and even though foobar-1.2.3-2.dsc does not mention the .orig.tar.gz. Neat! (Tested and working.)


9. Run 'edos-debcheck' against the repository and "catch-up" with whatever packages are needed to make the repository as a whole "trimmed" (no unresolved dependencies).


10. Keep the update script running as a cron task to pull updated packages from incoming.debian.org, run the .changes through emgrip and update your repository.


11. Clean out the .debs, leave the source packages and restart the process for another architecture. (Crafty!)


12. Possibly, to support those who can afford to do a small amount of compilation on their Emdebian Grip device, add a second repository that provides the Build-Depends that may be missing from the initial package set - stuff like automake and pkgconfig. Depending on how that works out, it might not be many packages and it may be easier to fold them into the mini-mirror anyway.


13. Instead of 20,000 packages, I'm expecting something closer to 2,000, per mini-mirror.


14. The process of "gripping" a package also drops numerous control fields (like Homepage and VCS*), as well as Recommends, Suggests and Enhances as well as truncating the long description at a maximum of four lines - all in the name of reduced package data sizes.


15. The end result is that each mini-mirror contains a component package set for Emdebian Grip, using Debian packages. Mini-mirrors are still "well-trimmed" (no unresolved dependencies) because you get to choose which packages you include and if a particular package tries to bring in both Gtk and KDE dependencies, you can choose to omit that package and use a different one.


16. Whether Emdebian Grip will or should support dropping certain binary packages from the repository is a difficult issue - Emdebian Crush is probably the preferred route if you need to start optimising an Emdebian Grip mini-mirror that harshly.


17. Another wrapper will be needed to clean out cruft (old dependencies, outdated packages etc.) - that one may be harder to achieve by automation.

The first Emdebian Grip repositories will be for ARM, armel and i386, based on an XFCE environment package set.

"Gripping" all the packages in my /var/cache/apt/archives (about 1Gb) took about 30 minutes on a fast laptop that was doing other things at the time - the real packages will be "gripped" on the Emdebian server and for multiple architectures in sequence. It's certainly conceivable that any number of mini-mirrors can exist with any number of package sets (components), all with "gripped" packages. You can choose to "grip" .debs upon download from a normal Debian mirror but that wastes time if there is more than one device, it wastes space in /var/lib/apt/lists and /var/lib/dpkg/ and wastes bandwidth. For portable devices that you want to be using, not updating, I think that a mini-mirror is a worthwhile bonus.

Expect a 20% reduction in installation size (for more, you'll need the cross-building, functionality-modifying Emdebian Crush shebang) but a large decrease in the size of all the apt-cache data and dpkg status data. I'm hoping to push towards 30% reductions but most of these figures are guesswork right now.

And beyond the mini-mirrors? Debian-Installer, of course. I really need to set some time aside to prepare an Emdebian Grip installer using current D-I. Simple things like:

• Preferring ext2 instead of ext3 for solid state file systems,


• preferring XFCE to GNOME for the Desktop System,


• turning off Recommends before even the first package gets chosen - let alone installed,
  
• supporting machine:variant customisation scripting so that particular devices can be configured from first boot instead of having to rely on defaults or manual fixes via Wiki pages.

It really, really annoys me that I have to spend ages getting devices like the Acer Aspire1 working properly under Debian and that I have to take particular care during the installer to change the Debian default to something more sane for that kind of device. For Debian to be The Universal Operating System, the installer needs to be as smooth on an NSLU2* as on a standard desktop, a fast server and a cluster. This isn't just an architecture thing, there are big and small devices in all architectures, it needs to be a machine:variant thing. Mini-mirrors of single components are a start and part of the bigger answer is to support machine-specific configuration packages in these mini-mirrors. Packages that only exist in a specific mini-mirror and are specifically designed to setup a host of machine-specific settings, blacklist certain modules and enable others, set sensible settings in /etc/fstab and other places based on results from things like powertop, turning off features for some devices and turning them on for others. The whole point of such packages is that what they do would be insane on what would currently be considered a "typical Debian machine".

*(I'm not talking about an NSLU2 with an external hard drive here, I'm talking about fitting Emdebian Crush into the 6.4Mb of space on the NSLU2 itself so that external hard drives can be hot-plugged, as well as devices like the OpenMoko and iPAQs etc.)

Don't get me wrong, D-I does a fantastic job for the kinds of devices Debian currently supports but there are a host of other devices out there that need customised solutions where a mini-mirror and a machine-specific configuration package can remove huge amounts of hassle from the actual installation process. Emdebian Crush already uses machine:variant specific packages and the current Emdebian packages repository is, itself, a mini-mirror, albeit not well trimmed.

Let's make the most of "Debian Squeeze" - let's squeeze it onto more devices and make it even easier to get Debian onto anything.

This is the point where I should be writing something witty and intelligent about the election going on across the pond, but I can't think of anything worth saying. If you can, go and vote. Even if you think your side is going to win, go and bloody do it anyway.

Anyway. Today I achieved FDMI success; my code successfully registered my test cluster against the fabric switch it's connected to and reported various bits of information about itself. Hardly earth shattering but does at least indicate that I'm beginning to understand some details of Fibre Channel at the packet level and how to make our code send/receive them. Go 'bama me.

(FDMI is the Fabric Device Management Interface and a way for Fibre Channel devices to report things like their manufacturer, serial number, OS, HBA model and driver/firmware versions to a management controller on the switch fabric. It's not a complex protocol in any manner, but there's a distinct lack of clear Fibre Channel documentation around that's at a lower level than "So you want to hook up a SAN?")

Dear United States of America, Please don’t fuck this up for everyone. With loving regards, Tore Sinding Bekkedal

Our DPL has spoken to The Register about the developer status proposal and how we’ll handle it after the release of Lenny. Too bad he hasn’t told us anything about it on our mailing-lists yet. Maybe The Register could create a Debian category so we can more easily look for what our DPL says? (If anyone asks, I’m a Reg reader myself)

Tomorrow I get the joy of watching an election go by that I can't participate in. I think Sen. Obama doesn't need one more voice from me at this point, but let's look at the California propositions:

Prop 1A - High speed rail. California really needs something like this. In fact, the whole US does, and maybe they'll get it once California shows that trains really can work on this side of the Atlantic.

Prop 2 - Increasing the sizes of cages for confined farm animals. I admit that I actually don't much care about this one either. I'd probably vote yet on it, but a cage is still a cage. =(

Prop 3 - Funding Children's hospitals. Uhhh. Why does this take a ballot measure? I'd do this one. Especially if it were seen as a path to the socialisation of health care.

Prop 4 - Parental notification before termination of pregnancy. Sounds like a great way to have more teen moms dead in back alleys. People either get illegal abortions or legal ones. Legal ones are much safer. This is a no-brainer for a no. If you think the incidence rate of abortions in teens is too high, fund some sex ed.

Prop 5 - Drug treatment facilities, good. Easier parole for drug-related offenses probably also good. Increased parole for serious and violent felonies? It's not clear to me that there's an actual problem here to solve. Is there really a problem with repeat offenders here that is going to be solved by this? I'd probably skip this one because there just isn't enough info.

Prop 6 - A bunch of gang related funding and such. It seems to me that if the legislature thought this was a real problem, they'd have solved it. The US already spends a stupid amount on policing, with the result of a lot of people in jail, and seemingly not alot of reduction in crime. Sounds like throwing good money after bad, then. I'd say no.

Prop 7 - Require utilities to focus on renewable energy. My electric company is already ~50% renewable according to their pamphlets. They also already are introducing smart meters, remote power cuts for air conditioning units, and have an offset program that isn't just cutting down trees. From what I understand, getting more than what they do is problematic because of reliability of supply. I don't think legislation will solve that. This legislation is probably too soon, sadly. A reluctant no from me here.

Prop 8 - A bill to remove existing rights from citizens. If you don't know my opinion on this one, you're clearly just tuning in. "no". It did occur to me today that what people should've been doing was trying to muddy the issue and put up billboards that said "In Defense of Marriage, No on prop 8". Because I really think that a yes here lessens the value of my marriage - it means that I'll be sharing in something I can't in turn wish for everyone else. And that would be sad.

Prop 9 - Victims rights bill. I wish there were more background on this site. It's not clear that there's a problem here that people are solving. A criminal sentence is supposed to be society's version of just punishment. Just because you personally still feel like crap doesn't mean that the punishment wasn't served. I don't see what relationship a person getting out of jail has to the victim at that point either than staying away from one another. No here.

Prop 10 - Funding for alternative fuels. I'm thinking back to "Who killed the electric car?" We're not doing what we can with the technology we already have. It would've been nice to perhaps see the old requirements kicked by in of a certain amount of cars having to be zero-emission or something like that now that we *know* we can do it. There are lots of electric car options now. I'd say no here.

Prop 11 - Redistricting. Nice to see that this includes people who belong to neither party in the requirements makeup. Taking the districts out of the hands of the representatives is probably a good idea. I hope the committee will opt to do something sane like simple cluster diagrams or something. I was talking with some folks at work about polling station finding and stuff, and it sounds really miserable.

Prop 12 - Veteran's bonds. Umm, sure? Are vets particularly hard hit by the current economic crisis? It seems odd to single them out. But the US has a culture of celebrating veterans that I've never understood. At home I suspect most people just don't know any. As opposed to a pile of their schoolmates *being* veterans.

Wow. That was exhausting. Do you guys have to do this every time you go to the polls?

I’ve flirted with it in the past and resisted it for a while, but you can now follow me on Identi.ca and Twitter if you are so inclined. Seems like I might have picked a good time to try this because TwiTip just soft-launched. There’s some overlap between my accounts, but they’re not identical because different third-party sites connect to each one. Both will get told when a new article appears on this website, but Twitter is likely to see more links to other sites and Identi.ca will get more status updates. Other than that, usage will be determined by popularity at events or with communities - or by suggestions here. How would you use them? At first glance, the two services are similar but different. Both sites seem to be “cooperative deserts” (not many cooperative members on them), both sites seem to be running on non-free software (I consider AGPL non-free until its big questions are resolved) and neither site seems to have an easy way to backup one’s data or participate without registration. Identi.ca seems to have some SEO-spammer problems, while Twitter’s interface has some silly needless javascript uses. Identi.ca supports OpenMicroBlogging (OMB) but I don’t understand that yet - it seems a terrible specification. It opens with references (but not web links) to three other things I’ve never heard of, then continues into a dictionary where terms like “listener” are redefined (ouch!) and doesn’t contain an examples section. New article announcements are done by a Wordpress plugin I’ll publish soon. Because of the above problems with OMB and that I can’t find whether Twitter supports it, the plugin uses the simpler status HTTP POST interface. Does anything other than Twitter and Identi.ca support that? I tried a few sites and got bored with them requiring me to apply for an API key before they’d tell me much.

I’ve flirted with it in the past and resisted it for a while, but you can now follow me on Identi.ca and Twitter if you are so inclined. Seems like I might have picked a good time to try this because TwiTip just soft-launched. There’s some overlap between my accounts, but they’re not identical because different third-party sites connect to each one. Both will get told when a new article appears on this website, but Twitter is likely to see more links to other sites and Identi.ca will get more status updates. Other than that, usage will be determined by popularity at events or with communities - or by suggestions here. How would you use them? At first glance, the two services are similar but different. Both sites seem to be “cooperative deserts” (not many cooperative members on them), both sites seem to be running on non-free software (I consider AGPL non-free until its big questions are resolved) and neither site seems to have an easy way to backup one’s data or participate without registration. Identi.ca seems to have some SEO-spammer problems, while Twitter’s interface has some silly needless javascript uses. Identi.ca supports OpenMicroBlogging (OMB) but I don’t understand that yet - it seems a terrible specification. It opens with references (but not web links) to three other things I’ve never heard of, then continues into a dictionary where terms like “listener” are redefined (ouch!) and doesn’t contain an examples section. New article announcements are done by a Wordpress plugin I’ll publish soon. Because of the above problems with OMB and that I can’t find whether Twitter supports it, the plugin uses the simpler status HTTP POST interface. Does anything other than Twitter and Identi.ca support that? I tried a few sites and got bored with them requiring me to apply for an API key before they’d tell me much.

Every Democrat was pretty sure Al Gore was going to win in 2000. Most polls said John Kerry was going to win in 2004. And yet both times they managed to snatch defeat from the jaws of victory. Just remember how you felt then and try your best so it doesn’t happen again. There is so much at stake … the US and the world can’t afford 4 more years of a Republican US President. Even if the GOP seems to be defeated, shattered and panicking, Republican voters always end up voting for their guy on election day, while Democrats have great problems closing the deal. It has to be different this time. We need Obama to win tomorrow, and we need him to win big. And a Democratic wave to get a filibuster-proof Senate majority and a comfortable victory in the House of Representatives, so they can enact a progressive agenda in the upcoming years and restore some of the damage from the last two Bush terms. We also need a solid ‘no’ to Proposition 8 in California and similar ones in other states, so no excuse to stay at your place just because you are in one of the bluest states. I would like to ask you to do your best tomorrow, but probably Larry Lessig does a better job at it. Once we are past this, the hard work starts, trying to hold every elected representative accountable, draft better candidates for future elections, spot DINOs, …, but for now, let’s focus on tomorrow. I am hoping for the best, and I wish I could do more about it, but it’s up to you Americans. Do the right thing and leave everything on the road, you don’t want to regret it tomorrow.

The National Council of Churches in the US [1] has produced some advice for Christian voters titled Christian Principles in an Election Year [2]. It starts by saying “Your church, your communion, and the National Council of Churches USA do not endorse any political party or any candidate” (which is in bold in their text) and then lists 10 issues that Christians should consider when voting. Here are the 10 section headings, the full article has a sentence or two explaining each one. I think that most good people (regardless of religion) will agree with all of this - maybe substitute “God” with the name of some other entity that has not been proven to exist or with “humanity”.

1. War is contrary to the will of God.


2. God calls us to live in communities shaped by peace and cooperation.


3. God created us for each other, and thus our security depends on the well being of our global neighbors.


4. God calls us to be advocates for those who are most vulnerable in our society.


5. Each human being is created in the image of God and is of infinite worth.


6. The earth belongs to God and is intrinsically good.


7. Christians have a biblical mandate to welcome strangers.


8. Those who follow Christ are called to heal the sick.


9. Because of the transforming power of God s grace, all humans are called to be in right relationship with each other.


10. Providing enriched learning environments for all of God s children is a moral imperative.

The Blogger Joy Reid [3] often uses the term “Matthew Christian” to refer to Christians who follow the book of Matthew and act in ways that would be considered to be good by most people regardless of belief. This is in stark contrast to some of the nasty people who call themselves Christian and who promote hatred, inequality, and war - such people give all Christians a bad reputation (see the comment section of any blog post concerning religion for examples). John Goerzen’s post titled Politics and the Church (which references the NCCUSA article [4]) is also worth reading. Interestingly his blog post had a Google advert for “Christian Masturbation” when I viewed it. John also has a good post explaning why he is voting for Obama - based on his Christian beliefs and “Traditional Values” [5].

• [1] http://www.ncccusa.org/


• [2] http://www.ncccusa.org/votingprinciples/


• [3] http://blog.reidreport.com/


• [4] http://changelog.complete.org/posts/771-Politics-and-the-Church.html


• [5] http://changelog.complete.org/posts/762-What-Traditional-Values-Mean-to-Me.html

Share This

sarge archived As already written a few days ago I was going to archive Sarge on archive.debian.org. That is done, you can get sarge from there now. The dists/sarge directory also already disappeared from the regular mirrors a few days ago. Im not simply deleting all source and binary packages from Sarge. Instead I am doing a removal of 1000 packages every second day, which ends up in batches of about 10000 removed files every other day. While that is a little more work on the archive side I think it pays off - many mirrors, including Debians own ones, limit the amount of files they delete in one mirror run and it is much less work than asking every mirror admin to make sure it will work on their system. Oh, just in case you mirror archive.debian.org somewhere: Make sure to use -H in your rsync call, will save about 30G of disk space.

---

Sarge AMD64, volatile and backports Now, while I was already working on archive.debian.org I thought it could do a little more than it did in the past. So I also went and put the Sarge AMD64 release on it, in a seperate directory. And as it would have felt lonely I put sarge-backports and sagre-volatile right beside it.

---

security and non-us non-us.debian.org is a service thats dead since the sarge release. Finally it also got archived on archive.debian.org (it helps that this service is, since a long time, no longer on a host in the US). security.debian.org now also appears on archive.debian.org, and the security archive will soon lose the files of old releases like slink, potato, woody and also sarge.

---

0.93R6, Rex and Buzz I got a few responses to my post about old releases and now we also provide 0.93R6. And to make it even better we now also have the binary packages for Rex and Buzz again. I still would like to see even older releases, including alpha and beta versions, so if you have such old things somewhere - tell me. Just make sure they somehow include the source too. I do have 0.91 for example, but it is missing any kind of source except for one package and one patchset. :(

---

Old list archives, old bugs On one of the CD images I thankfully got from J.H.M. Dassen I found another nice thing - old mailinglist archives from 1995 and 1996 as well as old bugs, #350 - #1626. The list- and bugpeople both got a tarball and just need to import them to make them available again. (Well, Don will kill me for that just, the bugs are html and textfile dumps, so he needs to parse them to get them in the right format for debbugs. But I’m sure he will manage. :) ). Comments: 2

First, let me make one thing clear. This isn't constructive criticism. This is just criticism. It's directed at software that's so wrong-headed that there's no way to make it significantly better, and everyone involved would be much better spending their time doing something else instead of trying to fix any of what I'm about to describe. It's not worth it. Sit in a park or something instead. Meet new and interesting people. Take up a hobby that doesn't involve writing shell scripts for Linux. You'll be happier. I'll be happier. Everyone wins.

Anyway. I wrote about Automatix some time ago. It died and the world became a better place. More recently it's been resurrected as something called Ultamatix. In summary, don't bother. It's crap. And dangerous. But mostly crap. Again, I'm going to utterly ignore the UI code and just concentrate on what it runs.

• function cleanup  
  echo "Cleaning up..."
  sudo apt-get autoremove --assume-yes --force-yes
   

  In other words, "Remove a bunch of packages that might have nothing to do with anything Ultamatix has installed, and don't ask the user first. Oh, and assume yes when asked whether to do anything potentially damaging". This gets called 103 times in various bits of Ultamatix.
  
  
• Oh, notice the sudo in there? Ultamatix is running as root already. Despite this, there are 429 separate calls to sudo.
  

• #Test O/S 64 or 32 bit...
  architecture= uname -m 
  targetarch="x86" #Set 64-bit machines to download 32-bit if no options are set
  if [ "$architecture" != "x86_64" ] && [ "$architecture" != "ia64" ]; then
      architecture="x86"
  else
      architecture="x86_64"
  fi

  It turns out that ia64 is not especially good at running x86_64 binaries. Never mind, eh?
  

• rm -rf $AXHOME/.gstreamer-0.10
  gst-inspect
  sudo gst-inspect
  

  Which translates as "Delete any self-installed plugins, run gst-inspect as root in an attempt to regenerate the plugin database, really run gst-inspect as root in an attempt to regenerate the plugin database". The flaws in this are left as an exercise for the reader.
  

• sudo apt-get --assume-yes --force-yes remove --purge

  Used 111 times. Will remove the packages it installed, but also any other packages the user has installed that happen to depend on them. Without asking.
  

• sudo cp /etc/apt/sources.list /etc/apt/sources.bak
  sudo echo "deb http://ppa.launchpad.net/project-neon/ubuntu hardy main" >> /etc/apt/sources.list
  sudo apt-get update
  if !    sudo apt-get install --assume-yes --force-yes amarok-nightly amarok-nightly-tools amarok-nightly-taglib
  then
          AX_fatal "An apt-based error occurred and installation was unsuccessful";
  fi
  echo "Restoring sources."
  sudo cp /etc/apt/sources.bak /etc/apt/sources.list
  sudo apt-get update

  The good news is that it backs up your sources.list before breaking things. The bad news is that it's still utterly horrifying.
  

• #since we have root we need to discover normal username so we can create the shortcut & set proper permissions
  NU=$(cat /etc/passwd   grep 1000   cut -d: -f1)
  sudo chown $NU:$NU "legends_linux-0.4.1.43.deb"
  sudo chmod +x legends_linux-0.4.1.43.deb
  sudo dpkg -i legends_linux-0.4.1.43.deb

  List of fail:
  1. Assuming that the user has uid 1000
  2. Chowning a deb to the user for no obvious reason (hint: a user can delete root owned files that are in the user's home directory)
  3. Making a deb executable for no reason whatsoever
  4. Assuming that user information will be in /etc/passwd
  5. Not just, say, passing the user's name to the application IN THE FIRST PLACE

• sudo apt-get --assume-yes --force-yes install f-spot dvgrab kino devede gtkpod-aac ipod gnupod-tools libgpod-common
  libipod-cil libipoddevice0 libipodui-cil libhfsp0 hfsplus hfsutils libipod0

  If only we had some way of saying that libraries used by programs should automatically be installed when a program is. Wouldn't that be great?
  

• echo "Adding mediabuntu repository"
  sudo cp /etc/apt/sources.list /etc/apt/sources.bak
  sudo wget http://www.medibuntu.org/sources.list.d/hardy.list -O /etc/apt/sources.list.d/medibuntu.list
  ...
  echo "Restoring sources."
  sudo cp /etc/apt/sources.bak /etc/apt/sources.list

  Yeah. that'll help.
  
  
• The Swiftweasel install that checks your CPU type and then has some insane number of cut and paste code chunks that differ only by the filename of the tarball it grabs. Rather than, say, using a variable and writing the code once.
  
  
• The cutting and pasting of the same code in order to install swiftdove.
  
  
• Code that installs packages differently depending on whether they happened to be in your home directory to start with or whether it had to download them for you
  

• if !    DEBIAN_FRONTEND=kde sudo apt-get --assume-yes --force-yesinstall virtualbox

  No, I didn't remove any spaces from that.
  

• #create directory incase they installed it elsewhere, no sense in scraping all thier games
  sudo mkdir /usr/local/games/WoP/ 2>/dev/null
  ...
  sudo rm -R /usr/local/games/WoP/ 2>/dev/null

  What, create a directory and then immediately delete it? How is this useful in any way whatsoever?

There's almost certainly more. I got bored. The worrying thing about this is that the Ultamatix author read my criticisms of Automatix and appears to have attempted to fix all of them. The problem with this is that there's clearly a complete lack of understanding of the fundamental problem in several cases. For example, one of my criticisms of Automatix:

sudo sed -i "s/^vboxusers\(.*\):$/vboxusers\1:$AXUSER/" /etc/group

- assumes that the system isn't using some sort of user directory service.

and the Ultamatix response:

Fixed...Got rid of Virtualbox

Except exactly the same problem is present at other points in Ultamatix, as noted above. Taking a bug list and slavishly fixing or deleting all the bugs isn't helpful if you then proceed to add the same bug back in 24 other places. In that respect, it's even worse than Automatix - the author's managed to produce a huge steaming pile of shite despite having been told how to avoid doing so beforehand. He may be no newbie to programming, but if not it's a perfect example of how experience doesn't imply competence.

Don't install this package. Don't let anyone else install this package. If you see anyone advocating the installation of this package, call them a fool. There's absolutely no excuse whatsoever for the existence of this kind of crap.

Minor update:
The above was looking at 1.8.0-4. It turns out that there's a 1.8.0-5 that's not linked off the website. There's no substantive difference, but some of the numbers may be slightly different.

One significant dividing factor between mailing lists is the difference between summary lists (where the person who asks a question receives replies off-list and then sends a summary to the list) and the majority of mailing lists which are discussion lists (where every reply goes to the list by default). I have seen an argument put forward that trusting the answers on a mailing list that operates under the summary list model is inherently risky and that peer review is required. It could be argued that the process of sending a summary to the list is the peer review. I’m sure that if someone posts a summary which includes some outrageously bad idea then there will be some commentary in response. Of course the down-side to this is that it takes a few days for responses to the question to arrive and as it’s common that computer problems need to be solved in hours not days the problem will be solved (one way or another) before the summary message is written. But the idea of peer review in mailing lists seems to fall down in many other ways. The first problem with the idea of peer review is that the usual aim of mailing lists is that most people will first ask google and only ask the list if a reasonable google search fails (probably most mailing lists would fall apart under the load of repeated questions otherwise). Therefore I expect that the majority of such problems to be solved by reading a web page (with no peer review that is easily accessible). Some of those web pages contain bad advice and part of the skill involved in solving any problem relates to recognising which advice to follow. Also it’s not uncommon for a question on a discussion list to result in a discussion with two or more radically different points of view being strongly supported. I think that as a general rule there is little benefit in asking for advice if you lack any ability to determine whether the advice is any good, and which of the possible pieces of good advice actually apply to your situation. Sometimes you can recognise good advice by the people who offer it, in a small community such as a mailing list it’s easy to recognise the people who have a history of offering reasonable advice. It seems that the main disadvantage of asking google when compared to asking a mailing list is that the google results will in most cases contain links to web sites written by people who you don’t know. Sometimes the advice is easy to assess, for example if someone recommends a little-known and badly documented command-line option for a utility it’s easy to read the man page and not overly difficult to read the source to discover whether it is a useful solution. Even testing a suggested solution is usually a viable option. Also it’s often the case that doing a google search on a recommended solution will be very informative (sometimes you see web pages saying “here’s something I tried which failed”). Recommendations based on personal experience are less reliable due to statistical issues (consider the the regular disagreements about the reliability of hard disks where some people claim that RAID is not necessary due to not having seen failures while others claim that RAID-5 is inadequate because it has failed them). There are also issues of different requirements, trivial issues such as the amount of money that can be spent will often determine which (if any) of the pieces of good advice can be adopted. The fact that a large number of people (possibly the majority of Internet users) regularly forward as fact rumors that are debunked by Snopes.com (the main site for debunking urban legends) seems to indicate that it is always going to be impossible to increase the quality of advice beyond a certain level. A significant portion of the people on the net are either unwilling to spend a small amount of effort in determining the accuracy of information that they send around or are so gullible that they believe such things beyond the possibility of doubt. Consider that the next time you ask for advice on a technical issue, you may receive a response from someone who forwarded a rumor that was debunked by Snopes. Sometimes technical advice is just inherently dangerous because it is impossible to verify the integrity of some code that is being shared, or because it may be based on different versions of software. In a previous blog post I analyse some issues related to security of the Amazon EC2 service [1]. While the EC2 service is great in many ways (and implements a good well-documented set of security features on the servers) the unsigned code for managing it and the old versions of the images that they offer to customers raise some serious issues that provide avenues for attack. Getting the EC2 management tools to work correctly on Debian is not trivial, I have released patches but will not release packages for legal reasons. It seems most likely to me that someone will release packages based on my patches (either because they don’t care about the legal issues or they have legal advice suggesting that such things are OK - maybe due to residing in a different jurisdiction). Then people who download such packages will have to determine whether they trust the person who built them. They may also have the issue of Amazon offering a newer version of the software than that which is packaged for Debian (for all I know Amazon released a new version yesterday). The term integrity when applied to computers refers to either accidental or malicious damage to data [2]. In the context of mailing list discussions this means both poorly considered advice and acts of malice (which when you consider spam and undisclosed conflicts of interest are actually quite common). If you ask for advice in any forum (and I use the term in it’s broadest sense to cover web “forums”, IRC, twitter, etc) then getting a useful result will depend on having the majority of members of the forum possessing sufficient integrity and skill, being able to recognise the people whose advice should be followed, or being able to recognise good advice on it’s own. I can think of few examples of forums of which I have been involved where the level of skill was sufficient to provide quality answers (and refutations for bad answers) for all areas of discussion that were on topic. People whose advice should generally be followed will often offer advice on areas where their skills are less well developed, someone whose advice can be blindly followed in regard to topic A may not be a reliable source for advice on topic B - which can cause confusion if the topics in question are closely related. Finally a fundamental difference between “peer review” (as applied to conferences and academic journals) is that review for conferences and journals is conducted before the presentation. Not only does the work have to be good enough to pass the review, but the people doing it will never be sure what the threshold is (and will generally want to do more than a minimal effort) so the quality will be quite high. While peer review in mailing lists is mostly based around the presence or absence of flames. A message which doesn’t attract flames will either have some minimal quality or be related to a topic that is not well known (so no-one regards it as being obviously wrong). Update: The “peer review” process of publishing a post on my blog revealed that I had incorrectly used who’s instead of whose.

• [1] http://etbe.coker.com.au/2008/10/13/ec2-security/


• [2] http://en.wikipedia.org/wiki/Data_integrity

Share This

Today everyone in the Netherlands received a letter about the new national electronic health record (EPD) and the possibility to object against registration. EPD aims to provide access to one's patient data to every care provider through a central information broker. I have submitted the form to disallow my data to be accessed through this system. First of all, there's no clear benefit for me, and I think that goes for the large majority of people. The possible situation where someone has a critical condition and isn't treated by his regular doctor and is unable to inform the stand-in of this and the stand-in has the time to delve through the entire EPD and actually finds and correctly interprets the necessary information seems extremely small for anyone, let alone the big majority that doesn't suffer such critical conditions in the first place. Hence, making it the default for everyone seems very inappropriate. See also this interesting article, written in Dutch by my uncle. Interestingly the same minister was recently opposed to a default-allow for organ donorship, which may address a problem that is much more real. The other concern is security. I am not worried by the technical security of the system, it seems to be of acceptable standard (see this report by my friend Niels). I am more concerned about access restrictions: these are implemented post hoc, that is, anyone can access my file and I can check who accessed my it and whether they had the right to. However, this procedure involves sending in paper forms which I think in practice will not bring about much review. Combined this project reminds me of voting computers - introducing new concerns while solving no actual problems.

I've been telling people that I plan to keep bicycling into the winter. I think about half the people I've told to don't really believe it. And the last two weeks, I haven't been able to ride to work due to muddy roads and scheduling conflicts. But today I did.

I left while it was still dark and 45 degrees outside (7C for the Fahrenheit-impaired). I think that's the coldest bike ride for me yet. I wore my regular shorts, shoes, and socks. I had my regular shirt on, plus a long-sleeve thicker shirt on above it.

It was cold the first few minutes, but this was pretty much the right outfit. Once I got going and got some heat built up, I was doing fine. In fact, I had to unzip my outer layer because I was getting hot.

So. I figure that if I can do 45 with shorts, then I ought to be able to take on 20 with proper winter gear.

Either that, or I've now dug myself a nice big hole if I wimp out.