If you walk up to the Swiss railways customer service desk and tell them that you're a group of six looking to travel from Zurich to Munich and would thus like to make a reservation, they'll charge you 15 CHF (~ 9.50 ) for a little piece of paper which documents that the guy behind the desk put his brain to use and reserved six contiguous seats to keep the group together. Well, he actually asked me whether that's what I wanted. If you walk up to the German railways customer service desk and tell them that you're a group of (now) eight looking to travel from Munich to Zurich and would thus like to make a reservation, they'll charge you 28 for a little piece of paper which documents that the guy behind the desk does not have a brain or didn't bother to switch it on: instead of two adjacent groups of four seats, we got four pairs of aisle seats. Given that we mainly reserved to minimise the annoyance for the other travellers while we kept up a conversation for the five hours of the journey, it did piss me off quite a bit to have paid three times as much in Germany as the reservation cost in Switzerland and not be able to talk among each other without annoying the others. This reservation was made two days in advance. You may, of course, think that the train was already full and no space to sit eight was left; however, I did not have to go further than two wagons to find copious amounts of unreserved spaces for eight, which were then, unfortunately, already occupied (of course). It shames me to expose my peers (I made the travel arrangements) to the crap and blather one has to put up in German railways: the quality of the speaker system is crap, everything is repeated in English by non-English-speaking conductors, and after every stop, we're yet again alerted to the fact that we could be relaxing to high-quality cuisine in the board restaurant, or should watch out for the little trolley passing by and happy to collect our money. Oh, and of course the brainless and often unfriendly service employees. And all that for a price more than it would have cost the lot of us to rent a van and drive back and forth. Going by train is my most preferable means of transport, not only, but in large part due to environmental issues. I don't understand why the German government puts this important bit of infrastructure in the hands of imbeciles and lets them overcharge for their low-quality service and their delays. NP: Mono: Under the Pipal Tree Update: A guy from Berlin wrote:

The seats you found in the other wagons may as well have been reserved, too. The electronic reservation signs will extinguish about 15 minutes after the train departs from the station. Or, there was too little time between reservation and departure, so you could only get one of the few express-reservation seats.

I am aware of this. However, the train between Zurich and Munich is ancient (part of the way it still has to run on Diesel) and the reservation signs were paper slips. The seats of eight I found were definitely unreserved, especially because I checked immediately after departure.

Training the Discriminators It has been a while since I posted on this category actually, it has been a long while since my last blog. When I last left you, I had mail (mbox format) folders called ham and/or junk, which were ready to be used for training either CRM114 or Spamassassin or both. Setting up Spamassassin This post lays the groundwork for the training, and details how things are set up. The first part is setting up Spamassassin. One of the things that bothered me about the default settings for Spamassassin was how swiftly Bayes information was expired; indeed, it seems really eager to dumb the Bayes information (don t they trust their engine?). I have spent some effort building a large corpus, and keeping ti clean, but Spamassassin would discard most of the information from the DB after training over my corpus, and the decrease in accuracy was palpable. To prevent this information from leeching away, I firstly increased the size of the database, and turned off automatic expiration, by putting the following lines into ~/.spamassassin/user_prefs:

bayes_expiry_max_db_size  4000000
bayes_auto_expire         0

I also have regularly updated spam rules from the spamassassin rules emporium to improve the efficiency of the rules; my current user_prefs is available as an example. Initial training I keep my Spam/Ham corpus under the directory /backup/classify/Done, in the subdirectories Ham and Spam. At the time of writing, I have approximately 20,000 mails in each of these subdirectories, for a total of 41,000+ emails. I have created a couple of scripts to train the discriminators from scratch using the extant Spam corpus; and these scripts are also used for re-learning, for instance, when I moved from a 32-bit machine to a 64-bit one, or when I change CRM114 discrimators. I generally run them from ~/.spamassassin/ and ~/var/lib/crm114 (which contains my CRM114 setup) directories. I have found that training Spamassassin works best if you alternate Spam and Ham message chunks; and this Spamassassin learning script delivers chunks of 50 messages for learning. With CRM114, I have discovered that it is not a good idea to stop learning based on the number of times the corpus has been gone over; since stopping before all messages i the Corpus are correctly handled is also disastrous. So I set the repeat count to a ridiculously high number, and tell mailtrainer to continue training until a streak larger than the sum of Spam and Ham messages has occurred. This CRM114 trainer script does the hob nicely; running it under screen is highly recommend. Routine updates Coming back to where we left off, we had mail (mbox format) folders called ham and/or junk sitting in the local mail delivery directory, which were ready to be used for training either CRM114 or Spamassassin or both. There are two scripts that help me automate the training. The first script, called mail-process, does most of the heavy listing. This processes a bunch of mail folders, which are supposed to contain mail which is either all ham or all spam, indicated by the command line arguments. We go looking though every mail, and any mail where either the CRM114 or the Spamassassin judgement was not what we expected, we strip out mail gathering headers, and then we save the mail, one to a file, and we train the approprite filter. This ensures that we only train on error, and it does not matter if we accidentally try to train on correctly classified mail, since that would be a no-op (apart from increasing the size of the corpus). The second script, called mproc is a convenience front-end; it just calls mail-process with the proper command line arguments, and feeds them the ham and junk in sequence; and takes no arguments. So, after human classification, just calling mproc does the classification. This pretty much finishes the series of posts I had in mind about spam filtering, I hope it has been useful.

The next release of DebTorrent is now available. This release includes new functionality for communicating with APT using a new transport method specifically designed for debtorrent, and many bug fixes. The major changes in this release are all in the communication between APT and DebTorrent. The HTTP server in DebTorrent that listens for APT requests has been upgraded to support HTTP/1.1 persistent connections and pipelining, which allows APT to have multiple outstanding requests for files. This is useful as DebTorrent suffers from the typical bittorrent slow start, so requesting multiple files at a time helps to speed up the download considerably. Though better, HTTP/1.1 is not ideal for DebTorrent however, as a maximum of 10 outstanding requests is maintained by APT's http method, and files must still be returned in the order they were requested (which is not ideal for bittorrent-type downloading since downloads occur randomly). To further improve the APT communication I have modified APT's http method to create a debtorrent method. This new debtorrent transport for APT is packaged separately as apt-transport-debtorrent, and once installed APT can be told to use it by replacing "http://" with "debtorrent://" in your sources.list file. This method sends all requests it receives immediately to DebTorrent, and will receive responses from DebTorrent in any order. You can find this new method on the Alioth project, or in my personal repository (only amd64 and i386 versions are currently available). Unfortunately, the story doesn't end here. The APT code responsible for sending requests to the method also limits the maximum number of outstanding requests that it will send to the method to 10, which is not really necessary since all existing methods limit the requests they send out themselves. I have therefore patched the current APT code to increase this limit to 1000 (a one line change), and released this patched version as 0.7.6-0.1. You can find this patched version in my personal repository (again, only for i386 and amd64). I have tested it with the other methods available and it causes no problems, and I hope to get the change included in the regular APT code soon. To sum up:

• new DebTorrent over HTTP = fast
• new DebTorrent with new apt-transport-debtorrent = faster
• new DebTorrent with new apt-transport-debtorrent and a patched APT = fastest

The last DebTorrent version (0.1.3.1) is currently in the NEW queue, and judging by the length of it, will be there for about another week. After DebTorrent is added to the archive, I will be upgrading it to this new version. I also hope to get the new apt-transport-debtorrent package into the NEW queue soon. This brings to an end the Google Summer of Code (which this project was created as a part of), but development of DebTorrent will of course continue (probably a little slower). The next major change will be the addition of unique piece numbers, which is almost complete but needs to be extensively tested. I'd like to thank Anthony Towns, Steve McIntyre, and Michael Vogt for their help over the last 4 months, and also the many others who sent me encouraging emails or engaged in interesting discussions about this project. It's the people who make a project like this a fun and memorable thing to do. Here's the changelog for the new DebTorrent release:

• APT communication supports HTTP/1.1 connections, including persistent connections and pipelining
• Add support for the new debtorrent APT transport method (see the new apt-transport-debtorrent package)
• Make the Packages decompression and torrent creation threaded
• Improve the startup initialization of files
• Add init and configuration files for the tracker
• bug fixes:
  • restarts would fail when downloaded files have been modified
  • deleting old cached data would fail
  • small tracker bug causing exceptions
  • prevent enabling files before the initialization is complete
  • only connect to unique peers from the tracker that are not already connected
  • tracker would return all torrents' peers for every request

The next release of DebTorrent is now available. This release includes new functionality for communicating with APT using a new transport method specifically designed for debtorrent, and many bug fixes. The major changes in this release are all in the communication between APT and DebTorrent. The HTTP server in DebTorrent that listens for APT requests has been upgraded to support HTTP/1.1 persistent connections and pipelining, which allows APT to have multiple outstanding requests for files. This is useful as DebTorrent suffers from the typical bittorrent slow start, so requesting multiple files at a time helps to speed up the download considerably. Though better, HTTP/1.1 is not ideal for DebTorrent however, as a maximum of 10 outstanding requests is maintained by APT's http method, and files must still be returned in the order they were requested (which is not ideal for bittorrent-type downloading since downloads occur randomly). To further improve the APT communication I have modified APT's http method to create a debtorrent method. This new debtorrent transport for APT is packaged separately as apt-transport-debtorrent, and once installed APT can be told to use it by replacing "http://" with "debtorrent://" in your sources.list file. This method sends all requests it receives immediately to DebTorrent, and will receive responses from DebTorrent in any order. You can find this new method on the Alioth project, or in my personal repository (only amd64 and i386 versions are currently available). Unfortunately, the story doesn't end here. The APT code responsible for sending requests to the method also limits the maximum number of outstanding requests that it will send to the method to 10, which is not really necessary since all existing methods limit the requests they send out themselves. I have therefore patched the current APT code to increase this limit to 1000 (a one line change), and released this patched version as 0.7.6-0.1. You can find this patched version in my personal repository (again, only for i386 and amd64). I have tested it with the other methods available and it causes no problems, and I hope to get the change included in the regular APT code soon. To sum up:

• new DebTorrent over HTTP = fast
• new DebTorrent with new apt-transport-debtorrent = faster
• new DebTorrent with new apt-transport-debtorrent and a patched APT = fastest

The last DebTorrent version (0.1.3.1) is currently in the NEW queue, and judging by the length of it, will be there for about another week. After DebTorrent is added to the archive, I will be upgrading it to this new version. I also hope to get the new apt-transport-debtorrent package into the NEW queue soon. This brings to an end the Google Summer of Code (which this project was created as a part of), but development of DebTorrent will of course continue (probably a little slower). The next major change will be the addition of unique piece numbers, which is almost complete but needs to be extensively tested. I'd like to thank Anthony Towns, Steve McIntyre, and Michael Vogt for their help over the last 4 months, and also the many others who sent me encouraging emails or engaged in interesting discussions about this project. It's the people who make a project like this a fun and memorable thing to do. Here's the changelog for the new DebTorrent release:

• APT communication supports HTTP/1.1 connections, including persistent connections and pipelining
• Add support for the new debtorrent APT transport method (see the new apt-transport-debtorrent package)
• Make the Packages decompression and torrent creation threaded
• Improve the startup initialization of files
• Add init and configuration files for the tracker
• bug fixes:
  • restarts would fail when downloaded files have been modified
  • deleting old cached data would fail
  • small tracker bug causing exceptions
  • prevent enabling files before the initialization is complete
  • only connect to unique peers from the tracker that are not already connected
  • tracker would return all torrents' peers for every request

One situation that you will occasionally encounter when running a Heartbeat cluster is a need to prevent a STONITH of a node. As documented in my previous post about testing STONITH the ability to STONITH nodes is very important in an operating cluster. However when the sys-admin is performing maintenance on the system or programmers are working on a development or test system it can be rather annoying. One example of where STONITH is undesired is when upgrading packages of software related to the cluster services. If during a package upgrade the data files and programs related to the OCF script are not synchronised (EG you have two programs that interact and upgrading one requires upgrading the other) at the moment that the status operation is run then an error may occur which may trigger a STONITH. Another possibility is that if using small systems for testing or development (EG running a cluster under Xen with minimal RAM assigned to each node) then a package upgrade may cause the system to thrash which might then cause a timeout of the status scripts (a problem I encounter when upgrading my Xen test instances that have 64M of RAM). If a STONITH occurs during the process of a package upgrade then you are likely to have consistency problems with the OS due to RPM and DPKG not correctly calling fsync(), this can cause the OCF scripts to always fail to run the status command which can cause an infinite loop of the cluster nodes in question being STONITHed. Incidentally the best way to test for this (given the problems of a STONITH sometimes losing log data) is to boot the node in question without Heartbeat running and then run the OCF status commands manually (I previously documented three ways of doing this). Of course the ideal (and recommended) way of solving this problem is to migrate all services from a node using the crm_resource program. But in a test or development situation you may forget to migrate all services or simply forget to run the migration before the package upgrade starts. In that case the best thing to do is to be able to remove the ability to call STONITH . For my testing I use Xen and have the nodes ssh to the Dom0 to call STONITH, so all I have to do to remove the STONITH ability is to stop the ssh daemon on the Dom0. For a more serious test network (EG using IPMI or an equivalent technology to perform a hardware STONITH as well as ssh for OS level STONITH on a private network) a viable option might be to shut down the switch port used for such operations - shutting down switch ports is not a nice thing to do, but to allow you to continue work on a development environment without hassle it’s a reasonable hack. When choosing your method of STONITH it’s probably worth considering what the possibilities are for temporarily disabling it - preferably without having to walk to the server room.

Share This

During my previous adventures with the Debian archive, I found that two packages in the archive use bzip2 compression inside the .deb instead of the traditional gzip compression, so I decided to try it out on emacs-snapshot (one of my larger packages). The combined size of the deb files goes from 36764KB to 33880KB, a 2884KB (7.8%) difference. It also makes both lintian and linda unhappy, the former gives me the following error:

E: emacs-snapshot-nox: deb-data-member-wrongly-compressed
N:
N:   The binary package contains a data member not compressed with gzip.
N:   From dpkg-dev 1.11 on, you can configure the way the data tarball is
N:   compressed. Though this is possible, you are not allowed to use it
N:   before dpkg 1.11 (or later) enters stable.

and linda just bombs:

E: emacs-snapshot-common; Package uses a newer feature of dpkg.
 This package uses a data.tar, or data.tar.bz2 member of the .deb. This
 was introduced in dpkg 1.11, but is not allowed to be used until dpkg
 1.11 or later hits stable.
File ...3_all.deb failed to process: Level 2 unpacking failed:
Could not unpack data tarball

Etch was released with dpkg 1.13.25, so bzip2 compression is probably allowed now. But is a 7.8% saving worth the incompatibility price? I'm not sure.

Growing up in Eugene, Oregon, which like Berkeley or Boulder could have the label "People's Republic of" applied to it, I always thought of "business friendly", as something along the lines of helping huge corporations avoid laws against pollution, or other antisocial behavior. Only after moving to Europe did I begin to get an idea of what the very positive side of "business friendly" is in the US. Truth be told, all countries tend to protect their 'big players' to some degree, be it the US propping up creaking airlines after September 11th, Italy finding various clever ways to get around rules about funding Fiat and Alitalia, or France finding it in their 'national interests' to discourage a potential bid for Danone (yogurt!) by PepsiCo. Some are better or worse (the UK has been pretty good about not interfering), but there is a tendency to want to intervene. Leaving be the discussion over whether those sorts of policies are good, bad, or ugly, the biggest difference between continental Europe and the US is the ease with which new companies - 'startups' can be created and enter a market. As a first hand example, I decided this summer to create DedaSys as a real company in order to better separate my business and personal financial dealings. Were I to do that in Italy or Austria, we would be talking about fees upwards of 3000 Euros (about 4000 dollars at market rates), which is a great deal of money for something that is not making a lot of it at this point in time. Contrast this with what it took to create a Limited Liability Company (LLC) in Oregon, my home state. To have things done up professionally, it's certainly possible to lay down a bit of cash there too, but by trading my time for money, and with the assistance of some nolo.com books about the creation and maintenance of an LLC, I was able to register DedaSys with the state of Oregon for the grand total of 55 dollars, and was actually able to complete the process remotely from Austria prior to going home on vacation, where I did the only thing I needed to do in person: open a bank account. So it costs 1% of what it costs in Italy or Austria to open a company that provides limited liability... a very impressive difference, especially to a small, new company that does not have the connections a Ford or a Fiat will likely have to enable it to deal with all the other paperwork, rules, and regulations to deal with. Add to that a culture of greater risk taking (meaning also more acceptance of failure), better funding opportunities, and... one comes to the conclusion that Paul Graham is right. It's a pity, because the people in Europe are top notch. In Italy alone, I know a bunch of really bright hackers. Granted, some of them aren't interested, and are probably better off not starting a company or otherwise dealing with the business side of the equation, but it's always nice to have that opportunity. In closing, here is another example of bureaucracy in action, from my personal on line journal about life in Italy, and now Austria, which I recently revamped by moving it to the Typo platform: Confronting the bureaucratic beast - registering an Italian domain Two months to accomplish what can be done in ten minutes with a .com!

Courtesy of an article in today's Guardian, I present Stone's Corollary to Burton's Low:

As a discussion about the contribution of vehicles to climate change grows longer, the probability of stating that China is putting 1,000 new cars on the road every week approaches one.

When used as a defence for driving an SUV, I'd also like to invoke the sudden death variant, where the discussion is finished immediately.

I was just reading the latest issue of Home Power magazine, which is celebrating 20 years of publication with this issue. One of the articles, by Kathleen Jarschke-Schultze, talks about the value of reusable cotton bags for shopping instead of accepting the usual "paper or plastic". It caught my eye because my wife Karen is an avid cotton-bag user. Kathleen ends the article with an idea that I think is worth repeating:

So how about this new etiquette: Conservation in everything except courtesy and generosity. Seems like that would be a refreshing change. We all need to just put one foot in front of the other to keep moving forward on conserving global resources. Even if it is slow or small, take that first step.

I like that. Conservation, courtesy, and generosity should never go out of style...

The great thing about the internet is that people can use it to make a fool of themselves in front of the whole world. Since this provides everyone with entertainment, I try and do something silly every once in a while. However, I'm easily topped by other people... Today's morons are these guys who have decided that ad blocking is evil, bad and wrong. Their argument is simple - that not looking at the ads would be stealing. They point out they can't block people who are using the Ad Blocker, so they have to block every firefox user. This is the MPAAs thinking - the user is the enemy. For me, all it means is that I can't visit these websites - I'll have to go spend money somewhere else...

ALISS have posted papers from their conference on electronic copyright, IPR and access issues in the emerging electronic landscape. PDFs are:- Librarians against plagiarism: how Imperial College London is using PRS and active learning to combat the cut and paste generation. Ruth Harrison and Julia Garthwaite. The practicalities of copyright in the online age. Helen Bartlett Copyright Manager, HERON. Copyright and data licensing, does electronic differ from print?' Richard Ebdon, Copyright Officer, The British Library. Rights and responsibilities: Copyright and digital images. Grant Young, Technical Research Officer, TASI - Technical Advisory Service for Images IPR and multimedia in institutional repositories: lessons from the MIDESS project (Management of Images in a Distributed Environment with Shared Services) Lesley Pitman Librarian and Director of Information Services, UCL SSEES Library. Irish Studies Online - a digital library of core resources for Irish studies. QUB. Norma Menabney, Queen University, Belfast. I've not finished reading them myself yet.

One situation that you will occasionally encounter when running a Heartbeat cluster is a need to prevent a STONITH of a node. As documented in my previous post about testing STONITH the ability to STONITH nodes is very important in an operating cluster. However when the sys-admin is performing maintenance on the system or programmers are working on a development or test system it can be rather annoying. One example of where STONITH is undesired is when upgrading packages of software related to the cluster services. If during a package upgrade the data files and programs related to the OCF script are not synchronised (EG you have two programs that interact and upgrading one requires upgrading the other) at the moment that the status operation is run then an error may occur which may trigger a STONITH. Another possibility is that if using small systems for testing or development (EG running a cluster under Xen with minimal RAM assigned to each node) then a package upgrade may cause the system to thrash which might then cause a timeout of the status scripts (a problem I encounter when upgrading my Xen test instances that have 64M of RAM). If a STONITH occurs during the process of a package upgrade then you are likely to have consistency problems with the OS due to RPM and DPKG not correctly calling fsync(), this can cause the OCF scripts to always fail to run the status command which can cause an infinite loop of the cluster nodes in question being STONITHed. Incidentally the best way to test for this (given the problems of a STONITH sometimes losing log data) is to boot the node in question without Heartbeat running and then run the OCF status commands manually (I previously documented three ways of doing this). Of course the ideal (and recommended) way of solving this problem is to migrate all services from a node using the crm_resource program. But in a test or development situation you may forget to migrate all services or simply forget to run the migration before the package upgrade starts. In that case the best thing to do is to be able to remove the ability to call STONITH . For my testing I use Xen and have the nodes ssh to the Dom0 to call STONITH, so all I have to do to remove the STONITH ability is to stop the ssh daemon on the Dom0. For a more serious test network (EG using IPMI or an equivalent technology to perform a hardware STONITH as well as ssh for OS level STONITH on a private network) a viable option might be to shut down the switch port used for such operations - shutting down switch ports is not a nice thing to do, but to allow you to continue work on a development environment without hassle it’s a reasonable hack. When choosing your method of STONITH it’s probably worth considering what the possibilities are for temporarily disabling it - preferably without having to walk to the server room.

Share This

Side note: According to the Skype RSS data feed, right now 174086 user are online. That less than 10% of the usual numbers, isn't it? To me, that's more than a few users.Skype is harmful. Here's the explantaion why:Skype uses P2P technology where it is not needed. There are parts of VoIP where it does make sense (when a direct connection is possible, transfer the data directly between clients), but pretty much any VoIP software does that.And there are parts, where it doesn't make sense. This includes not accessing the login servers directly, but trying to use other Skype users as proxies.Now what I've seen happening in some networks is the following: Skype fails to login directly, so it starts connecting random peers it has seen before. It keeps a list of 200 peers for that in the registry. So when it fails to connect, it will try contacting these 200 peers instead. Now if you have multiple users sharing your connection, let's say 10 and a cheap router with limited memory, 2000 connections is a lot to keep track of. It might start losing other connections. Such as your mail an web connections.This is exactly what I've seen today: the internet connection becoming pretty much unuseable (single packets would work fine, but not TCP/IP transfers of more than a few kb). When we found out it might be due to Skype malfunctioning, and gave the order for everyone to shut down Skype (which didn't work anyway, so people quickly complied), Internet became useable again. Let me emphasize that: it wasn't a bandwidth issue, but it was just the sheer number of connections opened by Skype that caused more important connections to be dropped by the routers.So why is Skype not using proper servers, like MSN, AIM and ICQ do? Because they're cheapskates. It's cheaper for them to let others use your bandwidth instead of having to provide enough to handle all the data. The other reason is that it's harder for system administrators to deny access to Skype, if any other Skype user can proxy your access to the actual login server. But if you want/need a central user registry, you can't rely on a true P2P system. It's bound to get disconnected at some point.And yes, this especially means that Skype is not a good idea in a corporate environment with a large number of users. The skype approach works well for a single user at home, but the Skype networking behaviour can take down your routers if you have too many users running Skype at the same time. Other software choices, such as the standardized, vendor independant SIP protocol for VoIP, however would even allow you to setup a proper VoIP telephone system, with conference rooms, complex call routing, dictionaries, callback, routing to ATM or mobiles, complex voice mail systems, you name it. Skype is designed for single home users, not for company use.P.S. Am I the only one who thinks the Skype UI is crap? No groups (at least not in my version, which is the latest there is for Linux), and totally idiotic button placement (like this ugly 'add' button I'm pretty much never ever, and a floding-up button next to it with the options, quit and recent chats (do they have anything in common?) hidden in there. How about using a menu bar like everybody else? How about organizing functionality in a human-comprehendable way?P.P.S. What I guess has happened: somehow, most nodes (especially super nodes) got convined that the actual login servers are bad. As long, as some still had the connection, login still worked. At least sometimes (many probably experienced an unstable Skype connection before they could no longer login). But at a certain point, a large part of the network got separated from the login nodes and started routing requests around chaotically, trying to find the actual login server. The only way they have for fixing it, is slowly convincing and nodes on the perimeter of the P2P network about the proper location for the login servers, and that way spreading the correct information around again. Or shipping a new version which will re-inject this information in clients again. Just my guesses.P.P.P.S. It would be interesting to know what had caused this. If it was some kind of subtle attack on the network. Like someone trying to convince supernodes that the login servers are bad and untrustworthy or something. Maybe just by sending spoofed bad data packets from the login servers IP or so.P^4.S. Interesting graph on the number of Skype users logged in. Still going down.

Ten years ago, Miguel de Icaza announced the GNU Network Object Model Environment project, an attempt to fix a dependency on a non-free library for free desktops. Today, GNOME is a large, healthy and fun project with a very steady mission and personality. Congratulations to everyone who made it possible!

Matthew, the situation you describe can be avoided by creating an explicit relationship between the values that need to change together, either by computing those that can be computed, or by having the build process error out at some point if you did something wrong. For example, I can use either char const *const keywords[] =

    "foo",
    "bar"
;
unsigned int const num_keywords = sizeof keywords / sizeof *keywords; (for more correctness, use "(&keywords[1] - &keywords[0])" as the divisor), or you can explicitly tell the compiler how many keywords you expect, using unsigned int const num_keywords = 2;
char const *const keywords[num_keywords] =

    "foo",
    "bar"
; What we lerned in our database courses is true in other places as well: redundant information has a habit of going out-of-sync, so we should avoid it (=> db normalisation), or create a connection between the redundant bits (=> foreign keys).

Michael Hirschorn in this month’s Atlantic reduces my generation’s entire cultural zeitgeist to a single word: quirk.

Quirk, loosed from its moorings, quickly becomes exhausting. It s easy for David Cross s character on Arrested Development to cover himself in paint for a Blue Man Group audition, or for the New Zealand duo on Flight of the Conchords to make a spectacularly cheesy sci-fi video about the future while wearing low-rent robot costumes. But the pleasures are passing. Like the proliferation of meta-humor that followed David Letterman and Jerry Seinfeld in the 90s, quirk is everywhere because quirkiness is so easy to achieve: Just be odd but endearing. It becomes a kind of psychographic marker, like wearing laceless Chuck Taylors or ironic facial hair a self-satisfied pose that stands for nothing and doesn t require you to take creative responsibility. Just because you can doesn t mean you should.

Hirschorn makes a fair point which, I think, can be restated that much of the content I enjoy is really just candy. The Atlantic seems to have recently figured out its readership (or at least figured out me). Hard to Swallow (by B.R. Myers, who more typically writes about Korean issues) is a pointed moral critique of modern food lovers (chowhounds?) and food writing (including Michael Pollan’s The Omnivore’s Dilemma: A Natural History of Four Meals, much read by my contemporaries). If I weren’t already a vegetarian, I might take umbrage. “Insider baseball” pieces on Karl Rove and Michael Gerson are also excellent, and in the case of the former quite timely. To wrap up this encomium, props to the magazine for its clean new website design, which I believe premiered today, and for including an embedded Youtube video in the online version of the quirk article.

Share This

As I said previously, abydos was dead. I bought new hardware some days after, mainly a new Enermax Liberty ELT 500 W power supply (ATX 2.2 compliant, sick, my old Enermax ATX 1.2 could not work with new mainboards), an Asus P5K, 2 GB of G.Skill DDR2 PC6400, an Intel Core 2 Duo E6750 and a GeForce 8400 GS 512 MB. At that time, I just switched the hard drives without reinstalling the box. However, I had to reinstall some days because my old was an old Intel Pentium 4 (i386) and the new is 64 bits-ready. Before that, I played with some kind of stuff. First, installing cpufrequtils is just magic, because now my processor is set at 2 GHz instead of 2.66 GHz, and rise to 2.66 GHz on demand. Just cool: less power, less warm, less noise. I tried to tweak my SATA drives. I did something very wrong: hdparm -s. Kids, don't do this at home, it killed my hard drives! Yes, you're warned if you read the hdparm manual and you have to force the command, but I'm mad anyway. After that, the drives were no more detected at boot time by the buggy BIOS. Why buggy? Because after some googling, I found that they were NOT dead, just sleeping. And the fucking BIOSes around never try to wake them up. Fortunately, hdparm people distribute an bootable ISO that will wake your drives... It saved me. The Asus P5K is not well supported by the 2.6.18 kernel, so you won't have network during install of etch (or lenny beta 1), and when you will reboot you won't be able to boot correctly because initrd is stuck with hard drive detection. Furthermore 2.6.18 on this mainboard is fucking slow anyway. Fortunately, I just added another NIC in the box, boot on the CD, mount disks, chrooted and dist-upgrade to sid with a 2.6.22 kernel which work perfectly. So now, everything's ready and set up, I can play!

Introducing debcheckout Cute little tiny teeny new addition to devscripts: debcheckout (not yet uploaded though, in the mean time you can get it from here. It checks out the versioning repository used to maintain a given package. Sample usage:

$ debcheckout devscripts
declared svn repository at svn://svn.debian.org/devscripts/trunk
svn co svn://svn.debian.org/devscripts/trunk ...
A    trunk/debian
A    trunk/debian/control
A    trunk/debian/links
A    trunk/debian/dirs
A    trunk/debian/compat
<snip>
 U   trunk
Checked out revision 749.
$

The information about where to find a repository is extracted parsing (in a rather dumb way actually, but I really can't stand libapt-pkg API!) Vcs-XXX fields. Intended usages:

1. NMU scenarios: when you're NMUing, please commit your patches (if possible of course: directly to the repository if it has already adhered to the open your VCS campaign I'm sponsoring, or somewhere else if you're using a distributed VCS); with debcheckout the first step it's easy
2. ease the creation of patches: isn't it better to checkout a repository, fiddle around, and then just invoke svn (or whatever) diff instead of remembering (I always forgot that!) to first create a .orig copy of the debianized source tree?
3. retrieving the bleeding edge version of a package which includes the patch for a pending bug you have been waiting for ages

RFC Let me know what you think of debcheckout, feature requests, whatever. In particular let me know if I did something wrong using some VCS, since I'm not proficient in all VCSs supported by debcheckout; for example: I'm quite sure the Arch part is not working ... help is appreciated! Vcs-Cvs proposed convention debcheckout can also give the ground for standardizing the VCS-specific meanings of the various Vcs-XXX fields. In writing it I've noticed that almost all VCSs have de facto standards about what to put in the field, with the notable exception of CVS. That's probably because all modern VCSs rely on some URL-like identifier for a repository location, while CVS does not. It needs a pair URL/module to be checkout. The format I'm currently supporting in debcheckout is a pair of space separated values "CVSROOT MODULENAME"; later on I'm using those values as in cvs -d CVSROOT checkout MODULENAME. Also please note that if you do not put the heading ":pserver:" string in the CVSROOT, users won't be able to checkout the repository without providing a password. Tip of the day: Pod::Usage Perl's Pod::Usage module is cool. Finally I can write the usage string only once instead of duplicating it in the manpage and in the string to be printed upon --help. Ruby had something similar, but the output on console was so horrible that I preferred duplicating stuff for Ruby scripts. Update: I've changed the link to debcheckout.pl so that it points to the "live" version in the devscripts repository, since some patches are already flowing in ...

Protx, a payment service, installed a major update on 1 August. They'd warned us about this in newsletters and we were prepared. We paid a bit towards another company's development of the protx_form module for OSCommerce to meet the updates and generally clean it up. This went slightly wrong in three ways. Firstly, protx went down for longer than expected, ultimately generating an apology from the MD. Not much we could do about that. Secondly, protx started rejecting commas in amounts. GB uses commas in between thousands, so this meant none of our British-localised shops could process orders over 1000. Ow. The Protx VSP Form docs still shows commas in the Amount examples, so I suspect that change was unintented. I've sent Protx Support an email to ask. I was offline, but Paul installed a fix before I got back. Finally, the mainline protx_form module has updated to v1.17 to include these changes and more (see Payment Modules forum threads), but our developer hasn't uploaded their work yet, so we're stuck up a branch and someone needs to merge. I hope it won't be us, as I can already see conflicting changes. What should I have done differently? Is it time to find a new payment provider? Should I insist that we publish paid developments ourselves as soon as possible? (2007-08-16: 1 comment)

When are two identical changes the same, and when aren't they? Theres a little bit of debate started by Andrew Cowie posting about unmixing the paint. Matt Palmer followed up with a claim that a particular technique used by Andrew is dangerous, and finally Andrew Bennetts makes the point that text conflicts are a small subset of merge conflicts. That said, one critical task for a version control system is the merge command. Lets define merge at a human level as "reproduce the changes made in branch A in my branch B". There are a lot of taste choices that can be made without breaking this definition. For instance, merge that combines all the individual changes into one - losing the individual commit deltas meets this description. So does a merge which requires all text conflicts to be resolved during the merge commands execution, or one that does not give a human a chance to review the merged tree before recording it as a commit. So if the goal of merge is to reproduce these other changes, then we are essentially trying to infer what the *change* was. For example, in an ideal world, merging a branch that changes all "log messages of floating points to 6 digit scale." would know enough to catch all new log messages added in my branch, regardless of language, actual api used etc etc. But that is fantasy at the moment. The best we can do today depends on how we capture the change. For instance, Darcs allows some changes to be captured as symbol changing patches, and others as regular textual diffs. So the problem about whether arriving at the same result can be rephrased 'when is arriving at the same result correct or incorrect'. For instance, if I write a patch and put it up as plain text on a website, then two people developing $foo download it and apply it, they have duplicate changes but its clearly correct that a merge between them should not error on this. On the other hand, the example Andrew Bennetts quotes in his post is a valid example of two people making the same change, but the line needing a change during the merge to remain correct. Here's another, example though. If I commit something faulty to my branch, and you pull from me before I fix it. Then while I fix the bug, you also fix it - the same way. That is another example of no-conflict being correct. If its possible for either answer - conflict, or do not conflict - to be correct, then what should a VCS author do? There are several choices here:

• Always conflict
• Never conflict conflict
• Conflict based on a heuristic

I think that our job is to assess what the maximum harm from choosing the wrong default is, and the likely hood of that occuring, and then make a choice. Short of fantasy no merge is, in general, definately good or bad - your QA process (such as an automatic test suite) needs to run regardless of the VCS's logic. The risk of a bad merge is relatively low, because you should be testing, and if the merge is wrong you can just not commit it, or roll it back. So our job in merge is to make it likely as possible that your test suite will pass when you have done the merge, without further human work. This is very different to trying to always conflict whenever we cannot be 100% sure that the text is what a human would have created. Its actually harder to take this approach than conflicting - conflicting is easy.