After seeing everyone shill for their favorite scm lately, with most of the predictable ones getting the attention, I thought I'd throw my choice out there, just to see it represented.

I primarily use svn with my work on X in Debian, and it's a wonderful system. Fortunately though, I don't have to maintain the repos, and when something goes wrong I can cry to our beloved DPL to get his admin on, and he fixes things up. This works out fairly well for me as a basic user, but I don't want to be dealing with these sorts of hassles for smaller projects. The distributed scm thing appealed to me in this sense, so when arch started to become a serious buzzword in the Debian crowd I had a look.

Well, more than a look. I fought for over a month with tla and baz and found that they were both so unuseable and poorly documented that I couldn't do more than scream and cry. bzr looked cool, but it wasn't ready (I sorta tried to manage one of the X.Org releases in Debian with it and failed miserably). Darcs was slow and in a minority language, which was a major turnoff for me. And I don't have a ton of diskspace, so git was out. Svk struck me as a hack (it still does) so I didn't want to go with that. Monotone was ok, but not good enough, so I sighed, gave up, and went back to svn thinking that this was as good as it could get. Indeed, we added a quilt-based patch system to the Xorg packages, so now I have a mini-scm (quilt) inside an scm (svn) and it works amazingly well.

But one of my friends just got hired by the people who make Xen, and he mentioned that they used mercurial, and asked if I had tried it yet. I scoffed and said no, and that all scm's suck harder than anything has ever sucked before, but I had a lot of time to kill while I was waiting on an X build, so I gave it a look.

Mama, I'm home. I was stunned that I was able to pick up mercurial in about half an hour after tearing my hair out for over a month with arch.

The UI is apparently very darcs-like (by accident, according to the mercurial author) which is known to be fantastic. Branching, merging, etc is simple as can be so far, although upstream is still working out better methods to deal with merging.

It's very fast, built for large trees (i.e. the kernel) so it will be suitable for managing X with (although I haven't tried this yet, but I plan to). For X, I'll need something fast to keep my sanity. This rules out darcs and baz (which has somehow managed to get amazingly slow in addition to being difficult as of late).

Unlike git, it keeps changesets in xdelta format rather than storing whole files, so it doesn't eat my hard drive.

It's written in python, so it's something I could hack on if I want to.

It supports an easy server setup with apache, and you can push your repos and serve them via static http if you don't have a server, like me.

Like monotone, it exposes hashes to the user, but unlike monotone it also provides a local revision number to make sure you don't spend all your time copying and pasting hashes, which is what put me off from monotone.

It has a fairly small command set, with 39 commands in the current tip (i.e. HEAD) as opposed to the > 150 in baz which made me weep.

It feels like using bzr, except it's here now and ready for production. Indeed, it's already used in production in making Xen.

There's a bunch more that I won't talk about, mainly because these are the things that have grabbed me. A note of caution is that I've only been hacking on local projects by myself (mainly configure-debian) with it, so I don't have the tons of real world experience with multiple trees that the thing is built for yet. I'm thinking about putting up a parallel mercurial tree to the XSF tree just for fun (the svn repo would remain the canonical one) if people are interested and I can find a server. I hope though, that all the people out there who, like me, were bouncing around trying out scm after scm and never being quite happy with any of them at least give mercurial a fair chance. I'm still amazed at how much I like it, and I think some others might feel the same.

Last time I blogged about a bug a lot of people had reported with X.Org, and at that time the workaround was to download a file and replace the installed version with it. Well, Eugene Konev came through and found a more complete patch and we've got that bug essentially fixed on our end, so I think most people can now run X.Org on Debian without fear. The bug in gcc4 is still present, but the next upload of gcc will include a fix for the issue, so we'll feel a lot more confident that the bug is really dealt with the next time we upload. Another thing that's happened is that it appears that the current upload does finally build on all arches (after I messed up -3, which should have been the one to do so) so that's one major issue happily out of the way.

We've already begun putting together another X.Org upload that should happen soonish. There's a pretty crippling bug on alpha that Steve Langasek has already provided a patch for. The big news though is that I've finally ripped out the X.Org package reliance on dbs and replaced it with quilt. Eugene Konev hacked up a very good version of the debian/rules file (which controls building of the package, for those of you who haven't dealt with the internals of a .deb before) that uses quilt instead of dbs. I did a little work on it myself so that it would provide the same basic features that dbs gave us, and now it's been rolled in to the svn archive.

Now this won't directly affect any users, but what it will do is let us process patches much faster than with dbs. dbs requires that you copy the whole source tree once, copy it again after applying patches, and then diff the whole tree when it's done. For small trees this isn't a huge deal, but for X.Org this is crippling, especially on my laptop where I do all my work. quilt lets me edit patches in place, and only diff those files that are edited. In addition, if there's a patch that I already pulled from somewhere like X.Org CVS or Redhat's bugzilla, I can simply tell quilt to import it and it will. This literally will save me an hour per patch. quilt also buys us some cool features like being able to check patch dependencies, tell what patches alter what files, and sort patches in to subdirectory heirarchies rather than having hundreds of patches sorted solely by name. Plus, it knows about patch documentation, which dbs will overwrite by default, and quilt will happily preserve.

So hopefully this will make the XSF ever so slightly faster at getting things done, although we're still obviously hampered by the scale of the source tree. Future patch audits should be on the order of days rather than weeks with this system, which lays the groundwork for work on 6.9/7.0.

Yesterday, the right-wing PAN candidate for the presidency of Mexico, Felipe Calder n, made a declaration that shadows the current president, Vicente Fox (from his same political party) almost look like a genuine statesman. (some links, La Jornada, El Universal) - And that's no easy feat, given that Mr. Fox is -so far- the nation's most prolific president... When it comes to stupid sayings.
How come Calder n rose so quickly to become a statesman? Let me first go through a bit of history. Not much, promise. Well, I'll do my best.
Twelve years and one month ago, Mexicans woke up with the frightening news we had a violent guerrilla group in Chiapas, one of Mexico's richest states, but one with the highest percentages of poor people - and, not coincidentally, of indigenous population as well. This uprising was very soon known all over the world - And its leaders quickly learned how to ensure they would not be prosecuted and exterminated: Simultaneously with the fighting, they made as large a media coverage as they could, strongly supported in the Internet democratization of the information that had just begun. Thousands of people all over the world understood their ideas -at least what they said at that time- and spread the word. I had my first trip in Europe with my (now) wife in 1996, and we saw many grafittis stating Viva Chiapas, Viva EZLN... It was amazing.
EZLN was (is) specially hard for the government to kill. As they don't do military sabotages, terrorism or so on, it is very hard to justify military action. After 12 days of fighting, a truce was decreed. The truce was broken once, in the early years of the Zedillo government, but officially there is a cease-fire, observed by both sides. The Zapatistas have took the ideas and further promoted them, with action. My Chiapas-born friend Alex Ju rez often tells me about the tremendous cultural difference between the Zapatista territories and the paramilitary ones - In the Zapatista areas, you see indians reading books, working their land, organizing themselves in the Fair Government councils, and strongly opposing alcohol and drugs. Just the opposite from the other areas.
EZLN shifted their strategy some months ago, going beyond stating their wishes for a fair regime and waiting for the government to look at them. I still don't really understand where they are heading, but the Sixth Declaration of the Lacandon Rainforest turns them into a political movement, clearly separated from the existing regime. They do not seek to be elected, they oppose every party and candidate. They define themselves as an anticapitalist movement, located down and to the left - Lets see where this goes to.
Well, back on track... One of the president's most famous stupid quotes is that, during his campaign, he promised to solve the Chiapas conflict in fifteen minutes. Well, it's been either fifteen very long minutes, or he just hasn't had any time to devote to those dirty indians. Mexicans are very frustrated by the lack of importance this man gave to one of our country's longest and most desperate needs.
Of course, Felipe Calder n must do something in this regard. Fox's shadow is a big one, and this little man must try to escape from Fox's biggest mistakes. So he just came with a formula that I don't know why no statesman thought of before: Instead of negotiating, instead of working to solve a very difficult problem, he declared he will unilaterally declare peace with EZLN.
I wonder if this year's Nobel prize will go to such an abnegated, smart and selfless person. Man, we could take this same solution to Darfur, to Israel/Palestine, to Congo, to Chechnya... If rulers all over the world start declaring peace unilaterally, all of the world's problems will be solved! What comes next, he will declare (maybe as Salinas did) that we are a first world country? Poor people are only sad reflections in the sand but don't exist in his reality?
Please, Mexicans: The man is clearly schizoid. Are you happy with Foxilandia? It seems that alternate realities are part of the PAN's action program. If you are not satisfied with Fox's results (who is?), don't vote for this guy. We will all regret it.

This took a bit too much work to figure out so here it is. If you're like me and had to buy a usb sound card (nearly as sensible a name as a cable modem) since the laptop's onboard headphone jack broke, and would like to have that card used by default when it's plugged in, but otherwise have builtin card be used, create an /etc/udev/rules.d/00_local.rules containing:

# Default to using additional (USB) sound cards when they are available.
KERNEL=="pcmC[D0-9cp]*", ACTION=="add", PROGRAM="/bin/sh -c 'K=%k; K=$$ K#pcmC ; K=$$ K%%D* ; echo defaults.ctl.card $$K > /etc/asound.conf; echo defaults.pcm.card $$K >>/etc/asound.conf'"
KERNEL=="pcmC[D0-9cp]*", ACTION=="remove", PROGRAM="/bin/sh -c 'echo defaults.ctl.card 0 > /etc/asound.conf; echo defaults.pcm.card 0 >>/etc/asound.conf'"

Only programs started after the sound card is plugged in will use it of course, and this might not work if you have it plugged in while booting.

Over one month ago, I truly hoped an evasive network misbehavior issue was solved. In the institute, we had a strange situation which caused a (seemingly random) stack of switches to go out to lunch and leave one of the floors without network access until it decided to restore itself (usually power-cycling it was not enough). The equipment, ten 4200 SuperStack switches, is new, we got it ~September, so I expected 3Com to give me at least some support on the issue. The behavior was apparently random, and I could not track it down, only (thanks to the very nice Cacti) prove it existed.
December 5 was terrible. The whole building's network refused to work. Switches were able to reply to my pings for no more than a minute after booting before they got flooded with "Spanning Tree CTRL MAC PAUSE: Quanta 65535" (according to another great tool, the Ethereal protocol analyzer).
I had been bugging the support people at 3Com for at least three weeks before this incident, they kept promising they would come soon to help us track the cause of the ellusive problem - We threatened to send the switches back and pursue reimbursement through whatever means we could (we had much better stability and performance with our old 10Mbps hubs, so my boss insisted on replacing them), so we got them to come. The answer they gave me? Upgrade your firmware to version 2.50 (not available at their website, of course) and disable spanning tree functionality at the switches.
I have to admit the big problem immediately ceased. This is not to say I was pleased with the response time or with the switches' software quality, but at least we could work.
Happily, ~7 weeks passed by with no major outages (although with a few small ones).
Yesterday, we got back the complete chaotic conditions. This time, though, I was able to get a good and useful tcpdump snapshot, passed it through Ethereal, and actually got something interesting (available to whoever requests it to me if it sounds interesting). In short: Something happens that confuses a switch (I guess it's a faulty NIC), and makes it request to all of the neighbouring switches (via spanning tree protocol - Yes, it's disabled, but still) for their complete ARP tables. But somehow, this noise or whatever keeps confusing it, so the spanning tree requests start repeating every two seconds for slightly over one minute. Then, the same switch which requested this information gets tired of too many ARP tables being thrown at it, and sends out a storm of packets (one each 0.04 seconds) asking every other switch _not_ to send updates anymore. This is broadcasted to all of each of the switches' ports, and the network dies.
My grudge against 3Com is: How come this frigging noise (at least that's what I assume it to be) on one of the ports kill all of the network? If a switch is supposed to be smart, would it be too much for it just to disable the misbehaving port?
Anyway... One and a half days went down the drain trying to find the cause for the problem, sending the report to the 3Com guy (who is very nice in person, yes, but that's by far not enough!), running up and down the stairs to reset different switches, lock down on the source of distress... And as my 3Com contact was busy with another client (who presumably had not yet given them money), I never got a call back from them.
All my good intentions to spend a nice quiet time coding for three projects we currently need to have ready soon (one of them is in production and many of you have used, my dear Comas conference management system, but requires heavy tinkering to be able to extend it) were stuck at that: Good intentions.
I seem to have locked down the problem in one of three computers (that is, network seems stable again once I disconnected them). I insist it must be a faulty NIC. My boss bets it's an ber-contaminated Windows machine. My boss' boss insists we should get the switches double-grounded just to be sure that's not the reason for failure. I am tired, bad mooded, and -again- bored you to death with a too-long blog entry.
Blame it all on 3Com.

People seem to still want to make big computers to beat chess masters, while Kasparov has been beaten by the upgraded Deep Blue, then nicknamed Deeper Blue, in 1997. But actually, what’s the point ?
I mean, all it demonstrates is that brutal brute force is better than human. And now human has been beaten, what’s the challenge ? So, let’s move on a bit and go for a real challenge : beat human at Go. You know, this simple game with simple rules but for which no computer has yet beaten average players. I’m not even saying advanced or masters, because computers are so boring and lame that strong players prefer to play humans. And if they happen to want to use the brute force as they did for chess, well, they’d need a computer to which Deep(er) Blue would be an abacus. More seriously, though, why isn’t there more IA research on Go, now that Chess is more or less mastered ? Another thing that puzzles me is how come gnugo is so good compared to other Go programs, while free software Chess programs are so lame ?

There's a fairly standard set of activities that go on at linux.conf.au each year -- keynotes, tutorials, seminars, miniconfs, BOFs, lightning talks, keysigning, etc. The hackfest is a standard part of the program, but one which (in the past) maybe hasn't gotten the attention it deserves. I think this is because, in 2004 and 2005, the hackfest was a fairly narrow task -- in Adelaide (2004) contestants had to write a tetrinet client (fairly complex), and in Canberra (2005) contestants had to write either a GUI or an AI for spellcast (a fairly esoteric game that nearly nobody has ever heard of). Prior to this year, I'd never participated in a hackfest. I've never played tetrinet in my life, and I knew I had no hope of writing an AI for spellcast. Also, since the contest in both cases ran for most of the week, to be competitive I figured you had to spend all of your spare time working on your program, and there was beer to be drunk, so I decided it was a bad use of my time. This year, though -- phwoar. A totally different problem set, a new way of running the hackfest. Far more approachable for a much broader collection of hackers, and since it was a 4-hour session, you knew how much time you had to invest up-front. I really appreciated that there was something for pretty much every type of competent programmer. One task, for instance, was to build a config GUI for the ccontrol tool, another was to write a Sudoku puzzle generator. These aren't simple problems, but I'd say that most people I'd call a programmer could make a good showing at one (or both) of these problems. At the same time, though, there were some really low-level problems (like optimising a C program or turning some PPC64 assembly code back into the original C function). These were interesting for me, as it's not something that I'd ever touch myself in the normal world, so it was nice to be able to submerge into something like this for a little while. There were even a couple of problems that didn't require much programming skill -- fix a corrupted gzip archive, and fix a corrupted filesystem image. These are difficult problems, but don't (necessarily) require the production of reams of code to solve. To be able to win the competition, though, you need to have a broad range of skills. This is entirely appropriate (although it rules me out of the running) as a true hacker is adaptable to new problems. With the previous competitions, if you happened to luck out and be a tetrinet or spellcast guru, you had the competition in the bag. This competition will slightly favour someone who is a PPC assembly expert over someone who doesn't know much about anything, for example, but your PPC guru (who can't do much else) is going to get 0wz0red by the problem-solving, google-loving generalist. The time limit aspect was a very, very good move as well. It meant that I could accurately estimate, up front, my total time commitment to the game. I didn't have to worry about getting flogged by someone who dedicated themselves day and night to cracking the problems while I was out stonegrilling my dinner and emptying barmates. Now I only have to worry about getting flogged by people who are just Better Hackers -- which I have no problem with. It was also great to be able to switch on the iRiver and bury myself in the problem for a few hours, instead of putting it off all week and then bodging up a poor solution in the last few hours. So, kudos to Chris and the rest of the team who ran this year's hackfest. I really enjoyed this year's competition, and I hope that next year will be equally interesting.

Sorry. Couldn't resist. Following on from my previous look at XTech proposals, I wanted to comment on one of the hot topics contributed to the Applications track. While this track will push all the buttons (we've got Rails, RSS, web services, content management and more) one of the really striking things was the number of proposals on the topic of DITA. If you're not a regular in the tech docs world, then DITA is probably at best just another acronym you've seen whizzing by. DITA stands for the Darwin Information Typing Architecture, and it's a new document format making waves in the world of technical documentation. From a casual documenter's point of view, the obvious question is probably "what does this mean for DocBook?" Does the mass of enthusiasm mean DocBook's on its way out? In short, no. DocBook and DITA compared  DITA and DocBook are actually rather different beasts. DocBook is a sound DTD for writing technical books and articles, with the structure that such forms imply. DITA is focused on creating modular technical documents, with the intent for easy reuse across varied delivery methods. It's intended for topic-oriented documentation, which may be chopped about and rearranged within a variety of contexts and for a variety of display devices. DITA is a DTD and a set of rules for writing online contextual documentation such as software help files. There's a lot more to it than I'll write here, but here a few more differences between DocBook and DITA:

• DocBook is hierarchical, DITA separates content from context allowing multiple architectures of information
• DocBook is a fixed element and attribute set, DITA is extensible, allowing the definition of custom information types
• DocBook is much more mature than DITA, with a variety of solid tool support options. DITA's the new kid on the block.
• DocBook is well documented, DITA's documentation is still in the early stages.

There's a real need for something like DITA, which addresses concerns that didn't really exist in the same way when DocBook first emerged at O'Reilly in the early 1990s. DITA and open source projects I'm looking forward to learning more about DITA, but one thing that immediately strikes me about it is its relevance to some of the documentation problems faced by open source projects. DocBook has never quite clicked as a documentation format for some collaboratively maintained projects, probably because the inevitable book-like nature of the result necessitates sole or restricted authorship, and heavy editing. Instead one of the increasingly popular ways for providing documentation has been the wiki. You guessed it, a topic-based architecture. What wikis seem to lack, however, is any way of providing a thought-out progression through the content. In essence, the information architecture side of things. Had I been able to attend every session from XTech last year, I'd have heard Paul Prescod talk about what DITA and wikis have in common.
There may be a downside, however. DITA looks impressively engineered, but it may be too industrial-strength for many. My initial investigations suggests that while there's emergent commercial tool support, there's not much going on in the open source field (although there is the DITA Open Toolkit, which looks to have been produced in close association with OASIS technical committee now in charge of DITA.)
As DocBook guru Norm Walsh observes, DocBook can play in this space too. With additions such as those Norm proposes, DocBook could play host to DITA's major features. (Norm writes "With a couple of hours of hacking, I've implemented on top of DocBook the four key features of DITA that I could identify.") Though these possibilities are intriguing, I think that, for open source projects, things need to get simpler rather than harder. More reading:

• DITA--Getting Started (XTech 2005 proceedings)
• To DITA or not to DITA? (Palimpsest blog)
• Cover Pages: Darwin Information Typing Architecture (Robin Cover)
• DITA FAQ (OASIS)
• Going DITA (O'Reilly weblogs)

For the first time in more that 10 days, I slept like a baby last night. It might hurt the xlibs-dev transition, but I hope it actually helps in the long run, I will make less mistakes, bug vorlon and dato a bit less, be more aware and awake. Now I just need to keep a balanced sleeping schedule again. I think have good allies.

I have started a small experiment that is against all rules but might spread some package love. So far, I have been successfully working with Victor Seva, Justin Pryzby, Thomas Viehmann and Nico Golde. Join us if you care about this transition!

On the other hand, I find recent threads in debian-devel, debian-private and several posts in debian-devel-announce very distracting. Although I am strongly opinionated on the subjects being discussed, I'd rather remain semi-silent and work in my "NMU love".

Today I had my first encounter with authenticated proxies (creepy). The client has a working LDAP directory and Samba servers for logins in their network, and until now they have plain old packet forwarding for WWW access. They didn’t even had Squid caching as a transparent proxy, and they suddenly wanted to authenticate users against the directory, and give particular access to users of certain groups. Luis was working on this using winbind and ntlm_auth in Squid (all from their corresponding packages in Sarge) and he was using the –require-membership-of stanza on auth_param. So his configuration was basically:

• An authentication required ACL: acl whocares auth_proxy REQUIRED
• Two authenticators (auth_param) using winbind (one for basic and one for digest) with the ntlm_auth utility, and specifying the SID of the group.
• Repeat the last step for every group wanted to have access to the proxy
• http_access rules involving the ACL made in step 1

This approach didn’t work as it meant to be used, since only the first authenticator worked. Seems like Squid parses the list of authenticators and only work with the first one. So only one group can be allowed to use the proxy this way, as far as I understand. I googled a bit through some Russian forums and started solving some problems with winbind and Samba themselves: no domain separators, no stanzas for winbind in the smb.conf (winbind use default domain = yes), bad mappings for uids and gids, etc. Then I found out about wbinfo_group.pl, a Squid helper written in PerlOfCourse which can find out the group of a certain user trying to authenticate with the proxy. So I thought that maybe this way I could made specific acl’s for each group and even control other stuff in Squid using the group. I made this in this way:

• An external ACL (the helper) defined as: external_acl_type nt_group ttl=0 concurrency=5 %LOGIN /usr/lib/squid/wbinfo_group.pl
• An ACL that tries to match the helper with a group name: acl allowed external nt_group “Allowed Navigation”
• Rulelists involving the above ACL
• Repeat steps 2 and 3 for each group with special needs.

This is magically working, and I think there’s more flexibility with this than the other way, but I’m not an expert and I have questions that will be answered in this week, hopefully:

• Do I need the proxy_auth REQUIRED ACL?
• Is there another way to allow access for several groups using winbind and NTLM?
• Why use winbind? Wouldn’t LDAP, SMB or other authenticators named in the Squid FAQ work?

For the first time in more that 10 days, I slept like a baby last night. It might hurt the xlibs-dev transition, but I hope it actually helps in the long run, I will make less mistakes, bug vorlon and dato a bit less, be more aware and awake. Now I just need to keep a balanced sleeping schedule again. I think have good allies.

I have started a small experiment that is against all rules but might spread some package love. So far, I have been successfully working with Victor Seva, Justin Pryzby, Thomas Viehmann and Nico Golde. Join us if you care about this transition!

On the other hand, I find recent threads in debian-devel, debian-private and several posts in debian-devel-announce very distracting. Although I am strongly opinionated on the subjects being discussed, I'd rather remain semi-silent and work in my "NMU love".

I've been using OpenVPN to provide secure wireless for a group of around 200 students here, of which around 60 frequently use it. It was running fine for the last few months, but started doing weird stuff on friday. Dear Lazyweb, I'm lost at debugging the cause... I've eliminated the wireless links as possible cause, so here is the simplified setup:

User #1   -- openvpn  --   tap0 \                 eth1   -- DMZ net
                                 - br0  Firewall
User #2   -- ethernet --   eth0 /                 eth2   -- internet

So the firewall host has three ethernet interfaces, one of which is bridged together with the OpenVPN in tap mode. Everything works just fine for User #2. The firewall rules used all use only br0 as device, and a netmask both User #1 is in and User #2. Rule counter in iptables verify that the firewall is working correctly. User #1 can:

• access hosts in the DMZ network perfectly all the time
• access the internet fine for like 2 seconds
• ping the internet

After these two seconds, packets start being retransmitted; some of the retransmissions will then arrive at the client, but rarely enough to actually receive a simple website. I am not aware of any changes to the client or server OpenVPN configurations. The latest changes (according to SVN) in the firewall rules were completely unrelated "accept" rules... (and the retransmissions arrive, so...) Any hint? Some MTU issue maybe? Maybe "upstream" routers increased their MTU, now the packets are hitting some triggers? But why would it work for 2 seconds then each time openvpn on the client is restarted?