Since Freenode has been compromised my nickserv password might be too. Unless they had the good sense to hash the passwords, which isn’t common. So after checking my 121 line long vim encrypted “accounts” file, I noticed that this particular weak password is used in 17 other systems. Woops. You know what be good? Web standards in regards to user accounts. Now I have to fiddle with umpteen annoyingly different systems to change my password. Pain and suffering. It would be far better is I could do this task with a CLI tool or some handy trusted Web application that would implement such a standard. Though would such a standard make it easier for “crackers” to script attacks to change your password and really screw you over? Actually after thinking about it, the best option would just be to reset the password via email. Though that process can be sniffed… If the cracker tried to change your email address too, the user management system should send you an email to the old address to confirm the change. If the user management system had any doubt, it should just send you email a link to proceed. Btw, I’ve made the move to the OFTC IRC network. I also found lilo’s “money badgering” a bit too annoying.

Last month I went to Mexico to attend Debconf6 and then I traveled with 4 geeks to Oaxaca. Here are a few photos from the trip, which was quite lovely, though I stalled writing anything about it here because I was thrown into a strom of sickness during my last week there and into my first week here. I have been to Mexico a few times before though usually in passing or in Baja, which is somewhat nice but desolate. This was the first time I had any sustained exposure to Mexico and it was quite nice. Some of my favorite things about the country and my visit are perhaps a little odd but they are what took me in and here are some of them… For example, during Debconf proper, I feel in love with the following reptile and very surprisingly, because, when it comes to animals, I usually like the soft and cuddly and not the scaly and spiny. But…. this lizard was…. special. Here is how I found him: We were eating at a restaurant after a visit to some ruins and there we found this little guy caged at the back of the restaurant. I was more than a little miffed that he was trapped (and by butting his head against the cage, he clearly was saying I WANT OUT) and so was about to pull a “PETA move” and just let him out. But I decided that was not such a good idea so instead I went over to the owners to ask why the little one was caged. In answer to my query, they promptly pulled him out and told me he first that he was a chameleon (which does not seem right to me) and was usually free to roam but when there are a lot of folks (as there were) they lock him up. He plopped the little one on my arm and alas, what I pleasantly found out was that a lizard can act an awful lot like a dog. He LOVED to be pet and basically had a look of all-out-pleasure when you stroked his back. And at times, with a particularily good stroke, he would even subtly smile and flutter his hind leg a little just like a doggie. Aside from a special dog-like reptile, there were many other things that I loved about Mexico. The food at the market was divine, divine, especially the corn smut a.k.a huitlacoche and all the salsas were better than you could imagine. It is tragic that every city does not have a local market with such divine food. The architecture of Oaxaca was especially stuning as was the whole city center. It reminded me of viejo San Juan in Puerto Rico but the buildings were a little lower (apparently because of earthquakes) and the environment was dryer and more expansive. Our hostel was courtyard style (as were many houses there) and I think the courtyard should be revived as the defacto style for housing as this is probably one of the most enticing and smart architectural styles ever (at least in warm places). While a home is suppsed to provide shelter to enclose, the courtyard brings some of the outside inside and some of the inside outside. While in Oaxaca we met some folks from espora, walked a lot in the city, met up with other Debian folks traveling about, and cleaned out Vagrant’s ear which was truly … well I won’t go into that. We also drove for a few hours up to the mountains for a few nice hikes. The town hoped we happy trip and I think we did. Mexico city, by a number of measures, is one of the largest in the world and I unfortunately only got to see the Zocalo area as well as the subway. The center was quite impressive and especially nice before a thunderous storm One night at the Zocalo we happened upon a free concert, the lineup including the Tijuana Nortec Collective. One of the most dynamite parts of Mexico City is its impressive and I mean impressive subway system. In a nutshell, the system is large, clean, efficient, cheap, and quiet (thanks to thick and black rubber tires). The trains come quickly, ferrying the millions of riders who pay 20 cents for a ride on a lovely bright orange train. Every time I rode it, without fail, a vendor entered the car, loaded with a backpack that doubled as a radio/stereo, who was selling pirated music. I appreciated the sample and bought one disc chock full of mp3s. Like most urban trains, the riders are silent and I have always wondered why this is so. When I go to PR, I ride the busses much more and there, as in the states on the busses, there seems to be more open flow of chatter. Perhaps it is because trains are quite comfortable and they lull already tired riders into a state of quite contemplation. Who knows. So these are some of the things that I liked about Mexico but perhaps the most exciting part about the country was the clear political heatwave passing through, just as it is with much of Latin America. In the zocalo of Mexico City there were various stands protesting the attacks in Atenco, as well as general protests against neoliberal policies. The city of Oaxca was also filled to the gills with protesting teachers, demanding a pay raise. Thousands of them were camped out all over the city center, which is apparently a year rite. Sadly, in the last week the police broke up the protest, violently…

It’s 3:51 in the morning and I am already awake and showered. In about 30 minutes or so, I leave for the airport, for a flight to NY, where I will meet my tour group and fly to Israel. Back in 3 weeks. Email me if you need me, I’ll respond if I have a chance. I will hopefully put pictures up at http://www.dlgeek.net/israel during the trip, but no promises.

I'm trying to replace SourceSafe at work with Subversion but it is going slowly. If anybody has any Java Web development using Subversion from Windows experiences they want to share, please toss me an email

Last week, I finally becamed annoyed enough by the huge amount of spam I receive, mostly on my @debian.org address. So, I decided that my current anti-spam measures (SpamAssassin running on my home server, which receives all my mail, as well as my laptop, which collects it) are probably not enough. In the same time, I was seeing a few people talking about CRM114 in various occasions. It seemed easy to setup enough for me to try doing somethign with it (I usually give up on stuff that requires too much effort....). Installing it on my laptop was fairly easy, as a Debian package exists for it. Using it through procmail recipes required some documentation reading, but no more than 1 hour. The most important part has been training it. I did so for about one week by feeding undetected spam to it and teach it that it is actually spam...and of course doing the opposite with false positives on ham. One week later, I feel safe enough to actually filter all mails detected by CRM114 as spam to a dedicated mailbox, which I of course will have to investigate once a day to sort out remaining false positives. But now, my incoming mailbox is clean again and it really seems that the time I invested in learning CRM114 and teaching it about spam/ham will very quickly be worth it. In short, just try it: there are chances that you won't regret it. PS: of course, things would probably be better if Debian mail machines were using some measures similar to those finally adopted on Alioth with the help of Pierre Habouzit (RBL+greylisting, IIRC). Old debate in the project....let's hope it will happen some day if nothing I'm unaware of does prevent it.

We've been having some interesting discussions on the Hecl mailing list about how to do concurrency: http://sourceforge.net/mailarchive/message.php?msg_id=19566676 (although it seems SF has managed to lose a portion of the conversation!) When all things are said and done, we have a couple of options:

• Try and immitate Tcl's select-based event loop. Not really concurrency, but it does a good job of "faking it" for things like GUI's, if done correctly. For the right kind of code, it makes life very easy for the programmer, as you don't have to think too much about concurrency issues, and you don't have to deal with threads. The downside for J2ME is that we don't have the equivalent of select, so we'd have to do the system with threads, which would probably be costly in terms of the resources required to set it up.
• Tcl style threads, where each thread gets its own interpreter, and data is mostly shared by message passing between the threads. Conceptually, I think this is a cleaner way of doing threads, compared with 'sharing everything and using mutexes', and it's a little closer to how Erlang handles concurrency, which is very nice indeed. For J2ME, I'm a little bit worried that this might be too resource intensive, and potentially slow to launch new threads.
• A low-level interface to threads that just exposes the Java API. This would be nice, and lightweight, but potentially quite complex to program if we attempt to make the interpreter reentrant. There are a lot of issues with things like stack frames - for instance, say one subroutine (proc) adds a stack frame, then a second thread kicks in and exits a proc, tearing down the stack frame again. If you put a big lock around procs, that just leads to blockage if a proc is doing some sort of long running calculation, defeating the whole point of the exersize.
• Deal with concurrency on a case-by-case basis, building it into each command that needs it. Not very elegant, and probably requires just as much work, if not more, than other options.

At this point, I'm tending towards each thread having an interp, because option three seems very prone to extremely difficult bugs. Not sharing the interpreter has worked well for Tcl both from the implementation and end user perspectives. It is, however, not an easy subject and I'm still thinking about it. Note comments are broken, so email me with questions or comments.

The german version of the official motto "A time to make friends" is when translated literally "the world for a visit with friends". So far, people have been taking that literally, as far as I can tell. Everybody is a friend here. And there are tons of people at the public viewing places (which are currently being extended, after there were 700.000 people in Berlin's fan mile during the GER-ECU game, and 72.000 in the stadium, not counting all the other viewing places, pubs etc.). There is a cool pun on this motto: Friends for a visit with the world champion. So I made some t-shirts with a free (for making, not ordering...) service. I hope they can ship them in reasonable time (especially before germany loses a game, that is): Now I only have to hope that noone has registered this motto as a trademark. I checked the EU, German and international databases; nothing in there yet.

Despite its own spam creation, Google is keen to be seen as helping to fight spam. To that end, it files invalid spam reports like this report from Erich Schubert. As Erich writes "That totally sucks, that a perfectly valid and correctly sent email, that is also delivered correctly, is reported as spam amongst the big ISPs." [cont with links...]

I'm willing to start a bounty for someone to fix/port/implement the required pieces of GStreamer so that Sound Juicer can rip a whole CD, fully tagged (all tags SJ writes must be correctly encoded), to MPEG4 AAC (as used natively by iTunes). A requirement is that a music player that support MPEG4 files can see the tags without any extra work: the primary test for this will be importing the ripped files into iTunes. I think there are many people out there who would like this to happen, so I want to arrange a group bounty. Bounty County looked promising but I have to specify how much the bounty is up front, I can't just let the fund grow as people donate what they can. The Bounties section on Launchpad has the same limitation. Does anyone know a reputable site for managing donations like this, or should I just get register a new email address on PayPal and get people to donate there? Also to gauge how much interest there is in this, can anyone who would consider donating leave a comment? NP: Vertigo, Groove Armada

Apparently, GMail has been filing invalid spam complaints with DFN (the germany-wide university and research network backbone). This is the second time we were forwarded this complaint all the way down to us. I know exactly which mail it is. The IP is the firewalls "private" IP, so no other service is on that IP (it's not the masquerading IP). The time matches exactly the time of an email delivered to GMail. Therefore I'm 100% sure which mail they are considering to be spam. However, it is not. It's not in the "spam" folder in GMail either. No SPF violation. Valid reverse lookup in the appropriate domain. Single-hop SMTP delivery. Firewall to gmail mailserver. It's our daily snort report, that is considered spam every few weeks. That totally sucks, that a perfectly valid and correctly sent email, that is also delivered correctly, is reported as spam amongst the big ISPs. Unfortunately I don't have any details on who is filing the complaint, but it certainly is somewhere among the big ISPs. And traceroute shows a direct peering of Google with the DFN at DeCIX.

So, a little while ago, I wrote about why I like HP. This week, I'm starting to be annoyed at them.

My employer just bought nearly $100,000 worth of HP hardware. We get a new MSA1500cs Fibre Channel SAN (with redundant controllers, FC switches, disks, etc), a new blade enclosure system, three blades to start with (all of them, at minimum, dual dual-core Opterons with 4GB RAM, and some considerably more), a rack to put all this in, etc.

So we're starting to set all this stuff up. I've got Debian installed on an NFS root for testing the blades and how they interact with the SAN.

The blades have an integrated dual-port QLogic QLA2312 Fibre Channel adapter. The Linux kernel has a built-in driver for this (qla2xxx), which detects it and, so far at least, works fine. We want to run kernel 2.6.17 because it's the first version where XFS has decent semantics for write ordering to prevent corruption after a power failure. Plus we want at least a 2.6.16.x kernel because we want to run the latest Xen 3.0 on these blades. (Live migration of virtual servers from blade to blade -- this will be great.)

But we learn that HP does not support the kernel qla2xxx driver. HP does not say WHY they don't support it, just that their own driver is the only one that they support.

After plowing through several annoying scripts to get to their driver, I realize why it fails to install: it is OLD. At BEST, 2.6.14 is the most recent kernel it would even compile against (release date: October 2005), and I think the most recent version it supports is more like 2.6.8 (almost TWO YEARS OLD now). They reference a whole bunch of kernel symbols and macros that were removed somewhere between 2.6.8 and 2.6.17.

I sent a ticket to HP support. Their first request was to run their system information gathering tool and send them the results. Fine, that's reasonable. I did so. Next they say, gee, you're running Debian, and we don't support that.

Argh.... If they tried to compile it against 2.6.17.1 on RedHat or SuSE, they'd get the exact same problem. I told them what symbols they were erroneously using, and a simple grep would have showed them that.

Besides, how many customers are going to be pleased with no upgrade path available for 2 years? I wouldn't want our kernel version to be held hostage to HP's slow driver development process.

Sigh.

It's been a fun day today, the pre-conference warm up day for RailsConf.Over breakfast with Surj Patel and Matt Biddulph, we got to talking about debuggers and poking around inside Rails applications. With Rails being so new, I was intrigued in what happens when you hand a Rails application over to somebody. Does all the Rails magic help or hinder their ability to understand what's going on?What kind of ways are there, beyond documentation (which nobody writes), to communicate the workings of a Rails app? We thought that what might help would be some kind of graphical illustration of the flow of control, linked to a source viewer, that showed requests being processed. Rails is built in a convenient way for instrumentation and introspection, by virtue of the aggressive use of DSLs and Ruby's language features. Based on these, are there tools we can write to help people understand what a Rails application is doing?
I've not yet had enough experience of either handing off, or being handed, mature Rails apps to say for sure what's needed. But it's going to be a problem we all face when Rails sneaks further into the system. Java's redeeming quality in the enterprise is that it's damn hard to write anything one might consider idiomatic. As a consequence, there are predictable ways in which one can approach code written by somebody else, all other factors being equal. Will Rails idioms cause trouble for long-term maintainability?
Lunchtime included an interesting exchange with Rich Kilmer. Rich used to work for DARPA on RDF and OWL stuff, and has apparently written all manner of interesting Ruby code that processes RDF. It's a crime I'd not come across his work before, but it's all open sourced in the Semitar project. The project web page looks modest, but if Rich's description is anything to go by, it sounds a wonderfully appropriate way to approach programming for the semantic web.Finally, I notice that, as the day finishes, IBM developerWorks have published my Introduction to Rails for DB2 developers.RailsConf proper starts tomorrow. 

I got some bounces today from Nationwide (no doubt spam backscatter), but the interesting thing about them was that they were PGP signed. Some digging reveals that Nationwide seem to sign their outgoing mail, which is nice to see. Further digging fails to find any nationwide.co.uk keys on pgp.net, keyserver.net or pgp.com. Well that's useful, isn't it? They're not the only people who do this - I regularly see posts to mailing lists that are signed but the key isn't available. Why bother signing mails if your public key isn't out there? It doesn't really give the impression you understand why you're signing things.

The Latin American Infest finally ended yesterday. In Caracas, it was a week of hard work with people of the Science and Technology Department and the IT Office (government offices) but the event in Caracas ended succesfully. It was a world-class event featuring renowned keynote speakers who spoke about Web Development using Frameworks (featuring TurboGears, Rails and Catalyst) and software development paradigms migration from privative platforms to GNU/Linux (Gambas and Mono). Other talks included basic security in Debian GNU/Linux and a speak about electronic signature security by an officer of the Venezuelan Government CA. We had a speak about Clusters, hosted by a female member of the Venezuelan Free Software Community and Debian User, and a speak about Free Software in the Venezuelan Education, by a 16-year-old Venezuelan GNU/Linux user and a willing Debian Maintainer. Of course, the Installation Festival took place in the Knowledge Square, in the Auditorium Building Lobby, with the extraordinary support of the Venezuelan Free Software Users Community. Debian GNU/Linux Sarge netinst and CD1 was distributed (several hundreds) as well as Knoppix and Ubuntu Hoary. Ernesto Hern ndez-Novich (maintainer) and me wore Debian T-Shirts on Saturday, and he made a great speech about the Project, named “Debian is inexorable“. We had demonstrations of Linux on a Mac Mini, using XGl and presenting several applications, as well as Debian running Enlightenment on an old Presario 1246 laptop. I would like to thank all the speakers and the attendees for making of FLISOL Caracas a great event. More information about FLISOL Caracas is in my blog and the homepage (spanish). Pics are here.

I was at a client today when I remembered that David was due to arrive to Caracas circa 1430, so I called him to his brand-new-venezuelan-GSM-cellphone and I found out that he was in problems. In big, dark, awful problems. As I told in previous posts, he went to San Crist bal (near the border with Colombia) to visit his girl and know the city. He departed from Caracas in a bus on Friday night and arrived on Saturday morning. He spent the whole weekend there, and in Monday he was due to leave San Crist bal through the Mayor Buenaventura Vivas Military Airbase, in a civilian airplane. He arrived just on time for his flight, rushing to take the plane on time, and he was stopped by two so-called Inmigration Officials who demanded to see his passport. He recently got a new one from the Passport Office in Mexico City, so he had a brand new passport with the Venezuelan Entry Stamp (Mexicans don’t need a visa to enter Venezuela)
ONIDEX, the National Office for Inmigration. Praised for lots of people for fighting against Passport Corruption, kidnaps people in San Cristobal. The Officials told him that his passport was a fake, and they told him that, from this time on, he was under arrest. They made him leave the Airport quickly and took him in a civilian vehicle with no identifications. They started the trip to San Crist bal City (1 hour away from the Airport in a very scary road) and they kept telling him that his passport was fake (they claimed that the Passport didn’t have the watermarks, and stuff) Soon their position changed and a common practice in Venezuelan officials started: extorsion. They told him that there were “several ways to fix the problem”, and asked him what was in his backpack (which had a laptop, but he denied it): they were demanding money, and David wasn’t prone to do that.
National Guard. Doesn’t have a /proc/clue about anything. They drove David to the Inmigration Office in San Crist bal and told him that they “made” some tests to the passport and they determined that the passport was fake. They told David that their “Commander” told they that he should be deported into Colombia, since he was a “very important member of a Cartel that they were willing to capture”
DISIP: the Venezuelan Political Police. They were helpful in this episode, but they’re not angels. Finally, hours after that charade of Officials telling him funny laws, stupid suppositions and wanting him to give them money, they left David in the Bus Terminal in San Crist bal, where he was able to talk to Ana, from Debian Venezuela, which picked him up and went back to the Airport. The two “Officials” weren’t there anymore, and he had a last problem with the National Guard which had stupid suppositions about his trip, and stuff (this is common, but at least it’s not illegal) before he was able to get into the plane. Being in Caracas, I was really worried. Between Ana, my mother and me we were able to move several people to check David status in the Airport (my mother called four members of the Venezuelan FAA, I called the Venezuelan CIA and Ana called the Vicechancellor) and this probably saved David’s life and money, since the “Officials” might have been scared by the movement of people caring about David. He’s safe now, back in Caracas, in home. He has all his stuff and his health is OK (yet he was scared when he arrived, which I fully understand) Why am I writing this post? I’m really concerned about what happened to David, and I want to make a public statement on this. Somehow, we venezuelans got used to this kind of practices in our Country. Probably 40 years of pseudo-democracy with messy governors helped, maybe it’s a matter of education and culture. I don’t really care about this, but I do care about what they’re doing with the people in my Country. Today, President Chavez declared that a dance against corruption was going to start. I think it’s the fourth time he’s making something about corruption, and he’s failing. There’s a gang of people in San Cristobal, Mr. President, who tries to rob and scare the visitors. They should be in jail. They arrested somebody illegally. It’s forbidden in our Constitution to arrest somebody without having proofs to do it (we have one of the best Constitutions in the World, experts say, but we don’t have people who obey it). If you’re going to arrest an alien, you need to have a Public Prosecutor in place, writing down a legal document and guaranteeing the safety of the detained. This people should be in jail.
Stop Corruption! Though, as my mother later said to David, two awful Venezuelan people kidnapped David, but probably twenty Venezuelan people quickly acted to find out what was happening. We might be a Third World Country, but we use to be able to get out of our problems by our own. And, hell it’s rewarding. I want to apologize in the name of all venezuelans to David and to make a clear statement against corruption: we venezuelan won’t tolerate more corruption. We’ve grown between that, buying places in a line to get our National ID Card, paying one million bolivares to get a Passport appointment. We’re sick of this. This is not the great Venezuela we live in. Please, losers, shoot yourselves. Or eat rat poison, whichever makes less stains.

I had a bit of a surprise today when I sat down to read this weeks DWN and found my own name and a link to and old post about some Lightweight Debian Archive Scripts that I had written in October last year staring back at me from the second sentence! As chance would have it, just last week I updated these scripts to use reprepro, as suggested in many of the comments that I received after writing the last post. I hadn’t got around to making the new scripts public yet, so I guess the link from DWN is a sign of sorts! My main motivation for updating the scripts to use reprepro was to give myself the ability to have a testing, and stable archive as recently we’ve been dealing with .debs manually for testing as we didn’t want to put them into the archive for fear of prematurely updating the production hosts with untested code. I’d heard that reprepro had some nice support for moving packages between local repositories which sounded exactly like what I wanted. Reprepro was indeed fairly easy to setup and install. I backported reprepro 0.9.1 and installed it alongside my existing sbuild setup as documented in the original post. The changes to the scripts were fairly minimal, although again, they assume some things for my environment, and its likely that they’ll need some modification before being useful to you:

• process_packages assumes you have pairs of distributions named like ‘foo’ and ‘foo-testing’. Every package picked up from incoming/foo is automatically added into foo-testing after it is successfully built
• add-packages is mostly redundant, used only to add packages manually if the automatic insertion fails because reprepro has got itself confused

You can grab the updated scripts and my reprepro configuration from:

• process_packages - Runs from cron to build new packages and add them to the -testing archive
• add-packages - Manually adds packages
• distributions - Reprepro distributions file
• options - Reprepro options file
• pulls - Reprepro pulls file

The new setup is great and I’m very happy with reprepro and the new archive. The only thing I haven’t had time to look into properly yet is how to pull specific packages from testing into stable, but not everything. Ideally I would be able to run a command like

reprepro <stable-dist> pull <pkgname>

If I could get that working then it would be awesome.

This Wednesday I'm heading out to Chicago to visit RailsConf. The full schedule has now been posted, and it looks like a pretty absorbing affair.It amuses me that US-based conferences often have evening sessions timetabled. In Europe, I've always preferred to leave evenings empty so people can eat, drink and socialise. But in big city hotel-land, I can understand the appeal of timetabling more of the day.Here's a selection of the talks I'm most interested in hearing: Internationalizing Rails (almost the number one question we got asked about Rails when we taught our tutorial at XTech), anything related to testing, Rails Application Optimization Techniques and Tools, Rails Deployment. Oh, and some guy's talking about his work at the BBC.I've just finished writing an article for IBM developerWorks introducing Rails to DB2 users. (I had surprisingly little problem getting DB2 up and running in a matter of minutes on my Ubuntu box.) IBM's DB2 integration for Rails is a great development, and it's nice to see them pushing forward with Rails as a platform. I'm hoping they'll post the article up before RailsConf starts.
Given a bit more hacking time, I'll be intrigued to see how DB2's XML support can play with Rails.I'll be staying at the conference hotel, and am very interested in meeting up with anyone building Rails apps of a decent scale, or looking for a project to work on. Do drop me an email.