eBox is growing a lot lately, and updating screenshots is always a cumbersome task, so, prompted by a workmate, I decided to do something to get automatic screenshots. The first step was to get an automatic way to install eBox from scratch to have something to get screenshots from. That was quite easy using the Xen facilities available in Debian. I only had to write a role for xen-create-image and now I can get an eBox up and running in a couple of minutes executing a simple “xen-create-image –role ebox –hostname ebox-screenshots”. Using eBox introspection capabilities and some Perl magic, it was quite easy to autogenerate a list of the URLs that we should take screenshots from. Once we had the URL list and a running eBox all we have to do was to have a browser logging in the eBox web interface and loop through the URLs taking screenshots. To achieve that I resorted to Ruby, Konqueror, KSnapshot and DCOP, and it took less than 40 lines of code to do it. The first problem was spawning Konqueror and KSnapshot instances and getting a DCOP reference for them, but that was easily done using Ruby’s fork, which is quiet convenient:

konqpid = Kernel.fork exec("konqueror --geometry 1024x768+0+0")
ksnappid = Kernel.fork exec("ksnapshot")
konqwin = KDE::DCOPRef.new("konqueror-# konqpid ","konqueror-mainwindow\#1")
ksnapshot = KDE::DCOPRef.new("ksnapshot-# ksnappid ","interface")
The second problem I faced was logging in the eBox interface, I had to enter the password and submit the form. Fortunately Konqueror’s DCOP interface offers a really nice “evalJS” function:

konqwin.openURL("https://# ip /ebox/")
konqhtml = KDE::DCOPRef.new("konqueror-# konqpid ","html-widget1")
konqhtml.evalJS("document.getElementById('credential_0').value = 'ebox'")
konqhtml.evalJS("document.login.submit()")

and we’re in! Now we only have to loop through the urls taking the screenshots. I had a little problem because ksnapshot expects you to click the left mouse button before taking the screenshot. Fortunately that could be easily fixed using xte from the xautomation package to simulate the click.

urls.each url
konqwin.openURL("https://# ip /ebox/# url ")
name = "# konqwin.caption.split(/-/)[1].strip.gsub(/ /,'_') "
ksnapshot.slotGrab
system("xte 'mousemove 100 100' 'mouseclick 1'")
ksnapshot.save("# outpath /# name .png")

The code I pasted lacks a bit of initialization and some sleeps I put there to synchronize the stuff, but basically that’s all. I just love Ruby, KDE and DCOP

You still got some money left after buying all those Christmas presents? How about donating some of it for a good cause:

• Wikipedia

Wikipedia is entering 2007 as one of the 10 most visited websites in the world. That's a great proof of our success. However, with this success comes a new set of challenges and responsibilities. As we plan for the future of Wikipedia and all of the Wikimedia projects, our two most important goals now are the reliability of our content and the long-term sustainability of every project in which we are involved. To meet these goals we have a lot of work to do, so I am asking for your help. In the coming year, the Wikimedia Foundation anticipates dramatically increasing spending to keep up with server and traffic capacity demands, add new staff on the organizational level, improve our software and develop methods to better ensure high quality content, all while making progress toward our goal of giving free knowledge to everyone.

-- Florence Devouard, Chair of the Wikimedia Foundation

Creative Commons

It's once again time to show your support for Creative Commons. Digital technologies are connecting people in ways that were never before possible but that network is fragile. Creative Commons needs your support to help enable a participatory culture a culture in which everyone can actively engage in the creativity that surrounds us. We need your support to assure access to cultural, scientific, and educational content that has been pre-cleared for use by its authors.

Free Software Foundation (FSF)

Freedom is more precious than anything else we have and we need to protect it while we still can.

-- Eben Moglen in his FSF year end video appeal (OGG, 24 MB)

Electronic Frontier Foundation (EFF)

EFF is a nonprofit group of passionate people lawyers, technologists, volunteers, and visionaries working to protect your digital rights.

Tor: anonymity online

Tor is a toolset for a wide range of organizations and people that want to improve their safety and security on the Internet. Using Tor can help you anonymize web browsing and publishing, instant messaging, IRC, SSH, and other applications that use the TCP protocol. Tor also provides a platform on which software developers can build new applications with built-in anonymity, safety, and privacy features.

Or how about some organizations not directly related to Free Culture or Free Software?

• Amnesty International (AI)

Amnesty International (AI) is a worldwide movement of people who campaign for internationally recognized human rights. AI s vision is of a world in which every person enjoys all of the human rights enshrined in the Universal Declaration of Human Rights and other international human rights standards.

Red Cross

The International Red Cross (and Red Crescent Movement) is an international humanitarian movement whose stated mission is to protect human life and health, to ensure respect for the human being, and to prevent and alleviate human suffering, without any discrimination based on nationality, race, religious beliefs, class or political opinions.

I've just been trying to set up a MythTV system, I've had the hardware for this for a while but until my TV broke I hadn't found the time to work on it.

My planned hardware was a P3 system for both frontend and backend. However the Debian MythTV packages take over 250M of memory for each of the frontend and backend programs (more than 500M for Myth programs alone), combine that with some memory use for MySQL, for the X server, and for the rest of a functional Linux system and a machine with 512M of RAM does not perform well. Unfortunately it seems that most (if not all) desktop P3 systems only support 512M of RAM (at least none of my collection of P3 machines does). P3 server systems support considerably more RAM (I used to have a SMP P3 server with 1G of RAM), but a server class machine is not what you want in your lounge room. P4 systems support more RAM but also take more than twice the electricity of P3 systems. The extra electricity use is a waste for a machine that will probably run 24*7, and also requires extra cooling (and therefore probably louder fans which is bad for a machine used for playing audio). Given that I desire the capabilities of a P4 in terms of RAM support and electricity use that is not excessive, my options are one of the recent Intel Core CPUs or AMD64 (both of which apparently use less electricity - not sure whether they would use as little as a P3) or a CPU designed for low power use (maybe a Via). Either option costs more than I prefer to spend (I want to use a second-hand machine valued at $129).

Of course if I could get Myth to use less memory then that would allow me to run both parts of MythTV on the one machine. Please offer any advice you can think of in regard to minimising memory use for Myth on Debian.

Given that I am unlikely to be able to optimise memory use as much as I desire it seems that I will have to go to a split model and try using an existing Pentium-D server machine as the backend and the P3 as a frontend. The up-side of this is that I can turn off the frontend machine when it's not being used (the server runs 24*7 anyway). I will need to get more RAM for the Pentium-D machine, but I had wanted to do that anyway.

The next thing I will have to do is to write SE Linux policy for MythTV and find out why /usr/lib/libmp3lame.so.0.0.0 needs both execmod and execstack (and either fix the library or write policy to permit that accss). I'm happy to have a MythTV server program running on a stand-alone machine with unrestricted access to all resources, but when it's going to run on a machine that is more important to me I need to lock it down.

Now to the design of MythTV. It's unfortunate that they don't support better multitasking options. One feature I would like to see is the ability to play MP3s in the background and then have the music pause whenever TV is selected. When I watch a live show (such as the news) I would like to listen to MP3s before it starts and then at the end of the show (and maybe during commercial breaks) have the MP3 playing resume where it left off. This is exactly the traditional lounge-room functionality we are all used to, when a commercial starts you mute the TV and un-pause the CD player! The advantage of MythTV is that you could do both with a single button - or even automatically with advert recognition!

The method of selecting MP3 files to play is also a little cumbersome and the option to add a song after the current one in the play list doesn't seem to work. Also it's a pity that there is no convenient option to sort the music list by artist, genre, or sone name - there are grouping options but they are all separate and I would like to change the sorting via a single key press.

The startup sequence of MythTV regenerates some images (which takes a few seconds), it seems that this is something that could be cached for a faster startup. Also when I add new MP3 files to the store I have to manually request a re-scan. It's a pity that it can't just check the size and time-stamp of the directories which contain MP3 files and do an automatic re-scan if they change.

We had a tough time getting out of Encinitas yesterday. It seems like everything was going wrong. I'd forgotten to order food for a sick aunt and her family in New Jersey the night before, so I was scrambling with apologies and new orders all morning. Then, when we finally got out of town, we hit traffic for about an hour near Santa Ana. Blech! Fortunately, it's a nice stretch of California highway, and we could see Catalina Island off on the horizon to the west. Our Christmas-eve-eve (or Festivus, if you will) was spent at Maj's parents' house in Long Beach, with about 30+ Jenkinses plus various offshoots and friendly relations. They have a huge house near Long Beach/Belmont Shore, with a giant tiled patio area that goes right into the house. It was a good place for a party. Amita June got some nice gifts from family, including a screaming monkey doll dressed as Superman. Yeah, we already have one screaming monkey doll with an "I LA" t-shirt, so this new one will join our growing collection of screaming monkeys with poor sartorial taste. tags: christmas belmont shore festivus long beach amita june

Typewriters are broken Yesterday Amita June and I were playing with pyKeys, from the awesome pyToddler suite of tools, and I realized that typewriter keyboards are broken. I was showing her that when she pushed on the key marked "Q", a "Q" would appear on the screen. But that's not what happened: instead, a "q" appeared on the screen. That's not a bug in pyToddler, of course: that's just the way typewriters work. If you click on the button marked with a capital letter, it prints the lowercase letter on the paper or screen. To get the capital letter itself, you have to put the keyboard in another mode, or use a modifying chord key. I'm not sure why this is the case. It's clear why we use modifier keys to make capitals, since we type lowercase keys much more often. But why do keyboards show the capitals printed on the keys, then? tags: typewriter keyboard pytoddler pykeys

Free Culture Christmas If there's anything that gives a bracing jolt of Christmas cheer, it's 235 Creative Commons Christmas songs. Thanks to Uwe Hermann for compiling this fine list. tags: cc christmas open content music christmas song

I'm pretty happy about the way the 'Ten Web 2.0 APIs you can really use' episode of the LinuxCast came out so well. I'm not very used to talking for recording, and like everyone I don't particularly enjoy hearing my own voice recorded (do I really sound like that?), but it was fun to do, and I think it's a nice adjunct to the Ten Web 2.0 APIs you can really use article itself. tags: api podcast linuxcast linuxworld

Tijuana We're on a trip in Southern California this week, so we decided yesterday to go to Tijuana for the day. Maj, Amita June and I got out of the condo we're staying at in Encinitas early in the morning -- well, early for a vacation day -- and headed down to downtown San Diego. It's a trip of about 25 miles, with one tricky turnoff at highway 805 that seems to get me every time I visit this part of the world. Going into Mexico with a car can be a bit of a hassle. Many American insurers don't cover cars going into Mexico, and you'll often get asked about insurance when you cross over. There are some quick-and-dirty insurers on the border that can give you one-day insurance or other short-term coverage, but we have a rental car right now, and most (all?) rental car companies forbid with bitter, mouth-foaming forbiddance driving the car across the border. Fortunately for us, San Diego has a great way around this problem. Their trolley system goes from downtown San Diego right up to the border in San Ysidro. So, we picked it up in the Civic Center downtown, after a brief stop at the Downtown San Diego Vistors Center (to get tourist material), a shopping foray at a stall outside of the downtown Macy's (to get Amita June a pair of purple Crocs), and a side trip into the Westin for a bathroom break. Man, the Westin in San Diego has nice bathrooms! I'd heard about these trolleys in the past, and I guess I'd expected a kind of fancy tourist train, but it's really just the Blue Line for typical light-rail trains for San Diego and its suburbs. The seats were comfortable, but the train right-of-way went through some of the least scenic parts of Southern California I've ever seen. There's something about this corner of the United States of America that's depressing and end-of-the-world; I get the same feeling about other borderlands, like around Plattsburgh and Laredo. The end of the line for the trolley is completely opaque as to how get into Mexico. We wandered around past some ugly strip malls until Maj spotted a tiny oblique walkway that said "US PROPERTY SUBJECT TO SEARCH AND FINE" and in much smaller letters "a Mexico - to Mexico". We went through a wiggly walkway and across the no-man's land between the scary metal fence at the US border and the actual Mexican customs area. Nobody was there, but we got the feeling we were being watched. At the end of this weird walk was a very nice tourist info center, where we got some maps and brochures and such. Another 10 minutes walk through the streets up against the fence, we got to Avenida Revolucion, the central way through the Tijuana/Zona Centro. It had been a while since we'd been to a city with aggressive touts, so getting yelled at by people in shop doorways or in the street was a bit of a jolt. I hate the invasion of my anonymity, and I'm pretty reflexive about saying "NO NO NO NO NO" to everyone, but Maj is a little more practical. "If everything is run by aggressive sideshow barkers, you're going to miss out on everything," she points out. "Sometimes, you have to say 'yes', even if you don't like the way the question was asked." By the time we hit av. Revolution, we were pretty tired, so we stumbled into the first restaurant we found -- El Torito, a big nightclub with go-go cages, a mechanical bull, and a tattoo parlour on premises. I was attracted by the racy Absinth Mexico posters. "Family restaurant, family place," said the touts, seeing the baby in the stroller. Sure. The food wasn't bad -- I made the mistake of ordering the fish Vera Cruz, but the waiter made a smelly-face and suggested I get the shrimp instead -- and the cocktails at 1PM were generous and inexpensive. Probably the nicest part was a guy with a guitar who came over and asked to sing. "We've got enough music," said Maj, pointing at the ceiling where loud disco tunes were booming. But he was insistent, so we let him play, and he sang a loud Besa Me Mucho that drowned out the Ricky Martin and had us, the waiter, and nearby diners singing along by the end. Amita June was fascinated by the sounds, and clapped along when he was done. After lunch we walked down av. Revolucion, which had a dense layer of souvenir shops and discount pharmacies offering Viagra and Cipro without a prescription. One weird repeated theme was donkeys painted as zebras, hooked up to carts where you could sit and get your picture taken for $2. Some of the donkeys looked pretty good in their zebra stripes, but most looked pretty bummed out. I'm not sure why tourists would expect to get their picture taken with a zebra while in northern Mexico. It got a lot nicer as we got down to the end of the block, and we kicked ourselves for being so lazy when we passed Chiki Jai, a charming little restaurant with tiled walls and rough wood furniture. Much more to our tastes than El Torito. We stopped at the monumental Jai Alai courts, with a beautiful facade and a nice statue out front. Ducking into the Tia Juana's next door, we did a diaper-change break and had some chips and salsa and pineapple juice (and Dos Equis Dark on tap for Papa). Then we started walking back to the border. On our way back, we got off of Revolucion and walked over to the city's Cathedral. It was a much more realistic part of the town -- shopping and bars and restaurants, true, but less ridiculous touristy crap. When we got to the cathedral, there was a holiday street fair going outside, with games and food stalls and people laughing and walking around with kids. It felt a little more human than the pressure-tactics-dense world of Revolucion. On our way back to the border crossing, we dipped briefly into the red light district at the head of av. Constitucion (who knew?), then made the long slog back to the USA. We didn't get much in the way of souvenirs, but Maj bought a nice set of luchador dolls for her brother, complete with a tiny wrestling ring. The line at the border was huge, and we had to wait at the trolley station for another 30 minutes before one left for SD again. By the time we got back downtown, around 5:30, we really weren't up for a dinner in the San Diego/Gaslamp Quarter like we'd originally planned. So we piled into the car, paid the $20 parking to get out, and headed right into rush-hour traffic that led to the over-populated and under-networked North County (San Diego County). Would I take the trolley to Tijuana again? Maybe. Next time, I'd probably go to one of the stops between San Diego and San Ysidro, where there are free parking lots and a little less hassle. tags: tijuana san diego travel san ysidro

Hazards of vacation homes I was just rifling through the fridge here at the condo in Encinitas and found what I thought was a delicious Abita beer, the fine drink you have when you can't handle another hurricane in New Orleans. But it turned out to be an Abita Purple Haze, a gross raspberry-wheat beverage. Unfair! It has probably been stuck in the back of the fridge of this condo for years. Fortunately there was still another Stone Pale Ale, from the North County local microbrewery, to satisfy my lunchtime beer needs. tags: stone brewing abita beer

Emre Sokullu is writing about a hypothetical “GoogleOS” again. As a long time OS guy, let me be the first to say that this makes absolutely no sense whatsoever. For one thing, Google already has an “operating system”. It’s the web, and Google dominates the web, so why in the world would they give their main competitor such an obvious advantage by moving into its turf? Furthermore, Google no more needs an operating system in the traditional sense of the word than it needs an office suite in the traditional sense of the word. Releasing yet another Linux distribution isn’t disruptive—redefining what an operating system is is disruptive, and Google’s already doing that. Two predictions for 2007 that I’m fairly sure will hold up: 1. This won’t be the last of the GoogleOS speculation; and 2. there won’t be a GoogleOS.

I'm in the process of converting my Fedora/rawhide laptop to Debian.

On Fedora the AES encrypted filesystems deliver about 38MB/s read speed according to dd. On Debian the speed is 2.4MB/s when running Xen and 2.7MB/s when not running Xen. The tests were done on the same block device.

Debian uses a SMP kernel (there are no non-SMP kernels in Debian), but I don't expect this to give an order of magnitude performance drop. Both systems use i686 optimised kernels.

Update: As suggested I replaced the aes module with the aes_586 module. Unfortunately it made no apparent difference.

Update2: As suggested by a comment I checked the drive settings with hdparm and discovered that my hard drive was not using DMA. After I configured the initramfs to load the piix driver first it all started working correctly. Thanks for all the suggestions, I'll post some benchmarks of encryption performance in a future blog entry.

Although you've not behaved very good, Santa Claus has a present for you: the entire GNOME 2.16.2 desktop in experimental. All core applications are here, only some bindings are still missing. Which means it's time for the most courageous to test it. With the experimental APT source configured, it should be a matter of apt-get install -t experimental gnome-desktop-environment. Be warned, it's still rough around the edges:

• GTK+ makes liferea crash repeatedly;
• there are a few glitches in the file chooser;
• there is only one working icon theme (the default one);
• evince seems to crash on some files;
• the keyboard switching stuff doesn't work at all.

But there are also many pleasant speed and UI improvements in most modules. Unfortunately for many, it is currently only available on amd64. Now, if a group of reindeers could rebuild it on i386, the children would be much happier. Otherwise, I will take some time to rebuild them in an i386 chroot in a few days.

With the uploads of slrn 0.9.8.1pl1-23 and ion3-scripts 20061214-1 this evening I nearly finished my TODO list for etch. The only pending issue is a problem in the alpha smp kernel which was discussed on the debian-alpha mailinglist, but unfortunately there's no working solution yet.

Hardware sucks. It just totally and utterly sucks. I purchased a 300 GB hard drive (plus a 3.5" USB disk drive enclosure) roughly 8 months ago which I encrypted using dm-crypt and then used it as a backup medium. And now the disk has died. No, this is not a software problem (that would be easy to deal with), the hardware is simply dead (it's making funny "klck" noises all the time). Mounting it via USB (using the USB enclosure) doesn't work at all anymore. I connected the disk via IDE in a PC and was able to mount it just fine (with cryptsetup luksOpen and all). A simple ls works, too (at least in the top-level directory). So I tried to copy the data over to another medium, but when accessing certain directories or files the system simply hangs and I get tons of funny kernel errors (and the disk makes even more and louder funny noises). Great. Stupid as I am, I also put data on the drive which I did not have backups of on other media (mostly because the data is huge, e.g. conference video recordings, large amounts of Creative Commons MP3s etc). OK, so I managed to get at least some small parts of my data, but now the disk is completely dead, I fear. I'll try to convice the vendor to give me a new drive, but I won't let them get their fingers on my data, i.e., I will not send the disk to them (yes, even though it is encrypted; crypto can be broken). Any ideas what else I could do? A professional "data recovery" service is not an option either, they usually cost 1000-2000 Euros (in the best case). What do you do for storing huge amounts of data nowadays? The only viable option I can think of is a RAID-5 (mdadm) with 3 or more disks in a silent PC which I can leave turned on 24/7. Unfortunately that costs non-trivial amounts of money. CDs are too small and they don't last too long either; the same is true for DVDs. Sigh...

Yesterday, Philip Hazel released exim 4.64. I have just uploaded the packages to Debian experimental. If you want to try the lastest and finest exim, please check out the packages. Unfortunately, the release is too late for etch. Debian etch will release with exim 4.63. I mean, unless the release team decides to bend their rules very badly, which I really do not assume.

It's pretty cool to see that Technorati is going to be acting as an OpenID identity provider. That is, you can use a Technorati account as an identity on other sites (like Wikitravel). Technorati OpenID URLs look like http://technorati.com/profile/evanprodromou (say). Nice job, Technorati. tags: technorati identity openid

Encinitas Maj and I finished up our business in Los Angeles this morning, so we picked up stakes from her parents' house in Long Beach and drove down to my parents' condo in Encinitas. There was a time that a big chunk of my family lived in and around San Diego. My dad worked for a company called Peregrine Semiconductor, and my brother went to San Diego State University at the same time. Nate lived in Mission Beach, but my folks bought a condo in Encinitas, near Dad's work. Mom split her time between the house in Los Altos Hills and down here. Times have changed and everybody's drifted back up to the Bay Area (California), but my parents still own a condo on the cliffs over looking the Pacific Ocean. We all use the place for vacation time, although it's a lot easier for my brothers and parents in Northern California than it is for Maj, Amita June, and I in Montreal. Encinitas is a nice town with a really pleasant little main street on Pacific Coast Highway, great beaches, clean air and the sound of waves as you're falling asleep. We spent 4 days down here last winter, and really enjoyed it -- probably the most relaxing time of the year. We're going to spend a few more days here this week. Last year, in the late afternoon, Maj took Amita June into the pool in the condo complex. It's about 20 feet from the edge of a cliff over the Pacific, and as they were swimming around the pool, the sunset was making a red-gold luminous carpet out of the ocean below. We were close enough that we could see dolphins jumping in and out of the water. My baby and wife laughing over a dolphin-dotted sunset; jeez. Mom and Dad are selling the condo -- we're going to have to dodge realtors while we stay here -- so it's unlikely we'll get another chance to spend time in this particular condo again. Good to appreciate the time while we can. tags: encinitas san diego sunsets dolphins

Wikitravel on CNN The AP article about online travel guides made it onto CNN. The last time we had an article about Wikitravel syndicated ( Web Denizens Contribute to Do-It-Yourself 'Wikimania'), originally by Ellen Lee of the Contra Costa Times and published December 2005, it kept popping up for months. I figure this one will be out there for a while, too. tags: cnn wikitravel

Lessig Steps Aside I'm sad to see that Lawrence Lessig will be stepping down from his position as chairman for Creative Commons. As he announced in his latest "Lessig Letter", CC Changes, Joi Ito will be taking over the position. I don't think there's a better person to do the job. I hope that both of them continue to work for free culture in their own ways and through CC. tags: cc lessig joi ito

MyBlogLog TechCrunch is effusive about how great MyBlogLog is. I tried it out a bit, and I have a MyBlogLog account, but I found the user tracking script a little creepy and invasive. So I took it off. I'm not sure the value of the service outweighs the automated click-tracking. tags: mybloglog privacy cookies

Joey and Josselin, I'm calling you childish, not because you "stand" your principles, but because of the way you try to convince others that your principles are better. What you are using is:

• Insults ("lying", and for the record, FUD means twisting the facts and telling only half the story, and you are twisting the facts, as I detailed in my post; "should read constitution", which of course means to imply that I'm trying to deny you some constitutional right, which I'm not, I'm just asking you to stop spreading FUD)
• Blackmail and threats ("I'll stop contributing" [if dunc-bank is paying anyone], "reduced it much less than I could, probably much less than I should")
• Schadenfreude (the libpng mishap of the new maintainer; noone broke libpng intentionally. That is not a nice character trait, and pollutes the community)
• You seriously annoy people by going on and on and on, while everybody has already understood your opinion and arguments (and just happens to have a different opinion)
• Playing big words without actual content ("read the constitution 2.1.1", which btw. also states that you "must not actively work against these rules and decisions properly made under them"; your attempt at a GR failed, and you are working against these decisions on planet; I've however never said that you're oblieged to still work for Debian if you don't like the decision)

Actually, I'm by now convinced that it would have been better for Debian, if you would just have really left. Your behaviour in the last months makes you qualify as "poisonous people" (except these usually aren't big contributors). In my opinion, your infighting has harmed the project, it gives people the impression that being involved in Debian means mostly having to deal with such things. While it's not - Debian is still fun to me, although I have little time, and didn't manage to go to the last few Debian meetings here in Munich - all these fights are a nuisance to me. Even though I'm not actually affected by them (neither Dunc-Tank, because it doesn't demotivate me, nur the powerpc fights, nor some other fights on -private), the way they are dicussed on the Debian lists does annoy me. Again: if anything is cutting on my motiviation, it's your behaviour (this btw. includes the powerpc issue and fighters, too, especially Sven Luther who has probably sent as many mails as all others together, just reiterating the same things again and again). So if you are planning to become a Debian contributor, learn to use the "delete" button in your email clients, and don't be afraid to just delete whole threads. Certain threads are just a waste of time and bring negative emotions. So: Joey, Josselin: grow up, and accept that people sometimes have a different opinion, and you can achieve more by accepting that your opinions differ and just try to make the most of it anyway. Reminding them of your differences won't make them change their mind, nor will it make them like you any better. Fortunately, most people in the project (well, everybody except me?) seem to obey above "poisoned people" recommendations, and just ignored your posts on this issue. I didn't, and one of the reasons it that I feared Joeys first post could grab some media attention, and I felt there should be at least one post saying that, well, the debian-installer and some other RC bugs are probably also a cause of the delay, and it's probably not worth rehashing the dunc-bank issue. So far, I've only seen one news article citing aba that the delay was caused by the debian installer among others, no "the delay is due to dunc-bank demotivating all developers" FUD.

In part 1, I described the problem of software installation on Linux; in part 2, I’ll describe the solution we came up with at the recent LSB Packaging Summit. After reading through the comments to part 1, let me first point out that our goal is to create a vibrant third party software ecosystem around Linux—you know, like the one Microsoft has built around Windows. No, it’s not about imitating Microsoft. It’s about being competitive. A platform is only as good as the applications that run on it. Bottom line: Many third parties have built their businesses around proprietary software, and we can’t just ignore them. And “ecosystem” implies decentralized, which I argued in part 1 was a key tenet of open source development anyway, i.e., this should be playing to one of our core strengths. So, if your “solution” is to tell ISVs (independent software vendors) to give us their source code so the distributions can include it because that’s just how we do things, you can safely skip the rest of the post below. You’re simply not going to agree that any of this is a problem. Ok. Assuming our goal is to create a vibrant third party software ecosystem (and everyone still reading agrees that’s a good goal, right?), we have the following challenges. First of all, the distribution vendors are hugely invested in their existing package systems, and for the most part, those package systems work extremely well. As I said in part 1, “if [an application] is in your distro of choice, you re only an apt-get or a yum install away from running it.” (I’m saying it again here because some of the commenters apparently didn’t see that. The tricky bit is “in your distro of choice”, which by definition is not the case with third party software.) Furthermore, a variety of highly sophisticated systems management solutions are built above those package systems, such as Red Hat Network and Novell ZENworks. This is a pretty important consideration, because these days, the “software management problem” largely involves managing software across the entire infrastructure, not just a single system. What problem then? This: ISVs have thus far been reluctant to use the native package systems. Why? THEY’RE hugely invested in “package systems” for other platforms, and every Linux specific thing they have to support costs money. In a world where decisions are ruled by cost vs. benefit, this is a pretty important consideration too. How do ISVs handle this today? For the most part, they ignore the package systems on Linux and do their own thing. Trouble is, while doing their own thing gives ISVs the flexibility to work cross platform, it ultimately makes their products integrate poorly in the broader systems management context because the package systems know nothing about them. What is needed is a way to bridge the gap between what the distros provide and what the ISVs want. But how? First off, we have to understand what ISVs want. The answer is simple: ISVs want to treat Linux as a single platform, which means they want to offer a single package for Linux, much as they do for Windows. So, if one commenter is right that “[t]he problem is that people (including software distributors) believe there s such [a] thing as ‘Linux’ as a target platform” and that “[i]f you re distributing software for ‘Linux’ then it won t be simple to install it, ever”, well, then Linux is destined to suffer the fate of UNIX. I’m not ready to give up so easily. Several commenters suggested that what we need is a brand new package system. That’s a non-starter—for one thing, the distros aren’t going to be too keen on replacing something they’re hugely invested in, and if ISVs aren’t going for RPM today, why would they go for something different tomorrow? No, to find a way forward, we need an evolutionary step from where we are today. From there, perhaps we can do more, but even the first steps can be quite valuable in their own right. To help find those first steps, we put some of the leading minds in Linux packaging in the same room (including the maintainers of RPM at Red Hat and Novell and the authors/maintainers of APT, yum, alien, and klik) along with ISVs large and small, server and desktop. The discussion pretty quickly converged on constructing a single API that could be implemented across the various package systems, because APIs make for nice evolutionary steps and can, done right, mask underlying implementation differences. Question is, what do ISVs need in such an API? Limiting the scope is key here, because providing an API that spans all the functionality of, say, RPM and dpkg is overwhelming to the point of being unworkable, not to mention more work to implement, which in turn makes it less likely to get into the distros so that ISVs can count on it being there, the whole point of this exercise in the first place. Fortunately, the ISVs don’t really need much. At the most basic level, an installer just needs to be able to query the system to see if it’s LSB compliant, and if it is, what version of the LSB it’s compliant with; and it needs to be able to “register” with the package system, so the package system knows about it, including what files it has installed. And that’s really about it. Importantly, because we assume an environment that’s LSB compliant, we don’t have to worry about dependencies, because everything is covered by the single LSB dependency, and dependency management is 95% of the package systems right there. We still need minimal dependency support—components can extend the LSB, and applications can depend on those other components being installed—but we’re talking on the order of a handful of components, not the tens of thousands of components typical package systems have to deal with. We only had a day, so we obviously don’t have a complete solution yet. For one thing, we barely touched on the issue of uninstallation (should we allow applications to register GUI uninstallers?), and the issue of how applications go about changing the system configuration still needs discussion (in a lot of cases, the assumption of LSB compliance means the installer is going to be well behaved, but there’s undoubtedly corner cases to be explored). And it’s going to take time for the API to be implemented and put into widespread enough use that ISVs can depend on it. However, everyone in the room did come to consensus pretty quickly that this was an important problem, and that providing a simple API that provides the minimum necessary functionality was the way to go. Perhaps most exciting, the very people whose support are needed to put the API into widespread use were in that room, and active participants in the discussion too. Everyone is certainly motivated: The distros get more applications, which makes the shared platform more attractive; and ISVs get lower cost, which tilts the cost vs. benefit equation in their favor, making Linux versions more economically attractive and potentially opening new markets. Because this is just a start, the FSG is launching a Packaging workgroup to continue the discussion we started at the Packaging Summit. If you’re interested in this topic, I’d encourage you to join the mailing list and get involved. There’s still plenty to be done. Update: One more thing: In addition to making it easier for ISVs to better support Linux by simply extending their existing installation scripts, the API approach really comes into its own when you imagine it implemented in things like Autopackage and InstallAnywhere. Let the market decide!

Last year, Binary 2.0 Conference had held, and it also held again. There is a book named Binary Hacks (Japanese) that contains many things focused for low-level stuff. Unfortunately, it is a Japanese book, but the English TOC is available. Anyway, Takeshi Yaegashi's presentation is spreading over the world. Engadget's article is a good brief summary.

I have a fairly sophisticated Spam filtering mechanism setup, using MIMeDefang for SMTP level rejection of Spam, grey-listing for mails not obviously ham or Spam, and using both crm114 and spamassassin for discrimination, since they compensate for each other when one of them can't tell what it is looking at. People have often asked me to write up the details of my setup, and the support infrastructure, and I have decided to oblige. What follows is a series of blog posts detailing how I set about redoing my crm114 setup, with enough detail that interested parties can tag along. I noticed that the new crm114 packages have a new facility called mailtrainer, which can be used to setup initial css database files. I liked the fact that it can run over the training data several times, it can keep back portions of the training data as a control group, and you can tell it to keep on going until it gets a certain number of mails discriminated correctly. This is cool, since I have a corpus of about 15K Spam and 16K ham messages, mostly stored on my previous train-on-error practice (whenever crm114 classified a message incorrectly, I train it, and store all training emails). I train whenever crm114 and spamassassin disagree with each other. This would also be a good time to switch from crm114's mailfilter to the new mailreaver, which is the new, 3rd Generation mail filter script for crm114. It is easier to maintain, and since it flags mails for which it is unsure, and you ought to train all such mails anyway, it learns faster. Also, since the new packages come with a brand new discrimination algorithms which are supposed to be as accurate but also faster, but may store data in incompatible ways, I figured that it might be time to redo my CRM mail filter from the ground up. The default now is "osb unique microgroom". This change requires me to empty out the css files. I also decided to change my learning scripts to not send command via email to a 'testcrm' user, instead, now I train the filter using a command line mode. Apart from saving mails incorrectly classified into folders ham and junk (Spam was already taken), I have a script that grabs and saves mails which are classified differently by crm114 and spamassassin from my incoming mail spool and saves it into a grey.mbox file, which I can manually separate out into ham and junk. Then I have a processing script, that takes the ham and junk folders and trains spamassassin, or crm114, or both; and stashes the training files away into my cached corpus for the future. In subsequent blog postings, I'll walk people through how I setup and initialized my filter, and provide examples of the scripts I used, along with various and sundry missteps and lessons learned and all.

For those who've been hiding under a rock for the last few months: dunc-tank is the (controversion) project that was founded by the current DPL, Anthony 'aj' Towns. Controversial, because a lot of people feel that this goes against the spirit of Debian; and that by paying some Debian people and not paying others, we're creating a two-class system that we'd best avoid. And probably a bunch of other arguments, too, but that's not what this blog post is about. Me, I've been staying rather quiet in the whole debate. It's not that I don't have an opinion; rather, I haven't been voicing it, mostly because my thoughts on the matter are rather convoluted, and I'm not sure I can put them in words pretty well. But since the debate seems to be going in circles, and since I think that I might be able to help it out, I'll try anyway. If that doesn't bring the debate any further, well—can't blame me for trying. Do I think that paying people to do work on Debian is necessarily a bad idea? No, not at all. Quite to the contrary; by paying people to work full-time on Debian for a while, we will increase the amount of work that's being put into Debian, which can only be a good thing; and I can only applaud aj for trying to get this done, since it should be obvious to anyone involved that by proposing this, he only had Debian's best interests (i.e., improve our distribution) in mind. The obvious question, however, is who we'll be paying. After all, while I don't think it's necessarily bad to pay people, I do think that with money, you introduce the possibility for corruption; so at the very least, it should be strictly checked what exactly happens with the money that's being spent. Accordingly, the way in which you pay people—your whole set-up and procedures—should have a number of safeguards. Which brings me to my next point. Do I think that dunc-tank, as a project, is a good idea? No, not at all. Quite to the contrary; and I have a number of reasons for that. First, I don't think that deciding by committee who can get paid and who can't is the way to go forward. A committee will always be biased, no matter how diverse its membership. Even if you could find a committee that would not be biased—an impossibility by definition—then there will still be people who will not agree with the committee's decision. Are they right not to agree? Not necessarily. Is it bad to create such controversy if it could be avoided? Sure as hell. Moreover, by creating a project which populated by Debian people and which has the sole purpose of paying Debian people, it's fairly possible to create the impression as if all other ways to get paid to do Debian work would be unsound; that if a developer accepts money for Debian work without going through the committee, he's somehow doing something morally wrong. This would be very bad indeed; I know for a fact that there are cases of people paying Debian Developers to work a few days on a certain package so that a certain bug which impedes their business is fixed, or so that a certain version of a random package which this company is using, is uploaded into the archive. I've done this myself in the past (although I can't disclose more, since the person who paid me requested to remain anonymous); I know of at least two other Debian Developers who've done so as well. So what's the alternative? Not paying people is one, but it would not necessarily be the best. One alternative, which the FreeBSD people have been doing, is just "no organization at all". People who want to get paid just put up a web page with the amount of money they would like to have, and it's up to the community to decide whether their request is valuable enough for it to actually be paid. To some extent, this is also what the Dunc-Tank committee is doing, except that as FreeBSD does it, there is no committee in between. The problem with this approach, however, is that if we all go ahead and ask for money at the same time, it'll not only make us look like a bunch of beggars; it's also very likely that nobody will actually be getting any money, because there's so many "good causes" to choose from that there might not be enough donations to fund them all. Another proposal was one made by Manoj Srivastava, before word got out that dunc-tank would be started (or about at that time). As it was made on the debian-private mailinglist, I can't say much about that; suffice to say that I feel his proposal was overly bureaucratic, and had problems that were similar in nature to the problems I feel exist with dunc-tank. So do I have any answers which might get us a way out of the current status quo? Nope, afraid not. But allow me to make this final observation: In early 2005, a bunch of people decided that Vancouver was a nice place, and travelled there. As it happened, they were all Debian people. What a coincidence. Since they were all there anyway, they decided to talk a bit about Debian, and as a result, came up with some plan for its future. This plan, too, was rather controversial. When I initially read the first few paragraphs, I suddenly fell upon this paragraph which seemed to say that "we suspect we won't be releasing with anything but i386, amd64, and powerpc for etch". Things suddenly turned kinda red from then on. And it wasn't the wine that I hadn't been drinking. I could imagine that what I felt back then is somewhat similar to what some other people felt when they first heard about the suggestion to pay people for Debian work; and this realization is the reason why I haven't been dismissing their arguments or thoughts, much like I've seen other people do. However, I cannot say that I feel the current crisis to have many parallels with the Vancouver one; this is mostly because in the current case, people on both side of the argument seem to be building up walls around themselves, to help them ignore whatever the other side was saying. In contrast, when someone suggested something which had the potential effect of throwing most of the hard work I'd been putting in for Debian during the last five years now down the drain, I went to talk with them—after the heat had cooled down, anyway. Unfortunately for me, the result is exactly the same; for all my effort, I haven't been able to prevent m68k to be kicked out of the release; and to say I'm not happy about this would be an understatement. But at least this way the port has been given a fair chance; if I hadn't been talking to the people on the other side of the argument, I would still be talking about Steve Langasek as if he were the devil himself, and m68k would probably have been kicked out of Debian—along with a bunch of other architectures. At least I got that bit fixed. I guess what I'm saying is that people, rather than farting in eachother's general direction, should be working together in finding a compromise. Sure, that means making concessions, and agreeing to something you'd rather not agree to. But at least this way, the thing you have to agree to isn't as bad as what it could be. And trust me, people will do the stuff you don't want them to do, with or without your agreement. Hell, they've already started. Whoa, long post. I'll stop preaching now.

Wouter and Russel: Wouter is right that 0.0.0.0 (or 0 in short) is not an alias to 127.0.0.1 (aka localhost). There are interesting things about specifying IP addresses though. As Matt said in a comment to Russel’s entry, 0.0.0.0 can be shortened to just 0. That’s right and also, 127.0.0.1 can be shortened to 127.1 with any software which implements parsing of IP addresses correctly. Actually, 2130706433 (127*256 +0*256 +0*256+1) would be an alternate alias to 127.0.0.1. A different example would be http://1426782553/ or http://85.719193/, which should take you to my blog at http://blog.incase.de/ (at IP:85.10.249.89) - Except that my Apache doesn’t seem to be able to handle the request with that hostname.
Update: a working example is http://1208933736/ or http://72.974184/ which takes you to google.

In the finest tradition it is Christmas-time and my computer is dead. OK slight exaguration. I have two computers I regularly use:

desktop.my.flat

This is my main development machine, which I sit in front of and work at most days.

itchy.my.flat

This has several roles:

• xen
  • So that I can run Sarge for building security updates.
• caching name server
• backup server

(Both of these machines run Sid.) The backup server is the important use for this machine. Every four hours I do a full database dump + rsync of d-a.org, my personal server, and my desktop system.

The backup machine, itchy, is currently down because of a failing hard drive. I’ve ordered a replacement computer (a bit extravogent considering I could have got by with just a new disk drive). This will be my first 64-bit machine, and also my first machine with hardware support for virtualisation. The new machine should arrive on Saturday. Once it does I’ll rotate it in. The new machine will become my desktop. My desktop will become the backup, and the old machine with the failing drive will be donated to my Sister. (Who I was due to give a machine to already. I will need to get a new disk but that will be done shortly.) I think I’ll try sticking with Etch on the new machine, and possibly make the backup machine run it too. I guess I don’t need sid yet… Besides Xen makes new installations easy

Russell, 0.0.0.0 is not the same thing as 127.0.0.1. You may get the same behaviour in some cases due to some implementation details, but that does not mean that both are the same! 127.0.0.1 means "The host this software is running on". In contrast, 0.0.0.0 means "Any host"; it can be used in software as the C constant "INADDR_ANY". While it is true that connecting to 0.0.0.0 from software such as ping will yield the same result as connecting to 127.0.0.1, it is absolutely not true that listening to 0.0.0.0 (in a bind(2) kind of way) yields the same result as listening to 127.0.0.1. After all, in the latter case you will only accept connections that originate from localhost; in the former case, you will accept any connection, whether from localhost, the local network, or the big and evil Internet. I presume you will have at least some firewall in between, but still. That being said, however, 127.0.0.2 is an alias for 127.0.0.1. So is 127.0.1.0. And 127.255.0.1. And, really, anything in the 127/8 network range. Now that has no utility—there can be use in having a /24 reserved for loopback, but a /8? Who would ever need 16M ip addresses for the local host? Is there even an operating system that can keep track of that many IP addresses? Oh well. not as if I care. And, after all, IPv6 has fixed that little issue. Although it might be fair to say that they overcompensated there a little bit.