Thanks everyone for the helpful comments yesterday on Apache vs. lighttpd. I ve taken the first few steps towards improving things. I ve eliminated mod_php5, switched all PHP to FastCGI, and switched from the prefork to the event MPM. The event MPM docs say that it s incompatible with mod_ssl, but it worked, and googling for that turned up mailing list posts from 2006 saying it was fixed. The only real glitch was that egroupware inexplicably depends on libapache2-mod-php5. It works fine without it, but I had to create a fakelibapache2-mod-php5 that provides libapache2-mod-php5 to convince the system not to remove egroupware when I switched to the event MPM. I went from 28 Apache processes to 4 Apache processes plus an average of around 2 php5-cgi processes which, despite the name, actually do grok FastCGI. Three of my Apache processes now have an RSS of about 20M, and the other of about 30M. The php5-cgi processes are at around 40M. My old Apache processes ranged from 18M to over 40M. So there is some memory savings here, though not drastic. My Redmine processes, two of them, each use over 50M. Ruby on Rails still is leaving me with a bad taste. At least it s not a Java app; it seems a lot of those allocate 2GB before they ever start accepting connections. I ll see how this goes for awhile, but for now I doubt that moving to lighttpd (or similar) will yield enough benefit to make it worth the effort. But there may be some benefit in inserting a caching proxy in front of Apache, so I may yet do that.

So Tuesday was another very productive day seeing lots of things done. And actually today also started (and will end) like this. I am now of the opinion that we need such a kind of meeting of our team every 6 months, if only we could manage to get the time off from our real lifes. Sitting together, discussing, working on the same thing, being able to just call over the table and get a different opinion just gains so much that we are missing otherwise As you already read, Monday was our day to change the database layer while Tuesday left us with adding the v3 source formats. Which was declared finished today, somewhere around noon. The first picture in this blogpost shows our still to fix up/todo list from Monday evening. Besides the v3 format we also worked on other things, like the already mentioned lintian autorejects. This patchset is also fully committed and tested working since today, so we could start on other issues. We had lots of general enhancements of our code committed, from improving our logging to fixing some issues with expiring old files from the pool all the way up to reducing code duplicates all over the place. I must confess that we did not start right away working on dak today. . We had to start by using hightech to fix a broken shoe first. (Some glue and a heavy laptop. NOT MINE). :) Back to reality - we are currently 6 persons all working on various parts of the dak code. We still have the archive mostly disabled (basically one large dinstall run in the evening always) and are happy with this, as this allows us immediate testing without the risk of accidently destroying someones package upload. And we do test a lot, haven t seen that many python backtraces before in one place :) The third picture shows our todo list status from today. We still have a good number of points open, but as we have 6 persons working it does seem manageable to at least get a very good start on each of them. What we currently work on is ACLs in the archive, which will soon allow us to fix another issue the old DM code has but even more important will allow automated signing of packages built by buildds soon. We basically are able to limit such keys to only be able to upload for the one architecture they are allowed to, as well as forbid them to upload any kind of source. (Together with various other considerations around the setup of the buildds (like, for example, only buildds administrated by DSA can have that, but other points too), the handling of the keys, etc, this will lead to much faster uploads of the build binary packages). And will take a tedious work away from buildd admins (who will still have enough to do to deal with the general administration of it, the build failures, such things). Another thing we work on is the handling of arch all packages, in a source that produces arch all as well as arch any binaries. Currently, as soon as a new source is uploaded for one architecture, every architecture will see the new arch all deb. Which can lead to uninstallability of other packages. Not very nice. And another one of us is also working on our contents generation. Something started long ago, but not yet finished. Recently we did take big steps in this, getting our SQL queries for it down from about 10 minutes to something around 10 seconds. That is for one suite, one architecture, the full contents file contents. While this is SQL only and we still need to write it out to disk, it is a nice improvement. And to my right we have someone working on getting the Packges/Sources files data stored within projectb too, so we can also use our own python scripts to write those files. This will finally enable us to completly ditch apt-ftparchive. And finally, to my left site sits someone who is working on a dak policy wrapper, with which we can allow other groups then ftpmaster to control suites. The initial usage will be to restore the (oldstable-)proposed-updates handling with their own queues in front, as that is no longer part of our NEW handling script. In the future it will also allow easy handling of other situations, like suites for volatile or backports.org. Basically everything where the people running the archive software are different to those people actually setting the policy for the suite. Another nice detail is the ability to throw away uploaded binaries and only ever take buildd built binaries into the archive. We will not go to source-only uploads, as that will only lead to lots of untested uploads (sorry people, we are all humans, it WILL happen), but we can have them uploaded, do regular checks (including the lintian ones), then take the files from the buildds. (Of course there is a way to get things bootstrapped, the system behind this is flexible to allow for a bootstrap mode for a new architecture). (I have multiple pictures, will probably put some more up when I am back home).

Anand Kumria has an ongoing dispute with Exetel, the latest is that a director of Exetel has libeled him in a blog comment [1]. Having public flame-wars with customers generally isn t a winning move for a corporation. But doing so in the context of the blog world is a particularly bad idea. The first issue is that almost everyone who regularly reads Anand s blog will trust him instead of a corporation (Anand is well regarded in the free software community). So it s not as if accusing Anand of lying will gain anything. But when a director of the company starts doing this it makes the issue more dramatic and interesting to many people on the net. Now Anand s side of the story will get even more readers, of course Anand s side was always going to get more readers than Exetel I m sure that Anand s blog is more popular than that of Steve Waddington. I wouldn t be surprised if my blog was more popular than Anand s and now my readers will be following the Exetel saga for the Lulz. I m sure that I won t be the last person to comment on this. The most amazing thing is that Steve Waddington talks about having to pay to take the TIO complaint. So I guess that means I should start complaining whenever I get bad service from an ISP and cost them some money! I should have stayed with Optus and started complaining all the time when they caused me problems! One thing that Steve and people like him should keep in mind is that members of our community are not only heavy users of the Internet, we generally recommend ISPs to other people, and many of us make money working for ISPs. If you want your ISP to get good reviews and to be able to hire good staff then attacking people like Anand is not the way to go.

• [1] http://www.progsoc.org/~wildfire/aum/2009/10/28#falsuminuno

On Thursday I will be helping to kick start a debate about some of the possible differences (or not) between free and open source software at Eyebeam, which will be hosted by Upgrade. If you are interested but are not in NYC, there will be a live stream and chat.

On Thursday I will be helping to kick start a debate about some of the possible differences (or not) between free and open source software at Eyebeam, which will be hosted by Upgrade. If you are interested but are not in NYC, there will be a live stream and chat.

Colophon This site is powered by Jekyll and based on styles and markup by Tom Preston Werner. Comments are run by Disqus and hosting is by Dreamhost. Before starting this site, I had an Advogato diary for writing about software. I also have a personal journal (mostly interesting to my friends and family).

It s been awhile since I ve made a blog post, and it will probably soon be evident why. For myself, I am taking two college classes philosophy and gerontology this semester, and still working full time. That s busy right there. I m really enjoying them, and in particular enjoying the philosophy class. I will, at long last, graduate this December. Oliver is about 3 months old now. He s starting to smile at us more, make his cute little baby noises more, and is very interested in taking in all of his surroundings. He has his opinions about things, but isn t expressing them too loudly just yet. Jacob, on the other hand, is sometimes. Our person from Parents As Teachers was talking to him for his 3-year-old evaluation. She said, Jacob, are you a boy or a girl? I a kitty. Are you a boy kitty or a girl kitty? I just a PLAIN kitty. Meow. He seems to delight in catching someone saying or doing something in a manner he considers wrong. No, not like THAT! is heard a lot in our house these days. Or perhaps, No, not RAILROAD tracks. They TRAIN tracks, dad! Jacob s imagination is very active. Sometimes if it is time to go use the potty, he will insist that we all stop because a freight train is going through the kitchen, the crossing guard lights are flashing, so we have to STOP. He pretends to be kitties, runaway bunnies (he has a book called Runaway Bunny), and occasionally other things.

This was the view yesterday from just outside my hotel. Needless to say, my vacation is going quite well. I've taken many pictures, read many books, written many reviews, and haven't thought about work (paid or volunteer) at all for about a week and a half. I'm slowly starting to recharge, although after this summer, a nice long vacation was exactly what I needed and I'm glad there's more of it left. I'll be back on line a bit more in about another week. In the meantime, I wish everyone a happy, productive time if you're not on vacation and a relaxing time if you are!

I am at the Debian FTPMaster meeting in the LinuxHotel Essen currently. We arrived Sunday evening and will stay until 1st November, giving us a nice week of time to work on outstanding archive issues. Currently we are 4 persons, Mark and me as FTPMasters and Frank and Chris as FTPTeam members. 2 more FTPTeam members join us Wednesday evening, so we only miss 2 members of our team in the end. Monday started out very well, with us merging a 7000 (plus something) lines patch, completly changing our database handling code. While we do have the archive disabled since we started working on it (and haven t reenabled it yet), we had a surprisingly small amount of errors to fix up. Though we do still have to test various codepaths, so it will still be a bit before normal service is back. But yay, we are now with a database layer that actually understands types (and has various other advantages). Today started with us getting to the next intrusive patch set waiting for our attention, the source v3 format. At the time of writing this blog post the patchset is merged (with changes to the way it originally looked) and we are now cleaning up and fixing the introduced bugs. Though I just heard live the first rejection of a sample v3 format package, so that looks pretty good too. And while that bugfixing is going on we are also working on the lintian autorejects and still have many other points on our agenda. I try to keep you updated, and we sure will send a report after the meeting, but I think it can already be said that every single cent spent here is worth it.

This was a Freedom of Information request to Companies House. The request was partially successful. I asked:

How many companies have been incorporated with the word co-operative in their name in the last few years and how many have been refused approval for the word co-operative?

After a bit of shuffling, the answer was:

we do not collate information on applications for company names that have been refused [...] As at the end of July there were 1,286 companies on the register with co-operative in their name. You may also wish to note that there were an additional 56 companies with cooperative in their name. [...] The figures for the companies that were incorporated within the last 3 years, and are still live, are 114 for co-operative and 14 for cooperative .

So, there has been about a 10% increase in cooperative names registered Companies House in the last three years, but there s no indication that Companies House are policing it effectively. Are you seeing more cooperatives around you recently? At least in the UK, the recession-support programmes offered to our co-op favour private companies over co-ops, but maybe new start-up co-ops can also access some funding? Have you met something called a co-op (apart from a computer game) which wasn t a co-op?

Remus Cernea was invited at Realitatea TV and the attempt to ridicule his and Mr. Pot rc 's (a counter roma/tzigane candidate) candidacies was obvious. The show was supposed to start at 23:00 and started 15 minutes later, all of that after a huge chunk of air time for a candidate from a big party.

In spite of that and the little time he spoke, Remus managed to avoid being trapped and managed to send his message accross.


For undecided Romanians, here are the recodings of the show:


http://www.youtube.com/watch?v=C02T-e2Gt28&feature=channel
http://www.youtube.com/watch?v=TSqRPBkA2GE&feature=channel
http://www.youtube.com/watch?v=dACUQnjM85Y&feature=channel

The Linux kernel regulator API requires that each system sets up the connections between the various voltage and current regulators in the system and the devices they supply, known as consumers within the regulator API. This is done using the struct device for the consumer device as the key for consumer access. This works well for things like platform devices which are generally always allocated at system startup but is not really usable with buses like I2C which only allocate the struct device late on during system startup. To help work with these buses Linux 2.6.32 will follow the clock API and allow the use of the dev_name() for the device instead. A new field dev_name has been added to struct regulator_consumer_supply which should be used instead of dev this is now the preferred mechanism. It is a simple string and so does not depend on any other initialization. For those backporting the relevant commit is 744ea1d0b59bf084f19559b8f199b644fbb0899c.

A try with OpenWRT 8.09 in April to setup a WDS with additional AP functionality and encryption, like described in the documentation, failed. So I keeped with my halfbroken solution running on DD-WRT, which is going a bit too commercial these days in my eyes.
Now I have found time to try OpenWRT again for this purpose, since the issue was fixed really fast. I just did setup a stock 8.09.1 installation and then dropped the following into /etc/config/wireless:

config wifi-device wl0
option type broadcom
option channel 5
option disabled 0
config wifi-iface
option device wl0
option network lan
option mode ap
option ssid OpenWrt
option encryption psk2
option key keyforclients
config wifi-iface
option device wl0
option network lan
option mode wds
option bssid 00:16:B6:19:63:C8
option ssid OpenWrtWDS
option encryption psk2
option key pskforWDS And guess what? It worked like a charm! So I could replace the odd DD-WRT boxes. Anyways ... does anybody have an idea where to find the sourcecode of anything > v23 SP1?

For software, the question whether some software can be considered free is relatively easy to decide: Check the license of the code against the list of free licenses of your trust (the OSI list, the FSF list or the judgement applied by debian-legal) and that s (mostly) it. When it comes to web services in the broader sense, including hosters, communication providers, social networks etc., this are not so clear. In this blog entry, I ll try to list the various requirements that a service should fulfill and which I usually consider before deciding to use a service.The RequirementsA) Availability of source codeComing from the Free Software movement, this is the first requirement that comes to mind. If a web service claims to be free, then I want to be able to inspect how it is working, suggest and implement improvements and set up my own instance. For example, Canonical s software housing setup Launchpad was itself not Free Software until very recently. Having access to the source code is also the reason why I prefer gitorious over github for git hosting.B) Access to my dataDepending on the kind of service, it might hold data from me or about me that are important, such as my emails on a webmail service. If I won t have a way to easily retrieve my data to back it up or to move to a competitor, I ll think twice before trusting my data to this service.C) Free data formatsIf there is accessible data, it should be in a format that I can easily get and use with a choice of (Free) software. Not sure if such a service exists, but an online office suite only supporting its own proprietary file format (possibly with its own proprietary offline office suit) would be no option for me.D) An APIFor an extensively used service, I will want to have the possibility to optimize my user experience by automation. This requires that I can use the service not only in person, but also from tools I write. A very good example for this is Twitter: Besides registration, there is no need to use their web site at all and still use all the features, using the tools I find most fit for a task.E) Choice of providersCompetition is good. If a service is only provided by a single company, and I become dependent on the service, I become dependent on the company. A few years ago, SourceForge was mostly the only provider of gratis, comprehensive software project hosting. By now, there are a few alternatives, such as alioth, savannah or google code, which is appreciated. Note that when it comes to the proprietary instant messaging services (ICQ, yahoo messenger, MSN messenger), there is no choice of providers: To talk to my buddies on a certain network, I am bound to that provider. The same applies to social networks (Facebook, StudiVZ, XING), and is one of the reasons why I shun these yet. F) Federation between providersIf there is a choice of providers, and the service in question provides some kind of communication, it is very desirable if I can interact with users of the service on other providers. A very good example for this is e-mail: People @gmx.net can send mails to people @aol.com, and I can have @joachim-breitner.de and still communicate with everyone. This was not always the matter of course: Before Internet e-mail was common, people on AOL could not get in touch with people at CompuServe. These days, the problem persists with instant messaging (Jabber a.k.a. XMPP being the notable exception) and social networks. Also sites like gitorious or github could support federation if they would allow to create branches of repositories that are hosted elsewhere, allowing their users to use the familiar interface with all repositories they work with.Analysis of a few servicesSo far the theory. Let s see how various services fare.e-mailBeing from the old times of the Internet, when everything was better, it does not surprise that this passes all requirements: There is a large amount of Free e-mail servers and clients to choose from, my data is easily moved via well-known protocols such as POP or IMAP and save in standardized format. There is an abundance of service providers, and they all talk to each other. One could argue that the openness of email causes problems like spam, but I d say that this might be a design issue, but not inherent to the above criteria.Pretty much the same analysis applies to Jabber/XMPP.ICQI joined when I was young and unwise. Now I am stuck to AOL to keep in touch with a relevent part of my online correspondence. To use it on the operating system of my choice, other people had to reverse engineer the protocol and any time, AOL could change how its servers work and I ll be locked out. I can not get in touch with people on other networks, my buddy roster is not easily backed up and restored, and no, we can not see the source of the ICQ servers. I d wish I could switch over to Jabber completely, but there are always a few important people who do not use that yet.Github/GitoriousAlthough gitorious is slightly older, github managed to become more famous. These web sites allow developers to publish their source code and other contributors to easily branch, work on their changes, and propose them for merging in the main code. They are based on git, so I can get my data. There is a choice of providers. I am not sure how well I can access their functionality (merge requests etc.) programmatically. As said before, they do not interact well, and only gitorious has published its code.Facebook/StudiVZVery disappointing. I have heard that Facebook offers you a rich API, but otherwise, they fail every category. This is one of the reasons why I am not to be found on these networks yet.Twitter/identi.caTwitter, similar to Facebook, excels in the category API , but fails in the others. A better alternative is identi.ca: The software behind it is free, I can download contact data etc. It even supports federation via OpenMicroBlogging, although that did not break through yet it, it seems.OpenIDAnother very good example for what I want is OpenID, a service that allows websites to accept a user s credentials from another site. This saves the user from having to create accounts everywhere. There are free implementations, multiple providers, I can easily switch between them. A great feature of OpenID that goes a bit beyond federation is delegation: On the start page of my homepage http://www.joachim-breitner.de/, I saved who my OpenID provider is. Now I can use http://www.joachim-breitner.de/ as my OpenID, and even if I change my provider, this stays valid! This is a feature that not even XMPP easily provides.ConclusionWhether a web service is really free is a complicated question and has a graded answer. One has to balance the benefits of using a web service against the freedoms it is missing. I hope that in the long run, services fulfilling most or all of these requirements will prevail, so that I ll be networking socially and blogging microly with a good conscience.

Brands are most often associated with the world of crass consumerism but they can play a key role in fomenting political change. Or so claim some pretty clever thinkers and activists and they will be giving talk about the importance of branding for democratic politics, this Monday at 7 PM, at the Change You Want to See

Please join us this Monday, October 26th as we continue our series on Symbols, Branding and Persuasion with an exploration of branding in the context of electoral and legislative politics. We ll start with a presentation by media theorist Stephen Duncombe, author of Dream: Reimagining Progressive Politics in an Age of Fantasy and the forthcoming Branding the New Deal. Afterward Jessica Teal, design manager for the Obama 2008 presidential campaign will join Duncombe for a conversation via video skype. Like it or not, propaganda and mass persuasion are part of modern democratic politics. Many progressives today have an adverse reaction to propaganda: ours is a politics based in reason and rationality, not symbols and fantasy. Given our last administration s fondness for selling fantasies as reality, this aversion to branding, marketing and propaganda is understandable. But it is also na ve. Mass persuasion is a necessary part of democratic politics, the real issue is what ethics it embodies and which values it expresses. Looking critically at how the Roosevelt Administration tried to brand the New Deal and how the Obama campaign leveraged principles of marketing and advertising gives us an opportunity to think about different models of political persuasion.

Brands are most often associated with the world of crass consumerism but they can play a key role in fomenting political change. Or so claim some pretty clever thinkers and activists and they will be giving talk about the importance of branding for democratic politics, this Monday at 7 PM, at the Change You Want to See

Please join us this Monday, October 26th as we continue our series on Symbols, Branding and Persuasion with an exploration of branding in the context of electoral and legislative politics. We ll start with a presentation by media theorist Stephen Duncombe, author of Dream: Reimagining Progressive Politics in an Age of Fantasy and the forthcoming Branding the New Deal. Afterward Jessica Teal, design manager for the Obama 2008 presidential campaign will join Duncombe for a conversation via video skype. Like it or not, propaganda and mass persuasion are part of modern democratic politics. Many progressives today have an adverse reaction to propaganda: ours is a politics based in reason and rationality, not symbols and fantasy. Given our last administration s fondness for selling fantasies as reality, this aversion to branding, marketing and propaganda is understandable. But it is also na ve. Mass persuasion is a necessary part of democratic politics, the real issue is what ethics it embodies and which values it expresses. Looking critically at how the Roosevelt Administration tried to brand the New Deal and how the Obama campaign leveraged principles of marketing and advertising gives us an opportunity to think about different models of political persuasion.

Over the past couple of months the machine which hosts the Debian Administration website has been struggling with two distinct problems:

The dreaded scheduler bug/issue

The machine would frequently hang with the messages of the form:

Task xxx blocked for more than 120 seconds

This would usually require the application of raised elephants to recover from.

OOM-issues

The system would exhaust the generous 2Gb of memory it possessed, and start killing random tasks until the memory usage fell - at which point the server itself stopped functioning in a useful manner.

Hopefully these problems are now over:

• Bytemark have generously increased the memory installed upon this (donated) machine from 2Gb to 4Gb.
• I've cleaned up some of the site code and switched to using a combination of nginx & apache 2 to serve the site. (I've documented this upon the site itself : Speeding up dynamic websites via an nginx proxy.)

The combination of these two changes should resolve the memory issues, and I've installed a home-made 2.6.31.4 kernel which appears to have corrected the task-blocking scheduler issue. ObTitle: Bridget Jones: The Edge of Reason

I ve just added the Wordpress Minify [1] plugin to my blog. It s purpose is to combine CSS and Javascript files and to optimise them for size and it s based on the Minify project [2]. On my documents blog this takes the main page from 313KB uncompressed, 169KB compressed, and a total of 23 HTTP transfers to 306KB uncompressed, 117KB compressed, and 21 HTTP transfers. In each case 10 of the HTTP transfers are from Google for advertising. It seems that a major obstacle to optimising the web page load times is Google adverts of course Google has faster servers than I do so I guess it s not that much of a performance problem. The minify plugin caches it s data files and I had to really hack at the code to make it use /var/cache/wordpress-minify a subdirectory of the plugins directory was specified in many places. deb http://www.coker.com.au lenny wordpress
I ve added a wordpress-minify package to my repository of Wordpress packages for Debian/Lenny with the above APT line. I ve also got the following packages:
adman
all-in-one-seo-pack
google-sitemap-generator
openid
permalink-redirect
stats
subscribe-to-comments
yubikey The Super Cache [3] plugin has some nice features. It generates static HTML files that are served to users who aren t logged in and who haven t entered a comment. This saves significant amounts of CPU time when there is high load. The problem is that installing this requires modifying the main .htaccess file, adding a new .htaccess file in the plugins directory, and lots of other hackery. The main reason for this is to avoid running any PHP code in the most common cases, it would be good for really heavy use. Also PHP safe mode has to be disabled for some reason, which is something I d rather not do. The Cache [4] plugin was used as the base for the Super Cache plugin. It seems less invasive, but requires the ability to edit the config file. Getting it into a shape that would work well in Debian would take more time than I have available at the moment. This combined with the fact that my blog will soon be running on a system with two quad-core CPUs that won t be very busy means that I won t be packaging it. If anyone would like to Debianise the Cache or Super Cache plugin then I would be happy to give them my rough initial efforts as a possible starting point. I m not planning to upload any of these packages to Debian, it would just add too much work to the Debian security team without adding enough benefit.

• [1] http://wordpress.org/extend/plugins/wp-minify/
• [2] http://code.google.com/p/minify/
• [3] http://ocaoimh.ie/wp-super-cache/
• [4] http://wordpress.org/extend/plugins/wp-cache/