Some of the more powerful tools combine data banks of information with robust technological features. One such tool is the Cause Caller developed by Fred Benenson, an ITP graduate student who developed it for his thesis project. Here is some more information and make sure to check out the blog for even more:

The promise of a truly participatory democracy has never been clearer than digital media has made it today. The unique distributive nature of the Internet provides a platform that has the potential to deliver on this promise by facilitating better group action at lower cost,thereby encouraging actual participation in democracy. There have been many meaningful developments in bringing better information and actions to citizens looking to use digital networks to serve their political needs, but many useful tools and databases remain proprietary and costly while others remain too general or ineffective for political action. Cause Caller uses open source technology to deliver on the promise of a participatory democracy by taking the hassle and difficulty out of organizing phone banks. Users of Cause Caller can create individual causes and associate new politicians to those causes.

These last two weeks most of my time has been sucked into getting Python 2.5 as default into testing. That’s done now. I made use of the block uploads thingie ftpmaster implemented for the release team to use. Basically, if your package could disrupt an “almost there” transition, the upload will be rejected. The blocks were in place for 5 days, which I think it’s acceptable. As long as we don’t end blocking stuff for very long, I think we should be fine. See the end of this post for more about this. Though it was quite a bit of work, I’m very glad I took care it myself, since now I really feel I’m 100% back to Debian, after the time I spent off for health reasons. Update: Oh, and I forgot to say: having control over britney has really really helped. Thanks a lot Joerg for that.

---

Dudesconf Tomorrow I’m leaving to Coru a for Dudesconf, which is a kind of Debconf-ES. I’m giving an introductory talk to Git, a semi-lighting talk about grep-dctrl (30 min.), and (gasp) a talk about Debian packaging with a VCS. We may have a Debian Quiz as well. I’m so looking forward to it, since many people who’ll attend are amongst my most loved ones, and I already missed last year’s since I wasn’t fully recovered yet. See you there!

---

Minirok and Sevilla One of the reasons I wasn’t fully back to release management during the past 5 months or so is because I spent as much time as I had doing development for Minirok. I don’t think I mentioned here before, but I was participating in a Free Software Contest for college students organized by the University of Sevilla, Spain. Such effort finally paid off, since Minirok was elected as one of the finalists. This means next week I’ll go to Sevilla, to make a presentation of the project, and who knows what more. ;-) I’m very excited.

---

Finally, more on blocking uploads This Python 2.5 transition was the first time the block uploads feature was used, and there were a couple bumps along the way. In particular, a couple packages were blocked, when they shouldn’t have been (libqt4-ruby and evolution-sharp), and one needed package was not blocked, though Rene Engelhard thankfully spotted it very quickly (mono). The problem is it’s not completely straightforward to generate a list of all the stuff that could possibly affect the transition. What I did was to make a run of britney on an arch that had all the needed bits in place, and block all the packages that migrated together as a result of the hint. This fails in two ways:

• it can include stuff that is not needed for the transition, namely unrelated packages that happen to become candidates for testing by one of the their dependencies having migrated as part of the hint. This is what happened with libqt4-ruby and evolution-sharp.
• it can miss stuff that is needed, like packages that can migrate by themselves, so the main run prior to the hint migrates them, and hence do not show up as part of the hint, but that some package of the transition depend on. This is what happened with mono.

The second problem, though, can be fixed by parsing the excuses list and blocking stuff that some bits of the transition depends on. Easy enough. Yet, there are more cases when things can go wrong, for example a shlibs-bumping upload of a package (say, sqlite3) linked against by some package still needing a couple of builds (say, qt4-x11). For that, blocking uploads is not an option, since that’d be an insane amounts of packages that, furthermore, can be uploaded if they don’t bump shlibs, so I guess we’ll have to ask in the next release update that shlibs-bumping uploads are coordinated in -release too, at least when close to finishing a transition.

A few weeks ago, NYU hosted an interesting event about the future of the Internet, appropriately tittled The Futures of the Internet, the video of which is now available here. One of the panelists was Jonathan Zittrain (who recently wrote an important new book bearing the same name as the event) and during the talk he provided a few ideas about how geeks and developers can help secure the Future of the Internet. While I agree with a lot in fact most of his assessments about the state and fate of the Internet as he lays out in his book and his talks, his characterization of geek/hacker/developer politics is not one of them. Basically, one of Zittrain’s claims is that developers are not doing enough to save the Future of the Internet and it is their rampant, Atlas-like libertarianism, which is, in part, to blame (first made 37:20 minutes into the video for those who want to listen to the actual comments). They have little-to-no political consciousness, are too cool to care about the fine print and they don’t care about the broader politics of the the Internet because they assume that they can just hack around any sort of barrier and impediment. While we can, without a doubt, identify a strain of libertarianism among hackers, it is by no means representative of all of geekdom and in fact, is becoming more and more a worn out 1990s stereotype/clich as time passes than an accurate representation of what is a far more variegated set of ethics and practices among hackers (and I will soon publish an article on this topic). It also completely fails to capture the ethical spirit as well as sociological, and political workings of one of the most important strains of hacking free and open source software which not only powers most of our (open) Internet but which in fact has provided a pretty hefty ethical backbone by which to conceptualize one of the ways we should think about the fight for the future of the Internet. Ok, time for a rant now Geeks not only designed the Internet, an indisputably revolutionary medium, but also implemented, and continue to maintain it, and then in their copious spare time, also engage in fighting back the political, legal and corporate encroachment which threatens to limit the very revolutionary nature of the Internet (as Chris Kelty’s new book on Free Software argues). If these acts by geeks are not enough political action, then maybe the development of not just one, but multiple entirely open and free alternatives to the only two proprietary operating systems that exist today might be a political act that would satisfy? Many would agree that even simply using a free operating system is a political act. It would be better to claim that individuals, lawyers and other political actors are not doing enough to save the Future of the Internet, rather than imploring the already overtaxed geeks to set aside everything that they are already doing to do something even more. (end rant) It also seems that when it comes to political questions related to the Internet, net neutrality being the hot topic now, or fighting restrictive and problematic laws like the DMCA, one of the only groups of people (outside of lawyers and librarians) to actually understand and dissect the fine print (and geeks actually are pretty attuned to and like to dissect the fine legal print), to protest these unsavory laws, and to support the organizations who are doing something about it (like the EFF), are geeks and hackers. While many geeks are not necessarily keen on conceptualizing their labor in traditional political terms, or aligning their technical projects with a political affiliation, and yes would rather just be writing good code, they do fight for their productive freedom and this productive freedom just happens to relate to most questions and concerns related to an open, accessible, and tweakable Internet, built by the geeks, lest we forget What was perhaps most surprising was that he also seemed to think that geeks and developers have not turned to “apprenticeship,” nor policies and procedure to coordinate their development projects, unlike Wikipedia, which he considers a shining example that geeks should look towards as a beacon of policy that geeks should consider emulating in their projects (comments made answering my question). He clearly has not been hanging out with any Debian developers in the last 10 years nor has he gone through their New Maintainer Process In other words, he seems to think they are allergic to regulation due to their accentuated libertarianism, or are against structure because of their anarchism, neither which is remotely true. I think I found this characterization most ironic and problematic for before Wikipedia was even an entry on a Wiki, projects like Debian (and most other F/OSS projects) were transforming and changing to integrate normative procedures and policies that allowed a group of people to work together, scale, grow and deal with crises’. No, they don’t have the Wikipedia badge system, but that system is emblematic of Wikipedia’s own transformation into integrating its own normative procedures and policies for working together, not an example of an idealized policy system that other projects are too primitive to have evolved into yet. About one hour into the talk when questions opened up, I objected to his characterization, but given his answer back to me, I did not make much of a dent in his thinking. Another lawyer Tim Wu (who also wrote a wonderful book on the Internet) chimed in to give me some props and also made a good point that even if geeks are the only groups of people who would storm AT&T and know intimately about the importance of net neutrality, there is a lot of room for thinking about how to strengthen and improve the tactics and politics among geeks and developers so that we can ensure the type of open and generative Internet and set of technologies we value. As part of thinking and rethinking new strategies, it is as key to acknowledge and honor the past. In this regard, free software development has been pivotal both in terms of providing software (and making it is an important political act as is choosing to use free software over propriety software) and a set of important set of ideas that a lot of lawyers like Yochai Benkler and Lawrence Lessig have run with to make some important political claims of their own. So despite my rant above, which was a rant and thus exaggerates things to some degree, I do think there is much more that geeks and non-geeks can do, such as help translate these uber-geeky issues into less geeky terms (and actually this is already being done by some geeks as the work of Jelena Karanovic has shown, or translate the technical issues into new domains as the uber-geek Karl Fogel is doing with question copyright but first lets give credit where credit it due and recognize that labor is political

With Hans Reiser convicted for murder, some seem to feel that reiserfs is more or less dead. Jason Perlow writes a very strange article on ZDNet to which I'm replying to it mainly because he alludes that Debian so far has failed to react. First, default installations of Debian create ext3 and not reiserfs filesystems (Please correct me if I'm wrong. I've just recently installed a fresh etch, but I didn't specifically look at the fs.) And even if it were reiserfs (v3), I don't see why a reaction would be called for now. The stability of reiserfs has come up every once, before the whole murderer story begun, and that the interaction between the reiserfs developers (including, of course, Hans Reiser) and the kernel team were always difficult has also been known for a long time. This is the kind of reason where I think it's appropriate for Debian to take steps (i.e. switching to a different filesystem), not a single event, where it is not even clear yet how the reiserfs (v3 and v4) efforts will move on. On to the technical stuff: Perlow tries, but doesn't really arrive at understanding the issues he's writing about. Reiser 4 is discounted without a single remark on its technical merits (I can't comment either as I have not looked at it so far.) Why he discounts ext4 is not clear to me (because it is not ripe for production use yet — but that's even more true of ZFS and this Linux-NTFS thingy he rambles about further down...) He discounts JFS2 because it hasn't got a new release for several years (is that bad in a filesystem?) but then touts ZFS as a great idea with minor licensing problems, without speaking of patents which is where the real problems lie (not to mention the fact that the Linux ZFS port probably is much less tested than ext4 or JFS.) And in a final jump into fantasy-land he mentions that NTFS might just be ideal for Linux, and Microsoft is said to have started cooperating nicely with the Free Software world, so all licensing and patent issues are certainly going away Real Soon Now™. At least for Novell, these issues shouldn't be a problem, I guess. Not mentioned by Jason are btrfs (which has a quite tightly coupled network filesystem brother, crfs and is in a very early state of development), and hammer, which comes from the BSD world and currently lacks a Linux port. Both efforts are probably more likely to replace ext3 or reiser on Linux than both ZFS and NTFS: no patent issues, no license issues, and the development is actually done by a community and not a single company. Update: Julien Blanche ha a much more succinct response to Jason Perlow. More intersting to read than mine, too.

In the last week, Nicholas Negroponte gave this unfortunate interview decrying "open source fundamentalism" and hinting the possibility of a warmer relationship with Microsoft. Predictably, this has elicited an ongoing response by OLPC News and on the OLPC development mailing lists. Just a few days before Negroponte's statements hit the press, I gave a talk at Penguicon called Laptop Liberation where I talked about why I thought that OLPC's use of a free software operating system and embrace of free software principles was essential for the initiative's success and its own goals of education reform and empowerment. I've been saying similar things for some time. My main point boiled down to something that, appropriately enough, Nicholas Negroponte was fond of saying back when the project was still called the $100 laptop: an extremely cheap laptop is not a matter of if, but of when and how. This technology will define the terms on which students communicate, collaborate, create, and learn. These terms are dictated by those with the ability to change the software -- by those with access to computers, the source necessary to make changes, and the freedom to share and collaborate. Constructionism -- OLPC's educational philosophy -- is about putting powerful tools and control over those tools into the hands of learners. It is about learning through exploration and creation -- about shaping one's own educational environment. Constructionist principles bear no small similarity to free software principles. Indeed, OLPC's stated commitment to free software did not happen by accident. OLPC convincingly argued that a free system was essential for creating a learning environment that could be used, tweaked, reinvented, and reapplied by its young users. Through these processes, the XO becomes a force for learning about computation and an environment through which children and their communities can use technology on their terms and in ways that are appropriate and self-directed. We know that laptop recipients will benefit from being able to fix, improve, and translate the software on their laptops into their own languages and contexts. Much more importantly, however, are all of the uses for the laptops that OLPC has not -- and can not -- think up. OLPC is a powerful tool for learning, but ultimate power is only in the hands of those that can freely use, change, and collaborate in defining the terms of their learning environments. In its commitment to software freedom, OLPC chose not to be arrogant by assuming that it knows how its users will use their laptops. Flexible environments designed for constructionist learning and a free software platform protect against this arrogance. Constructionism and free software, implemented and taught in a classroom, offer a profound potential for exploration, creation, and learning. If you don't like something, change it. If something doesn't work right, fix it. Free software and constructionism put learners in charge of their educational environment in the most explicit and important way possible. They create a culture of empowerment. Creation, collaboration, and critical engagement becomes the norm. OLPC does not get to choose if educational technology happens. If we work hard at it though we might get to influence the "how" and the "who." Proprietary software vendors like Microsoft want the "who" to be them. With free software, users can be in power. What's at stake is nothing less than autonomy. We can help foster a world where technology is under the control of its users, and where learning is under the terms of its students -- a world where every laptop owner has freedom through control over the technology they use to communicate, collaborate, create, and learn. This, to me, is the promise of OLPC and its mission. It is the reason I've been involved and in support of the project since nearly day one. It is the reason I left Canonical and Ubuntu to come back to school at MIT to be closer to the then nascent unincorporated project. It is the reason that OLPC's embrace of constructionist philosophy is so deeply important to its mission and the reason that its mission needs to continue to be executed with free and open source software. It is why OLPC needs to be uncompromising about software freedom. As an adviser and sometimes contractor to OLPC, OLPC does not need to listen to me. But I hope, for all our sake, that they do.

Update: Richard Stallman and the FSF have published another essay on the same topic focused more on pure free software (i.e., less education specific) objections.

The recent announcements by McCain and Clinton of their support for a temporary repeal of the Federal gas tax make me sick. More on why later, but first, I want to put forth my idea. I think both Republicans and Democrats would like it -- as it's based on market principles and achieves a reduction in costs to the average household, while simultaneously helping the environment and reducing our dependency on foreign oil. But of course, it's courageous, and we don't have many politicians of that type anymore.

What we need is a large, revenue-neutral, gas tax increase. Now, before people go nuts, let's explore what this means.

Revenue-neutral means that it doesn't result in a net increase of monies going to the government. The increase in the gas tax rate is offset by a decrease in the income tax, tied to the cost of direct and indirect taxable gasoline each family or business consumes. So on day 1, if you cost of filling up at the tank goes up by $10 in a week, if you are an average family, your total paychecks also go up by $10. Your cost for receiving a package might go up by $1, and your paycheck goes up by the same amount. So you're no worse off than before -- if you're average.

Let's look at the pros and cons of this sort of plan:

• The economic incentive to be efficient consumers of gas is magnified. This will eventually lead to Americans having more money in their pockets, increasing market incentives for fuel efficiency, and a decreasing (or increasing slower) price of oil as demand slows.


• Economic incentives to use mass transit, live close to urban centers, or drive fuel-efficient vehicles are magnified. Likewise, the economic incentives to invest in mass transit and efficient automobiles are also magnified.


• As more efficient technologies come on the market, and Americans decide that they'd like to pad their bank accounts by hundreds or thousands of dollars a year, more sustainable and environmentally-friendly development patterns will emerge. Also, the price of oil will be kept low. Of course, people that choose not to change will, on average, be no worse off than before.


• Alternative choices to the automobile will have a greater incentive to develop. Think the return of a fast, speedy national passenger and freight network, greater mass transit options, etc.


• The marketplace will drive Detroit to love making fuel-efficient vehicles, because they will be the new profit centers.


• This sort of thing is known to work well in other countries around the world.

If we think more long-term, we see even more positive effects:

• The return to local agriculture and manufacturing. Due to lower transportation costs, local farmers and manufacturers will be able to undercut Walmart's prices due to the larger relative costs of Walmart's much-vaunted national distribution network. Unless, that is, Walmart starts buying local -- which is a good thing too. This is a good thing for American jobs.


• Keeping all that oil money in the domestic economy is a good thing for American jobs, too.


• Our businesses will have a jump start on being competitive in the increasingly carbon-regulated global marketplace.

As for the cons:

• Eventually this will lead to a net reduction in Federal revenues as efficiencies develop in the marketplace and people save money on gas. Corresponding budget cuts will be required. (A good thing, I figure)


• Implementing this all at once would be a shock to some people living inefficiently now -- those that are far above average. It would have to be implemented gradually to avoid being a shock to the economy.

Now, for the McCain/Clinton plan: it's a farce. Reducing the gas taxes means more efficient gas, which means more consumption of gas, which in turn leads to -- yes -- higher gas prices. Its real effect will be minimal, and is a terrible long-term policy. It charges tens of billions of dollars to the national credit card (which we, and our children, will have to repay) while achieving almost no benefit now. It's a gimmick through and through, and something that says loud and clear that neither candidate is on track for the "Straight Talk Express".

Update 4/29/2008: One potential solution for the problem of declining revenues over time is to periodically re-index the averages to mirror current usage. Assuming this does really lead to the expected drop in consumption, there is no sense in 2020 of paying people for how much gas they would have used in 2008.

I've got most of the build system / packaging stuff done for estron now. There's a basic Yelp manual pending completion of the Estron GNOME application, there are some refreshed screenshots, updated the website CSS and added an icon, updated Glade files using Gtk stock items for the buttons.

http://estron.alioth.debian.org/

It's grown into yet more packages from the one source:

ii  estron                0.7.0-1  data-centric development interpreter for GNOME

ii  estron-doc            0.7.0-1  data-centric development interpreter (help files)

ii  estron-gnome          0.7.0-1  data-centric development environment for GNOME

ii  libestron-dbg         0.7.0-1  estron debugging library

ii  libestron-dbi-driver  0.7.0-1  DBI driver for estron

ii  libestron-dev         0.7.0-1  estron development interface

ii  libestron-doc         0.7.0-1  libestron development documentation

ii  libestron-odbc-driver 0.7.0-1  ODBC driver for estron

ii  libestron-pg-driver   0.7.0-1  PostgreSQL driver for estron

ii  libestron0            0.7.0-1  data-centric interface support library

Only the docs are of any particular size - the interpreter is tiny. (estron-gnome is simply an estron file and companion glade file, run by the estron interpreter so it's Architecture: all like all estron applications.)

Lots to do still - upgrading the Gtk code, adding new features to the interpreter to fully support the development environment and finally putting some details into the user manual (estron-doc) - but at least the package builds easily and cleanly. Once I have a working estron-gnome setup, I'll put the POT file out for translation too (around 120 messages so far).

There isn't any need for debconf and things, this will use existing database configuration tools like mergeant - that needs work too because current estron example files have hardcoded database connection strings.

As I wrote some time ago, I did consider switching to a new blog software, the OpenSource Edition of MovableType. I’ve actually done that some 6 weeks ago and must say - it is one browser based application I am actually happy with for myself. (Good god, I must be really ill). I know that about all the things that changed in my blog, compared to my old blosxom based version, are actually doable with blosxom too. For some reasons I just never got around to doing it. Anyways, some things I still remember that changed:

• I got comments and trackbacks support. Not that anyone except spammers and myself did use trackbacks yet, but comments got used, especially in the last few ftpmaster and delegation related posts. Also, the Anti-Spam plugins you get with MT version 4.1 are pretty good. I haven’t had a spam trackback show up yet. They all directly go into the Spam area, which I sometimes just delete. Same for the comments. As I’ve set it up that all comments need approval (unless someone already made a non-spam comment so I set them “Trusted Commenter”) and also have to pass the spamfilter I haven’t had a problem with spam comments. And while I offer 7 different ways of authenticating yourself when you want to post a comment here, including the famous anonymous way with a captcha - only one spam comment got in yet.
• Comments and Trackbacks are automagically closed by a Plugin when the entry reaches an age of 30 days.
• I now can do “Scheduled” posts. Within blosxom I would need a cronjob that copies the file with the blog entry into the right dir, in MT I just select the posting time in the creation window for an entry.
• The category system, especially with having one entry appear in multiple categories, seems to be way simpler. Including the ability to easily generate per-category rss feeds.
• MT generates (in default setup) static pages. Only the comment part is not static (hehe, of course not).
• Editing no longer resets the published time, which blosxom based on file timestamps. I can edit as much as I want, the entries keep their time. Unless I manually change that time.
• MT also seems to be able to handle your whole homepage. And file storage. And cms like (with plugins, afaik). And whatnot. And is “multi-author” capable. And multi-blog.

What it not has is the ability to use a version control system for your posts. Of course not, its a web based editor which stores its data in a database. Oh well, lets stop here, basically - I just like it. :) Comments: 0

Guess I got your attention with that title? :) Anyways, britney is no longer. And with her testing died too. Ok, not really, but todays work was hand over all of testing and put it where it should have been for years - in the hands of the release team. For those not following the “inner” workings of stuff:

• Testing is the place where our next stable release gets prepared. It is managed by a software called britney, which was written by Anthony Towns.
• Britney was, until today, run by the ftpmasters. The release team had no way to change anything, besides their hint files. They couldn’t change code, couldn’t rerun it themselve if needed, nothing. (Somewhere in the past, for some period, some of the release team was able to do extra britney runs, AFAIK, but thats long ago).

As the above is surely not nice, especially as Anthony (and as such the one in ftpmaster who really fully knew britney) left the ftpmaster group, all of the code managing testing now lives with the release team. They now just prepare a file in a special format and then tell us (via a ssh trigger) to read that in. Which simply sets testing to the new state. This new way of doing things has multiple advantages:

• The release team can run britney whenever they want to. It no longer needs to wait for a cronjob at some externally defined time or ping an ftpmaster to do an extra run. That should be pretty helpful, especially when we are near to a release again.
• They can do code changes or even replace the current britney with a totally new one. (There seems to be a new one available that does the same as the current one, just way faster. Or even more. Which never seems to have got any comment from our side. Badbad. Well, now they can play with it if they want to.). As long as they output valid data for us to import, it is now completly their decision how it gets generated.

The above action did need surprisingly few code changes in dak. It basically was

• disable the cronjob for britney as user dak
• Add a script that the release team now pushes via ssh. This script reads in the above mentioned special file and sets the database.
• Change process-accepted to write the urgencies to a different directory. And no longer try to mkdir this directory, but warn if it doesn’t exist (or is not writeable by dak) and then use a temporary one for the run.
• Move the transitions.yaml to a different location and add a redirect in apache, so everyone can still access it from outside. (It has to live in a ftpmaster controlled directory, thats all).
• Eliminate a britney call from our daily cronjob.

Comments: 8

About gobolinux

I can suspend my laptop and it restarts happily enough in X, but playing video results in a strange green square. I've found that starting and killing another X server (like X :1, wait for the grey mesh and then zap it) fixes the problem and I can switch back to my original :0 X and play video again. I wonder if these notes on X suspend and video BIOS by Matthew Garrett explain it.

About debian

Why on earth did someone change the .changes format so swiftly and why does a change that breaks a common upload process (build in a VM, sign on a stable system) only warrant a -devel-announce paragraph under the headline Misc Development News (#6)? "Small news" - my foot!

Since I didn't find a similar tool for Linux, I've hacked together a tiny tool to do BPM counting. The usual stuff: tap any key with the music to get a BPM estimation. There is no music analysis done.To achieve better results, my tool uses an exponentially weighted average. It also computes a confidence interval. So if the song changes tempo (and quite some jazz songs do) or doesn't keep the tempo constantly (live music with human drummers often shows some tempo drift. It's just the drum computers that keep their speed very well...) - the tool can handle that quite well, while also not being overly sensitive to tapping errors (as you can see in the image).I figure there might be even better estimation tricks (e.g. doing some error detection and removing them), but this one was just very simple to implement. And by looking at the graph and the error interval it's quite easy to check that you didn't any recent mistakes and the result is okay.The Python code is about 158 lines SLOC, and a small UI description generated with Glade. It's built with PyGTK and uses Cairo for the plots, so it should be very platform independant and even run on Windows.Download will be available sometime when I get around to clean it up a little. There is for example still the code in there to compute plain averages, which can't handle songs with multiple tempos at all.[Update: it now also has a tiny 'flasher' icon that will show you the current tempo estimation. This is great for checking the quality of the result - if it keeps on flashing in sync with the music after you've stopped tapping, it's good. I was also pointed to GNU GTick which is a full featured metronome application and also has a tap button.]

There’s been some controversy generated over my use of the terminology of “Organic” and “Non-Organic” Open Source. Asa Dotzler noted that it wasn’t Mozilla’s original intent to “make a distinction between how Mozilla does open source and how others do open source”. Nessance complained that he didn’t like the term “Non-Organic”, because it was “raw and vague - is it alien, poison, silicon-based?” and suggested instead the term “Synthetic Open Source”, referencing a paper by Siobh n O Mahony, ” What makes a project open source? Migrating from organic to synthetic communities”. Nessance referenced a series of questions and answers by Stephen O’ Grady from Red Monk, where he claimed the distinction between the two doesn’t matter. (Although given that Sun is a paying customer of Red Monk, Stephen admits that this might have influenced his thinking and so he might be “brainwashed” :-). So let’s take some of these issues in reverse order. Does the distinction matter? After all, if the distinction doesn’t matter, then there’s no reason to create or define specialized terminology to describe the difference. Certainly, Brian Aker, a senior technologist from MySQL, thinks it does, as do folks like me and Amanda McPherson and Mike Dolan; but does it really? Are we just saying that because we want to take a cheap shot at Sun? Well, to answer that, let’s go back and ask the question, “Why is Open Source a good thing in the first place?” It’s gotten to the point where people just assume that it’s a good thing, because everybody says it is. But if we go back to first principals maybe it will become much clearer why this dinction is so important. Consider the Apache web server; it was able to completely dominate the web server market, easily besting all of its proprietary competitors, including the super-deep-pocketed Microsoft. Why? It won because a large number of volunteers were able to collaborate together to create a very fully featured product, using a “stone soup” model where each developer “scratched their own itch”. Many, if not most, of these volunteers were compensated by their employers for their work. Since their employers were not in the web server business, but instead needed a web server as means (a critical means, to be sure) to pursue their business, there was no economic reason not to let their engineers contribute their improvements back to the Apache project. Indeed, it was cheaper to let their engineers work on Apache collaboratively than it was to purchase a product that would be less suited for their needs. In other words, it was a collective “build vs. buy” decision, with the twist that because a large number of companies were involved in the collaboration, it was far, far cheaper than the traditional “build” option. This is a powerful model, and the fact that Sun originally asked Roy Felding from the Apache Foundation to assist in forming the Solaris community indicates that at least some people in Sun appreciated why this was so important. There are other benefits of having code released under the Open Source license, such as the ability for others to see the implementation details of your operating system — but in truth, Sun had already made the Source Code for Solaris available for a nominal fee years before. And, of course, there are plenty of arguments over the exact licensing terms that should be used, such as GPLv2, GPLv3, CDDL, the CPL, MPL, etc., but sometimes those arguments can be a distraction from the central issue. While the legal issues that arise from the choice of license are important, at the end of the day, the most crucial issue is the development community. It is the strength and the diversity of the development community which is the best indicator for the health and the well-being of an Open Source project. But what about end-users, I hear people cry? End users are important, to the extent that they provide ego-strokes to the developers, and to the extent that they provide testing and bug reports to the developers, and to the extent that they provide an economic justification to companies who employ open source developers to continue to do so. But ultimately, the effects of end-users on an open source project is only in a very indirect way. Moreover, if you ask commercial end users what they value about Open Source, a survey by Computer Economics indicated that the number one reason why customers valued open source was “reduced dependence on software vendors”, which end users valued 2 to 1 over “lower total cost of ownership”. (Which is why Sun Salescritters who were sending around TCO analysis comparing 24×7 phone support form Red Hat with Support-by-email from Sun totally missed the point.) What’s important to commercial end users is that they be able to avoid the effects of vendor lock-in, which implies that if all of the developers are employed by one vendor, it doesn’t provide the value the end users were looking for. This is why whether a project’s developers are dominated by employees from a single company is so important. The license under which the code is released is merely just the outward trappings of an open source project. What’s really critical is the extent to which the development costs are shared across a vast global community of developers who have many different means of support. This saves costs to the companies who are using a product being developed in such a fashion; it gives choice to customers about whether they can get their support from company A or company B; programmers who don’t like the way things are going at one company have an easier time changing jobs while still working on the same project; it’s a win-win-win scenario. In contrast, if a project decides to release its code under an open source license, but nearly all the developers remain employed by a single company, it doesn’t really change the dynamic compared to when the project was previously under a closed-source license. It is a necessary but not sufficient step towards attracting outside contributors, and eventually migrating towards having a true open source development community. But if those further steps are not taken, the hopes that users will think that some project is “cool” because it is under an open-source license will ultimately be in vain. The “Generation Y”/Millennial Generation in particular are very sensitive indeed to Astroturfing-style marketing tactics. Ok, so this is why the distinction matters. Given that it does, what terms shall we use? I still like “Organic” vs “Non-organic”. While it may not have been intended by the Mozilla Foundation, the description in their web page, “only a small percentage of whom are actual employees [of the Mozilla Foundation]“, is very much what I and others have been trying to describe. And while I originally used the description “Projects which have an Open Source Development Community” vs “Projects with an Open Source License but which are dominated by employees from a single company”, I think we can all agree these are very awkward. We need a better shorthand. When Brian Aker from MySQL suggested “Organic” vs “Non-Organic” Open Source, and I think those terms work well. If some folks think that “Non-Organic” is somehow pejorative (hey, at least we didn’t say “genetically modified Open Source” :-), I suppose we could use Synthetic Open Source. I’m not really convinced that is any much more appetizing, myself, however. So what would be better terms to use? Please give me some suggestions, and maybe we can come up with a better set of words that everyone is happy with.

There’s been some controversy generated over my use of the terminology of “Organic” and “Non-Organic” Open Source. Asa Dotzler noted that it wasn’t Mozilla’s original intent to “make a distinction between how Mozilla does open source and how others do open source”. Nessance complained that he didn’t like the term “Non-Organic”, because it was “raw and vague - is it alien, poison, silicon-based?” and suggested instead the term “Synthetic Open Source”, referencing a paper by Siobh n O Mahony, ” What makes a project open source? Migrating from organic to synthetic communities”. Nessance referenced a series of questions and answers by Stephen O’ Grady from Red Monk, where he claimed the distinction between the two doesn’t matter. (Although given that Sun is a paying customer of Red Monk, Stephen admits that this might have influenced his thinking and so he might be “brainwashed” :-). So let’s take some of these issues in reverse order. Does the distinction matter? After all, if the distinction doesn’t matter, then there’s no reason to create or define specialized terminology to describe the difference. Certainly, Brian Aker, a senior technologist from MySQL, thinks it does, as do folks like me and Amanda McPherson and Mike Dolan; but does it really? Are we just saying that because we want to take a cheap shot at Sun? Well, to answer that, let’s go back and ask the question, “Why is Open Source a good thing in the first place?” It’s gotten to the point where people just assume that it’s a good thing, because everybody says it is. But if we go back to first principals maybe it will become much clearer why this dinction is so important. Consider the Apache web server; it was able to completely dominate the web server market, easily besting all of its proprietary competitors, including the super-deep-pocketed Microsoft. Why? It won because a large number of volunteers were able to collaborate together to create a very fully featured product, using a “stone soup” model where each developer “scratched their own itch”. Many, if not most, of these volunteers were compensated by their employers for their work. Since their employers were not in the web server business, but instead needed a web server as means (a critical means, to be sure) to pursue their business, there was no economic reason not to let their engineers contribute their improvements back to the Apache project. Indeed, it was cheaper to let their engineers work on Apache collaboratively than it was to purchase a product that would be less suited for their needs. In other words, it was a collective “build vs. buy” decision, with the twist that because a large number of companies were involved in the collaboration, it was far, far cheaper than the traditional “build” option. This is a powerful model, and the fact that Sun originally asked Roy Felding from the Apache Foundation to assist in forming the Solaris community indicates that at least some people in Sun appreciated why this was so important. There are other benefits of having code released under the Open Source license, such as the ability for others to see the implementation details of your operating system — but in truth, Sun had already made the Source Code for Solaris available for a nominal fee years before. And, of course, there are plenty of arguments over the exact licensing terms that should be used, such as GPLv2, GPLv3, CDDL, the CPL, MPL, etc., but sometimes those arguments can be a distraction from the central issue. While the legal issues that arise from the choice of license are important, at the end of the day, the most crucial issue is the development community. It is the strength and the diversity of the development community which is the best indicator for the health and the well-being of an Open Source project. But what about end-users, I hear people cry? End users are important, to the extent that they provide ego-strokes to the developers, and to the extent that they provide testing and bug reports to the developers, and to the extent that they provide an economic justification to companies who employ open source developers to continue to do so. But ultimately, the effects of end-users on an open source project is only in a very indirect way. Moreover, if you ask commercial end users what they value about Open Source, a survey by Computer Economics indicated that the number one reason why customers valued open source was “reduced dependence on software vendors”, which end users valued 2 to 1 over “lower total cost of ownership”. (Which is why Sun Salescritters who were sending around TCO analysis comparing 24×7 phone support form Red Hat with Support-by-email from Sun totally missed the point.) What’s important to commercial end users is that they be able to avoid the effects of vendor lock-in, which implies that if all of the developers are employed by one vendor, it doesn’t provide the value the end users were looking for. This is why whether a project’s developers are dominated by employees from a single company is so important. The license under which the code is released is merely just the outward trappings of an open source project. What’s really critical is the extent to which the development costs are shared across a vast global community of developers who have many different means of support. This saves costs to the companies who are using a product being developed in such a fashion; it gives choice to customers about whether they can get their support from company A or company B; programmers who don’t like the way things are going at one company have an easier time changing jobs while still working on the same project; it’s a win-win-win scenario. In contrast, if a project decides to release its code under an open source license, but nearly all the developers remain employed by a single company, it doesn’t really change the dynamic compared to when the project was previously under a closed-source license. It is a necessary but not sufficient step towards attracting outside contributors, and eventually migrating towards having a true open source development community. But if those further steps are not taken, the hopes that users will think that some project is “cool” because it is under an open-source license will ultimately be in vain. The “Generation Y”/Millennial Generation in particular are very sensitive indeed to Astroturfing-style marketing tactics. Ok, so this is why the distinction matters. Given that it does, what terms shall we use? I still like “Organic” vs “Non-organic”. While it may not have been intended by the Mozilla Foundation, the description in their web page, “only a small percentage of whom are actual employees [of the Mozilla Foundation]“, is very much what I and others have been trying to describe. And while I originally used the description “Projects which have an Open Source Development Community” vs “Projects with an Open Source License but which are dominated by employees from a single company”, I think we can all agree these are very awkward. We need a better shorthand. When Brian Aker from MySQL suggested “Organic” vs “Non-Organic” Open Source, and I think those terms work well. If some folks think that “Non-Organic” is somehow pejorative (hey, at least we didn’t say “genetically modified Open Source” :-), I suppose we could use Synthetic Open Source. I’m not really convinced that is any much more appetizing, myself, however. So what would be better terms to use? Please give me some suggestions, and maybe we can come up with a better set of words that everyone is happy with.

Following my first ftpmaster blog post, which got some nice replies, lets have the next, as I didn’t stop working on stuff. Not as much as I hoped, but well, something. So, what was done since then?

• Merged a little patch from Christoph Berg fixing usage of a function - which doesn’t exist in global namespace, so it needs the full name.
• Merged another patch from Christoph - now we write the content of the Changed-By field from .changes files into projectb. To cite him for the reason: This will allow to identify NMUs and sponsored uploads more precisely in tools querying projectb. Broke dinstall with this and as a little side-effect also the merkel copy of the projectb database. Ups. Well, fixed it immediately again. :) There is some old data available, which I could import into projectb, so it would have data for all packages. But there is some weirdness with the dataset I got from Christoph and am waiting for him to provide a solution. Anyway, all new uploads already store this data, so it is only old data missing, currently.
• Merged a small patch from Thomas Viehmann into our archive software which enables it to send a copy of all mails to the person sponsoring the upload. This is in affect for all future uploads, not for any package which already hit the queues (like those in NEW right now). We do
  • only sent the extra copy if it is a sourceful upload, to not spam buildd admins,
  • only sent the mails to sponsors who did not disable their @debian.org address.
  For those interested, the process works similar to:
  • We look if the Debian login attached to the key signing the upload has an active @debian.org address.
  • If that is the case we look for all uids on that key that look like an email address and compare them with the Maintainer: and Changed-By: field in the .changes file.
  • If there is no match, ie. none of the “emails” of the key uids is in those fields, the @debian.org mail is added to the set of valid receivers.
• Changed split-done to only move files older than 30 days and let it run by our weekly cronjob. Also run it once, which moved 120079 files into their $YEAR/$MONTH subdirectories. (split-done seperates the subdir where we store every .changes of uploads accepted. That quickly grows fast, and we all know how bad filesystems perform with such a number of files. The above 120079 files are from November/Dezember 2007 til now, so imagine how many we have in total, as we have them back to 2002).
• Merged a patch I wrote 7 days ago that does a little sanity checking in the Provides field. Do not try to version your provide, that is forbidden and actually breaks software. (See this bug about it.)
• Merged a patch, submitted by Marc Brockschmidt some long time ago, that lets cruft-report also display NBS for experimental. Changed the cron script to use that extra feature.

---

And as in my last blog post, lets add some items to the todo list:

• Enable dak to read a Built-By field from the .changes file. Thats the sanest solution to sponsored uploads. And also better for buildds, they then should no longer overwrite the Maintainer field with the -m parameter when building packages. Exact semantics to be defined, but dak probably would go something similar to
  • Built-By same as Changed-By or Maintainer?” - do not send mail there.
  • Built-By differs from Changed-By or Maintainer and its sourceful? Send mail to it too.
  • Built-By differs from Changed-By or Maintainer and binary-only? Only send mail there.
• Track the Uploaders: field in projectb. That would help the BTS to offer another display.

Having written all this - today (AKA Saturday, when writing this) won’t see code merges by me. I think most of my ftpmaster related work today will be fun with the NEW queue. For some unknown reason people keep uploading packages. Tsss. Comments: 3

For a university seminar I m currently writing a GUI program to view fractals based on simple iterated function systems (only similaritudes allowed). It supports three different drawing algorithms and you can edit the IFS by dragging squares around on the screen. But this post is not about this program (I might present it later), but how Haskell allowed me to solve a problem very nicely:The first instances of the code had a problem that a lot of GUI programmers know: The drawing of the fractal took quite some time, and during that time the gtk main loop is blocked and the program becomes unresponsive. At first I avoided this problem by manually splitting the drawing function (e.g. by repeatedly increasing the resolution) and kept re-drawing it at higher resolutions in an idle handler, so at least I could interact whenever one resolution has finished drawing. It worked somewhat but it was not easily done for the other, not pixel based, algorithms and it was ugly.So I wanted a way to (a) pause the drawing at any convenient point, to resume it later, and (b) safely abort the drawing if what I m drawing has changed and I need to restart.A common solution to this would be to do the drawing in a separate thread, so (a) is actually not needed, but I did not know if I can safely do (b), and I have heard that threads cause problems with gtk.So I tried to dig deeper for the hidden treasures of advanced haskell programming: I need a monad transformer! I expected the infamous ContT monad transformer to help, but I couldn t figure out how, and I started to create my own monad transformer, called CoroutineT.I tried to figure out what an action of type (CoroutineT IO a) should do, and I came up with this type signature:

pausingAction :: IO (Either (CoroutineT IO a) a)

which means that after the pausingAction is done, it is Either paused (and I get back another pausingAction to run when I want it), or it is done (and I get the result). Note that I m writing IO here, but it can be any monad.The definition of the datatype and the monad instance came mostly from trying to make this type work (yay to haskell s type system, less thinking required), and looks like this:

data CoroutineT m a = CoroutineT  unCoroutineT :: (m (Either (CoroutineT m a) a))  

instance (Monad m) => Monad (CoroutineT m) where
    return v = CoroutineT (return (Right v))
    a >>= b = CoroutineT $ do
          r <- unCoroutineT a
          case r of
            Left  paused   -> return $ Left (paused >>= b)
            Right unpaused -> unCoroutineT (b unpaused)

This translates to english like this: A call to return is not paused. When an action is already paused, further actions should be run after the paused action is resumed. When an action is not paused, further actions can happen now.Like every well behaving monad transfomer, I also need a "runCoroutineT" function to start the coroutine. I probably could have used unCoroutineT directly, but for my use case (GUI drawing) I did not need a return value, so this function is more handy:

runCoroutineT :: Monad m => CoroutineT m () -> m (Maybe (CoroutineT m ()))
runCoroutineT a = either (Just) (const Nothing)  liftM  unCoroutineT a

Nothing surprising here, basically just turning the Either into a Maybe. So it becomes clear how to do (b): We can just throw away the resume action returned by runCoroutineT (if any). The more interesting thing is how we do (a): We need a pause action of type (Coroutine m ()). But how should it work? I did not really try to understand why it works, but by looking at the types, I came up with this:

pause :: Monad m => CoroutineT m ()
pause = CoroutineT (return (Left (CoroutineT (return (Right ())))))

Yes, it sounds like some dance step instructions (read the second line out aloud!), but it works somehow.So here is some example code: I have a pausable IO action that counts from one to ten, pausing after each number. I also have function that resumes an pausable action up to n times:

example n = keepGoingFor n $ do
        liftIO $ putStrLn "This is the coroutine"
        forM_ [1..10] $ \i -> do
            liftIO $ putStrLn $ "Counting to "++ show i ++" while you keep calling it"
            pause
  where --keepGoing :: Monad m => CoroutineT m () -> m ()
      keepGoingFor 0 _   = putStrLn "Here I just abort the run"
      keepGoingFor n cor = do
        resume <- runCoroutineT cor
        case resume of
            Just runAgain -> keepGoingFor (n-1) runAgain
            Nothing       -> putStrLn "Finally stopped"

And here is the output of two different runs:

*CouroutineT> example 5
This is the coroutine
Counting to 1 while you keep calling it
Counting to 2 while you keep calling it
Counting to 3 while you keep calling it
Counting to 4 while you keep calling it
Counting to 5 while you keep calling it
Here I just abort the run
*CouroutineT> example 14
This is the coroutine
Counting to 1 while you keep calling it
Counting to 2 while you keep calling it
Counting to 3 while you keep calling it
Counting to 4 while you keep calling it
Counting to 5 while you keep calling it
Counting to 6 while you keep calling it
Counting to 7 while you keep calling it
Counting to 8 while you keep calling it
Counting to 9 while you keep calling it
Counting to 10 while you keep calling it
Finally stopped

So it does really works fine, and it proved very useful in my GUI drawing problem. For that, I created this nice control structure which works like mapM_, but calls pause every n iterations, and therefore hides the pausing stuff almost completely:

pausingForM_ :: Monad m => Int -> [a] -> (a -> CoroutineT m ()) -> CoroutineT m ()
pausingForM_ period list action = pausing' 0 list
  where pausing' _ []     = return ()
        pausing' n (x:xs) = do action x
                       if n==period then pause >> pausing' 0     xs
                                    else          pausing' (n+1) xs

I have put the complete module (including instances omitted here) in the darcs repository that might later also contain the fractal drawing program.

How to not start a service by default Use case: in my laptop, I sometimes need MySQL, PostgreSQL or Apache in order to test some software that I'm developing, but I do not want them on all the time. The solution is: rm /etc/rc*.d/S*mysql* (thanks to Wouter) update-rc.d will not touch your symlinks as long as there is at least one still around for a package. Also, this leaves the stop symlinks around, so that when I start one of these services for development, it will still be properly stopped on shutdown.