Search Results: "ema"

1 March 2009

Gustavo Noronha Silva: WebKitGTK+ 1.1.1 released

A new version of WebKit s GTK+ port is here, as also announced by Xan! Before I start listing the new features and fixes, let me give you a download link =). For this version we have changed the binary interface slightly, which requires a soversion bump, so rebuilding applications is in order. The API is still stable, though. Another major change worthy of note is that we have made libsoup the only HTTP backend available for WebKitGTK+, removing the CURL related code from our port (the CURL backend remains in the tree, though, as it is used by other ports). This should simplify implementation and allows us to expose Soup objects in the API, reducing the amount of wrapping code that is necessary. We are already exposing the SoupSession we use internally, providing yet more power and flexibility to applications using our API. In this release we finally get to see the official debut of the Web Inspector in the GTK+ port. Can t wait to see releases of our dear browsers featuring the inspector =). Numerous improvements to hading policy decisions regarding navigations and window creation requests to the application have also come with this release. We worked hard on finding and fixing crashes for this release, but also numerous fixes for those pet peeves that bother us just a bit, but enough to add up got landed. There are more of those to kill, but things look promising. We are hoping to release again in two weeks, so expect more fixes soon. So, that s it. Grab the new version, test it, report bugs, and enjoy!

27 February 2009

Neil Williams: blogging using 2.6.28 on Aspire1

This is my first entry from my Acer Aspire1 since installing the latest Emdebian Grip and the 2.6.28 kernel that now provides free wireless support. A few tweaks still needed to get a few packages sorted out so I've got a Debian source on this box too. Still, with my usual packages (iceweasel, sylpheed, grisbi, geany), all my email cached data, liferea data, a few documents and a few Debian packages and I'm still at only 753Mb used.

Using wicd from squeeze, I've got wireless working - there's no led lit but I'm sure that's fixable.

Copying a few files around from ~/.gnome2/ on my other laptop and suddenly I have drivel, sylpheed and liferea all pre-configured.

I'll be blitzing this install again numerous times in the next few days but by keeping the ~/.gnome2/ files on a USB stick, it will be simple to restore the system, especially as I'm using IMAP email.

(If you're wondering, the drivel upload is delayed by the churn in Debian Sid - waiting for the libmtp7 8 transition to make rhythmbox available in a clean chroot.)

26 February 2009

Martín Ferrari: When it changed

I didn't want to miss the opportunity to share this discovery, maybe the feminists or the sci-fans out there already know her well, but to me it's a completely new name. By this post I came to know Joanna Russ, a visionary intellectual, an incredible writer a, legendarily fiery critic [...], an unrepentant sci-fi fan [...], feminist crusader and one of the first major female and openly lesbian sci-fi scribe to hit major success. I was instantly hooked by her short story named When it changed (full text there). I cannot add more to what's said there, take a look. PS: Thanks, Coilhouse, one more book to already big list of wishlists... Tags: Planet Debian

25 February 2009

Andrew McMillan: Conference Audio: Problems, Procedures & Suggestions for the Future

I was going to reply to this in the mailing list, but after I spent half an hour furiously typing this, I thought it might be better to put it into a blog, for a slightly wider audience, perhaps a more permanent life and partly in an effort to end this thread on a more or less constructive note. The backchat for this is that the LCA 2009 video has started to go up on the web, and some talks have some gaps in the audio stream... Russell Coker wants to make a constructive suggestion, as follows: On Wed, 2009-02-25 at 18:02 +1100, Russell Coker wrote:
>
> I've pasted in the below paragraph (the third time I've included it in this
> thread) to reiterate my point. Note that I am not asking you to re-do any
> videos, I am making a suggestion for future people.
>
> # I suggest that in future when preparing such videos the sections with no
> # sound be omitted and replaced by a short note explaining the omission,
> # something like "sorry, due to technical difficulties four minutes of sound
> # were not available". Having a second ogg file for the lip-readers would be
> # an option although I expect that the demand would be minute.
> As someone who has not been at all involved with the production of the LCA videos, but who has been intimately involved with the process of streaming DebConf 6, 7 & 8, I think this sort of additional effort will only happen if people voluntarily do it themselves. Just the task of *watching* all of the produced videos is a huge job for one individual, but as a community of individuals it is quite likely that at least one individual in the community watches each video. If each person who was offended by the quality of post-production work took it upon themselves to fix up only one video then it is probable that we would see some changed. Otherwise I'm afraid it is unlikely. Further in his e-mail, Russell Coker wrote:
>
> My observation was that the microphones were put down AFTER they ceased
> operating. A non-functional lapel microphone is no better than a non-
> functional hand-held microphone.
>
> Having a large supply of new batteries would be one way of alleviating the
> problem (I believe that some of the instances were due to flat batteries).
> Another possibility is having two microphones on hand so that if one died the
> other could be turned on.
>
> A wired microphone that doesn't rely on battery power would probably be the
> most reliable option. That of course might not fit with OH&S issues. Most speakers are uncomfortable / unfamiliar with wired microphones. Spares are essential, of course, but the problem can usually be avoided by having a checklist for the audio person to confirm several things: Before the start of the talk: At the start of the talk: At the end of the talk: There are probably a couple of things that I've missed, but if you start with a checklist you pretty soon modify it into a *good* checklist, and it *really* helps when people are under fire. In fact lapel mics are not the best microphones for these kinds of presentations. The best mics are the ultra lightweight 'headset' models which place a the mic near to the speakers mouth. These do not suffer when the speaker turns their head hard to the left or right, at which points a lapel mic stops getting their speech. They also work well with that more hirsute minority so over-represented in our particular community (alas, that this set no longer includes Bdale :-) Of course DebConf has a large team of volunteers for streaming the conference, and has developed these kinds of procedures over a number of years. At DebConf there are usually only two main streams, each of which involves: Ideally that really is 7 people (times two streams), and you can get by if you can't find all of the last three, but the other four are increasingly desirable. We mostly manage to do that with volunteers for two streams at DebConf, but for five or more streams at LCA it is inevitably a lot harder, and the quality necessarily has to be cut back to match the resources available. While it is true that sound is critical to this communication, it is unfortunately also true that most people's (even most geek's) eyes glaze over when presented with a mysterious box with a mere 6 sliding potentiometers, let alone when there are 36 of them, each associated with another 8 rotary knobs, and a rats nest of cables worthy of the worst network nightmare they've seen, and where everything appears to be literally held in place with duct tape. So it is not so surprising then that while it is relatively easy to find a gadget mad geek capable of operating a camera, or even experienced with operating a camera, finding people with experience operating a sound mixer is an order of magnitude harder. At LCA the operation appears superficially to often involve a (single) speaker camera operator with a very basic sound mixer which has been configured once by an overworked person who is unobtainable during the actual talk to solve any audio problems. These people are volunteers, and are doing their best, but it simply isn't possible to get a consistently high quality of video and audio in those circumstances. So as one of the organising committee for LCA 2010 in Wellington I will be watching what we do, and although I don't intend to get personally involved in the video production (I've got plenty of other stuff on *my* plate :-) I do intend to provide what advice and assistance I can. I know that we hope to have some professionals involved (as volunteers), but how many volunteers, and with what levels of skill & experience, we are unlikely to know until much closer than the date. I'll also personally try and get a run-down on all of the audio equipment so that in an emergency, if I happen to be available, I can sub as a sound mixer operator. So we will try and do better in Wellington in 2010. Come to the conference, though, because we can make no guarantees, and if you do come to Wellington, and you see a single cameraman, and you know something about running a sound mixer, then perhaps you can come and volunteer to help out in that capacity, for the talks you are attending anyway. We'd really appreciate the help. And finally I must say thanks, in particular, to Holger Levsen for all the learning I have gained since joining the videoteam at DebConf5 in Helsinki in 2005. And too, to all of those past conference teams (LCA, DebConf and otherwise) who have put their best efforts into providing videos of the talks I couldn't see in person. I know it's a bloody hard job, and often a less high profile one, so:

Thank You.

Russ Allbery: Expectation and obligation

Some necessary background: George R.R. Martin is a science fiction and fantasy author who's been writing (and editing) in the field for quite a while. In 1996, he started a new epic fantasy series entitled The Song of Ice and Fire, a hugely complex and sprawling series telling the story of an entire medieval world, with four volumes published and something on the order of 19 viewpoint characters to date. The fourth and last book to date in the series was published in 2005. It contained only half the viewpoints expected (and in my opinion and that of at least some other readers, the least interesting half). There was an afterword saying basically that the book was cut somewhat artificially in half for length reasons (it was 753 hardcover pages for what was there), the next book would have the other viewpoints, and it was basically done. Three years and some later, and after a few other optimistic updates, it's not yet done. Some of Martin's fans are increasingly getting antsy and sometimes abusive about this. This has increasingly been getting to Martin, and he's responded in various ways in his personal blog, most recently February 19th. See also John Scalzi and Charlie Stross reacting to his post. Now, I'm not one of the people who is rabid about this series. I'm a little disappointed that I haven't gotten to read the next bits with the characters I really like in the series, and I didn't much care for A Feast for Crows because of how the book was split. But I have a thousand other books I want to read (literally I've read less than half the books I own) and if I don't get to see more of the series, oh well. I am, however, very interested in the fan reaction to this delay and Martin's reaction to that, because it has started reminding me of similar things in my own life. Some of the fans are just batshit insane, which is one of the things that happens to a best-selling author; I feel for Martin there but there isn't much anyone can really do about that. But among the sane people, there are a bunch who say something like "I understand why the constant nagging doesn't help, but on the other hand, when you start a big story with unresolved plots, you have an obligation to your readers to finish it." I call bullshit on that. This same effect happens in the free software world. If you write some program, you're perceived to have an obligation to support it, fix it, release new versions, and so forth. I've also had this happen in other volunteer contexts, such as Usenet. Often that obligation is based on some comments that one makes about future plans, things that will go into the next release, e-mail messages where you say that you'll look at something, and so forth. It's not entirely groundless, and it's not groundless with Martin either. However. As any software developer knows, time estimates for tasks are horribly difficult and almost always too short. This is clearly what happened to Martin, and bitching about the time estimates doesn't help at all. Some problems turn out to be so much harder than you expect that time estimates fail entirely. With creative work (including some programming) it's particularly hard to know how much work is remaining. You may know roughly how much you have left to write, but you don't know how hard each chunk will be until you start it, and sometimes until you finish it. With some projects, the only thing you can do is say "sorry, my estimates were completely off and I don't know how to estimate this" and then stop promising dates. Which Martin has already done. He was "wrong" in some sense for setting bad expectations, but estimation is insanely hard and the world usually doesn't let you get away with just refusing to estimate anything. So much for time. What about whether Martin has an obligation to write the next book at all? The equivalent question in the free software world is whether one can just walk away from a project, or away from a volunteer position, leaving a bunch of stuff undone that some people who were using the software were counting on having finished. It's a bit easier to deal with in free software, since usually just promises are at stake, whereas with Martin he's sold a commercial product and people get weird about money. There is a reasonable expectation that a series of this type will have a conclusion (although I'll point out that the SFF world is filled with unfinished series). Maybe you wouldn't have bought any of the series if you had known that it wouldn't end, so now you're out $100 for the hardcovers that you feel like you bought on false pretenses. Let's examine that. First, I have a serious problem with people who use the economic transaction as leverage in this sort of argument. This is a really piddly amount of money for each individual person, generally less than I get paid for one hour of work. Writing a new novel is way, way more than one hour of work. Martin's fans are demanding that he do work that's worth many orders of magnitude more than they're conceivably out monetarily. (And the reactions are individual reactions of anger; the level of upsetness of each individual shouldn't change based on the total size of the fan base.) Furthermore, their economic transaction was with the publisher, and they had no control and no say over how Martin got paid. From multiple different directions, they're not his employer, have no contractual control over how he spends his time, and have no legitimate economic basis for asserting any such control. They can make a decision to not buy his future work because they're disappointed, and that's it. Second, it is absolutely mandatory to give other people room to be wrong or change their mind. Human beings cannot keep every promise that they make. We fail at things that we fully intend to do, particularly around estimates, and particularly around one's dedication to decade-long major life projects. People are wrong about marriages and children, which involve far more serious consequences than a fiction series. I think it's a good reality check here to note that his fans are placing way more restrictions on what he can do with his life than would be legal for any employer. Third, when someone doesn't meet your expectations, the response should be proportional and rational. This is the one that really gets to me. Choosing not to read the next book in the series because it's been too long is entirely reasonable. Trying to make Martin feel like crap is not. I would go further: nothing that an artist could promise to their general public could warrant intentionally making them feel like crap for failing to do it. That level of emotional consequence should be reserved for things like marriage, parenting, or life-long friendship, not disappointment that you didn't get to hear the end of a story you were enjoying. I speak from personal experience here: this type of punishment by guilt is a nasty, nasty business. It burns people out faster than any other possible reaction, it makes it much less likely that Martin will continue the series at all, and it's probably poisoning his feelings about the entire world. It's burning down the bridge you're standing on. It always hurts and never helps. People already create feelings of obligation around things they've promised, implicitly or explicitly, to do, and generate plenty of their own guilt. That guilt is one of the major obstacles that one has to overcome to be productive; if you don't believe me, read any good productivity book, such as Allen's Getting Things Done, and notice how much it talks about getting past guilt. Intentionally going out of your way to contact someone you do not know personally and pile on more guilt is vicious, cruel, and tactically counterproductive. Trust me, Martin didn't fail to notice that he missed his own deadlines. I think Martin's post and the subsequent discussion is a great opportunity for all of us who create or use creative and technical works to think about how we apply the Golden Rule. There's a natural tendency to hold people tightly to promises that we'll get something that we want. There's a natural tendency for all of us to make those promises, since people are grateful for them, and then pile guilt on ourselves for not meeting them. All of the emotion wrapped up in both sides of that tension of expectation and obligation can be exhausting and poisonous. It makes relationships brittle because we don't let people renegotiate. We don't let people say they were wrong. We don't let people change their minds. And we don't let ourselves change our minds. Here's my promise: the next time someone doesn't meet an expectation I had, I will remember this discussion, give them space to be wrong, and not pile on more guilt. Like all promises, I'll try to keep it, but I might not be able to.

24 February 2009

Russell Coker: Tragedy and Profit

Every time something goes wrong there will be someone who tries to take advantage of the situation. The recent bushfires in Australia that have killed hundreds of people (the count is not known yet) are a good example. Pastor Nalliah of Catch the Fire Ministries [1] claims that it is due to legalising abortion. This is astoundingly wrong. In a more extreme example representatives of the Westboro Baptist Church were planning to visit Australia to launch a protest in support of the bushfires [2]. I have not yet found any news reports about whether they actually visited Australia or protested - it s most likely that they decided not to visit due to the Australian laws being very different to US laws regarding the relative importance of freedom of speech and incitement to violence. Apparently the insane Westboro Baptist Church people (who are best known for GodHatesFags.com and GodHatesAmerica.com) believe that God hates Australia and caused the fires (presumably due to Australia not persecuting homosexuals). Danny Nalliah has permanently damaged his own reputation by acting in a similar way to the Westboro Baptist Church. The reputation of Catch The Fire now depends on how quickly they get a new pastor Please note well that the vast majority of Christians have nothing in common with Westboro or Catch The Fire. I don t recall the last time I met an Australian Christian who was strongly opposed to homosexuality or abortion. Now we do have to try and investigate ways of avoiding future tragedies, and the work to do this needs to begin immediately. John Brumby (the Premier of Victoria) has announced that Victoria will get new strict building codes for fire resistant buildings [3]. There have been many anecdotes of people who claim to have been saved by attaching sprinkler systems to their homes, by building concrete bunkers to hide in while the fire passes, and using other techniques to save their home or save themselves. Some more research on the most effective ways of achieving such goals would be worthwhile, an increase in funding for the CSIRO to investigate the related issues would be a good thing. The article also has an interesting quote As the fallout from the disaster widened, the union representing the nation s 13,000 firefighters warned both the federal and state governments to take global warming seriously to prevent a repeat of last weekend s lethal firestorm . However given that traditionally Australia and the US have been the two nations most opposed to any efforts to mitigate global warming it seems unlikely that anything will change in this regard in a hurry. The attempts to link bushfires to abortion and homosexuality are offensive, but can be ignored in any remotely serious debate about politics. However there are some other groups trying to profit from the tragedy that make claims which are not as ridiculous. On the 9th of February the Australian Green party was compelled to release an official statement from Spokesperson Scott Ludlam, Sarah Hanson-Young, Rachel Siewert, Christine Milne, and Bob Brown following some political discussion about Greens policies [4]. There have been attempts to blame the Greens for the tragedy which were politically motivated, some of which came from groups that traditionally oppose the Greens for other reasons (I m not going to provide the detail - anyone who is really interested can do google searches on the people in question). On the 16th of February Bob Brown (the leader of the Green party) felt obliged to make another media release reiterating the fact that the Greens support prescribed burn-offs to limit the scope of wild fires [5], he also decried the hate mongering that has been occurring in the wake of the disaster. One of the strange memes that seems to be spread by opponents to the Greens is that the Greens are all supposedly from the city and know nothing about the country. To avoid being subject to such attack I feel obliged to note that on one of the bad fire days I visited my parents. I spent the morning with my father and some friends at a park that was not far from the fire area, my friends then returned to their home which was not far from the fire area. I then had lunch with my parents and watched the smoke through the dining room window. After that my friends didn t respond to email for a while and I was concerned that they may have lost their house or maybe suffered injury or death. I didn t know them well enough to feel it appropriate to try a dozen different ways of contacting them (I m sure that many other people were doing so), but I was rather concerned until my wife received an email from them. But I don t base my political beliefs on what I personally observe or my connections to people on the edge of the fire zone. I believe in the Green principles of Peace and Non Violence, Grassroots Democracy, Social and Economic Justice, Ecological Sustainability and the use of science and statistics to determine the best ways of achieving those goals.

Michal Čihař: Email easier than Google?

I really do not get, while people rather email somebody to ask question, which Google can answer in few seconds. Today I got question that some body was not able to find phpMyAdmin demo:
Hello, I ran your phpMyAdmin demo from some link, but I unfortunately do not know where it was. Could you please send me link to this demo.
Is it really that hard to find it? Typing phpMyAdmin demo into Google, does give correct result on first place to me... PS: I got to reading this email after five days, so that guy definitely did not save much time.

Adrian von Bidder: New Toy

Last week, I couldn't resist and bought myself an Acer Aspire One (AOA 150Ab) netbook. It has 1G RAM, 120G HDD, unfortunately needs a fan, and is the model without 3G modem. It comes with Linpus Linux pre-installed. Looks quite nice, but is obviously ultimately the wrong OS ... Besides, it's locked down quite a bit, there's not even an easy way to start a terminal :-) I still have kept it, in a dual-boot configuration, to play around or show to people. Installing Lenny went very well, and to make things more fun I'm running quite a few things from experimental: KDE 4.2, xorg 7.4 (1:7.4~5 right now) and Oo.org 3. And, to get DRI2, also the 2.6.28 kernel from the newer-than-sid repository of the kernel team (2.6.28-2~snapshot.12850, but I hear 2.6.28 has now been uploaded.) While the whole thing is fun to use and didn't make any real problems, there are a few remaining issues (yes, this is a Dear Lazyweb posting, feel free to comment. I'll try to add updates to this article as this progresses): Ok, this has become rather a long list. But at least, as you can see, it's mostly minor issues, and hopefully a few where it's just missing configuration. One other issue is battery life: I see that there should be a large battery available for this thing. If it doesn't cost me as much as the whole netbook again, I'll seriously have to think about this... The 3-cell battery does last about 2.5h (and has, right now, uncovered a little bug where the battery monitoring applet tells me No AC adaptor plugged in, battery capacity: 50%, charging . Whee! I have a perpetuum mobile! I'm gonna be RICH!)

Biella Coleman: hell_yes: debconf nyc 2010



hell_yes, originally uploaded by the biella.
w00t. NYC has been chosen. I was equally excited about Boston so I did not really care about which city won out. But since it is in NYC, I will definitely be helping out and one of the first things I want to do is organize a 1/2 day trip to this sauna/spa mega-facility in Queens

Biella Coleman: hell_yes: debconf nyc 2010



hell_yes, originally uploaded by the biella.
w00t. NYC has been chosen. I was equally excited about Boston so I did not really care about which city won out. But since it is in NYC, I will definitely be helping out and one of the first things I want to do is organize a 1/2 day trip to this sauna/spa mega-facility in Queens

Matt Brown: The government listened!

I was very pleased to wake up this morning to the news that National has delayed the introduction of S92A via an order-in-council. It s a nice short-term victory, but I ll save the champagne until the law is fundamentally rewritten. The most pleasing aspect of the decision is simply that it was made at all. Within two weeks, a small band of protesters were able to harness the power of the Internet to direct international attention and place enough pressure on a Government, whose Prime Minister admitted to not having read the bill prior, that he then took the time to understand the issues and personally announce the delay in implementation of the law. We owe much thanks to the Creative Freedom Foundation for all the effort they put into co-ordinating the protest and ensuring that a single coherent message was presented. Just a little bit of my cynicism and belief that politicians never listen to public opinion outside of election campaigns was chipped away today. The reason I m not breaking out the champagne yet is that we ve only achieved a temporary reprieve in the commencement of the law. While those present at the press conference seem somewhat confident that John Key didn t like what he found in the law and would have repealed it if given the chance, all that has actually been done is delay it in the hopes of an agreement between the TCF and the rights holders (aka big media companies) on how to implement the still fundamentally broken law. The Government has given until late March for that to occur. To put this into a more global context. My happiness as I took the bus to work after reading about the decision to delay the law was short lived as the front page of the local paper declared that Eircom (Ireland s equivalent of Telecom) has voluntarily agreed to block sites such as The Pirate Bay upon request by the media companies (this comes a week after they also announced an agreement to, again voluntarily , implement a 3-strikes S92A style policy). Now, with the biggest ISP in their pocket (so to speak), the media companies have sent threatening letters to the remaining ISPs in the country demanding they implement the same procedure. To me, this illustrates one of the fundamental problems with S92. The concept that an ISP is liable for the conduct of its users, or for policing where on the Internet users should and shouldn t be able to connect to does not belong in our laws. Most ISPs already have provision to disconnect customers for illegal activity in their terms and conditions. If an end-user is doing something illegal, that is an issue between the rights holder and the end-user to take up in the courts just like every other sector of society must do when wronged, at which point the existing ISP terms and conditions can be invoked and access terminated. The big media companies, having decided that it is too expensive/hard/inconvenient to follow standard legal procedures to resolve their grievances are launching multi-pronged attacks to shift the playing field in their favour. In countries like New Zealand, where our politicians yearn for a Free Trade Agreement with America, they use their lobbyists to ensure that S92 style laws are part of the conditions. In other jurisdictions, like Ireland, they use strong-arm, divide and conquer style bully tactics outside of the political and legal process. I don t support copyright infringement. I rely on copyright to protect much of the work I place on the Internet, I want strong laws that protect me when my rights have been infringed. I don t believe that such laws should come at the expense of due process, our legal tradition and the basic principle of fairness! I don t believe that copyright infringement is such a heinous crime that it demands punishments stronger than those we deliver to paedophiles, stalkers or any other class of criminal who uses the Internet to enable their crimes. To me, today s (yesterday s - depending on your timezone) decision is only the first step in clawing New Zealand back from the dangerous path that the big media companies have been leading our law makers down. From here we need to press on and demonstrate to the Government over the next month that even if the TCF and rights holders are able to come up with some sort of workable code of practice, the law is still fundamentally flawed. It is based on premise that we are guilty by accusation. Even if guilt were to be proved by a competent legal body (eg. court or copyright tribunal) we don t need laws placing further liabilites onto ISPs (and remember the definition of ISP under this amendment act includes businesses who provide Internet access to staff, libraries, schools and hospitals) when their existing terms and conditions already prohibit illegal activity. Finally, and most importantly of all, we need to remember that laws exist to serve all sectors of society. Yes, copyright infringement is against the law and rights holders are reasonable in expecting the law to protect their content and allow them to make a fair profit. On the other side of the fence, average New Zealanders are not being unreasonable in their desire to have media available electronically, on demand and non-inhibited by DRM following a legal purchase. The failure of the media businesses to adequately cater to this change in market demand and usage of technology is obviously a contributing factor to the widespread copyright problems that they are facing today. Obviously, I m not condoning copyright infringement simply because the media companies are failing to address demand. Even stupid laws must be obeyed (and the concept of copyright is far from stupid). What I want to see is the Government acknowledging that the problem is not solely with consumers infringing copyright for malicious purposes, and therefore that the solutions do not lie solely in increasing the enforcement and punishments available. Copyright has always been a balancing act between the rights of content producers and consumers. S92 and the act it is contained within are taking us far too far down the road of catering to big business and their outdated business models with far too little concern for the rights of the individual consumer. Despite the many submissions made on this act last year when it was first passing through parliament, there was no comprehensive debate on what copyright means and how it should balance the rights of content producers and consumers in our digital century where copying is a zero-cost, zero-thought activity. Without such a debate we re doomed to continue wasting time arguing over the symptoms of the problem, like S92. So, I m saving my champagne for the day when we as a country address these issues and come up with a fair and workable interpretation of what copyright means today.

23 February 2009

Theodore Ts'o: Reflections on a complaint from a frustrated git user

Last week, Scott James Remnant posted a series of Git Sucks on his blog, starting with this one here, with follow up entries here and here. His problem? To quote Scott, I want to put a branch I have somewhere so somebody else can get it. That s the whole point of distributed revision-control, collaboration. He thought this was a mind-numbingly trivial operation, and was frustrated when it wasn t a one-line command in git. Part of the problem here is that for most git workflows, most people don t actually use git push . That s why it s not covered in the git tutorial (this was a point of frustration for Scott). In fact, in most large projects, the number of people need to use the scm push command is a very small percentage of the developer population, just as very few developers have commit privileges and are allowed to use the svn commit command in a project using Subversion. When you have a centralized repository, only the privileged few will given commit privileges, for obvious security and quality control reasons. Ah, but in a distributed SCM world, things are more democratic anyone can have their own repository, and so everyone can type the commands git commit or bzr commit . While this is true, the number of people who need to be able to publish their own branch is small. After all, the overhead in setting up your own server just so people can pull changes from you is quite large; and if you are just getting started, and only need to submit one or two patches, or even a large series of patches, e-mail is a far more convenient route. This is especially true in the early days of git s development, before web sites such as git.or.cz, github, and gitorious made it much easier for people to publish their own git repository. Even for a large series of changes, tools such as git format-patch and git send-email are very convenient for sending a patch series, and on the receiving side, the maintainer can use git am to apply a patch series sent via e-mail. It turns out that from a maintainer s point of view, reviewing patches via e-mail is often much more convenient. Especially for developers who are just starting out with submitting patches to a project, it s rare that a patch is of sufficiently high quality that it can be applied directly into the repository without needing fixups of one kind or another. The patch might not have the right coding style compared to the surrounding code, or it might be fundamentally buggy because the patch submitter didn t understand the code completely. Indeed, more often than not, when someone submits a patch to me, it is more useful for indicating the location of the bug more than anything else, and I often have to completely rewrite the patch before it enters into the e2fsprogs mainline repository. Given that, publishing a patch that will require modification in a public repository where it is ready to be pulled just doesn t make sense for many entry-level patch submitters. E-mail is in fact less work, and more appropriate for review purposes. It is only when a mid-level to senior developer is trusted to create high quality patches that do not need review that publishing their branch in a pull-ready form really makes sense. And that is fairly rare, and why it is not covered in most entry-level git documentation and tutorials. Unfortunately, many people expect to see the command scm push in a distributed SCM, and since git pull is a commonly used command for beginning git users, they expect that they should use git push as well not realizing that in a distributed SCM, push and pull are not symmetric operations. Therefore, while most git users won t need to use git push , git tutorials and other web pages which are attempting to introduce git to new users probably do need to do a better job explaining why most beginning participants in a project probably don t need their own publically accessible repository that other people can pull from, and which they can push changes for publication. There is one exception to this, of course, and this is a developer who wants to get started using git for a new project which he or she is starting and is the author/maintainer, or someone who is interested in converting their project to git. And this is where bzr has an advantage over git, in that bzr is primarily funded by Canonical, which has a strong interest in pushing an on-line web service, Launchpad. This makes it easier for bzr to have relatively simple recipes for sharing a bzr repository, since the user doesn t need to have access to a server with a public IP address, or need to set up a web or bzr server; they can simply take advantage of Launchpad. Of course, there are web sites which make it easy for people to publish their git repositories; earlier, I had mentioned git.or.cz, github, and gitorious. Currently, the git documentation and tutorials don t mention them since they aren t formally affiliated with the git project (although they are used by many git users and developers and the maintainers of these sites have contributed a large amount of code and documentation to git). This should change, I think. Scott s frustrations which kicked off his git sucks complaints would have been solved if the Git tutorial recommended that the easist ways for someone to publicly publish their repository is via one of these public web sites (although people who want to set up their own server certainly free to do so). Most of these public repositories probably won t have much reason to exist, but they don t do much harm, and who knows? While most of the repositories published at github and gitoriuous will be like the hundreds of thousands of abandoned projects on Sourceforge, one or two of the new projects which someone starts experimenting on at github or gitorious could turn out to be the next Ruby on Rails or Python or Linux. And hopefully, they will allow more developers to be able to experiment with publishing commits on their own repositories, and lessen the frustrations of people like Scott who thought they needed their own repositories; whether or not a public repository is the best way for them to do what they need to do, at least this way they won t get as frustrated about git. :-) Related posts (automatically generated):
  1. Git and hg John Goerzen recently posted about Git, Mercurial and Bzr that...
  2. Batches of patched batches of patches I found the following from the Risks Digest, authored by...

Theodore Ts'o: Reflections on a complaint from a frustrated git user

Last week, Scott James Remnant posted a series of Git Sucks on his blog, starting with this one here, with follow up entries here and here. His problem? To quote Scott, I want to put a branch I have somewhere so somebody else can get it. That s the whole point of distributed revision-control, collaboration. He thought this was a mind-numbingly trivial operation, and was frustrated when it wasn t a one-line command in git. Part of the problem here is that for most git workflows, most people don t actually use git push . That s why it s not covered in the git tutorial (this was a point of frustration for Scott). In fact, in most large projects, the number of people need to use the scm push command is a very small percentage of the developer population, just as very few developers have commit privileges and are allowed to use the svn commit command in a project using Subversion. When you have a centralized repository, only the privileged few will given commit privileges, for obvious security and quality control reasons. Ah, but in a distributed SCM world, things are more democratic anyone can have their own repository, and so everyone can type the commands git commit or bzr commit . While this is true, the number of people who need to be able to publish their own branch is small. After all, the overhead in setting up your own server just so people can pull changes from you is quite large; and if you are just getting started, and only need to submit one or two patches, or even a large series of patches, e-mail is a far more convenient route. This is especially true in the early days of git s development, before web sites such as git.or.cz, github, and gitorious made it much easier for people to publish their own git repository. Even for a large series of changes, tools such as git format-patch and git send-email are very convenient for sending a patch series, and on the receiving side, the maintainer can use git am to apply a patch series sent via e-mail. It turns out that from a maintainer s point of view, reviewing patches via e-mail is often much more convenient. Especially for developers who are just starting out with submitting patches to a project, it s rare that a patch is of sufficiently high quality that it can be applied directly into the repository without needing fixups of one kind or another. The patch might not have the right coding style compared to the surrounding code, or it might be fundamentally buggy because the patch submitter didn t understand the code completely. Indeed, more often than not, when someone submits a patch to me, it is more useful for indicating the location of the bug more than anything else, and I often have to completely rewrite the patch before it enters into the e2fsprogs mainline repository. Given that, publishing a patch that will require modification in a public repository where it is ready to be pulled just doesn t make sense for many entry-level patch submitters. E-mail is in fact less work, and more appropriate for review purposes. It is only when a mid-level to senior developer is trusted to create high quality patches that do not need review that publishing their branch in a pull-ready form really makes sense. And that is fairly rare, and why it is not covered in most entry-level git documentation and tutorials. Unfortunately, many people expect to see the command scm push in a distributed SCM, and since git pull is a commonly used command for beginning git users, they expect that they should use git push as well not realizing that in a distributed SCM, push and pull are not symmetric operations. Therefore, while most git users won t need to use git push , git tutorials and other web pages which are attempting to introduce git to new users probably do need to do a better job explaining why most beginning participants in a project probably don t need their own publically accessible repository that other people can pull from, and which they can push changes for publication. There is one exception to this, of course, and this is a developer who wants to get started using git for a new project which he or she is starting and is the author/maintainer, or someone who is interested in converting their project to git. And this is where bzr has an advantage over git, in that bzr is primarily funded by Canonical, which has a strong interest in pushing an on-line web service, Launchpad. This makes it easier for bzr to have relatively simple recipes for sharing a bzr repository, since the user doesn t need to have access to a server with a public IP address, or need to set up a web or bzr server; they can simply take advantage of Launchpad. Of course, there are web sites which make it easy for people to publish their git repositories; earlier, I had mentioned git.or.cz, github, and gitorious. Currently, the git documentation and tutorials don t mention them since they aren t formally affiliated with the git project (although they are used by many git users and developers and the maintainers of these sites have contributed a large amount of code and documentation to git). This should change, I think. Scott s frustrations which kicked off his git sucks complaints would have been solved if the Git tutorial recommended that the easist ways for someone to publicly publish their repository is via one of these public web sites (although people who want to set up their own server certainly free to do so). Most of these public repositories probably won t have much reason to exist, but they don t do much harm, and who knows? While most of the repositories published at github and gitoriuous will be like the hundreds of thousands of abandoned projects on Sourceforge, one or two of the new projects which someone starts experimenting on at github or gitorious could turn out to be the next Ruby on Rails or Python or Linux. And hopefully, they will allow more developers to be able to experiment with publishing commits on their own repositories, and lessen the frustrations of people like Scott who thought they needed their own repositories; whether or not a public repository is the best way for them to do what they need to do, at least this way they won t get as frustrated about git. :-) Related posts (automatically generated):
  1. Git and hg John Goerzen recently posted about Git, Mercurial and Bzr that...
  2. Batches of patched batches of patches I found the following from the Risks Digest, authored by...

22 February 2009

Biella Coleman: No final decision but I am excited!

I can t believe Debconf will be on American soil next summer. This is exciting, whichever city (Boston/NY) wins!

Biella Coleman: No final decision but I am excited!

I can t believe Debconf will be on American soil next summer. This is exciting, whichever city (Boston/NY) wins!

Theodore Ts'o: Should Filesystems Be Optimized for SSD s?

In one of the comments to my last blog entry, an anonymous commenter writes:
You seem to be taking a different perspective to linus on the adapting to the the disk technology front (Linus seems to against having to have the OS know about disk boundaries and having to do levelling itself)
That s an interesting question, and I figure it s worth its own top-level entry, as opposed to a reply in the comment stream. One of the interesting design questions in any OS or Computer Architecture is where the abstraction boundaries should be drawn and which side of an abstraction boundary should various operations be pushed. Linus s arguments is that there a flash controller can do a better job of wear leveling, including detecting how worn a particular flash cell might be (for example, perhaps by looking at the charge levels at an analog level and knowing when the last time the cell was programmed), and so it doesn t make sense to try to do wear leveling in a flash file system. Some responsibilities of flash management, such as coalescing newly written blocks into erase blocks to avoid write amplification can be done either on the SSD or in the file system for example, by using a log-structured file system, or some other copy-on-write file system, instead of a rewrite-in-place style file system, you can essentially solve the write amplification problem. In some cases, it s necessary let additional information leak across the abstraction for example, the ATA TRIM command is a way for the file system to let the disk know that certain blocks no longer need to be used. If too much information needs to be pushed across the abstraction, one way or another, then maybe we need to rethink whether the abstraction barrier is in the right place. In addition, if the abstraction has been around for a long time, changing it also has costs, which has to be taken into account. The 512 byte sector LBA abstraction has been around long time, and therefore dislodging it is difficult and costly. For example, the same argument which says that because the underlying hardware details are changing between different generations of SSD is all of these details should be hidden in hardware, was also used to justify something that has been a complete commercial failure for years if not decades: Object Based Disks. One of the arguments of OBD s was that the hard drive has the best knowledge of how and where to store an contiguous stream of bytes, and so perhaps filesystems should not be trying to decide where on disk an inode should be stored, but instead tell the hard drive, I have this object, which is 134 kilobytes long; please store it somewhere on the disk . At least in theory the HDD or SSD could handle all of the details of knowing the best place to store the object on the spinning magnetic media or flash media, taking into account how worn the flash is and automatically move the object around in the case of an SSD, and in the case of the HDD, the drive could know about (real) cylinder and track boundaries, and store the object in the most efficient way possible, since the drive has intimate knowledge about the low-level details of how data is stored on the disk. This theory makes a huge amount of sense; but there s only one problem. Object Based Disks have been proposed in academia and advanced R&D shops of companies like Seagate et.al. have been proposing them for over a decade, with absolutely nothing to show for it. Why? There have been two reasons proposed. One is that OBD vendors were too greedy, and tried to charge too much money for OBD s. Another explanation is that the interface abstraction for OBD s was too different, and so there wasn t enough software or file systems or OS s that could take advantage of OBD s. Both explanations undoubtedly contributed to the commercial failure of OBD s, but the question is which is the bigger reason. And the reason why it is particularly important here is because at least as far as Intel s SSD strategy is concerned, its advantage is that (modulo implementation shortcomings such as the reported internal LBA remapping table fragmentation problem and the lack of ATA TRIM support) filesystems don t need to change (much) in order to take advantage of the Intel SSD and get at least decent performance. However, if the price delta is a stronger reason for its failure, then the X25-M may be in trouble. Currently the 80GB Intel X25-M has a street price of $400, so it costs roughly $5 per gigabyte. Dumb MLC SATA SSD s are available for roughly half the cost/gigabyte (64 GB for $164). So what does the market look like 12-18 months from now? If dumb SSD s are still available at 50% of the cost of smart SSD s, it would probably be worth it to make a copy-on-write style filesystem that attempts to do the wear leveling and write amplification reduction in software. Sure, it s probably more efficient to do it in hardware, but a 2x price differential might cause people will settle for a cheaper solution even if isn t the absolutely best technical choice. On the hand, if prices drop significantly, and/or dumb SSD s completely disappear from the market, then time spent now optimizing for dumb SSD s will be completely wasted. So for Linus to make the proclamation that it s completely stupid to optimize for dumb SSD s seems to be a bit premature. Market externalities for example, does Intel have patents that will prevent competing smart SSD s from entering the market and thus forcing price drops? could radically change the picture. It s not just a pure technological choice, which is what makes projections and prognostications difficult. As another example, I don t know whether or not Intel will issue a firmware update that adds ATA TRIM support to the X25-M, or how long it will take before such SSD s become available. Until ATA TRIM support becomes available, it will be advantageous to add support in ext4 for a block allocator option that aggressively reuses blocks above all else, and avoids using blocks that have never been allocated or used before, even if it causes more in-file system fragmentation and deeper extent allocation trees. The reason for this is at the moment, once a block is used by the file system, at least today, the X25-M has absolutely no idea whether we still care about the contents of that block, or whether the block has since been released when the file was deleted. However, if 20% of the SSD s blocks have never been used, the X25-M can use 20% of the flash for better garbage collection and defragmentation algorithms. And if Intel never releases a firmware update to add ATA TRIM support, then I will be out $400 out of my own pocket for an SSD that lacks this capability, and so adding a block allocator which works around limitations of the X25-M probably makes sense. If it turns out that it takes two years before disks that have ATA TRIM support show up, then it will definitely make sense to add such an optimization. (Hard drive vendors have been historically S-L-O-W to finish standardizing new features and then letting such features enter the market place, so I m not necessarily holding my breath; after all, the Linux block device layer and and file systems have been ready to send ATA TRIM support for about six months; what s taking the ATA committees and SSD vendors so long? <grin> On the other hand, if Intel releases ATA TRIM support next month, then it might not be worth my effort to add such a mount option to ext4. Or maybe Sandisk will make an ATA TRIM capable SSD available soon, and which is otherwise competitive with Intel, and I get a free sample, but it turns out another optimization on Sandisk SSD s will give me an extra 10% performance gain under some workloads. Is it worth it in that case? Hard to tell, unless I know whether such a tweak addresses an optimization problem which is fundamental, and whether or not such a tweak will either be unnecessary, or perhaps actively unhelpful in the next generation. As long as SSD manufacturers force us treat these devices as black boxes, there may be a certain amount of cargo cult science which may be forced upon us file system designers or I guess I should say, in order to be more academically respectable, we will be forced to rely more on empirical measurements leading to educated engineering estimations about what the SSD is doing inside the black box . Heh. Related posts (automatically generated):
  1. Aligning filesystems to an SSD s erase block size I recently purchased a new toy, an Intel X25-M SSD,...

Theodore Ts'o: Should Filesystems Be Optimized for SSD s?

In one of the comments to my last blog entry, an anonymous commenter writes:
You seem to be taking a different perspective to linus on the adapting to the the disk technology front (Linus seems to against having to have the OS know about disk boundaries and having to do levelling itself)
That s an interesting question, and I figure it s worth its own top-level entry, as opposed to a reply in the comment stream. One of the interesting design questions in any OS or Computer Architecture is where the abstraction boundaries should be drawn and which side of an abstraction boundary should various operations be pushed. Linus s arguments is that there a flash controller can do a better job of wear leveling, including detecting how worn a particular flash cell might be (for example, perhaps by looking at the charge levels at an analog level and knowing when the last time the cell was programmed), and so it doesn t make sense to try to do wear leveling in a flash file system. Some responsibilities of flash management, such as coalescing newly written blocks into erase blocks to avoid write amplification can be done either on the SSD or in the file system for example, by using a log-structured file system, or some other copy-on-write file system, instead of a rewrite-in-place style file system, you can essentially solve the write amplification problem. In some cases, it s necessary let additional information leak across the abstraction for example, the ATA TRIM command is a way for the file system to let the disk know that certain blocks no longer need to be used. If too much information needs to be pushed across the abstraction, one way or another, then maybe we need to rethink whether the abstraction barrier is in the right place. In addition, if the abstraction has been around for a long time, changing it also has costs, which has to be taken into account. The 512 byte sector LBA abstraction has been around long time, and therefore dislodging it is difficult and costly. For example, the same argument which says that because the underlying hardware details are changing between different generations of SSD is all of these details should be hidden in hardware, was also used to justify something that has been a complete commercial failure for years if not decades: Object Based Disks. One of the arguments of OBD s was that the hard drive has the best knowledge of how and where to store an contiguous stream of bytes, and so perhaps filesystems should not be trying to decide where on disk an inode should be stored, but instead tell the hard drive, I have this object, which is 134 kilobytes long; please store it somewhere on the disk . At least in theory the HDD or SSD could handle all of the details of knowing the best place to store the object on the spinning magnetic media or flash media, taking into account how worn the flash is and automatically move the object around in the case of an SSD, and in the case of the HDD, the drive could know about (real) cylinder and track boundaries, and store the object in the most efficient way possible, since the drive has intimate knowledge about the low-level details of how data is stored on the disk. This theory makes a huge amount of sense; but there s only one problem. Object Based Disks have been proposed in academia and advanced R&D shops of companies like Seagate et.al. have been proposing them for over a decade, with absolutely nothing to show for it. Why? There have been two reasons proposed. One is that OBD vendors were too greedy, and tried to charge too much money for OBD s. Another explanation is that the interface abstraction for OBD s was too different, and so there wasn t enough software or file systems or OS s that could take advantage of OBD s. Both explanations undoubtedly contributed to the commercial failure of OBD s, but the question is which is the bigger reason. And the reason why it is particularly important here is because at least as far as Intel s SSD strategy is concerned, its advantage is that (modulo implementation shortcomings such as the reported internal LBA remapping table fragmentation problem and the lack of ATA TRIM support) filesystems don t need to change (much) in order to take advantage of the Intel SSD and get at least decent performance. However, if the price delta is a stronger reason for its failure, then the X25-M may be in trouble. Currently the 80GB Intel X25-M has a street price of $400, so it costs roughly $5 per gigabyte. Dumb MLC SATA SSD s are available for roughly half the cost/gigabyte (64 GB for $164). So what does the market look like 12-18 months from now? If dumb SSD s are still available at 50% of the cost of smart SSD s, it would probably be worth it to make a copy-on-write style filesystem that attempts to do the wear leveling and write amplification reduction in software. Sure, it s probably more efficient to do it in hardware, but a 2x price differential might cause people will settle for a cheaper solution even if isn t the absolutely best technical choice. On the hand, if prices drop significantly, and/or dumb SSD s completely disappear from the market, then time spent now optimizing for dumb SSD s will be completely wasted. So for Linus to make the proclamation that it s completely stupid to optimize for dumb SSD s seems to be a bit premature. Market externalities for example, does Intel have patents that will prevent competing smart SSD s from entering the market and thus forcing price drops? could radically change the picture. It s not just a pure technological choice, which is what makes projections and prognostications difficult. As another example, I don t know whether or not Intel will issue a firmware update that adds ATA TRIM support to the X25-M, or how long it will take before such SSD s become available. Until ATA TRIM support becomes available, it will be advantageous to add support in ext4 for a block allocator option that aggressively reuses blocks above all else, and avoids using blocks that have never been allocated or used before, even if it causes more in-file system fragmentation and deeper extent allocation trees. The reason for this is at the moment, once a block is used by the file system, at least today, the X25-M has absolutely no idea whether we still care about the contents of that block, or whether the block has since been released when the file was deleted. However, if 20% of the SSD s blocks have never been used, the X25-M can use 20% of the flash for better garbage collection and defragmentation algorithms. And if Intel never releases a firmware update to add ATA TRIM support, then I will be out $400 out of my own pocket for an SSD that lacks this capability, and so adding a block allocator which works around limitations of the X25-M probably makes sense. If it turns out that it takes two years before disks that have ATA TRIM support show up, then it will definitely make sense to add such an optimization. (Hard drive vendors have been historically S-L-O-W to finish standardizing new features and then letting such features enter the market place, so I m not necessarily holding my breath; after all, the Linux block device layer and and file systems have been ready to send ATA TRIM support for about six months; what s taking the ATA committees and SSD vendors so long? <grin> On the other hand, if Intel releases ATA TRIM support next month, then it might not be worth my effort to add such a mount option to ext4. Or maybe Sandisk will make an ATA TRIM capable SSD available soon, and which is otherwise competitive with Intel, and I get a free sample, but it turns out another optimization on Sandisk SSD s will give me an extra 10% performance gain under some workloads. Is it worth it in that case? Hard to tell, unless I know whether such a tweak addresses an optimization problem which is fundamental, and whether or not such a tweak will either be unnecessary, or perhaps actively unhelpful in the next generation. As long as SSD manufacturers force us treat these devices as black boxes, there may be a certain amount of cargo cult science which may be forced upon us file system designers or I guess I should say, in order to be more academically respectable, we will be forced to rely more on empirical measurements leading to educated engineering estimations about what the SSD is doing inside the black box . Heh. Related posts (automatically generated):
  1. Aligning filesystems to an SSD s erase block size I recently purchased a new toy, an Intel X25-M SSD,...

Alastair McKinstry: Why did the frogs cross the road ?

While driving home from Letterfrack to Moycullen last night after a pleasant days hillwalking with the family, we saw frogs on the road. Hundreds of them. Over the course of over more than more than twenty kilometres, we saw frogs apparently crossing the road. Can anyone tell me what was going on? It didn't look as though they were going somewhere in particular. Has they all recently spawned (unlikely) or where they looking for somewhere to spawn ? This was in the early evening: its possible that they all got to the road at dusk. That part of Conemara is all boggy hillside, with individual widely spaced houses. From up on a hill, the linear feature of the road (the N59), especially in the wet, might have been mistaken for a river. Did they head to the road to spawn? Tags , ,

Antti-Juhani Kaijanaho: grep-dctrl is ten years old

My message Intent to package a Debian control file grepper to WNPP and debian-devel is today a decade old. Apparently, the message predates the invention of the acronym ITP for Intent to Package (the first instance I can find is from May 1999). The changelog reveals that the first upload was on March 1st, 1999. There is unfortunately no record of when the package hit unstable the first time, since dinstall did not send Installed (or even the later Accepted) mails to an archived mailing list at that time. I suppose I might have it in some old private email archive, but more likely it s just gone. Similarly, the first fixed bug (#35527) predates BTS archiving and is now lost. The grep-dctrl program came out of repeated awkward grepping of the dpkg available and status files. Eventually I decided there must be a better way, and failed to find any canned solutions (I was later pointed to sgrep, which didn t look useful enough, and even later to dpkg-awk, but I had already committed to my own solution by then). I wrote a simple C program that processed these files as a sequence of records and did simple substring searches in each record. I rapidly added support for field selection, regular expressions and output field selection. By version 1.3a of March 2000 (which was released with Debian 2.2 potato ), the program was as good as it was going to get with one exception. Make disjunctive searches possible, said my TODO file those days. Conjunctive searches (that is, AND-searches) were possible even then by using more than one grep-dctrl command in a pipeline. Disjunctive (OR) was not possible. The problem was not so much that it would be hard to program (although the program s internal structure wasn t very good, to be honest, making extensive changes difficult), it was more a problem of coming up with a good command line syntax. Another thing that bothered me with the old grep-dctrl was how to implement Ben Armstrong s feature request. Again, the programming part wasn t the problem, the problem was coming up with a good, clean semantics for the feature. It was finally the appearance of ara in 2003 that got me moving again. Ara s author proudly compared eir program to grep-dctrl, claiming that my program did not do disjunctive searches while ara does. Competition being good for the soul, I took it as a challenge. In April 2003 I announced a complete rewrite of grep-dctrl, which was completed in January 2004 (the 2.0 release). The rewrite changed the way the command line was handled even though the usual Unix switch style is still used, the command line is regarded as a language with a parser (first an operator-precedence parser, then a recursive-descent one). The command line is transformed into an interpreted stack-based language which drives the actual grepping. The rewrite also generalised the internal data structures into an internal library which could be used to write other tools. The first such tool was sort-dctrl (introcuded in 2.7, June 2005), which was soon followed by tbl-dctrl (2.8, July 2005). The later appearance of join-dctrl (2.11, August 2007) finally allowed me to close Ben Armstrong s longstanding feature request mentioned above. The unpronounceable part of the names, dctrl , is an abbreviation for Debian control , which I decided to call the file format used by dpkg. Some people call it a RFC-822 format, but that really is a misnomer, since the differences between dctrl and RFC-822 outweigh the mainly superficial similarities (and the historical connection). I did consider calling my program dpkg-grep, but I didn t feel like I had the right to invade the dpkg namespace. The later rename to dctrl-tools reflects the fact that there are now several tools, grep-dctrl being just the oldest. I have several plans for the dctrl-tools suite, but my time and energy are mostly claimed by other responsibilities. The suite is currently team-maintained, but unfortunately the team is not very active. I would love it if I weren t the most active one with my busy schedule! Feel free to pop in on the dctrl-tools-devel mailing list, and to look at the Git repository and the todo list. If you decide to participate, please follow the rules.

21 February 2009

Biella Coleman: Visualizing Copyright

Copyright Digital Slider and the Spinner

Next.

Previous.