Search Results: "Lucas Nussbaum"

2 March 2011

Lucas Nussbaum: Banshee and Ubuntu

[ If you haven't heard of this debate yet, you probably want to read Vincent Untz's and Mark Shuttleworth's blog posts. ] Last year, I gave a talk at FOSDEM about Debian and Ubuntu (slides of a slightly updated version). One of my points was that Debian has better values, being a volunteer-driven project where decisions are taken in the open. In contrast, Ubuntu is a project managed and controlled by Canonical, and recent history has shown that Canonical had no problem imposing some decisions to the developers community: first with the inclusion of UbuntuOne, then the switch from Google to Yahoo! to Google as the default search engine, both to increase revenue streams. So one should not be surprised by the Banshee story. I find Mark s justification quite difficult to buy, and similar to Apple s model where 30% of the revenues from the App Store go to Apple, and 70% to the seller of the application. For those wondering how much work was done by Canonical directly on the Banshee package: the banshee package in Ubuntu natty is based on the package currently in Debian experimental. The package is mainly maintained in Debian by an Ubuntu developer not paid by Canonical AFAIK, Chow Loong Jin. There are some differences between the Debian package and the Ubuntu package, which are fairly limited (full diff here) and mainly about enabling UbuntuOne and disabling the other music stores. That patch itself apparently was provided by Jo Shields, who doesn t seem to be a canonical employee. (Feel free to correct me) I think that one of the conclusions to draw from this story is that we now have several proofs that Ubuntu isn t a volunteer-driven project, and that volunteer contributors should really decide whether they are OK with working for free for Canonical, or if their free time would be better spent on other projects where they actually have a chance to influence decisions. From the Debian POV, I m still convinced that we should take the feedback that we receive from Ubuntu in consideration to improve our Debian packages (by looking at patches made by Ubuntu, or at bugs reported in Launchpad). But my motivation for contributing to Ubuntu directly has just diminished a bit more (not that it was very high before).

25 February 2011

Lucas Nussbaum: Debian is changing

I ve been playing with snapshot.d.o, which is a fantastic resource to look at how Debian has been changing over the years.
Below are some preliminary results about various aspects of source packages: format, packaging helpers, and patch systems.
For clarification: Random comments: If you have other ideas of things to investigate using snapshot.d.o, don t hesitate to ask in comments.

21 February 2011

Lucas Nussbaum: Debian Spring Cleanup

We have many orphaned packages in Debian. Those are packages that no longer have a maintainer willing to care for them. They are a problem for several reasons. Often, there have better alternatives (that s often why the maintainer gave up maintenance), but users keep installing them because they show up in apt-cache search. Or, they might be broken, without anybody noticing because they only have a few users. Also, developers need to take them into account. They might slow down transitions, have RC bugs that need fixing, etc. So, there are good reasons to remove some of the orphaned packages from the Debian archive. Not all of them. But those that have been orphaned for a long time without anybody stepping up to take over the maintenance, have very few users according to popcon, etc. Removing some of them also makes it easier to expose the ones that should really find an adopter. Also, that removal is not final. The good reason to do this now is that we are at the beginning of a release cycle, and that it will be easy to reintroduce packages that are really needed during the next two years. Also, packages (both source and binaries) are still available on snapshot.debian.org, so users can still install them from there, and re-uploading them to Debian is very easy if someone wants to adopt them. So, how do I proceed? I use bapase, and go through specific sets of packages (using a custom version of bapase to restrict the list to those packages). For example, orphaned packages that have a popcon lower than 50, have been orphaned for more than 500 days, and whose orphaning bugs have not been modified in the last 100 days. Then, I read the bug log, and decide if there s a reason to keep the package. If there s not, I request the removal. As of today, there are 471 orphaned packages in Debian. The goal is not to get down to zero, but it s certainly possible to get down to 200 or 250. How can you help? Start from the list, and find some easy targets. You can take a look at bug #483252 if you want to re-use my template. Also, one thing that is currently missing in the process is that orphaned packages that have reverse-dependencies should not be removed without removing their dependencies (if appropriate). An easy and efficient way to know if a given package has reverse (build-)?dependencies would be great, so I could integrate this info into bapase and exclude those packages from the list, or mark them somehow.

16 February 2011

Lucas Nussbaum: Introducing the Debian packaging tutorial

One of the common complains about Debian packaging is that it s hard to learn because, while there is quite a lot of high-quality documentation, it is often written more as a reference manual than as a tutorial: it s great if you already know everything and want to check some detail, but not so great if you want to learn everything from scratch. I have been volunteered (i.e, someone decided I volunteered) for a Debian packaging tutorial at work, so I decided to give a try at tackling this issue. I also volunteered (voluntarily this time) for a similar talk at RMLL 2011 to make sure I would be forced to do the work and prepare the actual tutorial. I m also considering teaching this next year in Licence Pro ASRALL, but I haven t made up my mind about it yet. The result is a work in progress (hey, I still have a lot of time), but in the release-early-release-often tradition, I m making it public now in the hope that someone will pick up the idea and do all the work for me (you never know). I ve decided to create a set of slides using Latex Beamer. The current version can be found here. The sources are available in a git repository, and all contributions are welcomed (including plain comments or suggestions). The last slide is the current TODO list.

15 February 2011

Torsten Werner: Are we slaves?

We still have more than 300 new packages in Debian that needs manual checks and we have a lot of open bugs. I don t get paid for working on dak (the Debian Archive Kit) nor do I feel a deep passion on doing such work. I am contributing to Debian because it is a great project and because contributing is my way to say Thank you to all who made (and still make) it possible. BUT: are we slaves that are supposed to do any stuff for other people? Yesterday someone asked for an enhancement for dak via IRC. I ve kindly asked him: do you want to send a patch? (words are copied from the IRC session). Instead of answering no he sent a patch for 2 plain text files within 4 minutes after I told him the URL to dak s git repository. I could not do it faster by myself and I ve got many other open tasks. I ve applied the patch and activated it on ftp-master.debian.org. But today he ranted about my rudeness and he did not even managed to quote me correctly ( Please send a patch ).

Lucas Nussbaum: Re: please send a patch

It seems that Matthew Palmer misread my blog post as a complaint against developers asking for patches in exchange of pet feature requests. He really should pay more attention, since I gave pet feature requests as an example of case where it would be appropriate to ask for a patch:
Of course, there are cases where it s perfectly reasonable to ask for a patch: when the task is expected to take hours, or when the result is of limited interest to everybody except the demander.
But even then, it s not clear. This morning I got an email from a someone involved in PHP packages maintenance, who said that Bugs Search @ UDD was a great tool, but that he would love to have a way to list all bugs affecting packages with the implemented-in::php debtag.
To produce a working patch for this would probably take him at least an hour. You need to set up a copy of the CGI on alioth, understand the DB structure, dig into the code, etc. If you don t understand SQL and Ruby, it could be a really difficult process. Also, it s probably quite uninteresting for him to do that, since he is unlikely to stick around developing UDD.
Instead, it didn t take me more than 5 minutes to produce a one-liner.
The net result for Debian in that case? 55 minutes saved by a developer. Update:
Torsten Werner wrote an angry reply to my post. It s true that yesterday s episode triggered my blog post, because I felt quite frustrated to have to provide a patch for something that simple, and would have preferred to use the time for a Debian task where I would be more efficient. But I was not particularly angry at that episode, since that s something I ve seen on several occasions. That s also why I did not mention any team in particular.
The feature request I was making was reasonable, and cannot really be considered a pet feature request (though I might be biased with my QA hat on): mentionning in the dak templates used for bug closure that packages removed from Debian can still be found on snapshot.d.o. The fact that he thinks that addressing this himself turns him into a slave raises interesting questions.

Lucas Nussbaum: Please send a patch

There s a frequent pattern in the Debian community where someone would suggest an improvement to some package or service, and the person responsible for it would reply with please send a patch . Of course, there are cases where it s perfectly reasonable to ask for a patch: when the task is expected to take hours, or when the result is of limited interest to everybody except the demander.
But often, it would be much more efficient for the person responsible for the service to take 5 or 10 minutes to make the change, than for the demander to spend half an hour learning how to contribute to this service, writing an untestable patch, and sending it back to the person responsible for the service for integration. It is important to see this problem as the issue of maximizing the usefulness of the resources we have. Often, we are in a situation where we could either: It would be much better if, instead of saying Please send a patch , people would say Are you interested in contributing a patch, or should I make the change myself? . Remember that each time you ask someone to take some time to contribute a patch in an area where he is not an efficient contributor, you take some time away from him that could be used to contribute something in an area where he is efficient.

6 February 2011

Lucas Nussbaum: mutt + notmuch = perfect mail setup?

Following Our Leader, I ve just switched to using notmuch with mutt for searching (instead of maildir-utils, previously). I strongly recommend this setup to all mutt users. As a bonus, accented characters ( , , ) are now handled correctly while searching, which is quite useful in french.

25 January 2011

Paul van Tilburg: Also Giving up on Ruby Packaging

I have just sent a mail to the Debian/Ruby Extras mailing list stating that I will also give up most of my Ruby packaging efforts and reduce my involvement in the team. I am following Lucas Nussbaum in his decision. Although our reasons are slightly different, I subscribe most of what he says in his blog post. Besides that, I haven t done much Ruby programming in the past years, making me lose the motivation for the huge load of packages I am responsible for and thus subconsciously perceive that as a burden, as I feel guilty not having/making time to work on them. So, what does this mean? I will handover (co)maintainership of over 50 libraries. Many of which have dead upstream, unfortunately. Also will I be looking for someone to take over the administrative tasks of heading the team such as mailing list moderation and Alioth repository access. I will keep maintaining some programs that I still use, such as Camping, and am approachable for (mentor) uploads. Meanwhile I will continue my work on my PhD thesis, which is progressing well and should lead to my defence (hopefully) somewhere around September this year. That said, I will try to keep tinkering with Ruby and other F/OSS projects. Oh, and also to get more motivated and some inspiration

6 January 2011

Gunnar Wolf: Back with Ruby on Debian polemics

Once again, a polemic regarding how to properly integrate the Ruby language and libraries with the Debian distribution has been ignited. Similar arguments were presented in November and December 2008 and September 2010 And excuse me for just refering to my own blog, but there are links to some other posts from there... and there will surely be many others I just missed. There were even nice collaboration attempts (such as DebGem, announced in January 2009... But which apparently never stuck on, as it is still listed as in its public Beta period and lists only support for Debian 4.0 and Ubuntu 8.04 and 8.10). Some days ago, while I was on vacation, I received a mail from Lucas Nussbaum expressing the burnout he had been suffering from this situation. Some days after that, he posted a corresponding blog post - Giving up on Ruby packaging. The comments following his post are most interesting The one comment I'd like to highlight (noting I skimmed a whole deal of them) is by Paul Brannan, one of the original RubyGems authors. Yes, possibly the design criteria for Gems included some mutually-conflicting goals, and they cover some of Debian's goals. Unfortunately, this conflicting criteria... Was resolved to the opposite way we would have wanted (specifically, that a sysadmin should be able to install software with the same tools he was already familiar with). And of course, there are deeper disagreements, which are rightly stemmed from different priorities: Debian (+derivatives) is meant to make user's and sysadmin's lifes simpler. Gems are meant to make developers' lifes simpler. And while all developers are some sort of users, and sysadminsthe reverse is (obviously) not so. A developer maintaining several different codebases will surely suffer (well, this particular developer has surely suffered) if he has to maintain them all using coherent versions of system libraries. When a system is programmed, its developers want to use the greatest, latest tools to offer the best experience and functionality. And it's natural for libraries to evolve over time. However, any sysadmin will grind their teeth at the prospect of having many different versions of libraries, as the Gem model proposes. Why? Well, I was bitten by a minor example of it Bug fixes do not get backported. Of course, if simple bugs are not backported, security and stability of the system as a whole suffers. Often, new software versions will not only add functionality, but improve on how things are done. Not only bugs get corrected, but we get better response times, better memory handling, etc. It might sound harsh to say this, and even more, as a developer it feels I am talking against myself But while development time should be minimized during the system design/implementation, once systems are in production, it should be used to allow for friendlier sysadmin relations. I have to wear both hats at my real-life job. So, what is my take on this? Of course, blaming myself for choosing the wrong version is a no-go. When evaluating a library before using it in my projects, I try to look at its history and API-stability. APIs that change often speak about immature libraries which are still trying to get the right way to implement their functionality; just-started projects might be great and revolutionary, but they do not yet show any kind of long-term committment... And can lead (and have often led me) to painful rewrites when the Next Big Thing is reached. As for the users: Even if your favorite language has the best and friendliest distribution method, users just do not care about what language was a particular piece of software implemented in. Users want to be able to install and uninstall with their system's usual tools. Directly using Gems, CPAN, PEAR and whatnot is just unnatural for users, however convenient they are for developers. Distributions offer a non-technical advantage in this regard as well: A human filter. As an example, as I write this there are 19392 Ruby Gems and 19092 Perl modules in CPAN (note that CPAN stores some older versions but discourages authors from keeping too many - So no, they are still very different in size). Debian has around 30,000 packages. Why is that? Because all Debian packages _must_ be human-generated, human-reviewed, human-submitted. This means, a person must think each packaged piece of code is worth packaging, is stable enough and provides value to users as it is, and is fit for being part of a stable release. I am not saying with this that 90% of CPAN and Gems are crap I am saying that they are probably early implementations, to be installed, tested and improved by developers, and still not apt for general public use. Or maybe not interesting enough to be packaged as a service for the non-techie public at large. Oh, and one last point: Ruby is not a new language anymore. It is a mature, powerful language with different implementations, every time more stable... But it is a language deeply affected by the (not so new anymore either) the appearance of Rails. Why do I say this? Because although the language is in no way tied to Web development, many of its strongest uses are Web-oriented. How does this affect the current discussion? Well, because many people argue that users are no longer needed to install the software. Web systems are installed by sysadmins and used via a Web browser, and sysadmins are expected be more skilled than casual users. Still, in Debian (and in other distributions, surely) we try to make sysadmin's lives simpler I have (again, talking out of personal experience) installed several webapps (and system tools, and whatnot) for which I never followed any instructions besides aptitude install foo Using different languages, frameworks, and so on. Can I troubleshoot their installs? Probably, as there is a common logic for how the distribution I have chosen and specialized in works. Can I find causes for bugs in them? Possibly, although there are some languages and frameworks I dare not touch. Can I get help on getting them out of a tight spot? Surely, as there is a central bug tracking system for my distribution And one of the maintainer's tasks is to separate the problems related to the distribution (packaging, installing, simple user questions and misconceptions) to those derived from real bugs upstream. Anyway I am not saying that our way is the best way. No, by a long shot. Again, developers should have an easy, convenient way for installing whatever they want to play with. And to publish it without jumping through hoops. With this post, I'm only trying to express again why Debian works the way it does. And hope for better cooperation in the future. And as for several comments of what I read in Lucas' post, I think that there is interest for this synergy among some of the most committed Ruby people.

2 January 2011

Lucas Nussbaum: Giving up on Ruby packaging

I have finally reached a decision regarding my involvement in the Debian Ruby packaging efforts. I have decided to stop. This has been a very hard decision to make. I have invested huge amounts of time in that work over the years. I still love the language, and will continue to use it on a daily basis for my own developments. I still hope that it will succeed. I know that some people will be disappointed by that decision (and that others will think your work was useless anyway, people should use RVM and rubygems ). But I also know that I won t be able to push for all the required changes alone. I just don t have the time, nor the motivation. For the record, here are the changes I would have liked to see in the Ruby community. The core Ruby development community should mature.
The core Ruby development community is still dominated by Japanese developers. While not a bad thing in itself, it is easily explained by the fact that the main development mailing list, where most of the important decisions are taken, is in japanese. ruby-dev@ should be closed, and all the technical discussions should happen on the english-speaking ruby-core@ list instead. The release management process should also improve. Currently, it looks like a total mess. The following Ruby development branches are actively maintained:
ruby_1_8 (106 commits over the last six months)
ruby_1_8_6 (4 commits over the last six months)
ruby_1_8_7 (35 commits over the last six months)
ruby_1_9_1 (4 commits over the last six months)
ruby_1_9_2 (227 commits over the last six months)
trunk (1543 commits over the last six months) While the state of the ruby_1_8_6 and ruby_1_9_1 branches is clear (very important bugfixes only), the state of all of the other branches is rather unclear.
What s the stable Ruby branch? 1.8 or 1.9? If it s 1.9, why are people still actively developing in the ruby_1_8 branch? How long will they continue to be maintained in parallel, dividing the manpower? Is a Ruby 1.8.8 release to be expected? Will it be ABI/API compatible with 1.8.7? Is the ruby_1_8_7 branch really bugfixes-only? How much testing of it has been done? If it s bugfixes-only and regression-free, I should push it to Debian squeeze, due to be released in a few weeks. But would you recommend that? Due to past breakages in the ruby_1_8_7 branch, it s unlikely that we will do it.
Is the ruby_1_9_2 a regression-free, bugfix-only branch? If yes, isn t 227 commits over 6 months a lot? What will be the version of the next release of trunk ? When is it expected? Will it be ABI-compatible with the current ruby_1_9_2 branch? API-compatible?
New releases in the 1.8.7 and 1.9.2 branches were done on december 25th. Why were they no betas or RCs allowing wider testing? How much testing has been done behind the scenes? Most of those questions have no clear answer. The Ruby development community should build a common understanding of the status of the various branches, and of their release expectations. Releasing on december 25th of each year sounds fun, but is releasing when everybody is on vacation really a good idea? It would be fantastic to have something similar to Python Enhancement Proposals in the Ruby community. But having open discussions in english about the major issues would already be great. Ruby is not just the interpreter.
The Ruby development community should clearly define what the Ruby platform is. There are some big players, like Rails, and newer interpreter releases should not be done before ensuring that those big players still work. Also, since we have alternative Ruby interpreters, like JRuby, Rubinius and MacRuby, we need a clear process on how they integrate with the rest of the ecosystem. For example, having each of them rely on their own outdated fork of the whole stdlib is ridiculous, since it s not where they compete. The Ruby community should acknowledge that RVM and Rubygems are not for everybody. People who say so should be laughed at. Of course, RVM and Rubygems are nice tools for some people. But it is completely wrong to believe that compiling from source using RVM should be the standard way of installing Ruby, or that all people interested in installing Redmine should know that Ruby has its own specific packaging system. The Ruby community should work with their target platforms to improve how Ruby is distributed instead of reinventing the wheel. That includes Debian, but also RedHat-based distros, for example. It is likely that it won t be possible to reach a one-size-fits-all situation. But that s real life. Some people in the Ruby community should stop behaving like assholes. As one of the Debian Ruby maintainers, I have been routinely accused of creating crippled packages on purpose (FTR, I don t think that the Debian packages are crippled, despite what the rumors says). Debian is not the only target of that. Just yesterday, someone called for abandonning YARV (the new Ruby VM in Ruby 1.9), calling it Yet Another Random Vailure. This kind of comments is really hurting the people who are investing their free time in Ruby, and is turning away people who consider getting involved. In Debian, we have had a lot of problems getting people to help with Ruby maintenance since they are getting shit from the community all the time. So, what s the future for Ruby in Debian? Update: there s also a number of interesting comments about this post on this site.
Update 2: First, thanks a lot for all the interesting comments. I will make some follow-up posts trying to summarize what was said. It seems that this post also triggered some reactions on ruby-core@, with Charles Olivier explaining the JRuby stdlib fork, and Yui Naruse clarifying that all questions are welcomed on ruby-core@. This is great, really.

25 December 2010

Lucas Nussbaum: Trouble connecting with Nokia N900 as 3G modem?

If you are having trouble connecting to the Internet using your Nokia N900 as a 3G modem with Network Manager, you should check those bugs. (In short: bug fixed upstream, but affecting the version in Debian. Using the package from Ubuntu works.)

Lucas Nussbaum: Merry christmas!

Seen on The Ruby Reflector (a blog aggregator for the Ruby community):
Under no circumstance should you install Ruby, Rubygems or any Ruby-related packages from apt-get. This system is out-dated and leads to major headaches. Avoid it for Ruby-related packages. We do Ruby, we know what s best. Trust us.
(source) It s always nice to see people appreciate your work. :-)

6 December 2010

Raphaël Hertzog: State of the Debian-Ubuntu relationship

Debian welcoming contributions from derivativesThe relationship between Debian and Ubuntu has been the subject of many vigorous debates over the years, ever since Ubuntu s launch in 2004. Six years later, the situation has improved and both projects are communicating better. The Natty Narwhal Ubuntu Developer Summit (UDS) featured like all UDS for more than 2 years a Debian Health Check session where current cooperation issues and projects are discussed. A few days after that session, Lucas Nussbaum gave a talk during the mini-Debconf Paris detailing the relationship between both projects, both at the technical and social level. He also shared some concerns for Debian s future and gave his point of view on how Debian should address them. Both events give valuable insights on the current state of the relationship. Lucas Nussbaum s Debian-Ubuntu talk Lucas started by introducing himself. He s an Ubuntu developer since 2006 and a Debian developer since 2007. He has worked to improve the collaboration between both projects, notably by extending the Debian infrastructure to show Ubuntu-related information. He attended conferences for both projects (Debconf, UDS) and has friends in both communities. For all of these reasons, he believes himself to be qualified to speak on this topic. Collaboration at the technical level He then quickly explained the task of a distribution: taking upstream software, integrating it in standardized ways, doing quality assurance on the whole, delivering the result to users, and assuring some support afterward. He pointed out that in the case of Ubuntu, the distribution has one special upstream: Debian. Indeed Ubuntu gets most of its software from Debian (89%), and only 7% are new packages coming from other upstream projects (the remaining 4% are unknown, they are newer upstream releases of software available in Debian but he was not able to find out whether the Debian packaging had been reused or not). From all the packages imported from Debian, 17% have Ubuntu-specific changes. The reasons for those changes are varied: bugfixes, integration with Launchpad/Ubuntu One/etc., or toolchain changes. The above figures are based on Ubuntu Lucid (10.04) while excluding many Ubuntu-specific packages (language-pack-*, language-support-*, kde-l10n-*, *ubuntu*, *launchpad*). The different agendas and the differences in philosophy (Debian often seeking perfect solutions to problems; Ubuntu accepting temporary suboptimal workarounds) also explain why so many packages are modified on the Ubuntu side. It s simply not possible to always do the work in Debian first. But keeping changes in Ubuntu requires a lot of work since they merge with Debian unstable every 6 months. That s why they have a strong incentive to push changes to upstream and/or to Debian. There are 3 channels that Ubuntu uses to push changes to Debian: they file bug reports (between 250 to 400 during each Ubuntu release cycle), they interact directly with Debian maintainers (often the case when there s a maintenance team), or they do nothing and hope that the Debian maintainer will pick up the patch directly from the Debian Package Tracking System (it relays information provided by patches.ubuntu.com). Lucas pointed out that those changes are not the only thing that Debian should take back. Ubuntu has a huge user base resulting in lots of bug reports sitting in Launchpad, often without anyone taking care of them. Debian maintainers who already have enough bugs on their packages are obviously not interested in even more bugs, but those who are maintaining niche packages, with few reports, might be interested by the user feedback available in Launchpad. Even if some of the reports are Ubuntu-specific, many of them are advance warnings of problems that will affect Debian later on, when the toolchain catches up with Ubuntu s aggressive updates. To make this easier for Debian maintainers, Lucas improved the Debian Package Tracking System so that they can easily get Ubuntu bug reports for their packages even without interacting with Launchpad. Human feelings on both sides Lucas witnessed a big evolution in the perception of Ubuntu on the Debian side. The initial climate was rather negative: there were feelings of its work being stolen, claims of giving back that did not match the observations of the Debian maintainers, and problems with specific Canonical employees that reflected badly on Ubuntu as a whole. These days most Debian developers find something positive in Ubuntu: it brings a lot of new users to Linux, it provides something that works for their friends and family, it brings new developers to Debian, and it serves as a technological playground for Debian. On the Ubuntu side, the culture has changed as well. Debian is no longer so scary for Ubuntu contributors and contributing to Debian is The Right Thing to do. More and more Ubuntu developers are getting involved in Debian as well. But at the package level there s not always much to contribute, as many bugfixes are only temporary workarounds. And while Ubuntu s community follows this philosophy, Canonical is a for-profit company that contributes back mainly when it has compelling reasons to do so. Consequences for Debian In Lucas s eyes, the success of Ubuntu creates new problems. For many new users Linux is a synonym for Ubuntu, and since much innovation happens in Ubuntu first, Debian is overshadowed by its most popular derivative. He goes as far as saying that because of that Debian becomes less relevant . He went on to say that Debian needs to be relevant because the project defends important values that Ubuntu does not. And it needs to stay as an independent partner that filters what comes out of Ubuntu, ensuring that quality prevails in the long term. Fixing this problem is difficult, and the answer should not be to undermine Ubuntu. On the contrary, more cooperation is needed. If Debian developers are involved sooner in Ubuntu s projects, Debian will automatically get more credit. And if Ubuntu does more work in Debian, their work can be showcased sooner in the Debian context as well. The other solution that Lucas proposed is that Debian needs to communicate on why it s better than Ubuntu. Debian might not be better for everybody but there are many reasons why one could prefer Debian over Ubuntu. He listed some of them: Debian has better values since it s a volunteer-based project where decisions are made publicly and it has advocated the free software philosophy since 1993. On the other hand, Ubuntu is under control of Canonical where some decisions are imposed, it advocates some proprietary web services (Ubuntu One), the installer recommends adding proprietary software, and copyright assignments are required to contribute to Canonical projects. Debian is also better in terms of quality because every package has a maintainer who is often an expert in the field of the package. As a derivative, Ubuntu does not have the resources to do the same and instead most packages are maintained on a best effort basis by a limited set of developers who can t know everything about all packages. In conclusion, Lucas explained that Debian can neither ignore Ubuntu nor fight it. Instead it should consider Ubuntu as a chance and should leverage it to get back in the center of the FLOSS ecosystem . The Debian health check UDS session While this session has existed for some time, it s only the second time that a Debian Project Leader was present at UDS to discuss collaboration issues. During UDS-M (the previous summit), this increased involvement from Debian was a nice surprise to many. Stefano Zacchiroli the Debian leader collected and shared the feedback of Debian developers and the session ended up being very productive. Six months later is a good time to look back and verify if decisions made during UDS-M (see blueprint) have been followed through. Progress has been made On the Debian side, Stefano set up a Derivatives Front Desk so that derivative distributions (not just Ubuntu) have a clear point of contact when they are trying to cooperate but don t know where to start. It s also a good place to share experiences among the various derivatives. In parallel, a #debian-ubuntu channel has been started on OFTC (the IRC network used by Debian). With more than 50 regulars coming from both distributions, it s a good place for quick queries when you need advice on how to interact with the distribution that you re not familiar with. Ubuntu has updated its documentation to prominently feature how to cooperate with Debian. For example, the sponsorship process documentation explains how to forward patches both to the upstream developers and to Debian. It also recommends ensuring that the patch is not Ubuntu-specific and gives some explanation on how to do it (which includes checking against a list of common packaging changes made by Ubuntu). The Debian Derivative Front Desk is mentioned as a fallback when the Debian maintainer is unresponsive. While organizing Ubuntu Developer Week, Ubuntu now reaches out to Debian developers and tries to have sessions on working with Debian . Launchpad has also been extended to provide a list of bugs with attached patches and that information has been integrated in the Debian Package Tracking system by Lucas Nussbaum. Still some work to do Some of the work items have not been completed yet: many Debian maintainers would like a simpler way to issue a sync request (a process used to inject a package from Debian into Ubuntu). There s a requestsync command line tool provided by the ubuntu-dev-tools package (which is available in Debian) but it s not yet usable because Launchpad doesn t know the GPG keys of Debian maintainers. Another issue concerns packages which are first introduced in Ubuntu. Most of them have no reason to be Ubuntu-specific and should also end up in Debian. It has thus been suggested that people packaging new software for Ubuntu also upload them to Debian. They could however immediately file a request for adoption (RFA) to find another Debian maintainer if they don t plan to maintain it in the long term. If Ubuntu doesn t make this effort, it can take a long time until someone decides to reintegrate the Ubuntu package into Debian just because nobody knows about it. This represents an important shift in the Ubuntu process and it s not certain that it s going to work out. As with any important policy change, it can take several years until people are used to it. Both issues have been rescheduled for this release cycle, so they re still on the agenda. This time the UDS session was probably less interesting than the previous one. Stefano explained once more what Debian considers good collaboration practices: teams with members from both distributions, and forwarding of bugs if they have been well triaged and are known to apply to Debian. He also invited Ubuntu to discuss big changes with Debian before implementing them. An interesting suggestion that came up was that some Ubuntu developers could participate in Debcamp (one week hack-together before Debconf) to work with some Debian developers, go through Ubuntu patches, and merge the interesting bits. This would nicely complement Ubuntu s increased presence at Debconf: for the first time, community management team member Jorge Castro was at DebConf 10 giving a talk on collaboration between Debian and Ubuntu. There was also some brainstorming on how to identify packages where the collaboration is failing. A growing number of Ubuntu revisions (identified for example by a version like 1.0-1ubuntu62) could indicate that no synchronization was made with Debian, but it would also identify packages which are badly maintained on the Debian side. If Ubuntu consistently has a newer upstream version compared to Debian, it can also indicate a problem: maybe the person maintaining the package for Ubuntu would be better off doing the same work in Debian directly since the maintainer is lagging or not doing their work. Unfortunately this doesn t hold true for all packages since many Gnome packages are newer in Ubuntu but are actively maintained on both sides. Few of those discussions led to concrete decisions. It seems most proponents are reasonably satisfied with the current situation. Of course, one can always do better and Jono Bacon is going to ensure that all Canonical teams working on Ubuntu are aware of how to properly cooperate with Debian. The goal is to avoid heavy package modifications without coordination. Conclusion The Debian-Ubuntu relationships used to be a hot topic, but that s no longer the case thanks to regular efforts made on both sides. Conflicts between individuals still happen, but there are multiple places where they can be reported and discussed (#debian-ubuntu channel, Derivatives Front Desk at derivatives@debian.org on the Debian side or debian@ubuntu.com on the Ubuntu side). Documentation and infrastructure are in place to make it easier for volunteers to do the right thing. Despite all those process improvements, the best results still come out when people build personal relationships by discussing what they are doing. It often leads to tight cooperation, up to commit rights to the source repositories. Regular contacts help build a real sense of cooperation that no automated process can ever hope to achieve.
This article was first published in Linux Weekly News. You can get my monthly summary of the Debian/Ubuntu news, all you have to do is to click here to subscribe to my free newsletter.

5 comments Liked this article? Click here. My blog is Flattr-enabled.

22 November 2010

Raphaël Hertzog: How Ubuntu builds up on Debian

I have been asked how Ubuntu relates to Debian, and how packages flow from one to the other. So here s my attempt at clarifying the whole picture.

Where do the packages come from? Most packages are created by Debian contributors and they are uploaded in Debian unstable (or Debian experimental). New packages are reviewed by the Debian ftpmasters before being accepted in the official archive. The packages are held in the NEW queue until the review is over, and the time spent there varies between a few hours and a few months (usually they are processed within one week or two). Ubuntu imports all the official Debian packages, but they also add some packages of their own. About 7% of the Ubuntu packages are third-party software that have been packaged for Ubuntu but not for Debian. What are the changes made by Ubuntu? From all the source packages coming from Debian, 17% have additional changes made by Ubuntu. Many of them are part of the main repository, which is actively maintained by Canonical and Ubuntu core developers. The universe repository is usually closer to the official Debian packages. Many of the changes made by Ubuntu are the results of the decisions taken during the Ubuntu Developer Summit in order to reach specific goals: provide a better user interface, offer faster boot times, become a better platform for third-party software developers, offer a good integration with their online services (Launchpad, Ubuntu One), etc. Other changes are simply the result of fixing bugs reported by Ubuntu users. Note that even non-modified source packages will result in different binary packages for Ubuntu. That s because Ubuntu has made changes to the build environment. They only support Intel-based computers with a 686-class (or newer) CPU, they enable some compiler options that Debian doesn t, etc. And all binary packages are modified by a program called pkgbinarymangler. Ubuntu s release cycle and the relation with Debian Ubuntu releases every 6 months (that s what time based releases is about). Debian has a very different schedule. How does Ubuntu manage to reuse Debian s work? Ubuntu imports packages from Debian unstable (even experimental sometimes) to get the newest packages. If the Ubuntu package already has Ubuntu-specific changes, they merge their changes in the updated Debian package. Otherwise the Debian package is simply grabbed and rebuilt in Ubuntu. This works well because Debian unstable is much more usable than the name suggests. And this process only goes on during the first 2 months of the cycle (until the Debian Import Freeze), so there s plenty of time afterward to fix the biggest problems. In the third and fourth month, it s still possible to pick updated packages from Debian but it must be requested by a developer, it won t be done automatically. At the end of the fourth month, the feature freeze is put in place. The 2 months left are dedicated to bug fixing and polishing the distribution. There are various sub-freezes that happen in this period, you can check the Natty release schedule as an example. Picking updated packages from Debian is now the exception, it will only be allowed if the update on the Debian side is a bug-fix only release. Credits: some figures taken from a talk of Lucas Nussbaum, they were collected based on the packages available in the Lucid Lynx release of Ubuntu. Click here to subscribe to my newsletter and get my monthly update on what s going on in Debian and Ubuntu.

15 comments Liked this article? Click here. My blog is Flattr-enabled.

16 November 2010

Lucas Nussbaum: Feedback on Gandi hosting?

For the last few years, I ve been a mostly happy customer of Sivit (now bought by Nerim). But recently, technical problems have become more frequent, and today, for the first time, a crash made me lose some data which was rather painful to restore from backups. Also, they are still stuck on with an ancient 2.6.16 kernel. So, I m looking into alternatives, and a promising one is Gandi hosting, based on virtual machines like Sivit. Thanks!

3 November 2010

Lucas Nussbaum: Triaging X Strike Force bugs with UDD Bugs Search

Thanks to the work of Julien Viard de Galbert, UDD Bugs Search just gained the ability to help the triaging of X Strike Force bugs. He worked on the patch by installing his own copy of bugs.cgi on alioth. Don t hesitate to do the same with your team!

1 November 2010

Julien Valroff: Mini DebConf Paris 2010

I am now back home after 2 days spent at the first mini DebConf Paris. It was really nice to meet a lot of people, and to be able to discuss various aspects of Debian and Free Software in general. People came from all over Europe (at least from the UK, Spain, Italy, Germany, Switzerland, Austria, Greece, and obviously a majority of French people). All the talks I have been able to attend were really interesting, and some of the slides have already been made available by their author: I have also taken part to the keysigning party, which will now allows me to use my newly generated 4096R key. The group picture is also online, as well as a more comprehensive gallery. Thanks a lot to the organisation team, you guys have made an awesome work! See you all next year!

31 October 2010

Lucas Nussbaum: Mini DebConf Paris 2010

So, I m (almost) back from the first edition of MiniDebConf Paris, which was a success thanks to the great organizing skills in panic mode of Carl Chenet and Mehdi Dogguy. I think that everybody is already looking forward to the next edition. I met lots of people, and it was great to be able to finally put faces on names from the french free software community. I gave two talk there, the first one on Debian and Ubuntu (slides), the second one on Debian Quality Assurance (slides), which both went well (I think). I also hacked a bit: If you want to help me, here are some easy tasks that need takers (it s unlikely that I ll be able to tackle them anytime soon): One of the other things I d like to do is discuss ways to improve the synchronization of UDD (currently, it s synchronized at arbitrary times, while we could sync Sources+Packages just after dinstall, for example).

30 October 2010

Lucas Nussbaum: Debian becoming upstream for ubuntu-dev-tools?

Launchpad bug #643691: Please sync ubuntu-dev-tools 0.103 (universe) from Debian experimental (main) Did development of ubuntu-dev-tools move from Ubuntu to Debian? :-)

Next.

Previous.