The Google books ngram datasets are a nice resource for playing with text generation. I've created a program that will, given a starting phrase, blather on by connecting phrases from the datasets. The results are much more varied, and seem to more often make sense than most Markov text generators I've seen before, such as dadadodo. It had better be -- it's working with data derived from 4% of all books ever written! Some examples. Do they tell a story? (The starting phrase I entered is in italics.)

• He is assailed they merge
• To be or not to be so deceived ! She denies abdominal scanning
• Beauty with enormous strides . The hospital was under the control of the communal organization of solitary mammals . In other words the mere failure of an individual .
• Time passes . It is the child's first day in town and at his own level ?
• Time above his left shirt pocket for safe keeping . At first I accepted the old man's work . He settled in that part of Paris .
• Until then no ill Accident would generate hydrogen sulfide
• Random nonsense put together huge boxes with ammunition against it itself stops . Working things out afterwards
• A change in voice and piano . He sat down at last at No . 5 in connection with another or with others ; and that by some strange disaster .
• O God ! What is the condition of morality is that they all hold hands in a chain of propositions . These propositions are the primary bearers of meaning making and the rearing of children

I have cheated slightly by picking where to stop quoting the blather each time. It tends to go on for quite a while otherwise. Although it often does manage some kind of sense at length too:

• Worst Case Scenarios in Search of Reality : The ERA and emendation . I can never recover . She became thoughtful . He looked at me gloomily above the tree tops . The women were asked how they used to behave more competitively . Forecasts should be made

---

This was accomplished using only 99 lines of code. I imported the datasets into Xapian databases using this program. And this program is the text generator. I threw out the metadata, to keep the Xapian databases of reasonable size. By which I mean, only a dozen gigabytes. It might be interesting to include the publication year data in Xapian, so that it could prefer ngrams that were published around the same time as the input text. I'm still importing data. I may put up a web interface later if there's interest. PS:

• What about Spam ? Who caused it ? You have said it so loud that the little lady in a black storm .
• What about Spam ? I think it cannot be any . The Church had the right men . They are our friends and relations who could be depended upon with regard to the UN . Some people paint the portraits of the writers .

PPS:

• Joey Hess was not the real author of these lines .

<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="http://rcm.amazon.com/e/cm?lt1=_blank&amp;bc1=000000&amp;IS2=1&amp;bg1=FFFFFF&amp;fc1=000000&amp;lc1=0000FF&amp;t=dedasys-20&amp;o=1&amp;p=8&amp;l=as1&amp;m=amazon&amp;f=ifr&amp;md=10FE9736YVPPT7A0FBG2&amp;asins=0470929839" style="width: 120px; height: 240px;"></iframe> I got this to read on my trip to Oregon. Initially I was a bit dubious, after reading the chapter headings on Amazon. Indeed, if you read a lot of articles about tech startups, a lot of the material is not particularly novel, and are things that you've already read elsewhere. If, on the other hand, you're looking for a quick introduction to some of the common ideas behind web/tech startups, it's a pretty good list, written by a variety of people, with a lot of practical advice. The list of concepts includes "execution, not ideas", "fail fast", get feedback early", "get a co-founder", and a lot of similar advice that permeates discussions of early stage internet companies. Other people have different tastes, but for me, this passage was a strikingly effective advertisement for living in Boulder, Colorado:

With this as the backdrop, it was natural to make cycling a central part of my work with Everlater's founders, Nate Abbott and Natty Zola. Nate, Natty, and I scheduled regular weekly rides (sometimes several times a week) and while we didn't keep a formal agenda, each ride had a number of topics planned out for us to cover. In this way, we worked through many of the early challenges the business faced as it moved from idea to reality in the course of the summer of 2009. Our typical rides lasted from 60 to 90 minutes during which time we were able to focus on the business without interruption. And while getting some fresh air helps bring perspective, it's hard to avoid thinking about problems in a new way when you're out of the office, surrounded by the beauty of the Boulder foothills while your mind is completely clear. Startups, cycling, mountains... sign me up! Another tidbit I found interesting:

We often talk about the Number One startup killer at TechStars making a product for which there is no interesting market. TechStars accepts just 10 of more than 600 startups that apply and presumably they are among the best. But still, we find that at least one-third of those startups are attempting to build a product that they want, or that no one wants, instead of what the market wants But wait, I thought "execution is everything", and "ideas are worthless". It seems like avoiding bad ideas is potentially worth a whole lot of money in wasted time and energy though, no? The bit about ideas being worthless is a load of crap. The reason there is no market for ideas is that they are not "excludable". There was never a market for lighthouses either, the classic example in economics of a public good. So the government built them, and they were valuable to people operating ships. Ideas for startups may not have a lot of value compared to people executing them well, but to say they are worthless is hyperbole. In any case, though, if you're interested in startups, or especially if you happen to be interested in the TechStars program, it's book full of good advice, even if none of it will come as a revalation to anyone who follows that sort of thing regularly.

I ve used Delicious (del.icio.us) for some time now for managing bookmarks. I have never really used its social features, just its management features. With word that it is closing (sigh, just after I had to leave Bloglines, too) I m looking for something else. There are a somewhat bewildering array of new options available, and I m wondering if people have had experience with them. My requirements are:

• Completely reliable syncing between multiple devices, even if each one is being used and actively bookmarking things simultaneously. My browser is Firefox. Back when I chose Delicious, nothing else accomplished this.
• The ability to present bookmarks as a toolbar in Firefox, either using its own system or a different one.
• I prefer tagging to hierarchical organization.
• I can run a service on my own server, IF it is Free Software, trivial to set up, and needs little care and feeding. If it s not part of Debian, that s a strong negative here.
• If it s hosted in the cloud, I am concerned about privacy, security, and long-term stability. I must have a way to export my data, preferably automatically. I will be reading ToS and privacy policies carefully.
• A system that can be trusted to encrypt my bookmarks on the server side is another nice to have.
• Being able to create bookmarks directly from my Android phone is nice but not required.
• Being able to access bookmarks from a regular web browser is a feature I use a few times a year; again, nice but not required.
What should I look into? Incidentally, if you need to export your Delicious data, go here.

Debian 6.0 Squeeze to be released with completely free Kernel(s) Today we have announced that, starting with the upcoming release of Squeeze, Debian will be even Free-er. Exceptions to the DFSG for non-free Linux firmware blobs, which have been granted in the past, will no longer be granted. Starting from Squeeze, Debian will be Free the bottom up, no matter where your own definition of software ends. The news has been known for a while within the Debian development community, but we had the pleasure of sharing that with our users only today. According to the reaction on identi.ca, where the news quickly made into the most popular notices, people are enthusiastic about the news and that in turn is very rewarding. If you are as happy and proud about this achievement as I am, please direct your kudos to all the teams who made this possible:

• the Debian kernel team who worked for the past 2 release cycles, together with Linux upstream, to split free-vs-non-free firmware bits and monitor new releases to avoid freedom regressions;
• the Debian CD team who basically "forked" different media production lines, for different user needs wrt firmware;
• the d-i team who added, long ago, support for loading user-provided firmware at installation time;
• whoever else I might have forgotten who contributed to this result (feel free to leave a comment to refresh my memory!).

Having been trol^W asked several times in the past about the "real freeness" of Debian at events, talks, and the like, I consider this episode to be a very nice story to tell. While we have granted exceptions for Linux firmware at the time of Etch and Lenny releases, during this time Debian people have worked, together with upstream, to make this final result possible. All this work has improved the situation not only for Debian users, but also for users of those derivatives which rely upon Debian kernels. Nonetheless, don't expect us to live under a rock. We know that there are users out there who need nasty non-free firmware bits to boot their machines or to access the Internet. That is why separate images with firmware included are being provided, although they are not part of Debian and are properly "discriminated": they should be looked for explicitly by users and can't be supported to the same extent of Free firmware, simply because we (as anybody else other than the hardware manufacturer) do not have access to the corresponding source code. This separation is a perfect example of the pragmatism embodied by the Debian Social contract: Debian consists of 100% Free software; at the same time, Debian enables its users to make their own choices in terms of Freedom, being explicit about what users risk if they cross the boundaries of DFSG. Awareness is the key here and I find it to be a very honest yet effective way of explaining to the world why Free Software is better. This is not the end of the story though, as we need to document this change properly to both increase awareness and avoid leaving users in the dark. A couple of bugs filed today might benefit from the help of some kind doc-writers: Debian bug #607191 (documenting the firmware change in the Squeeze release notes) and Debian bug #607193 (documenting the firmware change on the website).

As I'm not currently working on any suitable paper, I'll just post this to my blog so it does not completely slip off my radar ;-) Also, it might be interesting to my reader. Readers? Oh, there are two of you now? Good! Yesterday, I learnt thanks to Beatriz Busaniche that a group of South American Free Culture activists launched number zero of a magazine that promises to be very interesting: Cultura RWX, cultura en modo lectura, escritura y acci n (culture in reading, writing and action mode). Guys, best luck with this new project! Anyway, reading it, I found this asseveration I want to keep at hand:

( )cuando surge la industria musical aparecen los derechos de autor como forma de defensa de los productores musicales, espec ficamente los m sicos. No tanto frente a los usuarios, porque hasta el cassette no existi posibilidad de copiar una obra musical. Era una defensa frente a las discogr ficas, que buscaban cerrar contratos muchas veces abusivos con los artistas. M sica en Libertad: La industria musical frente al cambio de paradigma; Mat as Lennie, adaptaci n: Sebasti n Vazquez

Yes, yes, translating to English:

( )when the musical industry was born, copyright appeared as a means of defense of the musical producers, specifically of the musicians. Not so much against the users, because up until the invention of the cassette there was no possibility to copy a musical creation. It was a defense against the discographic companies, which tried to close often abusive contracts with the artists. Music in Freedom: Musical industry and the paradigm shift; Mat as Lennie, adaptation: Sebasti n V zquez

I have argued (i.e. in here) in this same line regarding the birth of copyright itself It was an arrangement that had to be made between writers and printers, back in the XVI/XVII centuries. Simple individuals were just unable to get anything of value out of the copying technology they had at hand. Copyright was born in a time where reproduction required specialized equipment. Today, massive reproduction technology is a given for a good portion of the planet's population. Copyright now only defends big corporations And will inevitably fade away as anachronic. Of course, it refuses to go without a fight... But it cannot win long-term. We cannot afford to allow it!

There s been a lot of discussion regarding whether or not Nokia is Doomed or not. The people who say Nokia are doomed basically point out that Nokia doesn t have any attractive products at the high end, and at the low end the margins are extremely thin. The high end products suffer from the Symbian being essentially dead (even Nokia is recommending that developers not develop native applications for Symbian, but to use Qt instead), and Nokia doesn t have much of a development community following it, and it certainly does have much in the way of 3rd party applications, either targetting Symbian or Qt at the moment. So what do I think of the whole debate between Tomi and Scoble? First of all, I think there is a huge difference in American and European assumptions and perspectives, and a big question is whether the rest of the world will end up looking more like Europe or America vis-a-vis two key areas: cost of data plans, and whether phones become much more application centric. Tomi took Apple to task in the comments section of his 2nd article for not having an SD card slot (how else would people share photos with their friends?) and for not supporting MMS in its earlier phones. My first reaction to that was: Um, isn t that what photo-sharing sites are for? Is it really that hard to attach a photo to an e-mail? And then it hit me. In Europe, data is still like MMS a few years ago a place for rapacious carriers to make way too much money. Many European telco s don t have unlimited data plans, and charge by the megabyte and even if you re lucky enough to live in a country which does have an American-like data plan, the cost of data roaming is still incredibly expensive. In contrast, in the US, I can pay $30/month for an unlimited data plan, and I can travel 2000 miles south or west and it will still be valid. Try doing that in Europe! The US had consumer-friendly data plans much earlier than Europe did, and so perhaps it s not surprising that Nokia has engineered phones that were far more optimized for the limitations caused by the Europe s Wireless carriers. The second area of debate where I think Scoble and Tomi are far apart is whether phones of the future are fundamentally about applications or well, making phone calls. Here I don t have proof that this is a fundamentally European vs. US difference, but I have my suspicions that it might be. Tomi spent a lot of time dwelling on how Nokia was much better at making phone calls (i.e., better microphones, better radios, etc). And my reaction to that was, Who cares? I rarely use my phone for making phone calls these days! And that was certainly one of the reasons why I gave up on Nokia after the E70 its contacts database was garbage! It was OK as a phone directory, but as a place for storing multiple addresses and e-mail addresses, it didn t hold a candle to the Palm PDA. And that s perhaps the key question how much is a smart phone and about being a phone , versus being a PDA (and these days I want a cloud-synchronized PDA, for my calendar, contacts, and todo lists), and how much is it about applications? This is getting long, so I think I ll save my comments about whether I think Meego will be an adequate savior for Nokia for another post. But it s worthwhile to talk here about Tomi s comments about most smartphones being much cheaper than the luxury iPhone, and so it doesn t matter that Nokia s attempt in the higher end smart phones has been a continuous history of fail. First of all, it s worth noting that there are much cheaper Android phones available on the market today, which are price-competitive with Nokia s low-end smartphones (i.e., available for free from T-Mobile in the States with a two year commitment). Secondly, the history in the computer market over the last twenty years is that features inevitably waterfall into the cheaper models, and prices will tend to drop over time as well. Apple started only with the iPod, but over time they added the iPod Nano and the iPod Shuffle. And it would not surprise me if they introduce a lower-end iPhone as well in time as well. It would shock me if they aren t experimenting with such models even as we speak, and have simply chosen not to push one out to the market yet. So even if you buy Tomi s argument that the high-end smartphones don t matter, and you only care about volume, and not about profit margins (talk to the people at Nokia that will need to be laid off to make their expenses match with their lowered revenue run rates; I bet they will care), the question is really about whether Nokia has time to execute on the Meego vision before it s too late and the current application-centric smartphone ecosystems (Android and iPhone) start eating into the lower-end smartphone segment. More on that in my next post. No related posts.

There s been a lot of discussion regarding whether or not Nokia is Doomed or not. The people who say Nokia are doomed basically point out that Nokia doesn t have any attractive products at the high end, and at the low end the margins are extremely thin. The high end products suffer from the Symbian being essentially dead (even Nokia is recommending that developers not develop native applications for Symbian, but to use Qt instead), and Nokia doesn t have much of a development community following it, and it certainly does have much in the way of 3rd party applications, either targetting Symbian or Qt at the moment. So what do I think of the whole debate between Tomi and Scoble? First of all, I think there is a huge difference in American and European assumptions and perspectives, and a big question is whether the rest of the world will end up looking more like Europe or America vis-a-vis two key areas: cost of data plans, and whether phones become much more application centric. Tomi took Apple to task in the comments section of his 2nd article for not having an SD card slot (how else would people share photos with their friends?) and for not supporting MMS in its earlier phones. My first reaction to that was: Um, isn t that what photo-sharing sites are for? Is it really that hard to attach a photo to an e-mail? And then it hit me. In Europe, data is still like MMS a few years ago a place for rapacious carriers to make way too much money. Many European telco s don t have unlimited data plans, and charge by the megabyte and even if you re lucky enough to live in a country which does have an American-like data plan, the cost of data roaming is still incredibly expensive. In contrast, in the US, I can pay $30/month for an unlimited data plan, and I can travel 2000 miles south or west and it will still be valid. Try doing that in Europe! The US had consumer-friendly data plans much earlier than Europe did, and so perhaps it s not surprising that Nokia has engineered phones that were far more optimized for the limitations caused by the Europe s Wireless carriers. The second area of debate where I think Scoble and Tomi are far apart is whether phones of the future are fundamentally about applications or well, making phone calls. Here I don t have proof that this is a fundamentally European vs. US difference, but I have my suspicions that it might be. Tomi spent a lot of time dwelling on how Nokia was much better at making phone calls (i.e., better microphones, better radios, etc). And my reaction to that was, Who cares? I rarely use my phone for making phone calls these days! And that was certainly one of the reasons why I gave up on Nokia after the E70 its contacts database was garbage! It was OK as a phone directory, but as a place for storing multiple addresses and e-mail addresses, it didn t hold a candle to the Palm PDA. And that s perhaps the key question how much is a smart phone and about being a phone , versus being a PDA (and these days I want a cloud-synchronized PDA, for my calendar, contacts, and todo lists), and how much is it about applications? This is getting long, so I think I ll save my comments about whether I think Meego will be an adequate savior for Nokia for another post. But it s worthwhile to talk here about Tomi s comments about most smartphones being much cheaper than the luxury iPhone, and so it doesn t matter that Nokia s attempt in the higher end smart phones has been a continuous history of fail. First of all, it s worth noting that there are much cheaper Android phones available on the market today, which are price-competitive with Nokia s low-end smartphones (i.e., available for free from T-Mobile in the States with a two year commitment). Secondly, the history in the computer market over the last twenty years is that features inevitably waterfall into the cheaper models, and prices will tend to drop over time as well. Apple started only with the iPod, but over time they added the iPod Nano and the iPod Shuffle. And it would not surprise me if they introduce a lower-end iPhone as well in time as well. It would shock me if they aren t experimenting with such models even as we speak, and have simply chosen not to push one out to the market yet. So even if you buy Tomi s argument that the high-end smartphones don t matter, and you only care about volume, and not about profit margins (talk to the people at Nokia that will need to be laid off to make their expenses match with their lowered revenue run rates; I bet they will care), the question is really about whether Nokia has time to execute on the Meego vision before it s too late and the current application-centric smartphone ecosystems (Android and iPhone) start eating into the lower-end smartphone segment. More on that in my next post. No related posts.

Someone recently pointed me to some economic/statistical models that were designed for eviews (a proprietary tool). They were looking for help converting the model into a format that could work with gretl (the Gnu Regression, Econometrics and Time-series Library). I'm afraid i'm an economics dunce, and i have no experience with this stuff. Any pointers?Tags: conversion, economics, gretl

as pointed out by Tolimar the number of RC bugs is decreasing. here's the list of my small contributions (which includes also one bug I've caused *cough*).

• #603036 lvm2: "lvm2: fails to install due to incorrect dependencies in init.d LSB header"
  update bug information, test solutions
• #603407 gearman-server: "gearman-server: Server hangs when job is submitted"
  investigate, tag unreproducible, lower severity (pkg-perl)
• #603429 hylafax-server: "hylafax-server: prompting due to modified conffiles which where not modified by the user"
  send new patches, later upload to DELAYED/2
• #604221 hylafax-server: "hylafax-server: installation fails"
  send another patch, later upload to DELAYED/2
• #604225 mailscanner: "mailscanner: installation fails"
  test for existence of /dev/tty in maintainer scripts, upload to DELAYED/2
• #604975 fractxtra,xfractint: "xfractint and fractxtra: error when trying to install together"
  update bug information, later fixed by maintainer
• #605136 python-apt-common: "python-apt-common: fails to install, trying to overwrite other packages files"
  add Replaces in debian/control. upload to DELAYED/2. later overridden by maintainer upload
• #605565 gnustep-base-runtime: "gnustep-base-runtime: unremovable if the daemon is not running"
  sponsor maintainer upload
• #606000 libmail-spf-query-perl: "libmail-spf-query-perl: Incorrect query results with IPv6 addresses; should warn about missing IPv6 support and/or fail graciously"
  investigate (pkg-perl)
• #606802 bacula-director-pgsql: "bacula-director-pgsql: make_catalog_backup is broken"
  investigate and make a proposal

squeeze is getting closer :)

Went to the Chicago Symphony yesterday to see Regina Carter as well as Jazz wunderkind Esperanza Spalding perform one seat each with their respective bands. Regina Carter was presenting material from her current record 'Reverse Thread'. This was a real nice set of African-themed world music featuring Carter herself on violin, Yacouba Sissoko on kora, Will Holshouser on accordion, Chris Lightcap on bass and Alvester Garnett on drums. Some of pieces were really, really nicely done and I particularly enjoyed Holshouser on the accordion. After the break, Esperanza Spalding come on for her Chamber Music Society'. Lovely setup with Spalding on acoustic bass and vocals, Leo Genovese on piano/keyboards, Sara Caswell on violin, Lois Martin on viola, Jody Redhage on cello, the always impressice Terry Lyne Carrington on drums and Leala Cyr on backing vocals (and one co-lead in a really nice duet with Spalding). This was clearly more experimental and a chunk of the audience left during the act. But there is room for improvided chamber music, and it was a good modern music act. And Spalding is really quite impressive and I will gladly go and see her again.

As I continue to explore BitCoin, I've starting to wonder what properties the system have, and how it will be affected by laws and regulations here in Norway. Here are some random notes. One interesting thing to note is that since the transactions are verified using a peer to peer network, all details about a transaction is known to everyone. This means that if a BitCoin address has been published like I did with mine in my initial post about BitCoin, it is possible for everyone to see how many BitCoins have been transfered to that address. There is even a web service to look at the details for all transactions. There I can see that my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b have received 16.06 Bitcoin, the 1LfdGnGuWkpSJgbQySxxCWhv8MHqvwst3 address of Simon Phipps have received 181.97 BitCoin and the address 1MCwBbhNGp5hRm5rC1Aims2YFRe2SXPYKt of EFF have received 2447.38 BitCoins so far. Thank you to each and every one of you that donated bitcoins to support my activity. The fact that anyone can see how much money was transfered to a given address make it more obvious why the BitCoin community recommend to generate and hand out a new address for each transaction. I'm told there is no way to track which addresses belong to a given person or organisation without the person or organisation revealing it themselves, as Simon, EFF and I have done. In Norway, and in most other countries, there are laws and regulations limiting how much money one can transfer across the border without declaring it. There are money laundering, tax and accounting laws and regulations I would expect to apply to the use of BitCoin. If the Skolelinux foundation (SLX Debian Labs) were to accept donations in BitCoin in addition to normal bank transfers like EFF is doing, how should this be accounted? Given that it is impossible to know if money can across the border or not, should everything or nothing be declared? What exchange rate should be used when calculating taxes? Would receivers have to pay income tax if the foundation were to pay Skolelinux contributors in BitCoin? I have no idea, but it would be interesting to know. For a currency to be useful and successful, it must be trusted and accepted by a lot of users. It must be possible to get easy access to the currency (as a wage or using currency exchanges), and it must be easy to spend it. At the moment BitCoin seem fairly easy to get access to, but there are very few places to spend it. I am not really a regular user of any of the vendor types currently accepting BitCoin, so I wonder when my kind of shop would start accepting BitCoins. I would like to buy electronics, travels and subway tickets, not herbs and books. :) The currency is young, and this will improve over time if it become popular, but I suspect regular banks will start to lobby to get BitCoin declared illegal if it become popular. I'm sure they will claim it is helping fund terrorism and money laundering (which probably would be true, as is any currency in existence), but I believe the problems should be solved elsewhere and not by blaming currencies. The process of creating new BitCoins is called mining, and it is CPU intensive process that depend on a bit of luck as well (as one is competing against all the other miners currently spending CPU cycles to see which one get the next lump of cash). The "winner" get 50 BitCoin when this happen. Yesterday I came across the obvious way to join forces to increase ones changes of getting at least some coins, by coordinating the work on mining BitCoins across several machines and people, and sharing the result if one is lucky and get the 50 BitCoins. Check out BitCoin Pool if this sounds interesting. I have not had time to try to set up a machine to participate there yet, but have seen that running on ones own for a few days have not yield any BitCoins througth mining yet. Update 2010-12-15: Found an interesting criticism of bitcoin. Not quite sure how valid it is, but thought it was interesting to read. The arguments presented seem to be equally valid for gold, which was used as a currency for many years.

The two first interviews were dedicated to long-time Debian developers. This time I took the opposite approach, I interviewed David Kalnischkies who is not (yet) a Debian developer. But he s contributing to one of the most important software within Debian the APT package manager since 2009. You can already see him in many places in Debian sharing his APT knowledge when needed. English is not his native language and he s a bit shy, but he accepted the interview nevertheless. I would like to thank him for the efforts involved and I hope his story can inspire some people to take the leap and just start helping My questions are in bold, the rest is by David. Who are you? I am David Kalnischkies, 22 years old, living in the small town Erbach near Wiesbaden in Germany and I m studying computer science at the TU Darmstadt. Furthermore I am for more than half a decade now young group leader of my hometown. I never intended to get into this position, but it has similarities with my career in this internet-thingy here. I don t remember why, but in April 2009 I was at a stage that some simple bugs in APT annoyed me so much that I grabbed the source, and most importantly I don t know why I did it but I published my changes in Mai with #433007, a few more bugs and even a branch on launchpad. And this public branch got me into all this trouble in June: I got a mail from Mr. package managment Michael Vogt regarding this branch A few days later I joined an IRC session with him and closely after that my name appeared for the first time in a changelog entry. It s a strange but also addicting feeling to read your own name in an unfamiliar place. And even now after many IRC discussions, bugfixes and features, three Ubuntu Developer Summits and a Google Summer of Code in Debian, my name still appear in places I have never even thought about e.g. in an interview. What s your biggest achievement within Debian? I would like to answer MultiArch in APT as it was my Google Summer of Code project, but as it has (not much) use for the normal user at this point will hopefully change for wheezy I chose three smaller things in squeeze s APT that many people don t even know yet:

• MMap run out of room , the most famous error related to the APT::Cache-Limit option is gone. The DynamicMMap in APT now deserves its name
• Download and usage of multiple Translations of Descriptions
• crazy stuff like apt-cache show ^apt-.*$/experimental works

If your impression is now that I only do APT stuff: that s completely right, but that s already more than enough for me for now as the toolchain behind the short name APT contains so many tools and use cases that you always have something different. You re an active member of the APT development team. Are there plans for APT in Debian Wheezy? What features can we expect? That s very hard to answer, as the team is too small to be able to really plan something. I mean, you can have fancy plans and everything and half a second later someone arrives on the mailing list with a small question which eats days of development time just for debugging But right now the TODO list contains (in no particular order):

• MultiArch managing packages built for different architectures like i386 on amd64 to ease the usage of packages which only work on specific architectures or cross-compiling.
• rewriting the subsystem responsible for downloading files to e.g. enable tools like apt-file or debtags to reuse the infrastructure by plugging into it.
• splitting long descriptions out of the Packages file to possibly decrease the size to download on an apt-get update . Includes mostly pulling the button only to enable it by the archive masters
• backporting popular features of some frontends to enable all frontends to use them, e.g. apt-get changelog or apt-get download
• support research on resolver improvements by implementing support for a distribution independent upgrade problem format (CUDF).
• an additional single binary for users wrapping apt-get, apt-cache and co.
• getting in contact (again) with various forks like apt-rpm and derivatives like maemo or telesphoreo to join forces instead of splitting them by working on different (and possibily very old) heads of the dragon

We will see what will get real for wheezy and what is postponed, but one thing is sure: more will be done for wheezy if you help! If you could spend all your time on Debian, what would you work on? I would spend it on APT s debbugs count zero would be cool to look at! We make progress in this regard, but with the current velocity we will reach it in ten years or so. Reading more mailing lists would be interesting, as I am kind of an information junky. Maintaining a package could be interesting to share the annoyance of a maintainer with handcrafted dependencies just to notice that APT doesn t get it in the way I intended it to be. Through, to make it feel real I need to train a few new APT contributors before so they can point my mistake out, but this unfortunately doesn t depend so much on time but on victims Maybe I could even be working on getting an official status. Beside that, I would love to be able to apt-get dist-upgrade the increasing mass of systems I and many others carry around in their pockets. In regards to my phone, this is already fixed, but there is much room for improvements. What s the biggest problem of Debian? You need to be lucky. You need to talk at the right time to the right person. That s not really a debian-only problem as such, but in a global project full of volunteers you can see it clearly as there are plenty of opportunities to be unlucky. For example, it s unlikely that an interview would be made with me now if Michael had not contacted me in June 2009. In a big project like Debian, you are basically completely lost without a mentor guiding you, so things like the debian-mentors list are good projects, but I am pretty certain they could benefit from some more helping hands. The other thing which I consider a problem is that and I read from time to time some people don t care for translations. That s bad. Yes, a developer is able to read English, otherwise s/he couldn t write code or participate on the mailinglists. Still, I personally prefer to use a translated application if I have the chance as it s simply easier for me to read in my mother tongue, not only because I am dyslexic, but because my mind still thinks in German and not in English. Yes, I could personally fix that by thinking in English only from now on, but its a quite big problem to convince my family which is not really familiar with tech-stuff to use something if they can t understand what is written on screen. It was hard enough to tell my mother how to write an SMS in a German interface. My phone with English words all over the place would be completely unusable for her despite the fact that my phone is powered by Debian and better for the task from a technical point of view. You are not yet an official Debian developer/maintainer, but you re already perceived in the community as one the most knowledgeable person about APT. It s a great start! What s your advice to other people who want to start contributing to Debian in general, and to APT in particular? It was never a goal in my life to start contributing . My goal was and still is to make my life easier by letting the computer work for me. At some point APT hindered the success of this goal, so it needed to be fixed. I didn t expect to open pandora s box. So, my advice is simple: Just start. Ignore the warning signs telling you that this is not easy. They are just telling you that you do something useful. Only artificial problems are easy. Further more, contribution to APT, dpkg or any other existing package is in no way harder than opening an ITP and working on your own, and it s cooler as you have a similar minded team around you to talk to. APT didn t accept release codenames as target release was one of the first things I fixed. If I had asked someone if that would be a good starting point the answer would have been a clear no , but I didn t search for a good starting point As a kid I can start playing football by just walking on the field and play or I can sit near the field, watching the others play, while analyzing which position would be the best for me to start ruling out one by one as the technical requirements seem too high Oh bicycle kick that sounds complicated I can t do that Julian Andreas Klode is working on a APT replacement, there s also Cupt by Eugene V. Lyubimkin. Both projects started because their authors are not satisfied with APT, they find APT s code difficult to hack partly due to the usage of C++. Do you share their concerns and what s your opinion on those projects? I don t think C++ is a concern in this regard, after all cupt is currently rewritten to C++0x and APT2 started in vala and is now C + glib last time I checked at least. I personally think that something is wrong if we need to advertise an application by saying in which language it is written The major problem for APT is probably that the code is old : APT does its job for more than 12 years now, under different maintainers with an always changing environment around it: so there are lines in APT which date from a time when nobody knew what a Breaks dependency is, that packages can have long descriptions which can be translated or even that package archives can be signed with a gpg key! And yet we take all those for granted today. APT has proven to adapt to these changes in the environment and became in this process very popular. So I don t think the point is near (if it will come at all) that APT can go into retirement as it is completely replaced by something else. The competitors one the other hand have their first 12 years still to go. And it will be interesting to see how they will evolve and what will be the state of the art in 2022 But you asked what I think about the competitors: I prefer the revolution from inside simply because I can see effects faster as more users will profit from it now. Cupt and co. obviously prefer the normal revolution. The goal is the same, creating the best package manager tool, but the chosen way to the goal is different. aptitude and cupt have an interactive resolver for example: that s something I dislike personally, for others that is the ultimate killer feature. cupt reading the same preference file as APT will have a different pinning result, which we should consider each time someone mentions the word drop-in replacement . APT2 isn t much more than the name which I completely dislike currently from a user point of view, so I can t really comment on that. All of them make me sad as each line invested in boilerplate code like configuration file parsing would be in my eyes better be spent in a bugfix or new feature instead, but I am not here to tell anyone what they should do in their free time But frankly, I don t see them really as competitors: I use the tools I use, if other do that too that s good, if not that s their problem. The thing that annoys me really are claims like plan is to remove APT by 2014 as this generates a vi vs. emacs like atmosphere we don t need. If some people really think emacs is a good editor who cares? I really hope we all can drink a beer in 2022 in Milliways, the restaurant at the end of the package universe, remembering the good old 2010 Is there someone in Debian that you admire for his contributions? No, not one, many! Michael Vogt who has nearly the monopole of package manager maintainer by being upstream of APT, synaptics and software center to name only the biggest and still has the time to answer even the dumbest of my questions. Jason Gunthorpe for being one of the initial developers behind deity who I will probably never meet in person beside in old comments and commit logs. Christian Perrier for caring so much about translations. Obey Arthur Liu as a great admin for Debian s participation in Google s Summer of Code. Paul Wise for doing countless reviews on debian-mentors which are a good source of information not only for the maintainer of the package under review. I guess I need to stop here because you asked for just one. So let s end with some big words instead: I am just a little cog in the big debian wheel

---

Thank you to David Kalnischkies for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

3 comments Liked this article? Click here. My blog is Flattr-enabled.

I attended a workshop sponsored by the IAB, W3C, ISOC and MIT on Internet Privacy. The workshop had much more of a web focus than it should have: the web is quite important should certainly cover a majority of the time, but backend issues, network issues, and mobile applications are certainly important too. For me this workshop was an excellent place to think about linkability and correlation of information. When people describe attacks such as using the ordered list of fonts installed in a web browser to distinguish one person from another, it s all too easy to dismiss people who want to solve that attack as the privacy fringe. Who cares if someone knows my IP address or what fonts I use? The problem is that computers are very good at putting data together. If you log into a web site once, and then later come back to that same website, it s relatively easy to fingerprint your browser and determine that it is the same computer. There s enough information that even if you use private browsing mode, clear your cookies and move IP addresses, it s relatively easy to perform this sort of linking. It s important to realize that partially fixing this sort of issue will make it take longer to link two things with certainty, but tends not to actually help in the long-run. Consider the font issue. If your browser returns the set of fonts it has in the order they are installed, then that provides a lot of information. Your fingerprint will look the same as people who took the same OS updates, browser updates and installed the same additional fonts in exactly the same order as you. Let s say that the probability that someone has the same font fingerprint as you is one in a million. For a lot of websites that s enough that you could very quickly be linked. Sorting the list of fonts reduces the information; in that case, let s say your probability of having the same font set as someone else is one in a hundred. The website gets much less information from the fonts. However it can combine that information with timing information etc. It can immediately rule out all the people who have a different font profile. However as all the other people who have the same font fingerprint access the website over time, differences between them and you will continue to rule them out until eventually you are left. Obviously this is at a high level. One important high-level note is that you can t fix these sorts of fingerprinting issues on your own; trying makes things far worse. If you re the only one whose browser doesn t give out a font list at all, then it s really easy to identify you. The big question in my mind now is how much do we care about this linking. Governments have the technology to do a lot with linking. We don t have anything we technical we can do to stop them, so we ll need to handle that with laws. Large companies like Google, Facebook and our ISPs are also in a good position to take significant advantage of linking. Again, though, these companies can be regulated; technology will play a part, especially in telling them what we re comfortable with and what we re not, but most users will not need to physically prevent Google and Facebook from linking their data. However smaller websites are under a lot less supervision than the large companies. Unless you take significant steps, such a website can link all your activities on that website. Also, if any group of websites in that space want to share information, they can link across the websites. I d like to run thought experiments to understand how bad this is. I d like to come up with examples of things that people share with small websites but don t want linked together or alternatively don t want linked back to their identity. Then look at how this information could be linked. However, I m having trouble with these thought experiments because I m just not very privacy minded. I can t think of something that I share on the web that I wouldn t link directly to my primary identity. I certainly can t find anything concrete enough to be able to evaluate how clearly I care to protect it. Helping me out here would be appreciated; if you can think of fairly specific examples. There s lots of important I prefer to keep private like credit card numbers, but there, it s not about linking at all. I can reasonably assume that the person I m giving my credit card number to has a desire to respect my privacy.a

As I said more than a week ago I was about to undergo an operation.

Now, 10 days after the operation I must say I am still not 100% recovered and I experience a weird sensation as if the nose is way too empty, especially the left nostril. During this period since the operation I have been experiencing head aches, and an almost continuous sensation of an empty left maxilar sinus, the side on which the cyst was before. This sensation is amplified after a nose blow or after the regular nose post-op cleanings.

I hope this is not Empty Nose Syndrome.

This all started this summer. My parents have some dimmer switches in their house, which Jacob loves. He also enjoyed turning their lights on an off rapidly to make them flash. Since we have CFLs almost exclusively, we have no dimmer switches, and we don t permit making the lights flash. (This hurts the lifespan of CFLs much more than traditional bulbs.) (I ll just put a few pictures from tonight within the story so you get an idea of what it turned out to be while you read.) So I had an idea: why not build him a box that has some switches that he can play with? And why not make it a project we can do together? So back in, I think, July, we went to the hardware store. We bought some AC power cord, some electrical boxes, and assorted supplies. I had some other supplies on hand already, so we didn t need to spend much. I asked Jacob today if he remembered that day, and he said, Oh yes! And I got popcorn to eat there! Which was, indeed, quite true. I had some surplus plywood, so I cut out two squares, one for a front and one for a back. Then, in 10-minute increments every so often over the last few months, Jacob would say, Dad, shall we go work on my switchbox? And by that, he meant that I should work on the switchbox while he pretends that my tape measures are train engines or raids the corner where disused toys are stored. After 10 minutes, he d be done. Which meant that after I gathered up my tools, remembered where I was, and got to work, I made only a few minutes progress each week. But that was fine. This week, I got it all wired up. It doesn t have its sides and back yet, but it was enough for Jacob to try it out. To add to the excitement just that much more, I spent a few dollars at the hardware store today and bought him four light bulbs: 25W red, green, and blue bulbs, and a 7.5W miniature white one. I let him choose two of them to use in the switchbox. So he started playing with it, and after just a few seconds, said, Dad, I am very very very very excited about my switchbox! And a little while later, he burst into applause, and announced, Dad, my switchbox is so so so so so fun! It is also so so so so so silly the lightbulb is green! I wired up a regular light switch that is a master on/off switch. Then the dimmer switch controls the light bulb socket but not the outlet. The dimmer switch also has a pushbutton on/off circuit, so there are plenty of opportunities for discovery. This is a play with it only with dad s supervision item for now, but after supper, he insisted in bringing Terah downstairs to show her how to use it too. He happily showed off the red and green lights, how to turn them on, and how to make the red light dimmer. I suggested he also show her what happens when he pushes on the dimmer switch, which he happily did, and also explained to her what it does. He was just amazingly excited. Every so often he d tell me how excited or happy he was, and some fun fact about it all. And, really, that s what I hoped would happen. I didn t accompany it with these words, but I hope that projects like this can serve to remind everyone that toys and gifts don t have to cost hundreds of dollars; they can be made with a few dollars supplies and some surplus materials laying around the house. But not only that, but the best parts of this project were the ones Jacob and I spent together from the time when just the two of us went shopping at the hardware store, through all those 10-minute times where he pretended my tape measures were locomotives and the tape was their track, right to today when he got to try it out. It took a long time, but when I asked him if it was worth the wait, he said Yes! and clapped while jumping up and down. And, I have to admit, as I got into the car with my $4 worth of light bulbs today, I was almost as excited as him, just to see what he would do. That s the best part: I got to experience Jacob s joy right along with him. It s a time when I really enjoy being a dad.

For my birthday back in October, my wonderful wife gave me a Kindle 3 from Amazon. I d been considering other e-book readers for quite some time, but I had mostly ignored the Kindle due to the lack of EPUB support and a general dislike of Amazon s DRM enforcement. In the end, the superior hardware and ecosystem of the Kindle overpowered those concerns and overall I m very pleased with the purchase. The screen is amazing, literally just like reading off a piece of paper and the selection of books is OK. I ve been buying almost all my books from Amazon to date since it s so easy (the Whispernet is amazingly quick!) but it s not terribly difficult to get EPUBs from elsewhere onto the device after a quick run through Calibre to turn them into a MOBI file, so I keep telling myself I ve still got some flexibility. Almost as much fun as reading on the device has been learning about how it works. The Mobile Read forums have lots of step by step posts on how to do specific tasks like replacing the screensaver image, but they don t give much background detail on how the Kindle is actually operating which is what really interests me. Luckily among all the step by step posts I also found a usbnetwork package which also adds an SSH server to the Kindle, so after installing that and then SSHing in to my Kindle I ve been poking around. Under the cover the Kindle reveals a fairly standard Linux installation. While the hardware and IO devices are obviously unique, compared to something like an Android phone, the Kindle is refreshingly normal . Hardware

• Freescale MX35 ARMv6 based CPU with Java specific instruction support.
• 256MB RAM
• 4GB of internal flash presented as an SDHC device with four partitions. A ~700MB root partition, a ~30MB /var/local partition, another roughly 30MB kernel partition and then the rest (~3.1G) as the writeable user partition where your books and other content are stored. The root and /var/local partitions are ext3! (not jffs or some other more traditional flash based file system) while the user partition is vfat for easy use with Windows, etc.
• The board is code-named luigi and there are lots of references to mario and fiona scattered around the device, and even in some URLs on Amazon s website. Someone was obviously a Super Mario fan.
• The wireless chipset is Atheros based using the ar6000 drivers.
• The WAN (3G) modem presents itself as a USB serial device and is controlled via a custom daemon named wand which uses the standard Linux pppd package to establish IP connections over a private APN Amazon (provided by Vodafone here in Ireland).
• The EInk display shows up as some special files under /proc rather than as a device. With a bit of digging I found some simple constants that when written to the proc files cause the screen to display the standard boot/progress/upgrading images. I haven t deciphered how to make more complex updates to the display yet.

Software

• The kernel is based on Linux 2.6.26, with a bunch of hardware specific patches and drivers from lab126.com, an Amazon subsidiary who appear to be responsible for much of the low-level driver and device development.
• Lots of familiar open source projects are present, e.g. syslog-ng, DBus, busybox, pppd, wpa_supplicant, gstreamer, pango, openssl and the list goes on. You can download all the sources from Amazon s website. I haven t spent any time to see what if anything has been modified.
• There were a few unexpected finds as well such as GDB and powertop! No doubt useful for the developers, but highly unlikely to actually be used on a shipping Kindle.
• Boot-up is controlled by a set of sysv style init scripts which setup the filesystems and then start a handful of daemons to look after the low-level subsystems (network, power, sound) as well as the standard syslog and cron daemons you d expect to see on any Linux box.
• Once the basic system is up and running the init scripts kick off the framework which lives under /opt/amazon/ebook and consists of lots of Java classes. The system uses the cvm Java environment from Sun/Oracle which is specialised for embedded low-memory devices like this. The framework appears to take over most of the co-ordination, management and interaction tasks once it has started up.

The application/framework code is heavily obfuscated apparently using the Allatori Java Obfuscator. The jrename and jd-gui utilities have proven very handy in helping to untangle the puzzle, although they still only leave you with a pile of Java source code with mostly single letter alphabetic variable and class names! I ve been using IntelliJ s support for refactoring/renaming Java code to slowly work through it (thanks in large part to error/log messages and string constants found through the code which can t be obfuscated easily and help to explain what is going on), and I m slowly beginning to piece together how the book reading functionality works. I ll maybe write more on this in a future post. In one of my initial tweets about the Kindle I mentioned that it seemed to be regularly uploading syslog data to Amazon based on some sendlogs scripts I d noticed and a few syslog lines containing GPS co-ordinates that had been pasted on the Mobile Read forums. I can t find any trace of GPS co-ordinates in any syslog messages I ve seen on my device, but there is definitely information about the cell sites that my Kindle can see, the books that I m opening and where I m up to in them:

101206:235431 wand[2515]: I dtp:diag: t=4cfd77b7,MCC MNC=272 01,Channel=10762,Band=WCDMA I IMT 2000,Cell ID=1362209,LAC=3021,RAC=1
,Network Time=0000/00/00 00.00.00,Local Time Offset=Not provided,Selection Mode=Automatic,Test Mode=0,Bars=4,Roaming=1,RSSI=-88,Tx
Power=6,System Mode=WCDMA,Data Service Mode=HSDPA,Service Status=Service,Reg Status=Success,Call Status=Conversation,MM Attach St
ate=Attach accept,MM LU State=LU update,GMM Attach State=Attach accept,GMM State=Registered,GMM RAU State=Not available,PDP State=
Active,Network Mode=CS PS separate attach mode,PMM Mode=Connected,SIM Status=Valid; PIN okay; R3,MM Attach Error=No error,MM LU Er
ror=No error,GMM Attach Error=No error,GMM RAU Error=Not available,PDP Rej Reason=No error,Active/Monitored Sets=0;39;-11 1;180;-1
5,RSCP=-111,DRX=64,HSDPA Status=Active,HSDPA Indication=HSDPA HSUPA unsupp,Neighbor Cells=,Best 6 Cells=,Pathloss=,MFRM=,EGPRS Ind
ication=,HPLMN=,RPLMN=272;01 ,FPLMN=234;33 234;30 234;20 272;05 ,n=1: 101206:235758 cvm[3426]: I Reader:BOOK INFO:book asin=B003IWZZ3Y,file size=233168,file last mod date=2010-11-27 19.18.22 +0000,con
tent type=ebook,length=MobiPosition_ 465747,access=2010-12-06 09.44.32 +0000,last read position=MobiPosition_ 464387,isEncrypted=f
alse,isSample=false,isNew=false,isTTSMetdataPresent=false,isTTSMetadataAllowed=true,fileExtn=azw: 101206:233416 udhcpc[5639]: Offer from server xxx.xxx.2.254 received
101206:233416 udhcpc[5639]: Sending select for xxx.xxx.2.10...
Interestingly you can see from the last two lines, that Amazon has taken some care to preserve privacy by not including the full IP address given to the device by my local Wifi network, so in light of that I find it interesting that they decided not to obfuscate the Cell and Book IDs in those respective log messages too. Seems rather inconsistent. As to how and when these logs are sent to Amazon, the picture is a little bit murky. Every 15 minutes tinyrot runs out of cron and rotates /var/log/messages if it is greater than 256k in size. Rotated logs are stored into /var/local/log under filenames like messages_00000044_20101207000006.gz and alongside the log files are a set of state files named nexttosendfile, messages_oldest, messages_youngest. Something regularly sweeps through this directory to update the state and remove the old logs (after sending them up to Amazon I assume). I suspect that something is buried in the Java application code mentioned above. On the whole the Kindle is a fascinating piece of technology. It delivers a wonderful reading experience on top of a familiar Linux system and is going to provide me with many more hours of entertainment as I unpack all the tricks and techniques that have gone into this device. I would recommended it as a present for geeks everywhere.

The relationship between Debian and Ubuntu has been the subject of many vigorous debates over the years, ever since Ubuntu s launch in 2004. Six years later, the situation has improved and both projects are communicating better. The Natty Narwhal Ubuntu Developer Summit (UDS) featured like all UDS for more than 2 years a Debian Health Check session where current cooperation issues and projects are discussed. A few days after that session, Lucas Nussbaum gave a talk during the mini-Debconf Paris detailing the relationship between both projects, both at the technical and social level. He also shared some concerns for Debian s future and gave his point of view on how Debian should address them. Both events give valuable insights on the current state of the relationship. Lucas Nussbaum s Debian-Ubuntu talk Lucas started by introducing himself. He s an Ubuntu developer since 2006 and a Debian developer since 2007. He has worked to improve the collaboration between both projects, notably by extending the Debian infrastructure to show Ubuntu-related information. He attended conferences for both projects (Debconf, UDS) and has friends in both communities. For all of these reasons, he believes himself to be qualified to speak on this topic. Collaboration at the technical level He then quickly explained the task of a distribution: taking upstream software, integrating it in standardized ways, doing quality assurance on the whole, delivering the result to users, and assuring some support afterward. He pointed out that in the case of Ubuntu, the distribution has one special upstream: Debian. Indeed Ubuntu gets most of its software from Debian (89%), and only 7% are new packages coming from other upstream projects (the remaining 4% are unknown, they are newer upstream releases of software available in Debian but he was not able to find out whether the Debian packaging had been reused or not). From all the packages imported from Debian, 17% have Ubuntu-specific changes. The reasons for those changes are varied: bugfixes, integration with Launchpad/Ubuntu One/etc., or toolchain changes. The above figures are based on Ubuntu Lucid (10.04) while excluding many Ubuntu-specific packages (language-pack-*, language-support-*, kde-l10n-*, *ubuntu*, *launchpad*). The different agendas and the differences in philosophy (Debian often seeking perfect solutions to problems; Ubuntu accepting temporary suboptimal workarounds) also explain why so many packages are modified on the Ubuntu side. It s simply not possible to always do the work in Debian first. But keeping changes in Ubuntu requires a lot of work since they merge with Debian unstable every 6 months. That s why they have a strong incentive to push changes to upstream and/or to Debian. There are 3 channels that Ubuntu uses to push changes to Debian: they file bug reports (between 250 to 400 during each Ubuntu release cycle), they interact directly with Debian maintainers (often the case when there s a maintenance team), or they do nothing and hope that the Debian maintainer will pick up the patch directly from the Debian Package Tracking System (it relays information provided by patches.ubuntu.com). Lucas pointed out that those changes are not the only thing that Debian should take back. Ubuntu has a huge user base resulting in lots of bug reports sitting in Launchpad, often without anyone taking care of them. Debian maintainers who already have enough bugs on their packages are obviously not interested in even more bugs, but those who are maintaining niche packages, with few reports, might be interested by the user feedback available in Launchpad. Even if some of the reports are Ubuntu-specific, many of them are advance warnings of problems that will affect Debian later on, when the toolchain catches up with Ubuntu s aggressive updates. To make this easier for Debian maintainers, Lucas improved the Debian Package Tracking System so that they can easily get Ubuntu bug reports for their packages even without interacting with Launchpad. Human feelings on both sides Lucas witnessed a big evolution in the perception of Ubuntu on the Debian side. The initial climate was rather negative: there were feelings of its work being stolen, claims of giving back that did not match the observations of the Debian maintainers, and problems with specific Canonical employees that reflected badly on Ubuntu as a whole. These days most Debian developers find something positive in Ubuntu: it brings a lot of new users to Linux, it provides something that works for their friends and family, it brings new developers to Debian, and it serves as a technological playground for Debian. On the Ubuntu side, the culture has changed as well. Debian is no longer so scary for Ubuntu contributors and contributing to Debian is The Right Thing to do. More and more Ubuntu developers are getting involved in Debian as well. But at the package level there s not always much to contribute, as many bugfixes are only temporary workarounds. And while Ubuntu s community follows this philosophy, Canonical is a for-profit company that contributes back mainly when it has compelling reasons to do so. Consequences for Debian In Lucas s eyes, the success of Ubuntu creates new problems. For many new users Linux is a synonym for Ubuntu, and since much innovation happens in Ubuntu first, Debian is overshadowed by its most popular derivative. He goes as far as saying that because of that Debian becomes less relevant . He went on to say that Debian needs to be relevant because the project defends important values that Ubuntu does not. And it needs to stay as an independent partner that filters what comes out of Ubuntu, ensuring that quality prevails in the long term. Fixing this problem is difficult, and the answer should not be to undermine Ubuntu. On the contrary, more cooperation is needed. If Debian developers are involved sooner in Ubuntu s projects, Debian will automatically get more credit. And if Ubuntu does more work in Debian, their work can be showcased sooner in the Debian context as well. The other solution that Lucas proposed is that Debian needs to communicate on why it s better than Ubuntu. Debian might not be better for everybody but there are many reasons why one could prefer Debian over Ubuntu. He listed some of them: Debian has better values since it s a volunteer-based project where decisions are made publicly and it has advocated the free software philosophy since 1993. On the other hand, Ubuntu is under control of Canonical where some decisions are imposed, it advocates some proprietary web services (Ubuntu One), the installer recommends adding proprietary software, and copyright assignments are required to contribute to Canonical projects. Debian is also better in terms of quality because every package has a maintainer who is often an expert in the field of the package. As a derivative, Ubuntu does not have the resources to do the same and instead most packages are maintained on a best effort basis by a limited set of developers who can t know everything about all packages. In conclusion, Lucas explained that Debian can neither ignore Ubuntu nor fight it. Instead it should consider Ubuntu as a chance and should leverage it to get back in the center of the FLOSS ecosystem . The Debian health check UDS session While this session has existed for some time, it s only the second time that a Debian Project Leader was present at UDS to discuss collaboration issues. During UDS-M (the previous summit), this increased involvement from Debian was a nice surprise to many. Stefano Zacchiroli the Debian leader collected and shared the feedback of Debian developers and the session ended up being very productive. Six months later is a good time to look back and verify if decisions made during UDS-M (see blueprint) have been followed through. Progress has been made On the Debian side, Stefano set up a Derivatives Front Desk so that derivative distributions (not just Ubuntu) have a clear point of contact when they are trying to cooperate but don t know where to start. It s also a good place to share experiences among the various derivatives. In parallel, a #debian-ubuntu channel has been started on OFTC (the IRC network used by Debian). With more than 50 regulars coming from both distributions, it s a good place for quick queries when you need advice on how to interact with the distribution that you re not familiar with. Ubuntu has updated its documentation to prominently feature how to cooperate with Debian. For example, the sponsorship process documentation explains how to forward patches both to the upstream developers and to Debian. It also recommends ensuring that the patch is not Ubuntu-specific and gives some explanation on how to do it (which includes checking against a list of common packaging changes made by Ubuntu). The Debian Derivative Front Desk is mentioned as a fallback when the Debian maintainer is unresponsive. While organizing Ubuntu Developer Week, Ubuntu now reaches out to Debian developers and tries to have sessions on working with Debian . Launchpad has also been extended to provide a list of bugs with attached patches and that information has been integrated in the Debian Package Tracking system by Lucas Nussbaum. Still some work to do Some of the work items have not been completed yet: many Debian maintainers would like a simpler way to issue a sync request (a process used to inject a package from Debian into Ubuntu). There s a requestsync command line tool provided by the ubuntu-dev-tools package (which is available in Debian) but it s not yet usable because Launchpad doesn t know the GPG keys of Debian maintainers. Another issue concerns packages which are first introduced in Ubuntu. Most of them have no reason to be Ubuntu-specific and should also end up in Debian. It has thus been suggested that people packaging new software for Ubuntu also upload them to Debian. They could however immediately file a request for adoption (RFA) to find another Debian maintainer if they don t plan to maintain it in the long term. If Ubuntu doesn t make this effort, it can take a long time until someone decides to reintegrate the Ubuntu package into Debian just because nobody knows about it. This represents an important shift in the Ubuntu process and it s not certain that it s going to work out. As with any important policy change, it can take several years until people are used to it. Both issues have been rescheduled for this release cycle, so they re still on the agenda. This time the UDS session was probably less interesting than the previous one. Stefano explained once more what Debian considers good collaboration practices: teams with members from both distributions, and forwarding of bugs if they have been well triaged and are known to apply to Debian. He also invited Ubuntu to discuss big changes with Debian before implementing them. An interesting suggestion that came up was that some Ubuntu developers could participate in Debcamp (one week hack-together before Debconf) to work with some Debian developers, go through Ubuntu patches, and merge the interesting bits. This would nicely complement Ubuntu s increased presence at Debconf: for the first time, community management team member Jorge Castro was at DebConf 10 giving a talk on collaboration between Debian and Ubuntu. There was also some brainstorming on how to identify packages where the collaboration is failing. A growing number of Ubuntu revisions (identified for example by a version like 1.0-1ubuntu62) could indicate that no synchronization was made with Debian, but it would also identify packages which are badly maintained on the Debian side. If Ubuntu consistently has a newer upstream version compared to Debian, it can also indicate a problem: maybe the person maintaining the package for Ubuntu would be better off doing the same work in Debian directly since the maintainer is lagging or not doing their work. Unfortunately this doesn t hold true for all packages since many Gnome packages are newer in Ubuntu but are actively maintained on both sides. Few of those discussions led to concrete decisions. It seems most proponents are reasonably satisfied with the current situation. Of course, one can always do better and Jono Bacon is going to ensure that all Canonical teams working on Ubuntu are aware of how to properly cooperate with Debian. The goal is to avoid heavy package modifications without coordination. Conclusion The Debian-Ubuntu relationships used to be a hot topic, but that s no longer the case thanks to regular efforts made on both sides. Conflicts between individuals still happen, but there are multiple places where they can be reported and discussed (#debian-ubuntu channel, Derivatives Front Desk at derivatives@debian.org on the Debian side or debian@ubuntu.com on the Ubuntu side). Documentation and infrastructure are in place to make it easier for volunteers to do the right thing. Despite all those process improvements, the best results still come out when people build personal relationships by discussing what they are doing. It often leads to tight cooperation, up to commit rights to the source repositories. Regular contacts help build a real sense of cooperation that no automated process can ever hope to achieve.

This article was first published in Linux Weekly News. You can get my monthly summary of the Debian/Ubuntu news, all you have to do is to click here to subscribe to my free newsletter.

5 comments Liked this article? Click here. My blog is Flattr-enabled.

The Ubuntu Technical Board is currently conducting a review of the top ten Brainstorm issues users have raised about Ubuntu, and Matt asked me to investigate and respond to Idea #25301: Keeping the time accurate over the Internet by default. My first reaction was "hey, that's odd - I thought we already did that?". We install the ntpdate package by default (although it's deprecated upstream in favour of other tools, but that shouldn't be important here). ntpdate is run from /etc/network/if-up.d/ntpdate, in other words every time you connect to a network, which should be acceptably frequent for most people, so it really ought to Just Work by default. But this is one of the top ten problems where users have gone to the trouble of proposing solutions on Brainstorm, so it couldn't be that simple. What was going on? I brought up a clean virtual machine with a development version of Natty (the current Ubuntu development version, which will eventually become 11.04), and had a look in its logs: it was indeed synchronising its time from ntp.ubuntu.com, and I didn't think anything in that area had changed recently. On the other hand, I had occasionally noticed that my own laptop wasn't always synchronising its time quite right, but I'd put it down to local weirdness as my network isn't always very stable. Maybe this wasn't so local after all? So, I started tracing through the scripts to figure out what was going on. It turned out that I had an empty /etc/ntp.conf file on my laptop. The /usr/sbin/ntpdate-debian script assumed that that meant I had a full NTP server installed (I don't), and fetched the list of servers from it; since the file was empty, it ended up synchronising time from no servers, that is, not synchronising at all. I removed the file and all was well. That left the question of where that file came from. It didn't seem to be owned by any package; I was pretty sure I hadn't created it by hand either. I had a look through some bug reports, and soon found ntpdate 1:4.2.2.p4+dfsg-1ubuntu2 has a flawed configuration file. It turns out that time-admin (System -> Administration -> Time and Date) creates an empty /etc/ntp.conf file if you press the reload button (tooltip: "Synchronise now"), as part of an attempt to update NTP configuration. Aha! Once I knew where the problems were, it was easy to fix them. I've uploaded the following changes, which will be in the 11.04 release:

• Disregard empty ntp.conf files in ntpdate-debian.
• Remove an empty /etc/ntp.conf file on fresh installation of the ntp package, so that it doesn't interfere with creating the normal configuration file.
• Don't create the NTP configuration file in the time-admin backend if it doesn't exist already.

I've also sent these changes to Debian and GNOME as appropriate. There are still a few problems. The "Synchronise now" button doesn't work quite right in general (bug #90524), and if your network doesn't allow time synchronisation from ntp.ubuntu.com then you'll have to change the value of NTPSERVERS in /etc/default/ntpdate. Furthermore, the time-admin interface is confusing and makes it seem as though the default is not to synchronise the time automatically; this interface is being redesigned at the moment, which should be a good opportunity to make it less confusing, and I will contact the designers to mention this problem. On the whole, though, I think that many fewer people should have this kind of problem in Ubuntu 11.04. It's always possible that I missed some other problem that breaks automatic time synchronisation for people. Please do file a bug report if it still doesn't work for you in 11.04, or contact me directly (cjwatson at ubuntu.com).

As a first step on the road to evaluating different configuration management systems, I set up two VMs to test out puppet. Puppet is conveniently packaged within Debian so client and server installation was very simple. By default, puppet clients will look for a host named puppet on the local network, so by naming the server sensibly the two started talking to each other. puppetca --sign [client hostname] on the server started the ball rolling. My first test configuration was to add a user to the sudoers file. With CF Engine 2, I would have achieved this by using editfiles to check for a regular expression and insert a line if necessary. I stumbled across some CF Engine-style definitions for puppet in a Debian Administration article (taken originally from puppet's own wiki, which have since been moved/deleted) and achieved it using

append_if_no_such_line   sudo:
  file => "/etc/sudoers",
  line => "jon ALL=(ALL) ALL"
 

I'm a little concerned that this approach to problem solving is no better than CF Engine 2, and in fact worse than how I'd do it with CF Engine 2: if the host had a line with a different quantity of whitespace, such as jon ALL=(ALL) ALL, it won't match exactly and a duplicate line will be appended, whereas with CF Engine 2 I would have checked for something like ^jon \+ALL=(ALL) \+ALL$. I set up my work laptop as another test node, and used puppet to install some packages that I commonly use:

class jon-desktop  
  package   ['icedove', 'git-core', 'vim-gnome', 'vinagre', 'build-essential',
             'devscripts', 'subversion', 'git-buildpackage', 'mutt',
             'offlineimap', 'ascii', 'gitk', 'chromium-browser'
            ]: ensure => installed  
 

This worked fine, but is a little verbose and the package list is awkward to maintain buried inside the puppet syntax. I spent some time trying to see whether I could populate a puppet list based on the contents of a file: then I could list the packages one-per-line in a separate file. I haven't found a way of doing that yet. Using my laptop as a second node also exposed some interesting puppet client behaviour. Once I went off-site, the laptop tried to look up puppet on foreign networks (such as my home network). I would have assumed that, once the strong SSL association had been made, the client would try to connect to the FQDN of the server. This probably isn't a serious problem in practice, as we probably won't need to rely on nodes off-site talking to the configuration server, but I hope that the client doesn't trust any host that happens to answer to puppet... I fixed this by hacking my hosts file to include an entry for puppet. I could maintain this hack using Puppet itself, which would protect me against stale definitions if any renumbering takes place. I could also have added a line

server = puppet.fqdn.example.org

in /etc/puppet/puppet.conf, which is a more robust way of solving the problem.