On Wednesday, February 1, a new virtual server May First/People Link recently rented went offline. We contracted the virtual server through Server4All because we need their un-metered 100Mbit connection to help us handle the bandwidth for Sahara Reporters, one of the most important independent news sources for Saharan Africa. With the server offline, the web site was down as well. We scrambled to setup alternative caching servers to handle the bandwidth. When I logged into our control panel, I saw the message: This virtual server has been suspended by the administrator. Please contact support. I immediately contacted support and then received the message:

Hello
We have received the following complaint associated with your server/service.
IP: 76.73.121.164
To prevent any further abuse we have suspended this service. In order to
resume, we request you to cooperate with our investigation as promptly as
possible. Please respond to us with the following details:
(1) What has caused the complaint
(2) What is the server used for. Purpose?
(3) How can you resolve the complaint and make sure it will not be repeated.
Depending on the nature of the complaint and your response, we will put back
the server online.  Please note, this has violated our Terms Of Service. We
expect your response within 24 hours, otherwise your account will be
terminated permanently.  Thank you

What?? What complaint?? I followed up but had to wait til the next day to get the response.

Here is the full log,
An email advertizing the Domain Name: saharareporters.com
has been sent to the blacklist.woody.ch spamtrap.
This Domain does resolve to IP addresses one of which your are responsible:
76.73.121.164
Please investigate why this Domain has been advertized.
Attached you find the headers and reports in ARF for automatic processing.
Feedback is appreciated.
Actual listing periods:
Bounce: 1 Hour in DNS.
Whitelisted IP: Not lised in DNS.
Spam: 24 hours in DNS.
Every Hit: 14 days in evidence DB.
For any questions or Feedback, contact abuse@woody.ch
From: is intentionally set to a bit-bucket.
Kind regards
-Benoit Panizzon-

There is no attachment. I went to woody.ch and it was in German. Then tried blacklist.woody.ch, but no luck. Finally I found the Woody's World Blacklist Page. I plugged in our IP address into their checker and I got:

Output from the Check, if empty the IP is not listed.
164.121.73.76.[name of the blacklist] being tested.
Host 164.121.73.76.blacklist.woody.ch not found: 3(NXDOMAIN)
Host 164.121.73.76.blacklist.woody.ch not found: 3(NXDOMAIN)
Host 164.121.73.76.rbl.maps.vix.com not found: 3(NXDOMAIN)
Host 164.121.73.76.rbl.maps.vix.com not found: 3(NXDOMAIN)
Host 164.121.73.76.relays.mail-abuse.org not found: 3(NXDOMAIN)
Host 164.121.73.76.relays.mail-abuse.org not found: 3(NXDOMAIN)
;; connection timed out; no servers could be reached
;; connection timed out; no servers could be reached
;; connection timed out; no servers could be reached
;; connection timed out; no servers could be reached
Host 164.121.73.76.relays.ordb.org not found: 3(NXDOMAIN)
Host 164.121.73.76.relays.ordb.org not found: 3(NXDOMAIN)
Host 164.121.73.76.dev.null.dk not found: 3(NXDOMAIN)
Host 164.121.73.76.dev.null.dk not found: 3(NXDOMAIN)
Host 164.121.73.76.blackholes.five-ten-sg.com not found: 3(NXDOMAIN)
Host 164.121.73.76.blackholes.five-ten-sg.com not found: 3(NXDOMAIN)
Host 164.121.73.76.bl.spamcop.net not found: 3(NXDOMAIN)
Host 164.121.73.76.bl.spamcop.net not found: 3(NXDOMAIN)
Host 164.121.73.76.relays.visi.com not found: 3(NXDOMAIN)
Host 164.121.73.76.relays.visi.com not found: 3(NXDOMAIN)
164.121.73.76.blacklist.spambag.org has address 208.91.197.182
164.121.73.76.blacklist.spambag.org descriptive text "v=spf1 -all"

So, Woody's World thinks we are listed in spambag.org. I went to spambag.org and learned that the domain is for sale. I then tried blacklist.spambag.org and got the same page. This page has many links all pointing to advertisements. The "RBL List" link takes me to a page advertising "5 foods you must not eat." Next, out of curiousity, I tried Mxtoolbox. I got one hit from Barricuda. Barricuda says the reputation of the IP address is "poor". Why? According to Barricuda, the reasons could be:

• Your email server contains a virus and has been sending out spam.
• Your email server may be improperly configured.
• Your PC may be infected with a virus or botnet software program.
• Someone in your organization may have an infected PC with a virus or botnet program.
• You may be using a dynamic IP address which was previously used by a known spammer.
• Your marketing department may be sending out bulk emails that do not comply with the CAN-SPAM Act.
• You may have an insecure wireless network which is allowing unknown users to use your network to send spam.
• In some rare cases, your recipient's Barracuda Spam Firewall may be improperly configured.

Keep in mind, this IP address is not used for sending email. It's just the web site. I then took a step back and re-read the complaint and noticed that it says that the domain name saharareporters.com was listed in a spam email. Hm. More concerted searching for the terms "woody spamtrap blacklist" and I found a pattern in URLs that suggested I plug in the following: http://news.scoutnet.org/rblhostlist.php?id=saharareporters.com.uri And sure enough, there was a result. In short, it was a classic Nigerian Oil scam in which the person claims to be "JAMES IBORI ex-governor of DELTA STATE oil city." The scammer acknowledges that he has been arrested, but promises lots of cash to the person who can help him. In an effort to boost their credibility, the scammer included a link to a Sahara Reporters article about the real James Ibori. And that, my friends, is enough to have one of the most prominent independent African news organization taken offline. But, Sahara Reporters shouldn't necessarily feel singled out. Sahara Reporters should feel singled out. They exist to illuminate news from Africa. They were taken offline because a series of individuals don't know the difference between a prominent independent African news service and a criminal scammer. To most of the Internet, Nigeria and email scams are synonymous. That has to change. Although Sahara Reporters is particularly vulnerable, any site hosted with Server4All can potentially be taken down. All you have to do is write a fake spam/scam email, including a link to the web site you want to be taken offline, and then send that email to: listme@blacklist.woody.ch. I'm currently following up with both Woody's World and Server4All. However, once this particular issue is resolved, we're left with a much bigger and ominous problem. If your hosting provider (or their upstream provider) takes your site offline when it receives a complaint first and then asks questions second, you have a big problem. All of our legal fights over our rights to keep content online are moot if our providers, without any legal pressure to do so, still take down our services based on spurious complaints.

I'm using GTK+ in my paid work, which is being cool because I like this library. It's not the first time I use it for paid work, and I like these opportunities to learn more about it and possibly report bugs. It's not easy to find bugs in such a widely used library, so much of them end up being simple cosmetic mistakes, like this one. Some others are only a question of interpretation, like this other, which even when I submitted a patch, it was not applied. Yesterday I couldn't make popup-menu create a, well, popup menu. It was working with button-press-event, but not with popup-menu. I created a small code sample to illustrate this problem and asked about it in #pygtk@irc.gimp.org --- yes, I'm using Python:

import gtk
import gobject
def menu():
  item = gtk.ImageMenuItem('gtk-add')
  item.show()
  menu = gtk.Menu()
  menu.add(item)
  return menu
def on_view_popup_menu(obj, *data):
  print 'popup-menu'
  menu().popup(None, None, None, 0, 0)
def on_view_button_press_event(obj, *data):
  print 'button-press-event'
  menu().popup(None, None, None, 0, 0)
store = gtk.ListStore(gobject.TYPE_STRING)
store.append(('teste',))
view = gtk.TreeView(store)
view.append_column(gtk.TreeViewColumn('Coluna', gtk.CellRendererText(), text=0))
view.connect('button-press-event', on_view_button_press_event)
view.connect('popup-menu', on_view_popup_menu)
view.show()
window = gtk.Window()
window.add(view)
window.show()
gtk.main()

When I clicked in the window, with any button in this case, 'button-press-event' is printed to stdout and the menu is displayed. When I press the Menu key, or Shift+F10, 'popup-menu' is printed to stdout, but the menu is not displayed. After getting no response in #pygtk, I decided to try #gtk+. Before that, I thought about trying the same problem in C to see if it was not a problem in the bindings, or in how I was using it. The code:

#include <gtk/gtk.h>
GtkMenu *menu()  
  GtkImageMenuItem *image_menu_item =
      GTK_IMAGE_MENU_ITEM(
          gtk_image_menu_item_new_from_stock(GTK_STOCK_ADD, NULL));
  gtk_widget_show(GTK_WIDGET(image_menu_item));
  GtkMenu *menu = GTK_MENU(gtk_menu_new());
  gtk_container_add(GTK_CONTAINER(menu), GTK_WIDGET(image_menu_item));
  return menu;
 
void on_tree_view_popup_menu(GtkWidget *widget, gpointer user_data)  
  printf("popup-menu\n");
  gtk_menu_popup(
      menu(), NULL, NULL, NULL, NULL, 0, gdk_event_get_time(NULL));
 
void
on_tree_view_button_press_event(
  GtkWidget *widget, GdkEventButton *event, gpointer user_data)  
  printf("button-press-event\n");
  gtk_menu_popup(
      menu(), NULL, NULL, NULL, NULL, 0, gdk_event_get_time(NULL));
 
int main(int argc, char **argv)  
  gtk_init(&argc, &argv);
  GtkListStore *list_store = gtk_list_store_new(1, G_TYPE_STRING);
  GtkTreeIter iter;
  gtk_list_store_append(list_store, &iter);
  gtk_list_store_set(list_store, &iter, 0, "teste", -1);
  GtkTreeView *tree_view =
      GTK_TREE_VIEW(
          gtk_tree_view_new_with_model(GTK_TREE_MODEL(list_store)));
  gtk_tree_view_append_column(
      tree_view,
      gtk_tree_view_column_new_with_attributes(
          "Coluna", gtk_cell_renderer_text_new(), "text", 0, NULL));
  g_signal_connect(
      tree_view, "popup-menu", G_CALLBACK(on_tree_view_popup_menu), NULL);
  g_signal_connect(
      tree_view,
      "button-press-event",
      G_CALLBACK(on_tree_view_button_press_event),
      NULL);
  gtk_widget_show(GTK_WIDGET(tree_view));
  GtkWindow *window = GTK_WINDOW(gtk_window_new(GTK_WINDOW_TOPLEVEL));
  gtk_container_add(GTK_CONTAINER(window), GTK_WIDGET(tree_view));
  gtk_widget_show(GTK_WIDGET(window));
  gtk_main();
 

Wow. The difference between the codes is impressive. I'm not that used to code in GTK+ using C, and I may have missed something here, but the C code is so verbose that I get tired only by looking at it. I'm glad we have other languages, and GTK+ bindings for them. As expected, the behavior was the same, and I asked in #gtk+ and got no answer. Yes, I did read the documentation before asking, and no, I didn't read it carefully enough. Maybe because of coming from Python, I didn't checked very much the type of the callback function for the signals, and didn't notice the point about the return value: "Returns: TRUE if a menu was activated". And that was not where I was looking for the problem, since I thought the return value was usually related to the treatment of the signal by another functions. Anyway, I changed the type of on_tree_view_popup_menu to return gboolean and returned TRUE. It works! I also updated the type of on_tree_view_button_press_event to do the same, just for compliance. In Python, a simple return True would do it. I know I could use the event parameter in on_tree_view_button_press_event, but the point was exactly to run the same code and see if it would work in one case and not in the other, as happened.

I wanted to give a final update of my experiment with adding symbols files to C++ library packages. In the end, I reverted the changes and have gone back to not providing a symbols file, and instead just using shlibs. Some of this is because of the general state of the available tools, and some of it is due to the nature of the packages I was working on. This set of packages has frequent SONAME bumps from upstream, so backward compatibility across Debian stable versions can't happen anyway. They also don't use symbol export control, which means that the number of exported symbols is quite large and contains a lot of things leaked from internal objects. But there are also tool issues. The biggest that I ran into is that symbols appear and disappear in the export list with different versions of the compiler, and the pkg-kde-tools utilities, while excellent, still interpret those changes as architecture-specific changes. So the symbols file accumulates supposedly arch-specific variations, which are actually about what version of the compiler happened to be installed on that buildd when the package was built. This also means that if a package that has had its symbols file updated for one architecture is built again on the same architecture but on another buildd with a different g++, it will often FTBFS since a different set of symbols will be present. Obviously, this is too fragile to be maintainable. Further debian-devel discussion pointed out that these symbol variations are probably all inline functions, where the compiler may or may not output a weak version of the symbol if it chooses not to inline it in some particular case. The right thing to do for these libraries (not all, since there are other uses of weak symbols) is to mark all weak symbols as optional and not care if they randomly appear and disappear. There are also some 32-bit vs. 64-bit variations that are not captured by the subst mechanism, which resulted in the symbols file developing a list of architecture restrictions that amounted to "all 32-bit systems" or "all 64-bit systems." These explicit lists are also fragile, and will break with each new architecture introduced in Debian. I filed three bugs with pkg-kde-tools with suggestions for improving the tools for other people going forward: #657805 pkgkde-getbuildlogs gets truncated logs
#657806 representation of covariant return thunks
#658333 option to mark weak symbols as optional but I think that the level of work and remaining fragility doesn't make sense for a lot of C++ libraries, at least right now without more direct support in dpkg-gensymbols and other tool improvements. I'm therefore also planning on changing the proposed Policy patch for symbols to make it more clear that shlibs is an acceptable alternative for C++, and to patch Lintian to suppress the info tag for no symbols file for C++ libraries.

Hello Planet :) You may have seen my ancient blog posts pop up on the planet when XTaran was nice enough to add me to the planet's aggregation. Not quite sure why ikiwiki/planet decided to do this, but I will try to find out. If you have an idea, which does not involve re-spamming the feed, do let me know. In any case, expect mostly shell-related posts from me. And, as every year, I will be going to FOSDEM.

If you re not into packaging and if you asked how you could help Debian, someone probably suggested that you help to translate it. It s true that translating Debian is essential if we want to make Debian available to everybody on the world. There are many persons who are stuck as soon as they get a message in English, so it s important to aim for 100% coverage in terms of localization. Some vocabulary: localization vs internationalization Internationalization (i18n) is the work that makes it possible to translate messages in a given application. Localization (l10n) is the work of translating messages of said application. So as a translator, you ll be doing localization but some knowledge of internationalization is still useful because it will define how you re supposed to provide the translations. We ll come back to that later. Join your localization team Usually the translation work is shared among multiple translators within a localization team. Check out the Debian International page on www.debian.org to find out instructions for translators for each language. Many teams have a debian-l10n-*@lists.debian.org mailing list used for coordination, feel free to ask questions on those lists when you start (but make sure that you have read the relevant documentation before). Each team has its own workflow, so observe for a while to get used to what s happening before asking your first questions. What is there to translate? The translation of most of the software provided by Debian is not handled by Debian. The Debian translation teams only handle the translation of:

• the software that are specific to Debian (debian-installer, dpkg, APT, etc.) (*);
• the Debconf prompts in all Debian packages (*);
• the Debian documentation (*);
• the Debian website;
• the Debian wiki;
• the descriptions of packages.

Now before contributing to your first translation, I have to come back to internationalization to teach you a few things. In the above list, the projects marked with (*) do use PO files for their translation and the next sections will explain you how to work with those files. Introduction to Gettext The free software community has mostly standardized on a single internationalization infrastructure known as Gettext. With this tool, you re provided a POT file which contains all the translatable strings. It looks like this:

# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR Software in the Public Interest, Inc.
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR <email>, YEAR.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: dpkg 1.16.1\n"
"Report-Msgid-Bugs-To: debian-dpkg@lists.debian.org\n"
"POT-Creation-Date: 2011-09-23 03:37+0200\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <email>\n"
"Language-Team: LANGUAGE <ll>\n"
"Language: \n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=INTEGER; plural=EXPRESSION;\n"
#: lib/dpkg/ar.c:66
#, c-format
msgid "invalid character '%c' in archive '%.250s' member '%.16s' size"
msgstr ""
#: lib/dpkg/ar.c:81 lib/dpkg/ar.c:97 lib/dpkg/ar.c:108 lib/dpkg/ar.c:112
#: lib/dpkg/ar.c:134 utils/update-alternatives.c:1154
#, c-format
msgid "unable to write file '%s'"
msgstr ""
[ ]

The lines starting with #: are comments that indicate the source files where the (English) string is used. This can be useful if you want check the source to have more information about how the string is used. The lines starting with #, contain flags that can be important. If the fuzzy flag is set, the translated string is not used because it must be updated (or at least verified) since the original string evolved. The c-format flags indicates that the string must be a C format string, this has some implications in what s allowed in the string (in particular when it embeds conversion specifier for arguments submitted to printf-like functions). Another thing to note is that the translation of the empty string is used to store some meta-information about the translation itself. Contributing a translation as a PO file When you start a new translation, you copy that POT file to create a PO file for your own language (eg. fr.po for the French language). You replace some template values (identified with the upper case words in the POT file) and you replace all the empty strings on msgstr lines with the translation of the string that appears in the previous msgid line. The result could be something like this:

# translation of fr.po to French
# Messages fran ais pour dpkg (Linux-GNU Debian).
msgid ""
msgstr ""
"Project-Id-Version: fr\n"
"Report-Msgid-Bugs-To: debian-dpkg@lists.debian.org\n"
"POT-Creation-Date: 2011-09-23 03:37+0200\n"
"PO-Revision-Date: 2012-01-16 07:57+0100\n"
"Last-Translator: Christian Perrier <bubulle>\n"
"Language-Team: French <debian-l10n-french>\n"
"Language: fr\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: Plural-Forms: nplurals=2; plural=n>1;\n"
"X-Generator: Lokalize 1.2\n"
#: lib/dpkg/ar.c:66
#, c-format
msgid "invalid character '%c' in archive '%.250s' member '%.16s' size"
msgstr "caract re invalide  %1$c  dans la taille du membre  %3$.16s  de l'archive  %2$.250s "
#: lib/dpkg/ar.c:81 lib/dpkg/ar.c:97 lib/dpkg/ar.c:108 lib/dpkg/ar.c:112
#: lib/dpkg/ar.c:134 utils/update-alternatives.c:1154
#, c-format
msgid "unable to write file '%s'"
msgstr "impossible d' crire le fichier  %s "
[ ]

If there s already a PO file for your language, there might still work to do: there might be strings that have not yet been translated and there might be fuzzy strings which have to be updated strings which were already translated but where the original string has been modified. There are software that can assist you to edit PO files: poedit, virtaal, lokalize, gtranslator. There are also special extensions for vim (packaged in vim-scripts) and for Emacs. Submit the translation for inclusion Once you have a complete PO file, you should submit it for inclusion. Sometimes you will have been granted commit rights to the source code repository so that you can include your translation by yourself. In the other cases, you should submit your translation with a bug report tagged l10n and someone else will include your work in the next release. Depending on the team, the workflow might require a review before the submission. In that case, you usually have to send a call for review on the coordination mailing list. Go ahead! Hopefully those explanations will be enough to get you started. There are many other things to learn but it s good to learn while practicing For example, can you find out why the French translation above changed %c in %1$c ?

Do you want to read more tutorials like this one? Click here to subscribe to my free newsletter, you can opt to receive future articles by email.

No comment Liked this article? Click here. My blog is Flattr-enabled.

When you read this, GNOME Shell 3.2 will (hopefully!) have finally transitioned to Debian s testing suite. Planet GNOME readers might think Debian now has outdated versions of software even in their development versions, or the distribution s development marches at glacial pace. Wheezy GNOME users will finally have a Shell that matches the rest of their GNOME components, something that works with the Shell extensions website and much less problems and limitations compared to 3.0.2. The reality is that GNOME 3.2 s packaging was quite ready back when it was released in late September, but a number of not-so-desirable situations held GNOME Shell from transitioning to testing until today, four months later. So, what happened? TL;DR: transitioning from GNOME 2 GNOME 3 is not so easy if you want to keep testing in a sane state, and when you need to deal with dozens of indirectly related packages, for more than 10 architectures but it shouldn t take nearly a full year, either Let s go back to the last months of 2010. Debian squeeze is in very deep freeze, and the release team and many Debian developers are focusing on squashing as many release critical bugs as they can, in order to make Debian 6.0 the great release it ended up being. The GNOME project has recently delayed the big launch of GNOME 3.0 again, until March 2011; Debian has already settled on GNOME 2.28 for its release, although it will end up cherry-picking many updates from the 2.30 release modules. With most of the stabilization work being done, many Debian GNOME team members were at that time working on packaging very early versions of what would end up being GNOME 3.0 technology: GTK+3.0, GNOME Shell, Mutter and some brave users even tried to use it via the experimental archive. On February 6th, Debian 6.0 was released, and soon after, on April 6th, GNOME made a huge step forward with the much anticipated release of GNOME 3.0. At that time, Debian developers were busy breaking unstable as much as they could, as it s tradition on the weeks following a major release, and the Debian GNOME team was able to start moving some GNOME 3.0 libraries (those which were parallel-installable with their GTK+2.0 versions) to unstable. However, moving the bulk of GNOME 3.0 to unstable wasn t so easy. When you start doing that, you need to be sure you re ready to have all affected packages in a transitionable state as soon as possible, to minimise the chances of blocking transitions of unrelated packages via the dependencies they pick up with rebuilds. All the packages involved in a transition need to be ready to go in the same testing run , for all supported architectures. When you re dealing with dozens of GNOME source packages at the same time, many of which introduce new libraries, or worse, introduce incompatible APIs that affect many more unrelated packages, things get hairy, and you need a plan. So, Joss outlined what a sane approach to this monster transition could look like. The amount of work to do was what we call fun on #debian-gnome. In a nutshell, we had to deal with quite a few transitions, starting with having a newer version of libnotify in unstable, and a pre-requisite for that was making sure all the packages using libnotify1 were ready to use the source-incompatible libnotify4, and this meant preparing patches and NMUs for many of our packages, as well as many others not under our control. Before starting a controlled transition like this one, we had to get an ACK from the release team, who was busy enough handling other huge transitions like Perl 5.12, so by the time we got our own slot, we were well into Summer. With libnotify done in August, it was time to get our hands dirty with more exciting stuff, like getting Nautilus in testing. This meant bumping a soname and requiring all packages providing Nautilus extensions to migrate to GTK+3.0, or drop the extension entirely, as you can t mix GTK+2.0 and GTK+3.0 symbols in the same process. However, in GNOME 3.0, automounting code had moved from Nautilus to gnome-settings-daemon, so in order to not break filesystem automounting in testing for an unreasonable amount of time, both Nautilus and g-s-d needed to go in at the same time. The fun thing is that g-s-d dragged glib2.0, gvfs, gnome-control-center, gdm3, gnome-media, gnome-session and gnome-panel into the equation, so this transition needed extra planning and a lot more work than initially expected: migrating all nautilus extensions, plus ensuring all Panel applets had migrated to GTK+3.0 and the new libpanel-applet-4 interface. In short, this was the monster transition we were trying to avoid. By the time all this mess was sorted out, GNOME 3.2 had been released, and for what users said, it was a lot better than 3.0. We still had no more than a few bits and pieces of 3.0 in testing, and we were working hard to get 3.0 in wheezy. With all the excitement around 3.2, at times it was difficult to explain outsiders why we were beating a dead 3.0 horse Going back to our huge transition, it was just a matter of time before all the packages would be built and be ready to enter, on the same run, in testing. A few weeks later, in early November and after several rounds of mass-bug-filings, fixing unrelated FTBFS, many NMUs, package removal requests and dealing with any possible problem that could block our transition, everything seemed to be set, and our release team magicians had everything in place for the big magic to happen. However, our first clash with the rest of Debian happened a few hours before our victory, in the form of an unannounced ruby-gnome2 upload which resetted the count for everyone. It was fun to see the release team trying all sorts of black magic in an attempt to mitigate the damage. Fortunately, after a few tries they managed to fool britney (the script that handles package transitions from unstable to testing) somehow, and the hardest part of the job was done with just one day of delay. At last, the core of GNOME 3 was in testing, and testing users found soon after. The rest of the week saw a cascade of hate posts against GNOME 3 in Planet Debian, and personally I didn t find that especially motivating to keep on working on the rest of GNOME bits. With experimental clear of GNOME 3.0 stuff, we finally were able to focus on packaging whatever GNOME 3.2 components were not already done, and preparing for what should be a plain simple transition of GNOME 3.0 to 3.2. After our share of wait for a transition slot, as Perl 5.14, ICU and OpenSSL were in the line before us, and after dealing with a minor tracker 0.12 transition, we were ready for our next episode: evolution-data-server. At first sight, we thought this would be a lot easier, but it still got a bit hairy due to evo-data-server massive soname bumps. We were given our slot just before Christmas, after a few weeks of wait for others to finish their migration rounds, and most of the pack entered wheezy a few days before the new year. No rejoicing, though, as GNOME Shell 3.2 didn t make it. First, we discovered it was FTBFS on kFreeBSD architectures, as NetworkManager had been promoted from optional to required, for apparently no good reason, leaving the BSD world in the cold, including our exotic GNU/kFreeBSD architectures. Now, let s clarify that I m a supporter of the Debian kFreeBSD architectures and was really happy to see it accepted as a technology preview in squeeze. However, as you know, GNOME Shell currently requires hardware acceleration to run, a requirement hardly met in kFreeBSD, unless you re using a DRI1 X driver. We seriously doubted anyone had ever ran a GNOME 3 session on kfreebsd-*. However, if it didn t build, it was a blocker bug for GNOME Shell. We considered creating different meta-packages for kFreeBSD architectures, to conclude it d be a mess, so our awesome Michael Biebl ended up cooking up a patch that restored the ability to build the Shell without NetworkManager support. With this out of our way, we just needed to upload Michael s fix and watch the buildds do their part of the job. Or maybe not? Enter Iceweasel 9. In parallel, and with incredible bad timing, Iceweasel 9.0 was uploaded to Debian the very same day it was released by Mozilla. Again, it greeted us with a nasty surprise: yet another mozjs API change, which made gjs FTBFS, which meant our kFreeBSD fixes would be unusable until someone who knew Gjs internals well enough bit the bullet and worked around the new API changes. Again, Michael Biebl tried to be our saviour, but unfortunately wasn t able to fix all the problems, so we tried to focus on plan B. Mozilla had released a fork of the mozjs that is included in Firefox, so that embedders would have a bit less of a hard time with these recurrent API changes. This was based on Firefox 4, and was already being packaged by Ubuntu. Gjs would build using this older version just fine, so we just needed to get it in Debian as soon as possible. We just needed to find a sucke^Wvolunteer that would be inclined to maintain the beast. Only after a few weeks we managed to get Chris Coulson, the Ubuntu packager, to maintain the package directly through the Debian archive via package syncs. However, his package had only been auto-compiled in the three Ubuntu architectures, that is amd64, armel and i386. It s late January 2012, and we ve been fighting this war for 10 months. After getting some help from Michael to get the new package in shape for Debian standards, we were excited to sponsor it for Chris. Duh, after a few days in the NEW fridge, it was rejected by the ftp-masters. The license statement was missing quite a few details, so I went ahead and sacrificed a few hours of my copious free time to get this sorted out. A few days later, mozjs was accepted, but the result was horrible. It was very red. mozjs didn t build on half of our targets. Mike Hommey was quick to file a bug and point us to the most obvious fuckups. As he had dealt with this in the past as the Iceweasel maintainer, all of these issues were fixed and patches were ready to be applied verbatim or with minimal changes to our sources. With mozjs finally built successfully (although with severe problems on ia64), we were finally able to rebuild Gjs against it, upload GNOME Shell with our kFreeBSD fixes and wait until today for this mess to be over. Whew. I can t say I ve enjoyed all the stages of this ride. Some bumps on the road were clearly there to test our patience, but it has helped me get back in touch with non-leaf GNOME packaging, which was all I was doing for a while due to being super-busy lately with studies. It also reminds me of the privilege of working side by side with some awesome people, not only Joss, Michael, Sjoerd, Laurent or Gustavo, to name just a few Debian GNOME team members, but also the receptive release team members like Julien or Cyril, and NEW-processing record-breaking ftp-master Luca. Without them, we might be trying to figure out the Nautilus transition since last Summer. We really hope GNOME 3.4 will be a piece of cake compared to this. ;)

I've managed to go for eleven years working on Debian and nearly eight on Ubuntu without ever needing to teach myself how APT's resolver works. I get the impression that there's a certain mystique about it in general (alternatively, I'm just the last person to figure this out). Recently, though, I had a couple of Ubuntu upgrade bugs to fix that turned out to be bugs in the resolver, and I thought it might be interesting to walk through the process of fixing them based on the Debug::pkgProblemResolver=true log files. Breakage with Breaks The first was Ubuntu bug #922485 (apt.log). To understand the log, you first need to know that APT makes up to ten passes of the resolver to attempt to fix broken dependencies by upgrading, removing, or holding back packages; if there are still broken packages after this point, it's generally because it's got itself stuck in some kind of loop, and it bails out rather than carrying on forever. The current pass number is shown in each "Investigating" log entry, so they start with "Investigating (0)" and carry on up to at most "Investigating (9)". Any packages that you see still being investigated on the tenth pass are probably something to do with whatever's going wrong. In this case, most packages have been resolved by the end of the fourth pass, but xserver-xorg-core is causing some trouble. (Not a particular surprise, as it's an important package with lots of relationships.) We can see that each breakage is:

Broken xserver-xorg-core:i386 Breaks on xserver-xorg-video-6 [ i386 ] < none > ( none )

This is a Breaks (a relatively new package relationship type introduced a few years ago as a sort of weaker form of Conflicts) on a virtual package, which means that in order to unpack xserver-xorg-core each package that provides xserver-xorg-video-6 must be deconfigured. Much like Conflicts, APT responds to this by upgrading providing packages to versions that don't provide the offending virtual package if it can, and otherwise removing them. We can see it doing just that in the log (some lines omitted):

Investigating (0) xserver-xorg-core [ i386 ] < 2:1.7.6-2ubuntu7.10 -> 2:1.11.3-0ubuntu8 > ( x11 )
  Fixing xserver-xorg-core:i386 via remove of xserver-xorg-video-tseng:i386
Investigating (1) xserver-xorg-core [ i386 ] < 2:1.7.6-2ubuntu7.10 -> 2:1.11.3-0ubuntu8 > ( x11 )
  Fixing xserver-xorg-core:i386 via remove of xserver-xorg-video-i740:i386
Investigating (2) xserver-xorg-core [ i386 ] < 2:1.7.6-2ubuntu7.10 -> 2:1.11.3-0ubuntu8 > ( x11 )
  Fixing xserver-xorg-core:i386 via remove of xserver-xorg-video-nv:i386

OK, so that makes sense - presumably upgrading those packages didn't help at the time. But look at the pass numbers. Rather than just fixing all the packages that provide xserver-xorg-video-6 in a single pass, which it would be perfectly able to do, it only fixes one per pass. This means that if a package Breaks a virtual package which is provided by more than ten installed packages, the resolver will fail to handle that situation. On inspection of the code, this was being handled correctly for Conflicts by carrying on through the list of possible targets for the dependency relation in that case, but apparently when Breaks support was implemented in APT this case was overlooked. The fix is to carry on through the list of possible targets for any "negative" dependency relation, not just Conflicts, and I've filed a patch as Debian bug #657695. My cup overfloweth The second bug I looked at was Ubuntu bug #917173 (apt.log). Just as in the previous case, we can see the resolver "running out of time" by reaching the end of the tenth pass with some dependencies still broken. This one is a lot less obvious, though. The last few entries clearly indicate that the resolver is stuck in a loop:

Investigating (8) dpkg [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( admin )
Broken dpkg:i386 Breaks on dpkg-dev [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( utils ) (< 1.15.8)
  Considering dpkg-dev:i386 29 as a solution to dpkg:i386 7205
  Upgrading dpkg-dev:i386 due to Breaks field in dpkg:i386
Investigating (8) dpkg-dev [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( utils )
Broken dpkg-dev:i386 Depends on libdpkg-perl [ i386 ] < none -> 1.16.1.2ubuntu5 > ( perl ) (= 1.16.1.2ubuntu5)
  Considering libdpkg-perl:i386 12 as a solution to dpkg-dev:i386 29
  Holding Back dpkg-dev:i386 rather than change libdpkg-perl:i386
Investigating (9) dpkg [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( admin )
Broken dpkg:i386 Breaks on dpkg-dev [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( utils ) (< 1.15.8)
  Considering dpkg-dev:i386 29 as a solution to dpkg:i386 7205
  Upgrading dpkg-dev:i386 due to Breaks field in dpkg:i386
Investigating (9) dpkg-dev [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( utils )
Broken dpkg-dev:i386 Depends on libdpkg-perl [ i386 ] < none -> 1.16.1.2ubuntu5 > ( perl ) (= 1.16.1.2ubuntu5)
  Considering libdpkg-perl:i386 12 as a solution to dpkg-dev:i386 29
  Holding Back dpkg-dev:i386 rather than change libdpkg-perl:i386

The new version of dpkg requires upgrading dpkg-dev, but it can't because of something wrong with libdpkg-perl. Following the breadcrumb trail back through the log, we find:

Investigating (1) libdpkg-perl [ i386 ] < none -> 1.16.1.2ubuntu5 > ( perl )
Broken libdpkg-perl:i386 Depends on perl [ i386 ] < 5.10.1-8ubuntu2.1 -> 5.14.2-6ubuntu1 > ( perl )
  Considering perl:i386 1472 as a solution to libdpkg-perl:i386 12
  Holding Back libdpkg-perl:i386 rather than change perl:i386

Investigating (1) perl [ i386 ] < 5.10.1-8ubuntu2.1 -> 5.14.2-6ubuntu1 > ( perl )
Broken perl:i386 Depends on perl-base [ i386 ] < 5.10.1-8ubuntu2.1 -> 5.14.2-6ubuntu1 > ( perl ) (= 5.14.2-6ubuntu1)
  Considering perl-base:i386 5806 as a solution to perl:i386 1472
  Removing perl:i386 rather than change perl-base:i386

Investigating (1) perl-base [ i386 ] < 5.10.1-8ubuntu2.1 -> 5.14.2-6ubuntu1 > ( perl )
Broken perl-base:i386 PreDepends on libc6 [ i386 ] < 2.11.1-0ubuntu7.8 -> 2.13-24ubuntu2 > ( libs ) (>= 2.11)
  Considering libc6:i386 -17473 as a solution to perl-base:i386 5806
  Added libc6:i386 to the remove list

Investigating (0) libc6 [ i386 ] < 2.11.1-0ubuntu7.8 -> 2.13-24ubuntu2 > ( libs )
Broken libc6:i386 Depends on libc-bin [ i386 ] < 2.11.1-0ubuntu7.8 -> 2.13-24ubuntu2 > ( libs ) (= 2.11.1-0ubuntu7.8)
  Considering libc-bin:i386 10358 as a solution to libc6:i386 -17473
  Removing libc6:i386 rather than change libc-bin:i386

So ultimately the problem is something to do with libc6; but what? As Steve Langasek said in the bug, libc6's dependencies have been very carefully structured, and surely we would have seen some hint of it elsewhere if they were wrong. At this point ideally I wanted to break out GDB or at the very least experiment a bit with apt-get, but due to some tedious local problems I hadn't been able to restore the apt-clone state file for this bug onto my system so that I could attack it directly. So I fell back on the last refuge of the frustrated debugger and sat and thought about it for a bit. Eventually I noticed something. The numbers after the package names in the third line of each of these log entries are "scores": roughly, the more important a package is, the higher its score should be. The function that calculates these is pkgProblemResolver::MakeScores() in apt-pkg/algorithms.cc. Reading this, I noticed that the various values added up to make each score are almost all provably positive, for example:

         Scores[I->ID] += abs(OldScores[D.ParentPkg()->ID]);

The only exceptions are an initial -1 or -2 points for Priority: optional or Priority: extra packages respectively, or some values that could theoretically be configured to be negative but weren't in this case. OK. So how come libc6 has such a huge negative score of -17473, when one would normally expect it to be an extremely powerful package with a large positive score? Oh. This is computer programming, not mathematics ... and each score is stored in a signed short, so in a sufficiently large upgrade all those bonus points add up to something larger than 32767 and everything goes haywire. Bingo. Make it an int instead - the number of installed packages is going to be on the order of tens of thousands at most, so it's not as though it'll make a substantial difference to the amount of memory used - and chances are everything will be fine. I've filed a patch as Debian bug #657732. I'd expected this to be a pretty challenging pair of bugs. While I certainly haven't lost any respect for the APT maintainers for dealing with this stuff regularly, it wasn't as bad as I thought. I'd expected to have to figure out how to retune some slightly out-of-balance heuristics and not really know whether I'd broken anything else in the process; but in the end both patches were very straightforward.

If you install Debian on a computer with an AMD graphic card (an integrated Radeon HD 6310 in my case), you will only get garbage on your screen. Even without X.Org. This is because these graphic card require a non-free firmware to work, even in 2D with free drivers. So, boot with the kernel option radeon.modeset=0 to get a working console, then install the non-free package linux-firmware-nonfree. After that, do not forget to complain to AMD, because that really sucks hard. At the beginning there were microwave network cards unusable without non-free firmware, then wired network cards, and they are even invading graphic cards!

With some computer hardware, one need non-free firmware blobs. This is the sad fact of todays computers. In the next version of Debian Edu / Skolelinux based on Squeeze, we provide several scripts and modifications to make firmware blobs easier to handle. The common use case I run into is a laptop with a wireless network card requiring non-free firmware to work, but there are other use cases as well. First and foremost, Debian Edu provide ISO images for DVD and CD with all firmware packages in the Debian sections main and non-free included, to ensure debian-installer find and can install all of them during installation. This take care firmware for network devices used by the installer when installing from from local media. But for example multimedia devices are not activated in the installer and are not taken care of by this. For non-network devices, we provide the script /usr/share/debian-edu-config/tools/auto-addfirmware which search through the dmesg output for drivers requesting extra firmware. The firmware file name is looked up in the Contents-ARCH.gz file available in the package repository, and the packages providing the requested firmware file(s) is installed. I have proposed to do something similar in debian-installer (BTS report #655507), to allow PXE installs of Debian to handle firmware installation better. Run the script as root from the command line to fetch and install the needed firmware packages. Debian Edu provide PXE installation of Debian out of the box, and because some machines need firmware to get their network cards working, the installation initrd some times need extra firmware included to be able to install at all. To fill the PXE installation initrd with extra firmware, the /usr/share/debian-edu-config/tools/pxe-addfirmware script is provided. Again, just run it as root on the command line to fill the PXE initrd with firmware packages. Last, some LTSP clients might also need firmware to get their network cards working. For this, /usr/share/debian-edu-config/tools/ltsp-addfirmware is provided to update the LTSP initrd with firmware blobs. It is used the same way as the other firmware related tools. At the moment, we do not run any of these during installation. We do not know if this is acceptable for the local administrator to use non-free software, and it is their choice. We plan to release beta3 this weekend. You might want to give it a try.

Josselin Mouette is one the leaders of the pkg-gnome team, he takes sound technical decisions and doesn t fear writing code to work-around upstream issues. He deserves kudos for the work he has put into packaging GNOME over the years. He can also be very sarcastic (sometimes he even enjoys participating to flamewars on debian lists), and there are quite a few topics where we have long agreed to disagree. But this kind of diversity is also what makes Debian a so interesting place Read on to learn more about the pkg-gnome team, its plans for Wheezy, Josselin s opinion on the GNOME 3 switch, and much more. Raphael: Who are you? Josselin: I am a 31 years old Linux systems engineer. I started in life with physics, which I studied at the ENS Lyon. I started a thesis on experimental and numerical models for optoelectronics, but when it became clear that research was not for me, I abandoned it and accepted a job at the CEA, which holds the largest computing center in Europe. Working on these machines has been the most awesome job ever (except for it being near Paris). After that I worked a bit on system monitoring technologies. I am married, currently living in Lyon, and working for EDF (the French historical electricity company) on scientific workstations using Debian. EDF is using Debian on more than a thousand workstations and holds the fastest Debian supercomputer in the world (200 Tflops), which makes it another obvious place for Debian developers. Raphael: How did you start contributing to Debian? Josselin: I discovered Debian in 1999 while studying at the ENS, which is one of the biggest nests of Debian developers while being a small place, it is producing almost one Debian developer per year on average. After wondering for a while what it could be useful for, hacking on a slink snapshot made me think that it was for, well, everything except for gaming. Later, in 2002, when I was working on optoelectronics computing codes, I started to package them for Debian in order to make them easier to install, for us as well as other labs over the world. I started the NM process, and it was going smoothly but also going to take time. However, at that moment, the frozen-bubble game went out and made quite some buzz. Since I knew a guy who knew the game s developer, he asked me to package it. The package found 3 sponsors in a very short time and was fast-tracked into the archive at a speed that was unseen before. After which the NM process was completed very quickly. At that time, I was a heavy WindowMaker user, but I didn t like the direction the project was taking (actually, I wonder if there was one). GNOME was starting to become attractive, but its packaging in Debian was very ineffective, with many inconsistent packages maintained by people who didn t ever talk to each other some of them didn t speak English, and some of them didn t talk at all. Together with awesome people, among which Jordi Mallach, Gustavo Noronha Silva, JHM Dassen, Ross Burton and S bastien Bacher, we started the GNOME team in 2003, introducing consistent packaging practices, and initiating synchronized uploads. Releasing a completely integrated GNOME 2.8 in sarge was a considerable achievement; proving (together with the Perl team) that a team was the best way to maintain large package sets changed the way people work on Debian.

Proving [ ] that a team was the best way to maintain large package sets changed the way people work on Debian.

Raphael: You re one of the most active contributors of the team which is packaging GNOME for Debian. What would you suggest to a new contributor who would like to help the team? Josselin: There are several ways to contact the team, but the recommended one has always been IRC. We hang on #debian-gnome on the OFTC network, so just come around and ask for us. The real question is what you want to do in the team. Of course, most new volunteers want to help packaging the latest and greatest version of GNOME into unstable as soon as possible, but unless they already have Debian background, this is not the easiest task. Since there are already people working on this, the big packages are usually waiting on dependencies. I used to direct newcomers towards bug triage, but it is a tedious task and I m now convinced that our huge bug backlog will never be dealt with. The most useful thing to do for newcomers now is probably to find a GNOME or GNOME-related package that needs improvement or is lagging behind, and simply try to work on it. You can also come and fix the bugs you find annoying. Find a patch on the GNOME bugzilla, or cook it yourself, propose it, and if it s worthy enough you ll soon get commit access.

Our huge bug backlog will never be dealt with.

At this point I feel worth mentioning that if no one answers in 10 minutes, it doesn t mean that no one will answer in 2 hours, so please stay on the channel after asking. Raphael: There s been some controversy about GNOME 3 and the direction that the project is taking. What s your personal stance on GNOME 3? And what s the position of the pkg-gnome team? Josselin: The controversy is not new to GNOME 3, but the large-scale changes made with it have put it more prominently. The criticism usually boils down to a few categories:

1. General lack of configurability
2. Strange design decisions
3. Red Hat centric development
4. Hardware requirements
5. Change resistance

The lack of configuration options has been an ongoing criticism since GNOME 2.0 has decided to rip off most of them. Of course, when the control center was redesigned again for 3.0, there was a surge of horrified exclamations from people who missed their favorite buttons. On this topic, I fully concur with GNOME developers. The configuration option that is useful for you is not necessarily useful for someone else. Of course, sometimes developers go a bit too far, but the general direction is right. At work, we found that only a minority of users actually configure anything on their desktops: they just want something that works to launch their applications. Apple and Google have sold millions of devices by making them the simplest possible and without any configuration. Design decisions are, on the contrary, individual decisions, and each of them, while having reasons behind it, can be questioned. I remember seeing a lot of complaints when the OK and Cancel buttons were reversed in dialog boxes, something that nobody questions anymore. GNOME Shell is full of such changes; some are easy to get accustomed with, some others just make eyebrows raise. The most obvious example is the user menu in GNOME 3.2, which contains an entry to configure your Google account, but no entry to shutdown the computer. Both decisions were taken independently, each of them with (good or bad) reasons, but the result is simply ridiculous. The default configuration in Debian will contain an extension to make it a bit better, but on the whole we don t intend to diverge from the upstream design, on which a lot of good work has been done.

On the whole we don t intend to diverge from the upstream design, on which a lot of good work has been done.

Point 3 is more complex. Red Hat being the company spending the most on GNOME, it is obvious that their employees work on making things work for their distribution. An example is the recurring discussions about relying on system services that are currently only implemented by systemd. Since there is a lot of (mostly unjustified) resistance against systemd in Debian, and since it won t work on kFreeBSD anyway, someone needs to develop an alternative implementation of these services for upstart and sysvinit. Everything is in place for someone else to do the job but it has to be done, and this can be frustrating. Especially since it can also be hard to integrate changes needed for other distributions . Hardware requirements are mostly a consequence of the previous criticism: there s hardware that most distributions just don t want to bother supporting. We ve seen it in squeeze with the introduction of a hard dependency on PulseAudio. The Debian GNOME team (together with the Gentoo maintainers) made this dependency optional, carrying heavy patches, in order to cover the cases where it does not work. Now that it has gained more maturity, making this effort obsolete, the new tendency is to require 3D acceleration. For various reasons, it is not available to everyone . On this matter, the position of the Debian GNOME team has always been to support as much different configurations as possible with reasonable effort. Thanks to efforts from the incredible Vincent Untz, upstream supports a so-called fallback mode , which is the GNOME panel from 2.x with a lot of its bugs fixed. We intend to support this mode for as long as reasonably possible in Debian, possibly even after upstream ends up dropping it. However, other applications are going to require 3D because GStreamer is moving to clutter too, affecting video playback performance on non-accelerated systems . For epiphany this is not a problem; only embedded video will be affected. But for totem, this is a major issue; because of that we will probably keep totem 3.0 in wheezy. Finally, there is a natural human tendency to dislike change (I have it too), and it applies a lot to desktop users habits. Needless to say a change of such a scale as introducing GNOME Shell can trigger reactions. However, I don t think it is reasonable, because of this resistance, to keep gnome-panel 2.x in Debian. This would be a lot of work on obsolete technology, and would prevent the upcoming removal of a lot of deprecated libraries. This time is much better spent improving gnome-panel 3.x in Debian and keeping the fallback mode great. One of the change that was made in Debian was to make it easier to find, being available as GNOME Classic directly from the login manager, instead of having to find it in an obscure configuration panel. In all cases, I would recommend to actually try GNOME Shell for a few hours before ditching it. I had never been accustomed to a new environment as quickly ever before.

In all cases, I would recommend to actually try GNOME Shell for a few hours before ditching it.

Having seen several of my GDM patches reverted without a warning, I know we are not finished with carrying patches in Debian packages.
Scientific workstations are a non-trivial example, since there is a measurable effect of using 3D in the window manager on heavy 3D applications.
On the other hand, on accelerated systems, this feature should end up improving performance a lot. Raphael: What are your plans for Debian Wheezy? Josselin: The first goal of the GNOME team is, of course, to provide again a great desktop environment to work on. For wheezy it will probably be based on GNOME 3.4. There also needs to be some work on package management interfaces. Upstream bases everything on PackageKit, but it is not as featureful as the aptdaemon Ubuntu technology. If I have time, I would also like to improve HTTP proxy support, since currently it is based on a stack of terrible hacks. Raphael: If you could spend all your time on Debian, what would you work on? Josselin: Obviously I would like to make GNOME in Debian even better. That would imply working on underneath dependencies (what we now like to call plumbing) to make sure everything is working great. This would also imply working more as GNOME upstream to make it more suitable for our needs. I would also work on large-scale improvements on the distribution, like conditional recommends which I d love to see implemented , or automatic build-dependency generation. I would also work on the installer to make it better for desktops machines. The idea is to automatically install language packs, or glues between two packages when both packages are installed. Raphael: What s the biggest problem of Debian? Josselin: The obvious answer is the same as the one most people you interviewed before gave: not enough members in core teams. A lot of developers join Debian to work on a small number of pet packages, and don t necessarily want to be involved with existing teams. It is probably still not obvious enough that the primary way to start contributing to Debian is to join an existing team. But if there is one thing that is preventing Debian from gaining more momentum now, it is a completely different one: the too short support timeframe. 3 years is really not enough for corporate users. One year to migrate from one version to another is too short, and it is not possible to skip a release. It is definitely possible to change that with reasonable effort: the long-term support after 3 years doesn t have to cover the same perimeter as the short-term one. For example, we could upgrade the kernel to the version in the current stable release, and stop fixing all non-remote security holes. The important thing is to cover the most basic needs: companies are ready to take the risk of having less support if it allows skipping a version, but not the risk of having no support at all. And even more important is to say that you do something. Red Hat says they support a release for 10 years, but of course after 5 years the supported perimeter is extremely small.

3 years [of support] is really not enough for corporate users.

Long-term support will not magically fix all problems in Debian, but it will bring more corporate users into the picture. And with corporate users come paid Debian developers, who can work on critical pieces of the system. Debian was built on the synergy between individuals and companies, and in recent years perhaps as a reaction against what happened with Ubuntu we ve kind of forgot the latter. A lot of individuals have joined the project, and they are actively working, for example, on shortening the release cycle, which goes against the interest of professionals. We should embrace again such users and developers, and that means adapting to the current needs of larger entities. Raphael: You re the maintainer of python-support, a packaging helper that was competing with python-central. Both helpers are now deprecated in favor of dh_python2. Does this mean that the situation of Python in Debian is now sane? Or are there remaining problems? Josselin: dh_python2 (and the Python3 version, dh_python3) has a sane enough design. It fixes a lot of issues in python-central and also python-support, at the expense of somehow reduced functionality for developers. However, just like the previous tools, it merely works around design mistakes in the Python interpreter. For example it is not possible to split binary modules, pure-Python modules and byte-compiled modules in different directory trees, like Perl does although PEP 3147 introduces a way to do so. There is still no sane and standardized way to deal with module versions. There is no difference made between the module (which is a part of language semantics) and the file containing it (an information which depends on the implementation). Developers heavily rely on introspection features and make assumptions based on the implementation, that make it impossible to work around problems with module files. Such problems are not restricted to Python. Those who fought against Ruby gems could tell even worse stories. While introducing GObject introspection packages in Debian (they can be used in JavaScript and Python to provide modules based on GObject libraries), I was pleased to see a clear distinction between file and module, but I was again struck by the fact you are not forced to declare API versions in your Python/JS code. In all cases, there is no reliable way to detect runtime dependencies in a given Python or JavaScript file, which leaves the maintainer to declare them by hand, and of course, often be wrong about them. Add to that the fact that most errors cannot be detected before runtime. For all these reasons, and while still being fond of Python for scripts and prototyping, I ve become really skeptical of using purely interpreted languages to write real applications. Some GNOME developers are moving away from Python and JavaScript, mostly towards Vala; I can only approve of that move and hope the same happens to other projects. Raphael: Is there someone in Debian that you admire for their contributions? Of course there is the never-sleeping, never-stopping, Michael Biebl who can upload a whole GNOME release in a single week-end. But there are a lot of awesome people who make Debian something that simply works. I could talk about Cyril Brulebois from the X strike force, Julien Cristau from the release team, Sjoerd Simons for his sound advice and work on plumbing, Luca Falavigna who is so fast at processing NEW, to quote only a few of those I work with frequently. And of course, Jordi and Sam for their humor.

---

Thank you to Josselin for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Note that you can find older interviews on http://wiki.debian.org/PeopleBehindDebian.

Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Google+, Twitter and Facebook .

8 comments Liked this article? Click here. My blog is Flattr-enabled.

As a Linux integration guy at Zend, most of my time is spent in compiling PHP related code or dealing with the variety of Linux distributions we support. With the coming release for PHP 5.4, we (at Zend) had some interesting stuff going on. As part of the RC phase, I got to check the status of the 50-60 PHP extensions we provide, especially the PECL extensions which have different release cycles than the main PHP. With minor versions, this usually doesn t really matter, but for major versions this means that some extensions need a little bit of love and fixes to work well with the new PHP version. This of course with the help of our developers. The changes are usually one-liners due to a variable type change, or finding commits in the extension s SCM and applying/back-porting it to the current versions (e.g. pecl memcache). Our policy regarding the patches we have, is that they should at least be sent to upstream (a core member or a bug report, e.g. #55703). I think I m in the best position to enforce that patch policy, so in a few recent cases, I found myself asking one of our developers if the patch he sent me was already accepted upstream before willing to take it into the build process (in this case they are used temporarily, till we ll work with the next RC or final release).
Filed under: PHP

I was doing some finite element (FE) calculation and I needed the sum of the lowest 7 eigenvalues of a symmetric matrix (that comes from the FE assembly) to converge to at least 1e-8 accuracy (so that I can check calculation done by some other solver of mine, that calculates the same but doesn't use FE). In reality I wanted the rounded value to 8 decimal digits to be correct, so I really needed 1e-9 accuracy (but it's ok if it is let's say 2e-9, but not ok if it is 9e-9). With my FE solver, I couldn't get it to converge more than to roughly 5e-7 no matter how hard I tried. Now what?

When doing the convergence, I take a good mesh and keep increasing "p" (the polynomial order) until it converges. For my particular problem, it is fully converged for about p=25 (the solver supports the order up to 64). Increasing "p" further will not increase the accuracy anymore, and the accuracy stays at the level 5e-7 for the sum of the lowest 7 eigenvalues. For optimal meshes, it converges at p=25, for not optimal meshes, it converges for higher "p", but in all cases, it doesn't get below 5e-7.

I know from experience, that for simpler problems, the FE solver can easily converge to 1e-10 or more using double precision. So I know it is doable, now the question is what the problem is: there
are a few possible reasons:

• The FE quadrature is not accurate enough
• The condition number of the matrix is high, thus LAPACK doesn't return very accurate eigenvalues
• Bug in the assembly/solver (like single/double corruption in Fortran, or some other subtle bug)

When using the same solver for simpler potential, it converged nicely to 1e-10. So this suggests there is no bug in the assembly or solver itself. It is possible that the quadrature is not accurate enough, but again, if it converges for simple problem, it's probably not it. So it seems it is the ill conditioned matrix, that causes this. So I printed the residuals (that I simply calculated in Fortran using the matrix and the eigenvectors returned by LAPACK), and it only showed 1e-9. For simpler problems, it can go to 1e-14 easily. So that must be it. How do we fix it?

Obviously by making the matrix less ill conditioned, which is caused by the mesh for the problem (the ratio of the longest/shortest elements is 1e9) but for my problem I really needed such a mesh. So the other option is to increase the real number accuracy.

In Fortran all real variables are defined as real(dp), where dp is an integer defined at a single place in the project. There are several ways to define it, but it's value is 8 for gfortran and it means double precision.So I increased it to 16 (quadruple precision), recompiled. Now the whole program calculates in quadruple precision (more than 30 significant digits). I had to recompile LAPACK using the "-fdefault-real-8" gfortran option, that promotes all double precision numbers to quadruple precision, and I used the "d" versions (double precision, now promoted to quadruple) of LAPACK routines.

I rerun the calculation ---- and suddenly LAPACK residuals are around 1e-13, and the solver converges to 1e-10 easily (for the sum of the lowest 7 eigenvalues). Problem solved.

Turning my Fortran program to quadruple precision is as easy as changing one variable and recompiling. Turning LAPACK to quadruple precision is easy with a single gfortran flag (LAPACK uses the old f77 syntax for double precision, if it used real(dp), then I would simply change it as for my program). The whole calculation got at least 10x slower with quadruple. The reason is that gfortran runtime uses the libquadmath library, that simulates quadruple precision (as current CPUs only support double precision natively).

I actually discovered a few bugs in my program (typically some constants in older code didn't use the "dp" syntax, but had the double precision hardwired). Fortran warns about all such cases, when the real variables have incompatible precision.

It is amazing how easy it is to work with different precision in Fortran (literally just one change and recompile). How could this be done with C++? This wikipedia page suggests, that "long double" is only 80bit in most cases (quadruple is 128bit), but gcc offers __float128, so it seems I would have to manually change all "double" to "__float128" in the whole C++ program (this could be done with a single "sed" command).

Cops in Tennessee routinely steal cash from citizens [1]. They are ordered to do so and in some cases their salary is paid from the cash that they take. So they have a good reason to imagine that any large sum of money is drug money and take it. David Frum wrote an insightful article for NY Mag about the problems with the US Republican Party [2]. TreeHugger.com has an interesting article about eco-friendly features on some modern cruise ships [3]. Dan Walsh describes how to get the RSA SecureID PAM module working on a SE Linux system [4]. It s interesting that RSA was telling everyone to turn off SE Linux and shipping a program that was falsely marked as needing an executable stack and which uses netstat instead of /dev/urandom for entropy. Really the only way RSA could do worse could be to fall victim to an Advanced Persistent Attack :-# The Long Now has an interesting summary of a presentation about archive.org [5]. I never realised the range of things that archive.org stores, I will have to explore that if I find some spare time! Jonah Lehrer wrote a detailed and informative article about the way that American high school students receive head injuries playing football[6]. He suggests that it might eventually be the end of the game as we know it. Fran ois Marier wrote an informative article about optimising PNG files [7], optipng is apparently the best option at the moment but it doesn t do everything you might want. Helen Keeble wrote an interesting review of Twilight [8]. The most noteworthy thing about it IMHO is that she tries to understand teenage girls who like the books and movies. Trying to understand young people is quite rare. Jon Masters wrote a critique of the concept of citizen journalism and described how he has two subscriptions to the NYT as a way of donating to support quality journalism [9]. The only comment on his post indicates a desire for biased news (such as Fox) which shows the reason why most US media is failing at journalism. Luis von Ahn gave an interesting TED talk about crowd-sourced translation [10]. He starts by describing CAPTCHAs and the way that his company ReCAPTCHA provides the CAPTCHA service while also using people s time to digitise books. Then he describes his online translation service and language education system DuoLingo which allows people to learn a second language for free while translating text between languages [11]. One of the benefits of this is that people don t have to pay to learn a new language and thus poor people can learn other languages great for people in developing countries that want to learn first-world languages! DuoLingo is in a beta phase at the moment but they are taking some volunteers. Cory Doctorow wrote an insightful article for the Publishers Weekly titles Copyrights vs Human Rights [12] which is primarily about SOPA. Naomi Wolf wrote an insightful article for The Guardian about the Occupy movement, among other things the highest levels of the US government are using the DHS as part of the crackdown [13]. Naomi s claim is that the right-wing and government attacks on the Occupy movement are due to the fact that they want to reform the political process and prevent corruption. John Bohannon gave an interesting and entertaining TED talk about using dance as part of a presentation [14]. He gave an example of using dancerts to illustrate some concepts related to physics and then spoke about the waste of PowerPoint. Joe Sabia gave an amusing and inspiring TED talk about the technology of storytelling [15]. He gave the presentation with live actions on his iPad to match his words, a difficult task to perform successfully. Thomas Koch wrote an informative post about some of the issues related to binary distribution of software [16]. I think the problem is evenm worse than Thomas describes.

• [1] http://www.youtube.com/watch?v=Xg4ArbQpsIU
• [2] http://nymag.com/print/?/news/politics/conservatives-david-frum-2011-11/
• [3] http://www.treehugger.com/natural-sciences/7-ocean-friendly-eco-cruises-hitting-the-high-seas.html
• [4] http://danwalsh.livejournal.com/48571.html
• [5] http://longnow.org/seminars/02011/nov/30/universal-access-all-knowledge/
• [6] http://www.grantland.com/story/_/id/7443714/jonah-lehrer-concussions-adolescents-future-football
• [7] http://feeding.cloud.geek.nz/2011/12/optimising-png-files.html
• [8] http://www.helenkeeble.com/2011/08/08/repost-what-i-thought-about-twilight
• [9] http://www.jonmasters.org/blog/2011/11/30/on-citizen-journalism/
• [10] http://www.ted.com/talks/luis_von_ahn_massive_scale_online_collaboration.html
• [11] http://duolingo.com/
• [12] http://www.publishersweekly.com/pw/print/20111205/49728-cory-doctorow-copyrights-vs-human-rights.html
• [13] http://www.guardian.co.uk/commentisfree/cifamerica/2011/nov/25/shocking-truth-about-crackdown-occupy
• [14] http://www.ted.com/talks/john_bohannon_dance_vs_powerpoint_a_modest_proposal.html
• [15] http://www.ted.com/talks/joe_sabia_the_technology_of_storytelling.html
• [16] http://www.koch.ro/blog/index.php?/archives/153-On-distributing-binaries.html

Related posts:

1. Links January 2011 Halla Tomasdottir gave an interesting TED talk about her financial...
2. Links January 2010 Magnus Larsson gave an interesting TED talk about using bacteria...
3. Links January 2009 Jennifer 8 Lee gave an interesting TED talk about the...

... a week ago, in the shape of Alexandra Daphne Scholz, my enchanting daughter. She seems to have inherited my hairstyle. Gundemarie Scholz (my wife) and she are both doing remarkably well, to the extent that they both keep surprising medical staff with their rapid progress. For those of you that care about such things: She was 2.9kg (6lb 6oz) at birth, and 49cm (1'7") tall ... or should that be "long", given that she's not doing so much standing up as yet? What she is doing is sleeping, breast feeding, and excreting ... rather a lot. Between times, it's mostly gurgling cheerfully, looking around at the world, and attempting to study her own hands -- I find her completely fascinating. That being the case, don't expect anything very sensible out of me for the foreseeable future (so no change there, eh? ) and because I cannot resist the urge, here are a few more snaps of her: As you can see, at only one day old, it seems that she'd already managed to get hold of the Amulet of Yendor, and was having a well earned nap

Wow that s really big demonstration.
I ve just uploaded video with people walking near Wawel castle plenty of people during almost 20 minutes.
I suppose there was more than 50k people and wonder what s going to be shown in TV I really like it.

When you start your journey with Debian, you tend to have lots of questions. You ll find some answers in various documentations but there always are remaining questions. Those can be asked on various support channels:

• on mailing lists (debian-user@lists.debian.org for the English-speaking one);
• on IRC (channel #debian on irc.debian.org);
• on questions & answers websites like ask.debian.net;
• on web forums like forums.debian.net;
• on every other places where Debian users hang out together (on a booth during a conference, during an install party of a Linux User Group, etc.).

Those are the places where you can also start your journey as a Debian contributor instead of asking questions, you just have to answer questions of other users! Let me share some advice if you want to do some user support. User support is difficult It s not always an easy task. Some users are more skilled than others and there might be difficulties related to the language, English is not always the native language of a user who asks a question in English. Be respectful and courteous when you answer user questions, even if they made mistakes. You re effectively representing Debian and you should give out a good image of the project. If you don t have the patience or the time needed to do a good answer, don t reply and let someone else take care of this user. I invite you to read (and follow!) the Debian Community Guidelines. Avoid RTFM answers, instead you should show the users how they could have found (alone) the solution to their problem. We don t want to scare people away, we want to grow our community. But it s also rewarding In some cases, the problem reported by the user will be a real problem and you ll have an opportunity to file a good bug report, thus helping to improve Debian for everybody. Often, you don t even have the answer to the user s question. But you re more skilled than him/her to do researches on the web, or you know of a good documentation that might contain the relevant bits of information, in any case you re doing further research to help this user. In this process, you also grow your own skills since you re learning stuff that you didn t know yet. At least that s how I learned many things during my first year in the Debian community there s no reason why you couldn t learn lots of stuff that way, in particular if you also read the answers of other skilled people on those channels (it takes a bit of training to learn who are the skilled people though). I still believe that doing user support is one of the best ways to join the Debian community and to start contributing. It helps you to grow your skills, and to slowly progress from average user to advanced user .

If you want to start contributing to Debian, click here to subscribe to my newsletter and get future updates for new contributors. You can also follow me on Identi.ca, Google+, Twitter and Facebook.

5 comments Liked this article? Click here. My blog is Flattr-enabled.

Below is an email I received from Representative Rick Larsen s office. I don t recall ever having identifying myself as PVT Adams-Collier in any communications with his office, though I could have done so at some point. Maybe. I also don t recall contacting his office directly concerning SOPA. I blogged a few things and mentioned him on facebook a couple of times, though. It is good to know that I don t always have to go directly to my representative to have my voice heard. For those playing along at home, I think the best course of action would be for producers to perform searches for their work on peer-to-peer file sharing search engines such as http://thepiratebay.org/ and http://scrapetorrent.com/. If producers find that their work is being distributed in a way which is contrary to their wishes, I advise the producer or their agent to contact the individuals operating seeds with high share ratio in order to ensure that they perform AAA before accepting new peers into the swarm. Those seed operators who refuse to negotiate reasonable AAA terms should be issued C&D orders, and legal action should be taken, should they fail to comply with the request. [edit 2012-01-24T05:27:44] It would probably be wise to impose a minimum fee of $5 for seed operators per content package they distribute. This will encourage them to pass this fee on to their subscribers in a legal and responsible manner. I recommend that it be distributed in the same fair and concise way in which the tickets are distributed for toll violations on the Tacoma Narrows Bridge. Ask me how I know. This will provide content producers a valuable distribution channel which is closely in touch with the consumers of the producers media. I believe that if the producer puts forth effort in good faith to develop and maintain relationships with key distribution points, we will see a situation where both content producers and consumers get what they want. The authorization phase of the AAA system could charge a Payment Card Industry or bitcoin account a reasonable fee. My employer produces content delivery appliances which would serve quite well the role of AAA gateway. For an example of this use case in action (sans account billing), I recommend contacting seeding participants of the Debian CD/DVD swarms. I will proxy such requests if there is interest. An alternative might be a tool developed for the online gaming community with a strange name: http://www.evenbalance.com/

---

Dear Pvt. Adams-Collier: Thank you for contacting me about the Stop Online Piracy Act (SOPA) and the Protect IP Act (PIPA). I wanted to update you on my views on this important issue. I am opposed to SOPA and PIPA in their current forms. I believe that these bills create unacceptable threats to free speech and free access to the internet. I have heard from many of you in Northwest Washington who are deeply concerned about the potential impacts of SOPA and PIPA. Online piracy is a serious problem that costs U.S. businesses billions of dollars. Government agencies must be empowered to stop and prosecute intellectual property thieves. But in doing so we cannot undermine freedom of speech or jeopardize the free flow of information on the internet. I will work with my colleagues to see that any final anti-online piracy legislation protects the internet and does not encroach on free speech rights. Please be assured that I will keep your thoughts in mind should I have the opportunity to vote on any legislation that would impact online piracy and internet freedom on the House floor. Again, thank you for contacting me. I encourage you to contact me in the future about this or any other issue of importance to you. Sincerely, Rick Larsen
United States Representative
Washington State, 2nd District

[edit 2012-01-24T05:28:09] Sincerely, C.J. Adams-Collier
San Juan County Democratic Central Committee (*whew*) PCO
San Juan County, Washington State, Orcas 3 District

As LibreOffice is approaching its final 3.5.0 release, I d like to sum up the RTL status for RC1. So far, 6 RTL related bugs were resolved in the 3.5 cycle (#32530, #34222, #40950, #43790, #43793, #44078), and a few minor issues reported directly to the developer s mailing list got quick responses. Most importantly, the new features of page break and header/footer not only support RTL but actually looks good. During the LibreOffice conference I was suggested to help with these features, providing feedback, and I m glade the needed attention was given to it. Besides that, a few l10n and translation issues were solved in the process of doing the Hebrew translation (which also reflects on other RTL languages). At a few cases, these issue because a general l10n issues which affects all the languages. In general, I found the core developers responsive to mails about RTL support. I m sure the talk about RTL problems during the conference helped, as well as being more active in the project and having more personal acquaintance with the developers. That s being said, RTL support for LibreOffice still has problems, which I hope will be pushed during the 3.5.x cycles (full list at Bug 43808, the rtl meta bug). As to get some focus regarding was is to be done, I m listing the top problems:

1. #44657 RTL UI: Horizontal scrollbar in calc main window is broken
2. #33302 brackets inverted in rtl text (mac only)
3. #37692 RTL list numbering reverses its direction
4. #42070 - RTL support in broken in presenter Console extension
5. #32531 - Incorrect cursor key movement between table cells of different directionality
6. #104515 - RTL UI: moving active embedded object to the left moves it to the right (reported for OO.org, but verified in LibO)
7. #37128 - Writer saves text alignment of RTL paragraph not according to the ODF specification

I hoped to have the first two done for 3.5.0, but didn t succeed in getting them fixed. Will keep trying
Filed under: i18n & l10n, Israeli Community, LibreOffice

I gave a presentation on UEFI at LCA 2012 - you can watch it here, with the bonus repeat (and different jokes) here. It's a gentle introduction to UEFI, followed by some discussion of the problems we've faced in dealing with implementation bugs.

The fundamental problem is that UEFI is a lot of code. And I really do mean a lot of code. Ignoring drivers, the x86 Linux kernel is around 30MB of code. A comparable subset of the UEFI tree is around 35MB. UEFI is of a comparable degree of complexity to the Linux kernel. There's no reason to assume that the people who've actually written this code are significantly more or less competent than an average Linux developer, so all else being equal we'd probably expect somewhere around the same number of bugs per line. Of course, not all else is equal.

Even today, basically all hardware is shipping with BIOS by default. The only people to enable UEFI are enthusiasts. Various machines will pop up all kinds of dire warnings if you try to turn it on. UEFI has had very little real world testing. And it really does show. In the few months I've been working on UEFI I've discovered machines where SetVirtualAddressMap() calls code that has already been (per spec) discarded. I've seen cases where it was possible to create variables, but not to delete them. I've seen a machine that would irreparably corrupt its firmware when you tried to set a variable. I've tripped over code that fails to parse invalid boot variables, bricking the hardware. Many vendors independently fail to report the correct framebuffer stride. And those are just the ones that have ended up on hardware which crosses my desk, which means I haven't even tested the majority of consumer-grade hardware with UEFI.

The problems with UEFI have very little to do with its design or the ability of the people implementing it. After a few years of iterative improvements it stands a good chance of being more reliable and useful than BIOS. Increased commonality of code between vendors is a blessing and a curse - in the long term it means that these bugs can be shaken out, but in the short term it means that at least one hardware-destroying bug has ended up carried by multiple vendors. Right now we're still in the short term and it's likely that we'll find yet more UEFI bugs that cause all sorts of problems. The next few years will probably be a bumpy ride.

comments

For my day job I build storage systems. A lot of what I do at present involves caring a lot about how different OSes deal with things like new LUNs being presented from a SCSI target, or errors along a subset of the available paths to a device. It will come as no surprise to you to discover that they all suck (for values of all equal to Linux, Solaris, Windows and VMWare). New LUNs are particularly annoying. I'm in the situation that creation and removal of a LUN is exceptionally easy. Hmmm. Maybe I need to back up here a bit first. SCSI has the concept of a target (think, device, eg hard drive). Each target can present multiple logical units. Each of these is assigned a number - a Logical Unit Number. Most devices (a hard drive, or a CDROM drive) will present a single LUN. A storage array will tend to present multiple LUNs; one for each volume that is exported to the host. At the host level each LUN really just looks like a separate device (for Linux /dev/sda and /dev/sdb may well be separate LUNs on the same array, rather than 2 separate arrays/hard drives, for example. At the block device level you don't care about the difference usually). Anyway. For various reasons I end up adding and removing LUNs quite often. And there are ways for the array to indicate that this has happened to the host (the UNIT ATTENTION/REPORT LUNS DATA CHANGED check condition seems to be favoured these days, as a complete Fibre Channel LIP can be disruptive). What I'd like to happen in that case is the host to pick up the check condition and drop and/or add the devices that have changed. Instead everything wants a manual rescan. rescan-scsi-bus tends to be simplest for Linux. Windows wants a manual refresh in Disk Administrator. VMWare a "Rescan HBAs" from vSphere. Solaris a "devfsadm -C" and possibly a "cfgadm -al" first. And all of these can be temperamental about picking up the changes. We've done a lot about hotplug for the desktop user experience, without doing the same level for the server experience. I appreciate that there are situations that you don't want your server to reconfigure things without being told to, but the current situation can be detrimental (for example Linux multipathing will hold a device open even after it's disappeared and is returning an "INVALID LUN" response; it would be much better if it could cleanly close that device and wait for it to return). Storage is capable of being much more than just a single block device these days, and it's a shame that nothing seems to deal fully with that fact. (Yes, yes, I should write and submit patches, but I appreciate that there's not always a simple answer, nor necessarily an answer that works for all situations automatically. Plus, y'know, not enough hours in the day and I hope you all appreciate I've taken a break from watching BSG to write this.)