Search Results: "tar"

1 February 2012

Richard Hartmann: hello-planet

Hello Planet :) You may have seen my ancient blog posts pop up on the planet when XTaran was nice enough to add me to the planet's aggregation. Not quite sure why ikiwiki/planet decided to do this, but I will try to find out. If you have an idea, which does not involve re-spamming the feed, do let me know. In any case, expect mostly shell-related posts from me. And, as every year, I will be going to FOSDEM.

Dominique Dumont: SDL packaging team update: most SDL packages are now up-to-date

Hello Thanks to the effort of Felix Geyer (debfx) and Manuel Montecelo (mafm), most of SDL packages in Debian unstable are now up-to-date. These guys did a really great job: most of the time, I could upload the packages they prepared without modification. These guys should become DD, they have the skills. Ok, debfx has started the process (congrats) and mafm is still thinking about it Anyway, here the messages for all those who gave up packaging new SDL games for Debian because of stale SDL packages: SDL team is alive, the core SDL packages are now up-to-date, the wait is over. This is true also for Perl SDL games: I ve uploaded yesterday the latest Perl bindings to SDL. Unfortunately, this upload broke some games like frozen-bubble, pangzero and dizzy. But new versions compatible with new SDL perl are available upstream so theses games won t be broken for long. All the best

Nikita Youshchenko: Skype everywhere

For years, I ve been using Jabber for all my IM needs. There is a private jabber server here, it runs several transports, so there was no problem to keep connectivity with people. until recent. Looks like now everybody is moving to skype. Quite a few of my contacts no longer use anything else. This is starting to cause problems. I feel quite discomfortable about moving to proprietary software that is a questionable quality (1) implementation of a secret protocol. But pressure increases: staying away from skype already does cause difficulties. (1) After recent unclean shutdown of server that hosts desktop sessions for thin clients here, quite a few users have been unable to login to Skype without erasing entire ~/.Skype directory. And there is no working jabber transport for skype. Bad. I m aware that some multi-protocol IM clients can talk skype via api provided by official skype client. This is better than nothing, but still uncomfortable.

Thomas Goirand: Uploading a new Git repo to Alioth

Over the years, I ve always uploaded my local git repo (git clone bare myrep repo.git) as a tarball, which I then uncompress. But I ve been tired of doing that by hand, so I wrote a tiny shell script for it. Nothing fancy. You just do: alioth-new-git openstack then it uploads your current repo to Alioth under /git/openstack, for example. It s a really stupid script, please don t point me to gbp-create-remote-repo, I know it exists, but I prefer my own tiny thing. Here s the (lame) script:
#!/bin/sh
set -e
CWD= pwd 
PKG_NAME= basename $ CWD 
usage ()  
echo "$0 creates a new Git repository out of your current working tree on alioth."
echo "You must be at the root of that local git repository"
echo "$0 will use the current folder as the name for the Alioth git repository."
echo ""
echo "Usage: $0 <destination-project-path-on-alioth>"
echo "example: $0 openstack will create a /git/openstack/$ PKG_NAME .git repository"
echo "note that you will need to have write access on the destination project,"
echo "which means you must be a member of that said project on Alioth."
echo ""
echo "Please send patch/comments to: Thomas Goirand <zigo@debian.org>"
exit 1
 
if [ $# != 1 ] ; then
usage
fi
DEST_PROJECT=$1
# Create the tarball and upload it to Alioth
cd ..
echo "===> Cloning $ PKG_NAME  as bare: $ PKG_NAME .git"
git clone --bare $ PKG_NAME  $ PKG_NAME .git
echo "===> Building tarball: $ PKG_NAME .git.tar.gz"
tar -czf $ PKG_NAME .git.tar.gz $ PKG_NAME .git
echo "===> Uploading $ PKG_NAME .git.tar.gz to vasks.debian.org"
scp $ PKG_NAME .git.tar.gz vasks.debian.org:
# Uncompress it on Alioth, fix perms and hook
# note that the below block should be put back in a single
# line, but has been broken into multiple lines for readability
# on this blog
ssh vasks.debian.org "cd /git/$ DEST_PROJECT  &&
echo '===> Uncompressing $ PKG_NAME .git.tar.gz in /git/$ DEST_PROJECT ' &&
tar -xzf ~/$ PKG_NAME .git.tar.gz &&
echo '===> Activating update-server-info hook' &&
mv $ PKG_NAME .git/hooks/post-update.sample $ PKG_NAME .git/hooks/post-update &&
cd /git/$ DEST_PROJECT /$ PKG_NAME .git &&
git --bare update-server-info &&
echo '===> Deleting tarball on alioth' &&
rm ~/$ PKG_NAME .git.tar.gz &&
echo '===> Fxing g+w unix permissions' &&
find /git/$ DEST_PROJECT /$ PKG_NAME .git -exec chmod g+w   \\;"
# Clean localy created files
echo "===> Cleaning local bare copy and tarball"
rm $ PKG_NAME .git.tar.gz
rm -rf $ PKG_NAME .git

Cyril Brulebois: dpkg with multiarch

(This page was edited a few times, details are available.) Grab it while it s hot! Thanks to the hard work of dpkg developers and many (generations of) developers, multiarch is becoming a reality. If you want to give it a try, install dpkg from experimental, add a foreign architecture, and start trying to install packages. Example on amd64:
# dpkg --add-architecture i386
# dpkg --print-foreign-architectures
i386
# apt-get update
[ lots of amd64 and i386 Packages files get downloaded ]
# apt-get install mksh:i386
Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
  ed:i386
The following NEW packages will be installed:
  mksh:i386
0 upgraded, 1 newly installed, 0 to remove and 9 not upgraded.
Need to get 414 kB of archives.
After this operation, 707 kB of additional disk space will be used.
Get:1 http://ftp.fr.debian.org/debian/ sid/main mksh i386 40.4-2 [414 kB]
Fetched 414 kB in 0s (664 kB/s)
Selecting previously unselected package mksh.
(Reading database ... 171933 files and directories currently installed.)
Unpacking mksh (from .../archives/mksh_40.4-2_i386.deb) ...
Processing triggers for menu ...
Processing triggers for man-db ...
Setting up mksh (40.4-2) ...
update-alternatives: using /bin/mksh to provide /bin/ksh (ksh) in auto mode.
Processing triggers for menu ...
# mksh
$ ldd $(which mksh)
linux-gate.so.1 =>  (0xf779c000)
libc.so.6 => /lib/i386-linux-gnu/i686/cmov/libc.so.6 (0xf75d6000)
libgcc_s.so.1 => /lib/i386-linux-gnu/libgcc_s.so.1 (0xf75b9000)
/lib/ld-linux.so.2 (0xf779d000)
Of course, installing an i386 mksh package isn t exactly what multiarch is about. Dozens of packages have been patched already to add Multi-Arch fields, but until their (recursive) dependencies have been multiarch-ified, foreign packages can be uninstallable, as can be seen below, with the usual why is it uninstallable? hunt (shortened output):
# apt-get install psutils:i386
The following packages have unmet dependencies:
 psutils:i386 : Depends: libpaper1:i386 but it is not going to be installed
# apt-get install libpaper1:i386
The following packages have unmet dependencies:
 libpaper1:i386 : Depends: ucf:i386 (>= 0.28) but it is not installable
                  Recommends: libpaper-utils:i386 but it is not going to be installed
# apt-get install ucf:i386
E: Package 'ucf:i386' has no installation candidate
Another example, successful handling of the installation of a foreign package, while it s already installed with the host architecture:
# apt-get install xz-utils:i386
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
  liblzma5:i386
Suggested packages:
  xz-lzma:i386
The following packages will be REMOVED:
  xz-utils
The following NEW packages will be installed:
  liblzma5:i386 xz-utils:i386
0 upgraded, 2 newly installed, 1 to remove and 9 not upgraded.
Need to get 440 kB of archives.
After this operation, 410 kB of additional disk space will be used.
Do you want to continue [Y/n]?
Get:1 http://ftp.fr.debian.org/debian/ sid/main liblzma5 i386 5.1.1alpha+20110809-3 [205 kB]
Get:2 http://ftp.fr.debian.org/debian/ sid/main xz-utils i386 5.1.1alpha+20110809-3 [235 kB]
Fetched 440 kB in 0s (478 kB/s)
dpkg: xz-utils: dependency problems, but removing anyway as you requested:
 dpkg depends on xz-utils.
 xz-lzma depends on xz-utils.
 dpkg-dev depends on xz-utils.
(Reading database ... 171952 files and directories currently installed.)
Removing xz-utils ...
Processing triggers for man-db ...
Selecting previously unselected package liblzma5:i386.
(Reading database ... 171908 files and directories currently installed.)
Unpacking liblzma5:i386 (from .../liblzma5_5.1.1alpha+20110809-3_i386.deb) ...
Selecting previously unselected package xz-utils.
Unpacking xz-utils (from .../xz-utils_5.1.1alpha+20110809-3_i386.deb) ...
Processing triggers for man-db ...
Setting up liblzma5:i386 (5.1.1alpha+20110809-3) ...
Setting up xz-utils (5.1.1alpha+20110809-3) ...
# dpkg -l xz-utils xz-utils:i386 'liblzma*:*'
[   ]
un  xz-utils            <none>
ii  xz-utils            5.1.1alpha+20110809-3
ii  liblzma5:amd64      5.1.1alpha+20110809-3
ii  liblzma5:i386       5.1.1alpha+20110809-3
# ldd $(which xz)
    linux-gate.so.1 =>  (0xf776b000)
    liblzma.so.5 => /usr/lib/i386-linux-gnu/liblzma.so.5 (0xf7724000)
    librt.so.1 => /lib/i386-linux-gnu/i686/cmov/librt.so.1 (0xf771b000)
    libpthread.so.0 => /lib/i386-linux-gnu/i686/cmov/libpthread.so.0 (0xf7701000)
    libc.so.6 => /lib/i386-linux-gnu/i686/cmov/libc.so.6 (0xf75a4000)
    /lib/ld-linux.so.2 (0xf776c000)
# zgrep -a '_("Activities")' gnome-shell-3.2.2.1.tar.xz
        this._label = new St.Label(  text: _("Activities")  );
zgrep is a shell script, but it calls xz, which is from the i386 package, and everything runs just fine. Running it through strace -f -e '', I discovered those messages, which I had never seen before:
[ Process PID=8900 runs in 32 bit mode. ]
[ Process PID=8899 runs in 64 bit mode. ]
[ Process PID=8900 runs in 32 bit mode. ]
[ Process PID=8899 runs in 64 bit mode. ]
 
What s next? We re late! It s time to check what happens with that dpkg package, report bugs, and convert more libraries! Please think of the kittens, and coordinate with the release team to make sure you don t delay a transition when uploading a shiny, multiarch-ified package. In a nutshell, if you received a patch from Steve Langasek or Riku Voipio, it s a good indication your package will be helpful quickly when it s multiarchified. Since zlib is directly depended on by 2000+ packages, #569697 was pinged right after the dpkg upload; but many other packages will need patches and heavy testing. Hurry up, the freeze is coming, we need to shake it up as soon as possible!

31 January 2012

Richard Hartmann: first post

Here, there be blog posts. For now, this blog is more or less a placeholder. In the past, I created a site to share thoughts about life, the universe and everything. Said site has never been picked up by archive.org and a good thing also; the HTML was horrible. Then, this new craze of "weblogs" arrived and I decided to go against the mainstream and not start another instance of those. As blogs and microblogs have become a useful addition to partake in some interesting discussions, I decided to give the whole thing another spin. All that being said, I do not expect to write a lot. You have been warned. A good place to start are the full lists of all posts and comments, respectively.

Olivier Berger: ADMS.F/OSS : standardizing meta-data for software description in forges or software catalogues

Maybe this could be of interest to a few of my readers who may have missed the announcement, in particular for ones related to forges which will be deployed for private administrations in Europe. The recently stared ADMS.F/OSS project is described as :
ADMS.F/OSS is an XML and RDF vocabulary to describe software, in
particular free and open-source software (F/OSS), making it possible to
more easily search and discover software. The ADMS.F/OSS specification is still under development.
It is developped in the frame of an EC (European Community) programme for interoperability between public administrations (more on the page above). I ll try and participate to the working group, bringing in some feedback from the efforts on similar issues conducted during the (now over) COCLICO project. Hope this helps.

Rapha&#235;l Hertzog: Contributing to the translation of Debian

If you re not into packaging and if you asked how you could help Debian, someone probably suggested that you help to translate it. It s true that translating Debian is essential if we want to make Debian available to everybody on the world. There are many persons who are stuck as soon as they get a message in English, so it s important to aim for 100% coverage in terms of localization. Some vocabulary: localization vs internationalization Internationalization (i18n) is the work that makes it possible to translate messages in a given application. Localization (l10n) is the work of translating messages of said application. So as a translator, you ll be doing localization but some knowledge of internationalization is still useful because it will define how you re supposed to provide the translations. We ll come back to that later. Join your localization team Usually the translation work is shared among multiple translators within a localization team. Check out the Debian International page on www.debian.org to find out instructions for translators for each language. Many teams have a debian-l10n-*@lists.debian.org mailing list used for coordination, feel free to ask questions on those lists when you start (but make sure that you have read the relevant documentation before). Each team has its own workflow, so observe for a while to get used to what s happening before asking your first questions. What is there to translate? The translation of most of the software provided by Debian is not handled by Debian. The Debian translation teams only handle the translation of: Now before contributing to your first translation, I have to come back to internationalization to teach you a few things. In the above list, the projects marked with (*) do use PO files for their translation and the next sections will explain you how to work with those files. Introduction to Gettext The free software community has mostly standardized on a single internationalization infrastructure known as Gettext. With this tool, you re provided a POT file which contains all the translatable strings. It looks like this:
# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR Software in the Public Interest, Inc.
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR <email>, YEAR.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: dpkg 1.16.1\n"
"Report-Msgid-Bugs-To: debian-dpkg@lists.debian.org\n"
"POT-Creation-Date: 2011-09-23 03:37+0200\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <email>\n"
"Language-Team: LANGUAGE <ll>\n"
"Language: \n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=INTEGER; plural=EXPRESSION;\n"
#: lib/dpkg/ar.c:66
#, c-format
msgid "invalid character '%c' in archive '%.250s' member '%.16s' size"
msgstr ""
#: lib/dpkg/ar.c:81 lib/dpkg/ar.c:97 lib/dpkg/ar.c:108 lib/dpkg/ar.c:112
#: lib/dpkg/ar.c:134 utils/update-alternatives.c:1154
#, c-format
msgid "unable to write file '%s'"
msgstr ""
[ ]
The lines starting with #: are comments that indicate the source files where the (English) string is used. This can be useful if you want check the source to have more information about how the string is used. The lines starting with #, contain flags that can be important. If the fuzzy flag is set, the translated string is not used because it must be updated (or at least verified) since the original string evolved. The c-format flags indicates that the string must be a C format string, this has some implications in what s allowed in the string (in particular when it embeds conversion specifier for arguments submitted to printf-like functions). Another thing to note is that the translation of the empty string is used to store some meta-information about the translation itself. Contributing a translation as a PO file When you start a new translation, you copy that POT file to create a PO file for your own language (eg. fr.po for the French language). You replace some template values (identified with the upper case words in the POT file) and you replace all the empty strings on msgstr lines with the translation of the string that appears in the previous msgid line. The result could be something like this:
# translation of fr.po to French
# Messages fran ais pour dpkg (Linux-GNU Debian).
msgid ""
msgstr ""
"Project-Id-Version: fr\n"
"Report-Msgid-Bugs-To: debian-dpkg@lists.debian.org\n"
"POT-Creation-Date: 2011-09-23 03:37+0200\n"
"PO-Revision-Date: 2012-01-16 07:57+0100\n"
"Last-Translator: Christian Perrier <bubulle>\n"
"Language-Team: French <debian-l10n-french>\n"
"Language: fr\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: Plural-Forms: nplurals=2; plural=n>1;\n"
"X-Generator: Lokalize 1.2\n"
#: lib/dpkg/ar.c:66
#, c-format
msgid "invalid character '%c' in archive '%.250s' member '%.16s' size"
msgstr "caract re invalide  %1$c  dans la taille du membre  %3$.16s  de l'archive  %2$.250s "
#: lib/dpkg/ar.c:81 lib/dpkg/ar.c:97 lib/dpkg/ar.c:108 lib/dpkg/ar.c:112
#: lib/dpkg/ar.c:134 utils/update-alternatives.c:1154
#, c-format
msgid "unable to write file '%s'"
msgstr "impossible d' crire le fichier  %s "
[ ]
If there s already a PO file for your language, there might still work to do: there might be strings that have not yet been translated and there might be fuzzy strings which have to be updated strings which were already translated but where the original string has been modified. There are software that can assist you to edit PO files: poedit, virtaal, lokalize, gtranslator. There are also special extensions for vim (packaged in vim-scripts) and for Emacs. Submit the translation for inclusion Once you have a complete PO file, you should submit it for inclusion. Sometimes you will have been granted commit rights to the source code repository so that you can include your translation by yourself. In the other cases, you should submit your translation with a bug report tagged l10n and someone else will include your work in the next release. Depending on the team, the workflow might require a review before the submission. In that case, you usually have to send a call for review on the coordination mailing list. Go ahead! Hopefully those explanations will be enough to get you started. There are many other things to learn but it s good to learn while practicing For example, can you find out why the French translation above changed %c in %1$c ?
Do you want to read more tutorials like this one? Click here to subscribe to my free newsletter, you can opt to receive future articles by email.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Paul Tagliamonte: python-sunlight (or: get at some awesome US Political data programmatically)

I ve spent a few days during work, after work and on the weekend working on python-sunlight, a unified API implementation of a few Sunlight services. This is very unstable, and not released yet (so please don t rely on it yet), but it will be shortly. Be sure to sign up for an API Key, and dump the key to ~/.sunlight.key there s a simple script to help with some of this in bin/, but nothing solid yet. I do, however, encourage you to use it and play around with it, and report your bugs. Contributions (in the form of code) are also very welcome, so please do fork the project and play around with it. Just to give everyone a taste of how cool this is this will pull up a list of twitter IDs of people who mention free market more then anyone else according to the congressional record:
from sunlight import capitolwords
from sunlight import congress
for person in capitolwords.phrases_by_entity(
    "legislator",
    phrase="free market",
    sort="count",
):
    n = congress.legislators( bioguide_id=person['legislator'],
        all_legislators="true" )
    if len(n) >= 1:
        n = n[0]
        if n['twitter_id'] != "":
            print n['twitter_id']
And the output:
RepRonPaul
stevekingia
SenSherrodBrown
JacksonLeeTX18
SenatorLeahy
ChuckGrassley
SenJohnMcCain
OrrinHatch
DanaRohrabacher
JudgeTedPoe
McConnellPress
edtowns
SenatorDurbin
SenatorSessions
SenatorHarkin
SenChrisDodd
russfeingold
RosLehtinen
sencarllevin
senjonkyl
Dennis_Kucinich
senatorsanders
SenatorReid
RepDanBurton
RepMikePence
SenSamBrownback
joebiden
FrankPallone
ToddAkin
senatorboxer
RepTrentFranks
JohnCornyn
Have fun! Show off what cha got, and please let me know if you do something cool!

30 January 2012

Matthew Garrett: The ongoing fight against GPL enforcement

GPL enforcement is a surprisingly difficult task. It's not just a matter of identifying an infringement - you need to make sure you have a copyright holder on your side, spend some money sending letters asking people to come into compliance, spend more money initiating a suit, spend even more money encouraging people to settle, spend yet more money actually taking them to court and then maybe, at the end, you have some source code. One of the (tiny) number of groups involved in doing this is the Software Freedom Conservancy, a non-profit organisation that offers various services to free software projects. One of their notable activities is enforcing the license of Busybox, a GPLed multi-purpose application that's used in many embedded Linux environments. And this is where things get interesting

GPLv2 (the license covering the relevant code) contains the following as part of section 4:

Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License.

There's some argument over what this means, precisely, but GPLv3 adds the following paragraph:

However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation

which tends to support the assertion that, under V2, once the license is terminated you've lost it forever. That gives the SFC a lever. If a vendor is shipping products using Busybox, and is found to be in violation, this interpretation of GPLv2 means that they have no license to ship Busybox again until the copyright holders (or their agents) grant them another. This is a bit of a problem if your entire stock consists of devices running Busybox. The SFC will grant a new license, but on one condition - not only must you provide the source code to Busybox, you must provide the source code to all other works on the device that require source distribution.

The outcome of this is that we've gained access to large bodies of source code that would otherwise have been kept by companies. The SFC have successfully used Busybox to force the source release of many vendor kernels, ensuring that users have the freedoms that the copyright holders granted to them. Everybody wins, with the exception of the violators. And it seems that they're unenthusiastic about that.

A couple of weeks ago, this page appeared on the elinux.org wiki. It's written by an engineer at Sony, and it's calling for contributions to rewriting Busybox. This would be entirely reasonable if it were for technical reasons, but it's not - it's explicitly stated that companies are afraid that Busybox copyright holders may force them to comply with the licenses of software they ship. If you ship this Busybox replacement instead of the original Busybox you'll be safe from the SFC. You'll be able to violate licenses with impunity.

What can we do? The real problem here is that the SFC's reliance on Busybox means that they're only able to target infringers who use that Busybox code. No significant kernel copyright holders have so far offered to allow the SFC to enforce their copyrights, with the result that enforcement action will grind to a halt as vendors move over to this Busybox replacement. So, if you hold copyright over any part of the Linux kernel, I'd urge you to get in touch with them. The alternative is a strangely ironic world where Sony are simultaneously funding lobbying for copyright enforcement against individuals and tools to help large corporations infringe at will. I'm not enthusiastic about that.

comment count unavailable comments

Jordi Mallach: GNOME Shell 3.2 in wheezy: a retrospective

When you read this, GNOME Shell 3.2 will (hopefully!) have finally transitioned to Debian s testing suite. Planet GNOME readers might think Debian now has outdated versions of software even in their development versions, or the distribution s development marches at glacial pace. Wheezy GNOME users will finally have a Shell that matches the rest of their GNOME components, something that works with the Shell extensions website and much less problems and limitations compared to 3.0.2. The reality is that GNOME 3.2 s packaging was quite ready back when it was released in late September, but a number of not-so-desirable situations held GNOME Shell from transitioning to testing until today, four months later. So, what happened? TL;DR: transitioning from GNOME 2 GNOME 3 is not so easy if you want to keep testing in a sane state, and when you need to deal with dozens of indirectly related packages, for more than 10 architectures but it shouldn t take nearly a full year, either Let s go back to the last months of 2010. Debian squeeze is in very deep freeze, and the release team and many Debian developers are focusing on squashing as many release critical bugs as they can, in order to make Debian 6.0 the great release it ended up being. The GNOME project has recently delayed the big launch of GNOME 3.0 again, until March 2011; Debian has already settled on GNOME 2.28 for its release, although it will end up cherry-picking many updates from the 2.30 release modules. With most of the stabilization work being done, many Debian GNOME team members were at that time working on packaging very early versions of what would end up being GNOME 3.0 technology: GTK+3.0, GNOME Shell, Mutter and some brave users even tried to use it via the experimental archive. On February 6th, Debian 6.0 was released, and soon after, on April 6th, GNOME made a huge step forward with the much anticipated release of GNOME 3.0. At that time, Debian developers were busy breaking unstable as much as they could, as it s tradition on the weeks following a major release, and the Debian GNOME team was able to start moving some GNOME 3.0 libraries (those which were parallel-installable with their GTK+2.0 versions) to unstable. However, moving the bulk of GNOME 3.0 to unstable wasn t so easy. When you start doing that, you need to be sure you re ready to have all affected packages in a transitionable state as soon as possible, to minimise the chances of blocking transitions of unrelated packages via the dependencies they pick up with rebuilds. All the packages involved in a transition need to be ready to go in the same testing run , for all supported architectures. When you re dealing with dozens of GNOME source packages at the same time, many of which introduce new libraries, or worse, introduce incompatible APIs that affect many more unrelated packages, things get hairy, and you need a plan. So, Joss outlined what a sane approach to this monster transition could look like. The amount of work to do was what we call fun on #debian-gnome. In a nutshell, we had to deal with quite a few transitions, starting with having a newer version of libnotify in unstable, and a pre-requisite for that was making sure all the packages using libnotify1 were ready to use the source-incompatible libnotify4, and this meant preparing patches and NMUs for many of our packages, as well as many others not under our control. Before starting a controlled transition like this one, we had to get an ACK from the release team, who was busy enough handling other huge transitions like Perl 5.12, so by the time we got our own slot, we were well into Summer. With libnotify done in August, it was time to get our hands dirty with more exciting stuff, like getting Nautilus in testing. This meant bumping a soname and requiring all packages providing Nautilus extensions to migrate to GTK+3.0, or drop the extension entirely, as you can t mix GTK+2.0 and GTK+3.0 symbols in the same process. However, in GNOME 3.0, automounting code had moved from Nautilus to gnome-settings-daemon, so in order to not break filesystem automounting in testing for an unreasonable amount of time, both Nautilus and g-s-d needed to go in at the same time. The fun thing is that g-s-d dragged glib2.0, gvfs, gnome-control-center, gdm3, gnome-media, gnome-session and gnome-panel into the equation, so this transition needed extra planning and a lot more work than initially expected: migrating all nautilus extensions, plus ensuring all Panel applets had migrated to GTK+3.0 and the new libpanel-applet-4 interface. In short, this was the monster transition we were trying to avoid. By the time all this mess was sorted out, GNOME 3.2 had been released, and for what users said, it was a lot better than 3.0. We still had no more than a few bits and pieces of 3.0 in testing, and we were working hard to get 3.0 in wheezy. With all the excitement around 3.2, at times it was difficult to explain outsiders why we were beating a dead 3.0 horse Going back to our huge transition, it was just a matter of time before all the packages would be built and be ready to enter, on the same run, in testing. A few weeks later, in early November and after several rounds of mass-bug-filings, fixing unrelated FTBFS, many NMUs, package removal requests and dealing with any possible problem that could block our transition, everything seemed to be set, and our release team magicians had everything in place for the big magic to happen. However, our first clash with the rest of Debian happened a few hours before our victory, in the form of an unannounced ruby-gnome2 upload which resetted the count for everyone. It was fun to see the release team trying all sorts of black magic in an attempt to mitigate the damage. Fortunately, after a few tries they managed to fool britney (the script that handles package transitions from unstable to testing) somehow, and the hardest part of the job was done with just one day of delay. At last, the core of GNOME 3 was in testing, and testing users found soon after. The rest of the week saw a cascade of hate posts against GNOME 3 in Planet Debian, and personally I didn t find that especially motivating to keep on working on the rest of GNOME bits. With experimental clear of GNOME 3.0 stuff, we finally were able to focus on packaging whatever GNOME 3.2 components were not already done, and preparing for what should be a plain simple transition of GNOME 3.0 to 3.2. After our share of wait for a transition slot, as Perl 5.14, ICU and OpenSSL were in the line before us, and after dealing with a minor tracker 0.12 transition, we were ready for our next episode: evolution-data-server. At first sight, we thought this would be a lot easier, but it still got a bit hairy due to evo-data-server massive soname bumps. We were given our slot just before Christmas, after a few weeks of wait for others to finish their migration rounds, and most of the pack entered wheezy a few days before the new year. No rejoicing, though, as GNOME Shell 3.2 didn t make it. First, we discovered it was FTBFS on kFreeBSD architectures, as NetworkManager had been promoted from optional to required, for apparently no good reason, leaving the BSD world in the cold, including our exotic GNU/kFreeBSD architectures. Now, let s clarify that I m a supporter of the Debian kFreeBSD architectures and was really happy to see it accepted as a technology preview in squeeze. However, as you know, GNOME Shell currently requires hardware acceleration to run, a requirement hardly met in kFreeBSD, unless you re using a DRI1 X driver. We seriously doubted anyone had ever ran a GNOME 3 session on kfreebsd-*. However, if it didn t build, it was a blocker bug for GNOME Shell. We considered creating different meta-packages for kFreeBSD architectures, to conclude it d be a mess, so our awesome Michael Biebl ended up cooking up a patch that restored the ability to build the Shell without NetworkManager support. With this out of our way, we just needed to upload Michael s fix and watch the buildds do their part of the job. Or maybe not? Enter Iceweasel 9. In parallel, and with incredible bad timing, Iceweasel 9.0 was uploaded to Debian the very same day it was released by Mozilla. Again, it greeted us with a nasty surprise: yet another mozjs API change, which made gjs FTBFS, which meant our kFreeBSD fixes would be unusable until someone who knew Gjs internals well enough bit the bullet and worked around the new API changes. Again, Michael Biebl tried to be our saviour, but unfortunately wasn t able to fix all the problems, so we tried to focus on plan B. Mozilla had released a fork of the mozjs that is included in Firefox, so that embedders would have a bit less of a hard time with these recurrent API changes. This was based on Firefox 4, and was already being packaged by Ubuntu. Gjs would build using this older version just fine, so we just needed to get it in Debian as soon as possible. We just needed to find a sucke^Wvolunteer that would be inclined to maintain the beast. Only after a few weeks we managed to get Chris Coulson, the Ubuntu packager, to maintain the package directly through the Debian archive via package syncs. However, his package had only been auto-compiled in the three Ubuntu architectures, that is amd64, armel and i386. It s late January 2012, and we ve been fighting this war for 10 months. After getting some help from Michael to get the new package in shape for Debian standards, we were excited to sponsor it for Chris. Duh, after a few days in the NEW fridge, it was rejected by the ftp-masters. The license statement was missing quite a few details, so I went ahead and sacrificed a few hours of my copious free time to get this sorted out. A few days later, mozjs was accepted, but the result was horrible. It was very red. mozjs didn t build on half of our targets. Mike Hommey was quick to file a bug and point us to the most obvious fuckups. As he had dealt with this in the past as the Iceweasel maintainer, all of these issues were fixed and patches were ready to be applied verbatim or with minimal changes to our sources. With mozjs finally built successfully (although with severe problems on ia64), we were finally able to rebuild Gjs against it, upload GNOME Shell with our kFreeBSD fixes and wait until today for this mess to be over. Whew. I can t say I ve enjoyed all the stages of this ride. Some bumps on the road were clearly there to test our patience, but it has helped me get back in touch with non-leaf GNOME packaging, which was all I was doing for a while due to being super-busy lately with studies. It also reminds me of the privilege of working side by side with some awesome people, not only Joss, Michael, Sjoerd, Laurent or Gustavo, to name just a few Debian GNOME team members, but also the receptive release team members like Julien or Cyril, and NEW-processing record-breaking ftp-master Luca. Without them, we might be trying to figure out the Nautilus transition since last Summer. We really hope GNOME 3.4 will be a piece of cake compared to this. ;)

Colin Watson: APT resolver bugs

I've managed to go for eleven years working on Debian and nearly eight on Ubuntu without ever needing to teach myself how APT's resolver works. I get the impression that there's a certain mystique about it in general (alternatively, I'm just the last person to figure this out). Recently, though, I had a couple of Ubuntu upgrade bugs to fix that turned out to be bugs in the resolver, and I thought it might be interesting to walk through the process of fixing them based on the Debug::pkgProblemResolver=true log files. Breakage with Breaks The first was Ubuntu bug #922485 (apt.log). To understand the log, you first need to know that APT makes up to ten passes of the resolver to attempt to fix broken dependencies by upgrading, removing, or holding back packages; if there are still broken packages after this point, it's generally because it's got itself stuck in some kind of loop, and it bails out rather than carrying on forever. The current pass number is shown in each "Investigating" log entry, so they start with "Investigating (0)" and carry on up to at most "Investigating (9)". Any packages that you see still being investigated on the tenth pass are probably something to do with whatever's going wrong. In this case, most packages have been resolved by the end of the fourth pass, but xserver-xorg-core is causing some trouble. (Not a particular surprise, as it's an important package with lots of relationships.) We can see that each breakage is:
Broken xserver-xorg-core:i386 Breaks on xserver-xorg-video-6 [ i386 ] < none > ( none )
This is a Breaks (a relatively new package relationship type introduced a few years ago as a sort of weaker form of Conflicts) on a virtual package, which means that in order to unpack xserver-xorg-core each package that provides xserver-xorg-video-6 must be deconfigured. Much like Conflicts, APT responds to this by upgrading providing packages to versions that don't provide the offending virtual package if it can, and otherwise removing them. We can see it doing just that in the log (some lines omitted):
Investigating (0) xserver-xorg-core [ i386 ] < 2:1.7.6-2ubuntu7.10 -> 2:1.11.3-0ubuntu8 > ( x11 )
  Fixing xserver-xorg-core:i386 via remove of xserver-xorg-video-tseng:i386
Investigating (1) xserver-xorg-core [ i386 ] < 2:1.7.6-2ubuntu7.10 -> 2:1.11.3-0ubuntu8 > ( x11 )
  Fixing xserver-xorg-core:i386 via remove of xserver-xorg-video-i740:i386
Investigating (2) xserver-xorg-core [ i386 ] < 2:1.7.6-2ubuntu7.10 -> 2:1.11.3-0ubuntu8 > ( x11 )
  Fixing xserver-xorg-core:i386 via remove of xserver-xorg-video-nv:i386
OK, so that makes sense - presumably upgrading those packages didn't help at the time. But look at the pass numbers. Rather than just fixing all the packages that provide xserver-xorg-video-6 in a single pass, which it would be perfectly able to do, it only fixes one per pass. This means that if a package Breaks a virtual package which is provided by more than ten installed packages, the resolver will fail to handle that situation. On inspection of the code, this was being handled correctly for Conflicts by carrying on through the list of possible targets for the dependency relation in that case, but apparently when Breaks support was implemented in APT this case was overlooked. The fix is to carry on through the list of possible targets for any "negative" dependency relation, not just Conflicts, and I've filed a patch as Debian bug #657695. My cup overfloweth The second bug I looked at was Ubuntu bug #917173 (apt.log). Just as in the previous case, we can see the resolver "running out of time" by reaching the end of the tenth pass with some dependencies still broken. This one is a lot less obvious, though. The last few entries clearly indicate that the resolver is stuck in a loop:
Investigating (8) dpkg [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( admin )
Broken dpkg:i386 Breaks on dpkg-dev [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( utils ) (< 1.15.8)
  Considering dpkg-dev:i386 29 as a solution to dpkg:i386 7205
  Upgrading dpkg-dev:i386 due to Breaks field in dpkg:i386
Investigating (8) dpkg-dev [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( utils )
Broken dpkg-dev:i386 Depends on libdpkg-perl [ i386 ] < none -> 1.16.1.2ubuntu5 > ( perl ) (= 1.16.1.2ubuntu5)
  Considering libdpkg-perl:i386 12 as a solution to dpkg-dev:i386 29
  Holding Back dpkg-dev:i386 rather than change libdpkg-perl:i386
Investigating (9) dpkg [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( admin )
Broken dpkg:i386 Breaks on dpkg-dev [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( utils ) (< 1.15.8)
  Considering dpkg-dev:i386 29 as a solution to dpkg:i386 7205
  Upgrading dpkg-dev:i386 due to Breaks field in dpkg:i386
Investigating (9) dpkg-dev [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( utils )
Broken dpkg-dev:i386 Depends on libdpkg-perl [ i386 ] < none -> 1.16.1.2ubuntu5 > ( perl ) (= 1.16.1.2ubuntu5)
  Considering libdpkg-perl:i386 12 as a solution to dpkg-dev:i386 29
  Holding Back dpkg-dev:i386 rather than change libdpkg-perl:i386
The new version of dpkg requires upgrading dpkg-dev, but it can't because of something wrong with libdpkg-perl. Following the breadcrumb trail back through the log, we find:
Investigating (1) libdpkg-perl [ i386 ] < none -> 1.16.1.2ubuntu5 > ( perl )
Broken libdpkg-perl:i386 Depends on perl [ i386 ] < 5.10.1-8ubuntu2.1 -> 5.14.2-6ubuntu1 > ( perl )
  Considering perl:i386 1472 as a solution to libdpkg-perl:i386 12
  Holding Back libdpkg-perl:i386 rather than change perl:i386
Investigating (1) perl [ i386 ] < 5.10.1-8ubuntu2.1 -> 5.14.2-6ubuntu1 > ( perl )
Broken perl:i386 Depends on perl-base [ i386 ] < 5.10.1-8ubuntu2.1 -> 5.14.2-6ubuntu1 > ( perl ) (= 5.14.2-6ubuntu1)
  Considering perl-base:i386 5806 as a solution to perl:i386 1472
  Removing perl:i386 rather than change perl-base:i386
Investigating (1) perl-base [ i386 ] < 5.10.1-8ubuntu2.1 -> 5.14.2-6ubuntu1 > ( perl )
Broken perl-base:i386 PreDepends on libc6 [ i386 ] < 2.11.1-0ubuntu7.8 -> 2.13-24ubuntu2 > ( libs ) (>= 2.11)
  Considering libc6:i386 -17473 as a solution to perl-base:i386 5806
  Added libc6:i386 to the remove list
Investigating (0) libc6 [ i386 ] < 2.11.1-0ubuntu7.8 -> 2.13-24ubuntu2 > ( libs )
Broken libc6:i386 Depends on libc-bin [ i386 ] < 2.11.1-0ubuntu7.8 -> 2.13-24ubuntu2 > ( libs ) (= 2.11.1-0ubuntu7.8)
  Considering libc-bin:i386 10358 as a solution to libc6:i386 -17473
  Removing libc6:i386 rather than change libc-bin:i386
So ultimately the problem is something to do with libc6; but what? As Steve Langasek said in the bug, libc6's dependencies have been very carefully structured, and surely we would have seen some hint of it elsewhere if they were wrong. At this point ideally I wanted to break out GDB or at the very least experiment a bit with apt-get, but due to some tedious local problems I hadn't been able to restore the apt-clone state file for this bug onto my system so that I could attack it directly. So I fell back on the last refuge of the frustrated debugger and sat and thought about it for a bit. Eventually I noticed something. The numbers after the package names in the third line of each of these log entries are "scores": roughly, the more important a package is, the higher its score should be. The function that calculates these is pkgProblemResolver::MakeScores() in apt-pkg/algorithms.cc. Reading this, I noticed that the various values added up to make each score are almost all provably positive, for example:
         Scores[I->ID] += abs(OldScores[D.ParentPkg()->ID]);
The only exceptions are an initial -1 or -2 points for Priority: optional or Priority: extra packages respectively, or some values that could theoretically be configured to be negative but weren't in this case. OK. So how come libc6 has such a huge negative score of -17473, when one would normally expect it to be an extremely powerful package with a large positive score? Oh. This is computer programming, not mathematics ... and each score is stored in a signed short, so in a sufficiently large upgrade all those bonus points add up to something larger than 32767 and everything goes haywire. Bingo. Make it an int instead - the number of installed packages is going to be on the order of tens of thousands at most, so it's not as though it'll make a substantial difference to the amount of memory used - and chances are everything will be fine. I've filed a patch as Debian bug #657732. I'd expected this to be a pretty challenging pair of bugs. While I certainly haven't lost any respect for the APT maintainers for dealing with this stuff regularly, it wasn't as bad as I thought. I'd expected to have to figure out how to retune some slightly out-of-balance heuristics and not really know whether I'd broken anything else in the process; but in the end both patches were very straightforward.

Pau Garcia i Quiles: HTML5 for everything?

Apparently HTML5 applications are the best thing after sliced bread. HTML5 is the first platform any mobile vendor supports: iPhone, Android, Windows Phone, BlackBerry, Symbian. All of them. Windows 8 is said to promote HTML5 as the preferred application development solution. I used to look kindly at that. But about a month ago I started to get worried: is HTML5 good for everything? Long-lived applications In military, industrial, warehouse management, medical, etc is not rare that bespoke applications are developed and stay in use for many years (and I really mean many: 10, 20 or even more!) with barely an update. It s not rare that those applications only receive very small updates once very 5 years. Those applications, not Angry Birds, are what keeps the world running: troops know what supplies they can count on, iPhones are manufactured, FedEx is able to deliver your package and your doctor is able to check your health. But now that everybody seems to be moving to HTML5 webapps, what happens when my warehouse management application is a webapp and the additions in the newest browsers make the webapp no longer work? Are vain upgrades the future? Say my webapp is released in 2014 and it works fine with Firefox 14.0 and Chrome 26.0, the newest browsers when I release the application in 2014. Fast-forward to 2020 and Firefox 14.0 and Chrome 26.0 do not even install on Windows 10 computer! What s the solution? Should the customer pay for a huge update and redesign to make it work with Firefox 27.1 and Chrome 41.0 in 2020? A virtual machine with Windows 8 and Firefox 14.0? A portable Mozilla Firefox 14.0 on Windows 10 in 2020 to be able to use that line-of-business application that only requires a small update once or twice every 5 years? How are the virtual machine and/or Portable Firefox 14.0 different from or better than a fat client? What s the advantage? I d say none! Native applications usually do not have that kind of problems because APIs are much more stable. You can still run Win16 applications on Windows 7! You don t believe me? We may soon be developing for 76 browsers! While HTML5 may be fine for applications which are updated very often, it makes me feel very uneasy to see it used in environments where applications will be rarely updated, such as SCADAs, warehouse management, control system, medical records, etc. A solution is needed It looks like that choice of technology is going to make those applications much more expensive in the medium and long term, paying for adaptations to new browsers (sorry, I resist to call update or upgrade to something that adds zero value other than being able to run on a newer browser). Or maybe it s about time to define actual HTML5 profiles . ACID3 seems to be too weak of a profile: two very different browsers may pass ACID3 yet a webapp would work with one browser and fail with the other due to bugs, missing features/added features, etc. Something needs to be done.

29 January 2012

Don Armstrong: Changing Grub2's default boot entry and halting

I have a mythtv box which (when working) records television shows for me. As I'm not interested in the vast majority of shows shown on US television, it spends most of it's time off, waiting for a show that I want to record. This requires using nvram-wakeup, and one of the oddities of my machine's bios is that it wants to be rebooted after setting the nvram. [This is likely due to Debian writing to the RTC after the nvram being updated, but not setting the RTC seems stupid.] After the reboot, the machine should halt, and grub should be configured to start the machine normally once the bios starts. As grub2 now supports named default entries, this is fairly straightforward. We create a menu entry like the following in /etc/grub.d/40_custom:
 menuentry 'halt'  
         set saved_entry=0;
         save_env saved_entry;
         load_env;
         halt;
  
make sure that GRUB_DEFAULT="saved" in /etc/default/grub; and set MythShutdownNvramRestartCmd to /usr/sbin/grub-set-default halt:
 mysql mythdb -e "UPDATE settings SET data='/usr/sbin/grub-set-default halt' WHERE value='MythShutdownNvramRestartCmd'";
and viola, the machine now behaves properly with grub2.

Gregor Herrmann: RC bugs 2012/04

good news: from looking through RC bugs in the BTS, it seems that more & more people are starting to join the RCBW effort!

& here's my usual list for the past week:

Sylvain Beucler: Cross-compile FreeType for Android ARM

If you want to use FreeType in your native C/C++ Android app, you'll need to cross-compile it. The Android system uses FreeType internally but it doesn't expose it to native apps. First, prepare the cross-compiler from the NDK:
/usr/src/android-ndk-r7/build/tools/make-standalone-toolchain.sh \
  --platform=android-9 \
  --install-dir=/usr/src/ndk-standalone-9
PATH=/usr/src/ndk-standalone-9/bin:$PATH
Then use it to cross-compile freetype:
tar xf freetype-2.4.8.tar.bz2
cd freetype-2.4.8/
CFLAGS="-std=gnu99" ./configure --host=arm-linux-androideabi --prefix=/freetype \
  --without-zlib
make
make install DESTDIR=$(pwd)
See this discussion for the --std option. --without-zlib uses the internal copy of zlib inside freetype rather than one independently recompiled for Android. Otherwise you'd get errors such as:
./obj/local/armeabi/libfreetype.a(ftgzip.o): In function  ft_gzip_file_init':
ftgzip.c:(.text+0x3c4): undefined reference to  inflateInit2_'
./obj/local/armeabi/libfreetype.a(ftgzip.o): In function  ft_gzip_file_done':
ftgzip.c:(.text+0x43c): undefined reference to  inflateEnd'
./obj/local/armeabi/libfreetype.a(ftgzip.o): In function  ft_gzip_file_reset':
ftgzip.c:(.text+0x514): undefined reference to  inflateReset'
./obj/local/armeabi/libfreetype.a(ftgzip.o): In function  ft_gzip_file_fill_output':
ftgzip.c:(.text+0x780): undefined reference to  inflate'
Then write an Android.mk file in the new freetype/ directory:
#####
LOCAL_PATH:= $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := freetype
LOCAL_SRC_FILES := lib/libfreetype.a
LOCAL_EXPORT_C_INCLUDES := $(LOCAL_PATH)/include $(LOCAL_PATH)/include/freetype2
include $(PREBUILT_STATIC_LIBRARY)
#####
See docs/STANDALONE-TOOLCHAIN.html and docs/PREBUILTS.html in the NDK for details. The CLEAR_VARS bit is not documented, but is necessary to avoid mixed-up paths; it is used in the native_app_glue NDK module. To use FreeType in your project, edit your Android.mk:
...
LOCAL_STATIC_LIBRARIES := ... freetype
...
$(call import-module,freetype)
You will need to specify the freetype directory location as well, using the NDK_MODULE_PATH variable. For instance, you can copy freetype in your project directory and build using:
    ndk-build NDK_MODULE_PATH=. ...
[Latest version of these instructions can be found at the OpenGL Wikibook]

28 January 2012

Russ Allbery: Review: Effective Java, Second Edition

Review: Effective Java, Second Edition, by Joshua Bloch
Publisher: Addison-Wesley
Copyright: 2008
Printing: October 2009
ISBN: 0-321-35668-3
Format: Trade paperback
Pages: 319
Guy Steele, in the forward to this book, writes:
Programs, unlike spoken sentences and unlike most books and magazines, are likely to be changed over time. It's typically not enough to produce code that operates effectively and is readily understood by other persons; one must also organize the code so that it is easy to modify. There may be ten ways to write code for some task T. Of those ten ways, seven will be awkward, inefficient, or puzzling. Of the other three, which is most likely to be similar to the code needed for the task T in next year's software release?
Teaching this, as well as how to avoid being awkward, inefficient, or puzzling, is the goal of this book. Clearly communicating effective, idiomatic, and maintainable usage to a newcomer to a programming language is one of the hardest types of programming books to write. Books like this are therefore quite scarce. Most introductory texts do try to communicate some degree of basic usage, but they rarely go far beyond the syntax, and when they do that usage is rarely both well-defended and inobvious. Bloch takes the concept quite far indeed, going deep not only into the Java language but also into object-oriented software construction in general. Effective Java is modeled after Effective C++ by Scott Meyers, a book I've not read (due to the lack of need for C++ in my programming life), but which I've heard a great deal about. This means the book is organized into 78 numbered items, each of which provides specific advice and analysis about one area of Java. Examples include item 16, "Favor composition over inheritance," or item 33, "Use EnumMap instead of ordinal indexing." As you can see, they run the gamut from high-level design principles to specific coding techniques. This sort of book demands a lot of the author. Everyone has a coding style, and everyone can make usage recommendations, but the merits or lack thereof of specific recommendations are often only visible with substantial later experience. More than any other type of programming language book, this sort of usage guide must be written by a language expert with years of experience with both good and bad code in the language. This is where Effective Java shines. Joshua Bloch led the design and implementation of significant portions of the Java core libraries at Sun and is currently the chief Java architect at Google, but even without knowing that background, his expertise is obvious. Every item in this book is backed up with specific examples and justification, and Bloch quotes extensively from the Java core library to illustrate both the advantages of the techniques he describes and the problems that result when they're not followed. This is not an introductory book, which is one of the things that makes it so efficient and concise. It's a book aimed at the intermediate or advanced Java programmer and assumes you already know the language and the basic pitfalls. There are only a few items in here that would be obvious to most experienced programmers, and even there Bloch ties them back to specific issues in Java in ways that are illuminating. I would not have expected to learn something new from a chapter on a hoary old problem like avoiding float and double for precise values, but I did: Bloch discusses the available alternatives within Java and their tradeoffs and then makes useful specific recommendations. If, like me, you're an experienced programmer already but relatively new to Java, you still should not read this book first. You need a general introduction to the language and libraries and a few projects under your belt before you can appreciate it. (I personally started with Thinking in Java by Bruce Eckel and it served me well, although on several points of style Bloch disagrees with advice in Eckel's book, and I find Bloch's arguments convincing.) But I think this is one of the best possible choices for your second book on Java, in large part because Bloch will head off bad design and style decisions that you don't realize you're making and catch them before they become entrenched. I'm glad I read it as soon as I knew the language well enough to absorb it, and it's the sort of book that I'm likely to re-read sections of whenever I work on Java code related to those topics. It's not entirely obvious that you should take my advice about this sort of book, since I'm not a Java expert and don't have those years of experience with it. But I've checked the recommendation with other programmers I know who are experts, and I've never heard anything but praise for it. It's also one of the books recommended in Coders at Work, and Bloch is one of the people interviewed there, which carries a lot of weight with me. And, apart from that, any long-time programmer who cares about their craft builds an internal sense of aesthetics around what a well-written program should look like and finds themselves recognizing a similar sense in other people's code, even in languages with which they're not familiar. Bloch's recommendations and analysis feel right; one can immediately see how they improve maintainability and robustness, and some of the techniques he shows are elegant and beautiful. This is not a general programming book. It's specifically focused on the Java language, and much of it deals with specific suggestions on how to use Java's core libraries and language features. If you're not going to be writing code in Java, I can't really recommend it. But one of the things I loved about it is that, while talking about Java, Bloch also talks about object-oriented software construction: techniques for extending foreign libraries one does not control, API design, and the proper use of inheritance, among other topics.. That advice is some of the best object-oriented software design advice I've ever read. There isn't enough of it to recommend the book to people with no interest in Java, but this book has even made my C and Perl code better, and has helped me grasp the tradeoff between inheritance and composition in a deeper way than I ever had before. It's a lovely side bonus. If you're writing Java, read this book. If you're learning Java, don't read it first, but read it second, and more than once, or alongside a project where you can apply the advice. It's dense and efficient in the information that it conveys, which means there's more in a couple pages here than in thirty or forty pages of some of the sprawling introductory programming books. I did read it cover to cover, which is one of the better ways to get a sense of Bloch's more general advice on software construction, but you'll hit information overload and will want to return to it piecemeal to fully absorb it. And do get the second edition. I'm sure the first edition is available cheap used, but the additions of enums and generics to the Java language are hugely important and provide some of the most elegant and graceful techniques in the book. Rating: 9 out of 10

Don Armstrong: Using wicd on UCR's WPA network

UC Riverside's wireless network uses WPA-EAP for the encrypted network. [The unencrypted network does a https based browser capture.] Unfortunately, none of the default wicd encryption templates support the precise brand of WPA that the network does, so you have to make your own template. Luckily, wicd makes this fairly simple: Create a new template, say, /etc/wicd/encryption/templates/eap-only, with appropriate contents.
 name = EAP
 author = Don Armstrong
 version = 1
 require identity *Identity passwd *Password 
 -----
 ctrl_interface=/var/run/wpa_supplicant
 network= 
    ssid="$_ESSID"
    key_mgmt=WPA-EAP
    identity="$_IDENTITY"
    password="$_PASSWD"
  
Then tell wicd about this new template by editing /etc/wicd/encryption/templates/active and adding eap-only to the existing list of templates, and restart wicd /etc/init.d/wicd restart. [I'm not sure if restarting wicd is necessary, but it shouldn't hurt.] Finally, configure the network using the appropriate wicd interface as usual.

27 January 2012

Jon Dowland: An Early Start

early start
An early start.

Rapha&#235;l Hertzog: People Behind Debian: Josselin Mouette, founder of the Debian GNOME team

Josselin Mouette is one the leaders of the pkg-gnome team, he takes sound technical decisions and doesn t fear writing code to work-around upstream issues. He deserves kudos for the work he has put into packaging GNOME over the years. He can also be very sarcastic (sometimes he even enjoys participating to flamewars on debian lists), and there are quite a few topics where we have long agreed to disagree. But this kind of diversity is also what makes Debian a so interesting place Read on to learn more about the pkg-gnome team, its plans for Wheezy, Josselin s opinion on the GNOME 3 switch, and much more. Raphael: Who are you? Josselin: I am a 31 years old Linux systems engineer. I started in life with physics, which I studied at the ENS Lyon. I started a thesis on experimental and numerical models for optoelectronics, but when it became clear that research was not for me, I abandoned it and accepted a job at the CEA, which holds the largest computing center in Europe. Working on these machines has been the most awesome job ever (except for it being near Paris). After that I worked a bit on system monitoring technologies. I am married, currently living in Lyon, and working for EDF (the French historical electricity company) on scientific workstations using Debian. EDF is using Debian on more than a thousand workstations and holds the fastest Debian supercomputer in the world (200 Tflops), which makes it another obvious place for Debian developers. Raphael: How did you start contributing to Debian? Josselin: I discovered Debian in 1999 while studying at the ENS, which is one of the biggest nests of Debian developers while being a small place, it is producing almost one Debian developer per year on average. After wondering for a while what it could be useful for, hacking on a slink snapshot made me think that it was for, well, everything except for gaming. Later, in 2002, when I was working on optoelectronics computing codes, I started to package them for Debian in order to make them easier to install, for us as well as other labs over the world. I started the NM process, and it was going smoothly but also going to take time. However, at that moment, the frozen-bubble game went out and made quite some buzz. Since I knew a guy who knew the game s developer, he asked me to package it. The package found 3 sponsors in a very short time and was fast-tracked into the archive at a speed that was unseen before. After which the NM process was completed very quickly. At that time, I was a heavy WindowMaker user, but I didn t like the direction the project was taking (actually, I wonder if there was one). GNOME was starting to become attractive, but its packaging in Debian was very ineffective, with many inconsistent packages maintained by people who didn t ever talk to each other some of them didn t speak English, and some of them didn t talk at all. Together with awesome people, among which Jordi Mallach, Gustavo Noronha Silva, JHM Dassen, Ross Burton and S bastien Bacher, we started the GNOME team in 2003, introducing consistent packaging practices, and initiating synchronized uploads. Releasing a completely integrated GNOME 2.8 in sarge was a considerable achievement; proving (together with the Perl team) that a team was the best way to maintain large package sets changed the way people work on Debian.
Proving [ ] that a team was the best way to maintain large package sets changed the way people work on Debian.
Raphael: You re one of the most active contributors of the team which is packaging GNOME for Debian. What would you suggest to a new contributor who would like to help the team? Josselin: There are several ways to contact the team, but the recommended one has always been IRC. We hang on #debian-gnome on the OFTC network, so just come around and ask for us. The real question is what you want to do in the team. Of course, most new volunteers want to help packaging the latest and greatest version of GNOME into unstable as soon as possible, but unless they already have Debian background, this is not the easiest task. Since there are already people working on this, the big packages are usually waiting on dependencies. I used to direct newcomers towards bug triage, but it is a tedious task and I m now convinced that our huge bug backlog will never be dealt with. The most useful thing to do for newcomers now is probably to find a GNOME or GNOME-related package that needs improvement or is lagging behind, and simply try to work on it. You can also come and fix the bugs you find annoying. Find a patch on the GNOME bugzilla, or cook it yourself, propose it, and if it s worthy enough you ll soon get commit access.
Our huge bug backlog will never be dealt with.
At this point I feel worth mentioning that if no one answers in 10 minutes, it doesn t mean that no one will answer in 2 hours, so please stay on the channel after asking. Raphael: There s been some controversy about GNOME 3 and the direction that the project is taking. What s your personal stance on GNOME 3? And what s the position of the pkg-gnome team? Josselin: The controversy is not new to GNOME 3, but the large-scale changes made with it have put it more prominently. The criticism usually boils down to a few categories:
  1. General lack of configurability
  2. Strange design decisions
  3. Red Hat centric development
  4. Hardware requirements
  5. Change resistance
The lack of configuration options has been an ongoing criticism since GNOME 2.0 has decided to rip off most of them. Of course, when the control center was redesigned again for 3.0, there was a surge of horrified exclamations from people who missed their favorite buttons. On this topic, I fully concur with GNOME developers. The configuration option that is useful for you is not necessarily useful for someone else. Of course, sometimes developers go a bit too far, but the general direction is right. At work, we found that only a minority of users actually configure anything on their desktops: they just want something that works to launch their applications. Apple and Google have sold millions of devices by making them the simplest possible and without any configuration. Design decisions are, on the contrary, individual decisions, and each of them, while having reasons behind it, can be questioned. I remember seeing a lot of complaints when the OK and Cancel buttons were reversed in dialog boxes, something that nobody questions anymore. GNOME Shell is full of such changes; some are easy to get accustomed with, some others just make eyebrows raise. The most obvious example is the user menu in GNOME 3.2, which contains an entry to configure your Google account, but no entry to shutdown the computer. Both decisions were taken independently, each of them with (good or bad) reasons, but the result is simply ridiculous. The default configuration in Debian will contain an extension to make it a bit better, but on the whole we don t intend to diverge from the upstream design, on which a lot of good work has been done.
On the whole we don t intend to diverge from the upstream design, on which a lot of good work has been done.
Point 3 is more complex. Red Hat being the company spending the most on GNOME, it is obvious that their employees work on making things work for their distribution. An example is the recurring discussions about relying on system services that are currently only implemented by systemd. Since there is a lot of (mostly unjustified) resistance against systemd in Debian, and since it won t work on kFreeBSD anyway, someone needs to develop an alternative implementation of these services for upstart and sysvinit. Everything is in place for someone else to do the job but it has to be done, and this can be frustrating. Especially since it can also be hard to integrate changes needed for other distributions . Hardware requirements are mostly a consequence of the previous criticism: there s hardware that most distributions just don t want to bother supporting. We ve seen it in squeeze with the introduction of a hard dependency on PulseAudio. The Debian GNOME team (together with the Gentoo maintainers) made this dependency optional, carrying heavy patches, in order to cover the cases where it does not work. Now that it has gained more maturity, making this effort obsolete, the new tendency is to require 3D acceleration. For various reasons, it is not available to everyone . On this matter, the position of the Debian GNOME team has always been to support as much different configurations as possible with reasonable effort. Thanks to efforts from the incredible Vincent Untz, upstream supports a so-called fallback mode , which is the GNOME panel from 2.x with a lot of its bugs fixed. We intend to support this mode for as long as reasonably possible in Debian, possibly even after upstream ends up dropping it. However, other applications are going to require 3D because GStreamer is moving to clutter too, affecting video playback performance on non-accelerated systems . For epiphany this is not a problem; only embedded video will be affected. But for totem, this is a major issue; because of that we will probably keep totem 3.0 in wheezy. Finally, there is a natural human tendency to dislike change (I have it too), and it applies a lot to desktop users habits. Needless to say a change of such a scale as introducing GNOME Shell can trigger reactions. However, I don t think it is reasonable, because of this resistance, to keep gnome-panel 2.x in Debian. This would be a lot of work on obsolete technology, and would prevent the upcoming removal of a lot of deprecated libraries. This time is much better spent improving gnome-panel 3.x in Debian and keeping the fallback mode great. One of the change that was made in Debian was to make it easier to find, being available as GNOME Classic directly from the login manager, instead of having to find it in an obscure configuration panel. In all cases, I would recommend to actually try GNOME Shell for a few hours before ditching it. I had never been accustomed to a new environment as quickly ever before.
In all cases, I would recommend to actually try GNOME Shell for a few hours before ditching it.
Having seen several of my GDM patches reverted without a warning, I know we are not finished with carrying patches in Debian packages.
Scientific workstations are a non-trivial example, since there is a measurable effect of using 3D in the window manager on heavy 3D applications.
On the other hand, on accelerated systems, this feature should end up improving performance a lot. Raphael: What are your plans for Debian Wheezy? Josselin: The first goal of the GNOME team is, of course, to provide again a great desktop environment to work on. For wheezy it will probably be based on GNOME 3.4. There also needs to be some work on package management interfaces. Upstream bases everything on PackageKit, but it is not as featureful as the aptdaemon Ubuntu technology. If I have time, I would also like to improve HTTP proxy support, since currently it is based on a stack of terrible hacks. Raphael: If you could spend all your time on Debian, what would you work on? Josselin: Obviously I would like to make GNOME in Debian even better. That would imply working on underneath dependencies (what we now like to call plumbing) to make sure everything is working great. This would also imply working more as GNOME upstream to make it more suitable for our needs. I would also work on large-scale improvements on the distribution, like conditional recommends which I d love to see implemented , or automatic build-dependency generation. I would also work on the installer to make it better for desktops machines. The idea is to automatically install language packs, or glues between two packages when both packages are installed. Raphael: What s the biggest problem of Debian? Josselin: The obvious answer is the same as the one most people you interviewed before gave: not enough members in core teams. A lot of developers join Debian to work on a small number of pet packages, and don t necessarily want to be involved with existing teams. It is probably still not obvious enough that the primary way to start contributing to Debian is to join an existing team. But if there is one thing that is preventing Debian from gaining more momentum now, it is a completely different one: the too short support timeframe. 3 years is really not enough for corporate users. One year to migrate from one version to another is too short, and it is not possible to skip a release. It is definitely possible to change that with reasonable effort: the long-term support after 3 years doesn t have to cover the same perimeter as the short-term one. For example, we could upgrade the kernel to the version in the current stable release, and stop fixing all non-remote security holes. The important thing is to cover the most basic needs: companies are ready to take the risk of having less support if it allows skipping a version, but not the risk of having no support at all. And even more important is to say that you do something. Red Hat says they support a release for 10 years, but of course after 5 years the supported perimeter is extremely small.
3 years [of support] is really not enough for corporate users.
Long-term support will not magically fix all problems in Debian, but it will bring more corporate users into the picture. And with corporate users come paid Debian developers, who can work on critical pieces of the system. Debian was built on the synergy between individuals and companies, and in recent years perhaps as a reaction against what happened with Ubuntu we ve kind of forgot the latter. A lot of individuals have joined the project, and they are actively working, for example, on shortening the release cycle, which goes against the interest of professionals. We should embrace again such users and developers, and that means adapting to the current needs of larger entities. Raphael: You re the maintainer of python-support, a packaging helper that was competing with python-central. Both helpers are now deprecated in favor of dh_python2. Does this mean that the situation of Python in Debian is now sane? Or are there remaining problems? Josselin: dh_python2 (and the Python3 version, dh_python3) has a sane enough design. It fixes a lot of issues in python-central and also python-support, at the expense of somehow reduced functionality for developers. However, just like the previous tools, it merely works around design mistakes in the Python interpreter. For example it is not possible to split binary modules, pure-Python modules and byte-compiled modules in different directory trees, like Perl does although PEP 3147 introduces a way to do so. There is still no sane and standardized way to deal with module versions. There is no difference made between the module (which is a part of language semantics) and the file containing it (an information which depends on the implementation). Developers heavily rely on introspection features and make assumptions based on the implementation, that make it impossible to work around problems with module files. Such problems are not restricted to Python. Those who fought against Ruby gems could tell even worse stories. While introducing GObject introspection packages in Debian (they can be used in JavaScript and Python to provide modules based on GObject libraries), I was pleased to see a clear distinction between file and module, but I was again struck by the fact you are not forced to declare API versions in your Python/JS code. In all cases, there is no reliable way to detect runtime dependencies in a given Python or JavaScript file, which leaves the maintainer to declare them by hand, and of course, often be wrong about them. Add to that the fact that most errors cannot be detected before runtime. For all these reasons, and while still being fond of Python for scripts and prototyping, I ve become really skeptical of using purely interpreted languages to write real applications. Some GNOME developers are moving away from Python and JavaScript, mostly towards Vala; I can only approve of that move and hope the same happens to other projects. Raphael: Is there someone in Debian that you admire for their contributions? Of course there is the never-sleeping, never-stopping, Michael Biebl who can upload a whole GNOME release in a single week-end. But there are a lot of awesome people who make Debian something that simply works. I could talk about Cyril Brulebois from the X strike force, Julien Cristau from the release team, Sjoerd Simons for his sound advice and work on plumbing, Luca Falavigna who is so fast at processing NEW, to quote only a few of those I work with frequently. And of course, Jordi and Sam for their humor.
Thank you to Josselin for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Note that you can find older interviews on http://wiki.debian.org/PeopleBehindDebian.

Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Google+, Twitter and Facebook .

8 comments Liked this article? Click here. My blog is Flattr-enabled.

Next.

Previous.