Whenever I want to login to a Debian porterbox to figure out some architecture-specific issue I typically do not care which particular host I am going to login to, just what architecture the host is. After discovering that it is not yet easy for the Debian sysadmins (DSA) to add aliases to DNS for this purpose, I whipped up a quick script to grab the relevant data about Debian machines from the Debian LDAP server and work around this in my OpenSSH config. To use the script you should run the script and place the magic comment lines suggested by the script into your ~/.ssh/config file and then run the script again, which will contact the Debian LDAP server using python-ldap, download the relevant information and replace the relevant part of your ~/.ssh/config file with some OpenSSH configuration directives to map Debian architecture names to hostnames. Within just a few seconds you will be able to login to armel, powerpc.port or kfreebsd-amd64.port.debian.org instead of needing to manually look up which servers to login for a particular architecture. Update: paravoid mentioned his dsa-porterbox script (see below) which is much simpler, does not need to modify your OpenSSH configuration and can be run as like ssh $(dsa-porterbox armel head -n1). I'm hoping we can add this and a possible ssh-porterbox to the devscripts package.

#!/bin/sh
test -z $1 && exit 1
exec ldapsearch -LLL -x -h db.debian.org -b ou=hosts,dc=debian,dc=org \
 "(&(purpose=porterbox)(architecture=$1))" hostname   \
 sed -n 's/^hostname: //p'

[Latest version of these instructions can be found at the SFML Wiki] CMake allows your project to be built in various environments, including command-line make, Code::Blocks project files, Eclipse project files, etc. It's used in SFML 2.0. In this tutorial we'll write a simple CMake configuration file with centralized version numbering, and see how to integrate SFML in it. The source files First, create a cmake_modules directory and copy FindSFML.cmake in it. FindXXXX is automatically searched by CMake's find_package command. Then create a main.cpp, for instance:

#include "config.h"
#include <iostream>
#include <SFML/Window.hpp>
using namespace std;
int main(int argc, char* argv[])  
  cout << "Version " << myproject_VERSION_MAJOR << "." << myproject_VERSION_MINOR << endl;
  sf::Window screen(sf::VideoMode(800, 600), "myproject");
  bool running = true;
  while (running)  
    screen.Display();
   
 

We use a config.h file that is built by CMake, let's also create a config.h.in file:

#define myproject_VERSION_MAJOR @myproject_VERSION_MAJOR@
#define myproject_VERSION_MINOR @myproject_VERSION_MINOR@

And place your project license (such as the GNU GPL) in a file named COPYING. CMake configuration Let's describe our project configuration in the CMakeLists.txt file:

cmake_minimum_required(VERSION 2.6)
project(myproject)
# Set version information in a config.h file
set(myproject_VERSION_MAJOR 1)
set(myproject_VERSION_MINOR 0)
configure_file(
  "$ PROJECT_SOURCE_DIR /config.h.in"
  "$ PROJECT_BINARY_DIR /config.h"
  )
include_directories("$ PROJECT_BINARY_DIR ")
# Define sources and executable
set(EXECUTABLE_NAME "myproject")
add_executable($ EXECUTABLE_NAME  main.cpp)
# Detect and add SFML
set(CMAKE_MODULE_PATH "$ CMAKE_SOURCE_DIR /cmake_modules" $ CMAKE_MODULE_PATH )
find_package(SFML 1.6 REQUIRED system window graphics network audio)
target_link_libraries($ EXECUTABLE_NAME  $ SFML_LIBRARIES )
# Install target
install(TARGETS $ EXECUTABLE_NAME  DESTINATION bin)
# CPack packaging
include(InstallRequiredSystemLibraries)
set(CPACK_RESOURCE_FILE_LICENSE "$ CMAKE_SOURCE_DIR /COPYING")
set(CPACK_PACKAGE_VERSION_MAJOR "$ myproject_VERSION_MAJOR ")
set(CPACK_PACKAGE_VERSION_MINOR "$ myproject_VERSION_MINOR ")
include(CPack)

The interesting part is:

# Detect and add SFML
set(CMAKE_MODULE_PATH "$ CMAKE_SOURCE_DIR /cmake_modules" $ CMAKE_MODULE_PATH )
find_package(SFML 1.6 REQUIRED system window graphics network audio)
target_link_libraries($ EXECUTABLE_NAME  $ SFML_LIBRARIES )

• First we specify where we stored the FindSFML.cmake module.
• Then we request CMake to look for it in the system, and search for the specified modules (here I specified them all).
• Last, we tell CMake to link our executable with the SFML libraries that it just found.

Compilation To build with Make and the GCC compiler on the command-line:

cd build/  # a separate directory so we can easily remove all the CMake work files
cmake ..  # generate Makefile's
make

To create Code::Blocks project files:

cmake .. -G "CodeBlocks - Unix Makefiles"

and open project.cbp with Code::Blocks. Installation CMake supports the classic:

make install

It also supports DESTDIR, useful in packaging:

make install DESTDIR= pwd /myworkdir

You can also define CMAKE_INSTALL_PREFIX when invoking cmake to change the default install path (/usr/local under Unix). DESTDIR would still override this setting. Packaging You then can use cpack to make a project release:

# Binary package:
cpack -C CPackConfig.cmake
# Source package:
cpack -C CPackSourceConfig.cmake

You will get a nice myproject-1.0.tar.gz, among others. Further information

• CMake has built-in help, for instance:

$ cmake --help-variable CMAKE_SOURCE_DIR
cmake version 2.8.5
  CMAKE_SOURCE_DIR
       The path to the top level of the source tree.
       This is the full path to the top level of the current CMake source
       tree.  For an in-source build, this would be the same as
       CMAKE_BINARY_DIR.

• Reference for v2.8
• CMake Tutorial

Some days ago, a hacker group, THC, released a denial of service tool for SSL web servers. As stated in its description, the problem is not really new: a complete SSL handshake implies costly cryptographic computations. There are two different aspects in the presented attack:

• The computation cost of an handshake is more important on the server side than on the client side. The advisory explains that a server will require 15 times the processing power of a client. This means a single average workstation could challenge a multi-core high-end server.
• The use of SSL renegotiation allows to trigger hundreds of handshakes in the same TCP connection. Even a client behind a DSL connection can therefore bomb a server with a lot of renegotiation requests.

Mitigation techniques There is no definitive solution to this attack but there exists some workarounds. Since the DoS tool from THC relies heavily on renegotiation, the most obvious one is to disable this mechanism on the server side but we will explore other possibilities.

Disabling SSL renegotiation Tackling the second problem seems easy: just disable SSL renegotiation. It is hardly needed: a server can trigger a renegotiation to ask a client to present a certificate but a client usually does not have any reason to trigger one. Because of a past vulnerability in SSL renegotiation, recent version of Apache and nginx just forbid it, even when the non-vulnerable version is available. openssl s_client can be used to test if SSL renegotiation is really disabled. Sending R on an empty line trigger renegotiation. Here is an example where renegotiation is disabled (despite being advertised as supported):

$ openssl s_client -connect www.luffy.cx:443 -tls1
[...]
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: zlib compression
Expansion: zlib compression
[...]
R
RENEGOTIATING
140675659794088:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake failure:s3_pkt.c:591:

Disabling renegotation is not trivial with OpenSSL. As an example, I have pushed a patch to disable renegotiation in stud, the scalable TLS unwrapping daemon.

Rate limiting SSL handshakes Disabling SSL renegotiation on the client side is not always possible. For example, your web server may be too old to propose such an option. Since those renegotiations should not happen often, a workaround is to limit them. When the flaw was first advertised, F5 Networks provided a way to configure such a limitation with an iRule on their load-balancers. We can do something similar with just Netfilter. We can spot most TCP packets triggering such a renegotiation by looking for encrypted TLS handshake record. They may happen in a regular handshake but in this case, they usually are not at the beginning of the TCP payload. There is no field saying if a TLS record is encrypted or not (TLS is stateful for this purpose). Therefore, we have to use some heuristics. If the handshake type is unknown, we assume that this is an encrypted record. Moreover, renegotiation requests are usually encapsulated in a TCP packet flagged with push .

# Access to TCP payload (if not fragmented)
payload="0 >> 22 & 0x3C @ 12 >> 26 & 0x3C @"
iptables -A LIMIT_RENEGOCIATION \
    -p tcp --dport 443 \
    --tcp-flags SYN,FIN,RST,PSH PSH \
    -m u32 \
    --u32 "$payload 0 >> 8 = 0x160300:0x160303 && $payload 2 & 0xFF = 3:10,17:19,21:255" \
    -m hashlimit \
    --hashlimit-above 5/minute --hashlimit-burst 3 \
    --hashlimit-mode srcip --hashlimit-name ssl-reneg \
    -j DROP

The use of u32 match is a bit difficult to read. The manual page gives some insightful examples. $payload allows to seek for the TCP payload. It only works if there is no fragmentation. Then, we check if we have a handshake (0x16) and if we recognise TLS version (0x0300, 0x0301, 0x0302 or 0x0303). At least, we check if the handshake type is not a known value. There is a risk of false positives but since we use hashlimit, we should be safe. This is not a bullet proof solution: TCP fragmentation would allow an attacker to evade detection. Another equivalent solution would be to use CONNMARK to record the fact the initial handshake has been done and forbid any subsequent handshakes. If you happen to disable SSL renegociation, you can still use some Netfilter rule to limit the number of SSL handshakes by limiting the number of TCP connections from one IP:

iptables -A LIMIT_SSL \
    -p tcp --dport 443 \
    --syn -m state --state NEW \
    -m hashlimit \
    --hashlimit-above 120/minute --hashlimit-burst 20 \
    --hashlimit-mode srcip --hashlimit-name ssl-conn \
    -j DROP

Your servers will still be vulnerable to a large botnet but if there is only a handful of source IP, this rule will work just fine¹. I have made all those solutions available in a single file.

Increasing server-side power processing SSL can easily be scaled up and out. Since SSL performance increases linearly with the number of cores, scaling up can be done by throwing in more CPU or more cores per CPU. Adding expensive SSL accelerators would also do the trick. Scaling out is also relatively easy but you should care about SSL session resume.

Putting more work on the client side In their presentation of the denial of service tool, THC explains:

Establishing a secure SSL connection requires 15 more processing power on the server than on the client.

I don t know where this figure comes from. To check it, I built a small tool to measure CPU time of a client and a server doing 1000 handshakes with various parameters (cipher suites and key sizes). The results are summarized on the following plot: For example, with 2048bit RSA certificates and a cipher suite like AES256-SHA, the server needs 6 times more CPU power than the client. However, if we use DHE-RSA-AES256-SHA instead, the server needs 34% less CPU power. The most efficient cipher suite from the server point of view seems to be something like DHE-DSS-AES256-SHA where the server needs half the power of the client. However, you can t really uses those shiny cipher suites:

1. Some browsers do not support them: they are limited to RSA cipher suites².
2. Using them will increase your regular load a lot. Your servers may collapse with just legitimate traffic.
3. They are expensive for some mobile clients: they need more memory, more processing power and will drain battery faster.

Let s dig a bit more on why the server needs more computational power in the case of RSA. Here is a SSL handshake when using a cipher suite like AES256-SHA: When sending the Client Key Exchange message, the client will encrypt TLS version and 46 random bytes with the public key of the certificate sent by the server in its Certificate message. The server will have to decrypt this message with her private key. Those are the two most expensive operations in the handshake. Encryption and decryption are done with RSA (because of the selected cipher suite). To understand why decryption is more expensive than encryption, let me explain how RSA works. First, the server needs a public and a private key. Here are the main steps to generate them:

1. Pick two random distinct prime numbers p and q , each roughly the same size.
2. Compute n=pq . It is the modulus.
3. Compute \varphi(n)=(p-1)(q-1) .
4. Choose an integer e such that 1<e<\varphi(n) and \gcd(\varphi(n),e) = 1 (i.e. e and \varphi(n) are coprime). It is the public exponent.
5. Compute d=e^ -1 \mod\varphi(n) . It is the private key exponent.

The public key is (n,e) while the private key is (n,d) . A message to be encrypted is first turned into an integer m<n (with some appropriate padding). It is then encrypted to a ciphered message c with the public key and should only be decrypted with the private key:

• c=m^e\mod n (encryption)
• m=c^d\mod n (decryption)

So, why is decryption more expensive? In fact, the key pair is not really generated like I said above. Usually, e is a small fixed prime number with a lot of 0, like 17 (0x11) or 65537 (0x10001) and p and q are choosen such that \varphi(n) is coprime with e . This allows encryption to be fast using exponentiation by squaring. On the other hand, its inverse d is a big number with no special property and therefore, exponentiation is more costly and slow. Instead of computing d from e , it is possible to choose d and compute e . We could choose d to be small and coprime with \varphi(n) and then compute e=d^ -1 \mod\varphi(n) and get blazingly fast decryption. Unfortunately, there are two problems with this:

• Most SSL implementation expects e to be a 32bit integer.
• If d is too small (less than one-quarter as many bits as the modulus n ), it can be computed efficiently from the public key.

Therefore, we cannot use a small private exponent. The best we can do is to choose the public exponent to be e =4294967291 (the biggest prime 32bit number and it contains only one 0). However, there is no change as you can see on our comparative plot. To summarize, no real solution here. You need to allow RSA cipher suites and there is no way to improve the computational ratio between the server and the client with such a cipher suite.

Things get worse Shortly after the release the denial of service tool, Eric Rescorla³ published a good analysis on the impact of such a tool. He asks himself about the efficiency to use renegotiation for such an attack:

What you should be asking at this point is whether a computational DoS attack based on renegotiation is any better for the attacker than a computational DoS attack based on multiple connections. The way we measure this is by the ratio of the work the attacker has to do to the work that the server has to do. I ve never seen any actual measurements here (and the THC guys don t present any), but some back of the envelope calculations suggest that the difference is small. If I want to mount the old, multiple connection attack, I need to incur the following costs:

1. Do the TCP handshake (3 packets)
2. Send the SSL/TLS ClientHello (1 packet). This can be a canned message.
3. Send the SSL/TLS ClientKeyExchange, ChangeCipherSpec, Finished messages (1 packet). These can also be canned.

Note that I don t need to parse any SSL/TLS messages from the server, and I don t need to do any cryptography. I m just going to send the server junk anyway, so I can (for instance) send the same bogus ClientKeyExchange and Finished every time. The server can t find out that they are bogus until it s done the expensive part. So, roughly speaking, this attack consists of sending a bunch of canned packets in order to force the server to do one RSA decryption.

I have written a quick proof of concept of such a tool. To avoid any abuse, it will only work if the server supports NULL-MD5 cipher suite. No sane server in the wild will support such a cipher. You need to configure your web server to support it before using this tool. While Eric explains that there is no need to parse any SSL/TLS messages, I have found that if the key exchange message is sent before the server send the answer, the connection will be aborted. Therefore, I quickly parse the server s answer to check if I can continue. Eric also says a bogus key exchange message can be sent since the server will have to decrypt it before discovering it is bogus. I have choosen to build a valid key exchange message during the first handshake (using the certificate presented by the server) and replay it on subsequent handshakes because I think the server may dismiss the message before the computation is complete (for example, if the size does not match the size of the certificate). With such a tool and 2048bit RSA certificate, a server is using 100 times more processing power than the client. Unfortunately, this means that most solutions, except rate limiting, exposed on this page may just be ineffective.

---

1. However, since this rule relies on source IP to identify the attacker, the risk of false positive is real. You can slow down legitimate proxies, networks NATed behind a single IP, mobile users sharing an IP address or people behind a CGN.
2. Cipher suites supported by all browsers are RC4-MD5, RC4-SHA and 3DES-SHA. Support for DHE-DSS-AES256-SHA requires TLS 1.2 (not supported by any browser).
3. Eric is one of the author of several RFC related to TLS. He knows his stuff.

As you will know, every file on a unixoid system has some meta-data like owner, group and permission bits. This is of course also true for files part of some Debian package. And it is not very surprising that different files should have different permissions. Or even different owners or groups. Which file has which settings is of course for the package maintainer to decide and Debian would not be Debian if there were not ways for the user to give their own preferences and have them preserved. This post is thus about how those settings end up in the package and what is to be observed when doing it. As you will also have heard, a .deb file is simply a tar archive stored as part of an ar archive, as you can verify by unpacking a package manually:

$ ar t reprepro_4.8.1-1_amd64.deb
debian-binary
control.tar.gz
data.tar.gz
$ ar p reprepro_4.8.1-1_amd64.deb data.tar.gz   gunzip   tar -tvvf -
drwxr-xr-x root/root         0 2011-10-10 12:05 ./
drwxr-xr-x root/root         0 2011-10-10 12:05 ./etc/
drwxr-xr-x root/root         0 2011-10-10 12:05 ./etc/bash_completion.d/
-rw-r--r-- root/root     19823 2011-10-10 12:05 ./etc/bash_completion.d/reprepro
drwxr-xr-x root/root         0 2011-10-10 12:05 ./usr/
drwxr-xr-x root/root         0 2011-10-10 12:05 ./usr/share/
--More--

(For unpacking stuff from scripts, you should of course use dpkg-deb --fsys-tarfile instead of ar gunzip. Above example is about the format, not a recipe to unpack files). This already explains how the information is usually encoded in the package: A tar file contains that information for each contained file and dpkg is simply using that information. (As tar stores numeric owner and group information, that limits group and owner information to users and groups with fixed numbers, i.e. 0-99. Other cases will be covered later.) The question for the maintainer is now: Where is the information which file has which owner/group/permissions in the .tar inside the .deb, and the answer is simple: It's taken from the files to be put into the .deb. This means that package tools could simply be implemented first by simply calling tar and there is no imminent need to write you own tar generator. It also means that the maintainer has full control and does not have to learn new descriptive languages or tools to change permissions, but can simply put the usually shell commands into debian/rules. There are some disadvantages, though: A normal user cannot change ownership of files and one has to make sure all files have proper permissions and owners. This means that dpkg-deb -b (or the usually used wrapper dh_builddeb) must be run in some context where you could change the file ownership to root first. This means you either need to be root, or at least to fake being root by using fakeroot. (While this could be considered some ugly workaround, it also means upstream's make install is run believing to be root, which also avoids some -- for a packager -- quite annoying automatisms in upstream build scripts assuming a package is not installed system wide if not installed as root). Another problem are random build host characteristics changing how files are created in the directory later given to dpkg-deb -b. For example an umask which might make all files non-world-readable by default. The usual workaround is to first fix up all those permissions. Most packages use dh_fixperms for this, which also sets executable bits according to some simple rules and has some more special cases so that the overall majority of packages does not need to look at permissions at all. So using some debhelper setup, every special permissions and all owner/group information for owner groups with fixed numbers only needs to be set using the normal command line tools between dh_fixperms and dh_builddeb. Everything else happens automatically. Note that games is a group with fixed gid. So it is not necessary (and usually a bug) to change group-ownership of files withing the package to group games in maintainer scripts (postinst,...). If a user wants to change permissions or ownership of a file, dpkg allows this using the dpkg-statoverride command. This command essentially manages a list of files to get special treatment and ownership and permission information they should get. This way a user can specify that files should have different permissions and this setting is applied if a new version of this file is installed by dpkg. Being a user setting especially means, that packages (that means their maintainer scripts) should not usually use dpkg-statoverride. There are two exceptions, though: Different permissions based on interaction with the user (e.g. asking question with debconf) and dynamically allocated users/groups with dynamic id. In both cases one should note that settings to dpkg-statoverride are settings of the user, so the same care should be given as to files in /etc, especially one should never override something the user has set in there. (I can think of no example where calling dpkg-statoverride --add without dpkg-statoverride --list in some maintainer script is not a serious bug: Either you override user settings or you are using debconf as a registry. Moral To recap, your package is doing something wrong if:

• Your maintainer scripts unconditionally change permissions of files within the package (Those should either be set within the package or you miss user configuration in dpkg-statoverride).
• Your maintainer scripts call dpkg-statoverride --add without having checked first whether the user already has done so with different values.

Many people around me switched to Chrome or Chromium. I also used it for a bit, but I was a bit disappointed about the extensions available. To show why, here's a list of the extensions I've currently installed:

• Adblock Plus: I guess everyone knows this. It sits quitely in the background and removes quite a bunch of eye-distracting stuff from web pages. You see the web differently and are always confused when viewing pages on your smartphone.
• British English Dictionary: Pretty self-explaining. Firefox has an integrated spell checker and that one needs a dictionary.
• BugMeNot: You can right-click on a login field and let it insert BugMeNot data automatically. I don't use it often enough to remember that it's there.
• Certificate Patrol: SSL's X.509 trust model is weak, to say the least. This extension implements the "save on first visit" trust model and warns you if the certificate or the CA of a URL changes.
• Download Statusbar: If you're used from, say, Chromium to see your download progress above the status bar, this extension will give you that. I don't like the separate window, especially with Awesome as my window manager.
• Firebug: Most people know this, too. Very useful when you do web development. You can see the effects of CSS as you type, for instance.
• Flashblock: YouTube has its HTML5 trial, so you don't need Flash for it. Sadly I had to give in recently (for live streaming sites like the German Parliament, go figure) and have it installed for "emergency" cases. But really nobody needs Flash advertisements or other silly Flash animations, so Flashblock will conveniently refuse to load the Flash plugin unless you tell it to.
• German Dictionary: As above.
• Google Translator for Firefox: Translates marked sections on a web page in place. It lets Google guess the source language, so you really only mark the area, click the button and be done.
• Greasemonkey: Modifies web pages in place to make them more sane, using little scripts. I use two currently:
• Better Outlook Web Access (OWA): For odd reasons I'm forced to use OWA 2007 at a company of Windows and Mac users. This makes it a little bit more bearable, by allowing you to save your password and by adding a message preview pane. You really don't want to use OWA Light 2007 without this.
• Tagesschau.de - video tag: A simple script that lets you watch videos on tagesschau.de without resorting to Flash. They ship Theora videos alongside H.264, which is supported by Firefox out of the box.
• Lazarus: Form Recovery: This already saved me many, many times. Sometimes I hit the stupid Thinkpad page back/page forward keys, or the browser crashes or I'm torn away from a page in another way. This extension keeps your form content mostly save, so that you don't need to start from scratch. A must have.
• Live HTTP headers: curl -D usually works. But sometimes you need authentication and cookies and stuff, hence doing it in the browser makes sort of sense.
• Modify Headers: I don't really use it anymore. There was this legend that YouTube lets you watch videos that are blocked in your country (Hello, GEMA, I'm looking at you!) if you provide a X-Forwarded-For header with an IP from another country. Never worked for me.
• Mozilla Archive Format: If you need to archive a web page or want to collect a bunch of pages for offline reading, this is the way to go. You can conveniently select either single pages or several tabs to be stored.
• Perspectives: I'm paranoid, so here's the second SSL certificate check. Perspectives uses a bunch of network notaries hosted on PlanetLab to check if everyone sees the same certificate. Together with Certificate Patrol that means if that you save a self-signed certificate on first visit if the notaries all agree that it's the currently installed one.
• PwdHash: Password re-use on different web sites is bad. There are sites where I'm not very concerned about the strength of my password, but where I don't want to leak my main ones. PwdHash uses the domain name and the password I give as components to a hash function. So if I'm at a computer without access to my saved passwords, I can easily reconstruct the hash. If need be, I can use the JavaScript on the developer's web site to do that.
• RequestPolicy: It's currently installed but I don't think it will stick around much longer. Too many sites are using embedded cross-site requests, which this extension will allow you to review. As an example every Blogger site that's on a custom domain (like my blog) will trigger it. Instead half of your web will have red flags everywhere full of blocked requests. Not that helpful. Interestingly enough it also polices Flash's outgoing web requests. Which breaks even more often than normal web pages.
• Secure Login: This gives you a button (or a keyboard shortcut) that allows you to choose from several logins and then goes on and posts those credentials securely to the page in question, bypassing any JavaScript that might want to see them. You also get auto-login bookmarks for those sites which don't allow you to remain logged-in across browser sessions. (Like OWA.)

If Firefox on Android were quicker to start and faster overall, I might even use it there. But as-is it's not very useful. Sadly this also means that I can't use Firefox Sync on my phone and as I don't use Chrome on my desktop I also can't use Chrome to Phone. So I usually go and build a QR code on my laptop and read that with Android's Barcode Scanner.

Of course I'm actually using Iceweasel and I'm very grateful for Mike Hommey's efforts to track the release channel on mozilla.debian.net.

This is my monthly summary of my Debian related activities. If you re among the people who made a donation to support my work (130.30 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Dpkg work The month started with fixing newly reported bugs to prepare the 1.16.1.1 release:

• #644492: there was a flaw in a change I made to the trigger setup code. This resulted in packages being incorrectly marked as configured while they were only unpacked in a new chroot.
• #642656: dpkg-source s refusal to build when it detects unrecorded changes broke the (mostly unused, except by the lintian test suite apparently) 2.0 source format.
• #644412: the Makefile snippet buildflags.mk did not respect the new maintainer specific environment variables (like DEB_CFLAGS_MAINT_APPEND) because make does not forward environment variable through $(shell ). Fixed that by manually exporting the required variables.
• I also disabled dpkg-buildpackage s output of the build flags since it was confusing several maintainers. dpkg-buildpackage invokes debian/rules and it has no (clean) way to discover the build flags changes that maintainer request by setting the dedicated environment variables in debian/rules. Maintainers expect to see the build flags with the modifications they have requested and not just the default values set by the distribution.

With the help of Guillem, we decided on a proper fix for a race condition sometimes triggered by parallel builds when 2 concurrent dpkg-gencontrol try to update debian/files (see #642608). This ended up requiring a new package (libfile-fcntllock-perl) that the Debian perl team kindly packaged for us. With all this sorted, it was a rather easy fix. Multiarch progress I also spent lots of time on multiarch. I fixed an old bug that requested to support the multi-arch paths in case of cross-building (see #595144), the discussion was not really conclusive on which of the two proposed patches was better so I ended up picking my own patch because it was closer to how we currently deal with cross-building. Then I fixed 2 issues that have been reported on Ubuntu s dpkg. The first one (LP #863675) was rather severe since an installed package ended being disappeared in favor of its foreign counterpart that was removed (but that had some config files left). The second one (LP #853679) only affected dselect users (apparently there are still some!) who had a self-conflicting library (Provides: foo, Conflicts: foo) installed for multiple architectures. But the bulk of the time spent on multiarch has been spent discussing with various parties on how to go forward with multiarch. The release team commented on the schedule of the merge to ensure it makes it into Wheezy, and the Debian project leader also commented on the problems encountered so far. While not the best course of action I could have hoped for, it certainly helped since Guillem started pushing some reviewed commits. Out of the 66 commits that were in my pu/multiarch/full branch one week ago, 20 have been merged in the master branch already. Python-django security update and RC bug Since python-django s maintainer did not manage to prepare the required security updates, I stepped in and prepared version 1.2.3-3+squeeze2 for Squeeze and 1.0.2-1+lenny3 for Lenny. Unfortunately this security update is an example of how an inactive maintainer is likely to result in a severe delay for the release of security updates. Furthermore in this specific case, the security team did not want to release the Squeeze security update until the Lenny one had been investigated (which required some time since upstream no longer supports the version in Lenny) but they did not make this very clear. Later another release critical bug had been filed against the package (#646634) but after investigation, it turned out to be a local configuration problem so I downgraded it. I still forwarded the test suite failure to upstream authors since the test could be enhanced. In any case, co-maintainers for python-django are welcome. I really preferred the situation where I can quietly sit down as backup maintainer WordPress packaging WordPress sounds similar to python-django. I m also only a backup maintainer but Giuseppe has been inactive for many months and I had to step in August because I wanted the new upstream version. I discovered a bit late that I was not subscribed to wordpress bugs and thus the release critical bug #639733 (that I introduced with my new upstream version) went unattended for a rather long time. Once aware, though, I quickly fixed it. I also took the opportunity to start a discussion on debian-devel about how to deal with embedded javascript libraries and proposed a mechanism of opportunistic replacement with symlinks . WordPress is my testbed package for this mechanism, you can check out its debian/dh_linktree that implements the replacement logic. The discussion has not been very interesting but at least I learned that Debian now requires that each source package shipping minified javascript files includes the original files too. It s somewhat of a pain since it s not a license requirement in many cases (many of those libraries are not under the GPL), but just a Debian requirement that many upstreams are not complying with. WordPress is affected and Jakub Wilk thus opened #646729 which is going to be a long-standing RC bug. To give good measures, I spent several hours investigating the case of each javascript file in the WordPress source package and I filed a new ticket on the upstream bugtracker. Dropbox packaging work A few months after the introduction of nautilus-dropbox to Debian and Ubuntu, I can say that the decision to only support the download of dropbox in the postinst has been a mistake. Because of this decision I had to make the postinst fail if the download failed. Even if the error message is relatively clear, this lead to many (mostly automated) bug reports on the Ubuntu side. Various other problems cropped up on top of this (trying to start dropbox while the package was not configured would result in an error because the user did not have the required rights to install the software, reinstalling the package while dropbox was running would result in a failure too, etc.). I have fixed all those issues in the version 0.7.0-2 of the package. Now if the user has to install dropbox, it will use PolicyKit to request the root rights. The postinst will no longer fail if the dropbox download fails since it can be run later by the user. And I fixed the download code to remove the replaced file before unpacking a new file (insead of overwriting the existing file). All this work has been forwarded upstream. The Debian Administrator s Handbook Update I m glad to tell you that the translation will happen because we reached the minimal funding goal on October 22th with the help of 380 supporters. Now the fundraising continues, but this time the goal is the liberation of the resulting book. For this to happen, we need to reach 25000 EUR in the liberation fund. So far we re at 37% of this goal with 9400 EUR in the liberation fund (which means that 59% of the money raised has been put in the liberation fund).

Click here if you want to contribute towards the liberation of this book. With (less than) 27 days left, it s going to be a challenge to meet the goal, but we do like challenges, don t we? Misc work

• I filed #644486 against dh-make so that new packages have proper support of dpkg-buildflags from the start.
• I merged lots of patches from Luca Falavigna in the developers-reference.
• I discussed debtags integration in the PTS with Enrico Zini and Paul Wise.
• I updated publican s packaging for the new upstream version 2.8. I had to write a new patch that I forwarded upstream.
• I filed an upstream bug on hamster-applet because just running hamster-time-tracker no longer brings its window forward.

Thanks See you next month for a new summary of my activities.

2 comments Liked this article? Click here. My blog is Flattr-enabled.

One of the things I liked about Gnome 2 was the ability to run a background 'slideshow' defined in an XML file with a list of background files to give me a change of scenery from time to time. Switching to XFCE4 I can't seem to find a simple way to do that, but the ingredients are all there:

• I can set the background image to a list
• There's a handy-dandy xfdesktop command-line utility I can call to switch desktops

What's missing from the Desktop Settings is a setting for "switching background ever N minutes", which is kind of odd to not have, given the list thing being there - without my script it's only going to change every couple of weeks when I log in, or something. Not nearly often enough for me! There's lots of stuff on the internet saying "Just run xfdesktop --reload in a cron job", but this does not work for me, since cron is running with a different environment, and so xfdesktop doesn't know where the X server is and doesn't have the necessary XAUTHORITY and DISPLAY settings. If it was just DISPLAY that was needed it would be easy enough to set that in the crontab and be done with it - after all it doesn't change very often. XAUTHORITY is harder, since on Debian systems (and presumably others too) it has a random component in the name of a directory which lives in a directory without read permissions. I solved it with this script, which steals those values from the environmnt of xfce4-panel, which will be running already:

#!/bin/sh
#
PANELPID=" /usr/bin/pgrep -U $ LOGNAME  xfce4-panel "
stealEnvironment()  
  tr '\000' '\012' < /proc/$PANELPID/environ   grep -a "^$1="   cut -f2- -d=
 
export DISPLAY=" stealEnvironment DISPLAY "
export XAUTHORITY=" stealEnvironment XAUTHORITY "
/usr/bin/xfdesktop --reload

So now I have my background image switching among my favourite photos again, and my Desktop is effectively back as it was a week ago.

Debian/Unstable Development deb http://www.coker.com.au wheezy selinux The above APT sources.list line has my repository for SE Linux packages that have been uploaded to Unstable and which will eventually go to testing and then the Wheezy release (if they aren t obsoleted first). I have created that repository for people who want to track SE Linux development without waiting for an Unstable mirror to update. In that repository I ve included a new version of policycoreutils that now includes mcstrans and also has support for newer policy such that the latest selinux-policy-default package can be installed. The version that is currently in Testing supports upgrading policy on a running system but doesn t support installing the policy on a system that previously didn t run SE Linux. I have also uploaded SE Linux Policy packages from upstream release 20110726 compared to the previous packages which were from upstream release 20100524. As the numbers imply there is 14 months of upstream policy development which changes many things. Many of the patches from my Squeeze policy packages are not yet incorporated in the policy I have uploaded to Unstable. I won t guarantee that an Unstable system in Enforcing mode will do anything other than boot up and allow you to login via ssh. It s definitely not ready for production but it s also very suitable for development (10 years ago I did a lot of development on SE Linux systems that often denied login access, it wasn t fun). Kyle Moffett submitted a patch for libselinux which dramatically changed the build process. As Manoj (who wrote the previous build scripts) was not contactable I accepted Kyle s patch as provided. Thanks for the patch Kyle, and thanks for all your work over the years Manoj. Anyway the result of these changes should mean that it s easier to bootstrap Debian on a new architecture and easier to support multi-arch but I haven t tested either of these. Squeeze The policy packages from Squeeze can t be compiled on Unstable. The newer policy compilation tool chain is more strict about how some things can be declared and used, thus some policy which was fairly dubious but usable is now invalid. While it wouldn t be difficult to fix those problems I don t plan to do so. There is no good reason for compiling Squeeze policy on Unstable now that I ve uploaded a new upstream release. deb http://www.coker.com.au squeeze selinux I am still developing Squeeze policy and releasing it in the above APT repository. I will also get another policy release in a Squeeze update if possible to smooth the transition to Wheezy the goal is that Squeeze policy will be usable on Wheezy even if it can t be compiled. Also note that the compilation failures only affect the Debian package, it should still be possible to make modules for local use on a Wheezy system with Squeeze policy. MLS On Wednesday I m giving a lecture at my local LUG about MLS on SE Linux. I hope to have a MLS demonstration system available to LUG members by then. Ideally I will have a MLS system running on a virtual server somewhere that s accessible as well as a Xen/KVM image on a USB stick that can be copied by anyone at the meeting. I don t expect to spend much time on any aspect of SE Linux unrelated to MLS for the rest of the week. Version Control I need to change the way that I develop SE Linux packages, particularly the refpolicy source package (source of selinux-policy-default among others). A 20,000 line single patch is difficult to work with! I will have to switch to using quilt, once I get it working well it should save me time on my own development as well as making it easier to send patches upstream. Also I need to setup a public version control system so I can access the source from my workstation, laptop, and netbook. While doing that I might as well make it public so any interested people can help out. Suggestions on what type of VCS to use are welcome. How You Can Help Sorting out the mess that is the refpolicy package, sending patches upstream and migrating to a VCS is a fair bit of work. But there are lots of small parts. Sending patches upstream is a job that could be done in small pieces. Writing new policy is not something to do yet. There s not much point in doing that while I still haven t merged all the patches from Squeeze maybe next week. However I can provide the missing patches to anyone who wants to review them and assist with the merging. I have a virtual server that has some spare capacity. One thing I would like to do is to have some virtual machines running Unstable with various configurations of server software. Then we could track Unstable on those images and use automated testing to ensure that nothing breaks. If anyone wants root access on a virtual server to install their favorite software then let me know. But such software needs to be maintained and tested!

Volker Theile is well known for his work on FreeNAS. But at one point, he decided to give up on it and to restart from scratch but this time based on Debian (instead of FreeBSD). OpenMediaVault is the name of this new Debian based product. Here s how he defines the project:

OpenMediaVault is the next generation network attached storage (NAS) solution based on Debian Linux. It contains services like SSH, (S)FTP, SMB/CIFS, DAAP media server, RSync, BitTorrent client and many more. Thanks to the modular design of the framework it can be enhanced via plugins. OpenMediaVault is primarily designed to be used in home environments or small home offices, but is not limited to those scenarios. It is a simple and easy to use out-of-the-box solution that will allow everyone to install and administrate a Network Attached Storage without deeper knowledge.

Even though all the work was private, he managed to attract an important following, and I must say that I was looking forward to this project. He regularly blogged on his progress, sharing some good-looking video of the resulting product (example here). The first public release (Version 0.2, codenamed Ix) happened on October 17th. I have yet to try it but I took a look on the website. As a Debian developer, I was keen on seeing the source code and how the project was managed. The GPLv3 license is presented as an important feature and I was expecting a well-managed open source project. The fact that it was a private one-man project up to now did not bother me, we re quite used to the scratch your itch kind of start for free software projects. Enough to say is that I have been very disappointed. First you come across a contributor agreement, it s frowned upon by many free software developers. But why not, maybe he bought the argument of Mark Shuttleworth and wants to give it a try. But then I looked at the subversion repository, it s obvious that it s just a dumping ground of files that are managed somewhere else in another repository. A bit like Android which is not developed in the open but released as a whole from time to time. But the worst was yet to be found on the licensing page:

Beside being freely available for personal end-users, System Builders and System Integrators, in general Installers, require a commercial license for OpenMediaVault.

Besides the fact that I m still not sure what this means, I would like to know how he reconciles this requirement with the terms of the GPLv3. It s a shame that a so promising project ends up being a disaster from a free software perspective. Have you tried OpenMediaVault already? If yes, I d be glad to read your thoughts in the comments.

9 comments Liked this article? Click here. My blog is Flattr-enabled.

(live from Mangalore, India, where I had a great time at the MiniDebconf India, Mangalore edition. Full report to come out soon) One more "full 100%" since last publication: Russian. Kudos to Yuri Kozlov for his very longstanding commitment to Debian localization efforts in his language. Today, I sent yet another nagging mail for level 1. If your language is not in the 100% list below, then you should start worrying about what the translator in charge is doing. Wheezy release is planned for "sometime in 2012" and it's never too early to start being the jerk who shakes the tree (Doh, I have 79 translation efforts to handle, remember!). So, I know that some oldtimer translators who always update at the last minute will be grumpy about me sending these nagging mails in advance. Sorry in advance for this, guys (you know who you are...:-)). I'm very fine with last minute updates....but this is not what will stop mefrom nagging the Debian l10n crowd anyway. Full stats are here Status for D-I level 1 (core D-I files):

• 19 languages 100% for all sublevels: bg cs de eo es fa fr gu it ja ko nl pl pt ru ta te th zh_CN
• 1 languages 99%: si
• 1 language 98%: sv
• 1 language 97%: eu
• 2 languages 96%: hi tr
• 5 languages 95%: be bn dz sk uk
• 4 langauges 94%: ast bs ga ro
• 4 languages 93%: hu km lo nb
• 2 languages 92%: id sl
• 5 languages 91%: et mr pa vi zh_TW
• others are 90% or below

Status for D-I level 2 (packages that have localized material that may appear during default installs, such as iso-codes, tasksel, etc.):

• 8 languages 100%: de es fa fr pt ru si zh_CN
• 14 languages 99%: be cs da eo id it ja kk nl sk sl sv th uk
• 4 languages 98%: bg ca eu ro
• 1 language 97%: pt_BR
• 4 langauges 95%: el fi he nb
• 7 languages 94%: ar ast dz gl hr vi zh_TW
• 1 language 93%: is
• 7 languages 92%: bn bs hu ko ne pl tr
• 9 languages 91%: ga gu ka km lt mk mr sr ta
• others are 90% or below

Status for D-I level 3 (packages that have localized material that may appear during non-default installs, such as win32-loader)

• 8 languages 100%: cs de fa fr nl pt ru sv
• 19 languages 99%: be bg bs ca el eo es fi ga gl he is ja ru sk sv th tr zh_CN
• 2 languages 97%: hu uk
• others are 90% or below

So, full 100% completeness (hall of fame) for: Persian (fa), French (fr), Portuguese (pt), Russian

I think that it will come as no surprise to my fellow software engineers if I note that I almost never write new software any more. I maintain, I augment, I refactor, I debug but very, very rarely do I start something new.

Saturday started well, reading two stimulating articles on Planet_Debian. One about GitTogether2011, and the other about the use of Git in the Baserock company. I like a lot the idea of using Git beyond source code management. This article is itself distributed via a network of Git repositories. For the Debian package I develop and that are maintained with Git, I started some time ago to store their build logs in a branch that I called meta. Since sbuild can filter some variable chains in its logs, the command git diff --word-diff=color became a nice tool to inspect the difference between two package builds. Here is for instance the update I made this morning for bedtools: we can see that -D_FORTIFY_SOURCE=2 is not passed anymore, and that consequently the -Wunused-result warnings disappeared. Another advantages of storing build logs is simply to make available information that was not previously: buildd.debian.org contains the build logs for every architecture except the one used by the package maintainer, often one of the most popular architectures, because the uploads combine source and binary packages. This problem will be solved when our archive will automatically rebuild the binary packages uploaded with the source packages, but it will not be entirely solved as it is still possible to upload binary packages alone. In the meta branch of my Git repositories, the logs are kept side-to-side for every architecture. I am unsure if that the best layout, but for the moment I am uncomfortable with having too many parallel branches. I am starting to automate the work with the logs, for instance with that small script to gather them from buildd.debian.org.

#!/bin/bash
BASEURL=buildd.debian.org:/srv/buildd.debian.org/db
PACKAGE=$1
shift
for version in "$*"
do
  scp $BASEURL/$ PACKAGE:0:1 /$PACKAGE/$ version /* .
  for arch in $(ls *bz2   sed 's/_.*//g')
    do bzcat $ arch *bz2 > $ arch .log
    rm $ arch _*_log.bz2;
  done
done
exit

Executive summary Debian should consider:

• less fine-grained software packaging for overall simplification of the system and its development
• less flexibility at the lower levels of the software stack, again for overall simplicity
• reducing friction in the development workflow, especially by making it possible to branch and merge at the whole system level

Introduction I've taken a job with Codethink, as part of a team to develop a new embedded Linux system, called Baserock. Baserock is not a distribution as such, but deals with many of the same issues. I thought it might be interesting to write out my new thoughts related to this, since they might be useful for proper distributions to think about too. I'll hasten to add that many of these thoughts are not originally mine. I shamelessly adopt any idea that I think is good. The core ideas for Baserock come from Rob Taylor, Daniel Silverstone, and Paul Sherwood, my colleagues and bosses at Codethink. At the recent GNOME Summit in Montreal, I was greatly influenced by Colin Walters. This is also not an advertisment for Baserock, but since my of my current thinking come from that project, I'll discuss things in the context of Baserock. Finally, I'm writing this to express my personal opinion and thoughts. I'm not speaking for anyone else, least of all Codethink. On the package abstraction Baserock abandons the idea of packages for all individual programs. In the early 1990s, when Linux was new, and the number of packages in a distribution could be counted on two hands in binary, packages were a great idea. It was feasible to know at least something about every piece of software, and pick the exact set of software to install on a machine. It is still feasible to do so, but only in quite restricted circumstances. For example, picking the packages to install a DNS server, or an NFS server, or a mail server, by hand, without using meta packages or tasks (in Debian terminology), is still quite possible. On embedded devices, there's also usually only a handful of programs installed, and the people doing the install can be expected to understand all of them and decide which ones to install on each specific device. However, those are the exceptions, and they're getting rarer. For most people, manually picking software to install is much too tedious. In Debian, we've realised this a great many years ago, and developed meta packages (whose only purpose is to depend on other packages) and tasks (solves the same problem, but differently). These make it possible for a user to say "I want to have the GNOME desktop environment", and not have to worry about finding every piece that belongs in GNOME, and install that separately. For much of the past decade, computers have had sufficient hard disk space that it is no longer necessary to be quite so picky about what to install. A new cheap laptop will now typically come with at least 250 gigabytes of disk space. An expensive one, with an SSD drive, will have at least 128 gigabytes. A fairly complete desktop install uses less than ten gigabytes, so there's rarely a need to pick and choose between the various components. From a usability point of view, choosing from a list of a dozen or two options is much easier than from a list of thirty five thousand (the number of Debian packages as I write this). This is one reason why Baserock won't be using the traditional package abstraction. Instead, we'll collect programs into larger collections, called strata, which form some kind of logical or functional whole. So there'll be one stratum for the core userland software: init, shell, libc, perhaps a bit more. There'll be another for a development environment, one for a desktop environment, etc. Another, equally important reason, to move beyond packages is the problems caused by the combinatorial explosion of packages and versions. Colin Walters talks about this very well. When every system has a fairly unique set of packages, and versions of them, it becomes much harder to ensure that software works well for everyone, that upgrades work well, and that any problems get solved. When the number of possible package combinations is small, getting the interactions between various packages right is easier, QA has a much easier time to test all upgrade paths, and manual test coverage improves a lot when everyone is testing the same versions. Even debugging gets easier, when everyone can easily run the same versions. Grouping software into bigger collections does reduce flexibility of what gets installed. In some cases this is important: very constrainted embedded devices, for example, still need to be very picky about what software gets installed. However, even for them, the price of flash storage is low enough that it might not matter too much, anymore. The benefit of a simpler overall system may well outweigh the benefit of fine-grained software choice. Everything in git In Baserock, we'll be building everything from source in git. It will not be possible to build anything, unless the source is committed. This will allow us to track, for each binary blob we produce, the precise sources that were used to bulid it. We will also try to achieve something a bit more ambitious: anything that affects any bit in the final system image can be traced to files committed to git. This means tracking also all configuration settings for the build, and the whole build environment, in git. This is important for us so that we can reproduce an image used in the field. When a customer is deploying a specific image, and needs it to be changed, we want to be able to make the change with the minimal changes compared to the previous version of the image. This requires that we can re-create the original image, from source, bit for bit, so that when we make the actual change, only the changes we need to make affect the image. We will make it easy to branch and merge not just individual projects, but the whole system. This will make it easy to do large changes to the system, such as transitioning to a new version of GNOME, or the toolchain. Currently, in Debian, such large changes need to be serialised, so that they do not affect each other. It is easy, for example, for a GNOME transition to be broken by a toolchain transition. Branching and merging has long been considered the best available solution for concurrent development within a project. With Baserock, we want to have that for the whole system. Our build servers will be able to build images for each branch, without requiring massive hardware investment: any software that is shared between branches only gets built once. Launchpad PPAs and similar solutions provide many of the benefits of branching and merging on the system level. However, they're much more work than "git checkout -b gnome-3.4-transition". I believe that the git based approach will make concurrent development much more efficient. Ask me in a year if I was right. Git, git, and only git There are a lot of version control systems in active use. For the sake of simplicity, we'll use only git. When an upstream project uses something else, we'll import their code into git. Luckily, there are tools for that. The import and updates to it will be fully automatic, of course. Git is not my favorite version control system, but it's clearly the winner. Everything else will eventually fade away into obscurity. Or that's what we think. If it turns out that we're wrong about that, we'll switch to something else. However, we do not intend to have to deal with more than one at a time. Life's too short to use all possible tools at the same time. Tracking upstreams closely We will track upstream version control repositories, and we will have an automatic mechanism of building our binaries directly from git. This will, we hope, make it easy to follow closely upstream development, so that when, say, GNOME developers make commits, we want to be able to generate a new system image which includes those changes the same day, if not within minutes, rather than waiting days or weeks or months. This kind of closeness is greatly enhanced by having everything in version control. When upstream commits changes to their version control system, we'll mirror them automatically, and this then triggers a new system image build. When upstream makes changes that do not work, we can easily create a branch from any earlier commit, and build images off that branch. This will, we hope, also make it simpler to make changes, and give them back to upstream. Whenever we change anything, it'll be done in a branch, and we'll have system images available to test the change. So not only will upstream be able to easily get the change from our git repository, they'll also be easily verify, on a running system, that the change fixes the problem. Automatic testing, maybe even test driven development We will be automatically building system images from git commits for Baserock. This will potentially result in a very large number of images. We can't possibly test all of them manually, so we will implement some kind of automatic testing. The details of that are still under discussion. I hope to be able to start adding some test driven development to Baserock systems. In other words, when we are requested to make changes to the system, I want the specification to be provided as executable tests. This will probably be impossible in real life, but I can hope. I've talked about doing the same thing for Debian, but it's much harder to push through such changes in an established, large project. Solving the install, upgrade, and downgrade problems All mainstream Linux distributions are based on packages, and they all, pretty much, do installations and upgrades by unpacking packages onto a running system, and then maybe running some scripts from the packages. This works well for a completely idle system, but not so well on systems that are in active use. Colin Walters again talks about this. For installation of new software, the problem is that someone or something may invoke it before it is fully configured by the package's maintainer script. For example, a web application might unpack in such a way that the web server notices it, and a web browser may request the web app to run before it is configured to connect to the right database. Or a GUI program may unpack a .desktop file before the executable or its data files are unpacked, and a user may notice the program in their menu and start it, resulting in an error. Upgrades suffer from additonal problems. Software that gets upgraded may be running during the upgrade. Should the package manager replace the software's data files with new versions, which may be in a format that the old program does not understand? Or install new plugins that will cause the old version of the program to segfault? If the package manager does that, users may experience turbulence without having put on their seat belts. If it doesn't do that, it can't install the package, or it needs to wait, perhaps for a very long time, for a safe time to do the upgrade. These problems have usually been either ignored, or solved by using package specific hacks. For example, plugins might be stored in a directory that embeds the program's version number, ensuring that the old version won't see the new plugins. Some people would like to apply installs and upgrades only at shutdown or bootup, but that has other problems. None of the hacks solve the downgrade problem. The package managers can replace a package with an older version, and often this works well. However, in many cases, any package maintainer scripts won't be able to deal with downgrades. For example, they might convert data files to a new format or name or location upon upgrades, but won't try to undo that if the package gets downgraded. Given the combinatorial explosion of package versions, it's perhaps just as well that they don't try. For Baserock, we absolutely need to have downgrades. We need to be able to go back to a previous version of the system if an upgrade fails. Traditionally, this has been done by providing a "factory reset", where the current version of the system gets replaced with whatever version was installed in the factory. We want that, but we also want to be able to choose other versions, not just the factory one. If a device is running version X, and upgrades to X+1, but that version turns out to be a dud, we want to be able to go back to X, rather than all the way back to the factory version. The approach we'll be taking with Baserock relies on btrfs and subvolumes and snapshots. Each version of the system will be installed in a separate subvolume, which gets cloned from the previous version, using copy-on-write to conserve space. We'll make the bootloader be able to choose a version of the system to boot, and (waving hands here) add some logic to be able to automatically revert to the previous version when necessary. We expect this to work better and more reliably than the current package based one. Making choices Debian is pretty bad at making choices. Almost always, when faced with a need to choose between alternative solutions for the same problem, we choose all of them. For example, we support pretty much every init implementation, various implementations of /bin/sh, and we even have at least three entirely different kernels. Sometimes this is non-choice is a good thing. Our users may need features that only one of the kernels support, for example. And we certainly need to be able to provide both mysql and postresql, since various software we want to provide to our uses needs one and won't work with the other. At other times, the inability to choose causes trouble. Do we really need to support more than one implemenation of /bin/sh? By supporting both dash and bash for that, we double the load on testing and QA, and introduce yet another variable to deal with into any debugging situation involving shell scripts. Especially for core components of the system, it makes sense to limit the flexibility of users to pick and choose. Combinatorial explosion d j vu. Every binary choice doubles the number of possible combinations that need to be tested and supported and checked during debugging. Flexibility begets complexity, complexity begets problems. This is less of a problem at upper levels of the software stack. At the very top level, it doesn't really matter if there are many choices. If a user can freely choose between vi and Emacs, and this does not add complexity at the system level, since nothing else is affected by that choice. However, if we were to add a choice between glibc, eglibc, and uClibc for the system C library, then everything else in the system needs to be tested three times rather than once. Reducing the friction coefficient for system development Currently, a Debian developer takes upstream code, adds packaging, perhaps adds some patches (using one of several methods), builds a binary package, tests it, uploads it, and waits for the build daemons and the package archive and user-testers to report any problems. That's quite a number of steps to go through for the simple act of adding a new program to Debian, or updating it to a new version. Some of it can be automated, but there's still hoops to jump through. Friction does not prevent you from getting stuff done, but the more friction there is, the more energy you have to spend to get it done. Friction slows down the crucial hack-build-test cycle of software development, and that hurts productivity a lot. Every time a developer has to jump through any hoops, or wait for anything, he slows down. It is, of course, not just a matter of the number of steps. Debian requires a source package to be uploaded with the binary package. Many, if not most, packages in Debian are maintained using version control systems. Having to generate a source package and then wait for it to be uploaded is unnecessary work. The build daemon could get the source from version control directly. With signed commits, this is as safe as uploading a tarball. The above examples are specific to maintaining a single package. The friction that really hurts Debian is the friction of making large-scale changes, or changes that affect many packages. I've already mention the difficulty of making large transitions above. Another case is making policy changes, and then implementing them. An excellent example of that is in Debian is the policy change to use /usr/share/doc for documentation, instead of /usr/doc. This took us many years to do. We are, I think, perhaps a little better at such things now, but even so, it is something that should not take more than a few days to implement, rather than half a decade. On the future of distributions Occasionally, people say things like "distributions are not needed", or that "distributions are an unnecessary buffer between upstream developers and users". Some even claim that there should only be one distribution. I disagree. A common view of a Linux distribution is that it takes some source provided by upstream, compiles that, adds an installer, and gives all of that to the users. This view is too simplistic. The important part of developing a distribution is choosing the upstream projects and their versions wisely, and then integrating them into a whole system that works well. The integration part is particularly important. Many upstreams are not even aware of each other, nor should they need to be, even if their software may need to interact with each other. For example, not every developer of HTTP servers should need to be aware of every web application, or vice versa. (It they had to be, it'd be a combinatorial explosion that'd ruin everything, again.) Instead, someone needs to set a policy of how web apps and web servers interface, what their common interface is, and what files should be put where, for web apps to work out of the box, with minimal fuss for the users. That's part of the integration work that goes into a Linux distribution. For Debian, such decisions are recorded in the Policy Manual and its various sub-policies. Further, distributions provide quality assurance, particularly at the system level. It's not realistic to expect most upstream projects to do that. It's a whole different skillset and approach that is needed to develop a system, rather than just a single component. Distributions also provide user support, security support, longer term support than many upstreams, and port software to a much wider range of architectures and platforms than most upstreams actively care about, have access to, or even know about. In some cases, these are things that can and should be done in collaboration with upstreams; if nothing else, portability fixes should be given back to upstreams. So I do think distributions have a bright future, but the way they're working will need to change.

Maybe you've already heard, that Google is starting to charge a fee for using the Maps API (Google may decide you're an non-profit organization and waive the fees). That is, in my opinion, Google telling you to look for better alternatives. And thanks to the awesome effort of the OpenStreetMaps community, there is such an alternative. Thus today's addition to the mini-tips series will tell you, how you get an OpenStreetMaps-powered map on your website.

1. First you need to determine the coordinates, where you want to center, the map. You can do this with OpenRouteService.org. Just search for the address, right-click on the marker highlighting the result, and set it as e.g. your start point, that'll give you the coordinates in the start field on the left. Alternatively you can just move your pointer over a place on the map and have a look at the lower right corner, where the current coordinates for the pointer position are displayed.
2. Then you need to add a few things to the website, where the card is to be shown:
   • Include the OpenLayers script in the <head> of your webpage:

     <script src="http://www.openlayers.org/api/OpenLayers.js" type="text/javascript" />
     

   • Add a short ECMAScript snippet right after that, which will initialize your map:

     function osmInit()  
       document.getElementById("ol_map").style.display = 'block';
       var lonLat = new OpenLayers.LonLat(13.376096, 52.518598) // Center of the map
       .transform(
         new OpenLayers.Projection("EPSG:4326"),   // transform from WGS 1984
         new OpenLayers.Projection("EPSG:900913")  // to Spherical Mercator Projection
       );
       var map = new OpenLayers.Map("ol_map");
       var osmlayer = new OpenLayers.Layer.OSM();
       map.addLayer(osmlayer);
       map.setCenter(lonLat,
         16 // Zoom level
       );
      
     

     The first line is not needed, but I find it nice to hide the map on default, in case the user has disabled ECMAScript they haven't a large empty area somewhere on the website. The first line changes then the display value from "none" to "block". The rest is, I hope, already explained by the code itself.
   • Now only two more changes are needed, first you need to add onload="osmInit();" to your <body> tag. And afterwards an element (say a <p>) with the ID ol_map somewhere in the website <body>. You should set a width and height for that tag (style is your friend), otherwise the map might be bigger than you like.
3. Done. Enjoy your free (not just as in free beer) map!

You can, of course, customize your map further by adding additional layers and changing the available options (e.g. the zoom level). If you would want to mark a location on your map, you'd need to add something like

var markers = new OpenLayers.Layer.Markers("POI1");
map.addLayer(markers);
markers.addMarker(new OpenLayers.Marker(lonLat));

to your ECMAScript snipped after you added the OpenStreetMaps layer. These three lines would mark the location given by the variable lonLat. For further information, you can check out the OpenStreetMaps wiki and the OpenLayers documentation. In case you wonder, where my coordinates from this example point, you can check. Oh, and don't forget to consider a donation to OpenStreetMaps, so the project can pay the bills for its servers.

This is an update to my last post regarding the Construcci n Colaborativa del Conocimiento book. The book is, as we have repeatedly stated, available online for download Both as a full PDF or chapter by chapter. In the website you will also find videos of all of the conferences held. But holding a printed book in your hands is just a different experience, isn't it? :-) Anyway, I said I would give here an update on how to get your hands on it. The main venue would be through my University's e-store. I recommend it to anybody interested in buying the book in Mexico. The book's list price is MX$300 (around US$27), but it is currently sold at half price I don't know how long will that price be offered. On the other hand, we also uploaded it to the lulu.com self-publishing service. Of course, given I have not seen the printed results, I cannot assure you the resulting product will be of the same quality as the one we got here, but I have a couple of books I have bought at lulu, and their quality is quite acceptable. So, you can also buy it from lulu.com. Note the 20% discount it shows will be permanent That's what I would get as an author, a payment I decided to forefit given we are 11 authors and it would be unfair to collect it all myself. So, the price at lulu.com is US$12.64 plus shipping Very similar to the price at UNAM. Enjoy!

As usual, a brief roundup of what happened in FusionForge land this month.

• 5.1.1 released. Nothing particularly Earth-shattering, mostly an acknowledgement of all the bugfixes that had been committed onto the branch since 5.1 was initially released. There are more already, including a large bunch of fixes to the search system. It's not part of an upstream FusionForge release, but the patches are in Debian unstable (and should trickle to testing in a few days).
• Much work has happened on the database upgrading system. It's not committed yet, because it needs more testing first, but it'll go to trunk, meaning that FusionForge 5.2 and following will (at last) have the same handling of database schema changes whatever the installation method (*.rpm, *.deb or from sources).
• An indirect consequence of that is that we'll use some features of PostgreSQL that are only available starting with PostgreSQL 8.3, so earlier versions will not be supported anymore. Since all major distros have at least 8.3 (more often 8.4 or 9.x), this shouldn't be much of a problem. (Note for CentOS 5 users: the postgresql package is 8.1, but there's a postgresql84 for 8.4.)
• The Coclico project is officially over. It's been a great two years, and we're all happy of the dynamics that have been brought about by it. We're not going to drop the results on the floor, either, and many of the features that were developed as part of Coclico are already being used by production forges.
• The server previously hosting Coclico has been moved to a VM, with the nice side effect that the SVN repository of FusionForge is now accessible on the standard ports even on IPv4 (IPv6 has worked for years ;-).
• A plugin integrating MoinMoinWiki into FusionForge has been pushed to trunk, and it will be part of 5.2.
• Speaking of 5.2: we don't have a release date yet, but we've agreed that the stabilization branch should start in early December, with a feature freeze on the 1st. So if you have a local feature that you'd like to see in upstream FusionForge 5.2, now is the time to bring it into shape and submit it.

Augmented reality is available on all relatively modern smart phones. I ve played with it on my Android phone but it hasn t delivered the benefits that I hoped, there is a game where you can walk through a virtual maze which didn t work for me, and a bunch of programs which show me the position of stars, pizza restaurants, and other things which are cool but not really useful. It has been proven that larger screen size can make a surprising difference in productivity for increasing monitor size. The general concept seems to be that ideally everything you are thinking about at one time should be on the screen at once. I m not aware of any research comparing phones to desktop monitors but it is obvious that some tasks become extremely difficult or nearly impossible when attempted on the tiny screen of a phone. One significant example is coding. One noteworthy thing about coding is that the amount of typing is often quite small when compared to the amount of time spent looking at code, so the lack of good keyboard options on phones isn t always a serious problem. The iPhone 4 has a resolution of 640*960 which seems to be the best available phone resolution (with 480*854 being the highest resolution that is available in many phones). The Dell Streak at 5 inches seemed to have the largest screen in a phone, but they have stopped selling them. It seems that the largest screen available in a phone is about 4.2 inches. Probably the minimum that would be considered usable for development would be a resolution of about 1280*1024 and a screen size of about 14 inches, while opinion will vary a lot about this I think that the vast majority of programmers will agree that the bigger tablet computers and Netbooks (at about 10 inches and something like 1366*768 resolution) are well below the minimum size. It seems to me that a possible solution to this problem involves using augmented reality to provide a virtual desktop that is significantly larger and which has a significantly higher resolution. The advantage of augmented reality over merely scrolling is that it should allow faster and more reliable seeking for the section of virtual desktop that is of interest, and seek speed is probably the bottleneck with small monitors. One problem for this would be turning corners when on public transport, but the camera button could be used to reset the current phone position to be the middle of the viewing area, if the process of resetting the angle is fast enough it wouldn t be a great distraction. I don t think that a mobile phone will ever be a great device for software development and I don t think that the places where a serious computer isn t available are good places to work. But sometimes I get inspiration for tracking down a difficult bug when on the move and it would be really good to be able to read the code immediately. I won t have any time to work on such things myself. I m just publishing the idea in case someone who likes it happens to have a lot of spare time

If you checkout Dropbox s Linux download page, you will see that Debian packages are provided. Up to a few days ago, they only provided packages for Ubuntu and Fedora. I m glad to see that packaging nautilus-dropbox for Debian and being in touch with them due to this led to this result. Another positive outcome is that the version 0.7.0 now ensures the origin of the downloaded binaries with GPG.

7 comments Liked this article? Click here. My blog is Flattr-enabled.

A few years ago I got up on stage and briefly talked about how the Linux community contained far too many people who were willing to engage in entirely inappropriate behaviour, how this discouraged people from getting involved and how we weren't very good at standing up against that sort of behaviour. Despite doing this in front of several hundred people, and despite the video of me doing so then being uploaded to the internet, this got me a sum total of:

• No death threats
• No discussion about any of my physical attributes or lack thereof
• No stalkers
• No accusations that I was selling out the Linux community
• No accusations that I was a traitor to my gender
• No real negative feedback at all[1]

Which is, really, what you'd expect, right? The internet seems intent on telling me otherwise:

Well, she didn't do herself any favors by talking at conferences about women in tech, or setting up a feminist movement. If you wanted to attract abuse, that's a good way to go about it. It should be expected.
(Source)

MikeeUSA is a troll. He has no means to actually harm anyone, and he does it purely for the lulz.

Thus, MikeeUSA trolled a woman, and she took the bait. I just don't get why this is news, I've been trolled before, I don't get a news story.
(Source)

I was going to start a rant about how this behavior is encouraged by the macho men online, but this was just one guy harassing her. "Due to harassment" reads as due to harassment from the community, but she gave in to one idiot. She let him win.
(Source)

The full comment thread has rather more examples. If you stand up and say anything controversial, you should expect abuse. And if you let that abuse change your behaviour in any way, you've let the trolls win.

These attitudes are problematic.

The immediate assumption underlying such advice is that the degree of abuse is related to what you've said, not who you are. I'm reasonably visible in the geek world. I've said a few controversial things. The worst thing that's happened to me has been Ryan Farmer deciding to subscribe me to several thousand mailing lists. Inconvenient, but not really threatening. I haven't, for instance, been sent death threats. Nobody has threatened to rape me. And even if they had done, I wouldn't need to worry too much - there's a rather stronger track record of violent antifeminism being targeted at women than men.

I don't have to worry about this kind of thing. That means I don't get to tell other people that they should have expected it. Nor do I get to tell them that they should ignore it, or that if they don't call the police then they have no grounds to complain. And nor does anyone else.

The trolls don't win because someone decides that getting out of the tech business is more appealing than continuing to face abuse. The trolls win because we consider their behaviour acceptable and then blame the victim for inviting them in the first place. That needs to change.

[1] It was justifiably pointed out that saying all this while standing on stage next to a mostly naked guy wearing a loincloth with a raccoon tail covering his penis may have weakened my message somewhat.

comments

Because of the very annoying new Mozilla release politics (which look like a pissing contest with the similar annoying Google Chrome/Chromium release schedule), Ubuntu kicked out Mozilla XULRunner with its recent release of 11.10 Oneiric. And with XULRunner, Ubuntu also kicked out Conkeror and all other XULRunner reverse dependencies, too. Meh. Sparked by this thread on the Conkeror mailing list, I extended the Debian package s /usr/bin/conkeror wrapper script so that it looks for firefox in the search path, too, if no xulrunner* is found, and added an alternative dependency on firefox versions greater or equal to 3.5, too. From now on, if the wrapper script finds no xulrunner but firefox in the search path, it calls firefox -app instead of xulrunner-$VERSION to start Conkeror. With the expection of the about:-page showing the orange-blue Firefox logo and claiming that this is Firefox $CONKEROR_VERSION , it works as expected on my Toshiba AC100 netbook running the armel port of Ubuntu 11.10. From version 1.0~~pre+git1110272207-~nightly1 on, the Conkeror Nightly Built Debian Packages will be installable on Ubuntu 11.10 Oneiric again without the need to install or keep XULRunner version from Ubuntu 11.04 Natty. For those who don t want to use the nightly builds, I created a (currently still empty) specific PPA for Conkeror where I ll probably upload all the conkeror packages I upload to Debian Unstable.