I came across the following compiler warning today:

    CC     libplurk_la-plurk-item-view.lo
  plurk-item-view.c: In function  construct_image_url :
plurk-item-view.c 233  warning: format  %lld  expects type  long long int ,
but argument 3 has type  gint64 

Tut tut tut .. a compiler warning! How could the commiter had let this happen? Lets look at the code

url = g_strdup_printf ("http://avatars.plurk.com/%s-medium%lld.gif", ...);

Ahaha .. they re probably using a 32-bit system, the nice thing to do here is:

url = g_strdup_printf ("http://avatars.plurk.com/%s-medium%" G_GINT64_FORMAT ".gif", ...);

To handle the case that on a 64-bit system you can represent a gint64 as a long rather than needing to go for a long long. I ve seen this quite a bit with debugging output for size_t for which G_GSIZE_FORMAT is definitely your friend.

The discussion about Debian rolling started sooner than expected on debian-devel (see the thread here). I initially wanted to iron out the biggest problems through discussions on my blog and try to submit a somewhat polished proposal instead we ended up discussing the same things both on -devel and on my blog. That said it s not that bad (except for the time I lost to have similar conversations in both places) because the debian-devel discussions included members of the release team and it looks like they are not fundamentally opposed to the idea. Despite this, the introduction of a frozen suite branched off from testing/rolling is not really consensual (yet?). But the idea of officially supporting testing on a best-effort basis appears to have almost no opposition. While some will undoubtedly believe that this is a useless exercise, I believe it would help if the project stated this in a somewhat official manner. My answer to the question in the title is thus: I am thus considering to submit a general resolution to that effect. My current draft is below.

Title: Debian endorses usage of testing by end-users, and renames it to rolling The Debian project recognizes that the Debian testing distribution initially created to make it easier to prepare and test the next stable release has become a useful product of its own. It satisfies the needs of users who are looking for the latest stable versions of software and who can cope (or even appreciate) a system that s constantly evolving. The Debian project decides to endorse this usage and will strive to provide a good experience to users of testing . To better communicate this policy change to our users, testing will be renamed rolling . While we believe that this is a good move, we would like to remind our users that Debian is a volunteer project and that our resources are not infinite. Package maintainers are contributing to Debian on a best-effort basis. This means that they might not be able to properly support their package(s) in all distributions. In that case, the project recommends that maintainers apply the following priorities:

1. Support in stable (security updates, release critical bugs)
2. Preparation of the next stable release
3. Support in rolling

Note that this general resolution could have amendments with s/rolling/current/ and that would solve the bikeshedding over the name that started several times already. I deliberately separated Preparation of the next stable release and Support in rolling so that priorities are clear even if we decide to not freeze rolling and to introduce a frozen distribution to finalize the next stable release. I also did not go into too much details on the implications that it might have, it s best to leave that up to each contributor/team/etc. I hesitated to add a paragraph stating that we want to try to gradually improve the usability of testing but in the end I think it would be somewhat redundant. We re always trying to do our best when we decide to take on something. All comments welcome (even if you just agree and would be willing to second such a GR). Update: I will tweak the draft included in this article when I get good suggestions. Thanks to Lucas Nussbaum for the first one.

10 comments Liked this article? Click here. My blog is Flattr-enabled.

Guido Trentalancia started an interesting discussion on the SE Linux policy development list about how to manage the evolution of the policy [1]. The Problem The SE Linux policy is the set of rules that determine what access is granted. It assigns types to files and domains to processes and has a set of rules that specify all the permitted interactions between processes and files (among many other things). The policy evolves over time to match the requirements of programs (applications and daemons). As a program evolves the things that it does will change and the SE Linux policy will tend to evolve to permit the set of all operations that were requested by all versions because people only complain when things stop working not when excessive privilege is granted. So we need to periodically remove old allow rules from the policy. One difficulty in this regard is the fact that multiple versions of programs are often available for use at the same time. Debian in particular has a good history of providing separate packages for the old and new versions of programs such as Apache to meet the needs of users who want the tried and tested version and of users who want the newer version with better performance, more features, better documentation, or something else good. There is also a demand to have the same policy work with multiple versions of a distribution without excessive effort. Finally all the distributions that have SE Linux support have different people deciding when the new version of a daemon is ready for inclusion and therefore there is a need to support multiple versions for multiple distributions. So support for older versions of daemons can t be removed easily. One of the things I do to make these things a little easier to manage is to put ifdef( distro_debian', before any Debian specific bits of policy. When policy is conditional and only used in Debian I can freely remove it at any future time if Debian works well without it. Also it doesn t matter if such Debian specific policy allows access that is not needed or desired in other distributions, the only down-side to this is that sometimes other distributions need to repeat work that I did, they determine what access is needed for their configuration and discover that it was already enabled for Debian. What is Valid Policy? We went to only have Valid Policy (as described by Christopher J. PeBenito), so the challenge is determining what is Valid Policy. It seems to me that there are three type of access granted by valid policy (it is debatable whether type #3 is valid):

1. Access that is needed for an application to perform it s minimal designed task.
2. Access that is needed for the application to perform all the optional configurations, EG an ftpd running from inetd or as a daemon, and daemons like http server being granted access to ssl keys or not.
3. Access that is needed to perform all the operations the application requests, but which the application doesn t require or shouldn t require if it worked correctly.

Some common operations that aren t required include opening utmp for write, searching /root, and many other relatively innocuous access attempts which don t affect the program operation if they are denied. There are also many things such as writing temporary files to /root that don t seem unusual if the application developer is not considering SE Linux (but which are often considered bad practice anyway). Some of these things (like using /root for stuff that belongs in /var/lib) have the potential to break things (for the daemon or for other system processes) even if you don t consider SE Linux. How to deal with those types:

1. In most cases this can be determined without too much effort. For example a web server needs to listen on port 80 and read files and directories that relate to data. When writing policy I can write a lot of the allow rules without even testing the application because I know from the design what it will do. A large part of the other access is obvious in a I can t believe I didn t realise it would need this sense.
2. The main question here is whether we have booleans (settings which can be tuned at run-time by the sysadmin which determine how the policy works) to specify which optional tasks or whether we allow all access for optional configurations by default. The secondary question is when certain unusual corner cases should be not supported at all such that the people who do such unusual corner cases need to use audit2allow to generate local policy to allow their operations.
3. Sometimes we have to allow things that we really don t like. Even when we write policy to allow a daemon to do unusual things (such as using /root instead of /var/lib) it s still a lot better than running without SE Linux. Also SE Linux policy to allow such obviously broken things stands out and is a constant reminder that the daemon needs fixing, this is better than allowing symptoms of broken design to be forgotten.

How to Improve the Situation We could have comments in the policy source for everything that is in category 3. If the comments had a fixed format so that a recursive grep could find them all then it would allow us to more easily remove the gross things from the policy at a later date. But it seems to me that the main problem is a lack of people working on this. I am not aware of any people actively testing Debian policy for excessive privilege in regard to such issues.

• [1] http://oss.tresys.com/pipermail/refpolicy/2011-March/004115.html

It s the time of the year again. LinuxTag is right up and, of course, Debian is part of it. But, it wouldn t be us if we didn t have any problems at all. Well, the situation isn t real bad but could deserve some improvement. Improvement about what, you ask? Isn t that obvious? It s man power that we need (women shall feel included :)). Now, LinuxTag 2011 is going to happen, with or without you, on May 11th to 14th. Needless to say that you ll be missing quite a lot if you don t attend. I mean, even Zack will be there. And so will I (most probably at least). And even more Who s willing to miss that, really? So, pack your stuff, get a train ticket or your car or your friend s car or the car of a friend of your friend who knows a friend who doesn t know you but still wants to give you his car which you don t even need since you already have the car of that friend of your friend and now you have two cars and can even bring another bunch of fellas who probably own cars themselves and now you re thinking about the environment because of all the cars that will come to Berlin and because of that you suddenly buy a train ticket and leave all three cars at home and in the end only one thing counts: you ll be there. Oh, and before you drive off to your local train station, please put your name into our neat little list so we can count on you as booth staff that s because we need man power in case you missed that section or somehow forgot it while thinking about that guy who is the friend of your friend that owns a car but rather comes by train as it saves the environment.

For a while I've been struggling with a little itch. I'm using my company notebook for company work and for Debian related stuff. Now, whenever I switch between those two contexts, I had to manually fix the environment configuration. This is mostly related to environment variables, because tools like dch et cetera rely on some, which need to be different for the different contexts, like DEBEMAIL.
A while ago I had the idea to use directory dependent configuration for that purpose, but I never found the time and mood to actually fix my itch.
Somewhere in the meanwhile I applied a quick hack ("case $PWD in ...; do export...; esac") to my zsh configuration to ease the pain, but it still did not feel right.

---

For the impatient: Below you find a way to just use whats described here. The rest of the article just contains detailed information on how to implement something like this.

---

The other day I were cleaning up and extending my zsh configuration and it came to my mind again. I then thought about what my requirements are and how I could solve it. First I thought about using a ready solution, like the one in the Grml Zsh configuration, but at that point I did not remember it (it needed a hint by a co-worker *after* I finished the first version of my solution). Then I came up with my requirements:

• Separate profile changing logic from configuration (as far as possible):I don't want to re-dive into a script logic every time I decide to change something, like adding a variable or changing it. Generally I find a declarative approach much cleaner.
• Avoid repeating myself
  Basically all I do when switching profiles is to change environment variables. Usually I don't want my shell to do extraordinary things, like brewing coffee when I switch the context, so I'd like to avoid typing an "export foobar.." for every single environment variable and every single profile.

typeset -A EMAILS ENV_DEBIAN ENV_COMPANY
EMAILS=(
"private" "foo@bar.org"
"company" "baz@foo.org"
"debian" "schoenfeld@debian.org"
)
ENV_DEBIAN=(
"DEBEMAIL" "$EMAILS[debian]"
"EMAIL" "$EMAILS[debian]"
)
ENV_COMPANY=(
"DEBEMAIL" "$EMAILS[company]"
)

The next part was selecting the right profile. In the first version I used the old case logic, but it was breaking my separate logic and configuration paradigm. Approximate at this point the co-worker lead me to the grml approach, which I borrowed an idea from:

# Configure profile mappings
zstyle ':chpwd:profiles:*company*' profile company
zstyle ':chpwd:profiles:*debian*' profile debian

and the following code to lookup profile based on $PWD:

1 function detect_env_profile
2 local profile
3 zstyle -s ":chpwd:profiles:$ PWD " profile profile profile='default'
4 profile=$ (U)profile
5 if [ "$profile" != "$ENV_PROFILE" ]; then
6 print "Switching to profile: $profile"
7 fi
8 ENV_PROFILE="$profile"
9

For an explanation: zstyle is a zsh-builtin which is used to "define and lookup styles", as the manpage says, or put different: Another way to store and lookup configuration values.
Its nice for my purpose, because it allows storing patterns instead of plain configuration values which can be compared against $PWD easily with all of the zsh globbing magic. This is basically whats done in line 3. zstyle then sets $profile to the matching zstyle configuration in the :chpwd:profiles: context or to 'default' if no matching zstyle is found.

The (almost) last part is putting it together with code to switch the profile:

1 function switch_environment_profiles
2 detect_env_profile
3 config_key="ENV_$ENV_PROFILE"
4 for key value in $ (kvP)config_key ; do
5 export $key=$value
6 done
7

The only non-obvious part in this are lines 3 and 4. Remember, the profiles were defined as ENV_PROFILE, where PROFILE is the name of the profile. We cannot know that key in advance, therefore we have to construct the right environment variable from the result of detect_env_profile. We do that in line 3 and lookup this environment variable in line 4.
The deciding aspect for that is the P-flag in the parameter expansion. It tells zsh that we do not want the value of $config_key, but instead the value of $WHATEVER_CONFIG_KEY_EXPANDS_TO.
The other flags k and v tell zsh that, from the array, we want both: keys and values. If we'd omitted those flags it would have given us the values only.
We then loop over that to configure the environment. Easy, hu?

We would be finished, if this would do anything. The code above needs to be called. Lucky for us thats pretty easy to achieve, as zsh has a hook for when a directory is changed. Making all this work is simply a matter of adding something like this:

function chpwd()
switch_environment_profiles

Now, one could say, that the solution in the grml configuration has an advantage. It allows calling arbitrary commands on profile changing, which might be useful to *unset* variables in a given profile or whatever you could think of.
Well, its a matter of three lines to extend the above code for that feature:

# Taken from grml zshrc, allow chpwd_profile_functions()
if (( $ +functions[chpwd_profile_$ENV_PROFILE] )) ; then
chpwd_profile_$ ENV_PROFILE
fi

to the end of switch_environment_profiles and now its possible to additionally add a function chpwd_profile_PROFILE which is called whenever the profile is changed to that profile.

---

USAGE: I have put the functions into a file which can be included into your zsh configuration, which can be found on github.
Please see this README and the comments in the file itself on further usage instructions.

Tomorrow Prince William will marry Kate Middleton. If we don t change anything he will probably become the King of Australia at some future time, so I think that now is the time to start discussing the options. Walter Block makes some interesting points in favor of longer terms for politicians and for having a monarch to get a long-term view of the national interest [1], he s not the only person to make such points, but he makes them in a better way than most. Of course the problem with this is the long history of kings not doing what s best for their country part of the ownership rights to property is the right to destroy it, so a monarch who owns a country therefore can be considered to have the right to cause the wholesale death of their subjects. There are some examples of President for Life political leaders demonstrating this at the moment. Even with a monarch who is generally a nice person and who has controls to prevent the worst abuses there is the possibility of Control Fraud. In the Constitutional Monarchy system that doesn t happen because a constitutional monarch has little power (no official position of power). But there is still the issue of whether the monarchy is any good. Charles Stross wrote an interesting post about the apparent human need to have a leadership figure [2]. So getting rid of a monarch tends to result in a president getting the trappings of a king, and if things go wrong (as they often do) then they get absolute power until the next revolution. It seems that having one person who is the head of government and the head of state (as done in the US for example) is a bad idea, they can start to think that it s all about them. I don t think that the US is at risk of getting a president for life in the near future and I don t think that Australia will do so if we become a republic, but that doesn t mean that the republican system works well. I think that the Australian system is working better than the US system and I will generally vote against any changes that make Australia more like the US. As long as the House of Windsor provides monarchs who are as sensible as Queen Elizabeth 2 I will vote in favor of the continued rule of the House of Windsor in preference to an Australian republic (if Prince Charles ever becomes king I may support a republic). A Way of Improving Things I want to have an Australian monarch. Someone who will live in Australia for most of the time (as opposed to a distant monarch who visits once a decade if we are lucky). Protocol should dictate that the Prime Minister and cabinet ministers are forced to show ritual respect to the monarch, bowing etc, and no touching. A separation between the person who performs most of the ceremonial functions and the person who actually makes the political decisions should help constrain political egos. When a Prime Minister feels the need to suck up to someone more powerful it would be better to have that person be an Australian monarch than the US president. Tradition has it that monarchs have to be descended from other monarchs (although there are cases of elected monarchs as happened in Danish history). An election for a monarch probably wouldn t work well in a modern political environment, so we need someone with royal ancestry. One possibility is to have a spare descendant of our current Queen become the monarch of Australia, I think it s quite likely that given a choice between remaining a UK prince or princess for the rest of their life and becoming the monarch of Australia there would be someone who would take the latter option and I expect that the Queen would consent to that arrangement if asked (she would have to prefer it to a Republic). Another possibility is the fact that Mary the Crown Princess of Denmark has more children than the Danish monarchy requires [3]. As she was born in Australia it seems likely that her children will have more interest in Australia than most royals and a skim read of some tabloid magazines indicates that her family is quite popular. I expect that if a Danish prince or princess was invited to become the monarch of Australia then this would be acceptable to the Queen of Denmark. In an ideal world there would not be such a thing as a monarchy. But as we don t get to have ideal voters and therefore our politicians are far from ideal it seems to me that the constitutional monarchy is the least bad system of government. Don t think that I am in favor of a monarchy, I just dislike it less than the other options. Finally the Queen is the Supreme Governor of the Church of England. I think that it would be good to have a separation between church and state and therefore anyone who is in a leadership position of any religious organisation should be considered unsuitable to be the monarch.

• [1] http://www.psychologytoday.com/blog/defending-the-undefendable/201101/term-limits-make-me-sick
• [2] http://www.antipope.org/charlie/blog-static/2011/04/make-it-stop.html
• [3] http://en.wikipedia.org/wiki/Mary,_Crown_Princess_of_Denmark

I use USAA for some of my finances. They specialize in remote banking (i've never been in a physical branch). Sadly, they can still be pretty clueless about how to use the web properly. My latest frustration with them was trying to use their Deposit@Home service, where you scan your checks and upload them. No problem, right? I've got a scanner and a web browser and i know how to use them both. Ha ha. Upon first connecting, i'm rejected, and i find the absurd System Requirements -- only Windows and Mac, and only certain versions of certain browsers. You also need Sun's Java plugin, apparently. Deliberately na ve, i call their helpdesk and ask them if they could just give me a link to let me upload my scanned checks. They tell me that they want 200dpi images, and then give an absurd runaround that includes references to the Gramm-Leach-Bliley Act as the reason they need to limit the system to Windows or Mac, and that security is the reason they need to control the scanner directly (apparently your browser can control your local scanner on Windows? yikes). But they let slip that Mac users don't have the scanner controlled directly, and can just upload images (apparently the federal law doesn't cover them or something). Preposterous silliness. The Workaround Of course, it turns out you can get it working on Debian GNU/Linux, mainly by telling them what they want to hear ("yes sir, i'm running Mac OS"), but you'll also have to run Sun's Java (non-free software) to do it, since their Java uploader fails with nasty errors when using icedtea6-plugin. I set up a dedicated system user for this goofiness, since i'm going to be running apparently untrustworthy applets on top of non-free software. I run a separate instance of iceweasel as that user; all configuration described is for that user's homedir. If you do this yourself, You'll need to decide if you want the same level of isolation for yourself. So i have the choice of installing sun-java6-plugin from non-free and having the plugin installed for all web browsers, or just doing a per-user install of java for my dedicated user (and avoiding the plugin for my normal user). I opted for the latter. As the dedicated user, I fetched the self-extracting variant from java.com, unpacked it, and added it to the iceweasel plugins:

chmod a+x ~/Download/jre-6u25-linux-i586.bin
mkdir -p ~/lib ~/.mozilla/plugins
cd ~/lib
~/Download/jre-6u25-linux-i586.bin
ln -s ~/lib/jre*/lib/*/libnpjp2.so ~/.mozilla/plugins/

Then i closed iceweasel and restarted it. In the relaunched iceweasel sesson, I told Iceweasel 4 to pretend that it was actually Firefox 3.6 on a Mac. I did this by going to about:config (checking the box that says i know what i'm doing), right-clicking, and choosing "new >> string". The new variable name is general.useragent.override and i found that setting it to the following (insane, false) value worked for me:

Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.9.2.3) Gecko/20060426 Firefox/3.5.18

Note that this configuration might make hotmail think that you are a mobile device. :P If you try to use this browser profile for anything other than visiting USAA, you might want to remove this setting, or install xul-ext-useragentswitcher to be able to set it more easily than using about:config. Once these changes were made, I was able to log into USAA, use the Deposit@Home service to upload my 200dpi grayscale scans. I guess they think i'm a Mac user now. The Followup After completing the upload, I wrote them this review (i doubt they'll post it):

The Deposit@Home service has great potential. Unfortunately, it currently appears to be overengineered and unnecessarily restrictive. The service requires two scans (front and back) of the deposited check, at 200dpi, in grayscale or black-and-white, in jpeg format, reasonably-cropped. The simplest way to do this would be to show some examples of good scans and bad scans, and provide two file upload forms. Once the user uploaded their images, the web site could run its own verification, and display them back for the user to confirm, optionally using a simple javascript-based image-cropper if any image seems wrong-sized. This would work fine with any reasonable browser on any OS. Instead, Deposit@Home requires the user to present a User-Agent header claiming to be from specific versions of Mac or Windows, running certain (older) versions of certain browsers, and requires the use of Sun's Java plugin. Entirely unnecessary system requirements to do a simple task. Disappointing. :(

Acknowledgements I found good background for this approach on the ubuntu forums. The Takeaway I continue to be frustrated and annoyed by organizations that haven't yet embraced the benefits of the open web. Clearly, USAA has spent a lot of money engineering what they think is a certain experience for their users. However, they didn't design with standard web browsers in mind, so they appear to have boxed themselves into a corner where they think they have to test and approve of the entire software stack running on the end-user's machine. This is not only foolish -- it's impossible. When you're designing a web-based application, just design it for the web. Keep it simple. If you want to offer some snazzy java-hooked-into-your-scanner insanity, i will only have a mild objection: it seems like a waste of time and engineering effort. My objection is much stronger if your snazzy/incompatible absurdity is the only way to use your service. A simple, web-based, browser-agnostic interface should be available to all your clients. Even more aggravating is the claim that they don't think they should engineer for everyone. I was told during the runaround that they would only support Linux if 4% of their users were using Linux (which they don't think is the case at the moment -- if you are a USAA customer, and you use something other than Mac and Windows, please tell them so). I tried to tell them that I wasn't asking for Linux support; i was asking for everyone support. If you just use generic engineering in the first place, there's no extra expense for special-casing other users. But they couldn't understand. And now, since i'll need to lie to them in my User Agent string every time i want to deposit a check online, those visits won't even show up in their logs to be counted. "Our web site deliberately disables itself for $foo users; we haven't written it for them; but that's OK, we don't have any $foo users anyway" is a nasty self-fulfilling prophecy. Why would you do that?

The Linux kernel attempts to protect portions of its memory from unexpected modification (through potential future exploits) by setting areas read-only where the compiler has allowed it (CONFIG_DEBUG_RODATA). This, combined with marking function pointer tables const , reduces the number of easily writable kernel memory targets for attackers. However, modules (which are almost the bulk of kernel code) were not handled, and remained read-write, regardless of compiler markings. In 2.6.38, thanks to the efforts of many people (especially Siarhei Liakh and Matthieu Castet), CONFIG_DEBUG_SET_MODULE_RONX was created (and CONFIG_DEBUG_RODATA expanded). To visualize the effects, I patched Arjan van de Ven s arch/x86/mm/dump_pagetables.c to be a loadable module so I could look at /sys/kernel/debug/kernel_page_tables without needing to rebuild my kernel with CONFIG_X86_PTDUMP. Comparing Lucid (2.6.32), Maverick (2.6.35), and Natty (2.6.38), it s clear to see the effects of the RO/NX improvements, especially in the Modules section which has no NX markings at all before 2.6.38:

lucid-amd64# awk '/Modules/,/End Modules/' /sys/kernel/debug/kernel_page_tables   grep NX   wc -l
0
maverick-amd64# awk '/Modules/,/End Modules/' /sys/kernel/debug/kernel_page_tables   grep NX   wc -l
0
natty-amd64# awk '/Modules/,/End Modules/' /sys/kernel/debug/kernel_page_tables   grep NX   wc -l
76

2.6.38 s memory region is much more granular, since each module has been chopped up for the various segment permissions:

lucid-amd64# awk '/Modules/,/End Modules/' /sys/kernel/debug/kernel_page_tables   wc -l
53
maverick-amd64# awk '/Modules/,/End Modules/' /sys/kernel/debug/kernel_page_tables   wc -l
67
natty-amd64# awk '/Modules/,/End Modules/' /sys/kernel/debug/kernel_page_tables   wc -l
155

For example, here s the large sunrpc module. RW is read-write, ro is read-only, x is executable, and NX is non-executable:

maverick-amd64# awk '/^'$(awk '/^sunrpc/  print $NF ' /proc/modules)'/','!/GLB/' /sys/kernel/debug/kernel_page_tables
0xffffffffa005d000-0xffffffffa0096000         228K     RW             GLB x  pte
0xffffffffa0096000-0xffffffffa0098000           8K                           pte
natty-amd64# awk '/^'$(awk '/^sunrpc/  print $NF ' /proc/modules)'/','!/GLB/' /sys/kernel/debug/kernel_page_tables
0xffffffffa005d000-0xffffffffa007a000         116K     ro             GLB x  pte
0xffffffffa007a000-0xffffffffa0083000          36K     ro             GLB NX pte
0xffffffffa0083000-0xffffffffa0097000          80K     RW             GLB NX pte
0xffffffffa0097000-0xffffffffa0099000           8K                           pte

The latter looks a whole lot more like a proper ELF (text segment is read-only and executable, rodata segment is read-only and non-executable, and data segment is read-write and non-executable). Just another reason to make sure you re using your CPU s NX bit (via 64bit or 32bit-PAE kernels)! (And no, PAE is not slower in any meaningful way.)

Yesterday I received another report for a game, which wasn't working with the wine-unstable packages built by me. The reason why the reporter wrote to me and suspected a bug in my builds was, that the same game worked on the system of a friend (using a different distribution). After some exchanged e-mails it became apparent that this was yet another installment of S3TC-compressed textures. And as this wasn't the first time I got such reports, I thought I blog about the solution today. As S3TC is one of those patent-encumbered texture compression formats (don't ask me why one can get a patent for such things, it's, IMHO, a very bad thing and we'd be better off, with a lot less patents), Mesa doesn't support it out of the box. You need to install an additional library or the proprietary driver for your graphics card. Before installing the library, make sure, that it is legal in your jurisdiction (i.e. check whether the patent is valid in your jurisdiction). If that is the case, you can install libtxc-dxtn0 from the Debian Multimedia repository or compile it yourself from source. If the patent is valid in your jurisdiction, you're most likely required to get a license from the rights holder for using the library, but I'm no lawyer so you might want to consult one prior to making your decision. For amd64 users, there is an additional caveat: Wine needs the 32 bit variant installed. At the moment, this means downloading the i386 package from debian-multimedia.org and extracting the library to /usr/lib32. For users with the r600c/g driver it gets (currently) really tricky, as there is no fully working support for loading libtxc_dxtn.so (current status is WIP). Here the only solution I know, would be to use the proprietary driver or writing patches for r600g (last option preferred *g*).

Linux 2.6.38 and 2.6.39 As you've probably seen, the testing suite finally got a new upstream kernel version (Linux 2.6.38) and is currently synchronised with unstable. I'm just about to upload a new version of linux-2.6 based on upstream stable update 2.6.38.4. Linux 2.6.39 is well on its way to release, so 2.6.39-rc4 should soon be in experimental. The Debian source package will add support for the new armhf (ARM with hardware floating-point) architecture. Splitting linux-2.6 The linux-base package, introduced to handle transitions and other support functions for 'image' packages, is now built from a separate source package. It won't be changed very often, so you will no longer need to upgrade linux-base every time you upgrade an 'image' package. (This doesn't apply within 'squeeze', but I may have a way to work around that.) The firmware-linux-free package, containing a few firmware and configuration blobs from Linux that are DFSG-compliant, is also now built from a separate source package. These blobs have not changed for a very long time, so again you will no longer be recommended to upgrade this package every time you upgrade an 'image' package. Secondly, this allows us to remove the entire 'firmware' directory from the upstream source rather than carefully removing and patching to leave just the DFSG-compliant bits. Wireless regulations The '2.4 GHz' and '5 GHz' ISM bands used for wifi are not defined identically around the world. The exact frequency ranges and maximum permitted power levels vary between countries. Radio equipment manufacturers have a legal responsibility to ensure their equipment operates within these regulations. Since it is expensive to implement such regulations in hardware (including many different variants for different regions), these restrictions are usually implemented in the driver using a country code retrieved from the hardware's EEPROM or flash. So this is now our responsibility. We need to be able to look up regulations by country codes, and we also need to support the case where a device sold in one country is used in another country with tighter regulations. For some time, the wifi configuration library (cfg80211) included regulatory information for a small number of countries and regions. However, the default since Linux 2.6.34 has been to build-in only the 'world' domain (intersection of all the restrictions) and to request information from user-space. In Debian, we didn't have anything to provide this information, so for example channels 12-13 became unavailable for users in Europe. The regulatory information should be provided by the Central Regulatory Domain Agent (CRDA) from a current database. These are now packaged as crda and wireless-regdb respectively. Drivers that do not use cfg80211 (mostly those from the 'staging' area) will not be affected, and may or may not implement correct regulations. When you travel to another country or use an imported wifi device with an incorrect country code (I hear 'CN' is common), you can use iw reg set to ensure that your wifi devices operate within the regulations of the country you are in. Set the REGDOMAIN variable in /etc/default/crda to make this persistent. These will be intersected with the regulations specified by the hardware.

Sebastian Thrun gave an interesting TED talk about the Google driverless car project and explains how his main aim is to avoid all the needless road deaths that are due to human error [1]. Finally a good use for the Google street-view type data! AnnMarie Thomas gave an interesting short TED talk about using play-dough to make circuits [2]. There are two recipies for play-dough, the one made with salt conducts well and the one made with sugar conducts poorly. That allows making wires with salty dough and insulators with the sugar dough. John Robb has published an interesting article about a Chinese fake revolutionary group that is triggering a backlash from Chinese security forces [3]. Even if this isn t accurate it seems like a good way to make people hate their local security forces and thus demand political change. Chris Rock made an interesting observation, we aren t making progress on racial issues, white people are getting less crazy [4]. Red Hill has an interesting article about 486 motherboards with fake cache chips that were sold in the 90 s [5]. One thing I disagree with is that they blame the customers for seeking low prices. When a white-box PC cost $2000 (which is $3000 in today s money) it made sense to try and get the cheapest option possible. Now that major department stores sell name-brand laptops for $400 it really makes sense to buy name-brand quality rather than white-box rubbish. IHollaBack.org is an interesting project to combat street harassment of women [6]. Psychology Today has an interesting article by Joe Navarro (former FBI counter-intelligence agent and author) about the serious implications of attempting to detect lies [7]. His main point is that most people over-estimate their ability to detect lies and because the legal system believes such claims from law enforcement officers many innocent people get found guilty and criminals get away free! Cory Doctorow wrote an informative article about the ways of persuading people to pay for content that can be obtained for free [8]. The main message seems to be that the big media companies are doing things the wrong way in everything that they do. The news satire site CBS Breaking News has an interesting about page explaining their mission [9]. They stopped their automated disaster generator after the Japanese Tsunami, while I can understand them wanting to keep some good taste and be sympathetic to the plight of the Japanese people it seems that they have forgotten that there is always a disaster somewhere. The typical bus plunge is just as bad as the Tsunami to the people on the bus and their relatives! Psychology Today has an interesting blog post by Satoshi Kanazawa explaining how criminals don t specialise, the psychological factors that make someone likely to commit one crime will make them likely to commit others, this makes it logical to collect DNA samples from all criminals [10]. Psychology Today has an interesting blog post by Dave Niose about a landmark US legal case in 1948 where Vashti McCollum had to escalate to the Supreme Court to allow her children to receive secular education [11]. We need something like this in Australia now as the religious extremists are going too far in indoctrinating children. Sam Richards gave a TED talk titled A Radical Experiment in Empathy which aims to teach Americans how to understand the way that people in the middle-east feel [12]. The comments suggest that his talk wasn t successful. Of course the fact that empathy doesn t have a clear definition in the English language doesn t help, and the fact that most people don t seem to interpret it in any way that corresponds to any dictionary and that most people seem unable to define what they mean by it makes things worse. At the moment I can t think of any examples of successfully teaching empathy to unwilling people. The people who want to learn will do so eventually, you can have some good success in helping them to learn faster. Kathryn Schulz gave an interesting TED talk about Being Wrong [13]. One interesting point that she makes concerns the way that people assume that people who disagree are ignorant, stupid, or evil instead of just having a different set of data or a different understanding of the same data. Of course it is possible for someone to be ignorant/stupid/evil AND have a different understanding. Marcin Jakubowski gave an inspiring TED talk about his project to develop free blueprints that allow anyone to create all machines needed to sustain civilisation with minimal cost [14]. His Open Source Ecology project has a blog and a wiki with blueprints and files for CAD/CAM [15]. Dave Meslin gave an insightful TED talk about apathy in the political process [16]. Among other things he compares council notices that supposedly request citizen input with adverts for running shoes which encourage people to buy them. One thing he didn t mention is the difference that technology can make, a short council advert with a QR code is probably a lot more useful than the current dense text-based adverts for today s audience.

• [1] http://www.ted.com/talks/sebastian_thrun_google_s_driverless_car.html
• [2] http://www.ted.com/talks/annmarie_thomas_squishy_circuits.html
• [3] http://globalguerrillas.typepad.com/globalguerrillas/2011/04/the-jasmine-revolution.html
• [4] http://www.youtube.com/v/z_L7qoP17-w
• [5] http://www.redhill.net.au/b/b-bad.html
• [6] http://www.ihollaback.org/
• [7] http://www.psychologytoday.com/blog/spycatcher/201010/detecting-lies-v-detecting-truth-serious-implications
• [8] http://www.guardian.co.uk/technology/gamesblog/2011/apr/20/digital-free-persuade-pay-cory-doctorow
• [9] http://cbsbreakingnews.com/about
• [10] http://www.psychologytoday.com/blog/the-scientific-fundamentalist/201008/does-it-make-sense-collect-dna-samples-misdemeanor-convict
• [11] http://www.psychologytoday.com/blog/our-humanity-naturally/201104/atheist-mom-goes-the-supreme-court-and-wins
• [12] http://www.ted.com/talks/sam_richards_a_radical_experiment_in_empathy.html
• [13] http://www.ted.com/talks/kathryn_schulz_on_being_wrong.html
• [14] http://www.ted.com/talks/marcin_jakubowski.html
• [15] http://opensourceecology.org/
• [16] http://www.ted.com/talks/dave_meslin_the_antidote_to_apathy.html

I just had a talk on How to contribute here in Taipei. The slides are available here in case you are interested.

A newly founded Logic Group in Mumbai, which seems to be a private thing but some professors from the IIT Bombay are taking part, has approached me and asked if I can give a talk about Church s lambda calculus and how he used it to show that Hilbert s Entscheidungsproblem is not solvable. They are starting a series of talk on logic on the occasion of Turing s 100th birth year and because I am finally leaving the IIT campus in two days, my talk turned out to be the opening talk for the series. Roughly guessed about fifty people attended, some professors but most of them students from fields other than mathematics and computer science I hope that they were not too confused by distinguishing truth and provability, expressability and computability, of which I did not give a proper explanation.
I started with a historical exposition of the state of logic in the beginning 20th century, then gave an introduction to lambda calculus, encoding of natural numbers in then, G del encoding. Finally I sketched the proof of the unsolvability of the question whether a lambda term has a normal form and then concluded by showing how this implies that the Entscheidungsproblem is not solvable.
I have before hand written out the talk in full, and again I ended up saying completely different things or at least said things completely different. Nevertheless, I am sharing the (planned) text of my talk, including the timeline that I draw on the whiteboard.

So how much wind energy can we realistically use? A paper (Earth System Dynamics, DOI: 10.5194/esd-2-1-2011) by Miller,Gans and Kleidon (MGK10) has generated a lot of controversy courtesy of a New Scientist review (See Climate Progress for a good rebuttal). They argue that the effective limit we can safely extract without (bad) climatic consequences is around 18--68 TW (we currently use 0.2 TW of wind power). This paper is a follow-on to Kleidons work on the thermodynamics of the climate, where he points out that the free energy is the important measure of what energy can be generated by renewables. The free energy of the atmosphere is on the scale of hundreds of TW, he argues, much less than the heat energy of ~10^5 TW that comes from the sun. He argues that not enough attention is paid to the free energy budget, and argues that from MEP theory that free energy flows through different geophysical processes are more important than people currently think: such energy and momentum flows then lead to faster depletion of gradients than people currently use in their models. Frequently we use "diffusive-like" approximations to stuff we can't explicitly handle,such as convection and eddies below the resolution of our computer models. That the free energy budget, and small-scale processes need more detailed study (such as the transfer of momentum and energy from wind to water), I agree. But paradoxically, MGK10's estimates based on diffusive processes are what gets it into trouble. The critiques (e.g. Jacobsen and Archer) seem to concentrate on the estimates for diffusive and other energy losses (eg. wake fields) and climatic consequences are very loose, and out by factors of 50-100. They also ignore the fact that using wind as a substitute for more destructive energy sources produces a net gain, climate-wise. Based on these numbers, the real limits on wind power are more limited by where we can realistically place turbines, and the use and availability of rare earth metals in turbine magnets (MW sized turbines use over a ton of neodynium each in magnets!) One of the focuses of research at ICHEC and Met Eireann is wind for wind power: this becomes increasingly important for Ireland as we have more energy production by wind. I've been recently involved in a European project Weather Intelligence for Renewable Energies on investigating weather and climate forecasting needs of wind energy. In our weather models, we are getting to finer and finer resolutions (down to 2-5 km operationally, but working on 1 km in research), but as yet do not take account of wind energy changes. Modelling the wake turbulence, etc. and other effects of turbines is a 'fluid dynamics' problem done by the wind turbine manufacturers, mostly, in order to lay out turbines in a farm efficiently; this is done on the timescale of metres and seconds, while on the other end of the spectrum work is done in climate research on the changes in potential winds to see if wind energy will change in a few decades. But little is done on 'weather' scales of kilometres, minutes and hours: it turns out there are wind patterns that we don't predict in our models (vortices, etc.; especially for offshore turbines) that affect wind energy production, but would be very useful for wind farm and electrical grid operators to know about. So we should think about adapting our weather models for these. Either way, it looks like more detailed work on the interactions between atmospheric dynamics and wind farms would be a good thing. Incidentally, whatever about the alarmist nature of the article, kudos to New Scientist for their linking to papers: they quoted the names of the journals (Earth System Dynamics,Philosophical Transactions of the Royal Society), but where the article was not yet published (or behind a paywall), linked to the Arxiv.org version. More could do this. Tags climate, energy, thermodynamics, science, wind, renewables

I have noticed some problems with how Debian is using the popularity-contest data.

• popcon units are unknown Using the popcon score of a package to measure its use is like using the bleeple score of a trip to measure its distance. Both scores have no sensible units attached, though they may be loosely derived from a unit value. Is a trip with a bleeple score of 99 a long trip? Is a package with a popcon score of 99 a rarely used package? The only way to resolve this ambiguity at all is to compare ratios of values, so the problimatic units cancel out. A flight from NYC to AMS with a bleepie score of 99 is 50 times as bleepie as my drive home, which scores 2. So, any statement like "low popcon score" is basically so lacking in context as to be meaningless. Such statements are deprecated, and should be ignored.
• not all popcon scores are comparable The above example is intentionally bad. Plane flights and car trips are not very comparable when you don't know what units (time / CO2 / distance / number of people sharing a confined space / security theater points) are being used. Similarly, comparing a high popcon package like gnome-terminal with a relatively low popcon package like udhcpc is very deceptive. The former is installed by default in the desktop task, but plenty of desktop users would not miss it. The latter is installed only on embedded systems, which can exist in absurd numbers, and none of which will tend to report to popcon. So, any attempt to compare popcon scores should include a rationalle about why the two scores are comparable. For example, gnome-terminal and rxvt are somewhat comparable since they are both terminal emulators. But, only the vote scores, not the inst scores should be compared, since gnome-terminal is installed by default. dhcp3-client and udhcpc are not comparable despite being similar packages.
• popcon scores do not measure long tail effects A strength of Debian is that not only commonly used, but also uncommon and niche software is packaged. Popcon does not measure the benefit of some little used peice of software being there, packaged and ready to use when a user needs it. For six years I kept satutils in Debian, despite it probably having no users. It has a very specific use case, to control a motorized internet satellite dish typically installed on an RV. I did that because it was essentially no work (the package was approximatly bug free, and required no changes since 2007), and because of the possible payoff if someone needed this thing and there it was, in Debian. The value of Debian in that occasion would spike to a value that, while not directly comparable with a popcon score, would be pretty epic, for that one user, as they pushed arrow keys to move a satellite dish around. (It also had the best WITHOUT WARRANTY statement I've had the pleasure to write: "If you break your dish off your vechicle using this software, you get to keep both pieces.") Every removal of a package for "low popcon score" runs the risk of silently degrading this overall value of Debian.
• who wants to be popular? Part of the problem is that popcon has been around long enough that the connotations of its name, "popularity contest" have been dulled by repetition (and abbreviation). Popularity contests are not pleasant things. They rarely reach the best result. They embody the tyranny of the majority. The name was originally, to the best of my knowledge, chosen exactly to imply all these failings, to say that hey, popularity-contest is deeply flawed, but is better than nothing for this one specific use case (ordering packages to place on CD sets). We no longer think of popcon with these caveats. That is a regression in your brain. Fix it. By removing packages that appear unpopular, we run the risk of Debian becoming bland and homogenous.

No, it hasn't been released yet, but they're working on it. We've not been doing much about beid since squeeze was released, mostly because we understood that version 4 of the software was quite close, and that working on some 3.x version in that light would not make much sense anymore. But that doesn't mean I haven't done anything about it, at all; a while back, the FedICT people contracted me to help them build Debian packages they could put an 'official' stamp on, and provide through their website or some such. Some pre-release versions of these packages are now available through their google code project, and it would be welcome if people could try them out and give feedback. Links:

• The middleware package contains the browser plugin, which doubles as the PKCS#11 library. It should even auto-install, which the packages currently in the archive don't (due to technical reasons that would take us too far).
• The viewer contains an application that allows you to view the data on your eID card, print out a page with identity information, etc. It's not required for everyone, but can be interesting none the less.

In both cases, look under 'featured downloads'. These contain snapshot builds that should be fairly stable, but are not officially supported yet. Alternatively, you can track the head of the code (packages that are built automatically upon commit) by going to some alternative pages for the middleware and the viewer. Feedback is welcome, preferably through the relevant Google Code bugtrackers.

You know, it would be really good if someone could take ShairPort, glue that to gst-rtsp-server, and then implement the Rygel MediaServer specification, letting me play music from my iPhone on my Raumfeld UPnP speakers. I'd actually like this so much that I'm willing to put up some of my own hard cash to see it happen. Are there sites that will let people pledge money towards projects like this? NP: Central Reservation, Beth Orton

One of the more exciting upcoming features of Farsight is the ability to change the amount of required bandwidth depending on network conditions. Which means Telepathy based VoIP clients will be able to use higher resolutions and better audio/video quality if the network connectivity allows. And ofcourse, when the available bandwidth is low, the bandwidth usage can be lowered such that calls are still possible with lower but hopefully still acceptable quality. One side of the full story is measuring the available bandwidth, which hopefully Olivier will tell at some point. The other part is actually using less bandwidth. In case technical details are not that interesting to you, probably best to stop reading at this point.. An important way to control the bandwidth we're using in a call is setting the target bitrate on the encoding GStreamer element. While previously it was enough for us to set the bitrate before encoding started, we now need to be able to change it at runtime. And not only that, the codec implementation should ideally respond within a few frames and have a reasonable constant per-frame size. All in all, a lot of requirements on codecs we didn't really have before. Which usually means we need to retest the various codecs we often use and make sure we configure them correctly for the behaviour we'd like to see (and fix them if necessary). To do this I've written a small test program, which simply changes the target bitrate over time and outputs some numbers suitable to feed into gnuplot to turn into graphs. For example, when using x264enc with the current default farsight settings a part of the resulting graph looks something like this (full graph here): The blue impulses is the per-frame size (scaled), the green line the bitrate average over 10 frames and the red line the target bitrate. Clearly this isn't an ideal picture, we will need to do some more tweaks to our default settings to make x264 behave the way we want. The same will be true for all other codecs and codec elements, at the moment the GStreamer VP8 element doesn't handle run-time bitrate changes at all and the Theora element acts it its very own interesting way (graph here)

So we're just over 2 weeks away from our next set of local elections in the UK (May 5th), and alongside those elections we're also being asked about switching to a new voting system for future elections. For a long time we've used the simple First Past The Post (FPTP) system here, but now we have the possbility of moving to Alternative Vote (AV) instead. First Past The Post FPTP is simple to understand - the person who receives more votes than any single other person wins. But that simplicity is the only good thing, and there are many problems with it. It's unfair: in an election with 10 candidate, it's possible for a winner to have just 11% of the vote, even in the case where the other 89% of voters would consider them to be the worst option. It's also very susceptible to tactical voting, leading to nasty tactics in parties' election literature like claiming "party foo cannot win here, so don't waste your vote on them - vote for us instead!". See this Wikipedia article for more background. Alternative Vote AV is slightly more complicated. Instead of just placing a mark against their single preferred candidate, voters are able to rank as many of the candidates as they like. In the case that there is not a clear winner with more than 50% of the votes from the initial count, the lowest-ranked candidate is eliminated and second-choice votes from their supporters are counted and re-distributed for the other candidates. Iterate this process until one candidate gets more than 50% of the total votes. By re-calculating the votes this way, supporters of less popular candidates / parties should no longer feel the pressure to vote tactically and a more accurate picture of voter intention should emerge. The downsides? AV will tend to lead to slower, more expensive counting due to the potential for several rounds. It's still not real proportional voting, but it's better than FPTP in this regard. Again, Wikipedia has a good article about this subject. Other options? I'd be much happier to see discussion / trials of other voting systems. For example, Debian uses a variation on Condorcet called Cloneproof Schwartz Sequential Dropping which is an excellent system for fair voting, but it's very difficult to explain and counting votes is comparatively very expensive. It's bad enough getting ostensibly-intelligent Debian developers to understand this system; extending this to a national election would be impossible in my opinion. It's also not an option on the ballot here... :-) Politicians spreading lies I know this won't come as a major shock for a nationwide referendum, but there's a lot of campaigning going on. And, in the best traditions of political campaigning, there's a huge amount of bullshit being spread. The worst is coming from the "No to AV" campaign, as far as I can see. Without many positive things to claim, various members of the Conservative party (current government, with most to fear from a change of voting system, of course) are spouting outright lies and sowing FUD in all directions:

1. Claiming that AV will cost 250 million, most of which would be for the cost of electronic voting machines. Except... there's no evidence that these would be needed, nor are there any plans to use them.
2. Continuing on, highlighting the alleged "extra costs" of AV: campaign poster FUD saying that we need to choose between cardiac facilities for babies and AV, or between equipment for our soldiers and AV. Except... there's no evidence that costs will be that high, nor that we have to make such binary choices.
3. Claims from senior Conservative figures that changing to AV would mean more votes and legitimacy for extremists like the British National Party. Except... there is no evidence that AV will boost minority extremist parties. The BNP themselves are urging their supporters to vote against AV. Finally, if these parties have a high enough proportion of votes that they should be getting seats in parliament then they should have those seats - this is one of the tenets of democracy. Why should we be choosing a voting system to deliberately disenfranchise people?
4. Finally, even David Cameron is at it: "too much of the debate about the alternative vote (AV) had so far been dominated by 'scientific' evaluation of the two systems' merits. But for me, politics shouldn't be some mind-bending exercise. It's about what you feel in your gut - about the values you hold dear and the beliefs you instinctively have. And I just feel it, in my gut, that AV is wrong." Well, it's nice that our Prime Minister wants to ignore all the scientific evidence and go with his gut feeling. After all, why would we want to think about choices like this?

Don't swallow the bullshit If you're eligible to vote in the UK, please ignore the bollocks. Make up your own mind how to vote in this referendum, by looking at the facts. I've done that and I'll be voting in favour of switching to AV.

After two weeks of work, I've just released git-annex version 0.20110417, with some big new features that open up some interesting use cases:

• Want to use The Cloud as a git remote? git-annex now supports storing data in Amazon S3 as if it were just another git remote. With full gpg encryption of the content stored in S3. * This is so easy to use and neato I just have to show it off:

joey@gnu:~/tmp/repo> git annex initremote cloud type=S3 encryption=joey@kitenet.net
initremote cloud (checking bucket) (creating bucket in US) ok
joey@gnu:~/tmp/repo> git annex add bigfile
add bigfile ok
(Recording state in git...)
joey@gnu:~/tmp/repo> git annex move bigfile --to cloud
move bigfile (gpg) (checking cloud...) (to cloud...) ok
(Recording state in git...)
joey@gnu:~/tmp/repo> file bigfile
bigfile: broken symbolic link
joey@gnu:~/tmp/repo> git annex get bigfile
get bigfile (copying from cloud...) (gpg) ok
(Recording state in git...)
joey@gnu:~/tmp/repo> file bigfile
bigfile: symbolic link

• Want to collaborate on some big files? Perhaps you're dealing with scientific datasets, or video game levels. git-annex can now use a bup repository as a special kind of git remote. The big files are stored forevermore in bup's git repository, while their metadata and the rest of your stuff can be kept in git as usual -- and both git repositories can be used collaboratively. It's turtles^Wgit all the way down, but without the large file scalability problems.
• Want to trade disk space with someone? You can set up a bup remote, or just bare directory on their system, and have git-annex encrypt the data it stores there.
• Want a DropBox like folder that's Free Software and stores data in the cloud, or in a more git-style distributed way? Well, about that... I had been planning to finish up S3 and encryption support, and then mention this could perhaps be used as the basis for a DropBox like thing. But Christophe-Marie Duquesne anticipated me by a month, and created ShareBox. It's a FUSE filesystem layered over git-annex. While not yet production ready (lacking eg, conflict resolution), it has promise.

---

* This feature isn't available in Debian yet, blocked by a lack of the Haskell hS3 library packaged for Debian. Someone should fix that, ideally not me. Getting missingh updated for the ghc7 transition so git-annex is buildable in unstable would also be nice..