Romain and I just posted our slides from yesterday's Rcpp workshop and class (preceding the now-ongoing R/Finance conference). You can access the slides via my presentation page, or directly from here as Part 1 (Introduction), Part 2 (Details), Part 3 (Advanced) and Part 4 (Applications). The page also contains a link to the examples as a tar.gz and zip file.

Update: One link corrected.

I came across the following compiler warning today:

    CC     libplurk_la-plurk-item-view.lo
  plurk-item-view.c: In function  construct_image_url :
plurk-item-view.c 233  warning: format  %lld  expects type  long long int ,
but argument 3 has type  gint64 

Tut tut tut .. a compiler warning! How could the commiter had let this happen? Lets look at the code

url = g_strdup_printf ("http://avatars.plurk.com/%s-medium%lld.gif", ...);

Ahaha .. they re probably using a 32-bit system, the nice thing to do here is:

url = g_strdup_printf ("http://avatars.plurk.com/%s-medium%" G_GINT64_FORMAT ".gif", ...);

To handle the case that on a 64-bit system you can represent a gint64 as a long rather than needing to go for a long long. I ve seen this quite a bit with debugging output for size_t for which G_GSIZE_FORMAT is definitely your friend.

The discussion about Debian rolling started sooner than expected on debian-devel (see the thread here). I initially wanted to iron out the biggest problems through discussions on my blog and try to submit a somewhat polished proposal instead we ended up discussing the same things both on -devel and on my blog. That said it s not that bad (except for the time I lost to have similar conversations in both places) because the debian-devel discussions included members of the release team and it looks like they are not fundamentally opposed to the idea. Despite this, the introduction of a frozen suite branched off from testing/rolling is not really consensual (yet?). But the idea of officially supporting testing on a best-effort basis appears to have almost no opposition. While some will undoubtedly believe that this is a useless exercise, I believe it would help if the project stated this in a somewhat official manner. My answer to the question in the title is thus: I am thus considering to submit a general resolution to that effect. My current draft is below.

Title: Debian endorses usage of testing by end-users, and renames it to rolling The Debian project recognizes that the Debian testing distribution initially created to make it easier to prepare and test the next stable release has become a useful product of its own. It satisfies the needs of users who are looking for the latest stable versions of software and who can cope (or even appreciate) a system that s constantly evolving. The Debian project decides to endorse this usage and will strive to provide a good experience to users of testing . To better communicate this policy change to our users, testing will be renamed rolling . While we believe that this is a good move, we would like to remind our users that Debian is a volunteer project and that our resources are not infinite. Package maintainers are contributing to Debian on a best-effort basis. This means that they might not be able to properly support their package(s) in all distributions. In that case, the project recommends that maintainers apply the following priorities:

1. Support in stable (security updates, release critical bugs)
2. Preparation of the next stable release
3. Support in rolling

Note that this general resolution could have amendments with s/rolling/current/ and that would solve the bikeshedding over the name that started several times already. I deliberately separated Preparation of the next stable release and Support in rolling so that priorities are clear even if we decide to not freeze rolling and to introduce a frozen distribution to finalize the next stable release. I also did not go into too much details on the implications that it might have, it s best to leave that up to each contributor/team/etc. I hesitated to add a paragraph stating that we want to try to gradually improve the usability of testing but in the end I think it would be somewhat redundant. We re always trying to do our best when we decide to take on something. All comments welcome (even if you just agree and would be willing to second such a GR). Update: I will tweak the draft included in this article when I get good suggestions. Thanks to Lucas Nussbaum for the first one.

10 comments Liked this article? Click here. My blog is Flattr-enabled.

The last few days I have spent some time trying to add support for the Open311 API in the Norwegian FixMyStreet service. Earlier I believed Open311 would be a useful API to use to submit reports to the municipalities, but when I noticed that the New Zealand version of FixMyStreet had implemented Open311 on the server side, it occurred to me that this was a nice way to allow the public, press and municipalities to do data mining directly in the FixMyStreet service. Thus I went to work implementing the Open311 specification for FixMyStreet. The implementation is not yet ready, but I am starting to get a draft limping along. In the process, I have discovered a few issues with the Open311 specification. One obvious missing feature is the lack of natural language handling in the specification. The specification seem to assume all reports will be written in English, and do not provide a way for the receiving end to specify which languages are understood there. To be able to use the same client and submit to several Open311 receivers, it would be useful to know which language to use when writing reports. I believe the specification should be extended to allow the receivers of problem reports to specify which language they accept, and the submitter to specify which language the report is written in. Language of a text can also be automatically guessed using statistical methods, but for multi-lingual persons like myself, it is useful to know which language to use when writing a problem report. I suspect some lang=nb,nn kind of attribute would solve it. A key part of the Open311 API is the list of services provided, which is similar to the categories used by FixMyStreet. One issue I run into is the need to specify both name and unique identifier for each category. The specification do not state that the identifier should be numeric, but all example implementations have used numbers here. In FixMyStreet, there is no number associated with each category. As the specification do not forbid it, I will use the name as the unique identifier for now and see how open311 clients handle it. The report format in open311 and the report format in FixMyStreet differ in a key part. FixMyStreet have a title and a description, while Open311 only have a description and lack the title. I'm not quite sure how to best handle this yet. When asking for a FixMyStreet report in Open311 format, I just merge title an description into the open311 description, but this is not going to work if the open311 API should be used for submitting new reports to FixMyStreet. The search feature in Open311 is missing a way to ask for problems near a geographic location. I believe this is important if one is to use Open311 as the query language for mobile units. The specification should be extended to handle this, probably using some new lat=, lon= and range= options. The final challenge I see is that the FixMyStreet code handle several administrations in one interface, while the Open311 API seem to assume only one administration. For FixMyStreet, this mean a report can be sent to several administrations, and the categories available depend on the location of the problem. Not quite sure how to best handle this. I've noticed SeeClickFix added latitude and longitude options to the services request, but it do not solve the problem of what to return when no location is specified. Will have to investigate this a bit more. My distaste for web forums have kept me from bringing these issues up with the open311 developer group. I really wish they had a email list available via Gmane to use for discussions instead of only a forum. Oh, well. That will probably resolve itself, one way or another. I've also tried visiting the IRC channel #open311 on FreeNode, but no-one seem to reply to my questions there. This make me wonder if I just fail to understand how the open311 community work. It sure do not work like the free software project communities I am used to.

Guido Trentalancia started an interesting discussion on the SE Linux policy development list about how to manage the evolution of the policy [1]. The Problem The SE Linux policy is the set of rules that determine what access is granted. It assigns types to files and domains to processes and has a set of rules that specify all the permitted interactions between processes and files (among many other things). The policy evolves over time to match the requirements of programs (applications and daemons). As a program evolves the things that it does will change and the SE Linux policy will tend to evolve to permit the set of all operations that were requested by all versions because people only complain when things stop working not when excessive privilege is granted. So we need to periodically remove old allow rules from the policy. One difficulty in this regard is the fact that multiple versions of programs are often available for use at the same time. Debian in particular has a good history of providing separate packages for the old and new versions of programs such as Apache to meet the needs of users who want the tried and tested version and of users who want the newer version with better performance, more features, better documentation, or something else good. There is also a demand to have the same policy work with multiple versions of a distribution without excessive effort. Finally all the distributions that have SE Linux support have different people deciding when the new version of a daemon is ready for inclusion and therefore there is a need to support multiple versions for multiple distributions. So support for older versions of daemons can t be removed easily. One of the things I do to make these things a little easier to manage is to put ifdef( distro_debian', before any Debian specific bits of policy. When policy is conditional and only used in Debian I can freely remove it at any future time if Debian works well without it. Also it doesn t matter if such Debian specific policy allows access that is not needed or desired in other distributions, the only down-side to this is that sometimes other distributions need to repeat work that I did, they determine what access is needed for their configuration and discover that it was already enabled for Debian. What is Valid Policy? We went to only have Valid Policy (as described by Christopher J. PeBenito), so the challenge is determining what is Valid Policy. It seems to me that there are three type of access granted by valid policy (it is debatable whether type #3 is valid):

1. Access that is needed for an application to perform it s minimal designed task.
2. Access that is needed for the application to perform all the optional configurations, EG an ftpd running from inetd or as a daemon, and daemons like http server being granted access to ssl keys or not.
3. Access that is needed to perform all the operations the application requests, but which the application doesn t require or shouldn t require if it worked correctly.

Some common operations that aren t required include opening utmp for write, searching /root, and many other relatively innocuous access attempts which don t affect the program operation if they are denied. There are also many things such as writing temporary files to /root that don t seem unusual if the application developer is not considering SE Linux (but which are often considered bad practice anyway). Some of these things (like using /root for stuff that belongs in /var/lib) have the potential to break things (for the daemon or for other system processes) even if you don t consider SE Linux. How to deal with those types:

1. In most cases this can be determined without too much effort. For example a web server needs to listen on port 80 and read files and directories that relate to data. When writing policy I can write a lot of the allow rules without even testing the application because I know from the design what it will do. A large part of the other access is obvious in a I can t believe I didn t realise it would need this sense.
2. The main question here is whether we have booleans (settings which can be tuned at run-time by the sysadmin which determine how the policy works) to specify which optional tasks or whether we allow all access for optional configurations by default. The secondary question is when certain unusual corner cases should be not supported at all such that the people who do such unusual corner cases need to use audit2allow to generate local policy to allow their operations.
3. Sometimes we have to allow things that we really don t like. Even when we write policy to allow a daemon to do unusual things (such as using /root instead of /var/lib) it s still a lot better than running without SE Linux. Also SE Linux policy to allow such obviously broken things stands out and is a constant reminder that the daemon needs fixing, this is better than allowing symptoms of broken design to be forgotten.

How to Improve the Situation We could have comments in the policy source for everything that is in category 3. If the comments had a fixed format so that a recursive grep could find them all then it would allow us to more easily remove the gross things from the policy at a later date. But it seems to me that the main problem is a lack of people working on this. I am not aware of any people actively testing Debian policy for excessive privilege in regard to such issues.

• [1] http://oss.tresys.com/pipermail/refpolicy/2011-March/004115.html

A few months ago dropbox was a Debian packge in non-free. Recently I noticed that there are several new versions available and tried to notifiy the maintainer and ask him to update the package. But unfortunately it looks like the package is not available any more (Here s why). However getting Dropbox is quite easy if you know what to do. The obvious choice would be to go to dropbox.com and download either the source package and compile it yourself or try to install the precompiled Ubuntu packages. The first option is not really an option for most users and the second one does currently not work on Debian/Sid (aka unstable). However, there is an easy solution: on the download page search for the CLI script (currently it is named dropbox.py) Download that and run it with python dropbox.py start -i. That will download the propritary binaries into ~/.dropbox-dist. Now go into that folder and start dropboxd. There you go, you have a fully working systray app for dropbox. Depending on your desktop environment you might want to put dropboxd in your autostart . So you don t have to start the daemon every time you login. And that s it. It is not really ideal, since a Debian package would be much better for updates, security, yada yada, but it s still better than nothing.

The main feature of rolling is that it would never freeze. This is not without consequences. Possible consequences It can divert developers from working on the release No freeze means developers are free to continue their work as usual in unstable. Will it be more difficult to release because some people will spend their time working on a new upstream version instead of fixing RC bugs in the version that is frozen? Would we lose the work of the people who do lots of NMU to help with the release? It makes it more difficult to cherry-pick updates from unstable No freeze also means that unstable is going to diverge sooner from testing and it will be more difficult to cherry-pick updates from unstable into testing. And the release team likes to cherry-pick updates that have been tested in unstable because updates that comes through testing-proposed-updates have often not been tested and need thus a more careful review. My responses to the objections Those are the two major objections that we ll have to respond to. Let s try to analyze them a bit more. It s not testing vs rolling On the first objection I would like to respond that we must not put testing and rolling/unstable in opposition. The fact that a contributor can t do its work as usual in unstable does not mean that he will instead choose to work on fixing RC bugs in testing. Probably that some do, but in my experience we simply spend our time differently, either working more on non-Debian stuff or doing mostly hidden work that is then released in big batches at the start of the next cycle (which tends to create problems of its own). I would also like to argue that by giving more exposure to rolling and encouraging developers to properly support their packages in rolling, it probably means that the overall state of rolling should become gradually better compared to what we re currently used to with testing. The objection that rolling would divert resources from getting testing in a releasable shape is difficult to prove and/or disprove. The best way to have some objective data would be to setup a questionnaire and to ask all maintainers. Any volunteer for that? Unstable as a test-bed for RC bugfixes? It s true that unstable will quickly diverge from testing and that it will be more difficult to cherry-pick updates from unstable into testing. This cannot be refuted, it s a downside given the current workflow of the release team. But I wonder if the importance of this workflow is not overdone. The reason why they like to cherry-pick from unstable is because it gives them some confidence that the update has not caused other regressions and ensures that testing is improving. But if they re considering to cherry-pick an update, it s because the current package in testing is plagued by an RC bug. Supposing that the updated package has introduced a regression, is it really better to keep the current RC bug compared to trading it for a new regression? It sure depends on the precise bugs involved so that s why they prefer to know up-front about the regression instead of making a blind bet. Given this, I think we should use testing-proposed-updates (tpu) as a test-bed for RC bug fixes. We should ask beta-testers to activate this repository and to file RC bugs for any regression. And instead of requiring a full review by a release manager for all uploads to testing-proposed-updates, uploads should be auto-accepted provided that they do not change the upstream version and that they do not add/remove binary packages. Other uploads would still need manual approval by the release managers. On top of this, we can also add an infrastructure to encourage peer-reviews of t-p-u uploads so that reviews become more opportunistic instead of systematic. Positive reviews would help reduce the aging required in t-p-u before being accepted into testing. This changes the balance by giving a bit more freedom to maintainers but still keeps the safety net that release managers need to have. It should also reduce the overall amount of work that the release team has to do. Comments welcome Do you see other important objections beside the two that I mentioned? Do you have other ideas to overcome those objections? What do you think of my responses? Does your experience infirm or confirm my point of view?

18 comments Liked this article? Click here. My blog is Flattr-enabled.

It s the time of the year again. LinuxTag is right up and, of course, Debian is part of it. But, it wouldn t be us if we didn t have any problems at all. Well, the situation isn t real bad but could deserve some improvement. Improvement about what, you ask? Isn t that obvious? It s man power that we need (women shall feel included :)). Now, LinuxTag 2011 is going to happen, with or without you, on May 11th to 14th. Needless to say that you ll be missing quite a lot if you don t attend. I mean, even Zack will be there. And so will I (most probably at least). And even more Who s willing to miss that, really? So, pack your stuff, get a train ticket or your car or your friend s car or the car of a friend of your friend who knows a friend who doesn t know you but still wants to give you his car which you don t even need since you already have the car of that friend of your friend and now you have two cars and can even bring another bunch of fellas who probably own cars themselves and now you re thinking about the environment because of all the cars that will come to Berlin and because of that you suddenly buy a train ticket and leave all three cars at home and in the end only one thing counts: you ll be there. Oh, and before you drive off to your local train station, please put your name into our neat little list so we can count on you as booth staff that s because we need man power in case you missed that section or somehow forgot it while thinking about that guy who is the friend of your friend that owns a car but rather comes by train as it saves the environment.

For a while I've been struggling with a little itch. I'm using my company notebook for company work and for Debian related stuff. Now, whenever I switch between those two contexts, I had to manually fix the environment configuration. This is mostly related to environment variables, because tools like dch et cetera rely on some, which need to be different for the different contexts, like DEBEMAIL.
A while ago I had the idea to use directory dependent configuration for that purpose, but I never found the time and mood to actually fix my itch.
Somewhere in the meanwhile I applied a quick hack ("case $PWD in ...; do export...; esac") to my zsh configuration to ease the pain, but it still did not feel right.

---

For the impatient: Below you find a way to just use whats described here. The rest of the article just contains detailed information on how to implement something like this.

---

The other day I were cleaning up and extending my zsh configuration and it came to my mind again. I then thought about what my requirements are and how I could solve it. First I thought about using a ready solution, like the one in the Grml Zsh configuration, but at that point I did not remember it (it needed a hint by a co-worker *after* I finished the first version of my solution). Then I came up with my requirements:

• Separate profile changing logic from configuration (as far as possible):I don't want to re-dive into a script logic every time I decide to change something, like adding a variable or changing it. Generally I find a declarative approach much cleaner.
• Avoid repeating myself
  Basically all I do when switching profiles is to change environment variables. Usually I don't want my shell to do extraordinary things, like brewing coffee when I switch the context, so I'd like to avoid typing an "export foobar.." for every single environment variable and every single profile.

typeset -A EMAILS ENV_DEBIAN ENV_COMPANY
EMAILS=(
"private" "foo@bar.org"
"company" "baz@foo.org"
"debian" "schoenfeld@debian.org"
)
ENV_DEBIAN=(
"DEBEMAIL" "$EMAILS[debian]"
"EMAIL" "$EMAILS[debian]"
)
ENV_COMPANY=(
"DEBEMAIL" "$EMAILS[company]"
)

The next part was selecting the right profile. In the first version I used the old case logic, but it was breaking my separate logic and configuration paradigm. Approximate at this point the co-worker lead me to the grml approach, which I borrowed an idea from:

# Configure profile mappings
zstyle ':chpwd:profiles:*company*' profile company
zstyle ':chpwd:profiles:*debian*' profile debian

and the following code to lookup profile based on $PWD:

1 function detect_env_profile
2 local profile
3 zstyle -s ":chpwd:profiles:$ PWD " profile profile profile='default'
4 profile=$ (U)profile
5 if [ "$profile" != "$ENV_PROFILE" ]; then
6 print "Switching to profile: $profile"
7 fi
8 ENV_PROFILE="$profile"
9

For an explanation: zstyle is a zsh-builtin which is used to "define and lookup styles", as the manpage says, or put different: Another way to store and lookup configuration values.
Its nice for my purpose, because it allows storing patterns instead of plain configuration values which can be compared against $PWD easily with all of the zsh globbing magic. This is basically whats done in line 3. zstyle then sets $profile to the matching zstyle configuration in the :chpwd:profiles: context or to 'default' if no matching zstyle is found.

The (almost) last part is putting it together with code to switch the profile:

1 function switch_environment_profiles
2 detect_env_profile
3 config_key="ENV_$ENV_PROFILE"
4 for key value in $ (kvP)config_key ; do
5 export $key=$value
6 done
7

The only non-obvious part in this are lines 3 and 4. Remember, the profiles were defined as ENV_PROFILE, where PROFILE is the name of the profile. We cannot know that key in advance, therefore we have to construct the right environment variable from the result of detect_env_profile. We do that in line 3 and lookup this environment variable in line 4.
The deciding aspect for that is the P-flag in the parameter expansion. It tells zsh that we do not want the value of $config_key, but instead the value of $WHATEVER_CONFIG_KEY_EXPANDS_TO.
The other flags k and v tell zsh that, from the array, we want both: keys and values. If we'd omitted those flags it would have given us the values only.
We then loop over that to configure the environment. Easy, hu?

We would be finished, if this would do anything. The code above needs to be called. Lucky for us thats pretty easy to achieve, as zsh has a hook for when a directory is changed. Making all this work is simply a matter of adding something like this:

function chpwd()
switch_environment_profiles

Now, one could say, that the solution in the grml configuration has an advantage. It allows calling arbitrary commands on profile changing, which might be useful to *unset* variables in a given profile or whatever you could think of.
Well, its a matter of three lines to extend the above code for that feature:

# Taken from grml zshrc, allow chpwd_profile_functions()
if (( $ +functions[chpwd_profile_$ENV_PROFILE] )) ; then
chpwd_profile_$ ENV_PROFILE
fi

to the end of switch_environment_profiles and now its possible to additionally add a function chpwd_profile_PROFILE which is called whenever the profile is changed to that profile.

---

USAGE: I have put the functions into a file which can be included into your zsh configuration, which can be found on github.
Please see this README and the comments in the file itself on further usage instructions.

Tomorrow Prince William will marry Kate Middleton. If we don t change anything he will probably become the King of Australia at some future time, so I think that now is the time to start discussing the options. Walter Block makes some interesting points in favor of longer terms for politicians and for having a monarch to get a long-term view of the national interest [1], he s not the only person to make such points, but he makes them in a better way than most. Of course the problem with this is the long history of kings not doing what s best for their country part of the ownership rights to property is the right to destroy it, so a monarch who owns a country therefore can be considered to have the right to cause the wholesale death of their subjects. There are some examples of President for Life political leaders demonstrating this at the moment. Even with a monarch who is generally a nice person and who has controls to prevent the worst abuses there is the possibility of Control Fraud. In the Constitutional Monarchy system that doesn t happen because a constitutional monarch has little power (no official position of power). But there is still the issue of whether the monarchy is any good. Charles Stross wrote an interesting post about the apparent human need to have a leadership figure [2]. So getting rid of a monarch tends to result in a president getting the trappings of a king, and if things go wrong (as they often do) then they get absolute power until the next revolution. It seems that having one person who is the head of government and the head of state (as done in the US for example) is a bad idea, they can start to think that it s all about them. I don t think that the US is at risk of getting a president for life in the near future and I don t think that Australia will do so if we become a republic, but that doesn t mean that the republican system works well. I think that the Australian system is working better than the US system and I will generally vote against any changes that make Australia more like the US. As long as the House of Windsor provides monarchs who are as sensible as Queen Elizabeth 2 I will vote in favor of the continued rule of the House of Windsor in preference to an Australian republic (if Prince Charles ever becomes king I may support a republic). A Way of Improving Things I want to have an Australian monarch. Someone who will live in Australia for most of the time (as opposed to a distant monarch who visits once a decade if we are lucky). Protocol should dictate that the Prime Minister and cabinet ministers are forced to show ritual respect to the monarch, bowing etc, and no touching. A separation between the person who performs most of the ceremonial functions and the person who actually makes the political decisions should help constrain political egos. When a Prime Minister feels the need to suck up to someone more powerful it would be better to have that person be an Australian monarch than the US president. Tradition has it that monarchs have to be descended from other monarchs (although there are cases of elected monarchs as happened in Danish history). An election for a monarch probably wouldn t work well in a modern political environment, so we need someone with royal ancestry. One possibility is to have a spare descendant of our current Queen become the monarch of Australia, I think it s quite likely that given a choice between remaining a UK prince or princess for the rest of their life and becoming the monarch of Australia there would be someone who would take the latter option and I expect that the Queen would consent to that arrangement if asked (she would have to prefer it to a Republic). Another possibility is the fact that Mary the Crown Princess of Denmark has more children than the Danish monarchy requires [3]. As she was born in Australia it seems likely that her children will have more interest in Australia than most royals and a skim read of some tabloid magazines indicates that her family is quite popular. I expect that if a Danish prince or princess was invited to become the monarch of Australia then this would be acceptable to the Queen of Denmark. In an ideal world there would not be such a thing as a monarchy. But as we don t get to have ideal voters and therefore our politicians are far from ideal it seems to me that the constitutional monarchy is the least bad system of government. Don t think that I am in favor of a monarchy, I just dislike it less than the other options. Finally the Queen is the Supreme Governor of the Church of England. I think that it would be good to have a separation between church and state and therefore anyone who is in a leadership position of any religious organisation should be considered unsuitable to be the monarch.

• [1] http://www.psychologytoday.com/blog/defending-the-undefendable/201101/term-limits-make-me-sick
• [2] http://www.antipope.org/charlie/blog-static/2011/04/make-it-stop.html
• [3] http://en.wikipedia.org/wiki/Mary,_Crown_Princess_of_Denmark

I use USAA for some of my finances. They specialize in remote banking (i've never been in a physical branch). Sadly, they can still be pretty clueless about how to use the web properly. My latest frustration with them was trying to use their Deposit@Home service, where you scan your checks and upload them. No problem, right? I've got a scanner and a web browser and i know how to use them both. Ha ha. Upon first connecting, i'm rejected, and i find the absurd System Requirements -- only Windows and Mac, and only certain versions of certain browsers. You also need Sun's Java plugin, apparently. Deliberately na ve, i call their helpdesk and ask them if they could just give me a link to let me upload my scanned checks. They tell me that they want 200dpi images, and then give an absurd runaround that includes references to the Gramm-Leach-Bliley Act as the reason they need to limit the system to Windows or Mac, and that security is the reason they need to control the scanner directly (apparently your browser can control your local scanner on Windows? yikes). But they let slip that Mac users don't have the scanner controlled directly, and can just upload images (apparently the federal law doesn't cover them or something). Preposterous silliness. The Workaround Of course, it turns out you can get it working on Debian GNU/Linux, mainly by telling them what they want to hear ("yes sir, i'm running Mac OS"), but you'll also have to run Sun's Java (non-free software) to do it, since their Java uploader fails with nasty errors when using icedtea6-plugin. I set up a dedicated system user for this goofiness, since i'm going to be running apparently untrustworthy applets on top of non-free software. I run a separate instance of iceweasel as that user; all configuration described is for that user's homedir. If you do this yourself, You'll need to decide if you want the same level of isolation for yourself. So i have the choice of installing sun-java6-plugin from non-free and having the plugin installed for all web browsers, or just doing a per-user install of java for my dedicated user (and avoiding the plugin for my normal user). I opted for the latter. As the dedicated user, I fetched the self-extracting variant from java.com, unpacked it, and added it to the iceweasel plugins:

chmod a+x ~/Download/jre-6u25-linux-i586.bin
mkdir -p ~/lib ~/.mozilla/plugins
cd ~/lib
~/Download/jre-6u25-linux-i586.bin
ln -s ~/lib/jre*/lib/*/libnpjp2.so ~/.mozilla/plugins/

Then i closed iceweasel and restarted it. In the relaunched iceweasel sesson, I told Iceweasel 4 to pretend that it was actually Firefox 3.6 on a Mac. I did this by going to about:config (checking the box that says i know what i'm doing), right-clicking, and choosing "new >> string". The new variable name is general.useragent.override and i found that setting it to the following (insane, false) value worked for me:

Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.9.2.3) Gecko/20060426 Firefox/3.5.18

Note that this configuration might make hotmail think that you are a mobile device. :P If you try to use this browser profile for anything other than visiting USAA, you might want to remove this setting, or install xul-ext-useragentswitcher to be able to set it more easily than using about:config. Once these changes were made, I was able to log into USAA, use the Deposit@Home service to upload my 200dpi grayscale scans. I guess they think i'm a Mac user now. The Followup After completing the upload, I wrote them this review (i doubt they'll post it):

The Deposit@Home service has great potential. Unfortunately, it currently appears to be overengineered and unnecessarily restrictive. The service requires two scans (front and back) of the deposited check, at 200dpi, in grayscale or black-and-white, in jpeg format, reasonably-cropped. The simplest way to do this would be to show some examples of good scans and bad scans, and provide two file upload forms. Once the user uploaded their images, the web site could run its own verification, and display them back for the user to confirm, optionally using a simple javascript-based image-cropper if any image seems wrong-sized. This would work fine with any reasonable browser on any OS. Instead, Deposit@Home requires the user to present a User-Agent header claiming to be from specific versions of Mac or Windows, running certain (older) versions of certain browsers, and requires the use of Sun's Java plugin. Entirely unnecessary system requirements to do a simple task. Disappointing. :(

Acknowledgements I found good background for this approach on the ubuntu forums. The Takeaway I continue to be frustrated and annoyed by organizations that haven't yet embraced the benefits of the open web. Clearly, USAA has spent a lot of money engineering what they think is a certain experience for their users. However, they didn't design with standard web browsers in mind, so they appear to have boxed themselves into a corner where they think they have to test and approve of the entire software stack running on the end-user's machine. This is not only foolish -- it's impossible. When you're designing a web-based application, just design it for the web. Keep it simple. If you want to offer some snazzy java-hooked-into-your-scanner insanity, i will only have a mild objection: it seems like a waste of time and engineering effort. My objection is much stronger if your snazzy/incompatible absurdity is the only way to use your service. A simple, web-based, browser-agnostic interface should be available to all your clients. Even more aggravating is the claim that they don't think they should engineer for everyone. I was told during the runaround that they would only support Linux if 4% of their users were using Linux (which they don't think is the case at the moment -- if you are a USAA customer, and you use something other than Mac and Windows, please tell them so). I tried to tell them that I wasn't asking for Linux support; i was asking for everyone support. If you just use generic engineering in the first place, there's no extra expense for special-casing other users. But they couldn't understand. And now, since i'll need to lie to them in my User Agent string every time i want to deposit a check online, those visits won't even show up in their logs to be counted. "Our web site deliberately disables itself for $foo users; we haven't written it for them; but that's OK, we don't have any $foo users anyway" is a nasty self-fulfilling prophecy. Why would you do that?

There is a whole business around books with worldly wisdoms. They get bought as gifts for friends to cheer them up, they are meant to help one through hard times. I though see a big issue with them:

• If you would really need them, you aren't able to adopt them.
• If you are able to adopt them, you don't need them.

This chicken-egg issue is a real pain here.

/personal permanent link Comments: 1 Flattr this

Donald Trump you magnificent bastard, you did it. You actually did it. What you did is irrelevant and counterproductive but still gold star and a tick for perseverence. In possibly related news, SETI is closing down their radio telescope array. I don't think I'd want to show this planet off to the neighbors either.

The Linux kernel attempts to protect portions of its memory from unexpected modification (through potential future exploits) by setting areas read-only where the compiler has allowed it (CONFIG_DEBUG_RODATA). This, combined with marking function pointer tables const , reduces the number of easily writable kernel memory targets for attackers. However, modules (which are almost the bulk of kernel code) were not handled, and remained read-write, regardless of compiler markings. In 2.6.38, thanks to the efforts of many people (especially Siarhei Liakh and Matthieu Castet), CONFIG_DEBUG_SET_MODULE_RONX was created (and CONFIG_DEBUG_RODATA expanded). To visualize the effects, I patched Arjan van de Ven s arch/x86/mm/dump_pagetables.c to be a loadable module so I could look at /sys/kernel/debug/kernel_page_tables without needing to rebuild my kernel with CONFIG_X86_PTDUMP. Comparing Lucid (2.6.32), Maverick (2.6.35), and Natty (2.6.38), it s clear to see the effects of the RO/NX improvements, especially in the Modules section which has no NX markings at all before 2.6.38:

lucid-amd64# awk '/Modules/,/End Modules/' /sys/kernel/debug/kernel_page_tables   grep NX   wc -l
0
maverick-amd64# awk '/Modules/,/End Modules/' /sys/kernel/debug/kernel_page_tables   grep NX   wc -l
0
natty-amd64# awk '/Modules/,/End Modules/' /sys/kernel/debug/kernel_page_tables   grep NX   wc -l
76

2.6.38 s memory region is much more granular, since each module has been chopped up for the various segment permissions:

lucid-amd64# awk '/Modules/,/End Modules/' /sys/kernel/debug/kernel_page_tables   wc -l
53
maverick-amd64# awk '/Modules/,/End Modules/' /sys/kernel/debug/kernel_page_tables   wc -l
67
natty-amd64# awk '/Modules/,/End Modules/' /sys/kernel/debug/kernel_page_tables   wc -l
155

For example, here s the large sunrpc module. RW is read-write, ro is read-only, x is executable, and NX is non-executable:

maverick-amd64# awk '/^'$(awk '/^sunrpc/  print $NF ' /proc/modules)'/','!/GLB/' /sys/kernel/debug/kernel_page_tables
0xffffffffa005d000-0xffffffffa0096000         228K     RW             GLB x  pte
0xffffffffa0096000-0xffffffffa0098000           8K                           pte
natty-amd64# awk '/^'$(awk '/^sunrpc/  print $NF ' /proc/modules)'/','!/GLB/' /sys/kernel/debug/kernel_page_tables
0xffffffffa005d000-0xffffffffa007a000         116K     ro             GLB x  pte
0xffffffffa007a000-0xffffffffa0083000          36K     ro             GLB NX pte
0xffffffffa0083000-0xffffffffa0097000          80K     RW             GLB NX pte
0xffffffffa0097000-0xffffffffa0099000           8K                           pte

The latter looks a whole lot more like a proper ELF (text segment is read-only and executable, rodata segment is read-only and non-executable, and data segment is read-write and non-executable). Just another reason to make sure you re using your CPU s NX bit (via 64bit or 32bit-PAE kernels)! (And no, PAE is not slower in any meaningful way.)

Curious about these shiny new things i keep hearing about, i set up a test desktop system using using systemd as the init system (yes, that means using systemd-sysv in place of sysvinit -- so i had to remove an essential package for this to work). A system-crippling bug was naturally the result of living on the bleeding edge, but fortunately it was resolved with a trivial patch. In all, i'm pretty happy with some of the things that systemd seems to get right. For example, native supervision of daemon processes, clean process states, elimination of initscript copy-paste-isms, and socket-based service initiation are all obvious, marked improvements over the existing sysvinit legacy cruft. But i'm a bit concerned about other parts. For example, all the above-mentioned features fit really well within a tightly-tuned, well-managed server. But systemd also appears to rely heavily on complex userland systems like dbus and policykit that would be much more at home on a typical desktop machine. I've never seen a well-managed server installation that warranted either policykit or dbus. Also, given the bug i ran into -- when PID 1 aborts due to a silly assertion, your system is well-and-truly horked. Shouldn't a lot more attention to detail be happening? I'd think that a "recover gracefully from failed assertions" approach would be the first thing to target for a would-be PID 1. I'm also concerned about the Linux-centricism of systemd. I understand that features like cgroups and reliance on the spiffiness of inotify are part of the appeal, but i also really don't want to see us stuck with only One Single Kernel as an option. The kFreeBSD folks (and the HURD folks) have done a lot of work to get us close to having some level of choice at this critical layer of infrastructure. It'd be good to see that possibility realized, to help us avoid the creeping monoculture. I worry that systemd's over-reliance on Linux-specific features is heading in the wrong direction. So my question is: why is this all being presented as a package deal? I'd be pretty happy if i could get just the "server-side" features without incurring the dbus/policykit/etc bloat. I already run servers with runit as pid 1 -- they're lean and quite nice, but runit doesn't have systemd's socket-based initiation (or the level of heavyweight backing that systemd seems to have picked up, for that matter). I understand that Lennart is resistant to UNIX's traditional "do one thing; do it well" philosophy. I can understand some of his reasoning, but i think he might be doing his work and his tools a disservice by taking it this far. Wouldn't systemd be better if it was clearer how to take advantage of parts of it without having to subscribe to the entire thing? Of course, i might be missing some nice ways that systemd can be effectively tuned and pared down. But i've read his blog posts about systemd and i haven't seen how to get some of the nice features without the parts i don't want. I'd love to be pointed to some explanations that show me how i'm wrong :)Tags: systemd

As you might know, I m of one the people who promote the idea of having a Constantly Usable Testing (CUT). I will post a set of articles on this topic to help me clarify my ideas and to get some early feedback of other Debian contributors. I plan to use the result of this process to kickstart a broader discussion on debian-devel (where I hope to get the release team involved). Let s start by defining my own objectives with Debian CUT. I won t speak of the part of Debian CUT that plans to make regular snapshots of testing with installable media because that s not where I will invest my time. I do hope other people will do it though. Instead I will concentrate on changes that would improve Debian testing for end-users. I consider that testing (in its current form) is largely usable already but there are two main obstacles to overcome. Testing without freezes = rolling The first one is that testing is no longer testing during freezes. The regular flow of new upstream versions that makes testing so interesting for many users is stalled because we re using testing to finalize the next stable version. That s why I d like to introduce a new suite called rolling that would work like the current testing except that it never freezes. Testing would no longer be a permanent suite, it would only exist during the freeze and it would be branched off from rolling. Rolling should be a supported distribution The second obstacle is not really technical. We must advertise rolling as a distribution that ordinary people can use but we should make it clear that it s never going to have the same level of polish that a stable release can have. And to back up this assertion, we must empower Debian developers to be able to provide good support of their software in rolling, this probably means using rolling-proposed-updates more often to push fixes and security updates when the natural flow of updates is blocked by transitions or other problems. Some maintainers won t have the time required to provide the same level of support as they currently do for a stable release and that s okay, it s not worse than the current testing. The goal is to treat it on a best-effort basis and to try to gradually improve the situation. My goal for wheezy This is the the minimal implementation of CUT that I would like to target in the wheezy timeframe. Having testing as a temporary branch of rolling is not a strict requirement, although I d argue that it s important to not waste resources towards maintaining two separate yet very similar distributions. Should Debian embrace those goals? Ignoring all possible problems that will surface while trying to implement those goals, can we agree that Debian should embrace those goals because it means providing a better service to a class of users that is not satisfied by the current stable release? I will come back to the expected problems in a subsequent post and we will have the opportunity to discuss them. Here I just want to see whether we can have broad buy-in on the principles behind Debian rolling and CUT.

23 comments Liked this article? Click here. My blog is Flattr-enabled.

Since my latest announcement of the Lua workshop, I received a couple of emails asking why I discourage the use of Lua. Actually, I already wrote out many of the things I dislike about Lua. I won't come back on this technical issues here, but since Lua 5.2 is not yet released (it's still at alpha stage), they are still relevant nowadays. Stack based API is harder

The ease of integration of Lua into a C program is one of the point of Lua. They claim it's very easy to integrate Lua into your C application, because it does not use pointer, nor reference counting, nor anything that requires a minimum amount of skills to be used. It uses a virtual stack based approach. You push or pop things on a stack, and refers to them using a relative or absolute index. In order to people who never wrote Lua code to understand, here's a quick example on how this work. The L pointer is a Lua environment.

/* Create a table on the stack: index 1 */
lua_newtable(L);
/* Push a string on the stack: index 2 */
lua_pushstring(L, "hello");
/* Push a number on the stack: index 3 */
lua_pushnumber(L, 123);
/* Set newtable["hello"] = 123 */
lua_settable(L, -3);

You first push a table (in Lua, a table is almost equivalent to what you'd call a hash table in other language), then push the key, the value, and do the assignment operation. In the settable, we use -3 as index, meaning the "3rd item on the stack counting from top". We could also have written lua_settable(L, 1), since the table is also the first item on the stack from the bottom. So far, so good. Problems arise when you do more complicated stuff. My previous example is what you would typically find in a tutorial, but of course, real life is different, and usually more complex. If you cut the things in different parts, it can start to be more complicated. Let's take a look at the following:

/* Create a table on the stack: index 1 */
lua_newtable(L);
/* Push a string on the stack: index 2 */
lua_pushstring(L, "hello");
/* Push a number on the stack: index 3 */
lua_pushnumbe(L, mycomputingfunction());
/* Set newtable["hello"] = 123 */
lua_settable(L, -3);

Here, we do exactly the same thing, but we do not push 123 directly: we compute it. And here's the trick: if your computing function is also using the Lua stack, things can become very messy. As long as your computing function use the stack cleanly by pushing and poping all its item, and returning the stack in the same state it was before, you're safe. The problem is that in a complex program, you also write bugs. You do not chose to, but you do. And sometimes, you forget to pop one of the item you fetched from a table. Imagine that mycomputingfunction is:

int
mycomputingfunctiong(void)
 
  /* Just push the table we want to fetch
     the number from on the stack */
  pushatableonstack(L);
  lua_pushstring(L, "mykey");
  lua_gettable(L, -2);
  return lua_tonumber(L, -1);
 

This function works perfectly. It pushes a table, then a key ("mykey"), then fetches mytable["mykey"] and pops the key (lua_gettable does push value/pop key itself), and then returns the numeric value of the last item (the fetched one) of the stack. However, this function has a bug: it does not pop the table! This does not prevent the function to work. It does not raise a segmentation fault. It does not show any problem under gdb. It does not show any leak under Valgrind. It does now show any problem under any standard C debugging tool. But when you'll start using it, your program will start to do weird things, and you'll have to spend a huge amount of time debugging it manually, dumping the stack content at each step of your program to watch out what's wrong. Another bad thing, that can happen, is some code poping accidentally an item from the stack, or worst, from an empty stack. This does not raise any error on the Lua side, but will break your program in very unfunny way. Even if I've been very meticulous writing awesome, but we hit that problem regularly. The easiest workaround is to use lua_settop(L, 0) to reset the stack to 0 element. Doing this regularly (like after each program event or treatment) can remove left-over items and avoid the never ending stack grow you may experience if your left-over items continue to pile up. Did I tell you I dislike work-around? You could also use lua_call(), which would avoid such an error, but this would require a huge amount of indirection, and would make write more (useless) code. This kind of problem does not exists with pointer based API. If you screw things up, the problem will cause a segmentation fault or leak memory, or cause things you can (easily) debug with standard tools like gdb or Valgrind. No reference counting is a pain in the ass

Userdata objects are variable Lua size objects embedding a C struct you define. It's the equivalent of an object in object oriented language. Lua does not provide any reference counting for the userdata objects. That means you can push this objects on the stack, use them, but they cannot directly reference each others. If you have a "car" userdata and a "wheel" one, the car cannot hold directly a reference to the wheel. This is not possible because userdata are allocated and garbage collected by Lua, and there's no way to increase the reference counting yourself. So the common hack is to store the wheel into a table as a value, and store the table index as an integer into the car data structure. This obviously makes memory leaks tracking harder, add huge level of reference indirection in usage (still more code), and does not make the whole process less error prone (at least in my opinion). No paradigm makes you lose time

Lua is proud to come with no paradigm and to provide metatables. I already showed 3 years ago that it has big flaws. To me, this ain't no good. Lua is not functional, nor it is object oriented. Most people, including me, want one of this paradigm, or any else. Plain old imperative is not enough. So you'll start to build more, or to use something like LOOP, which implements an object model. You'll implement your paradigm. I say life is too short to (re)write a paradigm. In awesome we wanted to have an object oriented approach (this is kind of typical in such a graphical application context), so we tried to build one. To me, this started to be a show stopper when I realized that I've ended writing Python object model into Lua while developing awesome (which aims to be a window manager, not a language). This is one of the reason I stopped hacking on Lua things. I liked Python object model and wanted to have it in Lua, and spending time rewriting Python is just not worth it. I probably should have chose Python, not Lua. YMMV. Embedding may not be a good choice

This is not Lua related, but I want to mention it. Googling for "embedding vs extending" will probably tell you more about why you should double check that you really need to embed Lua rather than to extend it. Being small is not an excuse

One common argument to choose Lua is that it has a small footprint. Yeah, that's true, but that's useless. Bummer! When I program, I don't have any resource usage pressure. People who have such pressure are either paranoids or playing in the world of embedded computers. This is also a no more existing conception since quad core processors equiped phones are coming into the market. I'm rather confident that what we used to call embedded devices are just dead and are now plain computers. But as usual, YMMV. So start to forget about it, run in your underpants and yelling "yay we killed that shit!", and then use real computers stuff. :-) Even if benchmarks show how Lua is damn fast, remember what a benchmark proves: that you can do useless things very fast. Too few extension modules

This is not directly Lua's fault, but there's too few extension modules for Lua. The community is quite small compared to other big languages' ones. So think twice

before you choose Lua (or any other language). My recommendations these days would be not to embed, but to extend. If you really have no choice and need to embed a language into your application, GNU Guile is probably worth considering, because it's a Scheme and therefore a functional language :-), and because it can provides also different languages. Including Lua.

I'd like to buy most of my books as e-books, particularly novels. I'd like buy to all of my movies and TV shows as downloads. I'd like to buy maps for my Garmin GPS device as a download. I can do none of these things, unless I consent to restrict my freedoms in essential ways. For example, I can buy books, but only if I read them on a Kindle or Nook or similar restricted device. I can buy TV shows on iTunes, but I can't watch them on my Linux laptop. I can buy the maps, but only if I get a Windows (or maybe Mac) computer so I can run Garmin's proprietary software to download the maps and put them on the device. There are ways to circumvent these restrictions. Doing that is illegal in various countries I would like to live in. If I'm going to be breaking the law, the easier route would be to copy thing illegally. I can choose to succumb to corporate enslavement, where they get to decide what I watch, and how. Even when I try do this, I fail: I installed Amazon's Kindle software for the Android, but it would still not let me buy anthing, saying that each title is not available in my country. It said this even for stuff long past copyright, while giving pointers to the same thing in Project Gutenberg. Alternatively, I can choose to reject doing things digitally, and relying on paper for books and maps, and rental DVDs for video stuff. I've heard rumors that there is a lot of good stuff legally downloadable for free. Maybe I should go find some more. I have a short list already, but finding the stuff that actually interests me is not easy.

As Evgeni Golov already blogged, there is going to be the next round of a IRC meeting of the Debian/Ubuntu Games Team on the upcoming Saturday. This time it will be held at April 30th at 12:00 UTC in #debian-games on irc.oftc.net, so if you are interested in bringing the Games Team up to pace again, want to join and wonder how you could help, please attend. The agenda contains a fair amount of leftovers from the first meeting, please see Meeting Page about it.

/debian permanent link Comments: 0 Flattr this

I d like to extend a warm welcome to our new batch of students selected for the 2011 Debian Google Summer of Code! They should soon be posting on Debian Planet and you re welcome to come talk to them on #debian-soc on irc.debian.org Further details will be posted in the coming days to our wiki: http://wiki.debian.org/gsoc Automated Multi-Arch Cross-Building and Bootstrapping aka autocrossbuild , by Gustavo Prado Alkmim, mentored by Wookey
Enable easy and automated setup of cross-platform automated build systems and bootstrapping for QA in the Multi-Arch era. This involves the creation of multi-stage bootstrap build sequencing tools and a reliable automated multi-arch cross-builder. APT/Dpkg Transaction Ordering for Safety and Performance aka aptordering , by Chris Baines, mentored by Michael Vogt
The ordering code in libapt is responsible for ordering the unpacking/configuration of debs so as to ensure dependencies are satisfied etc. Currently it organizes the ordering into big batches. This project further implements an ordering satisfying more constrains such as minimal amounts of dpkg invocations , minimal amount of broken packages at any point . DebDelta APT Native Integration aka debdelta , by Ishan Jayawardena, mentored by Michael Vogt
Improve user experience of APT and its front-ends by speeding up the upgrade process. This provides a better framework for unified handling of debdelta and future APT improvements such as parallelism. Support for stable and security ugprades as well as multiple APT related libraries is expected. Dpkg Declarative Diversions aka declarativediversions , by Sam Dunne, mentored by Steve Langasek
The dpkg-divert command should be replaced with a new control file with a declarative syntax which Dpkg will parse and process directly as part of the package unpack and removal phases, eliminating the problems resulting from non-atomic handling of diversions. Backend Tools and Infrastructure for DEX aka dextools , by Nathan Handler, mentored by Matt Zimmerman
EX is a new program designed to help improve Debian and its derivatives by merging in changes made downstream and encouraging discussions between the various projects. As this is a new project, most of the infrastructure does not exist (or is rather hackish and incomplete). This project will create the necessary backend tools and infrastructure so that all Debian derivatives can easily make use of the DEX project. Jigsaw Modularized Java in Debian aka jigsaw , by Guillaume Mazoyer, mentored by Tom Marble
The Java Development Kit (JDK) is a big monolithic software tool: many of its features are only useful in limited areas (GUI toolkits are useless for a web server). This project will bring the Jigsaw modular JDK to Debian, helping performance (start-up, size, etc) but also the dependency resolution (to match Debian packaging). Some work exists upstream does not fit with Debian. This project will package the current development version of Jigsaw, update Debian Java Policy, and create the necessary packaging tools for software depending on it. Python Multi-Build for Python Extensions Packaging aka pythonmultibuild , by Mesutcan Kurt, mentored by Piotr O arowski
This project creates a tool to build Python extensions for all Python versions supported by Debian at the time. The project should detect the upstream build system and testing frameworks and use them. It will be interfaced with CDBS and the dh sequencer, replacing their Python snippets. Debian Teams Activity Metrics aka teammetrics , by Sukhbir Singh, mentored by Andreas Tille
This project will gauge the performance of teams in Debian by measuring metrics such as: postings on relevant mailing lists, package upload records from the Ultimate Debian Database and commit statistics from project repositories The information gathered will help in evaluating team performance by measuring how people in a team are working together. An interface to access this information easily will also be developed. Compute Clusters Integration for Debian Development and Building aka computeclusters , by Rudy Godoy, mentored by Steffen M ller
The project s main goal is to enable developers to easily use compute clusters (Eucalyptus, OpenStack ) as environments for arch-specific development by providing a set of tools they can use to setup and run an extended platform for their development, testing and building tasks. Good luck to everyone!