While preparing my slides for the Rcpp workshop this Thursday, I had wondered about more nice examples motivating Rcpp. So I posed a quick question on the rcpp-devel list. And I received a few friendly answers. My favourite, so far, was a suggestion by Lance Bachmeier who sent me a short script which used both R and C++ (via Rcpp) to simulate a first-order vector autoregressive process (and he ensured me that it worked well enough on his graduate students). It is indeed a great example as it involves (simple) matrix multiplication in an iterative fashion. Which makes it a great example not only for Rcpp but also for our RcppArmadillo package (which wraps Conrad Sanderson's wonderful Armadillo C++ templated library for linear algebra and more). And at the same time, we can also add another look at the new and shiny R compiler I also blogged about recently. So Lance and I iterated over this a little more over email, and I now added this as a new (and initial) example file in the RcppArmadillo SVN repo. (As an aside: The newest version 0.2.19 of RcppArmadillo has been sitting in incoming at CRAN since earlier in the week while the archive maintainer takes a well-deserved vacation. It should hit the public archive within a few days, and is otherwise available too from my site.) So let's walk through the example:

R> ## parameter and error terms used throughout
R> a <- matrix(c(0.5,0.1,0.1,0.5),nrow=2)
R> e <- matrix(rnorm(10000),ncol=2)
R> ## Let's start with the R version
R> rSim <- function(coeff, errors)  
+   simdata <- matrix(0, nrow(errors), ncol(errors))
+   for (row in 2:nrow(errors))  
+     simdata[row,] = coeff %*% simdata[(row-1),] + errors[row,]
+    
+   return(simdata)
+  
R> rData <- rSim(a, e)                     # generated by R

This starts with a simple enough loop. After skipping the first row, each iteration multiplies the previous row with the parameters and adds error terms. We can then turn to the R compiler:

R> ## Now let's load the R compiler (requires R 2.13 or later)
R> suppressMessages(require(compiler))
R> compRsim <- cmpfun(rSim)
R> compRData <- compRsim(a,e)              # generated by R 'compiled'
R> stopifnot(all.equal(rData, compRData))  # checking results

Nice and easy: We load the compiler package, create a compiled function and use it. We check the results and surely enough find them to be identical. With that, time to turn to C++ using Armadillo via RcppArmadillo:

R> ## Now load 'inline' to compile C++ code on the fly
R> suppressMessages(require(inline))
R> code <- '
+   arma::mat coeff = Rcpp::as<arma::mat>(a);
+   arma::mat errors = Rcpp::as<arma::mat>(e);
+   int m = errors.n_rows; int n = errors.n_cols;
+   arma::mat simdata(m,n);
+   simdata.row(0) = arma::zeros<arma::mat>(1,n);
+   for (int row=1; row<m; row++)  
+     simdata.row(row) = simdata.row(row-1)*trans(coeff)+errors.row(row);
+    
+   return Rcpp::wrap(simdata);
+ '
R> ## create the compiled function
R> rcppSim <- cxxfunction(signature(a="numeric",e="numeric"),
+                        code,plugin="RcppArmadillo")
R> rcppData <- rcppSim(a,e)                # generated by C++ code
R> stopifnot(all.equal(rData, rcppData))   # checking results

Here we load the inline package to compile, link and load C++ snippets. We define a short C++ function in the code variable, declare a signature taking a and e as before and ask cxxfunction() to deploy the plugin for RcppArmadillo so that it and Rcpp are found during build. With that, we have a compiled function to generate data, and we once again check the result. The C++ code is pretty straightforward as well. We can instatiate Armadillo matrices directly from the R objects we pass down; we then run a similar loop building the result row by row. Now, with all the build-up, here is the final timing comparison, using the rbenchmark package:

R> ## now load the rbenchmark package and compare all three
R> suppressMessages(library(rbenchmark))
R> res <- benchmark(rcppSim(a,e),
+                  rSim(a,e),
+                  compRsim(a,e),
+                  columns=c("test", "replications", "elapsed",
+                            "relative", "user.self", "sys.self"),
+                  order="relative")
R> print(res)
            test replications elapsed relative user.self sys.self
1  rcppSim(a, e)          100   0.038   1.0000      0.04        0
3 compRsim(a, e)          100   2.011  52.9211      2.01        0
2     rSim(a, e)          100   4.148 109.1579      4.14        0

So in a real-world example involving looping and some algebra (which is of course already done by BLAS and LAPACK libraries), the new R compiler improves by more than a factor of two, cutting time from 4.14 seconds down to about 2 seconds. Yet, this still leaves the C++ solution, clocking in at a mere 38 milliseconds, ahead by a factor of over fifty relative to the new R compiler And compared to just R itself, the simple solution involving Rcpp and RcppArmadillo is almost 110 times faster. As I mentioned, I quite like this example ;-).

Just a note that the slides are available (non-slideshare link) for my presentation Tuning an old but free phone (description) that I held in the tremendously great event FSCONS 2010. It could be described as a smaller scale FOSDEM, but that would be actually down-playing it since the free software effects on society are something that I've actually never seen elsewhere on such a scale. My talk was among the purely technical ones, though.

I was planning to hold on with this blog post until the recorded videos arrive, but since it seems it might not be during this year I will just post this now that slides are available.

I've shared a few photos as well at Flickr...


Keynote: Karin Kosina, The Inanna Project. A tech + art workshop for female artists in Damascus, Syria. An experiment in art, technology, and the transformative power of Free Hardware and Software.


Erik de Bruijn, The Future of RepRap, a self-replicating open source 3D printer that fabricates arbitrary objects including parts of itself.


Social event at the Berg 211.


Malin Nilsson on Gender, class and global flows. Using free software to fuel a revolution in home based industrial work.


Keynote: Glyn Moody, Ethics of Intellectual Monopolies.


Keynote: Glyn Moody, Ethics of Intellectual Monopolies (audience).

A few summaries available on a Qaiku seminar channel.

So how much wind energy can we realistically use? A paper (Earth System Dynamics, DOI: 10.5194/esd-2-1-2011) by Miller,Gans and Kleidon (MGK10) has generated a lot of controversy courtesy of a New Scientist review (See Climate Progress for a good rebuttal). They argue that the effective limit we can safely extract without (bad) climatic consequences is around 18--68 TW (we currently use 0.2 TW of wind power). This paper is a follow-on to Kleidons work on the thermodynamics of the climate, where he points out that the free energy is the important measure of what energy can be generated by renewables. The free energy of the atmosphere is on the scale of hundreds of TW, he argues, much less than the heat energy of ~10^5 TW that comes from the sun. He argues that not enough attention is paid to the free energy budget, and argues that from MEP theory that free energy flows through different geophysical processes are more important than people currently think: such energy and momentum flows then lead to faster depletion of gradients than people currently use in their models. Frequently we use "diffusive-like" approximations to stuff we can't explicitly handle,such as convection and eddies below the resolution of our computer models. That the free energy budget, and small-scale processes need more detailed study (such as the transfer of momentum and energy from wind to water), I agree. But paradoxically, MGK10's estimates based on diffusive processes are what gets it into trouble. The critiques (e.g. Jacobsen and Archer) seem to concentrate on the estimates for diffusive and other energy losses (eg. wake fields) and climatic consequences are very loose, and out by factors of 50-100. They also ignore the fact that using wind as a substitute for more destructive energy sources produces a net gain, climate-wise. Based on these numbers, the real limits on wind power are more limited by where we can realistically place turbines, and the use and availability of rare earth metals in turbine magnets (MW sized turbines use over a ton of neodynium each in magnets!) One of the focuses of research at ICHEC and Met Eireann is wind for wind power: this becomes increasingly important for Ireland as we have more energy production by wind. I've been recently involved in a European project Weather Intelligence for Renewable Energies on investigating weather and climate forecasting needs of wind energy. In our weather models, we are getting to finer and finer resolutions (down to 2-5 km operationally, but working on 1 km in research), but as yet do not take account of wind energy changes. Modelling the wake turbulence, etc. and other effects of turbines is a 'fluid dynamics' problem done by the wind turbine manufacturers, mostly, in order to lay out turbines in a farm efficiently; this is done on the timescale of metres and seconds, while on the other end of the spectrum work is done in climate research on the changes in potential winds to see if wind energy will change in a few decades. But little is done on 'weather' scales of kilometres, minutes and hours: it turns out there are wind patterns that we don't predict in our models (vortices, etc.; especially for offshore turbines) that affect wind energy production, but would be very useful for wind farm and electrical grid operators to know about. So we should think about adapting our weather models for these. Either way, it looks like more detailed work on the interactions between atmospheric dynamics and wind farms would be a good thing. Incidentally, whatever about the alarmist nature of the article, kudos to New Scientist for their linking to papers: they quoted the names of the journals (Earth System Dynamics,Philosophical Transactions of the Royal Society), but where the article was not yet published (or behind a paywall), linked to the Arxiv.org version. More could do this. Tags climate, energy, thermodynamics, science, wind, renewables

Meike Reichle is a Debian developer since 2008 but has been involved for longer than that, in particular in Debian Women. She s a great speaker and shared her experience in a Debconf talk. She s also part of the Debian publicity team and managed the live coverage of the last release on identi.ca. Enough introduction, learn more about her by reading the interview. My questions are in bold, the rest is by Meike. Who are you? My name is Meike Reichle, I am a studied information scientist and work as a project manager at Pengutronix, an embedded Linux company probably best known for their ARM kernel work. I live in Germany, more exactly in Lower Saxony, but I was originally born and raised in Swabia. Although I moved here ten years ago I still have a rather strong Swabian cultural identity. (Among other things I pride myself on having introduced a number of fellow DDs to the true promise that are real hand-made Sp tzle ) I am married to Alexander Reichle-Schmehl, we ll have our third wedding anniversary this summer. Apart from Debian most of my spare time is used for all kinds of crafts and DIY activities. Making things with my hands always gives me a great sense of accomplishment. My Free software history is summed up pretty quickly. Like most women of my age I wasn t introduced to computers until well into my teens. I didn t have a computer of my own until I started studying at the university in 2001. From there on things developed rather quickly: Working on the University s Unix terminals got me hooked on *nixes, so I got me one of those Linuxes everyone talked about. I tried a couple of different distributions, ended up with Debian around 2004, started contributing in 2005, and finally became a full DD what a nice coincidence! exactly this day (Apr 18th) three years ago. You re part of Debian Women. How is the project going? I have the feeling that the number of women involved in Debian has not significantly increased. The amount of women active within Debian is a tricky thing to judge. Here s a quick example why: When the DPL was elected in 2004 there were 911 Debian Developers eligible to vote, 4 of them were female. Shortly after, during DebConf4, debian-women was founded. When the current DPL was reelected last month, there were again 911 Debian Developers eligible to vote, but this time 13 of them were women. You can look at these numbers and say The number of female DDs has more than tripled, what a success! Or you can pull out your calculator and it will tell you that in terms of ratio this puts us from a measly 0.4% to an only slightly less measly 1.4% ratio of female DDs. This still is pardon my language a bloody shame, but sadly also pretty close to the average ratio of women in Free Software. So, while I do think that the debian-women project did already have a significant impact on the Debian project as a whole, I don t think it has achieved its goals yet. Not for a long time. There s still a lot to be done but unfortunately the debian-women project has somewhat run out of steam at the moment. The seven years of its existence divide quite equally into the first half, which was very active and saw great results, and the second half, which was very slow and much more passive. In my impression debian-women is currently undergoing a rather bumpy generational change. On the one hand a lot of the original members, including myself, have reduced their involvement. Speaking for myself this is caused by shift of interests as much as general weariness. On the other hand there are only very few women following up. This development is also reflected quite harshly in DD numbers: If I don t misjudge any first names (and I desperately hope I do!) for the last three years not a single woman has joined Debian as a developer! After the great start debian-women has had, this is a very painful thing to see! That said, things don t look all bad. There is a number of women maintaining packages without being DDs and there s also at least one woman currently in NM, so there s hope this standstill won t last very much longer. But still, the fact remains that debian-women is suffering from a rather serious recruitment problem and I hope that this interview might actually help to spur some new or not yet active members into action. The aim of debian-women is far from achieved and now that its initial members are receding its time for new members to step up and take initiative. What should Debian do to be more attractive to women ? I think the general atmosphere has improved, we re less tolerant with rude behaviour, the usual tone on mailing lists has improved. Yet it doesn t seem to be enough. If there was a female DD for every time I answered that question First of all, I agree, Debian as a community has improved tremendously! Our general tone is much more friendly and cooperative and there is now a much better awareness of the impression we give to outsiders and newcomers. Now on to the difficult part: The question what should be done to get more women into Free Software has been around almost as long as Free Software exists, and it has been answered very well by a lot of people: Twenty years ago Ellen Spertus wrote Why are There so Few Female Computer Scientists? and most of it still holds true. Almost ten years ago Val Henson (now Aurora) wrote HOWTO Encourage Women in Linux and that also is still pretty accurate. In 2006 Floss Pols undertook extensive research to find out why there were so few women in Open Source and Free Software and how that could be changed. They also came up with a very good set of recommendations. All of these texts highlight different aspects of that question and all of them have very good points. I personally have, over the years, arrived at a rather sociological, not to say holistic point of view. In fact I answered the exact same question a few days ago, and the answer I gave then was this: After ~10 years of women in tech advocacy I d say the ultimate and final measure to get more women into Free Software is by finally achieving a truly equal society and at the same time dramatically improving child care support in almost any country. I ve come to the conclusion that what really holds women back in practice is not so much a lack of skill or interest but a simple lack of opportunity. For most of us Free Software is what we do in our spare time and that s something that women, even today, have considerably less of than men. Even in couples where both partners work full-time it is still mostly the woman who does the majority of the housework and child care duties. In most cultures men have a perceived right to their leisure time that does not to the same degree exist for women. That is one major reason, the other is instilled modesty, which has kind of become my personal arch-enemy by now. I ve talked to so many girls and women at all sorts of events about why they won t take up Computer Science studies or join a Free Software project and the answer I hear most often is that they do not consider themselves good enough in one or another aspect. Sometimes they will doubt their technical skills, sometimes their language skills, sometimes their stamina. Needless to say these girls and women were not any less qualified than the people already active in Free Software. So, yes, in the short and medium term making Debian a more welcoming and friendly place is the way to go. As many others pointed out already this will not only benefit prospective contributors but the community as a whole: those new to it as well as those who ve been in it for a long time. In the long term however what we need is empowerment! Women who are just as confident about their skills as men and are not discouraged by uncooperative environments. This is of course something that is culturally deep-rooted and can only happen in a very large time frame. So, for the moment the way to go in my view is accessibility: a cooperative atmosphere, a code of conduct, comprehensive documentation not only of technical aspects but also of structures and processes. The other thing we need to do is to have as many already active women as possible attend as many Linux/Debian/Free Software/Whatever events as possible. In my experience it happens quite often that other women see these women, feel very inspired by them, get to talk to them and then a few days later show up on some mailing list or IRC channel. From what I ve seen personal contact still beats any other kind of recruiting measures. You re a Debian developer but you re also married with a Debian developer (Alexander Reichle-Schmehl). Did you meet because of Debian? If not, who introduced Debian to the other one? We did in fact meet because of Debian. More specifically during our booth shift at the Debian booth at LinuxTag 2005, where I did a talk on the debian-women project and Alex organised the DebianDay. After that our relationship developed pretty much along our Debian activities: After our initial meeting we talked a lot on #debian.de, when Alex went to DebConf5 and I didn t we noticed that we kind of missed each other. The first gift he ever gave me was a Debconf5 shirt and a box Finnish chocolates (I still have one of them today. ) Our first secret kiss was at ApacheCon 2005, where we were both staffing the Debian booth (kudos to abe for pretending not to notice). We then became an official couple at Berlinux 2005 where we were both staffing the Debian booth and giving talks on packaging and user motivation. Our first real relationship stress test was when we both joined the DebConf6 orga team. It was a stressful time, but we passed it with flying colours! About a year later we announced our engagement via planet.debian.org. Our wedding was a veritable MiniDebConf, one of the best gifts we got was a Debian cookbook including the favourite recipes of DDs from around the world. By now we ve both finished university and work full-time jobs, so we don t do as many talks and attend as many Debian events as we used to. Instead we now mainly focus on press and publicity work, which is quite practical to work on as a pair. It s actually rather funny that way, Alex and I get confused with each other quite often, since we have almost the same name, often pick up on each other s E-Mail conversations and are most often quoted by our function rather than by name. Because of we have kind of merged into this virtual Debian Press Person in the perception of many of our contacts. You also have another hat : Debian Press Officer. What is this about? What would you suggest to people who would like get involved in that domain? Debian press work is mainly about providing an official and coordinated point of contact to anyone wanting information from or about Debian. The press team answers all sorts of inquiries (the most popular one is is of course always the next release date) and makes sure all important events and developments within Debian receive the attention and recognition they deserve. Debian is a diverse project where every sort of contributor is free to voice his or her opinion in any way. We don t have NDAs or prescribed terminology. That s one of the things I love about Debian but also something that makes us difficult to handle for conventional media. They want official statements, in generally understandable terms, at appointed times. That s what the press team takes care of. Almost all of the press work is done in the publicity team, which coordinates using IRC, Mail and SVN. The publicity team also publishes the Debian Project News, which are very popular among our users and developers. Press work is also an area of work that offers lots of possibilities for non-technical contribution. http://wiki.debian.org/Teams/Publicity lists a number of possibilities for contribution and, like most Debian Teams, we d be more than grateful to get some more helping hands and happy to introduce interested newcomers to our work. What s the biggest problem of Debian? In my view: Overwork. Debian has thousands of contributors but still a lot of the main work rests on very few shoulders. We need more contributors, especially, but not only in the key teams. In order to get more people we need to do some marketing which is very hard for us, since we are very proud of our independence and have a strong focus on purely technical aspects rather than aiming for popularity. However, with the current amount of Open Source and Free Software projects to join we find ourselves not only in a contest on technical excellence but also a sort of popularity contest that is about perception rather than hard facts. This popularity contest is difficult for Debian and currently costs us quite a bunch of very capable people. Do you have wishes for Debian Wheezy? My answer to that is a non-technical one: I think Debian is currently very under-appreciated, we do a lot of great work and maybe even more importantly we do a lot of important work for Software Freedom, sometimes even at the cost of our above-mentioned popularity. I hope people will appreciate that more again in the future. Is there someone in Debian that you admire for their contributions? Over the years I have made a lot of friends within the Debian community, some have even become family. That makes it somewhat hard to single out individual people. I think what I admire most is continuous commitment. I am very impressed by those among us who have kept up a high level of commitment over many years and at the same time managed to bring that in line with a fulfilled personal/family life. That s something that I hope I ll also be able to achieve in the years to come.

---

Thank you to Meike for the time spent answering my questions. I hope you enjoyed reading her answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

2 comments Liked this article? Click here. My blog is Flattr-enabled.

While we are waiting for Lintian 2.5.0~rc3, here is a little update on what you can look forward to. About a week ago I had the pleasure of merging my branch into the Lintian master branch. While the IRC bot DOS ed the #debian-qa channel[SRY], cross-package checks became part of the official Lintian. Beyond the reduced amount of binary-without-manpage false-positives, circular-dependency checking and a partial broken symlink detector, the 2.5.0~rc3 release will also feature java and multiarch checks (kudos to Vincent Fourmond and Steve R. Langasek, respectively). The new java checks are based on the Java Policy and currently mostly covers checking for possibly build misconfigurations (jars without class files etc.), weird classpaths and missing Main-Class/depends on jarwrapper for executable jars. Note that #539315 has not been fixed, so executable jars will still trigger a warning. For people wanting to upgrade their libraries to multiarch, Lintian will have a check to see if you remember the Pre-Depends on multiarch-support. Personally I have not followed the progress of multiarch too closely, but last I checked multiarch-support was not in unstable. I guess we are ahead on this one. Other highlights include dont install stuff in /run , armhf support, less false positive on missing B-D on dh_python 2,3 and recognition of 3.9.2 as the latest version of the Policy Manual. I am also planning on starting a new branch and this time I intend to work on Vendor-based customization of Lintian. If you have any ideas/suggestions, by all means follow up on the thread on the Lintian list. [SRY] Sorry to anyone present in the channel!

I have noticed some problems with how Debian is using the popularity-contest data.

• popcon units are unknown Using the popcon score of a package to measure its use is like using the bleeple score of a trip to measure its distance. Both scores have no sensible units attached, though they may be loosely derived from a unit value. Is a trip with a bleeple score of 99 a long trip? Is a package with a popcon score of 99 a rarely used package? The only way to resolve this ambiguity at all is to compare ratios of values, so the problimatic units cancel out. A flight from NYC to AMS with a bleepie score of 99 is 50 times as bleepie as my drive home, which scores 2. So, any statement like "low popcon score" is basically so lacking in context as to be meaningless. Such statements are deprecated, and should be ignored.
• not all popcon scores are comparable The above example is intentionally bad. Plane flights and car trips are not very comparable when you don't know what units (time / CO2 / distance / number of people sharing a confined space / security theater points) are being used. Similarly, comparing a high popcon package like gnome-terminal with a relatively low popcon package like udhcpc is very deceptive. The former is installed by default in the desktop task, but plenty of desktop users would not miss it. The latter is installed only on embedded systems, which can exist in absurd numbers, and none of which will tend to report to popcon. So, any attempt to compare popcon scores should include a rationalle about why the two scores are comparable. For example, gnome-terminal and rxvt are somewhat comparable since they are both terminal emulators. But, only the vote scores, not the inst scores should be compared, since gnome-terminal is installed by default. dhcp3-client and udhcpc are not comparable despite being similar packages.
• popcon scores do not measure long tail effects A strength of Debian is that not only commonly used, but also uncommon and niche software is packaged. Popcon does not measure the benefit of some little used peice of software being there, packaged and ready to use when a user needs it. For six years I kept satutils in Debian, despite it probably having no users. It has a very specific use case, to control a motorized internet satellite dish typically installed on an RV. I did that because it was essentially no work (the package was approximatly bug free, and required no changes since 2007), and because of the possible payoff if someone needed this thing and there it was, in Debian. The value of Debian in that occasion would spike to a value that, while not directly comparable with a popcon score, would be pretty epic, for that one user, as they pushed arrow keys to move a satellite dish around. (It also had the best WITHOUT WARRANTY statement I've had the pleasure to write: "If you break your dish off your vechicle using this software, you get to keep both pieces.") Every removal of a package for "low popcon score" runs the risk of silently degrading this overall value of Debian.
• who wants to be popular? Part of the problem is that popcon has been around long enough that the connotations of its name, "popularity contest" have been dulled by repetition (and abbreviation). Popularity contests are not pleasant things. They rarely reach the best result. They embody the tyranny of the majority. The name was originally, to the best of my knowledge, chosen exactly to imply all these failings, to say that hey, popularity-contest is deeply flawed, but is better than nothing for this one specific use case (ordering packages to place on CD sets). We no longer think of popcon with these caveats. That is a regression in your brain. Fix it. By removing packages that appear unpopular, we run the risk of Debian becoming bland and homogenous.

Generally speaking it is a really bad idea to hold passwords in cleartext. I am actually amazed people still do this! The standard way of holding passwords that has been around for years is to encrypt or hash the password and store the result, called a ciphertext. There have been many ways of hashing the password, starting off with plain old crypt with no salt (a random pair of characters) then crypt with salt through to MD5 and SHA. The thing is, each one of these hashing techniques results in a ciphertext in a different length. Now with most languages, this doesn t matter because you know what hash you are using; its simply the name of the function or some flag you set. PHP is different, because all of these methods use the one function called crypt which is a little confusing because it is more than plain old crypt. Around the PHP version 5.3 the developers started putting in the more complex hash algorithms which is good, but the ciphertext has been growing. A lot of applications store this hashed password in a database and the decision needs to be made; how big should this field be? For a long while, 50 characters would be enough and this is what programs like jffnms use. Unfortunately the SHA-512 algorithm needs 98 characters so what you get is half a hash stored. When the user enters their password, the program compares the full hash from that password to the half hash in the database and it always fails. I ve adjusted JFFNMS to use 200 character long fields which is fine for now. The problem is who knows what the future will bring?

Related articles

• France to require cleartext password storage (boingboing.net)
• Are You Sure SHA-1+Salt Is Enough For Passwords? (it.slashdot.org)
• What All This MD5 Hash Stuff Actually Means [Technology Explained] (makeuseof.com)
• Serious Security Flaw in Amazon.com Passwords (techie-buzz.com)

The password-gorilla package has lacked some love since a while and at some point in time I orphaned it.
That happened due to the fact, that the upstream author was pretty unresponsive and inactive and my own TCL skills are very limited. As a result password-gorilla package was in a bad state, at least from a user point of view, with several (apparently) random happening error message and alike, stalling feature development etc.

But in the meanwhile there was a promising event arising. A guy, named Zbigniew Diaczyszyn, wrote me a mail that he intended to continue upstream development. Well, meanwhile is kind of an understatement. That first mail already happened in December 2009. And he asked me, if I'd like to continue maintaining password-gorilla in Debian. I agreed to that, but as promising as it sounded to have a new upstream, I was not sure if that would work out. However: My doubt were not justified.

In the time between 2009 and now Zbigniew managed to become the official upstream (with the accreditation of the previous upstream), create a github project for it and make several releases.


I know there are several people out there who tested password-gorilla. I know there were magazine reviews including the old version, which were a bit buggy with recent tcl/tk versions. It made a quiet good multi-platform password manager, with support for very common password file formats, stand in a bad light.
I recommend previous users of password-gorilla to try the new version, which recently has been
uploaded to unstable.

After two weeks of work, I've just released git-annex version 0.20110417, with some big new features that open up some interesting use cases:

• Want to use The Cloud as a git remote? git-annex now supports storing data in Amazon S3 as if it were just another git remote. With full gpg encryption of the content stored in S3. * This is so easy to use and neato I just have to show it off:

joey@gnu:~/tmp/repo> git annex initremote cloud type=S3 encryption=joey@kitenet.net
initremote cloud (checking bucket) (creating bucket in US) ok
joey@gnu:~/tmp/repo> git annex add bigfile
add bigfile ok
(Recording state in git...)
joey@gnu:~/tmp/repo> git annex move bigfile --to cloud
move bigfile (gpg) (checking cloud...) (to cloud...) ok
(Recording state in git...)
joey@gnu:~/tmp/repo> file bigfile
bigfile: broken symbolic link
joey@gnu:~/tmp/repo> git annex get bigfile
get bigfile (copying from cloud...) (gpg) ok
(Recording state in git...)
joey@gnu:~/tmp/repo> file bigfile
bigfile: symbolic link

• Want to collaborate on some big files? Perhaps you're dealing with scientific datasets, or video game levels. git-annex can now use a bup repository as a special kind of git remote. The big files are stored forevermore in bup's git repository, while their metadata and the rest of your stuff can be kept in git as usual -- and both git repositories can be used collaboratively. It's turtles^Wgit all the way down, but without the large file scalability problems.
• Want to trade disk space with someone? You can set up a bup remote, or just bare directory on their system, and have git-annex encrypt the data it stores there.
• Want a DropBox like folder that's Free Software and stores data in the cloud, or in a more git-style distributed way? Well, about that... I had been planning to finish up S3 and encryption support, and then mention this could perhaps be used as the basis for a DropBox like thing. But Christophe-Marie Duquesne anticipated me by a month, and created ShareBox. It's a FUSE filesystem layered over git-annex. While not yet production ready (lacking eg, conflict resolution), it has promise.

---

* This feature isn't available in Debian yet, blocked by a lack of the Haskell hS3 library packaged for Debian. Someone should fix that, ideally not me. Getting missingh updated for the ghc7 transition so git-annex is buildable in unstable would also be nice..

I ve now had a Sony Ericsson Xperia X10 for almost two months (here is a link for my first review) [1]. This is a phone that people seem to really dislike because the battery life is poor and Sony doesn t allow replacing the kernel. I m happy with mine, happy enough that after buying one for my wife and trying it out I bought one for myself. I knew about it s problems in advance and wanted a relatively cheap phone with a large high resolution screen, and the Xperia X10 was the best match for my criteria.ChargingThere has been a recent trend towards using USB for charging devices. Sony provides a tiny wall-wart PSU which has a USB socket and a short (1m) USB cable that can be used to charge the phone from it s PSU or from a PC. The size is really convenient, as the phone has a short battery life I ll probably want to take the PSU to more places than I would for other phones I ve owned.The short USB cable saves weight and tangle when travelling, but can be inconvenient. I m currently working on the phone while it s plugged into my laptop, that works well and I can make calls while it s connected. If the phone was connected to a USB port on a tower computer that was on the ground then the cable could be too short to make a call, if it was charging on a power point near the floor then I wouldn t even be able to use the computer functionality let alone make calls without kneeling. My Viewty has a 175cm charging cable which alleviates these problems. I ve got a 50cm USB extension cable that I use for charging my phone while I m in bed that is just long enough to allow me to check my email without getting out of bed!The socket for connecting the USB cable is protected by a plastic lid that is connected by a strip of rubber. For a socket that will be used at least twice a day this isn t a good mechanical design. The plastic slide on the LG U990 Viewty seems like a much better design.As an aside it s a pity that they aren t designing cars with USB charging sockets built in. Some new cars have a socket for one USB flash drive to be attached to the radio, but really they need at least one USB charging port per seat. It would also be nice if they made power points with USB charging sockets built in, I d buy a few of those if they made them.Preserving the BatteryI ve been running Juice Defender to reduce the battery use, at the moment it is claiming to have extended battery life by 69%. That combined with turning off things that aren t needed (such as WiFi) has made the phone reasonably usable. I can survive more than 12 hours during which I use the phone a lot without charging it.XperiaX10.net has an interesting review of replacement batteries [5]. I m not going to buy them because the largest battery requires a replacement back case which prevents using an external case to prevent damage and the smaller one doesn t provide enough of a benefit and the design of the phone makes it difficult to change batteries so carrying a spare battery isn t a good option.Cursor Control KeysThe HP/Compaq iPaQs that I own from ~10 years ago have a single button that can be moved up/down/left/right to act as a cursor, it can also be pressed inwards to act as an ENTER key (or whatever the application might want for a fifth function). Having some sort of hardware cursor control is really handy, I often end up deleting several characters when I want to replace one because getting the cursor onto the desired character is too difficult.Adding more hardware keys would require making the phone bigger, but that would be fine by me. As described in my previous post about phone cameras I d like to have a phone that s thicker to have a better camera with a greater focal distance for a larger lens [2].Core Phone FunctionalityI really miss having separate green and red buttons for making/answering calls and for rejecting/ending calls.A really common operation is to call back the last person who called. To do this on my LG U990 Viewty I press the green button twice on my home screen which took less than a second. On the Xperia this involves selecting the phone icon from the home screen, then the call log icon, then the call that is desired, then the Call user line. That is four presses in different parts of the screen compared to pressing the same hardware button twice. I m investigating dialer applets right now, the Dialler One applet has a good interface for calling people who have called or been called recently but it still requires two presses on different parts of the screen. With my old Viewty I could call back the last person by feel without even looking at the phone!In the last 6 years the standard functionality of phones has been to include multiple profiles for noises. My Viewty has profiles named Normal , Silent , Vibrate only , Outdoor , Headset (automatically selected when a headset is connected), and three customised ones. The Xperia has only one setting, and that is three icon presses away from the home screen. It does allow changing the volume by hardware switches on the side which includes going to vibrate-only and silent mode. While this is useful, it s not the best way that two precious hardware keys could be used. It also doesn t allow control over all the different notifications, I d like to be able to activate a noise profile and have every application respond to it in an appropriate manner. This is a deficiency in Android 2.1 not in the phone itself.In many ways this phone has the worst phone functionality of any mobile phone I ve ever owned, I think that this is more the fault of the Android designers than Sony Ericsson.Android UpdatesSony had previously claimed that they wouldn t support Android later than 2.1. Now Sony has announced that they will support Gingerbread Android 2.3 [3]. So one of the major complaints about the Xperia will soon be addressed.TetheringThe build of Android that the Xperia runs at the moment doesn t support Wifi tethering. I m currently using with Easy Tether for USB tethering [4]. It s not free software and requires it s own code to run as root on your Linux system that is being tethered (which is easily locked down with SE Linux), but it basically works. The down-side with Easy Tether is that it proxies all the connections so you can t run traceroute etc and in the free version you can t use UDP.K9 MailK9 Mail fixes some of the problems in the default Android email program that I described in my previous post. It allows selection of SSL with a default port of 993 for IMAPs. It uses mail.example.com as the mail server address for an email address of user@example.com (so I ll add a mail CNAME to the domains I run). When I connected initially it told me that the SSL key was not signed by a CA and asked if I wanted to save it or reject it this is the correct and desired functionality. It also correctly parses URLs from the email (or at least has fixed the bug that I discovered in the default email app).One problem I ve found with K9 is that it seems to timeout on large folders. This is probably partly the fault of Virgin Mobile being slow for IP access, but I wonder whether K9 doesn t pipeline IMAP commands as Virgin can do bulk transfer at reasonable speeds (80KB/s) and it mainly fails on latency (900ms being typical). The result is that a folder with more than 500 messages that need to be copied to the phone will never get synchronised. When I started reading mail on my phone I had to move mail from some of the bigger folders into other folders to avoid timeouts. As an aside the amount of time I ve saved by reading email on the go has already paid for the phone.MediascapeThe Sony Mediascape software is used for categorising photos. One function of this is to assign names to photos, the names come from the contact list so if you photograph someone who you can t phone then you need to add a contact list entry for them. This also means that you couldn t conveniently add names to non-humans, I guess I could have added a contact list entry Mr Crash Dump for photos of system crash logs. But a bigger problem is that it decides what is a picture of a person, a picture of someone who is not centered in the photo or a profile picture can be regarded as not a photo of a person and therefore not subject to being associated with a contact list. Finally when selecting a name for a picture it displays the entire contacts list in a small font instead of displaying favorites, I have the phone numbers of many clients in my contacts list who I will never photograph Mediascape just isn t much good. If I feel the need to do something serious in this regard I ll search the app store for something free that s better.ProtectionFor $15 each I bought one grey and one clear Gel Case from J2K. This case covers the sides and back of the phone with a firm rubber layer that will hopefully allow it to bounce rather than break. It also extends slightly higher than the screen which should stop the screen being scratched if the phone is left face-down on a hard surface on the vibrate setting.One problem with the Gel Case is that by it s design it has to cover the buttons on the sides, due to a design or manufacturing problem the clear case that my wife uses can press slightly on the shutter button which disables the three main buttons on the front. So when the menu or back button stops working she has to slightly move the Gel Case out from the side. I m thinking of just cutting out a section of the Gel Case there, that will make it difficult to press the shutter button, but you can use the touch screen to take photos and there doesn t seem to be any other use for the shutter button so this shouldn t be a problem. The clear and grey cases have different designs, the grey one has a hole over the raised Sony logo which makes it fit a little better as well as not having the shutter problem, so it seems to be a later design I don t know whether all clear cases have the same problem but I recommend that someone who only has one Xperia X10 get the grey case just in case.Web BrowsingI m experimenting with using my own web proxy to compress the data sent to the phone. Unfortunately the Android settings for a web proxy only apply to the main web browser, not to all the other applications that use web services. Also the built-in web browser requires pressing the settings hardware button followed by at least two touch-screen presses to change between windows or close a window and it doesn t seem to support making the short-press action be to open a link in a new window. So I will have to find another web browser to use.Bandwidth UseThe 3G Watchdog applet is really good for tracking the data transfer and optionally cutting off 3G access before the quota is exceeded. Unfortunately Virgin has already sent me a bill with $234 in extra charges for bandwidth use because I used 1.6G instead of my 1.5G quota. A Virgin representative had told me over the phone that the billing period would be based on the end of the month, so while every calendar month has had less than 1.5G used because I used more than average in the end of March and the start of April that counts as excess data in the 14th March to 13th April period.I m going to appeal this to the Telecommunications Industry Ombudsman, I don t think that it s reasonable that I should be billed almost 7* the normal bill for exceeding the quota by 7.4% based on following the instructions of a Virgin employee. While I could have read the previous bill that they sent me to discover the end of billing period dates, I don t think that I should be expected to distrust everything that a Virgin employee says. If they billed me an extra $20 then I d just pay it, but $234 is unreasonable.ConclusionSince using the phone I ve found significant benefits in web browsing and reading email with the major limitation being the small screen (relative to a Netbook or Laptop). So if I was to buy another phone I would probably consider a Dell Streak which seems to be the largest Android phone on the market at the moment.Given the amount of use that I ve got I would be happy to spend more money and therefore consider a more expensive phone. But I don t regret the decision to save money by getting an Xperia X10.Finally when a telco tries to stick me with a $234 excess charge it really detracts from the value of having a phone contract. If I end up having to pay that then it s half the value of a smart-phone lost in one telco scam. This is enough to make a contract with Virgin a bad option, after this contract expires I may use VOIP and a pre-paid SIM from Telstra NextG if their network is still the best. Another possibility is to just use a small tablet and skip having a mobile phone, email and Jabber plus SMS from the people who lack net access will probably do.

• [1] http://etbe.coker.com.au/2011/02/18/xperia-x10i-first-experience/
• [2] http://etbe.coker.com.au/2010/11/26/quality-of-cameras-in-phones/
• [3] http://blogs.sonyericsson.com/products/2011/03/25/update-on-xperia%E2%84%A2-x10/
• [4] http://www.mobile-stream.com/easytether/android.html
• [5] http://www.xperiax10.net/2010/08/02/mugen-power-xperia-x10-battery-review-%E2%80%93-part-1/

Apparently I ve won the lottery. What is even more amazing is that it is one based not in Australia but in the UK ELO (England Lottery Organisation) and I didn t even buy a ticket. Even more amazing is even though this organisation is based in England, they don t write English very well; perhaps its declining school standards. They re so concerned about giving you the maximum return on the dollar (or pound) they don t even use a proper co.uk email address but a free webmail from umail. It is, of course a scam. Popularly known as Nigerian 419 or advanced-fee fraud. You can win the money but.. well it seems there is some holdup and you need to pay some release fee or some bribe to get your dollars. What makes me a little sad is it was for only 250,000 UK pounds. I feel ripped off as a few google searches showed people being offered over 500,000 pounds on the same scam. Don t these crooks know I have a high aussie dollar exchange rate to overcome? About the only interesting thing about it was that my dspam filters missed it but they ve now been retrained with that miss. I think sending it as a pdf was why it made it through. And I now cannot get ELO (Electric Light Orchestra) songs out of my head, thanks a lot scammers! (It s a livin thing, ya know)

Related articles

• FBI Warns of Spam, Lottery Scams and Potassium Iodide Price Gouging (walletpop.com)
• Detecting scams (cyberdefend.wordpress.com)
• Getting Scammed, Doesn t Make You an Idiot, Stupid (socyberty.com)
• Jamaican Lottery Scam: Woman loses $370,000 in Jamaican Lottery Fraud!! (darnellthenewsman.com)

New scratching post

Vi, taking a break as I make the bed.

Update: I reported the deluge issues and added a new problem about the lack of click on tapping on the touchpad.

---

I know that installation reports and upgrade reports need to be submitted to the BTS, but I just want to point out some issues for people that might hit the same issue on upgrade from lenny to squeeze.

But before that I must say I like the new Debian site(s) look.

Issue number one: When upgrading from lenny's deluge to squeeze's deluge, the new version of the app is quite different from the prevoious version. Here are some things to take into account:

• when starting for the first time the new version it will take ages to check (and probably recalculate some checksums); make sure you don't mind the I/O activity when you start it; a torrent will not be available until this check is done
  
• the new version relies on a client-server model which is disabled if you use the "classic mode" (Edit->Preferences->Interface)
• some features available in the old version are not available in the new version (e.g.: graph for traffic and embedded search function)
• some features are availbale as modules, but there are just a few modules
• when starting the graphical interface in the new mode, don't use the "Start daemon" button, use the "Connect" button. If you start the daemon via the "Start daemon" button the "Connect" button will become "Disconnect" although the client is NOT connected. Using the "Connect" button directly solves the problem.
• There is an ugly side panel (on the left side) which, IMHO has no real function or use, except filtering in the view the active downloads or similar things
• when choosing the place to save a torrent and trying to set that place as the default location, deluge will not remember that setting
• closing now (with the daemon option) the app will close just the client, but the server/daemon will remain in the background, unless is explicitly closed (there is a dedicated menu entry)
  

Issue number 2: I seem to experience some weird glyph/graphic area reshuffling in GNOME (maybe X?) after recovering from hibernate. With a total (according to my current sloppy counting) 5 resumes, things get back to normal, but I suspect the next hibernate-resume cycle will restart the glyph reshuffling cycle.

By glyph/graphic area reshuffling I mean alterations of the shapes of the glyphs (and some areas on the background picture) in such a manner that it seems that within a set of 8/16/N (?) lines are shifted/rotated sideways with some undefined and different ammount each, but in a reproducible manner ("b" will always be doodled in the same way, no matter if is in the word "be" or if is in the word "absurd".

I'll try to provide some picture in the bug report, once I report this issue in BTS.

Update: I reported this in Debian's BTS and upstream (with screenshots, too). I am not convinced the problem is in the kernel since I tried an old kernel and saw the same issue. I suspect X is at fault.

Issue number 3: This is more like a convenience issue. In the past I was using "hibernate" with uswsusp which I understand is now broken beyond repair and replaced by pm-utils. The thing I miss in the hibernate process now is the ability to abort the hibernation process as it was possible in lenny's uswsusp by pressing the backspace key during the storing of the state on disk phase.


Issue number 4: The upgrade process was quite tedious because once I tried upgrading aptitude, python2.6 was pulled in and almost all apps ended up needing upgrading due to the chaining of necessary package upgrades.


Issue number 5: For some weird reason pulseaudio was initially installed making playback of any audio impossible (the apps wanting to emit noises would just freeze but they were TERM-inatable) and later I've seen a default null audio sink was defined for me. Killing pulseaudio and removing the ~/.pulse directory fixed the issue (but I should probably see why pulseaudio didn't work properly in the first place because I suspect the problem will reappear at the next restart - I usually use hibernate).

Issue number 6: Tapping the touchpad no longer results in a click. Maybe some packages got removed? And, no, it is the same when I remove the external mouse, so it is not because of some smart behaviour of that sort.

Otherwise I am quite satisfied with the result of the upgrade. A huge "thank you" to all people involved in the development of squeeze.

We're heading to release candidates for phpMyAdmin 3.4.0 (probably once GSoC application/evaluation rush period is over), so the translators have less time to make the translations better. There is no new complete translation, so we still have 14 translations at 100%:

• Catalan
• Chinese (China)
• Czech
• Dutch
• English (United Kingdom)
• French
• German
• Greek
• Italian
• Russian
• Slovenian
• Spanish
• Swedish
• Turkish

There are still lot of translations which would need some love, for example:

• Finnish
• Norvegian
• Polish

Recently two of our prospective GSoC students started to contribute to translations, so there is great progress in Sinhala and Hindi translations. Looking forward to have these completed soon :-). You can also help make translation to your language better! If your language is missing just let me know (either in comments or by email).

Filed under: English Phpmyadmin 0 comments Flattr this!

A brand new 0.9.4 release of Rcpp is now on CRAN and Debian. This version contains an improvement to loading and initialization of Rcpp modules, a bug fix for vectors of factors, another build issue fix as well as (per common practice with JSS) citation information for the article Rcpp: Seamless R and C++ Integration which is now Volume 40, Issue 8 in the Journal of Statistical Software (or JSS for short). The complete NEWS entry is below; more details are in the ChangeLog file in the package and on the Rcpp Changelog page.

0.9.4   2011-04-12
    o   New R function "loadRcppModules" to load Rcpp modules automatically
        from a package. This function must be called from the .onLoad function
        and works with the "RcppModules" field of the package's DESCRIPTION file
    o   The Modules example wrapped the STL std::vector received some editing
        to disambiguate some symbols the newer compilers did not like
    o   Coercing of vectors of factors is now done with an explicit callback
        to R's "as.character()" as Rf_coerceVector no longer plays along
    o   A CITATION file for the published JSS paper has been added, and 
        references were added to Rcpp-package.Rd and the different vignettes

Thanks to CRANberries, there is also a diff to the previous release 0.9.3:

Diff between Rcpp versions 0.9.3 dated 2011-04-05 and 0.9.4 dated 2011-04-12
 Rcpp-0.9.3/Rcpp/build                                              only
 Rcpp-0.9.3/Rcpp/inst/skeleton/yada.Rd                              only
 Rcpp-0.9.3/Rcpp/inst/unitTests/testRcppModule/R/Modules.R          only
 Rcpp-0.9.3/Rcpp/inst/unitTests/testRcppModule/man/yada.Rd          only
 Rcpp-0.9.4/Rcpp/ChangeLog                                             55 
 Rcpp-0.9.4/Rcpp/DESCRIPTION                                           10 
 Rcpp-0.9.4/Rcpp/NAMESPACE                                             29 
 Rcpp-0.9.4/Rcpp/R/Rcpp.package.skeleton.R                             11 
 Rcpp-0.9.4/Rcpp/R/loadRcppModules.R                                only
 Rcpp-0.9.4/Rcpp/inst/CITATION                                      only
 Rcpp-0.9.4/Rcpp/inst/NEWS                                             17 
 Rcpp-0.9.4/Rcpp/inst/doc/Makefile                                      2 
 Rcpp-0.9.4/Rcpp/inst/doc/Rcpp-FAQ.pdf                              binary
 Rcpp-0.9.4/Rcpp/inst/doc/Rcpp-FAQ/Rcpp-FAQ.Rnw                       153 +-
 Rcpp-0.9.4/Rcpp/inst/doc/Rcpp-extending.pdf                        binary
 Rcpp-0.9.4/Rcpp/inst/doc/Rcpp-extending/Rcpp-extending.Rnw             2 
 Rcpp-0.9.4/Rcpp/inst/doc/Rcpp-introduction.Rnw                       734 ++++------
 Rcpp-0.9.4/Rcpp/inst/doc/Rcpp-introduction.pdf                     binary
 Rcpp-0.9.4/Rcpp/inst/doc/Rcpp-modules.pdf                          binary
 Rcpp-0.9.4/Rcpp/inst/doc/Rcpp-modules/Rcpp-modules.Rnw                36 
 Rcpp-0.9.4/Rcpp/inst/doc/Rcpp-package.pdf                          binary
 Rcpp-0.9.4/Rcpp/inst/doc/Rcpp-package/Rcpp-package.Rnw                 4 
 Rcpp-0.9.4/Rcpp/inst/doc/Rcpp-quickref.pdf                         binary
 Rcpp-0.9.4/Rcpp/inst/doc/Rcpp-quickref/Rcpp-quickref.Rnw              34 
 Rcpp-0.9.4/Rcpp/inst/doc/Rcpp-sugar.pdf                            binary
 Rcpp-0.9.4/Rcpp/inst/doc/Rcpp-sugar/Rcpp-sugar.Rnw                     2 
 Rcpp-0.9.4/Rcpp/inst/doc/Rcpp-unitTests.pdf                        binary
 Rcpp-0.9.4/Rcpp/inst/doc/Rcpp.bib                                     73 
 Rcpp-0.9.4/Rcpp/inst/doc/unitTests-results/Rcpp-unitTests.html        18 
 Rcpp-0.9.4/Rcpp/inst/doc/unitTests-results/Rcpp-unitTests.txt         40 
 Rcpp-0.9.4/Rcpp/inst/include/Rcpp/config.h                             2 
 Rcpp-0.9.4/Rcpp/inst/skeleton/zzz.R                                   11 
 Rcpp-0.9.4/Rcpp/inst/unitTests/runit.Module.client.package.R          27 
 Rcpp-0.9.4/Rcpp/inst/unitTests/runit.Vector.R                         18 
 Rcpp-0.9.4/Rcpp/inst/unitTests/testRcppModule/DESCRIPTION              2 
 Rcpp-0.9.4/Rcpp/inst/unitTests/testRcppModule/NAMESPACE                3 
 Rcpp-0.9.4/Rcpp/inst/unitTests/testRcppModule/R/zzz.R              only
 Rcpp-0.9.4/Rcpp/inst/unitTests/testRcppModule/src/rcpp_module.cpp      6 
 Rcpp-0.9.4/Rcpp/inst/unitTests/testRcppModule/src/stdVector.cpp       18 
 Rcpp-0.9.4/Rcpp/inst/unitTests/testRcppModule/tests/modules.R         21 
 Rcpp-0.9.4/Rcpp/man/Rcpp-package.Rd                                    6 
 Rcpp-0.9.4/Rcpp/man/loadRcppModules.Rd                             only
 Rcpp-0.9.4/Rcpp/src/r_cast.cpp                                         9 
 43 files changed, 647 insertions(+), 696 deletions(-)

As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

I've been running at capacity for a while now, and something went "snap" recently, so I decided to take a few days off to clear my head a bit. One of the things that came to mind is that I have too many 'side project' things. I have had lots of fun (and a few flame wars) over the years with LangPop.com but I decided that it's another thing that should probably go, in order to simplify my life some. It's a fairly popular site amongst programmers, as I think it's the best of the bunch in terms of guaging an admittedly tricky subject like language popularity, but there's lots of room to add to it. I've got various ideas for whoever ends up buying it! About money: it doesn't earn a lot. Programmers, I think, are fairly blind to advertisements. In the right hands I think it could do better, but most likely, it would work best as a bit of branding/advertising for your own business, much as TIOBE's index has spread their name far and wide. For the time being, I'll accept private bids, and depending on how well that does or doesn't go, may consider using something like Flippa.com later. Write me at davidw@dedasys.com if you're interested. I'd be happy to share some numbers and information with you. Perhaps I'll update this posting with them too. What you get: everything! Historicaly data, code, scripts to manage it, the domain. It could use some cleaning up and love, but it does get the job done. Running it just takes a few minutes a month, so I could hold on to it with no trouble, but I've decided that I want one less thing to think about. Update: Some Numbers In the last year, according to Google Analytics, the site had: 174,645 visits. In the last month, it had 17,047 visits. Revenue? Think of it as zero and you won't be too far off. You would have to rework things significantly to make money from ads on it, I think. Hosting costs? Not sure, particularly, it's on my Linode. It utilizes Ruby on Rails, but creates static HTML pages, so it's fairly cheap.

R 2.13.0 will be released tomorrow. It contains a new package by R Core member Luke Tierney: compiler. What a simple and straightforward name, and something that Luke has been working on for several years. The NEWS file says

    o Package compiler is now provided as a standard package.  See
      ?compiler::compile for information on how to use the compiler.
      This package implements a byte code compiler for R: by default
      the compiler is not used in this release.  See the 'R
      Installation and Administration Manual' for how to compile the
      base and recommended packages.

so the compiler is not yet used for R's own base and recommended packages. While working on my slides for the upcoming Rcpp workshop preceding R/Finance 2011, I thought of a nice test example to illustrate the compiler. Last summer and fall, Radford Neal had sent a very nice, long and detailed list of possible patches for R performance improvements to the development list. Some patches were integrated and some more discussion ensued. One strand was on the difference in parsing between normal parens and curly braces. In isolation, these seem too large, but (as I recall) this is due some things 'deep down' in the parser. However, some folks really harped on this topic. And it just doesn't die as a post from last week demonstrated once more. Last year, Christian Robert had whittled it down to a set of functions, and I made a somewhat sarcastic post argueing that I'd rather use Rcpp to get 80-fold speed increase than spend my time argueing over ten percent changes in code that could be made faster so easily. So let us now examine what the compiler package can do for us. The starting point is the same as last year: five variants of computing 1/(1+x) for a scalar x inside an explicit for loop. Real code would never do it this way as vectorisation comes to the rescue. But for (Christian's) argument's sake, it is useful to highlight differences in the parser. We once again use the nice rbenchmark package to run, time and summarise alternatives:

> ## cf http://dirk.eddelbuettel.com/blog/2010/09/07#straight_curly_or_compiled
> f <- function(n, x=1) for (i in 1:n) x=1/(1+x)
> g <- function(n, x=1) for (i in 1:n) x=(1/(1+x))
> h <- function(n, x=1) for (i in 1:n) x=(1+x)^(-1)
> j <- function(n, x=1) for (i in 1:n) x= 1/ 1+x 
> k <- function(n, x=1) for (i in 1:n) x=1/ 1+x 
> ## now load some tools
> library(rbenchmark)
> ## now run the benchmark
> N <- 1e6
> benchmark(f(N,1), g(N,1), h(N,1), j(N,1), k(N,1),
+           columns=c("test", "replications",
+           "elapsed", "relative"),
+           order="relative", replications=10)
     test replications elapsed relative
5 k(N, 1)           10   9.764  1.00000
1 f(N, 1)           10   9.998  1.02397
4 j(N, 1)           10  11.019  1.12853
2 g(N, 1)           10  11.822  1.21077
3 h(N, 1)           10  14.560  1.49119

This replicates Christian's result. We find that function k() is the fastest using curlies, and that explicit exponentiation in function h() is the slowest with a relative penalty of 49%, or an absolute difference of almost five seconds between the 9.7 for the winner and 14.6 for the worst variant. On the other hand, function f(), the normal way of writing things, does pretty well. So what happens when we throw the compiler into the mix? Let's first create compiled variants using the new cmpfun() function and then try again:

> ## R 2.13.0 brings this toy
> library(compiler)
> lf <- cmpfun(f)
> lg <- cmpfun(g)
> lh <- cmpfun(h)
> lj <- cmpfun(j)
> lk <- cmpfun(k)
> # now run the benchmark
> N <- 1e6
> benchmark(f(N,1), g(N,1), h(N,1), j(N,1), k(N,1),
+           lf(N,1), lg(N,1), lh(N,1), lj(N,1), lk(N,1),
+           columns=c("test", "replications",
+           "elapsed", "relative"),
+           order="relative", replications=10)
       test replications elapsed relative
9  lj(N, 1)           10   2.971  1.00000
10 lk(N, 1)           10   2.980  1.00303
6  lf(N, 1)           10   2.998  1.00909
7  lg(N, 1)           10   3.007  1.01212
8  lh(N, 1)           10   4.024  1.35443
1   f(N, 1)           10   9.479  3.19051
5   k(N, 1)           10   9.526  3.20633
4   j(N, 1)           10  10.775  3.62673
2   g(N, 1)           10  11.299  3.80310
3   h(N, 1)           10  14.049  4.72871

Now things have gotten interesting and substantially faster, for very little cost. Usage is straightforward: take your function and compile it, and reap more than a threefold speed gain. Not bad at all. Also of note, the difference between the different expressions essentially vanishes. The explicit exponentiation is still the loser, but there may be an additional explicit function call involved. So we do see the new compiler as a potentially very useful addition. I am sure more folks will jump on this and run more tests to find clearer corner cases. To finish, we have to of course once more go back to Rcpp for some <emph>real</emph> gains, and some comparison between compiled and byte code compiled code.

> ## now with Rcpp and C++
> library(inline)
> ## and define our version in C++
> src <- 'int n = as<int>(ns);
+         double x = as<double>(xs);
+         for (int i=0; i<n; i++) x=1/(1+x);
+         return wrap(x); '
> l <- cxxfunction(signature(ns="integer",
+                            xs="numeric"),
+                  body=src, plugin="Rcpp")
> ## now run the benchmark again
> benchmark(f(N,1), g(N,1), h(N,1), j(N,1), k(N,1),
+           l(N,1),
+           lf(N,1), lg(N,1), lh(N,1), lj(N,1), lk(N,1),
+           columns=c("test", "replications",
+           "elapsed", "relative"),
+           order="relative", replications=10)
       test replications elapsed relative
6   l(N, 1)           10   0.120   1.0000
11 lk(N, 1)           10   2.961  24.6750
7  lf(N, 1)           10   3.128  26.0667
8  lg(N, 1)           10   3.140  26.1667
10 lj(N, 1)           10   3.161  26.3417
9  lh(N, 1)           10   4.212  35.1000
5   k(N, 1)           10   9.500  79.1667
1   f(N, 1)           10   9.621  80.1750
4   j(N, 1)           10  10.868  90.5667
2   g(N, 1)           10  11.409  95.0750
3   h(N, 1)           10  14.077 117.3083

Rcpp still shoots the lights out by a factor of 80 (or even almost 120 to the worst manual implementation) relative to interpreted code. Relative to the compiled byte code, the speed difference is about 25-fold. Now, these are of course entirely unrealistic code examples that are in no way, shape or form representative of real R work. Effective speed gains will be smaller for both the (pretty exciting new) compiler package and also for our C++ integration package Rcpp. Before I close, two more public service announcements. First, if you use Ubuntu see this post by Michael on r-sig-debian announcing his implementation of a suggestion of mine: we now have R alpha/beta/rc builds via his Launchpad PPA. Last Friday, I had the current R-rc snapshot of R 2.13.0 on my Ubuntu box only about six hours after I (as Debian maintainer for R) uploaded the underlying new R-rc package build to Debian unstable. This will be nice for testing of upcoming releases. Second, as I mentioned, the Rcpp workshop on April 28 preceding R/Finance 2011 on April 29 and 30 still has a few slots available, as has the conference itself.

Spammer Number 2:Tata Technologies (aka myigetit.com) via Streamsend Again, it looks like Streamsend are being spamming bastards. Guilty of: spamming lots of Debian mailing lists. They claim to offer "Online Self-Paced Training for Engineers". I guess that's training in "how to be obnoxious to people on the internet" then. It looks like it's a good idea to simply drop all connections from machines in the mailengine1.com domain. Following up through whois, that's another domain owned by EZ Publishing, the same folks who own streamsend.com. Spamming bastards all over...

Registrations for DebConf11 are now open!

Dear guests, attendees and contributors, we are happy to announce that registrations for this year s Conference are now open. The instructions for registration process can be found under Register page. While the basic registrations will not close until the end of the Conference, sponsored registrations however, have the deadline. If you would like to apply for sponsored attendance, then it is imperative that you do it before 8th May! Should you have any further questions regarding registrations, please mail them at registration@debconf.org.

Call for Contributions!

We invite submissions of proposals for papers, presentations, discussion sessions and tutorials for Debconf11. Submissions don t have to be limited to traditional talks, you could propose a performance, art installation, debate, or anything else. Official submissions will be accepted until May 8th 2011, 23:59 UTC. For more information, please visit Call for contributions page.