A couple of days ago, an email arrived in which I am kindly asked to provide my PayPal coordinates. And before you dear reader start to laugh, it was actually perfectly genuine and came from a very happy user of the Finance::YahooQuote Perl module. DJ Padzensky wrote this well over a decade ago, and I first introduced and then maintained it as a package in Debian for what is now almost twelve years. I have also been acting as upstream maintainer for the last eight or so years providing the occassional patch to keep up with changes at the Yahoo! server side. After a short reflection, I decided to keep it simple. I undusted a wishlist I had once created over at Amazon, yet never publicised. And amazingly, within minutes of minutes of passing the URL on, I received a cheerful email stating that three books are their merry way. Appropriately enough, two were from Springer's useR! series and another one was a brand new GPU programming title. And they all arrived yesterday. I am still stunned. So with that, my time to say thanks! This was a genuinely nice gesture, and I really appreciate it. I have been doing open source work for over fifteen years now, and I would not still be doing it if I did not enjoy it myself. I continue to learn a lot in the process, and have had a chance to meet lot interesting folks. Yet it is also nice to get a little Thanks every now and them.

I did some experiments with the small tool I wrote in previous post in order to gather some startup data about Firefox. It turns out it can t flush directories and other metadata from the page cache, making it unfortunately useless for what I m most interested in. So, I gathered various startup information about Firefox, showing how page cache (thus I/O) has a huge influence on startup. The data in this post are mean times and 95% confidence interval for 50 startups with an existing but fresh profile, in a mono-processor GNU/Linux x86-64 virtual machine (using kvm) with 1GB RAM and a 10GB raw hard drive partition over USB, running, except when said otherwise, on an i7 870 (up to 3.6GHz with Intel Turbo Boost). The Operating System itself is an up-to-date Debian Squeeze running the default GNOME environment. Firefox startup time is measured as the difference between the time in ms right before starting Firefox and time in ms as returned by javascript in a data:text/html page used as home page. Startup vs. page cache

	Average startup time (ms)
Entirely cold cache (drop_caches)	5887.62 0.88%
Cold cache after boot	3382.0 0.51%
Selectively cold cache (see below)	2911.46 0.48%
Hot cache (everything previously in memory)	250.74 0.18%

The selectively cold cache case makes use of the flush program from previous post and a systemtap script used to get the list of files read during startup. This script will be described in a separate post. As you can see, profiling startup after echo 3 > /proc/sys/vm/drop_caches takes significantly more time than in the normal conditions users would experience, because of all system libraries that would normally be in the page cache being flushed, hence biasing the view one can have of the actual startup performance. Mozilla build bots were running, until recently, a ts_cold startup test that, as I understand it, had this bias (which is part of why it was stopped). The Hot cache value is also interesting because it shows that the vast majority of cold startup time is due to hard disk I/O (and no, there is no page faults number difference). I/O vs CPU Interestingly, testing on a less beefy machine (Core 2 Duo 2.2GHz) with the same USB disk and kvm setup shows something not entirely intuitive:

	Average (ms)
Entirely cold cache	6748.42 1.01%
Cold cache after boot	3973.64 0.53%
Selectively cold cache	3445.7 0.43%
Hot cache	570.58 0.70%

I, for one, would have expected I/O bound startups to only be slower by around 320ms, which is roughly the hot cache startup difference, or, in other words, the CPU bound startup difference. But I figured I was forgetting an important factor. I/O vs. CPU scaling Modern processors do frequency scaling, which allows the processor to run slowly when underused, and faster when used, thus saving power. It was first used on laptop processors to reduce power drawn from the battery, allowing batteries to last longer, and is now used on desktop processors to reduce power consumption. It unfortunately has a drawback, in that it introduces some latency when the scaling kicks in. A not so nice side effect of frequency scaling is that when a process is waiting for I/O, the CPU is underused, making the CPU usually run at its slowest frequency. When the I/O ends and the process runs again, the CPU can go back to full speed. This means every I/O can induce, on top of latency because of, e.g. disk seeks, CPU scaling latency. And it actually has much more impact than I would have thought. Here are results on the same Core 2 Duo, with frequency scaling disabled, and the CPU forced to its top speed :

	Average (ms)
Entirely cold cache	5824.1 1.13%
Cold cache after boot	3441.8 0.43%
Selectively cold cache	3025.72 0.29%
Hot cache	576.88 0.98%

(I would have liked to do the same on the i7, but Intel Turbo Boost complicates things and I would have needed to get two new sets of data) Update: I actually found a way to force one core at its max frequency and run kvm processes on it, giving the following results:

	Average (ms)
Entirely cold cache	5395.94 0.83%
Cold cache after boot	3087.47 0.31%
Selectively cold cache	2673.64 0.21%
Hot cache	258.52 0.35%

I haven t gathered enough data to have accurate figures, but it also seems that forcing the CPU frequency to a fraction of the fastest supported frequency gives the intuitive results where the difference between all I/O bound startup times is the same as the difference for hot cache startup times. As such, I/O bound startup improvements would be best measured as an improvement in the difference between cold and hot cache startup times, i.e. (cold2 - hot2) - (cold1 - hot1), at a fixed CPU frequency. Startup vs. desktop environment We saw above that the amount of system libraries in page cache directly influences application startup times. And not all GNU/Linux systems are made equal. While the above times were obtained under a GNOME environment, some other desktop environments don t use the same base libraries, which can make Firefox require to load more of them at cold startup. The most used environment besides GNOME is KDE, and here is what cold startup looks like under KDE:

	Average startup time (ms)
GNOME cold cache	3382.0 0.51%
KDE cold cache	4031.9 0.48%

It s significantly slower, yet not as slow as the entirely cold cache case. This is due to KDE not using (thus not pre-loading) some of the GNOME core libraries, yet using libraries in common, like e.g. libc (obviously), or dbus.

I put 5 EUR in Flattr each month and I like to spend those among other Debian contributors. That s why I keep a list of Debian people that I have seen on Flattr (for most of them I noticed through an article on Planet Debian). Directory of Debian contributors that you can thank I thought this list could be useful for others so I put it on a web page. Then I realized that limiting this to Flattr was not a good idea, and indeed several developers already propose multiple ways to be thanked. I went back through my list and looked up each developer s website to identify a Thank me page (it can be Donate , Support me , Amazon Wishlist , etc.). Obviously this means that Debian contributors who are not on Flattr do not appear on the initial list even if they have some Thank me page please help me fix this and send me the missing entries if you know of any. Click here to view the directory. The initial listing contains 16 developers and 8 of them have an additional (non-Flattr) Thank me link. Please note the warning I put on the page: the inclusion in the directory should not be taken as an endorsement of the amount or quality of the work done for Debian. You are supposed to make up your own judgment when deciding who you want to thank (but the links can help you learn more about what each contributor is doing). Flattr subscriptions explained Since this article replaces the traditional Flattr FOSS issue for this month, I wanted to introduce a new Flattr feature I recently discovered. With Flattr you have to click on some things every month or your monthly fee is given to a random charity. Now you can avoid this pitfall by subscribing to some things that you like. A subscription acts like an automatic click during a period of 3/6/12 months. If you want to subscribe to something, you just have to click a second time on the Flattr button and you will see this:
Once you clicked on the desired duration, the subscription is recorded and the button will appear like this:
Easy, isn t it? PS: I installed a WordPress plugin to make it super easy to share my articles on the most common social networks.

One comment Liked this article? Click here. My blog is Flattr-enabled.

I have finally reached a decision regarding my involvement in the Debian Ruby packaging efforts. I have decided to stop. This has been a very hard decision to make. I have invested huge amounts of time in that work over the years. I still love the language, and will continue to use it on a daily basis for my own developments. I still hope that it will succeed. I know that some people will be disappointed by that decision (and that others will think your work was useless anyway, people should use RVM and rubygems ). But I also know that I won t be able to push for all the required changes alone. I just don t have the time, nor the motivation. For the record, here are the changes I would have liked to see in the Ruby community. The core Ruby development community should mature.
The core Ruby development community is still dominated by Japanese developers. While not a bad thing in itself, it is easily explained by the fact that the main development mailing list, where most of the important decisions are taken, is in japanese. ruby-dev@ should be closed, and all the technical discussions should happen on the english-speaking ruby-core@ list instead. The release management process should also improve. Currently, it looks like a total mess. The following Ruby development branches are actively maintained:
ruby_1_8 (106 commits over the last six months)
ruby_1_8_6 (4 commits over the last six months)
ruby_1_8_7 (35 commits over the last six months)
ruby_1_9_1 (4 commits over the last six months)
ruby_1_9_2 (227 commits over the last six months)
trunk (1543 commits over the last six months) While the state of the ruby_1_8_6 and ruby_1_9_1 branches is clear (very important bugfixes only), the state of all of the other branches is rather unclear.
What s the stable Ruby branch? 1.8 or 1.9? If it s 1.9, why are people still actively developing in the ruby_1_8 branch? How long will they continue to be maintained in parallel, dividing the manpower? Is a Ruby 1.8.8 release to be expected? Will it be ABI/API compatible with 1.8.7? Is the ruby_1_8_7 branch really bugfixes-only? How much testing of it has been done? If it s bugfixes-only and regression-free, I should push it to Debian squeeze, due to be released in a few weeks. But would you recommend that? Due to past breakages in the ruby_1_8_7 branch, it s unlikely that we will do it.
Is the ruby_1_9_2 a regression-free, bugfix-only branch? If yes, isn t 227 commits over 6 months a lot? What will be the version of the next release of trunk ? When is it expected? Will it be ABI-compatible with the current ruby_1_9_2 branch? API-compatible?
New releases in the 1.8.7 and 1.9.2 branches were done on december 25th. Why were they no betas or RCs allowing wider testing? How much testing has been done behind the scenes? Most of those questions have no clear answer. The Ruby development community should build a common understanding of the status of the various branches, and of their release expectations. Releasing on december 25th of each year sounds fun, but is releasing when everybody is on vacation really a good idea? It would be fantastic to have something similar to Python Enhancement Proposals in the Ruby community. But having open discussions in english about the major issues would already be great. Ruby is not just the interpreter.
The Ruby development community should clearly define what the Ruby platform is. There are some big players, like Rails, and newer interpreter releases should not be done before ensuring that those big players still work. Also, since we have alternative Ruby interpreters, like JRuby, Rubinius and MacRuby, we need a clear process on how they integrate with the rest of the ecosystem. For example, having each of them rely on their own outdated fork of the whole stdlib is ridiculous, since it s not where they compete. The Ruby community should acknowledge that RVM and Rubygems are not for everybody. People who say so should be laughed at. Of course, RVM and Rubygems are nice tools for some people. But it is completely wrong to believe that compiling from source using RVM should be the standard way of installing Ruby, or that all people interested in installing Redmine should know that Ruby has its own specific packaging system. The Ruby community should work with their target platforms to improve how Ruby is distributed instead of reinventing the wheel. That includes Debian, but also RedHat-based distros, for example. It is likely that it won t be possible to reach a one-size-fits-all situation. But that s real life. Some people in the Ruby community should stop behaving like assholes. As one of the Debian Ruby maintainers, I have been routinely accused of creating crippled packages on purpose (FTR, I don t think that the Debian packages are crippled, despite what the rumors says). Debian is not the only target of that. Just yesterday, someone called for abandonning YARV (the new Ruby VM in Ruby 1.9), calling it Yet Another Random Vailure. This kind of comments is really hurting the people who are investing their free time in Ruby, and is turning away people who consider getting involved. In Debian, we have had a lot of problems getting people to help with Ruby maintenance since they are getting shit from the community all the time. So, what s the future for Ruby in Debian?

• For the interpreter, the two other maintainers, Akira Yamada and Daigo Moriwaki, are of course free to continue their work, and I wish them good luck.
• For the pkg-ruby-extras team, which maintains most of the Ruby libraries and applications, the future is less clear. The team was already badly understaffed, and I feel that I should probably clean up its status by orphaning/removing the packages that are unmaintained otherwise. This won t affect all the packages that the team maintains: some packages (like redmine) are actively maintained, and will stay.
• For me, it just means I will have more time for other things (possibly not Free Software-related). If things improve dramatically, I might also come back to Ruby packaging at some point.

Update: there s also a number of interesting comments about this post on this site.
Update 2: First, thanks a lot for all the interesting comments. I will make some follow-up posts trying to summarize what was said. It seems that this post also triggered some reactions on ruby-core@, with Charles Olivier explaining the JRuby stdlib fork, and Yui Naruse clarifying that all questions are welcomed on ruby-core@. This is great, really.

After trying to compare Ogg Theora to the Digistan definition of a free and open standard, I concluded that this need to be done for more standards and started on a framework for doing this. As a start, I want to get the status for all the standards in the Norwegian reference directory, which include UTF-8, HTML, PDF, ODF, JPEG, PNG, SVG and others. But to be able to complete this in a reasonable time frame, I will need help. If you want to help out with this work, please visit the wiki pages I have set up for this, and let me know that you want to help out. The IRC channel #nuug on irc.freenode.net is a good place to coordinate this for now, as it is the IRC channel for the NUUG association where I have created the framework (I am the leader of the Norwegian Unix User Group). The framework is still forming, and a lot is left to do. Do not be scared by the sketchy form of the current pages. :)

I just discovered that I'd been hit by #409864 since upgrading to Lenny. A very unfortunate bug that one. I'm surprised it hasn't been fixed in a point release.

... looking for an HOWTO? These days, possibly because it's holiday in several countries around the world, I'm receiving a higher number of "how can I help Debian?" mails than usual. This is good and makes me happy! Those mails show we have a community who cares about giving back to Free Software, in exchange of what they have received via Debian. Routinely, in my replies, I point people to the following list of documentation we have on how to contribute to Debian:

• "How can you help Debian", the official contribution page on www.d.o
• HelpDebian on the Debian wiki
• HelpDebian FAQ, companion FAQ of the above page
• Chapter "Contributing to the Debian Project" of the Debian FAQ

Am I missing something? At the same time, it happens that I reply to people who clearly went through (at least some of) the above resources, but which are still blocked at the "what I do next?" step. I guess this is because some of the day to day practices of Debian life are not described in the documentation we have. For instance, I find scarce references to "cultural" aspects of Debian technical life such as:

• a lot of coordination happens over IRC
• a typical way of starting collaborating on packaging is interacting with the package maintainers via the BTS, by doing triaging, submitting patches, etc.
• lots of packages are team-maintained (in fact, this is mentioned in a couple of places, but unfortunately our team list is not complete. Therefore by saying "choose from the list a team you're interested in and contact them", as I recurrently do, I know I'm leaving someone out)
• how to find a mentor
• ...

And I'm sure I'm missing quite some other cultural/folklore traits, which look obvious to longstanding members of the Debian community, but which are probably not so obvious to external people. So, dear lazyweb, have you started contributing to Debian and remember your experience? If so, I'd like to hear from you. In particular, I'd like to hear your suggestions on how we can improve our documentation on the matter (patches welcome, as usual!). Bonus point. Of the various pieces of documentation we have, it seems that a sufficiently detailed HOWTO-style document, covering both technical and cultural aspects, is missing. If you are into writing for Debian, that might be an experiment worth trying.

There s this great feature in modern operating systems called the page cache. Simply put, it keeps in memory what normally is stored on disk and helps both read and write performance. While it s all nice for a day to day use, it often gets in the way when one wants to track performance issues with cold cache (when the files you need to access are not in the page cache yet). A commonly used command to flush the Linux page cache is the following:

# echo 3 > /proc/sys/vm/drop_caches

Unfortunately, its effect is broad, and it flushes the whole page cache. When working on cold startup performance of an application like Firefox, what you really want is to have your page cache in a state close to what it was before you started the application. One way to get in a better position than flushing the entire page cache is to reboot: the page cache will be filled with system and desktop environment libraries during the boot process, making the application startup closer to what you want. But it takes time. A whole lot of it. In one of my what if moments, I wondered what happens to the page cache when using posix_fadvise with the POSIX_FADV_DONTNEED hint. Guess what? It actually reliably flushes the page cache for the given range in the given file. At least it does so with Debian Squeeze s Linux kernel. Provided you have a list of files your application loads that aren t already in the page cache, you can flush these files and only these from the page cache. The following source code compiles to a tool to which you give a list of files as arguments, and that flushes these files:

#include <unistd.h>
#include <fcntl.h>
#include <sys/stat.h>

int main(int argc, char *argv[])  
  int i, fd;
  for (i = 1; i < argc; i++)  
    if ((fd = open(argv[i], O_RDONLY)) != -1)  
      struct stat st;
      fstat(fd, &st);
      posix_fadvise(fd, 0, st.st_size, POSIX_FADV_DONTNEED);
      close(fd);
     
   
  return 0;
 

It s actually a pretty scary feature, especially on multi-user environments, because any user can flush any file she can open, repeatedly, possibly hitting system performance. By the way, on systems using lxc (and maybe other containers, I don t know), running the echo 3 > /proc/sys/vm/drop_caches command from a root shell in a container does flush the host page cache, which could be dangerous for VPS hosting services using these solutions. Update: I have to revise my judgment, it appears posix_fadvise(,,,POSIX_FADV_DONTNEED) doesn t flush (parts of) files that are still in use by other processes, which still makes it very useful for my usecase, but also makes it less dangerous than I thought. The drop_cache problem is still real with lxc, though.

One of the reasons I like the Digistan definition of "Free and Open Standard" is that this is a new term, and thus the meaning of the term has been decided by Digistan. The term "Open Standard" has become so misunderstood that it is no longer very useful when talking about standards. One end up discussing which definition is the best one and with such frame the only one gaining are the proponents of de-facto standards and proprietary solutions. But to give us an idea about the diversity of definitions of open standards, here are a few that I know about. This list is not complete, but can be a starting point for those that want to do a complete survey. More definitions are available on the wikipedia page. First off is my favourite, the definition from the European Interoperability Framework version 1.0. Really sad to notice that BSA and others has succeeded in getting it removed from version 2.0 of the framework by stacking the committee drafting the new version with their own people. Anyway, the definition is still available and it include the key properties needed to make sure everyone can use a specification on equal terms.

The following are the minimal characteristics that a specification and its attendant documents must have in order to be considered an open standard:

• The standard is adopted and will be maintained by a not-for-profit organisation, and its ongoing development occurs on the basis of an open decision-making procedure available to all interested parties (consensus or majority decision etc.).
• The standard has been published and the standard specification document is available either freely or at a nominal charge. It must be permissible to all to copy, distribute and use it for no fee or at a nominal fee.
• The intellectual property - i.e. patents possibly present - of (parts of) the standard is made irrevocably available on a royalty- free basis.
• There are no constraints on the re-use of the standard.

Another one originates from my friends over at DKUUG, who coined and gathered support for this definition in 2004. It even made it into the Danish parlament as their definition of a open standard. Another from a different part of the Danish government is available from the wikipedia page.

En ben standard opfylder f lgende krav:

1. Veldokumenteret med den fuldst ndige specifikation offentligt tilg ngelig.
2. Frit implementerbar uden konomiske, politiske eller juridiske begr nsninger p implementation og anvendelse.
3. Standardiseret og vedligeholdt i et bent forum (en s kaldt "standardiseringsorganisation") via en ben proces.

Then there is the definition from Free Software Foundation Europe.

An Open Standard refers to a format or protocol that is

1. subject to full public assessment and use without constraints in a manner equally available to all parties;
2. without any components or extensions that have dependencies on formats or protocols that do not meet the definition of an Open Standard themselves;
3. free from legal or technical clauses that limit its utilisation by any party or in any business model;
4. managed and further developed independently of any single vendor in a process open to the equal participation of competitors and third parties;
5. available in multiple complete implementations by competing vendors, or as a complete implementation equally available to all parties.

A long time ago, SUN Microsystems, now bought by Oracle, created its Open Standards Checklist with a fairly detailed description.

Creation and Management of an Open Standard

• Its development and management process must be collaborative and democratic:
  • Participation must be accessible to all those who wish to participate and can meet fair and reasonable criteria imposed by the organization under which it is developed and managed.
  • The processes must be documented and, through a known method, can be changed through input from all participants.
  • The process must be based on formal and binding commitments for the disclosure and licensing of intellectual property rights.
  • Development and management should strive for consensus, and an appeals process must be clearly outlined.
  • The standard specification must be open to extensive public review at least once in its life-cycle, with comments duly discussed and acted upon, if required.

Use and Licensing of an Open Standard

• The standard must describe an interface, not an implementation, and the industry must be capable of creating multiple, competing implementations to the interface described in the standard without undue or restrictive constraints. Interfaces include APIs, protocols, schemas, data formats and their encoding.
• The standard must not contain any proprietary "hooks" that create a technical or economic barriers
• Faithful implementations of the standard must interoperate. Interoperability means the ability of a computer program to communicate and exchange information with other computer programs and mutually to use the information which has been exchanged. This includes the ability to use, convert, or exchange file formats, protocols, schemas, interface information or conventions, so as to permit the computer program to work with other computer programs and users in all the ways in which they are intended to function.
• It must be permissible for anyone to copy, distribute and read the standard for a nominal fee, or even no fee. If there is a fee, it must be low enough to not preclude widespread use.
• It must be possible for anyone to obtain free (no royalties or fees; also known as "royalty free"), worldwide, non-exclusive and perpetual licenses to all essential patent claims to make, use and sell products based on the standard. The only exceptions are terminations per the reciprocity and defensive suspension terms outlined below. Essential patent claims include pending, unpublished patents, published patents, and patent applications. The license is only for the exact scope of the standard in question.
  • May be conditioned only on reciprocal licenses to any of licensees' patent claims essential to practice that standard (also known as a reciprocity clause)
  • May be terminated as to any licensee who sues the licensor or any other licensee for infringement of patent claims essential to practice that standard (also known as a "defensive suspension" clause)
  • The same licensing terms are available to every potential licensor
• The licensing terms of an open standards must not preclude implementations of that standard under open source licensing terms or restricted licensing terms

It is said that one of the nice things about standards is that there are so many of them. As you can see, the same holds true for open standard definitions. Most of the definitions have a lot in common, and it is not really controversial what properties a open standard should have, but the diversity of definitions have made it possible for those that want to avoid a level marked field and real competition to downplay the significance of open standards. I hope we can turn this tide by focusing on the advantages of Free and Open Standards.

If anyone mentions Dropbox to me one more time, I will scream. I'm sure it's a wonderful solution, but I have deep misgivings about handing my data over to someone I don't trust. Besides, my synchronisation needs are not that complicated. Here are my high-level requirements:

1. Two way synchronisation between multiple clients and a central server.
2. Automatic syncs - no requirement for manual triggering.
3. Graceful handling of network outages, suspends, etc.
4. If using ssh, running from within my X session (so that it can access my ssh-agent)
5. Some assurance that my data will not be passed to other people (e.g. by running everything on servers I control, or encrypting all the data with keys only I hold)
6. Regular backups of the central data.

I've been playing about with several ideas - I know some people try to do this sort of thing with git, but I don't have a need for version control right at the moment, and merging conflicts could be far too complicated. I did hack together a script involving lsyncd and NetworkManager's dbus signals, but then realised that synchronising in the other direction would need something quite different, and it was getting a bit out of hand by that point. So my current favourite is actually GNOME's Conduit project. It ticks almost all of the boxes (apart from the backups, which should be simple to put together). There are some rough edges - the author has tried to delegate the minor detail that it's accessing an SFTP share entirely to GIO. But GVFS shares are not mounted automatically by Conduit - so I would have to browse to the remote folder in Nautilus before any synching would happen in each session. Bad. But looking at how Conduit is used, I don't agree with hiding the nature of the remote folder from the user. By all means use GIO to perform the mechanics of SSH and so on, but the user surely thinks of the remote server as something different to a local folder? So from a UI perspective, this should be made explicit. After all, the user has to deal with it at some level by switching to Nautilus to set up the connection. And since I cannot find an SFTP plugin for Conduit anywhere, this still leaves me in the unfortunate position of having to write code if I want a nice solution. Bah, humbug.

The text below went out as a post to the r-packages list a few days ago, but I thought it would make sense to post it on the blog too. So with a little html markup... Summary Version 0.9.0 of the Rcpp package is now on CRAN and its mirrors. This release marks another step in the development of the package, and a few key points are highlighted below. More details are in the NEWS and ChangeLog files included in the package. Overview Rcpp is an R package and associated C++ library that facilitates integration of C++ code in R packages. The package features a complete set of C++ classes (Rcpp::IntegerVector, Rcpp:NumericVector, Rcpp::Function, Rcpp::Environment, ...) that makes it easier to manipulate R objects of matching types (integer vectors, functions, environments, etc ...). Rcpp takes advantage of C++ language features such as the explicit constructor / destructor lifecycle of objects to manage garbage collection automatically and transparently. We believe this is a major improvement over use of PROTECT / UNPROTECT. When an Rcpp object is created, it protects the underlying SEXP so that the garbage collector does not attempt to reclaim the memory. This protection is withdrawn when the object goes out of scope. Moreover, users generally do not need to manage memory directly (via calls to new / delete or malloc / free) as this is done by the Rcpp classes or the corresponding STL containers. A few key points about Rcpp:

• a rich API covering all core R data types including vectors, matrices, functions, environments, ... (with the exeception of factors which are less useful in C++)
• seamless (bi-directional) data interchange between R and C++
• possibility of inline use permitting definition, compilation, linking and loading of C++ functions directly from R
• extensive documentation now covering eight vignettes
• exception handling and error propagation back to R
• extensive test suite using RUnit covering over 700 tests
• extension packages RcppArmadillo and RcppGSL provide easy-to-use integration with the Armadillo (linear algebra) and GNU GSL librasries
• increasing adoption among R users and package developers with now twenty packages from CRAN or BioConductor depending on Rcpp
• support for the legacy 'classic' Rcpp is now provided by the RcppClassic package which is being released concurrently with Rcpp 0.9.0

Several key features were added during the 0.8.* cycles and are described below. Rcpp sugar Rcpp now provides syntactic sugar: vectorised expressions at the C++ level which are motivated by the corresponding R expressions. This covers operators (binary arithmetic, binary logical, unary), functions (producing single logical results, mathematical functions and d/p/q/r statistical functions). Examples comprises anything from ifelse() to pmin()/pmax() or A really simply example is a function

    SEXP foo( SEXP xx, SEXP yy) 
        NumericVector x(xx), y(yy) ;
        return ifelse( x < y, x*x, -(y*y) ) ;
     

which deploys the sugar 'ifelse' function modeled after the corresponding R function. Another simple example is

    double square( double x) 
        return x*x ;
     
    SEXP foo( SEXP xx ) 
        NumericVector x(xx) ;
        return sapply( x, square ) ;
     

where use the sugar function 'sapply' to sweep a simple C++ function which operates elementwise across the supplied vector. The Rcpp-sugar vignette describes sugar in more detail. Rcpp modules Rcpp modules are inspired by Boost.Python and make exposing C++ functions or classes to R even easier. A first illustration is provided by this simple C++ code snippet

    const char* hello( const std::string& who ) 
        std::string result( "hello " ) ;
        result += who ;
        return result.c_str() ;
     
    RCPP_MODULE(yada) 
        using namespace Rcpp ;
        function( "hello", &hello ) ;
     

which (after compiling and loading) we can access in R as

    yada <- Module( "yada" )
    yada$hello( "world" )

In a similar way, C++ classes can be exposed very easily. Rcpp modules are also described in more detail in their own vignette. Reference Classes R release 2.12.0 introduced Reference Classes. These are formal S4 classes with the corresponding dispatch method, but passed by reference and easy to use. Reference Classes can also be exposed to R by using Rcpp modules. Extension packackages The RcppArmadillo package permits use of the advanced C++ library 'Armadillo, a C++ linear algebra library aiming towards a good balance between speed and ease of use, providing integer, floating point and complex matrices and vectors with lapack / blas support via R. Armadillo uses templates for a delayed evaluation approach is employed (during compile time) to combine several operations into one and reduce (or eliminate) the need for temporaries. Armadillo is useful if C++ has been decided as the language of choice, rather than another language like Matlab or Octave, and aims to be as expressive as the former. Via Rcpp and RcppArmadillo, R users now have easy access to this functionality. Examples are provided in the RcppArmadillo package. The RcppGSL package permits easy use of the GNU Scientific Library (GSL), a collection of numerical routines for scientifc computing. It is particularly useful for C and C++ programs as it provides a standard C interface to a wide range of mathematical routines such as special functions, permutations, combinations, fast fourier transforms, eigensystems, random numbers, quadrature, random distributions, quasi-random sequences, Monte Carlo integration, N-tuples, differential equations, simulated annealing, numerical differentiation, interpolation, series acceleration, Chebyshev approximations, root-finding, discrete Hankel transforms physical constants, basis splines and wavelets. There are over 1000 functions in total with an extensive test suite. The RcppGSL package provides an easy-to-use interface between GSL data structures and R using concepts from Rcpp. The RcppGSL package also contains a vignette with more documentation. Legacy 'classic' API Packages still using code interfacing the initial 'classic' Rcpp API are encouraged to migrate to the new API. Should a code transition not be possible, backwards compatibility is provided by the RcppClassic package released alongside Rcpp 0.9.0. By including RcppClassic.h and building against the RcppClassic package and library, vintage code can remain operational using the classic API. The short vignette in the RcppClassic package has more details. Documentation The package contains a total of eight vignettes the first of which provides a short and succinct introduction to the Rcpp package along with several motivating examples. Links

• Rcpp main page
• R-forge project page
• Dirk's blog section about Rcpp
• Romain's blog section about Rcpp

Support Questions about Rcpp should be directed to the Rcpp-devel mailing list https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel

Dirk Eddelbuettel, Romain Francois, Doug Bates and John Chambers
December 2010

A few days ago an article in the Norwegian Computerworld magazine about how version 2.0 of European Interoperability Framework has been successfully lobbied by the proprietary software industry to remove the focus on free software. Nothing very surprising there, given earlier reports on how Microsoft and others have stacked the committees in this work. But I find this very sad. The definition of an open standard from version 1 was very good, and something I believe should be used also in the future, alongside the definition from Digistan. Version 2 have removed the open standard definition from its content. Anyway, the news reminded me of the great reply sent by Dr. Edgar Villanueva, congressman in Peru at the time, to Microsoft as a reply to Microsofts attack on his proposal regarding the use of free software in the public sector in Peru. As the text was not available from a few of the URLs where it used to be available, I copy it here from my source to ensure it is available also in the future. Some background information about that story is available in an article from Linux Journal in 2002.

Lima, 8th of April, 2002
To: Se or JUAN ALBERTO GONZ LEZ
General Manager of Microsoft Per Dear Sir: First of all, I thank you for your letter of March 25, 2002 in which you state the official position of Microsoft relative to Bill Number 1609, Free Software in Public Administration, which is indubitably inspired by the desire for Peru to find a suitable place in the global technological context. In the same spirit, and convinced that we will find the best solutions through an exchange of clear and open ideas, I will take this opportunity to reply to the commentaries included in your letter. While acknowledging that opinions such as yours constitute a significant contribution, it would have been even more worthwhile for me if, rather than formulating objections of a general nature (which we will analyze in detail later) you had gathered solid arguments for the advantages that proprietary software could bring to the Peruvian State, and to its citizens in general, since this would have allowed a more enlightening exchange in respect of each of our positions. With the aim of creating an orderly debate, we will assume that what you call "open source software" is what the Bill defines as "free software", since there exists software for which the source code is distributed together with the program, but which does not fall within the definition established by the Bill; and that what you call "commercial software" is what the Bill defines as "proprietary" or "unfree", given that there exists free software which is sold in the market for a price like any other good or service. It is also necessary to make it clear that the aim of the Bill we are discussing is not directly related to the amount of direct savings that can by made by using free software in state institutions. That is in any case a marginal aggregate value, but in no way is it the chief focus of the Bill. The basic principles which inspire the Bill are linked to the basic guarantees of a state of law, such as:

• Free access to public information by the citizen.
• Permanence of public data.
• Security of the State and citizens.

To guarantee the free access of citizens to public information, it is indispensable that the encoding of data is not tied to a single provider. The use of standard and open formats gives a guarantee of this free access, if necessary through the creation of compatible free software. To guarantee the permanence of public data, it is necessary that the usability and maintenance of the software does not depend on the goodwill of the suppliers, or on the monopoly conditions imposed by them. For this reason the State needs systems the development of which can be guaranteed due to the availability of the source code. To guarantee national security or the security of the State, it is indispensable to be able to rely on systems without elements which allow control from a distance or the undesired transmission of information to third parties. Systems with source code freely accessible to the public are required to allow their inspection by the State itself, by the citizens, and by a large number of independent experts throughout the world. Our proposal brings further security, since the knowledge of the source code will eliminate the growing number of programs with *spy code*. In the same way, our proposal strengthens the security of the citizens, both in their role as legitimate owners of information managed by the state, and in their role as consumers. In this second case, by allowing the growth of a widespread availability of free software not containing *spy code* able to put at risk privacy and individual freedoms. In this sense, the Bill is limited to establishing the conditions under which the state bodies will obtain software in the future, that is, in a way compatible with these basic principles. From reading the Bill it will be clear that once passed:

the law does not forbid the production of proprietary software

the law does not forbid the sale of proprietary software

the law does not specify which concrete software to use

the law does not dictate the supplier from whom software will be bought

the law does not limit the terms under which a software product can be licensed.

What the Bill does express clearly, is that, for software to be acceptable for the state it is not enough that it is technically capable of fulfilling a task, but that further the contractual conditions must satisfy a series of requirements regarding the license, without which the State cannot guarantee the citizen adequate processing of his data, watching over its integrity, confidentiality, and accessibility throughout time, as these are very critical aspects for its normal functioning. We agree, Mr. Gonzalez, that information and communication technology have a significant impact on the quality of life of the citizens (whether it be positive or negative). We surely also agree that the basic values I have pointed out above are fundamental in a democratic state like Peru. So we are very interested to know of any other way of guaranteeing these principles, other than through the use of free software in the terms defined by the Bill. As for the observations you have made, we will now go on to analyze them in detail: Firstly, you point out that: "1. The bill makes it compulsory for all public bodies to use only free software, that is to say open source software, which breaches the principles of equality before the law, that of non-discrimination and the right of free private enterprise, freedom of industry and of contract, protected by the constitution." This understanding is in error. The Bill in no way affects the rights you list; it limits itself entirely to establishing conditions for the use of software on the part of state institutions, without in any way meddling in private sector transactions. It is a well established principle that the State does not enjoy the wide spectrum of contractual freedom of the private sector, as it is limited in its actions precisely by the requirement for transparency of public acts; and in this sense, the preservation of the greater common interest must prevail when legislating on the matter. The Bill protects equality under the law, since no natural or legal person is excluded from the right of offering these goods to the State under the conditions defined in the Bill and without more limitations than those established by the Law of State Contracts and Purchasing (T.U.O. by Supreme Decree No. 012-2001-PCM). The Bill does not introduce any discrimination whatever, since it only establishes *how* the goods have to be provided (which is a state power) and not *who* has to provide them (which would effectively be discriminatory, if restrictions based on national origin, race religion, ideology, sexual preference etc. were imposed). On the contrary, the Bill is decidedly antidiscriminatory. This is so because by defining with no room for doubt the conditions for the provision of software, it prevents state bodies from using software which has a license including discriminatory conditions. It should be obvious from the preceding two paragraphs that the Bill does not harm free private enterprise, since the latter can always choose under what conditions it will produce software; some of these will be acceptable to the State, and others will not be since they contradict the guarantee of the basic principles listed above. This free initiative is of course compatible with the freedom of industry and freedom of contract (in the limited form in which the State can exercise the latter). Any private subject can produce software under the conditions which the State requires, or can refrain from doing so. Nobody is forced to adopt a model of production, but if they wish to provide software to the State, they must provide the mechanisms which guarantee the basic principles, and which are those described in the Bill. By way of an example: nothing in the text of the Bill would prevent your company offering the State bodies an office "suite", under the conditions defined in the Bill and setting the price that you consider satisfactory. If you did not, it would not be due to restrictions imposed by the law, but to business decisions relative to the method of commercializing your products, decisions with which the State is not involved. To continue; you note that:" 2. The bill, by making the use of open source software compulsory, would establish discriminatory and non competitive practices in the contracting and purchasing by public bodies..." This statement is just a reiteration of the previous one, and so the response can be found above. However, let us concern ourselves for a moment with your comment regarding "non-competitive ... practices." Of course, in defining any kind of purchase, the buyer sets conditions which relate to the proposed use of the good or service. From the start, this excludes certain manufacturers from the possibility of competing, but does not exclude them "a priori", but rather based on a series of principles determined by the autonomous will of the purchaser, and so the process takes place in conformance with the law. And in the Bill it is established that *no one* is excluded from competing as far as he guarantees the fulfillment of the basic principles. Furthermore, the Bill *stimulates* competition, since it tends to generate a supply of software with better conditions of usability, and to better existing work, in a model of continuous improvement. On the other hand, the central aspect of competivity is the chance to provide better choices to the consumer. Now, it is impossible to ignore the fact that marketing does not play a neutral role when the product is offered on the market (since accepting the opposite would lead one to suppose that firms' expenses in marketing lack any sense), and that therefore a significant expense under this heading can influence the decisions of the purchaser. This influence of marketing is in large measure reduced by the bill that we are backing, since the choice within the framework proposed is based on the *technical merits* of the product and not on the effort put into commercialization by the producer; in this sense, competitiveness is increased, since the smallest software producer can compete on equal terms with the most powerful corporations. It is necessary to stress that there is no position more anti-competitive than that of the big software producers, which frequently abuse their dominant position, since in innumerable cases they propose as a solution to problems raised by users: "update your software to the new version" (at the user's expense, naturally); furthermore, it is common to find arbitrary cessation of technical help for products, which, in the provider's judgment alone, are "old"; and so, to receive any kind of technical assistance, the user finds himself forced to migrate to new versions (with non-trivial costs, especially as changes in hardware platform are often involved). And as the whole infrastructure is based on proprietary data formats, the user stays "trapped" in the need to continue using products from the same supplier, or to make the huge effort to change to another environment (probably also proprietary). You add: "3. So, by compelling the State to favor a business model based entirely on open source, the bill would only discourage the local and international manufacturing companies, which are the ones which really undertake important expenditures, create a significant number of direct and indirect jobs, as well as contributing to the GNP, as opposed to a model of open source software which tends to have an ever weaker economic impact, since it mainly creates jobs in the service sector." I do not agree with your statement. Partly because of what you yourself point out in paragraph 6 of your letter, regarding the relative weight of services in the context of software use. This contradiction alone would invalidate your position. The service model, adopted by a large number of companies in the software industry, is much larger in economic terms, and with a tendency to increase, than the licensing of programs. On the other hand, the private sector of the economy has the widest possible freedom to choose the economic model which best suits its interests, even if this freedom of choice is often obscured subliminally by the disproportionate expenditure on marketing by the producers of proprietary software. In addition, a reading of your opinion would lead to the conclusion that the State market is crucial and essential for the proprietary software industry, to such a point that the choice made by the State in this bill would completely eliminate the market for these firms. If that is true, we can deduce that the State must be subsidizing the proprietary software industry. In the unlikely event that this were true, the State would have the right to apply the subsidies in the area it considered of greatest social value; it is undeniable, in this improbable hypothesis, that if the State decided to subsidize software, it would have to do so choosing the free over the proprietary, considering its social effect and the rational use of taxpayers money. In respect of the jobs generated by proprietary software in countries like ours, these mainly concern technical tasks of little aggregate value; at the local level, the technicians who provide support for proprietary software produced by transnational companies do not have the possibility of fixing bugs, not necessarily for lack of technical capability or of talent, but because they do not have access to the source code to fix it. With free software one creates more technically qualified employment and a framework of free competence where success is only tied to the ability to offer good technical support and quality of service, one stimulates the market, and one increases the shared fund of knowledge, opening up alternatives to generate services of greater total value and a higher quality level, to the benefit of all involved: producers, service organizations, and consumers. It is a common phenomenon in developing countries that local software industries obtain the majority of their takings in the service sector, or in the creation of "ad hoc" software. Therefore, any negative impact that the application of the Bill might have in this sector will be more than compensated by a growth in demand for services (as long as these are carried out to high quality standards). If the transnational software companies decide not to compete under these new rules of the game, it is likely that they will undergo some decrease in takings in terms of payment for licenses; however, considering that these firms continue to allege that much of the software used by the State has been illegally copied, one can see that the impact will not be very serious. Certainly, in any case their fortune will be determined by market laws, changes in which cannot be avoided; many firms traditionally associated with proprietary software have already set out on the road (supported by copious expense) of providing services associated with free software, which shows that the models are not mutually exclusive. With this bill the State is deciding that it needs to preserve certain fundamental values. And it is deciding this based on its sovereign power, without affecting any of the constitutional guarantees. If these values could be guaranteed without having to choose a particular economic model, the effects of the law would be even more beneficial. In any case, it should be clear that the State does not choose an economic model; if it happens that there only exists one economic model capable of providing software which provides the basic guarantee of these principles, this is because of historical circumstances, not because of an arbitrary choice of a given model. Your letter continues: "4. The bill imposes the use of open source software without considering the dangers that this can bring from the point of view of security, guarantee, and possible violation of the intellectual property rights of third parties." Alluding in an abstract way to "the dangers this can bring", without specifically mentioning a single one of these supposed dangers, shows at the least some lack of knowledge of the topic. So, allow me to enlighten you on these points. On security: National security has already been mentioned in general terms in the initial discussion of the basic principles of the bill. In more specific terms, relative to the security of the software itself, it is well known that all software (whether proprietary or free) contains errors or "bugs" (in programmers' slang). But it is also well known that the bugs in free software are fewer, and are fixed much more quickly, than in proprietary software. It is not in vain that numerous public bodies responsible for the IT security of state systems in developed countries require the use of free software for the same conditions of security and efficiency. What is impossible to prove is that proprietary software is more secure than free, without the public and open inspection of the scientific community and users in general. This demonstration is impossible because the model of proprietary software itself prevents this analysis, so that any guarantee of security is based only on promises of good intentions (biased, by any reckoning) made by the producer itself, or its contractors. It should be remembered that in many cases, the licensing conditions include Non-Disclosure clauses which prevent the user from publicly revealing security flaws found in the licensed proprietary product. In respect of the guarantee: As you know perfectly well, or could find out by reading the "End User License Agreement" of the products you license, in the great majority of cases the guarantees are limited to replacement of the storage medium in case of defects, but in no case is compensation given for direct or indirect damages, loss of profits, etc... If as a result of a security bug in one of your products, not fixed in time by yourselves, an attacker managed to compromise crucial State systems, what guarantees, reparations and compensation would your company make in accordance with your licensing conditions? The guarantees of proprietary software, inasmuch as programs are delivered AS IS'', that is, in the state in which they are, with no additional responsibility of the provider in respect of function, in no way differ from those normal with free software. On Intellectual Property: Questions of intellectual property fall outside the scope of this bill, since they are covered by specific other laws. The model of free software in no way implies ignorance of these laws, and in fact the great majority of free software is covered by copyright. In reality, the inclusion of this question in your observations shows your confusion in respect of the legal framework in which free software is developed. The inclusion of the intellectual property of others in works claimed as one's own is not a practice that has been noted in the free software community; whereas, unfortunately, it has been in the area of proprietary software. As an example, the condemnation by the Commercial Court of Nanterre, France, on 27th September 2001 of Microsoft Corp. to a penalty of 3 million francs in damages and interest, for violation of intellectual property (piracy, to use the unfortunate term that your firm commonly uses in its publicity). You go on to say that: "The bill uses the concept of open source software incorrectly, since it does not necessarily imply that the software is free or of zero cost, and so arrives at mistaken conclusions regarding State savings, with no cost-benefit analysis to validate its position." This observation is wrong; in principle, freedom and lack of cost are orthogonal concepts: there is software which is proprietary and charged for (for example, MS Office), software which is proprietary and free of charge (MS Internet Explorer), software which is free and charged for (Red Hat, SuSE etc GNU/Linux distributions), software which is free and not charged for (Apache, Open Office, Mozilla), and even software which can be licensed in a range of combinations (MySQL). Certainly free software is not necessarily free of charge. And the text of the bill does not state that it has to be so, as you will have noted after reading it. The definitions included in the Bill state clearly *what* should be considered free software, at no point referring to freedom from charges. Although the possibility of savings in payments for proprietary software licenses are mentioned, the foundations of the bill clearly refer to the fundamental guarantees to be preserved and to the stimulus to local technological development. Given that a democratic State must support these principles, it has no other choice than to use software with publicly available source code, and to exchange information only in standard formats. If the State does not use software with these characteristics, it will be weakening basic republican principles. Luckily, free software also implies lower total costs; however, even given the hypothesis (easily disproved) that it was more expensive than proprietary software, the simple existence of an effective free software tool for a particular IT function would oblige the State to use it; not by command of this Bill, but because of the basic principles we enumerated at the start, and which arise from the very essence of the lawful democratic State. You continue: "6. It is wrong to think that Open Source Software is free of charge. Research by the Gartner Group (an important investigator of the technological market recognized at world level) has shown that the cost of purchase of software (operating system and applications) is only 8% of the total cost which firms and institutions take on for a rational and truly beneficial use of the technology. The other 92% consists of: installation costs, enabling, support, maintenance, administration, and down-time." This argument repeats that already given in paragraph 5 and partly contradicts paragraph 3. For the sake of brevity we refer to the comments on those paragraphs. However, allow me to point out that your conclusion is logically false: even if according to Gartner Group the cost of software is on average only 8% of the total cost of use, this does not in any way deny the existence of software which is free of charge, that is, with a licensing cost of zero. In addition, in this paragraph you correctly point out that the service components and losses due to down-time make up the largest part of the total cost of software use, which, as you will note, contradicts your statement regarding the small value of services suggested in paragraph 3. Now the use of free software contributes significantly to reduce the remaining life-cycle costs. This reduction in the costs of installation, support etc. can be noted in several areas: in the first place, the competitive service model of free software, support and maintenance for which can be freely contracted out to a range of suppliers competing on the grounds of quality and low cost. This is true for installation, enabling, and support, and in large part for maintenance. In the second place, due to the reproductive characteristics of the model, maintenance carried out for an application is easily replicable, without incurring large costs (that is, without paying more than once for the same thing) since modifications, if one wishes, can be incorporated in the common fund of knowledge. Thirdly, the huge costs caused by non-functioning software ("blue screens of death", malicious code such as virus, worms, and trojans, exceptions, general protection faults and other well-known problems) are reduced considerably by using more stable software; and it is well known that one of the most notable virtues of free software is its stability. You further state that: "7. One of the arguments behind the bill is the supposed freedom from costs of open-source software, compared with the costs of commercial software, without taking into account the fact that there exist types of volume licensing which can be highly advantageous for the State, as has happened in other countries." I have already pointed out that what is in question is not the cost of the software but the principles of freedom of information, accessibility, and security. These arguments have been covered extensively in the preceding paragraphs to which I would refer you. On the other hand, there certainly exist types of volume licensing (although unfortunately proprietary software does not satisfy the basic principles). But as you correctly pointed out in the immediately preceding paragraph of your letter, they only manage to reduce the impact of a component which makes up no more than 8% of the total. You continue: "8. In addition, the alternative adopted by the bill (I) is clearly more expensive, due to the high costs of software migration, and (II) puts at risk compatibility and interoperability of the IT platforms within the State, and between the State and the private sector, given the hundreds of versions of open source software on the market." Let us analyze your statement in two parts. Your first argument, that migration implies high costs, is in reality an argument in favor of the Bill. Because the more time goes by, the more difficult migration to another technology will become; and at the same time, the security risks associated with proprietary software will continue to increase. In this way, the use of proprietary systems and formats will make the State ever more dependent on specific suppliers. Once a policy of using free software has been established (which certainly, does imply some cost) then on the contrary migration from one system to another becomes very simple, since all data is stored in open formats. On the other hand, migration to an open software context implies no more costs than migration between two different proprietary software contexts, which invalidates your argument completely. The second argument refers to "problems in interoperability of the IT platforms within the State, and between the State and the private sector" This statement implies a certain lack of knowledge of the way in which free software is built, which does not maximize the dependence of the user on a particular platform, as normally happens in the realm of proprietary software. Even when there are multiple free software distributions, and numerous programs which can be used for the same function, interoperability is guaranteed as much by the use of standard formats, as required by the bill, as by the possibility of creating interoperable software given the availability of the source code. You then say that: "9. The majority of open source code does not offer adequate levels of service nor the guarantee from recognized manufacturers of high productivity on the part of the users, which has led various public organizations to retract their decision to go with an open source software solution and to use commercial software in its place." This observation is without foundation. In respect of the guarantee, your argument was rebutted in the response to paragraph 4. In respect of support services, it is possible to use free software without them (just as also happens with proprietary software), but anyone who does need them can obtain support separately, whether from local firms or from international corporations, again just as in the case of proprietary software. On the other hand, it would contribute greatly to our analysis if you could inform us about free software projects *established* in public bodies which have already been abandoned in favor of proprietary software. We know of a good number of cases where the opposite has taken place, but not know of any where what you describe has taken place. You continue by observing that: "10. The bill discourages the creativity of the Peruvian software industry, which invoices 40 million US$/year, exports 4 million US$ (10th in ranking among non-traditional exports, more than handicrafts) and is a source of highly qualified employment. With a law that encourages the use of open source, software programmers lose their intellectual property rights and their main source of payment." It is clear enough that nobody is forced to commercialize their code as free software. The only thing to take into account is that if it is not free software, it cannot be sold to the public sector. This is not in any case the main market for the national software industry. We covered some questions referring to the influence of the Bill on the generation of employment which would be both highly technically qualified and in better conditions for competition above, so it seems unnecessary to insist on this point. What follows in your statement is incorrect. On the one hand, no author of free software loses his intellectual property rights, unless he expressly wishes to place his work in the public domain. The free software movement has always been very respectful of intellectual property, and has generated widespread public recognition of its authors. Names like those of Richard Stallman, Linus Torvalds, Guido van Rossum, Larry Wall, Miguel de Icaza, Andrew Tridgell, Theo de Raadt, Andrea Arcangeli, Bruce Perens, Darren Reed, Alan Cox, Eric Raymond, and many others, are recognized world-wide for their contributions to the development of software that is used today by millions of people throughout the world. On the other hand, to say that the rewards for authors rights make up the main source of payment of Peruvian programmers is in any case a guess, in particular since there is no proof to this effect, nor a demonstration of how the use of free software by the State would influence these payments. You go on to say that: "11. Open source software, since it can be distributed without charge, does not allow the generation of income for its developers through exports. In this way, the multiplier effect of the sale of software to other countries is weakened, and so in turn is the growth of the industry, while Government rules ought on the contrary to stimulate local industry." This statement shows once again complete ignorance of the mechanisms of and market for free software. It tries to claim that the market of sale of non- exclusive rights for use (sale of licenses) is the only possible one for the software industry, when you yourself pointed out several paragraphs above that it is not even the most important one. The incentives that the bill offers for the growth of a supply of better qualified professionals, together with the increase in experience that working on a large scale with free software within the State will bring for Peruvian technicians, will place them in a highly competitive position to offer their services abroad. You then state that: "12. In the Forum, the use of open source software in education was discussed, without mentioning the complete collapse of this initiative in a country like Mexico, where precisely the State employees who founded the project now state that open source software did not make it possible to offer a learning experience to pupils in the schools, did not take into account the capability at a national level to give adequate support to the platform, and that the software did not and does not allow for the levels of platform integration that now exist in schools." In fact Mexico has gone into reverse with the Red Escolar (Schools Network) project. This is due precisely to the fact that the driving forces behind the Mexican project used license costs as their main argument, instead of the other reasons specified in our project, which are far more essential. Because of this conceptual mistake, and as a result of the lack of effective support from the SEP (Secretary of State for Public Education), the assumption was made that to implant free software in schools it would be enough to drop their software budget and send them a CD ROM with Gnu/Linux instead. Of course this failed, and it couldn't have been otherwise, just as school laboratories fail when they use proprietary software and have no budget for implementation and maintenance. That's exactly why our bill is not limited to making the use of free software mandatory, but recognizes the need to create a viable migration plan, in which the State undertakes the technical transition in an orderly way in order to then enjoy the advantages of free software. You end with a rhetorical question: "13. If open source software satisfies all the requirements of State bodies, why do you need a law to adopt it? Shouldn't it be the market which decides freely which products give most benefits or value?" We agree that in the private sector of the economy, it must be the market that decides which products to use, and no state interference is permissible there. However, in the case of the public sector, the reasoning is not the same: as we have already established, the state archives, handles, and transmits information which does not belong to it, but which is entrusted to it by citizens, who have no alternative under the rule of law. As a counterpart to this legal requirement, the State must take extreme measures to safeguard the integrity, confidentiality, and accessibility of this information. The use of proprietary software raises serious doubts as to whether these requirements can be fulfilled, lacks conclusive evidence in this respect, and so is not suitable for use in the public sector. The need for a law is based, firstly, on the realization of the fundamental principles listed above in the specific area of software; secondly, on the fact that the State is not an ideal homogeneous entity, but made up of multiple bodies with varying degrees of autonomy in decision making. Given that it is inappropriate to use proprietary software, the fact of establishing these rules in law will prevent the personal discretion of any state employee from putting at risk the information which belongs to citizens. And above all, because it constitutes an up-to-date reaffirmation in relation to the means of management and communication of information used today, it is based on the republican principle of openness to the public. In conformance with this universally accepted principle, the citizen has the right to know all information held by the State and not covered by well- founded declarations of secrecy based on law. Now, software deals with information and is itself information. Information in a special form, capable of being interpreted by a machine in order to execute actions, but crucial information all the same because the citizen has a legitimate right to know, for example, how his vote is computed or his taxes calculated. And for that he must have free access to the source code and be able to prove to his satisfaction the programs used for electoral computations or calculation of his taxes. I wish you the greatest respect, and would like to repeat that my office will always be open for you to expound your point of view to whatever level of detail you consider suitable. Cordially,
DR. EDGAR DAVID VILLANUEVA NU EZ
Congressman of the Republic of Per .

A couple of months ago, Debian project membership voted, after extensive discussion, to implement a fundamental change in the Debian community: to welcome as members people who make a valuable contribution to the project, even if they are contributing something other than source code. This was a tremendous milestone for Debian, and one which made me feel proud to have been a part of the project. Historically, only developers had been eligible for membership, including voting and other formal privileges. Although other kinds of contributions were welcome, this disparity gave the impression that they were less valued than code contributions. It seemed to me at the time that Debian s mission was to package all of the free software in the world, and if one s efforts didn t go directly to improving packages, they just weren t as important. I don t remember when I first installed Debian, but I made my first contributions to the project in 1999, and officially joined as a developer in 2000. After several fun and rewarding years of packaging and development, I started a very demanding day job, and spent more and more of my energy into that, and less and less coding for Debian as a volunteer. However, my job with Canonical involved working with Debian, and that was a primary reason why it was interesting to me. It was an opportunity to introduce a whole new population of people to the things I loved about Debian. The reality, of course, was more complicated. Following the launch in 2004, Ubuntu grew quickly in popularity and scope, diverged from Debian in significant ways, and relations between Debian and Ubuntu became strained. Canonical grew quickly as well, and the combination of a growing community, a growing company and growing user adoption was a challenge for everyone concerned. As a Canonical manager and a Debian developer, I felt the strain as much as anyone. Meanwhile, and I felt more and more alienated from Debian. Debian developers who had been friendly in the past became suspicious of Ubuntu and me and I quickly became an outsider. My code contributions to Debian continued to decline, and I was no longer maintaining any packages. In Debian at the time, that meant that I didn t exist. I saw it as an important part of my job to work with my counterparts in Debian, in a coordinating role, but found this increasingly impractical. In 2007, I received an inquiry from the Debian Account Manager, who had noticed I wasn t actively involved in packaging, and wished to disable my account for security reasons if I wasn t using it. Although I wanted to remain active in the Debian community, I had to agree that it wasn t good security practice for me to hold onto my developer privileges. I relinquished my upload rights, with the option to come back if I resumed my development work, and officially became a nobody: I lost the right to vote, my email address and mailing list subscriptions, and all other official ties with Debian, except for the record of my GPG key in a special emeritus keyring for informational purposes. Last month, Enrico Zini announced instructions for contributors to apply for membership under the new guidelines, which recognize many kinds of contributions, not only code. Today, after a three year hiatus, I am proud to be the first Debian member to be accepted through this new process. I expect to continue to submit the occasional patch, but my primary interest is in healing the rift which still exists between Debian and Ubuntu by contributing in a more personal way. Please feel free to contact me if you d like to work together on this. You can reach me as mdz at either debian.org or ubuntu.com, or on IRC. I would like to thank Stefano Zacchiroli, for proposing the General Resolution which enabled Debian to make this transition, and for all of his other work as Debian Project Leader to help Debian grow and improve. I also appreciate Enrico Zini, Jonathan McDowell and Martin Zobel-Helas for expediently processing me and working through the technical changes needed to implement the resolution correctly. It s good to be back.

The last few days I have spent at work here at the University of oslo testing if the new batch of computers will work with Linux. Every year for the last few years the university have organized shared bid of a few thousand computers, and this year HP won the bid. Two different desktops and five different laptops are on the list this year. We in the UNIX group want to know which one of these computers work well with RHEL and Ubuntu, the two Linux distributions we currently handle at the university. My test method is simple, and I share it here to get feedback and perhaps inspire others to test hardware as well. To test, I PXE install the OS version of choice, and log in as my normal user and run a few applications and plug in selected pieces of hardware. When something fail, I make a note about this in the test matrix and move on. If I have some spare time I try to report the bug to the OS vendor, but as I only have the machines for a short time, I rarely have the time to do this for all the problems I find. Anyway, to get to the point of this post. Here is the simple tests I perform on a new model.

• Is PXE installation working? I'm testing with RHEL6, Ubuntu Lucid and Ubuntu Maverik at the moment. If I feel like it, I also test with RHEL5 and Debian Edu/Squeeze.
• Is X.org working? If the graphical login screen show up after installation, X.org is working.
• Is hardware accelerated OpenGL working? Running glxgears (in package mesa-utils on Ubuntu) and writing down the frames per second reported by the program.
• Is sound working? With Gnome and KDE, a sound is played when logging in, and if I can hear this the test is successful. If there are several audio exits on the machine, I try them all and check if the Gnome/KDE audio mixer can control where to send the sound. I normally test this by playing a HTML5 video in Firefox/Iceweasel.
• Is the USB subsystem working? I test this by plugging in a USB memory stick and see if Gnome/KDE notices this.
• Is the CD/DVD player working? I test this by inserting any CD/DVD I have lying around, and see if Gnome/KDE notices this.
• Is any built in camera working? Test using cheese, and see if a picture from the v4l device show up.
• Is bluetooth working? Use the Gnome/KDE browsing tool to see if any bluetooth devices are discovered. In my office, I normally see a few.
• For laptops, is the SD or Compaq Flash reader working. I have memory modules lying around, and stick them in and see if Gnome/KDE notice this.
• For laptops, is suspecd/hibernate working? I'm testing if the special button work, and if the laptop continue to work after resume.
• For laptops, is the extra buttons working, like audio level, adjusting background light, switching on/off external video output, switching on/off wifi, bluetooth, etc? The set of buttons differ from laptop to laptop, so I just write down which are working and which are not.
• Some laptops have smart card readers, finger print readers, acceleration sensors etc. I rarely test these, as I do not know how to quickly test if they are working or not, so I only document their existence.

By now I suspect you are really curious what the test results are for the HP machines I am testing. I'm not done yet, so I will report the test results later. For now I can report that HP 8100 Elite work fine, and hibernation fail with HP EliteBook 8440p on Ubuntu Lucid, and audio fail on RHEL6. Ubuntu Maverik worked with 8440p. As you can see, I have most machines left to test. One interesting observation is that Ubuntu Lucid has almost twice the framerate than RHEL6 with glxgears. No idea why.

A few months ago, my company made the decision to switch from our previous mail provider, to use Google Apps. It was a no-brainer really, most of the company were using it already, for the calendars, and Documents, and well we were a bit fed up with our previous mail provider. So we made the move. As an online retailer, we obviously get a lot of Financial related emails, and our Accountancy Department has an email address setup as a group, where the group sends to all the members in the Accounts Team. A pretty simple setup. However, for the last 2 months, I ve been fighting an uphill battle to get this to work properly. When an email gets delivered to a Google Group, it stops there, and then Google re-sends the email to the people in the group. Somehow, this triggers the receiving account to believe that it s originated from the Google Groups servers, rather than the actual originating server. Google Apps emails have a pretty nifty spam filtering service. For those well-known services on the web, it ll check whether it actually came from them (I think through some combination of DKIM and SPF), and bounce it if not. Can you see where I m going yet? Google Groups has a nifty feature to stop it attempting to send emails to addresses that don t exist. If an email address bounces at a certain rate, it ll flag it as undeliverable. If all the emails in a group are flagged as undeliverable, it ll bounce that email to the original party, so that they know no-one received it. The Accounts team are setup to receive information regarding Paypal Payments, Disputes, etc etc. A couple of days in, I had our Financial controller tell us that the Accounts team had stopped receiving emails. Well, being who I am I sent a test email out guess what I got back? Hello <redacted>, Your message could not be delivered because of previous failures during delivery attempts to this mailing list. Please update the list with valid addresses. This is an example of some of the bounces we received: ----------------------------------------------
Recipient: pa.......@mobilefun.co.uk 550 550 5.7.1 Unauthenticated email is not accepted from this domain. u22si29486554yba.55 (state 18).
Message-Id: <redacted> ----------------------------------------------
Recipient: ni........@mobilefun.co.uk 550 550 5.7.1 Unauthenticated email is not accepted from this domain. u22si29486554yba.55 (state 18).
Message-Id: <redacted> ----------------------------------------------
Recipient: lu..........@mobilefun.co.uk 550 550 5.7.1 Unauthenticated email is not accepted from this domain. u22si29486554yba.55 (state 18).
Message-Id: <redacted> *sigh* time to play with the group. I change the members of the group to use one of our alias domains instead of the @mobilefun.co.uk this gets emails working again. 2 days later . our Financial Controller comes to me and complains that clients are complaining that they re receiving bounce emails when sending to his team I check, and once again, get the same message. Time to open a support ticket. That was in October since then we ve tried everything Changing to user-defined groups and switching off the spam filters, whitelisting the groups, setting up DKIM on our domain, changing the spam levels, clearing bounce statuses all to no avail I ve been back and forthing with Google Enterprise Support for 2 months and have still not found a suitable solution *sigh*. I must say however, that the person dealing with the case at Google (Josephine H) has been professional and helpful all the way along, even with my rising frustration at the issue. Today I ended up calling their service unusable number, after having tried a couple of times to change the groups so that the Accounts Team could receive emails again and finding that none of my previous tricks worked. While on hold, I found that I could use a local part extension to an email address, and the group would recognise it as a new email address and therefore have no bounce status. I ve now made a script using the Provisioning APIs, and a bit of python-fu that will generate a local part extension based on the current date/time, and replace the users in the group for the Accounts team with those. Say for example, the primary email address for a user was test@example.com it d add a user of test+20101221235641@example.com. This is set to run each day so it s pretty much the same as resetting the bounce status (which fixes things for a short while) on a daily basis. According to my latest email from Google Enterprise support:-

At the moment there is an incentive going on to fix this outright as messages from Paypal etc have been causing bounce s for other domains. This fix is supposedly due very early in the New Year and will solve this problem indefinitely.

I wait with baited breath but for now, I m happy with my hack.

As of tonight, all translatable debconf templates for packages in squeeze are translated in German and French, after udev entered testing. Both teams are very happy to join the Russian team who was the very first to reach that goal on November 5th. Unfortunately, for our Russian friends, debian-edu-config is currently missing for them now. We of course hope for Debian Edu developers to correct this before squeeze. The Swedish team should join the 100% gang as soon as recently uploaded console-setup reaches testing. The Portuguese team is now only missing two complete packages to reach that 100%: sympa and (again) udev. udev is likely to be updated but, unfortunately, this is much less sure for sympa. The Czech team is only missing bugzilla. Again, unfortunately, that package needs an upload to testing-proposed-updates (that should fix and RC bug) and I am technically unable to do it (and the maintainer is apparently not motivated for doing it, arguing that a more recent upstream version should go in instead, which is unacceptable for the release team). The Spanish team made a great effort, but I'm not sure that the 16 packages they're missing will make it. Next time, folks! Full list for the "7 languages trying to reach 100%" at http://i18n.debian.net/debian-l10n/l10n-testing.txt

I would appreciate if you could test the proposed fixes for the exim4 privilege escalation bug CVE-2010-4345. Preliminary binary and source packages for squeeze/sid and lenny are available here:
deb http://www.bebt.de/debian/ sid exim4+cve
deb-src http://www.bebt.de/debian/ sid exim4+cve
deb http://www.bebt.de/debian/ lenny exim4+cve
deb-src http://www.bebt.de/debian/ lenny exim4+cve
You can also browse the changes in SVN (lenny and sid) or build your own binaries.

---

Changelog:

• 67_unnecessaryCopt.dpatch: Do not use exim's -C option in utility scripts. This would not work with ALT_CONFIG_PREFIX.
• Pull changes related to fixing CVE-2010-4345 from exim 4.73 rc1. Closes: #606527
  • 1_cfile_norw_eximuid: Don't allow a configure file which is writeable by the Exim user or group.
  • 2_permcheck_configurefile: Check configure file permissions even for non-default files if still privileged.
  • 3_remove_ALT_CONFIG_ROOT_ONLY: Remove ALT_CONFIG_ROOT_ONLY build option, effectively making it always true.
  • 4_FD_CLOEXEC: Set FD_CLOEXEC on SMTP sockets after forking in the daemon, to ensure that rogue child processes cannot use them.
  • 5_TRUSTED_CONFIG_LIST: Add TRUSTED_CONFIG_LIST compile option.
  • 6_nonroot_system_filter_user: If the system filter needs to be run as root, let that be explicitly configured. The default is now the Exim run-time user.
  • 7_filter_D_option: Add a (compiletime) whitelist of acceptable values for the -D option.
  • 8_updatedocumentation: Update documentation to reflect the changes.
• 4_FD_CLOEXEC replaces 80_fdleak.dpatch, drop the latter.
• Build with WHITELIST_D_MACROS=OUTGOING. Post patch 7_filter_D_option exim will not regain root privileges (usually necessary for local delivery) if the -D option was used. Macro identifiers listed in WHITELIST_D_MACROS are exempted from this restriction. mailscanner (4.79.11-2.2) uses -DOUTGOING.
• Build with TRUSTED_CONFIG_LIST=/etc/exim4/trusted_configs. Post patch 3_remove_ALT_CONFIG_ROOT_ONLY exim will not re-gain root privileges (usually necessary for local delivery) if the -C option was used. This makes it impossible to start a fully functional damon with an alternate configuration file. /etc/exim4/trusted_configs (can) contain a list of filenames (one per line, full path given) to which this restriction does not apply.

NEWS entry: Exim versions up to and including 4.72 are vulnerable to CVE-2010-4345. This is a privilege escalation issue that allows the exim user to gain root privileges by specifying an alternate configuration file using the -C option. The macro override facility (-D) might also be misused for this purpose. In reaction to this security vulnerability upstream has made a number of user visible changes. This package includes these changes. If exim is invoked with the -C or -D option the daemon will not regain root privileges though re-execution. This is usually necessary for local delivery, though. Therefore it is generally not possible anymore to run an exim daemon with -D or -C options. However this version of exim has been built with TRUSTED_CONFIG_LIST=/etc/exim4/trusted_configs. TRUSTED_CONFIG_LIST defines a list of configuration files which are trusted; if a config file is owned by root and matches a pathname in the list, then it may be invoked by the Exim build-time user without Exim relinquishing root privileges. As a hotfix to not break existing installations of mailscanner we have also set WHITELIST_D_MACROS=OUTGOING. i.e. it is still possible to start exim with -DOUTGOING while being able to do local deliveries. If you previously were using -D switches you will need to change your setup to use a separate configuration file. The ".include" mechanism makes this easy. The system filter is run as exim_user instead of root by default. If your setup requies root privileges when running the system filter you will need to set the system_filter_user exim main configuration option.

We had a small celebration with friends at our local Italian restaurant, Mario's Italiano today, to mark Sarah's 30th birthday. The reason this milestone is a big deal is because, to put it simply, two of her direct ancestors didn't make it to it. The undetermined genetic connective tissue disorder that Sarah has, claimed the lives of her father and paternal aunt when they were both in their twenties, suddenly and two weeks apart from each other. The case made the journal of the American College of Chest Physicians at the time. We're just very fortunate that with advances in medical diagnostic and surgical technology, Sarah was able to be diagnosed and have her aortic aneurysm repaired before it dissected. So there's the heart thing. That's a big deal. Then there's all the other crap that she's had to go through in only thirty years of life. She's had to endure more tragedies and medical challenges than some people have to go through in a lifetime. The fact that she's still such a happy, positive person is a testament to just what an amazing person she is, and it is an honour to be able to call her my wife. So happy birthday Sarah, here's to the next thirty years being a whole lot less eventful than the first thirty, and the last three in particular.

On the last weekend my team and I participated in the Mobile App Camp (link in Lithuanian) organized by Omnitel, which is one of the dominant mobile operators in Lithuania. The event seems to be part of Omnitel's push for increasing the share of smartphones in the Lithuanian market, which has been lagging behind the European trends.

Our team consisted of Povilas Kytra, who is behind the TV.LT project, Mantas Kanaporis from A-Gain and me. In the weekend we built an app that shows the TV programme for the coming day for all the Lithuanian TV channels. (The app is not yet available on the Android Market, but we are working on it.)

Here's a screenshot of the main screen of the application:

When I first started using Debian properly, I played with gpg-agent and pinentry but I didn t really understand the various bash initialisation scripts, and my botched setup annoyed me so much I disabled it again quite quickly (for example, if I left the machine logged in to GDM at home then logged in through SSH, pinentry dialogs stayed in GDM and had to be killed). Now I ve finally had time to get it sorted out, and this is part aide-memoire and part idiot check (hello, lazyweb). The brief:

• gpg-agent should take care of both GPG keys and SSH keys, transparently;
• pinentry should ask with pinentry-gtk2 at home, or at the current SSH session remotely;
• nothing should be left lying around after a session.

I assume appropriate packages are already installed at a minimum, gpg-agent and whichever pinentry you prefer. Installing gpg-agent drops a hook into /etc/X11/Xsession.d, but now there are both gpg-agent and ssh-agent running. Disable ssh-agent because we re going to let gpg-agent take care of everything: in /etc/X11/Xsession.options, comment out the line use-ssh-agent. In /etc/X11/Xsession.d/90gpg-agent, add the enable-ssh-support parameter so the STARTUP variable looks something like:

STARTUP="$GPGAGENT --daemon --sh --enable-ssh-support --write-env-file=$PID_FILE $STARTUP"

and also disable ssh in gnome-keyring:

$ gconftool-2 --set -t bool /apps/gnome-keyring/daemon-components/ssh false

Now (after restarting your X session) you can use ssh-add to import a key into gpg-agent, and you should see a pinentry dialog asking for a password. That takes care of X sessions, but it won t work for SSH. This is where the different bash login scripts matter: ~/.bashrc is run for all non-login shells (that is, ones spawned from e.g. gnome-terminal) but ~/.profile is only run for login shells (that is, ones where you have to authenticate, such as SSH connections). So, at the bottom of ~/.profile, we just start another gpg-agent who doesn t know about X:

eval $(gpg-agent --enable-ssh-support --daemon)

After restarting your SSH session and triggering a key operation, like signing a document, pinentry-curses should ask for your passphrase at the SSH shell and not leave dialogs stranded in the X session. Finally, that second agent shouldn t be left hanging around after we ve finished, or there ll be hundreds of them soon. Fortunately, the PID is recorded in the environment variable GPG_AGENT_INFO, but parsing that is a bit awkward when it s also recorded alone in SSH_AGENT_PID. So in ~/.bash_logout, we just

kill $SSH_AGENT_PID

(Most of the pointers for this setup came from http://wiki.kumina.nl/index.php/Managing_ssh_keys_with_gpg-agent and http://stefaanlippens.net/bashrc_and_others)