Search Results: "neal"

13 June 2011

Joachim Breitner: 401 page family book published

Over the course of the last five years, some relatives of me and I have been working on a family book which lists all descendants of my grandgrandgrandgrandfather Isaak Riehm, born 1799, and his wife Charlotte. My task was everything related to maintaining the data, running the infrastructure and generating the final book. In the end, we have collected biographical details, often with the person s live story, of 703 descendants, 285 partners and 416 parents-in-law. Together with an extensive preface the book now has a respectable size of 401 pages, includes 144 pictures and maps and over 200 copies have been printed and sold through the family.
The book cover
Although I do not think that the details of the lives of my family is very interesting for the general public, or even suitable for general release, I do want to show the fruit of our work. Therefore, I have created a stripped-down on-line version that only covers the first four generations. I bundled the book body with the cover and the end sheets into one complete PDF file (caution, large: 54MB, and of course in German). Some nifty embellishments of the book that I especially like are the abstract looking descendant fan in the front-end and back-end sheets, the separating bars between the persons which indicate the livespan of that person between 1799 and 2011 and the maps of the birth places and Isaak s travel destinations in the preface. Now to the technical details: Naturally, I used only Free Software in the genesis of the work (The cover is an exception, as it was not created by me). The most important is definitely gramps, a free genealogy system that held all our data. It is written in python and extensible with plug-ins, so I was able to create a custom report that takes the data relevant for us, filters for people who have agreed to appear in the book, and then generate LaTeX code that typesets the book, ready to be printed. I learned a great deal of LaTeX for this project, which uses over 30 different packages and required me to create a font of my own (which, as a side product, lead to a publication in the Linux-Magazin). For the maps I found the Generic Mapping Tools (GMT) and the freely available data for that of great use. I still had to manually collect the coordinates for all the places I wanted to show up in the map. Here, I relied mostly on Wikipedia. The fan which appears in the end sheets is laid out using a Haskell program (naturally, I have to use Haskell somewhere).
The front-end sheet
Of course, generating the book is only one side of the story; first, the data has to be collected and edited. For that, I wrote a plug-in to gramps that creates forms for each person. These forms contain the current known data and are used to correct and extend our knowledge. One variant of the form, created using LaTeX, was printed and mailed by snail-mail around the country and beyond, for those who are not reachable by mail. The other variant is an HTML form, hosted on a password-protected web site. Changes to that form cause the server to send an e-mail with a description of the changes (and only the changes, not the fields that are left untouched) to the mailing list of our team. There, the changes can be reviewed, e.g. for spelling and stylistic issues, and finally I copy n pasted them into gramps. These parts are implemented in python as well. Since genealogical data is never complete or up-to-date, I have set up a password-protected mediawiki based web site for the book where we collect corrections and additions to the data in the book. We have never anticipated that it would be so much work and that it would take us so long to finish this project, but we pulled it through and we are very happy with the result. Also, the reception has been very good and it was definitely a success. At this point I d like to express my gratitude to the hundreds of Free Software developers out there whose code we used and without whom it would not have been possible publish the book in the current form at all! Update: Due to some requests, I am making the code that I wrote available. You can download the tarball. Note that the code is a big mess and will not be usable right away. I guess if you can get the code to run and actually adjust to your needs, then you can as well write it yourself. Nevertheless it might be useful as inspiration or reference.

12 April 2011

Dirk Eddelbuettel: The new R compiler package in R 2.13.0: Some first experiments

R 2.13.0 will be released tomorrow. It contains a new package by R Core member Luke Tierney: compiler. What a simple and straightforward name, and something that Luke has been working on for several years. The NEWS file says
    o Package compiler is now provided as a standard package.  See
      ?compiler::compile for information on how to use the compiler.
      This package implements a byte code compiler for R: by default
      the compiler is not used in this release.  See the 'R
      Installation and Administration Manual' for how to compile the
      base and recommended packages.
so the compiler is not yet used for R's own base and recommended packages. While working on my slides for the upcoming Rcpp workshop preceding R/Finance 2011, I thought of a nice test example to illustrate the compiler. Last summer and fall, Radford Neal had sent a very nice, long and detailed list of possible patches for R performance improvements to the development list. Some patches were integrated and some more discussion ensued. One strand was on the difference in parsing between normal parens and curly braces. In isolation, these seem too large, but (as I recall) this is due some things 'deep down' in the parser. However, some folks really harped on this topic. And it just doesn't die as a post from last week demonstrated once more. Last year, Christian Robert had whittled it down to a set of functions, and I made a somewhat sarcastic post argueing that I'd rather use Rcpp to get 80-fold speed increase than spend my time argueing over ten percent changes in code that could be made faster so easily. So let us now examine what the compiler package can do for us. The starting point is the same as last year: five variants of computing 1/(1+x) for a scalar x inside an explicit for loop. Real code would never do it this way as vectorisation comes to the rescue. But for (Christian's) argument's sake, it is useful to highlight differences in the parser. We once again use the nice rbenchmark package to run, time and summarise alternatives:
> ## cf http://dirk.eddelbuettel.com/blog/2010/09/07#straight_curly_or_compiled
> f <- function(n, x=1) for (i in 1:n) x=1/(1+x)
> g <- function(n, x=1) for (i in 1:n) x=(1/(1+x))
> h <- function(n, x=1) for (i in 1:n) x=(1+x)^(-1)
> j <- function(n, x=1) for (i in 1:n) x= 1/ 1+x 
> k <- function(n, x=1) for (i in 1:n) x=1/ 1+x 
> ## now load some tools
> library(rbenchmark)
> ## now run the benchmark
> N <- 1e6
> benchmark(f(N,1), g(N,1), h(N,1), j(N,1), k(N,1),
+           columns=c("test", "replications",
+           "elapsed", "relative"),
+           order="relative", replications=10)
     test replications elapsed relative
5 k(N, 1)           10   9.764  1.00000
1 f(N, 1)           10   9.998  1.02397
4 j(N, 1)           10  11.019  1.12853
2 g(N, 1)           10  11.822  1.21077
3 h(N, 1)           10  14.560  1.49119
This replicates Christian's result. We find that function k() is the fastest using curlies, and that explicit exponentiation in function h() is the slowest with a relative penalty of 49%, or an absolute difference of almost five seconds between the 9.7 for the winner and 14.6 for the worst variant. On the other hand, function f(), the normal way of writing things, does pretty well. So what happens when we throw the compiler into the mix? Let's first create compiled variants using the new cmpfun() function and then try again:
> ## R 2.13.0 brings this toy
> library(compiler)
> lf <- cmpfun(f)
> lg <- cmpfun(g)
> lh <- cmpfun(h)
> lj <- cmpfun(j)
> lk <- cmpfun(k)
> # now run the benchmark
> N <- 1e6
> benchmark(f(N,1), g(N,1), h(N,1), j(N,1), k(N,1),
+           lf(N,1), lg(N,1), lh(N,1), lj(N,1), lk(N,1),
+           columns=c("test", "replications",
+           "elapsed", "relative"),
+           order="relative", replications=10)
       test replications elapsed relative
9  lj(N, 1)           10   2.971  1.00000
10 lk(N, 1)           10   2.980  1.00303
6  lf(N, 1)           10   2.998  1.00909
7  lg(N, 1)           10   3.007  1.01212
8  lh(N, 1)           10   4.024  1.35443
1   f(N, 1)           10   9.479  3.19051
5   k(N, 1)           10   9.526  3.20633
4   j(N, 1)           10  10.775  3.62673
2   g(N, 1)           10  11.299  3.80310
3   h(N, 1)           10  14.049  4.72871
Now things have gotten interesting and substantially faster, for very little cost. Usage is straightforward: take your function and compile it, and reap more than a threefold speed gain. Not bad at all. Also of note, the difference between the different expressions essentially vanishes. The explicit exponentiation is still the loser, but there may be an additional explicit function call involved. So we do see the new compiler as a potentially very useful addition. I am sure more folks will jump on this and run more tests to find clearer corner cases. To finish, we have to of course once more go back to Rcpp for some <emph>real</emph> gains, and some comparison between compiled and byte code compiled code.
> ## now with Rcpp and C++
> library(inline)
> ## and define our version in C++
> src <- 'int n = as<int>(ns);
+         double x = as<double>(xs);
+         for (int i=0; i<n; i++) x=1/(1+x);
+         return wrap(x); '
> l <- cxxfunction(signature(ns="integer",
+                            xs="numeric"),
+                  body=src, plugin="Rcpp")
> ## now run the benchmark again
> benchmark(f(N,1), g(N,1), h(N,1), j(N,1), k(N,1),
+           l(N,1),
+           lf(N,1), lg(N,1), lh(N,1), lj(N,1), lk(N,1),
+           columns=c("test", "replications",
+           "elapsed", "relative"),
+           order="relative", replications=10)
       test replications elapsed relative
6   l(N, 1)           10   0.120   1.0000
11 lk(N, 1)           10   2.961  24.6750
7  lf(N, 1)           10   3.128  26.0667
8  lg(N, 1)           10   3.140  26.1667
10 lj(N, 1)           10   3.161  26.3417
9  lh(N, 1)           10   4.212  35.1000
5   k(N, 1)           10   9.500  79.1667
1   f(N, 1)           10   9.621  80.1750
4   j(N, 1)           10  10.868  90.5667
2   g(N, 1)           10  11.409  95.0750
3   h(N, 1)           10  14.077 117.3083
Rcpp still shoots the lights out by a factor of 80 (or even almost 120 to the worst manual implementation) relative to interpreted code. Relative to the compiled byte code, the speed difference is about 25-fold. Now, these are of course entirely unrealistic code examples that are in no way, shape or form representative of real R work. Effective speed gains will be smaller for both the (pretty exciting new) compiler package and also for our C++ integration package Rcpp. Before I close, two more public service announcements. First, if you use Ubuntu see this post by Michael on r-sig-debian announcing his implementation of a suggestion of mine: we now have R alpha/beta/rc builds via his Launchpad PPA. Last Friday, I had the current R-rc snapshot of R 2.13.0 on my Ubuntu box only about six hours after I (as Debian maintainer for R) uploaded the underlying new R-rc package build to Debian unstable. This will be nice for testing of upcoming releases. Second, as I mentioned, the Rcpp workshop on April 28 preceding R/Finance 2011 on April 29 and 30 still has a few slots available, as has the conference itself.

3 March 2011

Rapha&#235;l Hertzog: People behind Debian: Christian Perrier, translation coordinator

Christian is a figure of Debian, not only because of the tremendous coordination work that he does within the translation project, but also because he s very involved at the social level. He s probably in the top 5 of the persons who attended most often the Debian conference. Christian is a friend (thanks for hosting me so many times when I come to Paris for Debian related events) and I m glad that he accepted to be interviewed. He likes to speak and that shows in the length of his answers :-) but you ll be traveling the world while reading him. My questions are in bold, the rest is by Christian. Who are you? I am a French citizen (which is easy to guess unless you correct my usual mistakes in what follows). I m immensely proud of being married for nearly 26 years with Elizabeth (who deserves a statue from Debian for being so patient with my passion and my dedication to the project). I m also the proud father of 3 wonderful kids , aged 19 to 23. I work as team manager in the Networks and Computers Division of Onera the French Aerospace lab , a public research institute about Aeronautics, Space and Defense. My team provides computer management services for research divisions of Onera, with a specific focus put on individual computing. I entered the world of free software as one of the very first users of Linux in France. Back in the early 1990 s, I happened (though the BBS users communities) to be a friend of several early adopters of Linux and/or BSD386/FreeBSD/NetBSD in France. More specifically, I discovered Linux thanks with my friend Ren Cougnenc (all my free software talks are dedicated to Ren , who passed away in 1996). You re not a programmer, not even a packager. How did you come to Debian? I m definitely not a programmer and I never studied computing (I graduated in Materials Science and worked in that area for a few years after my PhD). However, my daily work always involved computing (I redesigned the creep testing laboratory and its acquisition system all by myself during my thesis research work). An my hobbies often involved playing with home computers, always trying to learn about something new. So, first learning about a new operating system then trying to figure out how to become involved in its development was quite a logical choice. Debian is my distro of choice since it exists. I used Slackware on work machines for a while, but my home server, kheops, first ran Debian 1.1 when I stopped running a BBS on an MS-DOS machine to host a news server. That was back in October 1996. I then happened to be a user, and more specifically a user of genealogy software, also participating very actively in Usenet from this home computer and server, that was running this Debian thing. So, progressively, I joined mailing lists and, being a passionate person, I tried to figure out how I could bring my own little contribution to all this. This is why I became a packager (yes, I am one!) by taking over the geneweb package, which I was using to publish my genealogy research. I applied as DD in January 2001, then got my account in July 2001. My first upload to the Debian archive occurred on August 22nd 2001: that was of course geneweb, which I still maintain. Quite quickly, I became involved in the work on French localization. I have always been a strong supporter of localized software (I even translated a few BBS software back in the early 90 s) as one of the way to bring the power and richness of free software to more users. Localization work lead me to work on the early version of Debian Installer, during those 2003-2005 years where the development of D-I was an incredibly motivating and challenging task, lead by Joey Hess and his inspiring ideas. From user to contributor to leader, I suddenly discovered, around 2004, that I became the coordinator of D-I i18n (internationalization) without even noticing :-) You re the main translation coordinator in Debian. What plans and goals have you set for Debian Wheezy? As always: paint the world in red. Indeed, this is my goal for years. I would like our favorite distro to be able to be used by anyone in the world, whether she speaks English, Northern Sami, Wolof, Uyghur or Secwepemcts n. As a matter of symbol, I use the installer for this. My stance is that one should be able to even install Debian in one s own language. So, for about 7 years, I use D-I as a way to attract new localization contributors. This progress is represented on this page where the world is gradually painted in red as long as the installer supports more languages release after release. The map above tries to illustrate this by painting in red countries when the most spoken language in the country is supported in Debian Installer. However, that map does not give enough reward to many great efforts made to support very different kind of languages. Not only various national languages, but also very different ones: all regional languages of Spain, many of the most spoken languages in India, minority languages such as Uyghur for which an effort is starting, Northern Sami because it is taught in a few schools in Norway, etc., etc. Still, the map gives a good idea of what I would like to see better supported: languages from Africa, several languages in Central Asia. And, as a very very personal goal, I m eagerly waiting for support of Tibetan in Debian Installer, the same way we support its sister language, Dzongkha from Bhutan. For this to happen, we have to make contribution to localization as easy as possible. The very distributed nature of Debian development makes this a challenge, as material to translate (D-I components, debconf screens, native packages, packages descriptions, website, documentation) is very widely spread. A goal, for years, is to set a centralized place where translators could work easily without even knowing about SVN/GIT/BZR or having to report bugs to send their work. The point, however, would be to have this without making compromises on translation quality. So, with peer review, use of thesaurus and translation memory and all such techniques. Tools for this exist: we, for instance, worked with the developers of Pootle to help making it able to cope with the huge amount of material in Debian (think about packages descriptions translations). However, as of now, the glue between such tools and the raw material (that often lies in packages) didn t come. So, currently, translation work in Debian requires a great knowledge of how things are organized, where is the material, how it can be possible to make contribution reach packages, etc. And, as I m technically unable to fulfill the goal of building the infrastructure, I m fulfilling that role of spreading out the knowledge. This is how I can define my coordinator role. Ubuntu uses a web-based tool to make it easy to contribute translations directly in Launchpad. At some point you asked Canonical to make it free software. Launchpad has been freed in the mean time. Have you (re)considered using it? Why not? After all, it more or less fills in the needs I just described. I still don t really figure out how we could have all Debian material gathered in Rosetta/Launchpad .and also how Debian packagers could easily get localized material back from the framework without changing their development processes. I have always tried to stay neutral wrt Ubuntu. As many people now in Debian, I feel like we have reached a good way to achieve our mutual development. When it comes at localization work, the early days where the everything in Rosetta and translates who wants stanza did a lot of harm to several upstream localization projects is, I think, way over. Many people who currently contribute to D-I localization were indeed sent to me by Ubuntu contributors .and by localizing D-I, apt, debconf, package descriptions, etc., they re doing translation work for Ubuntu as well as for Debian. Let s say I m a Debian user and I want to help translate Debian in my language. I can spend 1 hour per week on this activity. What should I do to start? Several language teams use Debian mailing lists to coordinate their work. If you re lucky enough to be a speaker of one of these languages, try joining debian-l10n-<yourlanguage> and follow what s happening there. Don t try to immediately jump in some translation work. First, participate to peer reviews: comment on others translations. Learn about the team s processes, jargon and habits. Then, progressively, start working on a few translations: you may want to start with translations of debconf templates: they are short, often easy to do. That s perfect if you have few time. If no language team exists for your language, try joining debian-i18n and ask about existing effort for your language. I may be able to point you to individuals working on Debian translations (very often along with other free software translation efforts). If I am not, then you have just been named coordinator for your language :-) I may even ask you if you want to work on translating the Debian Installer. What s the biggest problem of Debian? We have no problems, we only have solutions :-) We are maybe facing a growth problem for a few years. Despite the increased welcoming aspects of our processes (Debian Maintainers), Debian is having hard times in growing. The overall number of active contributors is probably stagnating for quite a while. I m still amazed, however, to see how we can cope with that and still be able to release over the years. So, after all, this is maybe not a problem :-) Many people would point communication problems here. I don t. I think that communication inside the Debian project is working fairly well now. Our famous flame wars do of course still happen from time to time, but what large free software project doesn t have flame wars? In many areas, we indeed improved communication very significantly. I want to take as an example the way the release of squeeze has been managed. I think that the release team did, even more this time, a very significant and visible effort to communicate with the entire project. And the release of squeeze has been a great success in that matter. So, there s nearly nothing that frustrates me in Debian. Even when a random developer breaks my beloved 100% completeness of French translations, I m not frustrated for more than 2 minutes. You re known in the Debian community as the organizer of the Cheese & Wine Party during DebConf. Can you tell us what this is about? This is an interesting story about how things build themselves in Debian. It all started in July 2005, before DebConf 5 in Helsinki. Denis Barbier, Nicolas Fran ois and myself agreed to bring at Debconf a few pieces of French cheese as well as 1 or 2 bottles of French wine and share them with some friends. Thus, we settled an informal meeting in the French room where we invited some fellows: from memory, Benjamin Mako Hill, Hannah Wallach, Matt Zimmermann and Moray Allan. All of us fond of smelly cheese, great wine plus some extra p t home-made by Denis in Toulouse. It finally happened that, by word of mouth, a few dozens of other people slowly joined in that French room and turned the whole thing into an improvized party that more or less lasted for the entire night. The tradition was later firmly settled in 2006, first in Debconf 6 in Mexico where I challenged the French DDs to bring as many great cheese as possible, then during the Debian i18n meeting in Extremadura (Sept 2006) where we reached the highest amount of cheese per participant ever. I think that the Creofonte building in Casar de C ceres hasn t fully recovered from it and is still smelling cheese 5 years after. This party later became a real tradition for DebConf, growing over and over each year. I see it as a wonderful way to illustrate the diversity we have in Debian, as well as the mutual enrichment we always felt during DebConfs. My only regret about it is that it became so big over the years that organizing it is always a challenge and I more and more feel pressure to make it successful. However, over the years, I always found incredible help by DebConf participants (including my own son, last year a moment of sharing which we will both remember for years, i think). And, really, in 2010, standing up on a chair, shouting (because the microphone wasn t working) to thank everybody, was the most emotional moment I had at Debconf 10. Is there someone in Debian that you admire for their contributions? So many people. So, just like it happens in many awards ceremonies, I will be very verbose to thank people, sorry in advance for this. The name that comes first is Joey Hess. Joey is someone who has a unique way to perceive what improvements are good for Debian and a very precise and meticulous way to design these improvements. Think about debconf. It is designed for so long now and still reaching its very specific goal. So well designed that it is the entire basis for Joey s other achievement: designing D-I. Moreover, I not only admire Joey for his technical work, but also for his interaction with others. He is not he loudest person around, he doesn t have to .just giving his point in discussion and, guess what? Most of the time, he s right. Someone I would like to name here, also, is Colin Watson. Colin is also someone I worked with for years (the D-I effect, again ) and, here again, the very clever way he works on technical improvements as well as his very friendly way to interact with others just make it. And, how about you, Rapha l? :-) I m really admirative of the way you work on promoting technical work on Debian. Your natural ability to explain things (as good in English as it is in French) and your motivation to share your knowledge are a great benefit for the project. Not to mention the technical achievements you made with Guillem on dpkg of course! Another person I d like to name here is Steve Langasek. We both maintain samba packages for years and collaboration with him has always been a pleasure. Just like Colin, Steve is IMHO a model to follow when it comes at people who work for Canonical while continuing their involvment in Debian. And, indeed, Steve is so patient with my mistakes and stupid questions in samba packaging that he deserves a statue. We re now reaching the end of the year where Stefano Zacchiroli was the Debian Project Leader. And, no offense intended to people who were DPL before him (all of them being people I consider to be friends of mine), I think he did the best term ever. Zack is wonderful in sharing his enthusiasm about Debian and has a unique way to do it. Up to the very end of his term, he has always been working on various aspects of the project and my only hope is that he ll run again (however, I would very well understand that he wants to go back to his hacking activities!). Hat off, Zack!I again have several other people to name in this Bubulle hall of Fame : Don Armstrong, for his constant work on improving Debian BTS, Margarita Manterola as one of the best successes of Debian Women (and the most geeky honeymoon ever), Denis Barbier and Nicolas Fran ois because i18n need really skilled people, Cyril Brulebois and Julien Cristau who kept X.org packaging alive in lenny and squeeze, Otavio Salvador who never gave up on D-I even when we were so few to care about it. I would like to make a special mention for Frans Pop. His loss in 2010 has been a shock for many of us, and particularly me. Frans and I had a similar history in Debian, both mostly working on so-called non technical duties. Frans has been the best release manager for D-I (no offense intended, at all, to Joey or Otavio .I know that both of them share this feeling with me). His very high involvment in his work and the very meticulous way he was doing it lead to great achievements in the installer. The Installation Guide work was also a model and indeed a great example of non technical work that requires as many skills as more classical technical work. So, and even though he was sometimes so picky and, I have to admit, annoying, that explains why I m still feeling sad and, in some way, guilty about Frans loss. One of my goals for wheezy is indeed to complete some things Frans left unachieved. I just found one in bug #564441: I will make this work reach the archive, benefit our users and I know that Frans would have liked that.
Thank you to Christian for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

7 comments Liked this article? Click here. My blog is Flattr-enabled.

25 December 2010

Dirk Eddelbuettel: Rcpp 0.9.0 announcement

The text below went out as a post to the r-packages list a few days ago, but I thought it would make sense to post it on the blog too. So with a little html markup... Summary Version 0.9.0 of the Rcpp package is now on CRAN and its mirrors. This release marks another step in the development of the package, and a few key points are highlighted below. More details are in the NEWS and ChangeLog files included in the package. Overview Rcpp is an R package and associated C++ library that facilitates integration of C++ code in R packages. The package features a complete set of C++ classes (Rcpp::IntegerVector, Rcpp:NumericVector, Rcpp::Function, Rcpp::Environment, ...) that makes it easier to manipulate R objects of matching types (integer vectors, functions, environments, etc ...). Rcpp takes advantage of C++ language features such as the explicit constructor / destructor lifecycle of objects to manage garbage collection automatically and transparently. We believe this is a major improvement over use of PROTECT / UNPROTECT. When an Rcpp object is created, it protects the underlying SEXP so that the garbage collector does not attempt to reclaim the memory. This protection is withdrawn when the object goes out of scope. Moreover, users generally do not need to manage memory directly (via calls to new / delete or malloc / free) as this is done by the Rcpp classes or the corresponding STL containers. A few key points about Rcpp:

Several key features were added during the 0.8.* cycles and are described below. Rcpp sugar Rcpp now provides syntactic sugar: vectorised expressions at the C++ level which are motivated by the corresponding R expressions. This covers operators (binary arithmetic, binary logical, unary), functions (producing single logical results, mathematical functions and d/p/q/r statistical functions). Examples comprises anything from ifelse() to pmin()/pmax() or A really simply example is a function
    SEXP foo( SEXP xx, SEXP yy) 
        NumericVector x(xx), y(yy) ;
        return ifelse( x < y, x*x, -(y*y) ) ;
     
which deploys the sugar 'ifelse' function modeled after the corresponding R function. Another simple example is
    double square( double x) 
        return x*x ;
     
    SEXP foo( SEXP xx ) 
        NumericVector x(xx) ;
        return sapply( x, square ) ;
     
where use the sugar function 'sapply' to sweep a simple C++ function which operates elementwise across the supplied vector. The Rcpp-sugar vignette describes sugar in more detail. Rcpp modules Rcpp modules are inspired by Boost.Python and make exposing C++ functions or classes to R even easier. A first illustration is provided by this simple C++ code snippet
    const char* hello( const std::string& who ) 
        std::string result( "hello " ) ;
        result += who ;
        return result.c_str() ;
     
    RCPP_MODULE(yada) 
        using namespace Rcpp ;
        function( "hello", &hello ) ;
     
which (after compiling and loading) we can access in R as
    yada <- Module( "yada" )
    yada$hello( "world" )
In a similar way, C++ classes can be exposed very easily. Rcpp modules are also described in more detail in their own vignette. Reference Classes R release 2.12.0 introduced Reference Classes. These are formal S4 classes with the corresponding dispatch method, but passed by reference and easy to use. Reference Classes can also be exposed to R by using Rcpp modules. Extension packackages The RcppArmadillo package permits use of the advanced C++ library 'Armadillo, a C++ linear algebra library aiming towards a good balance between speed and ease of use, providing integer, floating point and complex matrices and vectors with lapack / blas support via R. Armadillo uses templates for a delayed evaluation approach is employed (during compile time) to combine several operations into one and reduce (or eliminate) the need for temporaries. Armadillo is useful if C++ has been decided as the language of choice, rather than another language like Matlab or Octave, and aims to be as expressive as the former. Via Rcpp and RcppArmadillo, R users now have easy access to this functionality. Examples are provided in the RcppArmadillo package. The RcppGSL package permits easy use of the GNU Scientific Library (GSL), a collection of numerical routines for scientifc computing. It is particularly useful for C and C++ programs as it provides a standard C interface to a wide range of mathematical routines such as special functions, permutations, combinations, fast fourier transforms, eigensystems, random numbers, quadrature, random distributions, quasi-random sequences, Monte Carlo integration, N-tuples, differential equations, simulated annealing, numerical differentiation, interpolation, series acceleration, Chebyshev approximations, root-finding, discrete Hankel transforms physical constants, basis splines and wavelets. There are over 1000 functions in total with an extensive test suite. The RcppGSL package provides an easy-to-use interface between GSL data structures and R using concepts from Rcpp. The RcppGSL package also contains a vignette with more documentation. Legacy 'classic' API Packages still using code interfacing the initial 'classic' Rcpp API are encouraged to migrate to the new API. Should a code transition not be possible, backwards compatibility is provided by the RcppClassic package released alongside Rcpp 0.9.0. By including RcppClassic.h and building against the RcppClassic package and library, vintage code can remain operational using the classic API. The short vignette in the RcppClassic package has more details. Documentation The package contains a total of eight vignettes the first of which provides a short and succinct introduction to the Rcpp package along with several motivating examples. Links Support Questions about Rcpp should be directed to the Rcpp-devel mailing list https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
Dirk Eddelbuettel, Romain Francois, Doug Bates and John Chambers
December 2010

8 September 2010

Dirk Eddelbuettel: Straight, curly, or compiled?

Christian Robert, whose blog I commented-on here once before, had followed up on a recent set of posts by Radford Neal which had appeared both on Radford's blog and on the r-devel mailing list. Now, let me prefix this by saying that I really enjoyed Radford's posts. He obviously put a lot of time into finding a number of (all somewhat small in isolation) inefficiencies in R which, when taken together, can make a difference in performance. I already spotted one commit by Duncan in the SVN logs for R so this is being looked at. Yet Christian, on the other hand, goes a little overboard in bemoaning performance differences somewhere between ten and fifteen percent -- the difference between curly and straight braces (as noticed in Radford's first post). Maybe he spent too much time waiting for his MCMC runs to finish to realize the obvious: compiled code is evidently much faster. And before everybody goes and moans and groans that that is hard, allow me to just interject and note that it is not. It really doesn't have to be. Here is a quick cleaned up version of Christian's example code, with proper assigment operators and a second variable x. We then get to the meat and potatoes and load our Rcpp package as well as inline to define the same little test function in C++. Throw in rbenchmark which I am becoming increasingly fond of for these little timing tests, et voila, we have ourselves a horserace:
# Xian's code, using <- for assignments and passing x down
f <- function(n, x=1) for (i in 1:n) x=1/(1+x)
g <- function(n, x=1) for (i in 1:n) x=(1/(1+x))
h <- function(n, x=1) for (i in 1:n) x=(1+x)^(-1)
j <- function(n, x=1) for (i in 1:n) x= 1/ 1+x 
k <- function(n, x=1) for (i in 1:n) x=1/ 1+x 
# now load some tools
library(Rcpp)
library(inline)
# and define our version in C++
l <- cxxfunction(signature(ns="integer", xs="numeric"),
                 'int n = as<int>(ns); double x=as<double>(xs);
                  for (int i=0; i<n; i++) x=1/(1+x);
                  return wrap(x); ',
                 plugin="Rcpp")
# more tools
library(rbenchmark)
# now run the benchmark
N <- 1e6
benchmark(f(N, 1), g(N, 1), h(N, 1), j(N, 1), k(N, 1), l(N, 1),
          columns=c("test", "replications", "elapsed", "relative"),
          order="relative", replications=10)
And how does it do? Well, glad you asked. On my i7, which the other three cores standing around and watching, we get an eighty-fold increase relative to the best interpreted version:
/tmp$ Rscript xian.R
Loading required package: methods
     test replications elapsed relative
6 l(N, 1)           10   0.122    1.000
5 k(N, 1)           10   9.880   80.984
1 f(N, 1)           10   9.978   81.787
4 j(N, 1)           10  11.293   92.566
2 g(N, 1)           10  12.027   98.582
3 h(N, 1)           10  15.372  126.000
/tmp$ 
So do we really want to spend time arguing about the ten and fifteen percent differences? Moore's law gets you those gains in a couple of weeks anyway. I'd much rather have a conversation about how we can get people speed increases that are orders of magnitude, not fractions. Rcpp is one such tool. Let's get more of them.

7 April 2010

Biella Coleman: Nameless, Freak, Phreak

So I tend to focus on the cultural present but during grad school it was hammered into my brain that to understand the contemporary moment, history matters, a lot.. The problem with hacking and related activities is that the history is a bit sparse and fragmented. Recently I have been working on a piece that examines some of the aesthetics similarities between phrakers, underground hackers, and trollers and it sort of hit me that I had no idea when the term phreak or freak came into being. So I dropped a note to Phil Lapsely who is writing (finally) a proper history of phreaking and he was thoughtful enough to pen down a genealogy of the term.

Biella Coleman: Nameless, Freak, Phreak

So I tend to focus on the cultural present but during grad school it was hammered into my brain that to understand the contemporary moment, history matters, a lot.. The problem with hacking and related activities is that the history is a bit sparse and fragmented. Recently I have been working on a piece that examines some of the aesthetics similarities between phrakers, underground hackers, and trollers and it sort of hit me that I had no idea when the term phreak or freak came into being. So I dropped a note to Phil Lapsely who is writing (finally) a proper history of phreaking and he was thoughtful enough to pen down a genealogy of the term.

26 February 2010

Dirk Eddelbuettel: R and Sudoku solvers: Plus ca change...

Christian Robert blogged about a particularly heavy-handed solution to last Sunday's Sudoku puzzle in Le Monde. That had my symapthy as I like evolutionary computing methods, and his chart is rather pretty. From there, this spread on to the REvolutions blogs where David Smith riffed on it, and showed the acual puzzle. That didn't stop things as Christian blogged once more about it, this time welcoming his post-doc Robin Ryder who posts a heavy analysis on all this that is a little much for me at this time of day. But what everybody seems to be forgetting is that R has had a Sudoku solver for years, thanks to the sudoku package by David Brahm and Greg Snow which was first posted four years ago. What comes around, goes around. With that, and about one minute of Emacs editing to get the Le Monde puzzle into the required ascii-art form, all we need to do is this:
R> library(sudoku)
R> s <- readSudoku("/tmp/sudoku.txt")
R> s
      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9]
 [1,]    8    0    0    0    0    1    2    0    0
 [2,]    0    7    5    0    0    0    0    0    0
 [3,]    0    0    0    0    5    0    0    6    4
 [4,]    0    0    7    0    0    0    0    0    6
 [5,]    9    0    0    7    0    0    0    0    0
 [6,]    5    2    0    0    0    9    0    4    7
 [7,]    2    3    1    0    0    0    0    0    0
 [8,]    0    0    6    0    2    0    1    0    9
 [9,]    0    0    0    0    0    0    0    0    0
R> system.time(solveSudoku(s))
      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9]
 [1,]    8    4    9    6    7    1    2    5    3
 [2,]    6    7    5    2    4    3    9    1    8
 [3,]    3    1    2    9    5    8    7    6    4
 [4,]    1    8    7    4    3    2    5    9    6
 [5,]    9    6    4    7    8    5    3    2    1
 [6,]    5    2    3    1    6    9    8    4    7
 [7,]    2    3    1    8    9    4    6    7    5
 [8,]    4    5    6    3    2    7    1    8    9
 [9,]    7    9    8    5    1    6    4    3    2
   user  system elapsed
  5.288   0.004   5.951
R>
That took all of five seconds while my computer was also compiling a particularly resource-hungry C++ package.... Just in case we needed another illustration that it is hard to navigate the riches and wonders that is CRAN...

19 November 2009

Biella Coleman: Phreaking your way into Egypt?

At times I am secretly (actually more openly) jealous of my colleagues in the English department who get to teach literature day in and day out. What a treat. So I always make sure to sneak at least one fictional story whether short such as Kurt Vonnegut s Unready to Wear in Welcome to the Monkey House, which is superb or something longer and way more verbose than Vonnegut who is the epitome of sparsity, such as Snow Crash by Neal Stephenson (Cryptonomicon is too long). I have just been informed by Phil Lapsley of one of the only fictional accounts on phone phreakers Loving Little Egypt and I might just make this the fictional account for my hacker class I am teaching next semester. I can never get enough of phreaking and since there is so little on the topic, this can top things off. Has anyone read it? Any thoughts? Any other fictional accounts on phreakers?

Biella Coleman: Phreaking your way into Egypt?

At times I am secretly (actually more openly) jealous of my colleagues in the English department who get to teach literature day in and day out. What a treat. So I always make sure to sneak at least one fictional story whether short such as Kurt Vonnegut s Unready to Wear in Welcome to the Monkey House, which is superb or something longer and way more verbose than Vonnegut who is the epitome of sparsity, such as Snow Crash by Neal Stephenson (Cryptonomicon is too long). I have just been informed by Phil Lapsley of one of the only fictional accounts on phone phreakers Loving Little Egypt and I might just make this the fictional account for my hacker class I am teaching next semester. I can never get enough of phreaking and since there is so little on the topic, this can top things off. Has anyone read it? Any thoughts? Any other fictional accounts on phreakers?

23 May 2009

John Goerzen: Review: In The Beginning. . . Was The Command Line

A few dud universes can really clutter up your basement. - Neal Stephenson, In The Beginning. . . was the Command Line
What a fun read. It s about technology, sure, but more about culture. Neal takes a good look at operating systems, why we get emotionally involved with them, and why Windows is still so popular. He does this with a grand detour to Disneyland, and a hefty dose of humor. The above quote was from near the end of the book, where he imagines hackers creating big bangs from the command line. He starts out the book from some anecdotes from the early 1970s, when he had his first computer class in high school. His school didn t have a computer, but they did have a teletype (the physical kind that used paper) with a modem link to some university s system. But time on that system was so expensive that they couldn t just dial in and run things interactively. The teletype had a paper tape device. You d type your commands in advance, and it would punch them out on the tape. Then when you dial in, it would replay the tape at high speed . Neal liked this because the stuff punched out of the tape were, actually, bits in both the literal and the mathematical sense. This, of course, led to a scene at the end of the schoolyear where a classmate dumped the bin of bits on the teacher, and Neal witnessed megabytes falling to the floor. Although the book was written in 1999, and needs an update in some ways, it still speaks with a strong voice today and is now also an interesting look at what computing was like 10 years ago. He had an analogy of car dealerships to operating systems. Microsoft had the big shiny dealership selling station wagons. Their image was all wrapped up in people feeling good about their purchase like they got something for their money. And he said that the Linux folks were selling tanks, illustrated with this exchange:
Hacker with bullhorn: Save your money! Accept one of our free tanks! It is invulnerable, and can drive across rocks and swamps at ninety miles an hour while getting a hundred miles to the gallon! Prospective station wagon buyer: I know what you say is true but er I don t know how to maintain a tank! Bullhorn: You don t know how to maintain a station wagon either! Buyer: But this dealership has mechanics on staff. If something goes wrong with my station wagon, I can take a day off work, bring it here, and pay them to work on it while I sit in the waiting room for hours, listening to elevator music. Bullhorn: But if you accept one of our free tanks we will send volunteers to your house to fix it for free while you sleep! Buyer: Stay away from my house, you freak! Bullhorn: But Buyer: Can t you see that everyone is buying station wagons?
That doesn t mean that Stephenson is just a Linux apologetic. He points out that the CLI has its place, and has a true love-hate relationship with the text-based config files (remember XF86Config before the days of automatic modelines? Back when you had to get out a calculator and work some things out with pencil and paper, or else risk burning out your monitor?) He points out that some people want to just have the thing work reasonably well. They don t want control in fact, would gladly give it up if offered something reasonably pretty and reasonably functional. He speaks to running Linux at times:
Sometimes when you finish working with a program and shut it down, you find that it has left behind a series of mild warnings and low-grade error messages in the command-line interface window from which you launched it. As if the software were chatting to you about how it was doing the whole time you were working with it.
Even if the application is imploding like a damaged submarine, it can still usually eke out a little S.O.S. message.
Or about booting Linux the first time, and noticing all sorts of cryptic messages on the console:
This is slightly alarming the first time you see it, but completely harmless.
I use emacs, which might be thought of as a thermonuclear word processor. . . Microsoft Word, were devoted to features like mail merge, and the ability to embed feature-length motion pictures in corporate memoranda, were, in the case of emacs, focused with maniacal intensity on the deceptively simple-seeming problem of editing text. If you are a professional writer i.e., if someone else is getting paid to worry about how your words are formatted and printed emacs outshines all other editing software in approximately the same way that the noonday sun does the stars. It is not just bigger and brighter; it simply makes everything else vanish. For page layout and printing you can use TeX: a vast corpus of typesetting lore written in C and also available on the Net for free.
I love these vivid descriptions: programs secretly chatting with us, TeX being a corpus of typesetting lore rather than a program. Or how about this one: Unix. . . is not so much a product as it is a painstakingly compiled oral history of the hacker subculture. It is our Gilgamesh epic. Yes, my operating system is an oral history project, thankyouverymuch. The book feels like a weird (but well-executed and well-written) cross between Douglas Adams and Cory Doctorow. Which makes is so indescribably awesome that I can t help but ending this review with a few more quotes.
Because Linux is not commercial because it is, in fact, free, as well as rather difficult to obtain, install, and operate it does not have to maintain any pretensions as to its reliability. Consequently, it is much more reliable.
what really sold me on it [Debian] was its phenomenal bug database (http://www.debian.org/Bugs), which is a sort of interactive Doomsday Book of error, fallibility, and redemption. It is simplicity itself. When had a problem with Debian in early January of 1997, I sent in a message describing the problem to submit@bugs.debian.org. My problem was promptly assigned a bug report number (#6518) and a severity level (the available choices being critical, grave, important, normal, fixed, and wishlist) and forwarded to mailing lists where Debian people hang out.
That should be our new slogan for bugs.debian.org: Debian s interactive Doomsday Book of error, fallibility, and redemption.
Unix is hard to learn. The process of learning it is one of multiple small epiphanies. Typically you are just on the verge of inventing some necessary tool or utility when you realize that someone else has already invented it, and built it in, and this explains some odd file or directory or command that you have noticed but never really understood before.
I ve been THERE countless times.
Note the obsessive use of abbreviations and avoidance of capital letters; this is a system invented by people to whom repetitive stress disorder is what black lung is to miners. Long names get worn down to three-letter nubbins, like stones smoothed by a river.
It is obvious, to everyone outside of the United States, that our arch-buzzwords, multiculturalism and diversity, are false fronts that are being used (in many cases unwittingly) to conceal a global trend to eradicate cultural differences. The basic tenet of multiculturalism (or honoring diversity or whatever you want to call it) is that people need to stop judging each other-to stop asserting (and, eventually, to stop believing ) that this is right and that is wrong, this true and that false, one thing ugly and another thing beautiful, that God exists and has this or that set of qualities.
The stone tablets bearing the Ten Commandments carved in immutable stone the original command-line interface
Apparently this actually works to some degree, for police in many lands are now complaining that local arrestees are insisting on having their Miranda rights read to them, just like perps in American TV cop shows. When it s explained to them that they are in a different country, where those rights do not exist, they become outraged. Starsky and Hutch reruns, dubbed into diverse languages, may turn out, in the long run, to be a greater force for human rights than the Declaration of Independence.
Unix has always lurked provocatively in the background of the operating system wars, like the Russian Army.
Available for free online, or as a 160-page book from Amazon.

28 January 2009

Russell Coker: Links January 2009

Jennifer 8 Lee gave an interesting TED talk about the spread and evolution of what is called Chinese food [1]. In that talk she compares McDonalds to Microsoft and Chinese restaurants to Linux. Her points comparing the different local variations of Chinese food to the variations of Linux make sense. The CentOS Plus repository has a kernel with support for the XFS filesystem, Postfix with MySQL support, and some other useful things [2]. Mary Gardiner comments about the recent loss of a blog server with all content [3]. One interesting point is that when you start using a service that maintains your data you should consider how to make personal backups in case the server goes away or you decide to stop being a customer. Val Henson makes some interesting points about the reliability of Solid State Disks (SSD) [4]. Some people are planning to replace RAID arrays of disks with a single SSD with the idea that a SSD will be more reliable, this seems like a bad idea. Also with the risk of corruption it seems that we have a greater need for filesystems that store block checksums. Lior Kaplan describes how to have multiple Linux bonding devices [5], the comment provides some interesting detail too. programmableweb.com has a set of links to sites that have APIs which can be used to create mashups [6]. One of the many things I would do if I had a lot more spare time is to play with some of the web APIs that are out there. Gunnar Wolf has written some insightful comments about the situation in Israel and Palestine [7]. He used to be a Zionist and spent some time living in Israel so he knows more about the topic than most commentators. Charles Stross has written an informative post about Ubuntu on the EeePC [8]. What is noteworthy about this is not that he s summarised the issues well, but that he is a well known science-fiction writer and he was responding to a SFWA member. One of his short stories is on my free short stories page [9]. He also wrote Accelerando which is one of the best sci-fi novels I ve read (and it s also free) [10]. Don Marti has written about Rent Seeking and proprietary software [11]. It s an interesting article, nothing really new for anyone who has followed the news about the coal and nuclear industries. Erik writes about The Setting Sun and points out that Scott McNealy had tried to capitalise on the SCO lawsuit but Red Hat has ended up beating them in the market [12].

23 December 2008

Joey Hess: personal wishlists 40319, 32

40319 An e-book reader where I can click on any word and enter a definition for it. The definition will then appear as a tooltip when I hover over the word. Take that, Iain Banks, and Neal Stephenson! 32 I want Deskview X's terminal program. The one that resizes the font when the window is resized. Except free and modern. With modern (ie, auto-layout tiling) window managers, this is less of a wishlist and more of a missing necessity. Ideally, it would always keep the window 80 columns (or some other user-defined value) wide. I've poked around in gnome-terminal's source, and this looks fairly doable, but I have not found the time to get down and do it.

22 December 2008

David Welton: Hecl on the G1

Thanks to Neal McBurnett, Hecl for Android finally got to run wild and free on a real phone. There were some glitches, but by and large, things seem to work, which is always exciting, and something of a relief. Neil says it runs fairly quickly, and looks good. Once again, a big thanks to Neil, and now on to fixing the bugs! Android itself continues to evolve: http://fredstechblog.blogspot.com/2008/12/big-updates-ahead-for-android.html Now, if only I could get ahold of one of those dev phones! Although, truth be told, I think the "G2" or whatever it is, will probably be a significant improvement in terms of fixing up some of what needs improvement over the first generation, so I'll probably wait until sometime next year and get one of those.

David Welton: Hecl on the G1

Thanks to Neal McBurnett, Hecl for Android finally got to run wild and free on a real phone. There were some glitches, but by and large, things seem to work, which is always exciting, and something of a relief. Neil says it runs fairly quickly, and looks good. Once again, a big thanks to Neil, and now on to fixing the bugs! Android itself continues to evolve: http://fredstechblog.blogspot.com/2008/12/big-updates-ahead-for-android.html Now, if only I could get ahold of one of those dev phones! Although, truth be told, I think the "G2" or whatever it is, will probably be a significant improvement in terms of fixing up some of what needs improvement over the first generation, so I'll probably wait until sometime next year and get one of those.

17 September 2008

Joey Hess: time

Sitting next to the fire after heating up fajitas for dinner. Making sure that no hot coals make it over to the canvas topped yurt. Maybe I should have made the fire further off. I've gotten 1/3 of the way through Anathem, and Neal Stephenson has thoroughly hooked me now. I know I won't have enough battery power to read all I'd like to tonight. But for now, I've been enjoying the evening, which has been unusually long today. Maybe it's a remnant of Hurricane Ike that's led to such a cool, cloudy day, but it's seemed to be on the verge of dusk for hours. Someone asked if the yurt has wifi. Nope, but I measured the distance today, and at 400 feet it's too far for cat 5 ethernet w/o a repeater. Anyway, I seem to be in a very good frame of mind for starting Anathem, and am enjoying its take on the Long Now. Out at the falls this weekend I kept imagining how they'd change in geologic time, tracking where the water seemed to have fallen earler, downstream. And sitting up under the overhang of the falls, was briefly nervous about it coming down (as it will in oh, a few thousand years perhaps), before getting back into the present moment.

15 September 2008

Erich Schubert: On Java Generics and subclassing

Due to their implementation by erasure, they face certain limitations.For example, the following constructor for a class with both compile time and runtime type checking:
class BagOf<T>  
  BagOf(Class<T> restrictionClass);
 
is not satisfiable when T is a generic class itself (since there is no ArraySet<Double>.class syntax, for example). The best work-around I know is to drop the T subclassing restriction for restrictionClass:
class BagOf<T>  
  BagOf(Class<?> restrictionClass);
 
The cost is low (obviously no difference at runtime) - you just don't assert that the developer using your class specifies a restriction class derived from the class T used in the generics. That won't prevent certain programming errors such as this anymore
BagOf<Integer> bar = BagOf<Integer>(Double.class)
but these shouldn't be too hard to find/fix anyway.Before submitting too clever suggestions, please make sure you've tested them. For example "if (obj instanceof T)" is not valid java code: since generics are implemented by erasure, T cannot be referenced in runtime statements.P.S. It would obviously be nice if the Java syntax would allow Foo<Bar>.class (which at runtime would be the same as Foo.class, and at complie time would have the result type Class<Foo<Bar>>), but currently it does not for all I know.P.P.S. I'm not looking for "Class<? extends T>", that is a different situation. The difficult case is when T is a Generic itself, not a subclass.Update: JM Ibanez pointed me to Neal Gafter's Super Type Tokens, which apparently are the same as TypeLiteral in Google Guice. Thanks!

14 July 2008

Simon Richter: It's all in my head now

I've spent four hours listening to muzak yesterday. Apparently, antitrust legislation prohibits T-Online from getting better access to T-Com's ticket system than their competitors. Of course that means "none at all". I do however, disagree with their corollary that it is the customer's duty to call T-Com and tell them what is wrong. On a Sunday evening. After a lightning strike. With a Windows update breaking PPPoE for ZoneAlarm users. Seriously, get your processes straight.

15 June 2008

Christian Perrier: On getting Ubuntu bug reports for our packages

Lucas pointed that the Package Tracking System now links to Ubuntu bug reports in Launchpad, for Debian packages. This is of course an interresting feature, which I immediately tried on geneweb, my pet package. Indeed, the only bug I found there comes from a user who was apparently not able to read a damn README.Debian file, which nicely explains what people are expected to do if they want to share the genealogical databases with Geneweb with the system-wide daemon. Maybe they should renamed README.Debian to README.Debian-but-readme-also-if-you-use-Ubuntu. In short, the only bug is user error. Now I really wonder how I could close that thing (yes, I did log in into Launchpad) and be able to claim that I have zero bugs on my pet package. I should do the same for samba, but I guess that Steve Langasek is more or less monitoring things there (LP is apparently Yet Another Thing You Have To Be Online To Use, which is a no-no for me).

8 June 2008

Biella Coleman: Anthropologocial Wonders and Myopias North and South

Four years ago, the last time I was in Brazil, I came as an anthropologist-in-training to attend and give a talk at Debconf4 held in Porto Alegre. I have returned to this city in the south, again in the winter, but this time I have come back as an anthropologist who is giving a talk at the department of anthropology at the UFRGS, based on my research conducted many years ago. This time, thanks to my (really friendly and energetic) hosts, I am seeing far more of the actual city, its beautiful and legendary sunsets, its outlying neighborhoods, and even its subcultures. This has been my first foreign trip where I am interacting primarily with anthropologists and this has been really interesting for me. On the one hand, because of the language difference, there is a lot that is strange and hard for me to follow and I am acutely aware of the general and particular economic gulf between north and south and its impact on students (books are expensive, traveling for conferences is very difficult, and even applying to graduate schools in the north can be impossible because it costs so much to apply to each school!) On the other hand, the methods and subjects of study, the style of analysis, and the teaching are all very familiar, making me feel like I am part of a discipline that (thankfully) transcends national borders. Many of the projects I learned about–the tensions between free software advocacy and development, the role of memory among fan s of a corny country/Gaucho singer from this area and the surgical management of intersex (hermaphrodite) newborns were some of the most interesting projects I have come across and gobbled as much information about them as I could. I was also struck, yet again, by the deep myopias of anthropology, north and south. My own project, which focuses mostly on white, American and European hackers, often does not strike as culturally authentic enough because, well, the people I study are white, American and European (and I am slowly coming to see that if you study the so-called white and male or the North American/European elements of technology and the Internet and the law, you are probably white and male yourself and anything having to do with ethnicity/gender is usually the province of female academics, which I find really problematic). If I had carried out my first project in Brazil, where there is a foreign language, where there is a long tradition of studying various groups, then my project would have been stamped by that mysterious aura of authenticity/approval. On the other hand, two of the students here (those that worked on free software and the country singer) complained that their papers have yet to be accepted by the Brazilian Anthropological Association for being too strange, non-traditional, and it seems in their case, the problem is that they are studying urban Brazilian, popular culture (which for me would have just been just the thing to study). We are located far apart but find ourselves in similar positions by virtue of studying something geographically close to where we live, which is confused and misperceived as being culturally close to our world. But if there is something I wish to hammer into my students and other anthropologists is that there is tremendous plurality and multiplicity within our own societies. We can travel far, in the cultural sense, just by staying home and opening our eyes a little wider. Another nice experience is that I am learning a tremendous amount about free software politics and development from the graduate student, Luis Felipe, who is really responsible for getting me down here. He found me on IRC many moons ago and finally we have been able to compare notes and have long conversations about free software, the differences in how we can gather data due to gender, and a topic close to me, which is the tension between the political and apolitical in free software. Our relationship is at once based on friendship and also one of mentoring. And it reminds me of others who have also mentored me and how crucial this mentoring was to the development of my own work. One of my most important mentors, Chris Kelty, just published his book on Free Software (and I will soon write an entry about the book but the WHOLE THING IS ONLINE) and so it felt quite nice that we were discussing what I think his one of his most amazing chapters on the creation of the copyleft in our class on Friday. Not only was the topic about the genesis of the first free software license appropriate for a class on IP, but in many ways, because of his mentoring, I have gotten to where I am and so it felt also appropriate to honor and recognize that genealogy in my own work.

Next.

Previous.