LWN has an article by Valerie Aurora titled The dark side of open source conferences [1] which is about sexual harassment and sexual assault at Free Software conferences. Apparently some conferences create such a bad environment that some people won t attend, it s a well researched article that everyone in the community should read.The Autism DerailmentThe comments have the usual mix of insight, foolishness, and derailment that you expect from such discussions. One derailment thread that annoyed me is the discussion about men on the Autism Spectrum started by Joe Buck [2]. Joe seems to believe that the 1% of males on the Autism Spectrum (and something greater than 1% but a lot less than 50% in the Free Software community) are a serious part of the problem because they supposedly hit on women who aren t interested in them in spite of the fact that the article in question is about women who are being insulted, harassed, and groped at at open source conferences . The article had no mention of men who try to chat up women presumably this was a deliberate decision to focus on sexual assault and harassment rather than what Joe wanted to talk about.In response Mackenzie made the following insightful point:I don t think any autistic person who is high-functioning enough to A) contribute to open source B) want to be at an event with so many people and C) carry on any sort of conversation is low-functioning enough not to understand stop or no. If you can understand your patch has been rejected, you can likely understand don t do that again. Understanding how Other People FeelBruce Perens claimed What they [Aspies] don t understand is how the other person in the situation feels . Like many (possibly most) people Bruce doesn t seem to get the fact that no-one can really understand how other people feel. The best logical analysis of this seems to be the Changing Emotions article on Less Wrong [3]. While Less Wrong deals with Male to Female conversion as the example (which may be relevant to the discussion about the treatment of women) the same logic also applies to smaller changes. Anyone who even thinks that if they would always be able understand how their identical twin felt (if they had one) probably hasn t considered these issues much. As an aside, having a psychologist diagnose you as being on the Autism Spectrum and therefore by implication thinking differently to 99% of the population really makes you consider the ways in which other people might have different thought processes and experiences.Every time we have a discussion about issues related to sexism in the Free Software community we get a lot of documented evidence that there are many people who are apparently neuro-typical (IE not Autistic) who don t understand how other people think in many cases they go so far as to tell other people what their emotional state should be.What Really HappensNix said However, in that situation our natural reflex is to *get out of there*, not to jump on women like some sort of slobbering caveman which is a really good summary.In more detail, I think that the vast majority of guys who are on the Autism Spectrum and who are able to do things like attend computer conferences (*) realise that chatting up a random girl that they meet is something that just isn t going to work out. Generally people don t attempt things that they expect to fail so I don t think that Autistic guys are going to be hitting on girls at conferences.(*) Having never met any Autistic people who aren t capable of attending such conferences I can t speak for them. I really doubt that the Low Functioning Autistic guys are as much of a problem as some people claim, but lack evidence. In any case the actions of people who don t attend conferences aren t relevant to a discussion about things that happen at conferences.Update: It Keeps GoingDion claims that the misogyny at conferences is due to socially inept people, he also casually switches between discussing people who misunderstand when someone is flirting and people who hire almost-naked booth-babes (two very different classes of action) [4]. Several people asked for supporting evidence, naturally none was provided.In response njs posted a link to Marissa Lingen s blog post Don t blame autism, dammit [5]. Marissa points out that people who offend other people due to lacking social skills will tend to do so in times and places that are likely to get a bad reaction if you don t know that you are doing something wrong then there s no reason to hide it. If someone offends a senior manager at a corporate event then it could be because they are on the Autism Spectrum (I ve apparently done that). If someone offends junior people at a times and places where there are no witnesses but is always nice to managers and other powerful people then it s not related to Autism.One final note, I have little tolerance for anyone who claims to be an Aspie when they do something wrong. You are either on the Autism Spectrum all the time or none of it. Anyone who wants any sympathy for me for an occasion where they stuffed up due to being an Aspie can start by making a clear statement about where they are on the Autism Spectrum.Update2: Yet More from Bruce PerensBruce wrote IMO, the kind of men who go in to software engineering suffer a lack of healthy interaction with women who are their peers, and it may be that the high incidence of empathy disorders in our field is involved (which seems to be part of the inspiration for Joe Buck later in that thread) and now claims Nobody here was trying to connect Asperger s or autism with the touching incidents or violent crime .Matthew Garrett responded to that with If you weren t trying to say that the high incidence of empathy disorders in our field was related to a lack of healthy interaction with women who are their peers, and that that has something to do with incidents of sexual harassment or assault at conferences, what were you trying to say? Because that sounds awfully like We wouldn t have so many problems if it weren t for all the autists .Bruce s latest comment is If you choose to read something that nasty into my writing, that s your problem. Get therapy .Through this discussion I ve been unsure of whether to interpret the statements by Bruce et al the way Matthew does or whether I should consider them as merely a desperate attempt to derail the discussion. I can t imagine any possible way of interpreting such comments in connection with the discussion of sexual assault as anything other than either trivialising violent crimes against women (suggesting that they are no worse than asking out someone who s not interested) or claiming that anyone who lacks social skills should be treated as a violent sexual predator. It s just not reasonable to believe that every single person who wrote such comments referring to Autism was misunderstood and really meant something nice.As a general rule I don t think that it s the responsibility of other people to try and find a non-offensive interpretation of something that one might say. I don t think that all the people who strongly disagree with the most obvious and reasonable interpretations of Bruce s comments should get therapy. I think that Bruce should explain what he means clearly.

• [1] http://lwn.net/Articles/417952/
• [2] http://lwn.net/Articles/418209/
• [3] http://lesswrong.com/lw/xe/changing_emotions/
• [4] http://lwn.net/Articles/418281/
• [5] http://mrissa.livejournal.com/746160.html

I have this project in my head, I want to work on it but I always lack the time. In order to go forward, I thought I could write about it, at least it would let me clarify my ideas and the core goals. So here I am, I will present you Librement (I have registered the alioth project but it s empty). The core goal is to make it easy for every user to contribute to free software in some way. I will now present the main features that I envision. Defining skills and interests In order to propose tasks that the user can do, we must have an idea of his skills. So on the first run (and later through a preferences menu) the user will be invited to define his skills:

• his native languages (multiple allowed)
• other languages he can understand
• programming languages he knows
• version control systems he can use
• markup language he knows (HTML, DocBook, Wiki-like formats, etc.)
• etc.

Maybe we can also ask which skills he would like to learn. Because contributing to free software is a nice opportunity to learn new skills! We should also find out what the user is interested in. What are his favorite free software projects? What kind of contributions would he like to do (documentation, translation, coding, bug fixing, bug triaging, creating artwork, donations, etc.)? Choose activities and pick concrete tasks Based on the user s skills and his interests, the software shows a list of possible activities. The user can then sort that list, from the most interesting one to those that he doesn t want to do. Each activity can generate concrete tasks. For example, the activity Do translation for Debian could generate a task Translate strings in debconf/fr.po or Review translations in partman/fr.po . Work on tasks When the user decides to work on a task, a step-by-step assistant helps him/her. It can automate some steps and provide explanations for the remaining ones, for example in the case of a translation for Debian:

• grab the PO file (from a VCS, from an HTTP URL, from a translation server, etc.);
• select and install a software to work with PO file (if not already done);
• edit the PO file with the preferred program;
• check the PO file (is it complete? is there no mistakes like missing substitutions?);
• send back the completed PO file in a mail to the Debian bugtracking system.

If the tasks is not completed in one go, the user can resume it the next time. Each free software project must provide some meta-information describing the various workflows involved for contributing to the different parts of the project. If necessary the project can also provide new plugins to support new operations that are not available in the default library. Setting goals In order to keep the user motivated, the software could track how much time he spent contributing to free software and it could verify if the user reached the goals he picked up for himself. Maybe it can also hook into the OMG Trophy Awarding System. The sky is the limit I hope that you now have a clearer idea of what this desktop application is supposed to be. There are literally hundreds of ways to contribute to free software and I like the idea that we can streamline the process for most users. All the plugins implementing activities can use local information (list of packages installed with their versions, configuration settings, etc.) to propose tasks targetted to the user and highly beneficial for the corresponding free software projects. For example, a bug tagged unreproducible might benefit from a few more users trying to reproduce it. The software could direct the user to this bug report if it detects that he/she runs the same version on the same architecture and that this software is regularly run on the system. Many projects have created operations or events to encourage people to contribute, they could all be implemented as dedicated activities in Librement. I m thinking of stuff like Gnome Love, Ubuntu s 5-a-day, Ubuntu s 100 papercuts, etc. Even for people who have no time to contribute, the application can still be useful by referencing the various ways to donate money (or material) to projects that they are using. Feedback welcome I m excited by the potential of such an application, but it s normal since it s my idea. Do you believe it can be useful and popular? Do you have ideas of exciting activities that such a framework can offer? PS: If you wonder how I came up with the name Librement , here s the explanation. It s a French word which means freely . And users who want to give back are trying to live up to the principles of free software, which I sum up by they are trying to live freely .

Follow me on Identi.ca, Twitter and Facebook. Or subscribe to this blog by RSS or by email.

11 comments Liked this article? Click here. My blog is Flattr-enabled.

As I previously blogged about, I wrote an Annual Review of Anthropology on digital media last year. About a month ago, I found out that anyone can download it thanks to a link provided by the ARA, which we are allowed to put on one institutional web page. So go here (and go to the citation for the link) for those who are interested in a way too short review of some of the ethnographic literature on digital media. Writing the piece left me many psychological wounds and scars, one of which had to do with the fact that I probably overlooked some folks. I have been left out of review type essays and honestly, it sucks. I tried to be as comprehensive as possible: I chose not to massively whittle down the scope (which was an option) and was able to smuggle in more citations than originally allowed and yet I still cut out 200 citations. But in the end I overlooked some folks as I found out about them too late. If I could go back in time, this is who I would include (well there are others but I have chosen these for now). So Shaka McGlotten: not only does he have a cool name, he studies some cool stuff like DIY online porn, race, and zombies. He has published a bunch of articles and a book is forthcoming. Check out his work here. I missed this book Online a Lot of the Time: Ritual, Fetish, and Sign by Ken Hills which looks quite pertinent and a great read. Jonathan Marshall is an anthropologist in Sydney who has been working on digital stuff for a long while now and recently published Living Cybermind, which covers in detail modes of interacting and communicating in a detailed examination of a mailing list by the name of Cybermind. Although this book is not out yet, it will be soon and looks fascinating: Digital Jesus (great cover). From what I understand, and one of the reasons I want to dive into the book , is because it is so longitudinal, at least when measured in Internet years. Rob Howard has studied Christians online since the days of Usenet to the present and thus this ethnography promises to have some real meat to it. I am eagerly anticipating the forthcoming book by Beth Coleman . I had a chance to read a chapter recently and it looks fantastic. While in some respects grounded in virtual worlds, it is far more expansive than that topic, addressing a range of issues from desire, experience, emotion and race. Can t wait. I cite four dissertations in the ARA mostly because I read them, thought they were great, and material based on the dissertation is en route to being published in some form in the next future. The one dissertation I wish I had read is Jenny Cool s thesis on cyberorganic.

i suspect a lot of people are used to forwarding TCP sockets with SSH -- for example, to connect locally to a mysql daemon that runs only on the loopback interface of a remote machine (this is debian's default mysql-server configuration):

ssh -N -T -oExitOnForwardFailure=yes -L 3306:localhost:3306 remoteuser@mysqlserver.example

But sometimes, the remote service runs on a UNIX-domain socket, not on a TCP socket -- for example, debian's default configuration for postgresql is to have it listen only on a UNIX domain socket in /var/run/postgresql, and use SO_PEERCRED with a simple system account == psql account mapping scheme to authenticate users without needing any extra credentials. This is not quite as simple to forward over ssh, but it's doable as long as socat is installed on both your local host and on the remote postgres server. Here's one way to do it if $SOCKET_DIR points to the full path of a directory under the user's control (this is all one command, split across lines for easier reading):

socat "UNIX-LISTEN:$SOCKET_DIR/.s.PGSQL.5432,reuseaddr,fork" \
   EXEC:'ssh remoteuser@psqlserver.example socat STDIO UNIX-CONNECT\:/var/run/postgresql/.s.PGSQL.5432'

Then, you'd connect with something like:

psql "user=remoteuser host=$SOCKET_DIR"

Each such psql connection will trigger an ssh connection to be made. Of course, this won't work well if ssh has to prompt for passwords, but you should be using ssh-agent anyway, right? There are at least a couple nice features of being able to use postgresql from a local client like this:

• your psql client can load files from your local machine, and can dump/export files to the local machine.
• your ~/.psql_history stays local, so you can review what you did even when you're offline
• you can run local RDBMS administrative GUIs like pgadmin3 with minimal network traffic and no extra packages installed on the server.
• unlike forwarding TCP ports (where any other user account on the machine can hop onto your connection), you can control access to your local UNIX-domain socket with standard filesystem permissions on $SOCKET_DIR.

Of course, postgresql itself already comes with a nice range of high-quality network-capable authentication mechanisms you could use. But many of them (like GSSAPI or X.509 mutual key-based authentication over TLS) require additional infrastructure setup; and you probably already have sshd up and running on that machine -- so why not make use of it?Tags: postgresql, socat, ssh, tip, unix-domain socket

(This is a part of a series of posts on my recent experience with Windows. See Windows (I) for the first post.)

For the past month, I got disconnected from basically all of my usual activities. Not only from Debian work (well, yes, I followed up on a couple of important bugs and releases, but I was in a very low power consumption mode of sorts), DebConf organization/following (some people have contacted me regarding the DebConf 12 venue decision, for which we should start working and have a presentation soon) Even the always-so-important Real Life got pushed aside. I barely participated in this year's EDUSOL (On-line Encounter of Education and Free Software). Even work suffered I asked my boss to hold any incoming requests for me (save, of course, for urgent stuff). I am now shyly coming up from the hole I dug myself in for this time. Why did I do such a thing? I do not have a formal education beyond high school. Due to several personal issues, I was accepted to the best university in my country, and to the first generation of the study program I would definitively love to have followed, but didn't attend. Of course, I had a fair bit of computing background, enough to be competitive and start working. After amazingly few years, I didn't only get a decent job, but I got a job precisely at the very university I wanted to study at (and where my father has worked for the past 40 years). And, even with some changes along the years, here I am with 10 years of laboral history in this university, with an academic position (although the smallest academic position, but still). It is one of my life's achievements, one of the things that makes me proudest of. Still, I don't want to stay at this level for the rest of my life. And an institution such as this one has rules Yes, flexible enough to allow me in, but rules whose importance I cannot deny. I aspire to become a professor/researcher at some point, and that is just not possible without filling in the prerequisites. Besides, I want to try my hand at teaching at a universitary level. So, what choices did I have besides devoting at least four years to full-time study? Well, there is a government entity that evaluates and regulates formal education given by smaller universities CENEVAL. For probably around ten years now, they have offered the ACREL (Acuerdo 286 Licenciatura) program to validate professional knowledge acquired through work experience And I'm writing this detail as several people have asked me for details on it. So, once you are accepted in the program (basic requisites: Having a high school certificate, over 30 year old, and with over 10 years of provable work experience), they can grant a Licenciatura degree (i.e. the equivalent to a four-year study program). So, in order to get it granted, I have to:

• Present a written exam, covering all of the areas of study. In my case (it varies depending on the program you are presenting), it was a 12 hour exam (over three sessions IIRC I finished each session in around three hours of the four allowed). Questions are all multiple choice. For the Software Engineering degree, you are allowed to enter each session with up to five reference books. Rules vary depending on the area. The exam was long, although not too hard. And I'm proud to say that I reached outstanding level on each of the areas of the exam
  If this information intersts you, do take a look at the links under the Study guides bullet at CENEVAL's information page. The areas to study, the style of the questions, and the general information on the exams are very close to what you will actually receive.
• The second phase (oral exam) is split in two sub-phases. The first one, the one I have just finished, is the longest one: I was assigned a practical case to develop. In my case, I was requested to write a document detailing the methodology for writing a request-tracking system for a restaurant. I was given a month for this task And in part, the reason that kept me so busy is that I subscribe to the idea that the best methodology to write an application is to write it, not just to document what you are about to do (and then fail to recognize you missed so many details). Then, there is yet another period (between one and three months, assigned randomly AFAICT) during which I would have to develop the application and a presentation, and defend it against a jury.
• Some study areas (mainly those related to health) have a third step: A practical exam. Of course, I cannot go into details on this one.

Anyway... Enough of a blog posting. I hope this is useful for somebody. And I hope it explains why I have disappeared so much lately. If everything goes as planned, I expect to be receiving my "licenciatura" title around March, and afterwards... Well, we shall see! Now... Back to work. And back to life. And back to my huge backlog of other pending things :)

We ve come a long way since the times when you needed to configure 2 X servers in XDM just to be able to use 2 X sessions at once. However there was still some way to go until recently. A number of bugs that could be wrongly attributed to bugs in the X server or in the desktop environment were actually caused by the display manager doing crap. GDM up to 2.20 Since the introduction of the flexible X servers feature, GDM hadn t evolved much on the matter of user switching. What it used to do was pretty straightforward:

• a specific protocol can be invoked by the gdmflexiserver command;
• the gdm daemon spawns a new X server on an empty console;
• it initiates another login process in it;
• when the session exits, or if the user clicks on Quit instead of logging in, the X server exits.

It is interesting to note that VT (console) switching is purely handled by the X server. When starting, the new server switches the current VT to where it is. When exiting, it automatically switches back to the VT from which it was launched. While very simple, this idea fails to work correctly every time you try to do something more complicated than starting a temporary session for a guest and exiting it. For example, if you start two of them, there is a chance that, when the X server switches back to the console it was run from, there is nothing left running in this console, leaving you with the funny Control-Alt-Fn shortcuts to find your way back to a X server. You will also meet interesting race conditions when trying to switch back to an existing session from the login window. GDM 2.28 and above In the process of rewriting the code entirely, the GDM developers tried to address a number of those shortcomings, making use of D-Bus and ConsoleKit. The new design is slightly more complicated, however.

• The gdmflexiserver tool will first try to look for an existing login window in another X server, and just switch to the VT it is in if it finds one.
• Otherwise, the daemon starts a new slave process with a new X server and a new login window, in a very similar way to what older versions did.
• When logging in as a user with an existing session, it switches to the VT it is in, but leaves the login window and its X server running.
• When going into a new session, the X server is simply left to die at the end of the session, and to switch back to the VT from which it was launched.

Not killing the X server in some cases partly addresses the problems caused by letting it switch back to the original VT when exiting. However in several ways the cure is worse than the disease.

• First of all, it will leave unused X servers, with all processes used by the login window - and that makes quite a number of them, with GDM now using a minimal GNOME session.
• When there is such a login window remaining, ConsoleKit will refuse to let you shut down your computer, being lured into thinking there is someone else using it.
• It doesn t solve the inconsistency issue. When you leave a session, you can find either of: a login window, a screensaver unlock dialog, or a black screen.

Getting it to work The modular architecture of GDM makes it possible to improve the situation. (Possible but not easy because of the millefeuille of classes.) However, it is merely a band-aid unless you fix the root issue: the X server knowing better than you which VT it should switch to when exiting. Fortunately Xorg now features an option to avoid that behavior: -novtswitch. So the first step is to enable it, and let the GDM daemon (or slave) handle VT switching through ConsoleKit. With that, the following changes are possible.

• When switching to an existing session, don t leave a X server behind. You can now kill it safely without risking a VT switch.
• On the opposite, when exiting a session, always respawn a login window on the same VT.
• The last step is to stop making a difference between the first launched X server (called a static server) and the flexible servers. The only remaining difference between a static display and a flexible one is that the static one honors automatic login/timed login settings.

The result With all these changes the behavior of the display manager is finally completely consistent.

• When exiting a session, regardless of it, you always find the same login window.
• There is never an unused process left.
• You will never find yourself facing a black screen with only keyboard shortcuts to leave you out of it.

Interestingly enough, this is very similar to what user switching looks like on Vista or MacOS X. So what now? These changes are stabilized for Debian squeeze, but of course it has been long overdue to get them accepted upstream, along with the very large number of Debian-specific changes that still lie in our packages.

Earlier in the year, Romain and I did a bunch of initial work on a wrapper from R to the GNU GSL by way of our Rcpp package for seamless R and C++ integration. But other work kept us busy and this fell a little to the side. We have now found some time to finish this work for a first release, together with a nicely detailed eleven page package vignette. As of today, the package is now a CRAN package, and Romain already posted a nice announcement on his blog and on the rcpp-devel list. So what does RcppGSL do? I gave the package its own webpage here as well and listed these points as key features of RcppGSL:

• templated vector and matrix classes: these are similar to Rcpp's own vector and matrix classes, but really are just smart pointers around the C structure expected by the library
• this means you can transfer data from R to your GSL-using programs in pretty much the same way you would in other C++ programs using Rcpp---by relying on the Rcpp::as() and Rcpp::wrap() converterrs
• at the C++ level you can use these GSL vectors in a more C++-alike way (using eg foo[i] to access an element at index i)
• yet at the same time you can just pass these vector and matrix objects to the GSL functions expecting its C objects: thanks to some cleverness in these classes they pass the right object on (see the example below)
• we also provide the lightweight views for vectors and matrices as the GSL API uses these in many places.

Also provided is a simple example which is a simple implementation of a column norm (which we could easily compute directly in R, but we are simply re-using an example from Section 8.4.14 of the GSL manual):

#include <RcppGSL.h>
#include <gsl/gsl_matrix.h>
#include <gsl/gsl_blas.h>
extern "C" SEXP colNorm(SEXP sM)  
  try  
        RcppGSL::matrix<double> M = sM;     // create gsl data structures from SEXP
        int k = M.ncol();
        Rcpp::NumericVector n(k);           // to store results
        for (int j = 0; j < k; j++)  
            RcppGSL::vector_view<double> colview = gsl_matrix_column (M, j);
            n[j] = gsl_blas_dnrm2(colview);
         
        M.free() ;
        return n;                           // return vector
    catch( std::exception &ex )  
        forward_exception_to_r( ex );
    catch(...)  
        ::Rf_error( "c++ exception (unknown reason)" );
   
  return R_NilValue; // -Wall
 

This example function is implemented in an example package contained in the RcppGSL package itself -- so that users have a complete stanza to use in their packages. This will then build a user package on Linux, OS X and Windows provided the GSL is installed (and on Windows you have to do all the extra steps of defining an environment variable pointing to and of course install Rtools to build in the first place---Linux and OS X are so much easier for development). Another complete example is in the package itself and provides a faster (compiled) alternative to the standard lm() function in R; this example is the continuation of the same example I had in several versions of my Intro to HPC with R tutorials and in the Rcpp package itself as an early example. We will try to touch base with CRAN package authors using both GSL and Rcpp to see how this can help them. The API in our package may well be incomplete, but we are always happy to try to respond to requests for additional features brought to our attention, preferably via the rcpp-devel list. More information is on the RcppGSL page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

The new RQuantLib release 0.3.5 is now on CRAN and in Debian. RQuantLib combines (some of) the quantitative analytics of QuantLib with the R statistical computing environment and language. Most of the changes were made two and four weeks ago: first in response to some warnings triggered by R 2.12.0 on the included manual pages which needed a brush-up, and then again is some consolidation of manual pages and some other minor tweaks. The release was then held back at CRAN as we noticed that manual pages, when collated to a single large document, triggered a segmentation fault in the latex compiler. Oddly enough only in Europe (if the a4paper option was used) and not here (where I use uspaper). Long story short, this turns out to be a bug in the latex toolchain (which we reported as Debian bug report 604754) which is apparently is known but has no known fix yet (a sample file was supplied with the bug report if you want to take a look). With that, special thanks go to Kurt Hornik and Brian Ripley on the R Core team who made a change to how R processes the manual which made it resilient to the latex bug so that normal release of the package could proceed (and the shiny manual is available too). Thanks to CRANberries, there is also a diff to the previous release 0.3.4. Full changelog details, examples and more details about this package are at my RQuantLib page.

Plack is one of these wonderful adventures in the modern Perl world that makes it fun to write Web applications in Perl again. But I have a few applications written with Apache/mod_perl and they are not fun to work with. So what would you do? One option would be to take the long road and port these apps to use Plack instead of messing around with Apache2::RequestRec. For this to work you might need to review the full code base before seeing any signs of progress. Another option is to mock your Apache2::RequestRec object using Plack. This is the road explored by Plack::App::WrapApacheReq. With very little work this enables you to run your mod_perl application with any Plack handler you want. You can run your application as a stand alone server or serve it with nginx trough FastCGI. But the fun doesn t stop here. Debugging and profiling mod_perl have always been a PITA, but with Plack::App::WrapApacheReq it is easy. Just take the generic.psgi example and it enables you to run the Perl debugger easy or just to use NYTProf on your request handling code. The initial ideas for writing Plack::App::WrapApacheRec came from my mocking code to test a legacy mod_perl application. Even this gets more fun by using Plack. I havn t tested it yet, but with Plack::Test and Plack::Test::ExternalServer it should be trivial to run the same set of tests directly in a single process test suite or against your deployed server. Plack::App::WrapApacheRec is still in it s infancy. It only mocks as much of Apache2::RequestRec as I need to run a single legacy application and to run Plack::Handler::Apache2 (as an absurd example, but yes we are self hosting). But I think that with very little work we should be able to run most mod_perl applications. Take it out for a ride, if it complains about a missing method please report it with CPAN s RequestTracker. Patches would be appreciated (or pull requests on github), but just a list of unimplemented method you need would be excellent. Plack is fun, now working with mod_perl applications might become fun too.

As part of the Training Sessions initiative organized by the Debian Women project, this week a lesson about Python packaging will be held. The lesson, which is aimed at an intermediate audience, will focus specifically modules and applications packaging and will be held by Piotr O arowski who is member of Python Applications Packaging Team and Debian Python Modules Team.

The session will take place, as usual, on #debian-women channel on irc.debian.org on Thursday, 2nd December at 20:00 UTC.

Cause of the advanced topic is madatory to have some knowledge about Debian packaging (see, for example Lars Wirzenius tutorial) and about general Python development.

The lesson will consists of a first, theoretical, part in wich Piotr will explain us some basics; and of a Q&A part, entirely dedicated to more advanced problems.

See you on Thursday!

Contributed by: Francesca Ciceri

At the end of September, things were quite exciting as we had our first project meeting. At that meeting those in the room saw a demonstration of the Moonshot GSS EAP mechanism and we discussed a number of open issues and began to plan for our test infrastructure. We ve made significant progress on the specification front and on explaining Moonshot to important communities since then. However there has been little public progress on the implementation front. Unfortunately, getting the necessary legal clearance and agreements to release code often takes longer than anyone would like; that is what is happening here. We re all eagerly awaiting final approval from the lawyers and JANET(UK) management. However, things have been moving behind the scenes. Throughout much of October, Luke Howard and Linus Nordberg were working on their respective parts of the code. I ve also been working on putting together the test and build infrastructure. As we discussed at the meeting, we re going to use Debian and Ubuntu as the basis for our testing. For example, we hope to release virtual machine images for these platforms for the major Moonshot components. Thus the primary build environment for our testing and virtualization will be for Debian. I ve been putting together that here. Right now, that branch will pull together packages of the SAML infrastructure that we need. I ve also been looking into virtualized test frameworks and believe I ve found one that meets our needs. I ve also put together some primitive build infrastructure that is independent of packaging available here. I ve set up a buildbot that builds both environments. So, as the code becomes available we ll be in a good position to start making it available.

Once again, there s been another story about how Google is having trouble retaining talent. Despite all Eric Schmidt s attempts to tell folks that Google s regretted attrition rate has not changed in seven years, this story just doesn t want to seem to die. (And those stories about Google paying $3.5 million and $7 million to keep an engineer from defecting to Facebook? As far as I know, total bull. I bet it s something made up by some Facebook recruiter who needed to explain how she let a live prospect get away. :-) At least for me, the complete opposite is true. There are very few companies where I can do the work that I want to do, and Google is one of them. A startup is totally the wrong place for me. Why? Because if you talk to any venture capitalist, a startup has one and only one reason to exist: to prove that it has a scalable, viable business model. Take diapers.com for example. As Business Week described, while they were proving that they had a business model that worked, they purchased their diapers at the local BJ s and shipped them via Fedex. Another startup, Chegg, proved its business model by using Amazon.com to drop ship text books to their first customers. (The venture capitalist Mark Maples talked about this in a brilliant talk at the Founders Showcase; the Chegg example starts around 20:50 minutes in, but I d recommend listening to the whole thing, since it s such a great talk.) You don t negotiate volume discounts with textbook publishers, or build huge warehouses to hold all of the diapers that you re going to buy until you prove that you have a business model that works. Similarly, you don t work on great technology at a startup. Startups, by and large, aren t about technology at least, not the Web 2.0 startups like Facebook, Foursquare, Twitter, Groupon, etc. They are about business model discovery. So if you are fundamentally a technologist at heart, whose heart sings when you re making a better file system, or fixing a kernel bug, you re not going to be happy at a startup. At least, not if the startup is run competently. If you have the heart of an entrepreneur, and you are willing to roll the dice (since 9 out of 10 startups go belly up; those are the ones that failed to find a viable business model), then sure, go for a startup. And understand that your job will be to make something that works well at a small scale, quick, dirty, and cheap. If that means using some proprietary software, then that s what you should do. Hopefully you ll get lucky and win the IPO lottery. But if your primary interest is to doing great engineering work, then you want go to company that has a proven business model. Google has enough scale that I can sleep well knowing that improvements I am making to Linux and the ext4 file system are saving enough money for the company that Google is making a profit on me. That is, the value Google is getting out of my work is worth many multiples of my total compensation. And that s a position that every engineer should strive for, since that s how you can be confident that you will remain gainfully employed. The fact that I can do what I love, and go to conferences, and all of my work is open source that s just total icing on the cake. :-) No related posts.

Paul Wayper has written a blog post criticising the main chains of grocery stores in Australia [1]. That is the cartel of Coles and Woolworths (which was formerly known as Safeway in some parts of Australia), and the German newcomer Aldi.While he does make some good points I think he was a little harsh on Aldi. As a portion of the stock on shelves (by quantity stocked, shelf space, number of products, or almost any other metric) it seems that Aldi easily beats every other decent sized grocery chain for the portion of the stock being organic food. Aldi also has a very high portion of their stock being fair-trade also probably beating the other chains handily. Having organic fair-trade food sold in bulk at low prices is a really good thing!Paul criticises the lack of range on offer in the major chains. Aldi has the smallest range as their entire business model is based around offering a small range of products in a relatively small store at very low prices. That dramatically decreases the amount of stock that they have and therefore decreases the amount that is spoiled and the expense of storing and tracking it all. I wonder if the success of Aldi can drive other stores to getting a better range of products. Why buy a small range of products from Coles or Woolworths when you can get a similar small range from Aldi at low prices that the business models of Coles and Woolworths can t match? This may even drive consumers to buying core products (bread, milk, pasta, etc) and some basic luxury products (cooking wine, basic chocolate, cheap soda, etc) from Aldi and then buying other products from specialist stores. This might result in more delicatessens!Aldi also has a weekly selection of special items. For example selling snow boots etc just before winter starts. In most parts of Australia we don t see snow much and a cheap pair of snow boots is a good investment for the one day a year that most people spend at the snow. These specials provide real benefits to consumers although not to the Australian economy.Paul complains about own brand products. But for some products such as sugar you will get the same product from home brand as from other brands (there aren t really many options for sugar) and it ll probably end up being Australian just due to economics.When you buy big-name products you are mostly paying for marketing. For some things there are real taste differences (such as Coke for me), but for lots of things there are no difference, so I m happy to buy own brand . Generally not paying for marketing is a good thing.I totally agree with Paul s point about buying Australian to help the local economy and also to avoid the transmission of diseases. Any treaty which forces us to accept the import of products from countries that have diseases that can be transmitted to Australian crops should be voided. It seems that the WTO is specifically designed to benefit corporations at the expense of member countries and their citizens.Paul gave a positive review to IGA, I wasn t impressed with my local IGA the last time I visited it. But I ll give it another go.

• [1] http://www.mabula.net/tbfw/blosxom.cgi/2010/11/29#2010-11-29-scarcity-in-abundance

The Debian archive is known to be one of the largest software collections available in the free software world. With more than 16,000 source packages and 30,000 binary packages, users sometimes have trouble finding packages that are relevant to them. Debian developer Enrico Zini has been working on infrastructure to solve this problem. During the recent mini-debconf Paris, Enrico gave a talk presenting what he has been working on in the last few years, which hasn t gotten yet the attention it deserves . Enrico is known in the Debian community for the introduction of debtags, a system used to classify all packages using facets. Each facet describes a specific kind of property: type of user-interface, programming language it s written in, type of document manipulated, purpose of the software, etc. His most recent work builds on that. It is available in Debian and Ubuntu in the apt-xapian-index package. Its purpose is to allow advanced queries over the database of available packages. Users of apt-xapian-index He started by presenting some early users of the infrastructure. The most widely know is Ubuntu s software center. Its search feature provides results almost instantly thanks to apt-xapian-index. But it is a very simple interface that doesn t exploit many of the advanced features provided by the apt-xapian-index. Another early adopter, making use of some more advanced features, is GoPlay!. It s a graphical user interface to find games. It makes use of debtags to classify games so that you can browse, for example, all 3D action/arcade games related to cars. GoPlay has even been extended to be a more generic debtags based package browser and the package now also provides GoLearn!, GoAdmin!, GoNet!, GoOffice!, GoSafe!, and GoWeb!. Fuss-launcher is an application launcher and not a package browser, but by using apt-xapian-index, it s able to reuse information provided at the package level to make it easier to find installed applications. Package descriptions tend to be more verbose than those embedded in .desktop files. Enrico also showed another nice feature to the audience: if you drag a document onto its window, it will show you a list of applications that can open it. Last but not least, apt-xapian-index provides a command line search tool that is vastly superior to the traditional apt-cache search: it s axi-cache search (axi stands for apt-xapian-index). Enrico compared the output of a search on the letter r . While apt-cache spits out an infinite list of packages containing this letter somewhere in the description, axi-cache only listed packages related to GNU R. He also demonstrated the contextual tab completion. It makes it easy to use debtags and to refine your search. Once you have typed a first keyword, the tab-completion for the second one only contains keywords or debtags that are actually able to provide more restrictive results. Advanced queries with logical operations (AND, OR, NOT, XOR) are also supported. Features of the backend Enrico then dived into the internals. Xapian s search engine is at the root of this infrastructure. He likes it because it s a simple library (i.e. no daemon) and it has nice Python bindings. While apt-xapian-index s core work is to index the descriptions of all the packages, it actually stores much more and can be easily extended with plugins (written in Python). For instance, the information stored encompasses:

• words appearing in the description of the packages (including the translated descriptions if the user uses a non-English locale);
• their origin;
• their section;
• their size and installed size;
• the time they have been first seen;
• icons, categories, descriptions from the .desktop files they contain (through app-install-data);
• aliases for names of some popular applications that are not available on Linux (for instance excel maps to the debtag office::spreadsheet).

He already has plans to store more: adding popularity contest data (see wishlist bugs #602180 and #602182) will make it possible to sort query results in a useful way. The most widely used applications are good choices when it comes to community support, and they are likely of better quality due to the larger user base. Adding timestamps of the last installation/upgrade/removal, will make it easier to pin-point a regression to a specific package update. The generated index is world-readable and can be used from any application provided it can use the Xapian library which is written in C++ but has bindings for Perl, Python, PHP, Java, Tcl, C#, and Ruby. Call for experimentation Enrico believes that many useful applications have yet to be invented on top of apt-xapian-index s features. He s calling for experimentation and asking for new ideas. The only practical limit that he has encountered is the size of the index: currently it varies between 50 Mb (Debian unstable without translation) and 70 Mb (Debian stable/testing/unstable with one translation). He would like it to not grow over 100 Mb since it s installed by default (due to aptitude recommending it) and he s not comfortable with the idea of using more than 20% of the disk footprint of a basic install just for this service. That s why the index was configured to not store the position of the terms: it s thus not possible to find out packages whose description contains the word statistical immediately followed by the word computing . You can however find those which have both terms somewhere in their description. Enrico wondered if apt-xapian-index offers too much freedom. That could explain why few people experimented with it despite his numerous blog posts with code samples and information on how to get started using it. But it s not difficult to imagine use cases for this data. It could be used to extend tools like rc-alert or wnpp-alert, for example. They provide a long list of Debian packages that are looking for some help and are installed on the machine. With apt-xapian-index, it would be possible to restrict the results to the set of packages written in a specific programming language or for a particular desktop environment. The more likely explanation is that too few people know about the tool. There are many more itches to scratch where apt-xapian-index s features could be very useful, and my guess is that Enrico s wishes will eventually come true.

This article was first published in Linux Weekly News. Click here to subscribe to my newsletter and keep up with the Debian/Ubuntu news thanks to my monthly digest.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Yesterday in Harald K nig s commandline talk at LinuxDay.at, I first heard of cat s -e option, and noticed that cat has quite some interesting options. From the man page of cat:

-A, --show-all

equivalent to -vET

-b, --number-nonblank

number nonempty output lines

-e

equivalent to -vE

-E, --show-ends

display $ at end of each line

-n, --number

number all output lines

-s, --squeeze-blank

suppress repeated empty output lines

-t

equivalent to -vT

-T, --show-tabs

display TAB characters as ^I

-v, --show-nonprinting

use ^ and M- notation, except for LFD and TAB

Some examples made on an UTF-8 terminal:

user@host:demo $ cat bla
            
abc
user@host:demo $ cat -Ab bla
     1  M-CM-$  M-CM-6  M-CM-<^IM-CM-^_   $
$
$
     2  abc$
user@host:demo $ cat -Ens bla
     1              $
     2	$
     3  abc$
user@host:demo $ 

I hope this blog posting will reduce the percentage of useless uses of cat by raising the amount of useful uses of cat . :-)

It is Sunday, and I was bored, and wanted to experiment a bit with a possible future direction for piuparts. To do that, I needed to create a virtual machine image with Debian installed in it. I failed. I failed, because I am not familiar with all the stuff I need to be to pull this off. More shame to me. There's a lot of possible ways to do this, and I tried a few, but ran out of time before I could exhaust all opportunities. Somewhere out there is probably the magic rune I am looking for, and if so, please point it out to me. I might want to get back to this problem say day, and having a working rune would simplify things a lot. Here are things I tried:

• Create a base system with debootstrap, then convert that into a bootable image: create image with qemu-img, partition it with parted (so I can have a swap partition), create devices for the partitions with kpartx, create filesystem and copy chroot into the partition. Reason for failure: could not find a way to install kernel or bootloader into the image.
• Live-helper, with various options. It wants to create a live CD, but I was told if I can build an image without the live-boot and live-config packages installed, it would be what I want. Reason for failure: --initramfs none either causes the image build to fail or the resulting image to not have a bootloader at all. I assume that is the right option, but don't know for sure. (#605255)
• qemubuilder might do what I want, but failed to build anything at all, even after I worked around a bug by specifying architecture explicitly. (#440944, open since 2007)

I did not try running debian-installer with a pre-seed file under kvm. I might try that some day, but today I did not get there yet. Ideally, I would like to build the image without running things inside a VM, just so it is as independent of specific VM technologies as possible. Quite a lot of people want to create virtual machine images easily. It would be really nifty to have a tool like this:

vmdebootstrap squeeze foo.img http://cdn.debian.net/debian 

It would be a fairly low-level tool, just like debootstrap is, but having just one place where all the trickiness to create the initial image happens would simplify the overall complexity of other tools. Making it simple to use would allow other stuff to be easily built on top of the new tool. For example, piuparts could use it to create a VM image in which to run tests, rather than using chroot, which would allow more code paths to be tested. But I'm not ready to write the tool today, alas.

A few months ago I realised it has been 10 years already since Spacelabs ¹ was founded. Today we will celebrate that with a reunion and probably reminisce about past projects, events and experiences. Therefore, I thought it would be nice to accompany this with a post. The past Spacelabs was founded to create something that sits in between research and consumer appliances. Since we are associated to the Electro-Optical Communication group these things often had to do with bandwidth. For example, showing the ease of use of plastic optical fiber or participate in GPRS/UMTS trials. However, due to the loose structure of Spacelabs, we ventured into many other projects of our own such a information system-enabled fridge with a barcode reader, a dynamic DNS system, etc. At some point we got our own Internet connection with an accompanying subnet. It has been fun and rewarding to learn how to setup and run a mini ISP. Spacelabs doesn t have any formal structure: when you are on the mailing list, you are a member. There are no regular meetings, no hierarchy, no obligations. So in some ways, we are a group that is much like a free/open source software community. I think that this approach has provided us the drive to work both on projects during our free time as well as our own ideas. I feel that Spacelabs has contributed so much to my life. It might not be the typical study association, but it was a big supplement to my studies, it was fun and it was completely ours. Not only did it teach me many, many technical things from networks to program languages to software design but it also gave me experiences on other levels, such as very attuned group work and responsibilities for key systems. The future Soon, everyone connected to Spacelabs will have finished their Master s or PhD (in my case) and nobody will be left to continue the project. Although this is a bit sad, I think we can be proud of what we have accomplished, learnt and experienced. I will miss our location high in the Potentiaal/E-hoog building overlooking Eindhoven, and our being together as a group. We are however not gone yet, there is still a lot to plan and to work on if we want to rescue the huge amount of servers and services that we have accumulated over the years if we were to leave

¹ Yes, the website is very much from the past.

Ever wondered how to easily convert e.g. 10 seamiles into kilometres? Use units:

$ units 
2411 units, 71 prefixes, 33 nonlinear units
You have: 10 seamiles
You want: km
        * 18.288
        / 0.054680665
You have: ^C
$

Of course this is interactive. There s also a non-interactive mode:

$ units '10 seamiles' 'km'
        * 18.288
        / 0.054680665
$ 

The line with the asterisk means that 10 seamiles are 18.288 kilometres or 1 kilometre is the 0.054680665th part of 10 seamiles. This quite non-intuitive output is caused by the fact that unit is designed to be used with units only:

$ units seamiles km
        * 1.8288
        / 0.54680665
$

Now this makes more sense: You have to multiply seamiles with 1.8288 to get kilometres and you have to multiply (not divide) kilometres with 0.54something to get seamiles. But this output is still a little bit cumbersome, and annoying if you want to use it in shell scripts. But for luck, units knows some nice options, especially -v ( verbose ) and -t ( terse ):

$ units -v '10 seamiles' km
        10 seamiles = 18.288 km
        10 seamiles = (1 / 0.054680665) km
$ units -t '10 seamiles' km
18.288

Now that s way easier to read and script! You can also script more complex things like

$ units -v '100 attoparsec/microfortnight' m/s
        100 attoparsec/microfortnight = 2.5509901 m/s
        100 attoparsec/microfortnight = (1 / 0.39200466) m/s

Unfortunately not all common units are unambiguous for units:

units -v '100 km/h' m/s
conformability error
        100 km/h = 1.5091905e+38 s / kg m
        m/s = 1 m / s

Well, h seems not to be units unit for hours , so lets tell it explicity that we want km per hour:

units -v '100 km/hour' m/s
        100 km/hour = 27.777778 m/s
        100 km/hour = (1 / 0.036) m/s

Looks more like what I expected. units behaves though a little bit strange when I try to convert litres per 100 kilometres into miles per gallon:

$ units -t '6L/100km' 'mpg'
conformability error
6e-08 m^2
425143.71 / m^2
$

Interestingly changing the verbosity helps already in this case:

$ units -v '6L/100km' 'mpg'
        reciprocal conversion
        1 / (6L/100km) = 39.202431 mpg
        1 / (6L/100km) = (1 / 0.025508622) mpg

Greetings from the Debian booth at LinuxDay.at and thanks to Y_Plentyn, rhalina and bzed for example ideas and the suggestion to write a blog posting about units. :-)

In the latest issue of Linux Journal, the readers choices were presented, and the winner among the multimedia player were VLC. Personally, I like VLC, and it is my player of choice when I first try to play a video file or stream. Only if VLC fail will I drag out gmplayer to see if it can do better. The reason is mostly the failure model and trust. When VLC fail, it normally pop up a error message reporting the problem. When mplayer fail, it normally segfault or just hangs. The latter failure mode drain my trust in the program. But even if VLC is my player of choice, we have choosen to use mplayer in Debian Edu/Skolelinux. The reason is simple. We need a good browser plugin to play web videos seamlessly, and the VLC browser plugin is not very good. For example, it lack in-line control buttons, so there is no way for the user to pause the video. Also, when I last tested the browser plugins available in Debian, the VLC plugin failed on several video pages where mplayer based plugins worked. If the browser plugin for VLC was as good as the gecko-mediaplayer package (which uses mplayer), we would switch. While VLC is a good player, its user interface is slightly annoying. The most annoying feature is its inconsistent use of keyboard shortcuts. When the player is in full screen mode, its shortcuts are different from when it is playing the video in a window. For example, space only work as pause when in full screen mode. I wish it had consisten shortcuts and that space also would work when in window mode. Another nice shortcut in gmplayer is [enter] to restart the current video. It is very nice when playing short videos from the web and want to restart it when new people arrive to have a look at what is going on.