Search Results: "spa"

A few days ago I watched a Q/A session with Linus Torvalds at Debconf 14. One of the main complaint of Linus towards Linux distribution was the way that distribution ends up using different versions of libraries than what has been used during application development. And the fact that it s next to impossible to support properly all Linux distributions at the same time due to this kind of differences.

And now I just discovered a new proposal of the systemd team that basically tries to address this: Revisiting how we put together Linux Systems. They suggest to make extensive use of btrfs subvolumes to host multiple variants of the /usr tree (that is supposed to contain all the invariant system code/data) that you could combine with multiple runtime/framework subvolumes thanks to filesytem namespaces and make available to individual applications. This way of grouping libraries in runtime subvolumes reminds me a bit of the concepts of baserock (they are using git instead of btrfs) and while I was a bit dubious of all this (because it goes against quite a few of the principles of distribution integration) I m beginning to believe that there s room for both models to work together. It would be nice if Debian could become the reference distribution that upstream developers are using to develop against Linux. This would in turn mean that when upstream distribution their application under this new form, they will provide (or reference) Debian-based subvolumes ready for use by users (even those who are not using Debian as their main OS). And those subvolumes would be managed by the Debian project (probably automatically built from our collection of .deb). We re still quite far from this goal but it will interesting to see this idea mature and become reality. There are plenty of challenges facing us.

18 comments Liked this article? Click here. My blog is Flattr-enabled.

USER=username
API_KEY=1234ABC
OUTPUTDIR=/var/spool/sms
LOG_SERVICE=local1 I ve just written the below script to send SMS via the inteltech.com/clicksend.com service. It takes the above configuration in /etc/sms-pass.cfg where the username is assigned with the clicksend web page and the API key is a long hex string that clicksend provides as a password. The LOG_SERVICE is which syslog service to use for the log messages, on systems that are expected to send many messages I use the service local1 and I use user for development systems. I hope this is useful to someone, and if you have any ideas for improvement then please let me know. #!/bin/sh
# $1 is destination number
# text is on standard input
# standard output gives message ID on success, and 0 is returned
# standard error gives error from server on failure, and 1 is returned . /etc/sms-pass.cfg
OUTPUT=$OUTPUTDIR/out.$$
TEXT= tr "[:space:]" + cut -c 1-159 logger -t sms -p $LOG_SERVICE.info "sending message to $1"
wget -O $OUTPUT "https://api.clicksend.com/http/v2/send.php?method=http&username=$USER&key=$API_KEY&to=$1&message=$TEXT" > /dev/null 2> /dev/null if [ "$?" != "0" ]; then
echo "Error running wget" >&2
logger -t sms -p $LOG_SERVICE.err "failed to send message \"$TEXT\" to $1 wget error"
exit 1
fi if ! grep -q ^.errortext.Success $OUTPUT ; then
cat $OUTPUT >&2
echo >&2
ERR=$(grep ^.errortext $OUTPUT sed -e s/^.errortext.// -e s/..errortext.$//)
logger -t sms -p $LOG_SERVICE.err "failed to send message \"$TEXT\" to $1 $ERR"
rm $OUTPUT
exit 1
fi ID=$(grep ^.messageid $OUTPUT sed -e s/^.messageid.// -e s/..messageid.$//)
rm $OUTPUT logger -t sms -p $LOG_SERVICE.info "sent message to $1 with ID $ID"
exit 0

With yesterday's announcement of littler 0.2.0, I kept thinking about a few not-so-frequently-asked but recurring questions about littler. And an obvious one if of course the relationship to Rscript. As we have pointed out before, littler preceded Rscript. Now, with Rscript being present in every R installation, it is of course by now more widely known. But there is one important aspect which I stressed once or twice in the past and which bears repeating. Due to the lean and mean way in which littler is designed and set-up, it actually starts a lot faster than either Rscript or R. How, you ask? Well we actually query the environement at build time and hardcode a number of settings which R and Rscript re-acquire and learn each time they start. Which is more flexible. But slower. So consider the following, really simple example. In it, we create a simple worker function f which launches the given (and simplest possible) R command of just quitting 250 times (by launching a shell command which loops). We then let littler, Rscript and R (in its just execute this expression mode) run this, and time it via one of the common benchmarking packages.

R> library(rbenchmark)
R> f <- function(cmd)   system(paste0("bash -c \"for i in \\$(seq 1 250); do ", cmd, " -e 'q()'; done\""));  
R> res <- benchmark(f("r"), f("Rscript"), f("R --slave"), replications=5, order="relative")
R> res
            test replications elapsed relative user.self sys.self user.child sys.child
1         f("r")            5  32.256    1.000     0.001    0.004     26.538     5.706
2   f("Rscript")            5 180.154    5.585     0.001    0.004    152.751    28.522
3 f("R --slave")            5 302.714    9.385     0.001    0.004    270.569    33.098
R>

So there: littler does "nothing" about five times as fast as Rscript, and about nine times as fast as R. Not that this matters greatly -- but when you design something for repeated (unsupervised) execution, say in a cronjob, it might as well be lightweight.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

This time, for personal reasons I wasn t able to attend the full DebConf, which started on the Saturday August 22nd. I arrived at Portland on the Tuesday the 26th by noon, at the 4th of the conference. Even though I would like to arrive earlier, the loss was alleviated by the work of the amazing DebConf video team. I was able to follow remotely most of the sessions I would like to attend if I were there already. As I will say to everyone, DebConf is for sure the best conference I have ever attended. The technical and philosophical discussions that take place in talks, BoF sessions or even unplanned ad-hoc gathering are deep. The hacking moments where you have a chance to pair with fellow developers, with whom you usually only have contact remotely via IRC or email, are precious. That is all great. But definitively, catching up with old friends, and making new ones, is what makes DebConf so special. Your old friends are your old friends, and meeting them again after so much time is always a pleasure. New friendships will already start with a powerful bond, which is being part of the Debian community. Being only 4 hours behind my home time zone, jetlag wasn t a big problem during the day. However, I was waking up too early in the morning and consequently getting tired very early at night, so I mostly didn t go out after hacklabs were closed at 10PM. Despite all of the discussion, being in the audience for several talks, other social interactions and whatnot, during this DebConf I have managed to do quite some useful work. debci and the Debian Continuous Integration project I gave a talk where I discussed past, present, and future of debci and the Debian Continuous Integration project. The slides are available, as well as the video recording. One thing I want you to take away is that there is a difference between debci and the Debian Continuous Integration project:

debci is a platform for Continuous Integration specifically tailored for the Debian repository and similar ones. If you work on a Debian derivative, or otherwise provides Debian packages in a repository, you can use debci to run tests for your stuff.
- a (very) few thinks in debci, though, are currently hardcoded for Debian. Other projects using it would be a nice and needed peer pressure to get rid of those.
Debian Continuous Integration is Debian s instance of debci, which currently runs tests for all packages in the unstable distribution that provide autopkgtest support. It will be expanded in the future to run tests on other suites and architectures.

A few days before DebConf, C dric Boutillier managed to extract gem2deb-test-runner from gem2deb, so that autopkgtest tests can be run against any Ruby package that has tests by running gem2deb-test-runner --autopkgtest. gem2deb-test-runner will do the right thing, make sure that the tests don t use code from the source package, but instead run them against the installed package. Then, right after my talk I was glad to discover that the Perl team is also working on a similar tool that will automate running tests for their packages against the installed package. We agreed that they will send me a whitelist of packages in which we could just call that tool and have it do The Right Thing. We might be talking here about getting autopkgtest support (and consequentially continuous integration) for free for almost ~~2000~~ 4000 packages. The missing bits for this to happen are:

making debci use a whitelist of packages that, while not having the appropriate Testsuite: autopkgtest field in the Sources file, could be assumed to have autopkgtest support by calling the right tool (gem2deb-test-runner for Ruby, or the Perl team s new tool for Perl packages).
make the autopkgtest test runner assume a corresponding, implicit, debian/tests/control when it not exists in those packages

During a few days I have mentored Lucas Kanashiro, who also attended DebConf, on writing a patch to add support for email notifications in debci so maintainers can be pro-actively notified of status changes (pass/fail, fail/pass) in their packages. I have also started hacking on the support for distributed workers, based on the initial work by Martin Pitt:

updated the amqp branch against the code in the master branch.
added a debci enqueue command that can be used to force test runs for packages given on the command line.
I sent a patch for librabbitmq that adds support for limiting the number of messages the server will send to a connected client. With this patch applied, the debci workers were modified to request being sent only 1 message at a time, so late workers will start to receive packages to process as soon as they are up. Without this, a single connected worker would receive all messages right away, while a second worker that comes up 1 second later would sit idle until new packages are queued for testing.

Ruby I had some discusion with Christian about making Rubygems install to $HOME by default when the user is not root. We discussed a few implementation options, and while I don t have a solution yet, we have a better understanding of the potential pitfalls. The Ruby BoF session on Friday produced a few interesting discussions. Some take away point include, but are not limited to:

Since the last DebConf, we were able to remove all obsolete Ruby interpreters, and now only have Ruby 2.1 in unstable. Ruby 2.1 will be the default version in Debian 8 (jessie).
There is user interest is being able to run the interpreter from Debian, but install everything else from Rubygems.
We are lacking in all the documentation-related goals for jessie that were proposed at the previous DebConf.

Redmine I was able to make Redmine work with the Rails 4 stack we currently have in unstable/testing. This required using a snapshot of the still unreleased version 3.0 based on the rails-4.1 branch in the upstream Subversion repository as source. I am a little nervous about using a upstream snapshot, though. According to the "roadmap of the project ":http://www.redmine.org/projects/redmine/roadmap the only purpose of the 3.0 release will be to upgrade to Rails 4, but before that happens there should be a 2.6.0 release that is also not released yet. 3.0 should be equivalent to that 2.6.0 version both feature-wise and, specially, bug-wise. The only problem is that we don t know what that 2.6.0 looks like yet. According to the roadmap it seems there is not much left in term of features for 2.6.0, though. The updated package is not in unstable yet, but will be soon. It needs more testing, and a good update to the documentation. Those interested in helping to test Redmine on jessie before the freeze please get in touch with me. Noosfero I gave a lighting talk on Noosfero, a platform for social networking websites I am upstream for. It is a Rails appplication licensed under the AGPLv3, and there are packages for wheezy. You can checkout the slides I used. Video recording is not available yet, but should be soon. That s it. I am looking forward to DebConf 15 at Heidelberg. :-)

Part 1 - Debconf 2014 This year I went to my first Debconf, which took place in Portland, OR during the last week of August 2014. All in all I have to rate my experience as very enlightening and in the end quite fun. First of all, it was a little daunting to go to a conference in 1 - A city I d never been to before; 2 - A conference with 300+ people, only 3 of which I knew and even then I only knew them virtually. Not to mention I was in the presence of some extremely brilliant and known contirbutors in the Debian community which was somewhat intimidating. Just to give you an idea, Linus Torvalds showed up for a Q&A session last Friday morning! Jealous? Actually I missed that too. It was kind of a last minute thing, booked for coincidentally the exact time I d be flying out of Portland. I found out about it much too late. But luckily for me and maybe you, the session was filmed and can be seen here. Isn t that a treat? Point made, there are lots of really talented people there, both techies and non-techies. It s easy to feel you re out of league, at least I did. But I d highly encourage you to ignore such feelings if you re ever in the same situation. Debian has been built on for a long time now, but although a lot has been done, a lot still needs to be done. The Debian community is very welcoming of new contributors and users, regardless of the level of expertise. So far I haven t been snubbed by anyone. To the contrary, all my interactions with the Debian community members has been extremely positive. So go ahead and attend the meetings and presentations, even if you think it s not your area of expertise. Debconf was organized (or at least this one was) as a series of talks, meet ups and ad hoc sessions, some of which occured simultaneously. The sessions were all about different components of the Debian universe, from presenting new features to overviews of accomplishments to discussing issues and how to fix them. A schedule with the location and description of each session was posted on the Debconf wiki. Sometimes none of the sessions at a certain time was on a topic I knew very much about. But I d sit in anyways. There s no rule to attending the sessions, no minimum qualifications required. You ll likely learn something new and you just might find out there is something you can do to contribute. There are also hackathons that are quite the thing or so I heard. Or you could walk about and meet new people, do some networking. I have to say networking was the highlight of the Debconf for me. Remember I said I knew about 3 people who were at the conference? Well, I had actually just corresponded with those people. I didn t really know them. So on my first day I spent quite some time shyly peeking at people s name tags, trying to recognize someone I had met over email or IRC. But with 300 or so people at the conference, I was unsuccessful. So I finally gave up on that strategy and walked up to a random person, stuck out my hand and said, Hi. My name is Juliana. This is my first Debconf. What s your name and what do you do for Debian? This may not be according to protocol, but it worked for me. I got to meet lots of people that way, met some Debian contributos from my home country (Brazil), some from my current city (NYC), and yet others that had similar interests as I do who I might work with in the near future. For example, I love Machine Learning, I m currently beginning my graduate studies on that track. Several Debian contributors offered to introduce me to a well known Machine Learning researcher and Debian contributor who is in NYC. Others had tried out JSCommunicator and had lots of suggestions for new features and fixes, or wanted to know more about the project and WebRTC in general. Also, not everyone there is a super experienced Debian contributor or user. There are a lot of newbies like me. I got to do a quick 20-min presentation and demo of the work I had done on JSCommunicator during GSoC 2014. Oh my goodness that was nerve-wracking, but not half as painful as I expected. My mentor (Daniel Pocock) wisely suggested that when confronted with a question I didn t know how to answer, to redirect the question to to the audience. Chances are, there is someone there that knows the answer. If not, it will at least spark a good discussion. When meeting new people at Debian, a question almost everyone asked is How did you start working with/for Debian? . So I thought it would be a good topic to post about. Part 2 - How I Became a Debian Contributor Sometime in late October of 2013 (I think) I received an email from one of my professors at UNIRIO forwarding a description of the Outreach Program for Women. OPW is a program organized by the GNOME which endeavors to get more women involved in FOSS. OPW is similar to Google Summer of Code; you work remotely from home, guided by an assigned mentor. Debian was one of the 8 participating organizations that year. There was a list of project proposals which I perused, a few of them caught my eye and these projects were all Debian. I d already been a fan of FOSS before. I had used the Ubuntu and Debian OS, I d migrated to GIMP from Photoshop and to Open Office from Microsoft Office, for example. I d strongly advocated the use of some of my prefered open source apps and programs to my friends and family. But I hadn t ever contributed to a FOSS project. There s no time like the present, so I reached out the the mentor responsible for one of the projects I was interested in, Daniel Pocock. Daniel guided me through making a small contribution to a FOSS project, which serves as a token demonstration of my abilities and is part of the application process. I added a small feature to JMXetric and suggested a fix for an issue in the xTuple project. Actually, I had forgotten about this. Recently I made another contribution to xTuple, it s funny to see things come full circle. I also had to write a profile-ish description of my experience and how I intended on contributing during OPW on the Debian wiki, if you d like you can check it out here. I wouldn t follow my example to a T, because in the end I didn t make the OPW selection. Actually, I take that back. The fact I wasn t chosen for OPW that year doesn t mean I was incompetent or incapable of making a valuable contribution. OPW and GSoC do not have unlimited resources; they can t include everyone they d like to. They receive thousands of proposals from very talented engineers and not everyone can participate at a given moment. But even though I wasn t selected, like I said, I could still pitch in. It s good to keep in mind that people usually aren t paid to contribute to FOSS. It s usually volunteer based, which I think is one of the beauties of the FOSS community and in my opinion one of the causes of it s success and great quality. People contribute because they want to, not because they have to. I will say I was a little disappointed at not being chosen. But after being reassured that this rejection wasn t due to any lack on my part, I decided to continue contributing to the Debian project I d applied to. I was begining the final semester of my undergraduate studies which included writing a thesis. To be able to focus on my thesis and graduate on time, I d stopped working temporarily and was studying full time. But I didn t want to lose practice and contributing to a FOSS project is a great way to stay in coding shape while doing something useful. So continue contributing I did. It paid off. I gained experience, added value to a FOSS project and I think my previous contributions added weight to the application I later made for GSoC 2014. I passed this time. To be honest, I really wasn t counting on it. Actually, I was certain I wouldn t pass for some reason - insecure much? But with GSoC I wasn t too anxious about it as I was with the OPW application because by then, I was already hooked . I d learned about all the benefits of becoming a FOSS contributor and I wasn t stopping anytime soon. I had every intention of still working on my FOSS project with or without GSoC. GSoC 2014 ended a week ago (August 18th 2014). There s a list of things I still want to do with JSCommunicator and you can be sure I ll keep working on them. P.S. This is not to say that programs like OPW and GSoC aren t amazing programs. Try it out if you can, it s really a great experience.

Monday morning, 1:45AM. Laura and I walk into the boys room. We turn on the light. Nothing happens. (They re sound sleepers.) Boys, it s time to get up to go get on the train! Four eyes pop open. Yay! Oh I m so excited! And then, Meow! (They enjoy playing with their stuffed cats that Laura got them for Christmas.) Before long, it was out the door to the train station. We even had time to stop at a donut shop along the way. We climbed into our family bedroom (a sleeping car room on Amtrak specifically designed for families of four), and as the train started to move, the excitement of what was going on crept in. Yes, it s 2:42AM, but these are two happy boys:

Jacob and Oliver love trains, and this was the beginning of a 3-day train trip from Newton to Seattle that would take us through Kansas, Colorado, the Rocky Mountains of New Mexico, Arizona, Los Angeles, up the California coast, through the Cascades, and on to Seattle. Whew! Here we are later that morning before breakfast:

Here s our train at a station stop in La Junta, CO:

And at the beautiful small mountain town of Raton, NM:

Some of the passing scenery in New Mexico:

Through it all, we found many things to pass the time. I don t think anybody was bored. I took the boys exploring the train several times we d walk from one end to the other and see what all was there. There was always the dining car for our meals, the lounge car for watching the passing scenery, and on the Coast Starlight, the Pacific Parlor Car. Here we are getting ready for breakfast one morning.

Getting to select meals and order in the train restaurant was a big deal for the boys.

Laura brought one of her origami books, which even managed to pull the boys away from the passing scenery in the lounge car for quite some time.

Origami is serious business:

They had some fun wrapping themselves around my feet and challenging me to move. And were delighted when I could move even though they were trying to weight me down!

Several games of Uno were played, but even those sometimes couldn t compete with the passing scenery:

The Coast Starlight features the Pacific Parlor Car, which was built over 50 years ago for the Santa Fe Hi-Level trains. They ve been updated; the upper level is a lounge and small restaurant, and the lower level has been turned into a small theater. They show movies in there twice a day, but most of the time, the place is empty. A great place to go with little boys to run around and play games.

The boys and I sort of invented a new game: roadrunner and coyote, loosely based on the old Looney Tunes cartoons. Jacob and Oliver would be roadrunners, running around and yelling MEEP MEEP! Meanwhile, I was the coyote, who would try to catch them even briefly succeeding sometimes but ultimately fail in some hilarious way. It burned a lot of energy. And, of course, the parlor car was good for scenery-watching too:

We were right along the Pacific Ocean for several hours sometimes there would be a highway or a town between us and the beach, but usually there was nothing at all between us and the coast. It was beautiful to watch the jagged coastline go by, to gaze out onto the ocean, watching the birds apparently so beautiful that I didn t even think to take some photos. Laura s parents live in California, and took a connecting train. I had arranged for them to have a sleeping car room near ours, so for the last day of the trip, we had a group of 6. Here are the boys with their grandparents at lunch Wednesday:

We stepped off the train in Seattle into beautiful King Street Station.

Our first day in Seattle was a quiet day of not too much. Laura s relatives live near Lake Washington, so we went out there to play. The boys enjoyed gathering black rocks along the shore.

We went blackberry picking after that filled up buckets for a cobbler. The next day, we rode the Seattle Monorail. The boys have been talking about this for months a kind of train they ve never been on. That was the biggest thing in their minds that they were waiting for. They got to ride in the very front, by the operator.

Nice view from up there.

We walked through the Pike Market I hadn t been in such a large and crowded place like that since I was in Guadalajara:

At the Seattle Aquarium, we all had a great time checking out all the exhibits. The please touch one was a particular hit.

Walking underneath the salmon tank was fun too. We spent a couple of days doing things closer to downtown. Laura s cousin works at MOHAI, the Museum of History and Industry, so we spent a morning there. The boys particularly enjoyed the old periscope mounted to the top of the building, and the exhibit on chocolate (of course!)

They love any kind of transportation, so of course we had to get a ride on the Seattle Streetcar that comes by MOHAI.

All weekend long, we had been noticing the seaplanes taking off from Lake Washington and Lake Union (near MOHAI). So finally I decided to investigate, and one morning while Laura was doing things with her cousin, the boys and I took a short seaplane ride from one lake to another, and then rode every method of transportation we could except for ferries (we did that the next day). Here is our Kenmore Air plane:

The view of Lake Washington from 1000 feet was beautiful:

I think we got a better view than the Space Needle, and it probably cost about the same anyhow.

After splashdown, we took the streetcar to a place where we could eat lunch right by the monorail tracks. Then we rode the monorail again. Then we caught a train (it went underground a bit so it was a subway to them!) and rode it a few blocks. There is even scenery underground, it seems.

We rode a bus back, and saved one last adventure for the next day: a ferry to Bainbridge Island.

Laura and I even got some time to ourselves to go have lunch at an amazing Greek restaurant to celebrate a year since we got engaged. It s amazing to think that, by now, it s only a few months until our wedding anniversary too! There are many special memories of the weekend I could mention visiting with Laura s family, watching the boys play with her uncle s pipe organ (it s in his house!), watching the boys play with their grandparents, having all six of us on the train for a day, flying paper airplanes off the balcony, enjoying the cool breeze on the ferry and the beautiful mountains behind the lake. One of my favorites is waking up to high-pitched Meow? Meow meow meow! Wake up, brother! sorts of sounds. There was so much cat-play on the trip, and it was cute to hear. I have the feeling we won t hear things like that much more. So many times on the trip I heard, Oh dad, I am so excited! I never get tired of hearing that. And, of course, I was excited, too.

My recently accepted Licentiate Thesis, which I posted about a couple of days ago, is now available in JyX. Here is the abstract again for reference: Kaijanaho, Antti-Juhani
The extent of empirical evidence that could inform evidence-based design of programming languages. A systematic mapping study.
Jyv skyl : University of Jyv skyl , 2014, 243 p.
(Jyv skyl Licentiate Theses in Computing,
ISSN 1795-9713; 18)
ISBN 978-951-39-5790-2 (nid.)
ISBN 978-951-39-5791-9 (PDF)
Finnish summary Background: Programming language design is not usually informed by empirical studies. In other fields similar problems have inspired an evidence-based paradigm of practice. Central to it are secondary studies summarizing and consolidating the research literature. Aims: This systematic mapping study looks for empirical research that could inform evidence-based design of programming languages. Method: Manual and keyword-based searches were performed, as was a single round of snowballing. There were 2056 potentially relevant publications, of which 180 were selected for inclusion, because they reported empirical evidence on the efficacy of potential design decisions and were published on or before 2012. A thematic synthesis was created. Results: Included studies span four decades, but activity has been sparse until the last five years or so. The form of conditional statements and loops, as well as the choice between static and dynamic typing have all been studied empirically for efficacy in at least five studies each. Error proneness, programming comprehension, and human effort are the most common forms of efficacy studied. Experimenting with programmer participants is the most popular method. Conclusions: There clearly are language design decisions for which empirical evidence regarding efficacy exists; they may be of some use to language designers, and several of them may be ripe for systematic reviewing. There is concern that the lack of interest generated by studies in this topic area until the recent surge of activity may indicate serious issues in their research approach. Keywords: programming languages, programming language design, evidence-based paradigm, efficacy, research methods, systematic mapping study, thematic synthesis

After I moved to a new OpenPGP key (see key transition statement) I have received comments about the short life length of my new key. When I created the key (see my GnuPG setup) I set it to expire after 100 days. Some people assumed that I would have to create a new key then, and therefore wondered what value there is to sign a key that will expire in two months. It doesn t work like that, and below I will explain how OpenPGP key expiration works; how to extend the expiration time of your key; and argue why having a relatively short validity period can be a good thing.
The OpenPGP message format has a sub-packet called the Key Expiration Time, quoting the RFC:

5.2.3.6. Key Expiration Time
   (4-octet time field)
   The validity period of the key.  This is the number of seconds after
   the key creation time that the key expires.  If this is not present
   or has a value of zero, the key never expires.  This is found only on
   a self-signature.

You can print the sub-packets in your OpenPGP key with gpg --list-packets. See below an output for my key, and notice the created 1403464490 (which is Unix time for 2014-06-22 21:14:50) and the subpkt 9 len 4 (key expires after 100d0h0m) which adds up to an expiration on 2014-09-26. Don t confuse the creation time of the key ( created 1403464321 ) with when the signature was created ( created 1403464490 ).

jas@latte:~$ gpg --export 54265e8c   gpg --list-packets  head -20
:public key packet:
	version 4, algo 1, created 1403464321, expires 0
	pkey[0]: [3744 bits]
	pkey[1]: [17 bits]
:user ID packet: "Simon Josefsson "
:signature packet: algo 1, keyid 0664A76954265E8C
	version 4, created 1403464490, md5len 0, sigclass 0x13
	digest algo 10, begin of digest be 8e
	hashed subpkt 27 len 1 (key flags: 03)
	hashed subpkt 9 len 4 (key expires after 100d0h0m)
	hashed subpkt 11 len 7 (pref-sym-algos: 9 8 7 13 12 11 10)
	hashed subpkt 21 len 4 (pref-hash-algos: 10 9 8 11)
	hashed subpkt 30 len 1 (features: 01)
	hashed subpkt 23 len 1 (key server preferences: 80)
	hashed subpkt 2 len 4 (sig created 2014-06-22)
	hashed subpkt 25 len 1 (primary user ID)
	subpkt 16 len 8 (issuer key ID 0664A76954265E8C)
	data: [3743 bits]
:signature packet: algo 1, keyid EDA21E94B565716F
	version 4, created 1403466403, md5len 0, sigclass 0x10
jas@latte:~$

So the key will simply stop being valid after that time? No. It is possible to update the key expiration time value, re-sign the key, and distribute the key to people you communicate with directly or indirectly to OpenPGP keyservers. Since that date is a couple of weeks away, now felt like the perfect opportunity to go through the exercise of taking out my offline master key and boot from a Debian LiveCD and extend its expiry time. See my earlier writeup for LiveCD and USB stick conventions.

user@debian:~$ export GNUPGHOME=/media/FA21-AE97/gnupghome
user@debian:~$ gpg --edit-key 54265e8c
gpg (GnuPG) 1.4.12; Copyright (C) 2012 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Secret key is available.
pub  3744R/54265E8C  created: 2014-06-22  expires: 2014-09-30  usage: SC  
                     trust: ultimate      validity: ultimate
sub  2048R/32F8119D  created: 2014-06-22  expires: 2014-09-30  usage: S   
sub  2048R/78ECD86B  created: 2014-06-22  expires: 2014-09-30  usage: E   
sub  2048R/36BA8F9B  created: 2014-06-22  expires: 2014-09-30  usage: A   
[ultimate] (1). Simon Josefsson 
[ultimate] (2)  Simon Josefsson 
gpg> expire
Changing expiration time for the primary key.
Please specify how long the key should be valid.
         0 = key does not expire
        = key expires in n days
      w = key expires in n weeks
      m = key expires in n months
      y = key expires in n years
Key is valid for? (0) 150
Key expires at Fri 23 Jan 2015 02:47:48 PM UTC
Is this correct? (y/N) y
You need a passphrase to unlock the secret key for
user: "Simon Josefsson "
3744-bit RSA key, ID 54265E8C, created 2014-06-22
pub  3744R/54265E8C  created: 2014-06-22  expires: 2015-01-23  usage: SC  
                     trust: ultimate      validity: ultimate
sub  2048R/32F8119D  created: 2014-06-22  expires: 2014-09-30  usage: S   
sub  2048R/78ECD86B  created: 2014-06-22  expires: 2014-09-30  usage: E   
sub  2048R/36BA8F9B  created: 2014-06-22  expires: 2014-09-30  usage: A   
[ultimate] (1). Simon Josefsson 
[ultimate] (2)  Simon Josefsson 
gpg> key 1
pub  3744R/54265E8C  created: 2014-06-22  expires: 2015-01-23  usage: SC  
                     trust: ultimate      validity: ultimate
sub* 2048R/32F8119D  created: 2014-06-22  expires: 2014-09-30  usage: S   
sub  2048R/78ECD86B  created: 2014-06-22  expires: 2014-09-30  usage: E   
sub  2048R/36BA8F9B  created: 2014-06-22  expires: 2014-09-30  usage: A   
[ultimate] (1). Simon Josefsson 
[ultimate] (2)  Simon Josefsson 
gpg> expire
Changing expiration time for a subkey.
Please specify how long the key should be valid.
         0 = key does not expire
        = key expires in n days
      w = key expires in n weeks
      m = key expires in n months
      y = key expires in n years
Key is valid for? (0) 150
Key expires at Fri 23 Jan 2015 02:48:05 PM UTC
Is this correct? (y/N) y
You need a passphrase to unlock the secret key for
user: "Simon Josefsson "
3744-bit RSA key, ID 54265E8C, created 2014-06-22
pub  3744R/54265E8C  created: 2014-06-22  expires: 2015-01-23  usage: SC  
                     trust: ultimate      validity: ultimate
sub* 2048R/32F8119D  created: 2014-06-22  expires: 2015-01-23  usage: S   
sub  2048R/78ECD86B  created: 2014-06-22  expires: 2014-09-30  usage: E   
sub  2048R/36BA8F9B  created: 2014-06-22  expires: 2014-09-30  usage: A   
[ultimate] (1). Simon Josefsson 
[ultimate] (2)  Simon Josefsson 
gpg> key 2
pub  3744R/54265E8C  created: 2014-06-22  expires: 2015-01-23  usage: SC  
                     trust: ultimate      validity: ultimate
sub* 2048R/32F8119D  created: 2014-06-22  expires: 2015-01-23  usage: S   
sub* 2048R/78ECD86B  created: 2014-06-22  expires: 2014-09-30  usage: E   
sub  2048R/36BA8F9B  created: 2014-06-22  expires: 2014-09-30  usage: A   
[ultimate] (1). Simon Josefsson 
[ultimate] (2)  Simon Josefsson 
gpg> key 1
pub  3744R/54265E8C  created: 2014-06-22  expires: 2015-01-23  usage: SC  
                     trust: ultimate      validity: ultimate
sub  2048R/32F8119D  created: 2014-06-22  expires: 2015-01-23  usage: S   
sub* 2048R/78ECD86B  created: 2014-06-22  expires: 2014-09-30  usage: E   
sub  2048R/36BA8F9B  created: 2014-06-22  expires: 2014-09-30  usage: A   
[ultimate] (1). Simon Josefsson 
[ultimate] (2)  Simon Josefsson 
gpg> expire
Changing expiration time for a subkey.
Please specify how long the key should be valid.
         0 = key does not expire
        = key expires in n days
      w = key expires in n weeks
      m = key expires in n months
      y = key expires in n years
Key is valid for? (0) 150
Key expires at Fri 23 Jan 2015 02:48:14 PM UTC
Is this correct? (y/N) y
You need a passphrase to unlock the secret key for
user: "Simon Josefsson "
3744-bit RSA key, ID 54265E8C, created 2014-06-22
pub  3744R/54265E8C  created: 2014-06-22  expires: 2015-01-23  usage: SC  
                     trust: ultimate      validity: ultimate
sub  2048R/32F8119D  created: 2014-06-22  expires: 2015-01-23  usage: S   
sub* 2048R/78ECD86B  created: 2014-06-22  expires: 2015-01-23  usage: E   
sub  2048R/36BA8F9B  created: 2014-06-22  expires: 2014-09-30  usage: A   
[ultimate] (1). Simon Josefsson 
[ultimate] (2)  Simon Josefsson 
gpg> key 3
pub  3744R/54265E8C  created: 2014-06-22  expires: 2015-01-23  usage: SC  
                     trust: ultimate      validity: ultimate
sub  2048R/32F8119D  created: 2014-06-22  expires: 2015-01-23  usage: S   
sub* 2048R/78ECD86B  created: 2014-06-22  expires: 2015-01-23  usage: E   
sub* 2048R/36BA8F9B  created: 2014-06-22  expires: 2014-09-30  usage: A   
[ultimate] (1). Simon Josefsson 
[ultimate] (2)  Simon Josefsson 
gpg> key 2
pub  3744R/54265E8C  created: 2014-06-22  expires: 2015-01-23  usage: SC  
                     trust: ultimate      validity: ultimate
sub  2048R/32F8119D  created: 2014-06-22  expires: 2015-01-23  usage: S   
sub  2048R/78ECD86B  created: 2014-06-22  expires: 2015-01-23  usage: E   
sub* 2048R/36BA8F9B  created: 2014-06-22  expires: 2014-09-30  usage: A   
[ultimate] (1). Simon Josefsson 
[ultimate] (2)  Simon Josefsson 
gpg> expire
Changing expiration time for a subkey.
Please specify how long the key should be valid.
         0 = key does not expire
        = key expires in n days
      w = key expires in n weeks
      m = key expires in n months
      y = key expires in n years
Key is valid for? (0) 150
Key expires at Fri 23 Jan 2015 02:48:23 PM UTC
Is this correct? (y/N) y
You need a passphrase to unlock the secret key for
user: "Simon Josefsson "
3744-bit RSA key, ID 54265E8C, created 2014-06-22
pub  3744R/54265E8C  created: 2014-06-22  expires: 2015-01-23  usage: SC  
                     trust: ultimate      validity: ultimate
sub  2048R/32F8119D  created: 2014-06-22  expires: 2015-01-23  usage: S   
sub  2048R/78ECD86B  created: 2014-06-22  expires: 2015-01-23  usage: E   
sub* 2048R/36BA8F9B  created: 2014-06-22  expires: 2015-01-23  usage: A   
[ultimate] (1). Simon Josefsson 
[ultimate] (2)  Simon Josefsson 
gpg> save
user@debian:~$ gpg -a --export 54265e8c > /media/KINGSTON/updated-key.txt
user@debian:~$

I remove the transport USB stick from the offline computer, and back on my laptop I can inspect the new updated key. Let s use the same command as before. The key creation time is the same ( created 1403464321 ), of course, but the signature packet has a new time ( created 1409064478 ) since it was signed now. Notice created 1409064478 and subpkt 9 len 4 (key expires after 214d19h35m) . The expiration time is computed based on when the key was generated, not when the signature packet was generated. You may want to double-check the pref-sym-algos, pref-hash-algos and other sub-packets so that you don t accidentally change anything else. (Btw, re-signing your key is also how you would modify those preferences over time.)

jas@latte:~$ cat /media/KINGSTON/updated-key.txt  gpg --list-packets   head -20
:public key packet:
	version 4, algo 1, created 1403464321, expires 0
	pkey[0]: [3744 bits]
	pkey[1]: [17 bits]
:user ID packet: "Simon Josefsson "
:signature packet: algo 1, keyid 0664A76954265E8C
	version 4, created 1409064478, md5len 0, sigclass 0x13
	digest algo 10, begin of digest 5c b2
	hashed subpkt 27 len 1 (key flags: 03)
	hashed subpkt 11 len 7 (pref-sym-algos: 9 8 7 13 12 11 10)
	hashed subpkt 21 len 4 (pref-hash-algos: 10 9 8 11)
	hashed subpkt 30 len 1 (features: 01)
	hashed subpkt 23 len 1 (key server preferences: 80)
	hashed subpkt 25 len 1 (primary user ID)
	hashed subpkt 2 len 4 (sig created 2014-08-26)
	hashed subpkt 9 len 4 (key expires after 214d19h35m)
	subpkt 16 len 8 (issuer key ID 0664A76954265E8C)
	data: [3744 bits]
:user ID packet: "Simon Josefsson "
:signature packet: algo 1, keyid 0664A76954265E8C
jas@latte:~$

Being happy with the new key, I import it and send it to key servers out there.

jas@latte:~$ gpg --import /media/KINGSTON/updated-key.txt 
gpg: key 54265E8C: "Simon Josefsson " 5 new signatures
gpg: Total number processed: 1
gpg:         new signatures: 5
jas@latte:~$ gpg --send-keys 54265e8c
gpg: sending key 54265E8C to hkp server keys.gnupg.net
jas@latte:~$ gpg --keyserver keyring.debian.org  --send-keys 54265e8c
gpg: sending key 54265E8C to hkp server keyring.debian.org
jas@latte:~$

Finally: why go through this hassle, rather than set the key to expire in 50 years? Some reasons for this are:

I don t trust myselt to keep track of a private key (or revocation cert) for 50 years.
I want people to notice my revocation certificate as quickly as possible.
I want people to notice other changes to my key (e.g., cipher preferences) as quickly as possible.

Let s look into the first reason a bit more. What would happen if I lose both the master key and the revocation cert, for a key that s valid 50 years? I would start from scratch and create a new key that I upload to keyservers. Then there would be two keys out there that are valid and identify me, and both will have a set of signatures on it. None of them will be revoked. If I happen to lose the new key again, there will be three valid keys out there with signatures on it. You may argue that this shouldn t be a problem, and that nobody should use any other key than the latest one I want to be used, but that s a technical argument and at this point we have moved into usability, and that s a trickier area. Having users select which out of a couple of apparently all valid keys that exist for me is simply not going to work well. The second is more subtle, but considerably more important. If people retrieve my key from keyservers today, and it expires in 50 years, there will be no need to refresh it from key servers. If for some reason I have to publish my revocation certificate, there will be people that won t see it. If instead I set a short validity period, people will have to refresh my key once in a while, and will then either get an updated expiration time, or will get the revocation certificate. This amounts to a CRL/OCSP-like model. The third is similar to the second, but deserves to be mentioned on its own. Because the cipher preferences are expressed (and signed) in my key, and that ciphers come and go, I would expect that I will modify those during the life-time of my long-term key. If I have a long validity period of my key, people would not refresh it from key servers, and would encrypt messages to me with ciphers I may no longer want to be used. The downside of having a short validity period is that I have to do slightly more work to get out the offline master key once in a while (which I have to once in a while anyway because I m signing other peoples keys) and that others need to refresh my key from the key servers. Can anyone identify other disadvantages? Also, having to explain why I m using a short validity period used to be a downside, but with this writeup posted that won t be the case any more.

To round up the discussion of the Debian Administration site yesterday I flipped the switch on the load-balancing. Rather than this:

  https -> pound \
                  \
  http  -------------> varnish  --> apache

We now have the simpler route for all requests:

http  -> haproxy -> apache
https -> haproxy -> apache

This means we have one less HTTP-request for all incoming secure connections, and these days secure connections are preferred since a Strict-Transport-Security header is set. In other news I've been juggling git repositories; I've setup an installation of GitBucket on my git-host. My personal git repository used to contain some private repositories and some mirrors. Now it contains mirrors of most things on github, as well as many more private repositories. The main reason for the switch was to get a prettier interface and bug-tracker support. A side-benefit is that I can use "groups" to organize repositories, so for example:

http://git.steve.org.uk/websites - Contains website source code.
http://git.steve.org.uk/blogspam - Contains grouped repositories on a single theme.

Most of those are mirrors of the github repositories, but some are new. When signed in I see more sources, for example the source to http://steve.org.uk. I've been pleased with the setup and performance, though I had to add some caching and some other magic at the nginx level to provide /robots.txt, etc, which are not otherwise present. I'm not abandoning github, but I will no longer be using it for private repositories (I was gifted a free subscription a year or three ago), and nor will I post things there exclusively. If a single canonical source location is required for a repository it will be one that I control, maintain, and host. I don't expect I'll give people commit access on this mirror, but it is certainly possible. In the past I've certainly given people access to private repositories for collaboration, etc.

This year I started to widen up my research; and one data source of interest was text because of the lack of structure in it, that makes it often challenging. One of the data sources that everybody seems to use is Twitter: it has a nice API, and few restrictions on using it (except on resharing data). By default, you can get a 1% random sample from all tweets, which is more than enough for many use cases.

We've had some exciting results which a colleague of mine will be presenting tomorrow (Tuesday, Research 22: Topic Modeling) at the KDD 2014 conference:

SigniTrend: Scalable Detection of Emerging Topics in Textual Streams by Hashed Significance Thresholds
Erich Schubert, Michael Weiler, Hans-Peter Kriegel
20th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

You can also explore some (static!) results online at signi-trend.appspot.com

In our experiments, the "news" data set was more interesting. But after some work, we were able to get reasonable results out of Twitter as well. As you can see from the online demo, most of these fall into pop culture: celebrity deaths, sports, hip-hop. Not much that would change our live; and even less that wasn't before captured by traditional media.

The focus of this post is on the preprocessing needed for getting good results from Twitter. Because it is much easier to get bad results!

The first thing you need to realize about Twitter is that due to the media attention/hype it gets, it is full of spam. I'm pretty sure the engineers at Twitter already try to reduce spam; block hosts and fraud apps. But a lot of the trending topics we discovered were nothing but spam.

Retweets - the "like" of Twitter - are an easy source to see what is popular, but are not very interesting if you want to analyze text. They just reiterate the exact same text (except for a "RT " prefix) than earlier tweets. We found results to be more interesting if we removed retweets. Our theory is that retweeting requires much less effort than writing a real tweet; and things that are trending "with effort" are more interesting than those that were just liked.

Teenie spam. If you ever searched for a teenie idol on Twitter, say this guy I hadn't heard of before, but who has 3.88 million followers on Twitter, and search for Tweets addressed to him, you will get millions over millions of results. Many of these tweets look like this:

@5SOS @Luke5SOS @Ashton5SOS @Michael5SOS @Calum5SOS CAN YOU FOLLOW M ? It would mean the world to me! Make my dream come true! x804 Chantal (@Chantal_San22) 25. August 2014

Now if you look at this tweet, there is this odd "x804" at the end. This is to defeat a simple spam filter by Twitter. Because this user did not tweet this just once: instead it is common amongst teenie to spam their idols with follow requests by the dozen. Probably using some JavaScript hack, or third party Twitter client. Occassionally, you see hundreds of such tweets, each sent within a few seconds of the previous one. If you get a 1% sample of these, you still get a few then...

Even worse (for data analysis) than teenie spammers are commercial spammers and wannabe "hackers" that exercise their "sk1llz" by spamming Twitter. To get a sample of such spam, just search for weight loss on Twitter. There is plenty of fresh spam there, usually consisting of some text pretending to be news, and an anonymized link (there is no need to use an URL shortener such as bit.ly on Twitter, since Twitter has its own URL shortener t.co; and you'll end up with double-shortened URLs). And the hacker spam is even worse (e.g. #alvianbencifa) as he seems to have trojaned hundreds of users, and his advertisement seems to be a nonexistant hash tag, which he tries to get into Twitters "trending topics".

And then there are the bots. Plenty of bots spam Twitter with their analysis of trending topics, reinforcing the trending topics. In my opinion, bots such as trending topics indonesia are useless. No wonder there are only 280 followers. And of the trending topics reported, most of them seem to be spam topics...

Bottom line: if you plan on analyzing Twitter data, spend considerable time on preprocessing to filter out spam of various kind. For example, we remove singletons and digits, then feed the data through a duplicate detector. We end up discarding 20%-25% of Tweets. But we still get some of the spam, such as that hackers spam.

All in all, real data is just messy. People posting there have an agenda that might be opposite to yours. And if someone (or some company) promises you wonders from "Big data" and "Twitter", you better have them demonstrate their use first, before buying their services. Don't trust visions of what could be possible, because the first rule of data analysis is: Garbage in, garbage out.

While I imagine Johannes Brahms was referring to music I think the sentiment applies to other endeavours just as well. The trap of believing an idea is worth something without an implementation occurs all too often, however this is not such an unhappy tale.

Lars Wirzenius, Steve McIntyre and myself were chatting a few weeks ago about several of the ongoing Debian discussions. As is often the case these discussions had devolved into somewhat unproductive noise and yet amongst all this was a voice of reason in Russ Allbery.

Lars decided that would take the opportunity of the upcoming opportunity of Debconf 14 to say thank you to Russ for his work. It was decided that a plaque would be a nice gift and I volunteered to do the physical manufacture. Lars came up with the idea of a DEBCON scale similar to the DEFCON scale and got some text together with an initial design idea.

CAD drawing of cut paths in clear acrylic

I took the initial design and as is often the case what is practically possible forced several changes. The prototype was a steep learning curve on using the Cambridge makespace laser cutter to create all the separate pieces.

The construction is pretty simple and consisted of three layers of transparent acrylic plastic. The base layer is a single piece of plastic with the correct outline. The next layer has the DEBCON title, the Debian swirl and level numbers. The top layer has the text engraved in its back surface giving the impression the text floats above the layer behind it.

For the prototype I attempted to glue the pieces together. This was a complete disaster and required discarding the entire piece and starting again with new materials.

The final version with stand ready to be presented

For the second version I used four small nylon bolts to hold the sandwich of layers together which worked very well.

Presentation of plaque photo by Aigars Mahinovs

Yesterday at the Debconf 14 opening Steve McIntyre presented it to Russ and I think he was pleased, certainly he was surprised (photo from Aigars Mahinovs).

The design files are available from my design git repo, though why anyone would want to reproduce it I have no idea ;-)

Stefano Zacchiroli opened DebConf 14 with an insightful talk titled Debian in the Dark Ages of Free Software (slides available, video available soon). He makes the point (quoting slide 16) that the Free Software community is winning a war that is becoming increasingly pointless: yes, users have 100% Free Software thin client at their fingertips [or are really a few steps from there]. But all their relevant computations happen elsewhere, on remote systems they do not control, in the Cloud. That give-up on control of computing is a huge and important problem, and probably the largest challenge for everybody caring about freedom, free speech, or privacy today. Stefano rightfully points out that we must do something about it. The big question is: how can we, as a community, address it? Towards a Free Service Definition? I believe that we all feel a bit lost with this issue because we are trying to attack it with our current tools & weapons. However, they are largely irrelevant here: the Free Software Definition is about software, and software is even to be understood strictly in it, as software programs. Applying it to services, or to computing in general, doesn t lead anywhere. In order to increase the general awareness about this issue, we should define more precisely what levels of control can be provided, to understand what services are not providing to users, and to make an informed decision about waiving a particular level of control when choosing to use a particular service. Benjamin Mako Hill pointed out yesterday during the post-talk chat that services are not black or white: there aren t impure and pure services. Instead, there s a graduation of possible levels of control for the computing we do. The Free Software Definition lists four freedoms how many freedoms, or types of control, should there be in a Free Service Definition, or a Controlled-Computing Definition? Again, this is not only about software: the platform on which a particular piece of software is executed has a huge impact on the available level of control: running your own instance of WordPress, or using an instance on wordpress.com, provides very different control (even if as Asheesh Laroia pointed out yesterday, WordPress does a pretty good job at providing export and import features to limit data lock-in). The creation of such a definition is an iterative process. I actually just realized today that (according to Wikipedia) the very first occurrence of an attempt at a Free Software Definition was published in 1986 (GNU s bulletin Vol 1 No.1, page 8) I thought it happened a couple of years earlier. Are there existing attempts at defining such freedoms or levels of controls, and at benchmarking such criteria against existing services? Such criteria would not only include control over software modifications and (re)distribution, but also likely include mentions of interoperability and open standards, both to enable the user to move to a compatible service, and to avoid forcing the user to use a particular implementation of a service. A better understanding of network effects is also needed: how much and what type of service lock-in is acceptable on social networks in exchange of functionality? I think that we should inspire from what was achieved during the last 30 years on Free Software. The tools that were produced are probably irrelevant to address this issue, but there s a lot to learn from the way they were designed. I really look forward to the day when we will have:

a Free Software Definition equivalent for services
Debian Free Software Guidelines-like tests/checklist to evaluate services
an equivalent of The Cathedral and the Bazaar, explaining how one can build successful business models on top of open services

Exciting times!

Yesterday I received my official diploma for the degree of Licentiate of Philosophy. The degree lies between a Master s degree and a doctorate, and is not required; it consists of the coursework required for a doctorate, and a Licentiate Thesis, in which the student demonstrates good conversance with the field of research and the capability of independently and critically applying scientific research methods (official translation of the Government decree on university degrees 794/2004, Section 23 Paragraph 2). The title and abstract of my Licentiate Thesis follow:

Kaijanaho, Antti-Juhani
The extent of empirical evidence that could inform evidence-based design of programming languages. A systematic mapping study.
Jyv skyl : University of Jyv skyl , 2014, 243 p.
(Jyv skyl Licentiate Theses in Computing,
ISSN 1795-9713; 18)
ISBN 978-951-39-5790-2 (nid.)
ISBN 978-951-39-5791-9 (PDF)
Finnish summary Background: Programming language design is not usually informed by empirical studies. In other fields similar problems have inspired an evidence-based paradigm of practice. Central to it are secondary studies summarizing and consolidating the research literature. Aims: This systematic mapping study looks for empirical research that could inform evidence-based design of programming languages. Method: Manual and keyword-based searches were performed, as was a single round of snowballing. There were 2056 potentially relevant publications, of which 180 were selected for inclusion, because they reported empirical evidence on the efficacy of potential design decisions and were published on or before 2012. A thematic synthesis was created. Results: Included studies span four decades, but activity has been sparse until the last five years or so. The form of conditional statements and loops, as well as the choice between static and dynamic typing have all been studied empirically for efficacy in at least five studies each. Error proneness, programming comprehension, and human effort are the most common forms of efficacy studied. Experimenting with programmer participants is the most popular method. Conclusions: There clearly are language design decisions for which empirical evidence regarding efficacy exists; they may be of some use to language designers, and several of them may be ripe for systematic reviewing. There is concern that the lack of interest generated by studies in this topic area until the recent surge of activity may indicate serious issues in their research approach. Keywords: programming languages, programming language design, evidence-based paradigm, efficacy, research methods, systematic mapping study, thematic synthesis

A Licentiate Thesis is assessed by two examiners, usually drawn from outside of the home university; they write (either jointly or separately) a substantiated statement about the thesis, in which they suggest a grade. The final grade is almost always the one suggested by the examiners. I was very fortunate to have such prominent scientists as Dr. Stefan Hanenberg and Prof. Stein Krogdahl as the examiners of my thesis. They recommended, and I received, the grade very good (4 on a scale of 1 5). The thesis has been ~~accepted for publication~~ published in our faculty s licentiate thesis series and ~~will in due course appear~~ has appeared in our university s electronic database (along with a very small number of printed copies). ~~In the mean time, if anyone wants an electronic preprint, send me email at antti-juhani.kaijanaho@jyu.fi.~~

Figure 1 of the thesis: an overview of the mapping process

As you can imagine, the last couple of months in the spring were very stressful for me, as I pressed on to submit this thesis. After submission, it took me nearly two months to recover (which certain people who emailed me on Planet Haskell business during that period certainly noticed). It represents the fruit of almost four years of work (way more than normally is taken to complete a Licentiate Thesis, but never mind that), as I designed this study in Fall 2010.

Figure 8 of the thesis: Core studies per publication year

Recently, I have been writing in my blog a series of posts in which I have been trying to clear my head about certain foundational issues that irritated me during the writing of the thesis. The thesis contains some of that, but that part of it is not very strong, as my examiners put it, for various reasons. The posts have been a deliberately non-academic attempt to shape the thoughts into words, to see what they look like fixed into a tangible form. (If you go read them, be warned: many of them are deliberately provocative, and many of them are intended as tentative in fact if not in phrasing; the series also is very incomplete at this time.) I closed my previous post, the latest post in that series, as follows:

In fact, the whole of 20th Century philosophy of science is a big pile of failed attempts to explain science; not one explanation is fully satisfactory. [...] Most scientists enjoy not pondering it, for it s a bit like being a cartoon character: so long as you don t look down, you can walk on air.

I wrote my Master s Thesis (PDF) in 2002. It was about the formal method called B ; but I took a lot of time and pages to examine the history and content of formal logic. My supervisor was, understandably, exasperated, but I did receive the highest possible grade for it (which I never have fully accepted I deserved). The main reason for that digression: I looked down, and I just had to go poke the bridge I was standing on to make sure I was not, in fact, walking on air. In the many years since, I ve taken a lot of time to study foundations, first of mathematics, and more recently of science. It is one reason it took me about eight years to come up with a doable doctoral project (and I am still amazed that my department kept employing me; but I suppose they like my teaching, as do I). The other reason was, it took me that long to realize how to study the design of programming languages without going where everyone has gone before. Debian people, if any are still reading, may find it interesting that I found significant use for the dctrl-tools toolset I have been writing for Debian for about fifteen years: I stored my data collection as a big pile of dctrl-format files. I ended up making some changes to the existing tools (I should upload the new version soon, I suppose), and I wrote another toolset (unfortunately one that is not general purpose, like the dctrl-tools are) in the process. For the Haskell people, I mainly have an apology for not attending to Planet Haskell duties in the summer; but I am back in business now. I also note, somewhat to my regret, that I found very few studies dealing with Haskell. I just checked; I mention Haskell several times in the background chapter, but it is not mentioned in the results chapter (because there were not studies worthy of special notice). I am already working on extending this work into a doctoral thesis. I expect, and hope, to complete that one faster.

After a bit more than 9 years, I am replacing Serendipity, which as been hosting my blog, by a self-made static solution. This means that when you are reading this, my server no longer has to execute some rather large body of untyped code to produce the bytes sent to you. Instead, that happens once in a while on my laptop, and they are stored as static files on the server. I hope to get a little performance boost from this, so that my site can more easily hold up to being mentioned on hackernews. I also do not want to worry about security issues in Serendipity static files are not hacked. Of course there are down-sides to having a static blog. The editing is a bit more annoying: I need to use my laptop (previously I could post from anywhere) and I edit text files instead of using a JavaScript-based WYSIWYG editor (but I was slightly annoyed by that as well). But most importantly your readers cannot comment on static pages. There are cloud-based solutions that integrate commenting via JavaScript on your static pages, but I decided to go for something even more low-level: You can comment by writing an e-mail to me, and I ll put your comment on the page. This has the nice benefit of solving the blog comment spam problem. The actual implementation of the blog is rather masochistic, as my web page runs on one of these weird obfuscated languages (XSLT). Previously, it contained of XSLT stylesheets producing makefiles calling XSLT sheets. Now it is a bit more-self-contained, with one XSLT stylesheet writing out all the various html and rss files. I managed to import all my old posts and comments thanks to this script by Michael Hamann (I had played around with this some months ago and just spend what seemed to be an hour to me to find this script again) and a small Haskell script. Old URLs are rewritten (using mod_rewrite) to the new paths, but feed readers might still be confused by this. This opens the door to a long due re-design of my webpage. But not today...

I've mentored a number of students in 2013 and 2014 for Debian and Ganglia and most of the companies I've worked with have run internships and graduate programs from time to time. GSoC 2014 has just finished and with all the excitement, many students are already asking what they can do to prepare and be selected in 2015. My own observation is that the more time the organization has to get to know the student, the more confident they can be selecting that student. Furthermore, the more time that the student has spent getting to know the free software community, the more easily they can complete GSoC. Here I present a list of things that students can do to maximize their chance of selection and career opportunities at the same time. These tips are useful for people applying for GSoC itself and related programs such as GNOME's Outreach Program for Women or graduate placements in companies. Disclaimers There is no guarantee that Google will run the program again in 2015 or any future year. There is no guarantee that any organization or mentor (including myself) will be involved until the official list of organizations is published by Google.

Do not follow the advice of web sites that invite you to send pizza or anything else of value to prospective mentors. Following the steps in this page doesn't guarantee selection. That said, people who do follow these steps are much more likely to be considered and interviewed than somebody who hasn't done any of the things in this list. Understand what free software really is You may hear terms like free software and open source software used interchangeably. They don't mean exactly the same thing and many people use the term free software for the wrong things. Not all open source projects meet the definition of free software. Those that don't, usually as a result of deficiencies in their licenses, are fundamentally incompatible with the majority of software that does use genuinely free licenses. Google Summer of Code is about both writing and publishing your code and it is also about community. It is fundamental that you know the basics of licensing and how to choose a free license that empowers the community to collaborate on your code well after GSoC has finished. Please review the definition of free software early on and come back and review it from time to time. The The GNU Project / Free Software Foundation have excellent resources to help you understand what a free software license is and how it works to maximize community collaboration. Don't look for shortcuts There is no shortcut to GSoC selection and there is no shortcut to GSoC completion. The student stipend (USD $5,500 in 2014) is not paid to students unless they complete a minimum amount of valid code. This means that even if a student did find some shortcut to selection, it is unlikely they would be paid without completing meaningful work. If you are the right candidate for GSoC, you will not need a shortcut anyway. Are you the sort of person who can't leave a coding problem until you really feel it is fixed, even if you keep going all night? Have you ever woken up in the night with a dream about writing code still in your head? Do you become irritated by tedious or repetitive tasks and often think of ways to write code to eliminate such tasks? Does your family get cross with you because you take your laptop to Christmas dinner or some other significant occasion and start coding? If some of these statements summarize the way you think or feel you are probably a natural fit for GSoC. An opportunity money can't buy The GSoC stipend will not make you rich. It is intended to make sure you have enough money to survive through the summer and focus on your project. Professional developers make this much money in a week in leading business centers like New York, London and Singapore. When you get to that stage in 3-5 years, you will not even be thinking about exactly how much you made during internships. GSoC gives you an edge over other internships because it involves publicly promoting your work. Many companies still try to hide the potential of their best recruits for fear they will be poached or that they will be able to demand higher salaries. Everything you complete in GSoC is intended to be published and you get full credit for it. Imagine a young musician getting the opportunity to perform on the main stage at a rock festival. This is how the free software community works. It is a meritocracy and there is nobody to hold you back.

Having a portfolio of free software that you have created or collaborated on and a wide network of professional contacts that you develop before, during and after GSoC will continue to pay you back for years to come. While other graduates are being screened through group interviews and testing days run by employers, people with a track record in a free software project often find they go straight to the final interview round. Register your domain name and make a permanent email address Free software is all about community and collaboration. Register your own domain name as this will become a focal point for your work and for people to get to know you as you become part of the community. This is sound advice for anybody working in IT, not just programmers. It gives the impression that you are confident and have a long term interest in a technology career. Choosing the provider: as a minimum, you want a provider that offers DNS management, static web site hosting, email forwarding and XMPP services all linked to your domain. You do not need to choose the provider that is linked to your internet connection at home and that is often not the best choice anyway. The XMPP foundation maintains a list of providers known to support XMPP. Create an email address within your domain name. The most basic domain hosting providers will let you forward the email address to a webmail or university email account of your choice. Configure your webmail to send replies using your personalized email address in the From header. Update your ~/.gitconfig file to use your personalized email address in your Git commits. Create a web site and blog Start writing a blog. Host it using your domain name. Some people blog every day, other people just blog once every two or three months. Create links from your web site to your other profiles, such as a Github profile page. This helps reinforce the pages/profiles that are genuinely related to you and avoid confusion with the pages of other developers. Many mentors are keen to see their students writing a weekly report on a blog during GSoC so starting a blog now gives you a head start. Mentors look at blogs during the selection process to try and gain insight into which topics a student is most suitable for. Create a profile on Github Github is one of the most widely used software development web sites. Github makes it quick and easy for you to publish your work and collaborate on the work of other people. Create an account today and get in the habbit of forking other projects, improving them, committing your changes and pushing the work back into your Github account. Github will quickly build a profile of your commits and this allows mentors to see and understand your interests and your strengths. In your Github profile, add a link to your web site/blog and make sure the email address you are using for Git commits (in the ~/.gitconfig file) is based on your personal domain. Start using PGP Pretty Good Privacy (PGP) is the industry standard in protecting your identity online. All serious free software projects use PGP to sign tags in Git, to sign official emails and to sign official release files. The most common way to start using PGP is with the GnuPG (GNU Privacy Guard) utility. It is installed by the package manager on most Linux systems. When you create your own PGP key, use the email address involving your domain name. This is the most permanent and stable solution. Print your key fingerprint using the gpg-key2ps command, it is in the signing-party package on most Linux systems. Keep copies of the fingerprint slips with you.

This is what my own PGP fingerprint slip looks like. You can also print the key fingerprint on a business card for a more professional look. Using PGP, it is recommend that you sign any important messages you send but you do not have to encrypt the messages you send, especially if some of the people you send messages to (like family and friends) do not yet have the PGP software to decrypt them. If using the Thunderbird (Icedove) email client from Mozilla, you can easily send signed messages and validate the messages you receive using the Enigmail plugin. Get your PGP key signed Once you have a PGP key, you will need to find other developers to sign it. For people I mentor personally in GSoC, I'm keen to see that you try and find another Debian Developer in your area to sign your key as early as possible. Free software events Try and find all the free software events in your area in the months between now and the end of the next Google Summer of Code season. Aim to attend at least two of them before GSoC. Look closely at the schedules and find out about the individual speakers, the companies and the free software projects that are participating. For events that span more than one day, find out about the dinners, pub nights and other social parts of the event. Try and identify people who will attend the event who have been GSoC mentors or who intend to be. Contact them before the event, if you are keen to work on something in their domain they may be able to make time to discuss it with you in person. Take your PGP fingerprint slips. Even if you don't participate in a formal key-signing party at the event, you will still find some developers to sign your PGP key individually. You must take a photo ID document (such as your passport) for the other developer to check the name on your fingerprint but you do not give them a copy of the ID document. Events come in all shapes and sizes. FOSDEM is an example of one of the bigger events in Europe, linux.conf.au is a similarly large event in Australia. There are many, many more local events such as the Debian France mini-DebConf in Lyon, 2015. Many events are either free or free for students but please check carefully if there is a requirement to register before attending. On your blog, discuss which events you are attending and which sessions interest you. Write a blog during or after the event too, including photos.

Quantcast generously hosted the Ganglia community meeting in San Francisco, October 2013. We had a wild time in their offices with mini-scooters, burgers, beers and the Ganglia book. That's me on the pink mini-scooter and Bernard Li, one of the other Ganglia GSoC 2014 admins is on the right. Install Linux GSoC is fundamentally about free software. Linux is to free software what a tree is to the forest. Using Linux every day on your personal computer dramatically increases your ability to interact with the free software community and increases the number of potential GSoC projects that you can participate in. This is not to say that people using Mac OS or Windows are unwelcome. I have worked with some great developers who were not Linux users. Linux gives you an edge though and the best time to gain that edge is now, while you are a student and well before you apply for GSoC. If you must run Windows for some applications used in your course, it will run just fine in a virtual machine using Virtual Box, a free software solution for desktop virtualization. Use Linux as the primary operating system. Here are links to download ISO DVD (and CD) images for some of the main Linux distributions:

If you are nervous about getting started with Linux, install it on a spare PC or in a virtual machine before you install it on your main PC or laptop. Linux is much less demanding on the hardware than Windows so you can easily run it on a machine that is 5-10 years old. Having just 4GB of RAM and 20GB of hard disk is usually more than enough for a basic graphical desktop environment although having better hardware makes it faster. Your experiences installing and running Linux, especially if it requires some special effort to make it work with some of your hardware, make interesting topics for your blog. Decide which technologies you know best Personally, I have mentored students working with C, C++, Java, Python and JavaScript/HTML5. In a GSoC program, you will typically do most of your work in just one of these languages. From the outset, decide which language you will focus on and do everything you can to improve your competence with that language. For example, if you have already used Java in most of your course, plan on using Java in GSoC and make sure you read Effective Java (2nd Edition) by Joshua Bloch. Decide which themes appeal to you Find a topic that has long-term appeal for you. Maybe the topic relates to your course or maybe you already know what type of company you would like to work in. Here is a list of some topics and some of the relevant software projects:

System administration, servers and networking: consider projects involving monitoring, automation, packaging. Ganglia is a great community to get involved with and you will encounter the Ganglia software in many large companies and academic/research networks. Contributing to a Linux distribution like Debian or Fedora packaging is another great way to get into system administration.
Desktop and user interface: consider projects involving window managers and desktop tools or adding to the user interface of just about any other software.
Big data and data science: this can apply to just about any other theme. For example, data science techniques are frequently used now to improve system administration.
Business and accounting: consider accounting, CRM and ERP software.
Finance and trading: consider projects like R, market data software like OpenMAMA and connectivity software (Apache Camel)
Real-time communication (RTC), VoIP, webcam and chat: look at the JSCommunicator or the Jitsi project
Web (JavaScript, HTML5): look at the JSCommunicator

Before the GSoC application process begins, you should aim to learn as much as possible about the theme you prefer and also gain practical experience using the software relating to that theme. For example, if you are attracted to the business and accounting theme, install the PostBooks suite and get to know it. Maybe you know somebody who runs a small business: help them to upgrade to PostBooks and use it to prepare some reports. Make something Make some small project, less than two week's work, to demonstrate your skills. It is important to make something that somebody will use for a practical purpose, this will help you gain experience communicating with other users through Github. For an example, see the servlet Juliana Louback created for fixing phone numbers in December 2013. It has since been used as part of the Lumicall web site and Juliana was selected for a GSoC 2014 project with Debian. There is no better way to demonstrate to a prospective mentor that you are ready for GSoC than by completing and publishing some small project like this yourself. If you don't have any immediate project ideas, many developers will also be able to give you tips on small projects like this that you can attempt, just come and ask us on one of the mailing lists. Ideally, the project will be something that you would use anyway even if you do not end up participating in GSoC. Such projects are the most motivating and rewarding and usually end up becoming an example of your best work. To continue the example of somebody with a preference for business and accounting software, a small project you might create is a plugin or extension for PostBooks. Getting to know prospective mentors Many web sites provide useful information about the developers who contribute to free software projects. Some of these developers may be willing to be a GSoC mentor. For example, look through some of the following:

Planet / Blog aggregation sites: these sites all have links to the blogs of many developers. They are useful sources of information about events and also finding out who works on what.
Developer profile pages. Many projects publish a page about each developer and the packages, modules or other components he/she is responsible for. Look through these lists for areas of mutual interest.
- my package list in Debian
- my package list in Fedora
Developer github profiles. Github makes it easy to see what projects a developer has contributed to. To see many of my own projects, browse through the history at my own Github profile

Getting on the mentor's shortlist Once you have identified projects that are interesting to you and developers who work on those projects, it is important to get yourself on the developer's shortlist. Basically, the shortlist is a list of all students who the developer believes can complete the project. If I feel that a student is unlikely to complete a project or if I don't have enough information to judge a student's probability of success, that student will not be on my shortlist. If I don't have any student on my shortlist, then a project will not go ahead at all. If there are multiple students on the shortlist, then I will be looking more closely at each of them to try and work out who is the best match. One way to get a developer's attention is to look at bug reports they have created. Github makes it easy to see complaints or bug reports they have made about their own projects or other projects they depend on. Another way to do this is to search through their code for strings like FIXME and TODO. Projects with standalone bug trackers like the Debian bug tracker also provide an easy way to search for bug reports that a specific person has created or commented on. Once you find some relevant bug reports, email the developer. Ask if anybody else is working on those issues. Try and start with an issue that is particularly easy and where the solution is interesting for you. This will help you learn to compile and test the program before you try to fix any more complicated bugs. It may even be something you can work on as part of your academic program. Find successful projects from the previous year Contact organizations and ask them which GSoC projects were most successful. In many organizations, you can find the past students' project plans and their final reports published on the web. Read through the plans submitted by the students who were chosen. Then read through the final reports by the same students and see how they compare to the original plans. Start building your project proposal now Don't wait for the application period to begin. Start writing a project proposal now. When writing a proposal, it is important to include several things:

Think big: what is the goal at the end of the project? Does your work help the greater good in some way, such as increasing the market share of Linux on the desktop?
Details: what are specific challenges? What tools will you use?
Time management: what will you do each week? Are there weeks where you will not work on GSoC due to vacation or other events? These things are permitted but they must be in your plan if you know them in advance. If an accident or death in the family cut a week out of your GSoC project, which work would you skip and would your project still be useful without that? Having two weeks of flexible time in your plan makes it more resilient against interruptions.
Communication: are you on mailing lists, IRC and XMPP chat? Will you make a weekly report on your blog?
Users: who will benefit from your work?
Testing: who will test and validate your work throughout the project? Ideally, this should involve more than just the mentor.

If your project plan is good enough, could you put it on Kickstarter or another crowdfunding site? This is a good test of whether or not a project is going to be supported by a GSoC mentor. Learn about packaging and distributing software Packaging is a vital part of the free software lifecycle. It is very easy to upload a project to Github but it takes more effort to have it become an official package in systems like Debian, Fedora and Ubuntu. Packaging and the communities around Linux distributions help you reach out to users of your software and get valuable feedback and new contributors. This boosts the impact of your work. To start with, you may want to help the maintainer of an existing package. Debian packaging teams are existing communities that work in a team and welcome new contributors. The Debian Mentors initiative is another great starting place. In the Fedora world, the place to start may be in one of the Special Interest Groups (SIGs). Think from the mentor's perspective After the application deadline, mentors have just 2 or 3 weeks to choose the students. This is actually not a lot of time to be certain if a particular student is capable of completing a project. If the student has a published history of free software activity, the mentor feels a lot more confident about choosing the student. Some mentors have more than one good student while other mentors receive no applications from capable students. In this situation, it is very common for mentors to send each other details of students who may be suitable. Once again, if a student has a good Github profile and a blog, it is much easier for mentors to try and match that student with another project.

Conclusion Getting into the world of software engineering is much like joining any other profession or even joining a new hobby or sporting activity. If you run, you probably have various types of shoe and a running watch and you may even spend a couple of nights at the track each week. If you enjoy playing a musical instrument, you probably have a collection of sheet music, accessories for your instrument and you may even aspire to build a recording studio in your garage (or you probably know somebody else who already did that). The things listed on this page will not just help you walk the walk and talk the talk of a software developer, they will put you on a track to being one of the leaders. If you look over the profiles of other software developers on the Internet, you will find they are doing most of the things on this page already. Even if you are not selected for GSoC at all or decide not to apply, working through the steps on this page will help you clarify your own ideas about your career and help you make new friends in the software engineering community.

So I previously talked about the setup behind Debian Administration, and my complaints about the slownes. The previous post talked about the logical setup, and the hardware. This post talks about the more interesting thing. The code. The code behind the site was originally written by Denny De La Haye. I found it and reworked it a lot, most obviously adding structure and test cases. Once I did that the early version of the site was born. Later my version became the official version, as when Denny setup Police State UK he used my codebase rather than his. So the code huh? Well as you might expect it is written in Perl. There used to be this layout:

]
yawns/cgi-bin/index.cgi
yawns/cgi-bin/Pages.pl
yawns/lib/...
yawns/htdocs/

Almost every request would hit the index.cgi script, which would parse the request and return the appropriate output via the standard CGI interface. How did it know what you wanted? Well sometimes there would be a paramater set which would be looked up in a dispatch-table:

/cgi-bin/index.cgi?article=40         - Show article 40
/cgi-bin/index.cgi?view_user=Steve    - Show the user Steve
/cgi-bin/index.cgi?recent_comments=10 - Show the most recent comments.

Over time the code became hard to update because there was no consistency, and over time the site became slow because this is not a quick setup. Spiders, bots, and just average users would cause a lot of perl processes to run. So? What did I do? I moved the thing to using FastCGI, which avoids the cost of forking Perl and loading (100k+) the code. Unfortunately this required a bit of work because all the parameter handling was messy and caused issues if I just renamed index.cgi -> index.fcgi. The most obvious solution was to use one parameter, globally, to specify the requested mode of operation. Hang on? One parameter to control the page requested? A persistant environment? What does that remind me of? Yes. CGI::Application. I started small, and pulled some of the code out of index.cgi + Pages.pl, and over into a dedicated CGI::Application class:

Application::Feeds - Called via /cgi-bin/f.fcgi.
Application::Ajax - Called via /cgi-bin/a.fcgi.

So now every part of the site that is called by Ajax has one persistent handler, and every part of the site which returns RSS feeds has another. I had some fun setting up the sessions to match those created by the old stuff, but I quickly made it work, as this example shows:

Viewing recent comments:
- https://www.debian-administration.org/recent/comments
- http://debian-administration.org/cgi-bin/f.fcgi?mode=recent_comments

The final job was the biggest, moving all the other (non-feed, non-ajax) modes over to a similar CGI::Application structure. There were 53 modes that had to be ported, and I did them methodically, first porting all the Poll-related requests, then all the article-releated ones, & etc. I think I did about 15 a day for three days. Then the rest in a sudden rush. In conclusion the code is now fast because we don't use CGI, and instead use FastCGI. This allowed minor changes to be carried out, such as compiling the HTML::Template templates which determine the look and feel, etc. Those things don't make sense in the CGI environment, but with persistence they are essentially free. The site got a little more of a speed boost when I updated DNS, and a lot more when I blacklisted a bunch of IP-space. As I was wrapping this up I realized that the code had accidentally become closed - because the old repository no longer exists. That is not deliberate, or intentional, and will be rectified soon. The site would never have been started if I'd not seen Dennys original project, and although I don't think others would use the code it should be possible. I remember at the time I was searching for things like "Perl CMS" and finding Slashcode, and Scoop, which I knew were too heavyweight for my little toy blog. In conclusion Debian Administration website is 10 years old now. It might not have changed the world, it might have become less relevant, but I'm glad I tried, and I'm glad there were years when it really was the best place to be. These days there are HowtoForges, blogs, spam posts titled "How to install SSH on Trusty", "How to install SSH on Wheezy", "How to install SSH on Precise", and all that. No shortage of content, just finding the good from the bad is the challenge. Me? The single best resource I read these days is probably LWN.net. Starting to ramble now. Go look at my quick hack for remote command execution https://github.com/skx/nanoexec ?

A lecture at LCA 2011 which included some inappropriate slides was followed by long discussions on mailing lists. In February 2011 I wrote a blog post debunking some of the bogus arguments in two lists [1]. One of the noteworthy incidents in the mailing list discussion concerned Ted Ts o (an influential member of the Linux community) debating the definition of rape. My main point on that issue in Feb 2011 was that it s insensitive to needlessly debate the statistics. Recently Valerie Aurora wrote about another aspect of this on The Ada Initiative blog [2] and on her personal blog. Some of her significant points are that conference harassment doesn t end when the conference ends (it can continue on mailing lists etc), that good people shouldn t do nothing when bad things happen, and that free speech doesn t mean freedom from consequences or the freedom to use private resources (such as conference mailing lists) without restriction. Craig Sanders wrote a very misguided post about the Ted Ts o situation [3]. One of the many things wrong with his post is his statement I m particularly disgusted by the men who intervene way too early without an explicit invitation or request for help or a clear need such as an immediate threat of violence in womens issues . I believe that as a general rule when any group of people are involved in causing a problem they should be involved in fixing it. So when we have problems that are broadly based around men treating women badly the prime responsibility should be upon men to fix them. It seems very clear that no matter what scope is chosen for fixing the problems (whether it be lobbying for new legislation, sociological research, blogging, or directly discussing issues with people to change their attitudes) women are doing considerably more than half the work. I believe that this is an indication that overall men are failing. Asking for Help I don t believe that members of minority groups should have to ask for help. Asking isn t easy, having someone spontaneously offer help because it s the right thing to do can be a lot easier to accept psychologically than having to beg for help. There is a book named Women Don t Ask which has a page on the geek feminism Wiki [4]. I think the fact that so many women relate to a book named Women Don t Ask is an indication that we shouldn t expect women to ask directly, particularly in times of stress. The Wiki page notes a criticism of the book that some specific requests are framed as complaining , so I think we should consider a complaint from a woman as a direct request to do something. The geek feminism blog has an article titled How To Exclude Women Without Really Trying which covers many aspects of one incident [5]. Near the end of the article is a direct call for men to be involved in dealing with such problems. The geek feminism Wiki has a page on Allies which includes Even a blog post helps [6]. It seems clear from public web sites run by women that women really want men to be involved. Finally when I get blog comments and private email from women who thank me for my posts I take it as an implied request to do more of the same. One thing that we really don t want is to have men wait and do nothing until there is an immediate threat of violence. There are two massive problems with that plan, one is that being saved from a violent situation isn t a fun experience, the other is that an immediate threat of violence is most likely to happen when there is no-one around to intervene. Men Don t Listen to Women Rebecca Solnit wrote an article about being ignored by men titled Men Explain Things to Me [7]. When discussing women s issues the term Mansplaining is often used for that sort of thing, the geek feminism Wiki has some background [8]. It seems obvious that the men who have the greatest need to be taught some things related to women s issues are the ones who are least likely to listen to women. This implies that other men have to teach them. Craig says that women need space to discover and practice their own strength and their own voices . I think that the best way to achieve that goal is to listen when women speak. Of course that doesn t preclude speaking as well, just listen first, listen carefully, and listen more than you speak. Craig claims that when men like me and Matthew Garrett comment on such issues we are making women s spaces more comfortable, more palatable, for men . From all the discussion on this it seems quite obvious that what would make things more comfortable for men would be for the issue to never be discussed at all. It seems to me that two of the ways of making such discussions uncomfortable for most men are to discuss sexual assault and to discuss what should be done when you have a friend who treats women in a way that you don t like. Matthew has covered both of those so it seems that he s doing a good job of making men uncomfortable I think that this is a good thing, a discussion that is comfortable and palatable for the people in power is not going to be any good for the people who aren t in power. The Voting Aspect It seems to me that when certain issues are discussed we have a social process that is some form of vote. If one person complains then they are portrayed as crazy. When other people agree with the complaint then their comments are marginalised to try and preserve the narrative of one crazy person. It seems that in the case of the discussion about Rape Apology and LCA2011 most men who comment regard it as one person (either Valeria Aurora or Matthew Garrett) causing a dispute. There is even some commentary which references my blog post about Rape Apology [9] but somehow manages to ignore me when it comes to counting more than one person agreeing with Valerie. For reference David Zanetti was the first person to use the term apologist for rapists in connection with the LCA 2011 discussion [10]. So we have a count of at least three men already. These same patterns always happen so making a comment in support makes a difference. It doesn t have to be insightful, long, or well written, merely I agree and a link to a web page will help. Note that a blog post is much better than a comment in this regard, comments are much like conversation while a blog post is a stronger commitment to a position. I don t believe that the majority is necessarily correct. But an opinion which is supported by too small a minority isn t going to be considered much by most people. The Cost of Commenting The Internet is a hostile environment, when you comment on a contentious issue there will be people who demonstrate their disagreement in uncivilised and even criminal ways. S. E. Smith wrote an informative post for Tiger Beatdown about the terrorism that feminist bloggers face [11]. I believe that men face fewer threats than women when they write about such things and the threats are less credible. I don t believe that any of the men who have threatened me have the ability to carry out their threats but I expect that many women who receive such threats will consider them to be credible. The difference in the frequency and nature of the terrorism (and there is no other word for what S. E. Smith describes) experienced by men and women gives a vastly different cost to commenting. So when men fail to address issues related to the behavior of other men that isn t helping women in any way. It s imposing a significant cost on women for covering issues which could be addressed by men for minimal cost. It s interesting to note that there are men who consider themselves to be brave because they write things which will cause women to criticise them or even accuse them of misogyny. I think that the women who write about such issues even though they will receive threats of significant violence are the brave ones. Not Being Patronising Craig raises the issue of not being patronising, which is of course very important. I think that the first thing to do to avoid being perceived as patronising in a blog post is to cite adequate references. I ve spent a lot of time reading what women have written about such issues and cited the articles that seem most useful in describing the issues. I m sure that some women will disagree with my choice of references and some will disagree with some of my conclusions, but I think that most women will appreciate that I read what women write (it seems that most men don t). It seems to me that a significant part of feminism is about women not having men tell them what to do. So when men offer advice on how to go about feminist advocacy it s likely to be taken badly. It s not just that women don t want advice from men, but that advice from men is usually wrong. There are patterns in communication which mean that the effective strategies for women communicating with men are different from the effective strategies for men communicating with men (see my previous section on men not listening to women). Also there s a common trend of men offering simplistic advice on how to solve problems, one thing to keep in mind is that any problem which affects many people and is easy to solve has probably been solved a long time ago. Often when social issues are discussed there is some background in the life experience of the people involved. For example Rookie Mag has an article about the street harassment women face which includes many disturbing anecdotes (some of which concern primary school students) [12]. Obviously anyone who has lived through that sort of thing (which means most women) will instinctively understand some issues related to threatening sexual behavior that I can t easily understand even when I spend some time considering the matter. So there will be things which don t immediately appear to be serious problems to me but which are interpreted very differently by women. The non-patronising approach to such things is to accept the concerns women express as legitimate, to try to understand them, and not to argue about it. For example the issue that Valerie recently raised wasn t something that seemed significant when I first read the email in question, but I carefully considered it when I saw her posts explaining the issue and what she wrote makes sense to me. I don t think it s possible for a man to make a useful comment on any issue related to the treatment of women without consulting multiple women first. I suggest a pre-requisite for any man who wants to write any sort of long article about the treatment of women is to have conversations with multiple women who have relevant knowledge. I ve had some long discussions with more than a few women who are involved with the FOSS community. This has given me a reasonable understanding of some of the issues (I won t claim to be any sort of expert). I think that if you just go and imagine things about a group of people who have a significantly different life-experience then you will be wrong in many ways and often offensively wrong. Just reading isn t enough, you need to have conversations with multiple people so that they can point out the things you don t understand. This isn t any sort of comprehensive list of ways to avoid being patronising, but it s a few things which seem like common mistakes. Anne Onne wrote a detailed post advising men who want to comment on feminist blogs etc [13], most of it applies to any situation where men comment on women s issues.

A new upstream release of the Eigen C++ template library for linear algebra was released a few days ago. And Yixuan Qiu did some really nice work rolling this into a new RcppEigen released and then sent me a nice pull requent. The new version is now on CRAN, and I will prepare a Debian in a moment too. Upstream changes for Eigen are summarized in their changelog. On the RcppEigen side, Yixuan also rolled in some more changes on

as() and
wrap() converters as noted below in the NEWS entry.


Changes in RcppEigen version 0.3.2.2.0 (2014-08-19)

   Updated to version 3.2.2 of Eigen  
   Rcpp::as() now supports the conversion from R vector
  to  row array , i.e., Eigen::Array<T, 1,
  Eigen::Dynamic>   
   Rcpp::as() now supports the conversion from
  dgRMatrix (row oriented sparse matrices, defined in Matrix package) to Eigen::MappedSparseMatrix<T,
  Eigen::RowMajor>   
   Conversion from R matrix to Eigen::MatrixXd and
  Eigen::ArrayXXd using Rcpp::as() no longer gives
  compilation errors
  
  


Courtesy of CRANberries, there
are diffstat reports for the
most recent release.

Questions, comments etc about RcppEigen should go to the
rcpp-devel mailing list
off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

I still consider myself a newbie teacher. I'm just starting my fourth semester. And yes, I really enjoy it. Now, how did I come to teaching? Well, my training has been mostly on stages for different conferences. More technical, more social, whatever I have been giving ~10 talks a year for ~15 years, and I must have learnt something from that. Some good things, some bad habits. When giving presentations, a most usual technique is to prepare a set of slides to follow/support the ideas. And yes, that's what I did for my classes: Since my first semester, I prepared a nice set of slides, thematically split in 17 files, with ~30 to ~110 pages each (yes, huge variation). Given the course spans 32 classes (72 hours, 2 hours per class), each slide lasts for about two classes. But, yes, this tends to make the class much less dynamic, much more scripted, rigid, and... Boring. From my feedback, I understand the students don't think I am a bad teacher, but still, I want to improve! So, today I was to give the introduction to memory management. Easy topic, with few diagrams and numbers, mostly talking about the intuitive parts of a set of functions. I started scribbling and shortening the main points on a piece of paper (yes, the one on the picture). I am sure I can get down to more reduction But this does feel like an improvement! The class was quite successful. I didn't present the 100% of the material (which is one of the reasons I cling to my presentations I don't want to skip important material), and at some point I do feel I was a bit going in circles. However, Operating Systems is a very intuitive subject, and getting the students to sketch by themselves the answers that describe the working of real operating systems was a very pleasant experience! Of course, when I use my slides I do try to make it as interactive and collaborative as possible. But it is often unfeasible when I'm following a script. Today I was able to go around with the group's questions, find my way back to the outline I prepared. I don't think I'll completely abandon my slides, specially for some subjects which include many diagrams or pictures. But I'll try to have this alternative closer to my mind.

Search Results: "spa"

4 September 2014

Raphaël Hertzog: The problem of distributing applications

Russell Coker: Inteltech/Clicksend SMS Script

3 September 2014

Dirk Eddelbuettel: littler is faster at doing nothing!

2 September 2014

Antonio Terceiro: DebConf 14: Community, Debian CI, Ruby, Redmine, and Noosfero

1 September 2014

Juliana Louback: Debconf 2014 and How I Became a Debian Contributor

31 August 2014

Russell Coker: Links August 2014

30 August 2014

John Goerzen: 2AM to Seattle

29 August 2014

Antti-Juhani Kaijanaho: Licentiate Thesis is now publicly available

26 August 2014

Simon Josefsson: The Case for Short OpenPGP Key Validity Periods

25 August 2014

Steve Kemp: Updates on git-hosting and load-balancing

Erich Schubert: Analyzing Twitter - beware of spam

24 August 2014

Vincent Sanders: Without craftsmanship, inspiration is a mere reed shaken in the wind.

Lucas Nussbaum: on the Dark Ages of Free Software: a Free Service Definition ?

23 August 2014

Antti-Juhani Kaijanaho: A milestone toward a doctorate

Joachim Breitner: This blog goes static

Daniel Pocock: Want to be selected for Google Summer of Code 2015?

Steve Kemp: Updating Debian Administration, the code

22 August 2014

Russell Coker: Men Commenting on Women s Issues

21 August 2014

Dirk Eddelbuettel: RcppEigen 0.3.2.2.0

Gunnar Wolf: Walking without crutches