Hadley is a
popular figure, and rightly so as he successfully introduced many newcomers to the wonders offered by
R. His approach strikes some of us old greybeards as wrong---I particularly take exception with some of his writing which frequently portrays a particular approach as both
the best
and only one. Real programming, I think, is often a little more nuanced and aware of tradeoffs which need to be balanced. As a book on another language once popularized: "There is more than one way to do things." But let us leave this discussion for another time.
As the reach of the
Hadleyverse keeps
spreading, we sometimes find ourselves at the receiving end of a cost/benefit tradeoff. That is what this post is about, and it uses a very concrete case I encountered yesterday.
As
blogged earlier, the
RcppZiggurat package was updated. I had not touched it in a year, but
Brian Ripley had sent a brief and detailed note concerning something flagged by the Solaris compiler (correctly suggesting I replace
fabs() with
abs() on integer types). (Allow me to stray from the main story line here for a second to stress just how insane a work load he is carrying, essentially for all of us. R and the R community are so just so indebted to him for all his work---which makes the usual social media banter about him so unfortunate. But that too shall be left for another time.) Upon making the simple fix, and submitting to
GitHub the usual
Travis CI was triggered. And here is what I saw:
All happy, all green. Previous build a year ago, most recent build yesterday, both passed. But hold on: test time went from 2:54 minutes to 7:47 minutes for an increase of almost five minutes! And I knew that I had not added any new dependencies, or altered any build options. What did happen was that among the dependencies of my package, one had decided to now also depend on
ggplot2. Which leads to a chain of sixteen additional packages being loaded besides the four I depend upon---when it used to be just one. And that took five minutes as all those packages are installed from source, and some are big and take a long time to compile.
There is however and easy alternative, and for that we have to praise
Michael Rutter who looks after a number of things for R on Ubuntu. Among these are the
R builds for Ubuntu but also the
rrutter PPA as well as the
c2d4u PPA. If you have not heard this alphabet soup before, a
PPA is a package repository for Ubuntu where anyone (who wants to sign up) can upload (properly setup) source files which are then turned into Ubuntu binaries. With full dependency resolution and all other goodies we have come to expect from the Debian / Ubuntu universe. And Michael uses this facility with great skill and calm to provide us all with Ubuntu binaries for R itself (rebuilding what yours truly uploads into Debian), as well as a number of key packages available via the CRAN mirrors. Less know however is this "c2d4u" which stands for
CRAN to Debian for Ubuntu. And this builds on something
Charles Blundell once built under my mentorship in a
Google Summer of Code. And Michael does a tremdous job covering well over a thousand
CRAN source packages---and providing binaries for all. Which we can use for Travis!
What all that means is that I could now replace the line
- ./travis-tool.sh install_r RcppGSL rbenchmark microbenchmark highlight
which implies
source builds of the four listed packages and all their dependencies with the following line implying
binary installations of already built packages:
- ./travis-tool.sh install_aptget libgsl0-dev r-cran-rcppgsl r-cran-rbenchmark r-cran-microbenchmark r-cran-highlight
In this particular case I also needed to build a binary package of my
RcppGSL package as this one is not (yet) handled by Michael. I happen to have (re-)discovered the beauty of PPAs for Travis earlier this year and revitalized an older and largely dormant
launchpad account I had for
this PPA of mine. How to build a simple .deb package will also have to left for a future post to keep this more concise.
This can be used with the existing
r-travis setup---but one needs to use the older, initial variant in order to have the ability to install .deb packages. So in the
.travis.yml of RcppZiggurat I just use
before_install:
## PPA for Rcpp and some other packages
- sudo add-apt-repository -y ppa:edd/misc
## r-travis by Craig Citro et al
- curl -OL http://raw.github.com/craigcitro/r-travis/master/scripts/travis-tool.sh
- chmod 755 ./travis-tool.sh
- ./travis-tool.sh bootstrap
to add my own PPA and all is good. If you do not have a PPA, or do not want to create your own packages you can still benefit from the PPAs by Michael and "mix and match" by installing from binary what is available, and from source what is not.
Here we were able to use an all-binary approach, so let's see the resulting performance:
Now we are at 1:03 to 1:15 minutes---much better.
So to conclude, while the every expanding universe of R packages is fantastic for us as
users, it can be seen to be placing a burden on us as
developers when installing and testing. Fortunately, the packaging infrastructure built on top of Debian / Ubuntu packages can help and
dramatically reduce build (and hence test) times. Learning about PPAs can be a helpful complement to learning about Travis and continued integration. So maybe now I need a new reason to blame
Hadley? Well, there is always
snake case ...
Follow-up: The post got some pretty immediate feedback shortly after I posted it. Craig Citro pointed out (quite correctly) that I could use r_binary_install which would also install the Ubuntu binaries based on their R packages names. Having built R/CRAN packages for Debian for so long, I am simply more used to the r-cran-* notations, and I think I was also the one contributing install_aptget to r-travis ... Yihui Xie spoke up for the "new" Travis approach deploying containers, caching of packages and explicit whitelists. It was in that very (GH-based) discussion that I started to really lose faith in the new Travis approach as they want use to whitelist each and every package. With 6900 and counting at CRAN I fear this simply does not scale. But different approaches are certainly welcome. I posted my 1:03 to 1:15 minutes result. If the "New School" can do it faster, I'd be all ears.
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.
There is something about that moment. Hiking in the mountains near Durango, Colorado, with Laura and the boys, we found a beautiful spot with a view of the valley. We paused to admire, and then
The sound of a steam locomotive whistle from down below, sounding loud all the way up there, then echoing back and forth through the valley. Then the quieter, seemingly more distant sound of the steam engine heading across the valley, chugging and clacking as it goes. More whistles, the sight of smoke and then of the train full of people, looking like a beautiful model train from our vantage point.
I ve heard that sound on a few rare recordings, but never experienced it. I ve been on steam trains a few times, but never spent time in a town where they still run all day, every day. It is a different sort of feeling to spend a week in a place where Jacob and Oliver would jump up several times a day and rush to the nearest window in an attempt to catch sight of the train.
Airplanes really can be a time machine in a sense what a wondrous time to be alive, when things so ancient are within the reach of so many. I have been transported to L beck and felt the uneven 700-year-old stones of the Marienkirche underneath my feet, feeling a connection to the people that walked those floors for centuries. I felt the same in Prague, in St. George s Basilica, built in 1142, and at the Acropolis of Lindos, with its ancient Greek temple ruins. In Kansas, I feel that when in the middle of the Flint Hills rolling green hills underneath the pure blue sky with billowing white clouds, the sounds of crickets, frogs, and cicadas in my ears; the sights and sounds are pretty much as they ve been for tens of thousands of years. And, of course, in Durango, arriving on a plane but seeing the steam train a few minutes later.
It was fitting that we were in Durango with Laura s parents to celebrate their 50th anniversary. As we looked forward to riding the train, we heard their stories of visits to Durango years ago, of their memories of days when steam trains were common. We enjoyed thinking about what our lives would be like should we live long enough to celebrate 50 years of marriage. Perhaps we would still be in good enough health to be able to ride a steam train in Durango, telling about that time when we rode the train, which by then will have been pretty much the same for 183 years. Or perhaps we would take them to our creek, enjoying a meal at the campfire like I ve done since I was a child.
Each time has its unique character. I am grateful for the cameras and airplanes and air conditioning we have today. But I am also thankful for those things that connect us with each other trough time, those rocks that are the same every year, those places that remind us how close we really are to those that came before.
I have just released version 1.13 of Obnam, my backup program. See the
website at 
VLANd is a python program intended to make it easy to manage
port-based VLAN setups across multiple switches in a network. It is
designed to be vendor-agnostic, with a clean pluggable driver API to
allow for a wide range of different switches to be controlled
together.
There's more information in
the 
My monthly report covers a large part of what I have been doing in the free software world. I write it for 
The 
DebConf15 Schedule
The DebConf content team is pleased to announce the schedule of
DebConf15, the forthcoming Debian Developers Conference. From a total of
nearly 100 talk submissions, the team selected 75 talks. Due to the high
number of submissions, several talks had to be shortened to 20 minute
slots, of which a total of 30 talks have made it to the schedule.
In addition, around 50 meetings and discussions (BoFs) have been
submitted so far, as well as several other events like lightning talk
sessions, live demos, a movie screening, a poetry night or stand-up
comedy.
The Schedule is 
So we're in Finland, and the differences start out immediately.
We're renting a flat, in building ten, on a street. You'd think "10
Streetname" was a single building, but no. It is a pair of buildings: 10A, and 10B.
Both of the buildings have 12 flats in them, with 10A having 1-12, and 10B having 13-24.
There's a keypad at the main entrance, which I assumed was to let you press a button and talk to the people inside "Hello I'm the postmaster", but no. There is no intercom system, instead you type in a magic number and the door opens.
The magic number? Sounds like you want to keep that secret, since it lets people into the common-area? No. Everybody has it. The postman, the cleaners, the DHL delivery man, and all the ex-tenants. We invited somebody over recently and gave it out in advance so that they could knock on our flat-door.
Talking of cleaners: In the UK I lived in a flat and once a fortnight somebody would come and sweep the stair-well, since we didn't ever agree to do it ourselves. Here somebody turns up every day, be it to cut the grass, polish the hand-rail, clean the glass on the front-door, or mop the floors of the common area. Sounds awesome. But they cut the grass, right outside our window, at 7:30AM. On the dot. (Or use a leaf-blower, or something equally noisy.)
All this communal-care is paid for by the building-association, of which all flat-owners own shares. Sounds like something we see in England, or even like Americas idea of a Home-Owners-Association. (In Scotland you own your own flat, you don't own shares of an entity which owns the complete building. I guess there are pros and cons to both approaches.)
Moving onwards other things are often the same, but the differences when you spot them are odd. I'm struggling to think of them right now, somebody woke me up by cutting our grass for the second time this week (!)
Anyway I'm registered now with the Finnish government, and have a citizen-number, which will be useful, I've got an appointment booked to register with the police - which is something I had to do as a foreigner within the first three months - and today I've got an appointment with a local bank so that I can have a euro-bank-account.
Happily I did find a gym to join, the owner came over one Sunday to
give me a tiny-tour, and then gave me a list of other gyms to try if
his wasn't good enough - which was a nice touch - I joined a couple of
days later, his gym is awesome.
(I'm getting paid in UK-pounds, to a UK-bank, so right now I'm getting local money by transferring to my wifes account here, but I want to do that to my own, and open a shared account for paying for rent, electricity, internet, water, & etc).
My flat back home is still not rented, because the nice property management company lost my keys. Yeah you can't make that up can you? With a bit of luck the second set of keys I mailed them will arrive soon and the damn thing can be occupied, while I'm not relying on that income I do wish to have it.
Dear Debian/KDE users,
Long long time ago I wanted to have 
I have already updated the packaging for the latest version 0.8.7, which will be included in Debian/sid rather soon. Thanks again especially Mattia for his great work.
What happened in the 
Some time ago we've decided to move 