This blog, together with
Romain's, is one of the main homes of stories about how
Rcpp can help with getting code to run faster in the context of the
R system for statistical programming and analysis. By making it easier to get already existing C or C++ code
to R, or equally to extend R with new C++ code,
Rcpp can help in
getting stuff done.
And it is often fairly straightforward to do so.
In this context, I have a nice new example. And for once, it is work-related. I generally cannot share too much of what we do there as this is, well,
proprietary, but I have this nice new example. The other day, I was constructing (large) time series of implied volatilities. Implied volatilities
can be thought of as the complement to an option's price: given a price (and all other observables which can be thought of as fixed), we compute an
implied volatility price (typically via the standard
Black-Scholes model).
Given a changed implied volatility, we infer a new price -- see
this
Wikipedia page for more details. In essence, it opens the door to all sorts of arbitrage and relative value pricing adventures.
Now, we observe prices fairly frequently to create somewhat sizeable time series of option prices. And each price corresponds to one matching implied
volatility, and for each such price we have to solve a small and straightforward optimization problem: to compute the implied volatility given the
price. This is usually done with an iterative root finder.
The problem comes from the fact that we have to do this (i) over and over and over for large data sets, and (ii) that there are a number of callbacks
from the (generic) solver to the (standard) option pricer.
So our first approach was to just call the corresponding function
GBSVolatility from the
fOption package from the trusted
Rmetrics project by Diethelm Wuertz et al. This worked fine, but even with the usual tricks of splitting over
multiple cores/machines, it simply took too long for the resolution and data amount we desired. One of the problems is that this function (which
uses the proper
uniroot optimizer in R) is not inefficient per se, but simply makes to many function call back to the option pricer as
can be seen from a quick glance at the code. The helper function
.fGBSVolatility gets called time and time again:
R> GBSVolatility
function (price, TypeFlag = c("c", "p"), S, X, Time, r, b, tol = .Machine$double.eps,
maxiter = 10000)
TypeFlag = TypeFlag[1]
volatility = uniroot(.fGBSVolatility, interval = c(-10, 10),
price = price, TypeFlag = TypeFlag, S = S, X = X, Time = Time,
r = r, b = b, tol = tol, maxiter = maxiter)$root
volatility
<environment: namespace:fOptions>
R>
R> .fGBSVolatility
function (x, price, TypeFlag, S, X, Time, r, b, ...)
GBS = GBSOption(TypeFlag = TypeFlag, S = S, X = X, Time = Time,
r = r, b = b, sigma = x)@price
price - GBS
<environment: namespace:fOptions>
So the next idea was to try the corresponding function from my
RQuantLib package which
brings (parts of)
QuantLib to R. That was seen as been lots faster already. Now, QuantLib is pretty big and so
is RQuantLib, and we felt it may not make sense to install it on a number of machines just for this simple problem. So one evening this week I
noodled around for an hour or two and combined (i) a basic Black/Scholes calculation and (ii) a standard univariate zero finder (both of which can be
found or described in numerous places) to minimize the difference between the observed price and the price given an implied volatility. With about
one hundred lines in C++, I had something which felt fast enough. So today I hooked this into R via a two-line wrapper in quickly-created package using
Rcpp.
I had one more advantage here. For our time series problem, the majority of the parameters (strike, time to maturity, rate, ...) are fixed, so we can
structure the problem to be vectorised right from the start. I cannot share the code or more the details of my new implementation. However, both
GBSVolatility and
EuropeanOprionImpliedVolatility are on CRAN (and as I happen to maintain these for
Debian, also just one
sudo apt-get install r-cran-foptions r-cran-rquantlib away if you're
on
Debian or
Ubuntu). And writing the other solver is really not that
involved.
Anyway, here is the result, courtesy of a quick run via the
rbenchmark package. We create a vector of length
500; the implied volatility computation will be performed at each point (and yes, our time series are much longer indeed). This is replicated 100
times (as is the default for
rbenchmark) for each of the three approaches:
xyz@xxxxxxxx:~$ r xxxxR/packages/xxxxOptions/demo/timing.R
test replications elapsed relative user.self sys.self user.child sys.child
3 zzz(X) 100 0.038 1.000 0.040 0.000 0 0
2 RQL(X) 100 3.657 96.237 3.596 0.060 0 0
1 fOp(X) 100 448.060 11791.053 446.644 1.436 0 0
xyz@xxxxxxxx:~$
The new local solution is denoted by
zzz(X). It is already orders of magnitude faster than the
RQL(x) function using
RQuantLib (which is, I presume, due to my custom solution internalising the loop). And
the new approach is a laughable amount faster than the basic approach (shown as
fOp) via
fOptions. For one hundred replications of solving implied volatilities for
all elements of a vector of size 500, the slow solution takes about 7.5 minutes --- while the fast solution takes 38 milliseconds. Which comes to a
relative gain of over 11,000.
So sitting down with your C++ compiler to craft a quick one-hundred lines, combining two well-known and tested methods, can reap sizeable benefits. And
Rcpp makes it trivial to call this from
R.
Since October 2011 my flatmate and I where quite busy realizing a little pet project of ours called The Art of Asking. The ultimate goal is to visualize the world s opinion in an intuitive fashion and make it easy for everyone to play around with the data.
The idea behind The Art of Asking is that users submit interesting questions which are answered by users around the world. But instead of showing only the boring result, we want to provide interesting insights and statistics about the answers given.
For now the users can see the results of the question visualized by geographical regions. For example on the page for the question How are you today? you can see the interactive map with the pie chart. The map shows the average/dominating answer for each continent encoded by color, and the pie chart the distribution of the different answers for that region. If you move the mouse over a continent the pie chart updates and shows the distribution of answers for that continent. You can also click to zoom into the map to see the same for countries and regions. This allows you to investigate how the answers are distributed around the world.
This is already quite nice and fancy to play around with, but of course we want much more. Right now we re working on a feature which will users allow to combine two arbitrary questions and see how the answers are related. This doesn t sound like much, but it is very addictive to crawl through the list of questions and find interesting correlations. Here is a little plot how it could look like (the data is from the actual data base of answers).
For each possible combination of answers it shows the percentage of people who answered in that combination. Of course those plots only make sense when enough users answered for both questions you want to compare which is right now not very often the case, so I guess we ll roll out that feature sometime later when we have more data. But we have lots of ideas and are working on further ways to investigate the answers. Obviously one low hanging fruit for example would be the distribution of the answers over time.
We ve been working for over a year now on this project in our spare time and we built it more or less from scratch. I wrote a small WSGI framework in Python and on top of that the WSGI application which runs the site. We use MongoDB for the storage of the data, uWSGI and nginx for the server, Jinja2 and Bootstrap for the HTML and D3.js for the visualization of the data, where Maci did a wonderful job realizing the interactive map and charts.
We re running this site since July 2012 now and are already quite satisfied with the number of users, and the quality of the questions. But of course we could always use more (especially more answers). So if you want to try it out, go to theartofasking.com and fill out the blank spots on the map! We re happy about every answer and question we can get and are eager to hear your suggestions.
While digging Debian Qt-KDE's bugs, it has become more common to find people reporting that their KDE is very slow. Most of the time (if not every time) it turns out that the loopback interface is down. Sometimes the user did something wrong, sometimes some network manager, but the end result is the same: KDE is terribly slow.
I've written rather a lot about how we're implementing our Secure Boot support, but there's been a range of subtle changes over time. Here's an overview of how it all fits together.Secure Boot and signingSecure Boot is part of the UEFI specification. When enabled, systems will refuse to boot any UEFI executables (such as an OS bootloader) unless they've been signed by a trusted key. The only trusted key effectively guaranteed to be present is Microsoft's, and Microsoft will sign binaries if you have access to the Windows hardware dev center. You gain access to this by purchasing a certificate from Verisign that verifies your identity and then use that to create a developer account at the 
A friend of mine recently reminded me of the 
PS: yes, I'm talking here about GitHub, which is not Open Source
software itself. I regard that as a (temporary) inconvenience, which
dies not detract from the significant advantage it brings to the
table.
[1] 
I've just finished deploying automatic installability checking for Ubuntu's development release, which is more or less equivalent to the way that uploads are promoted from Debian unstable to testing. See 
I've had a few weeks with very little time spent on Debian. I did set aside
some time to try to find and fix an RC bug, to help the release. Steven C
commented to let me know about the 
as zack has 
To experiment with network stuff, I was using
# Copyright (C) YEAR Free Software Foundation, Inc.
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR <email>, YEAR.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"POT-Creation-Date: 2012-10-12 11:14+0300\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <email>\n"
"Language-Team: LANGUAGE <ll>\n"
"Language: \n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
#. type: Title #
#, no-wrap
msgid "Status hos doktoren"
msgstr ""
#. type: Plain text
#, no-wrap
msgid "\n"
msgstr ""
#. type: Plain text
#, no-wrap
msgid "\n"
msgstr ""
#. type: Plain text
#, no-wrap
msgid ""
"**Jonas Smedegaard** [dr.jones at pobox.com "
"](mailto:friends%40jones.dk?Subject=Status%20hos%20doktoren&In-Reply-To=1.5.4.32.19971011010004.0067de34%40kaospilot.dk) "
"\n"
msgstr ""
#. type: Plain text
msgid "_Sat Oct 11 15:22:00 CEST 1997_"
msgstr ""
#. type: Bullet: ' * '
#:
msgid "Previous message: [Spamming? ](000001.html)"
msgstr ""
#. type: Bullet: ' * '
#:
msgid ""
"**Messages sorted by:**