Search Results: "xam"

8 December 2012

Dirk Eddelbuettel: Rcpp attributes: Even easier integration of GSL code into R

Following the Rcpp 0.10.0 release, I had written about simulating pi easily by using the wonderful new Rcpp Attributes feature. Now with Rcpp 0.10.1 released a good week ago, it is time to look at how Rcpp Attributes can help with external libraries. As this posts aims to show, it is a breeze! One key aspect is the use of the plugins for the inline package. They provide something akin to a callback mechanism so that compilation and linking steps can be informed about header and library locations and names. We are going to illustrate this with an example from GNU Scientific Library (GSL). The example I picked uses B-Spline estimation from the GSL. This is a little redundant as R has its own spline routines and package, but serves well as a simple illustration---and, by reproducing an existing example, followed an established path. So we will look at Section 39.7 of the GSL manual which has a complete example as a standalone C program, generating both the data and the fit via cubic B-splines. We can decompose this two parts: data generation, and fitting. We will provide one function each, and then use both from R. These two function will follow the aforementioned example from Section 39.7 somewhat closely. We start with the first function to generate the data.
// [[Rcpp::depends(RcppGSL)]]
#include <RcppGSL.h>
#include <gsl/gsl_bspline.h>
#include <gsl/gsl_multifit.h>
#include <gsl/gsl_rng.h>
#include <gsl/gsl_randist.h>
#include <gsl/gsl_statistics.h>
const int N = 200;                              // number of data points to fit 
const int NCOEFFS = 12;                         // number of fit coefficients */
const int NBREAK = (NCOEFFS - 2);               // nbreak = ncoeffs + 2 - k = ncoeffs - 2 since k = 4 */
// [[Rcpp::export]]
Rcpp::List genData()  
    const size_t n = N;
    size_t i;
    double dy;
    gsl_rng *r;
    RcppGSL::vector<double> w(n), x(n), y(n);
    gsl_rng_env_setup();
    r = gsl_rng_alloc(gsl_rng_default);
    //printf("#m=0,S=0\n");
    /* this is the data to be fitted */
    for (i = 0; i < n; ++i)  
        double sigma;
        double xi = (15.0 / (N - 1)) * i;
        double yi = cos(xi) * exp(-0.1 * xi);
        sigma = 0.1 * yi;
        dy = gsl_ran_gaussian(r, sigma);
        yi += dy;
        gsl_vector_set(x, i, xi);
        gsl_vector_set(y, i, yi);
        gsl_vector_set(w, i, 1.0 / (sigma * sigma));
                
        //printf("%f %f\n", xi, yi);
     
    Rcpp::DataFrame res = Rcpp::DataFrame::create(Rcpp::Named("x") = x,
                                                  Rcpp::Named("y") = y,
                                                  Rcpp::Named("w") = w);
    x.free();
    y.free();
    w.free();
    gsl_rng_free(r);
    return(res);
 
We include a few header files, define (in what is common for C programs) a few constants and then define a single function genData() which returns and Rcpp::List as a list object to R. A primary importance here are the two attributes: one to declare a dependence on the RcppGSL package, and one to declare the export of the data generator function. That is all it takes! The plugin of the RcppGSL will provide information about the headers and library, and Rcpp Attributes will do the rest. The core of the function is fairly self-explanatory, and closely follows the original example. Space gets allocated, the RNG is setup and a simple functional form generates some data plus noise (see below). In the original, the data is written to the standard output; here we return it to R as three columns in a data.frame object familiar to R users. We then free the GSL vectors; this manual step is needed as they are implemented as C vectors which do not have a destructor. Next, we can turn the fitting function.
// [[Rcpp::export]]
Rcpp::List fitData(Rcpp::DataFrame ds)  
    const size_t ncoeffs = NCOEFFS;
    const size_t nbreak = NBREAK;
    const size_t n = N;
    size_t i, j;
    Rcpp::DataFrame D(ds);              // construct the data.frame object
    RcppGSL::vector<double> y = D["y"]; // access columns by name, 
    RcppGSL::vector<double> x = D["x"]; // assigning to GSL vectors
    RcppGSL::vector<double> w = D["w"];
    gsl_bspline_workspace *bw;
    gsl_vector *B;
    gsl_vector *c; 
    gsl_matrix *X, *cov;
    gsl_multifit_linear_workspace *mw;
    double chisq, Rsq, dof, tss;
    bw = gsl_bspline_alloc(4, nbreak);      // allocate a cubic bspline workspace (k = 4)
    B = gsl_vector_alloc(ncoeffs);
    X = gsl_matrix_alloc(n, ncoeffs);
    c = gsl_vector_alloc(ncoeffs);
    cov = gsl_matrix_alloc(ncoeffs, ncoeffs);
    mw = gsl_multifit_linear_alloc(n, ncoeffs);
    gsl_bspline_knots_uniform(0.0, 15.0, bw);   // use uniform breakpoints on [0, 15] 
    for (i = 0; i < n; ++i)                     // construct the fit matrix X 
        double xi = gsl_vector_get(x, i);
        gsl_bspline_eval(xi, B, bw);            // compute B_j(xi) for all j 
        for (j = 0; j < ncoeffs; ++j)           // fill in row i of X 
            double Bj = gsl_vector_get(B, j);
            gsl_matrix_set(X, i, j, Bj);
         
     
    gsl_multifit_wlinear(X, w, y, c, cov, &chisq, mw);  // do the fit 
    
    dof = n - ncoeffs;
    tss = gsl_stats_wtss(w->data, 1, y->data, 1, y->size);
    Rsq = 1.0 - chisq / tss;
    
    Rcpp::NumericVector FX(151), FY(151);       // output the smoothed curve 
    double xi, yi, yerr;
    for (xi = 0.0, i=0; xi < 15.0; xi += 0.1, i++)  
        gsl_bspline_eval(xi, B, bw);
        gsl_multifit_linear_est(B, c, cov, &yi, &yerr);
        FX[i] = xi;
        FY[i] = yi;
     
    Rcpp::List res =
      Rcpp::List::create(Rcpp::Named("X")=FX,
                         Rcpp::Named("Y")=FY,
                         Rcpp::Named("chisqdof")=Rcpp::wrap(chisq/dof),
                         Rcpp::Named("rsq")=Rcpp::wrap(Rsq));
    gsl_bspline_free(bw);
    gsl_vector_free(B);
    gsl_matrix_free(X);
    gsl_vector_free(c);
    gsl_matrix_free(cov);
    gsl_multifit_linear_free(mw);
    
    y.free();
    x.free();
    w.free();
    return(res);   
 
The second function closely follows the second part of the GSL example and, given the input data, fits the output data. Data structures are setup, the spline basis is created, data is fit and then the fit is evaluated at a number of points. These two vectors are returned along with two goodness of fit measures. We only need to load the Rcpp package and source a file containing the two snippets shown above, and we are ready to deploy this:
library(Rcpp)
sourceCpp("bSpline.cpp")                # compile two functions
dat <- genData()                        # generate the data
fit <- fitData(dat)                     # fit the model, returns matrix and gof measures
And with that, we generate a chart such as Spline fitting example from GSL manual redone with Rcpp Attributes via a simple four lines, or as much as it took to create the C++ functions, generate the data and fit it!
op <- par(mar=c(3,3,1,1))
plot(dat[,"x"], dat[,"y"], pch=19, col="#00000044")
lines(fit[[1]], fit[[2]], col="orange", lwd=2)
par(op)
The RcppArmadillo and RcppEigen package support plugin use in the same way. Add an attribute to export a function, and an attribute for the depends -- and you're done. Extending R with (potentially much faster) C++ code has never been easier, and opens a whole new set of doors.

Russ Allbery: Consent: Do you speak it?

Yes, humor is a matter of taste. Does this mean that practical jokes, pranks, and social engineering are just a matter of taste? No. What's different, and what's missing, is consent. A key characteristic of practical jokes is that they involve a third party in the joke without getting specific prior consent because to get prior consent would ruin the joke. There are situations in which that's fine: if the people involved know each other well, they can give implicit consent to that sort of joke, for example. If the victim isn't known to the pranksters, they are outside of the set of situations in which this is okay. Consent is something that human society in general seems to be struggling with right now. Consent to sex. Consent to have one's personal information used by a corporation. Consent to have one's pictures posted on-line. And in every case, I think the default position should be the same as the gold standard for sex: enthusiastic consent. If you want to make someone else part of your business, your art work, your joke, your sex act, or anything else that happens outside of your head in a way that affects them, the required step is very simple: you need to get their enthusiastic consent. If that's too much trouble for you, if that makes it less fun for you, if that means you can't make money, if you think your great idea overrides their rights over their own life, body, ideas, information, or participation, then you fail. That simple. If you want to use someone else, you have to ask first. It's not a hard concept. One might even say that it's basic politeness. Now, the role of governments in this is limited and often not practical. For example, I don't think one can write laws about radio performers who like prank calling, at least without causing so many negative side effects that the effort isn't really practical. Doing that doesn't make you a criminal. It just makes you an asshole. Also, doing things to other people without their consent doesn't mean that you're then morally responsible for anything that subsequently happens that may or may not be related. But you're still an asshole. And if your employer decides that you being an asshole as part of your job has suddenly stopped being funny and fires you, hey, works for me.

7 December 2012

Richard Hartmann: Release Critical Bug report for Week 49

The UDD bugs interface currently knows about the following release critical bugs: How do we compare to the squeeze cycle ?
Week Squeeze Wheezy Diff
43 284 (213+71) 468 (332+136) +184 (+119/+65)
44 261 (201+60) 408 (265+143) +147 (+64/+83)
45 261 (205+56) 425 (291+134) +164 (+86/+78)
46 271 (200+71) 401 (258+143) +130 (+58/+72)
47 283 (209+74) 366 (221+145) +83 (+12/+71)
48 256 (177+79) 378 (230+148) +122 (+53/+69)
49 256 (180+76) 360 (216+155) +104 (+36/+79)
50 204 (148+56)
51 124 (124+54)
52 115 (78+37)
1 93 (60+33)
2 82 (46+36)
3 25 (15+10)
4 14 (8+6)
5 2 (0+2)
6 release!
Graphical overview of bug stats thanks to azhag:

Joachim Breitner: Calculating the internal rate of return with hledger

For more than ten years now, I am a regular and happy user of GnuCash, the free accountancy program. Especially the integrated online banking support for HBCI that allows me not only to fetch my transactions, but also to transfer money using a cryptographic makes it a great tool. Nevertheless, it has shortcomings, especially when it comes to reports. And for some reason I never felt the urge to hack on that C/Guile code (although I recently hacked a bit on the UI).
Joey Hess blog post about Simon Michael s hledger made me look at that command line based accountancy tool. It certainly won t replace Gnucash for me, but luckily ledger, of which hledger is a re-implementation, can read GnuCash s file format and convert it to a ledger file that hledger can read as well. So now I can do analysis on my financial data in Haskell what would you want more? One report that I was missing was the internal rate of investment of some account. This is the annual rate of a hypothetical fixed interest-bearing savings account that would, given the same deposits and payouts, result in the same final value. I now implemented this as the hledger-irr command and as you would expect you can find it on hackage. See the package description for documentation and example output. It is very fresh, hardly tested code, so don t base important decisions on it yet. Also, it needs work to work correctly with multiple currencies or things with varying prices, and I am not yet sure what to do there. Update: Simon asked me to elaborate the difference to hledger-interest by Peter Simons (on which my code was initial based, but eventually I rewrote everything but the option parsing code). The difference is that hledger-interest answers the question I have this investment with this yearly interest rate (which may even change), how large are the interest payments going to be. I imagine a usecase is that you have borrowed some money to a friend at a certain (surely amicable) rate and need to figure out what he owes you, even when you gave him the money mid year and he payed you some back shortly after easter. hledger-irr serves a different purpose: Here you already know your investment payments, gains and losses and you want to have a number describing how good your investment was. One usecase might be that, at the end of a live of paying into a pension fund, you ask yourself: Given all the feeds and commissions, was this a good idea or should I just have invested the money myself? Then the internal rate of investment tells you how lucrative an alternative fixed-rate investment would have had to been to match your pension fund investment. (Or course this is not the only aspect that you should look at when comparing investments, as it totally ignores risk and possible insurance elements of the investment.)

Dominique Dumont: How to fix a configuration model

Hello I ve written a tutorial to show how a configuration model can be updated to cope with new features provided by new software release. This tutorial shows how to add a new parameter provided by OpenSsh 5.9 into Ssh model. This example can be applied to other models (like Dpkg model). I hope that people will be able now to perform simple enhancements to current models. Feedback are welcome
Tagged: config-model, OpenSsh, Perl

Christoph Berg: apt.postgresql.org

So we finally made it, and sent out an official announcement for apt.postgresql.org. This new repository hosts packages for all PostgreSQL server versions (at the moment 8.3, 8.4, 9.0, 9.1, 9.2) for several Debian/Ubuntu distributions (squeeze, wheezy, sid, precise) on two architectures (amd64, i386). Now add packages for extension modules on top of all these, and you get a really large amount of binaries from a small number of sources. Right now there's 1670 .deb files and 148 .dsc files, but the .dsc count includes variants that only differ in the version number per distribution (we attach .pgdg60+1 for squeeze packages, .pgdg70+1 for wheezy and so on), so the real number of different sources is rather something like 81, with 38 distinct source package names. Dimitri Fontaine, Magnus Hagander, and I have been working on this since I first presented the idea at PGconf.EU 2011 in Amsterdam. We now have a Jenkins server building all the packages, an archive server with the master repository, and a feed that syncs the repository to the postgresql.org FTP (well, mostly http) server. If you were previously using pgapt.debian.net, that's the same archive as on apt.postgresql.org (one rsync away). Please update your sources.list to point to apt.postgresql.org, I'll shut down the archive at that location at the end of January. Here's the Quickstart instructions from the Wiki page: Import the repository key from http://apt.postgresql.org/pub/repos/apt/ACCC4CF8.asc:
wget -O - http://apt.postgresql.org/pub/repos/apt/ACCC4CF8.asc   sudo apt-key add -
Edit /etc/apt/sources.list.d/pgdg.list. The distributions are called codename-pgdg. In the example, replace squeeze with the actual distribution you are using:
deb http://apt.postgresql.org/pub/repos/apt/ squeeze-pgdg main
Configure apt's package pinning to prefer the PGDG packages over the Debian ones in /etc/apt/preferences.d/pgdg.pref:
Package: *
Pin: release o=apt.postgresql.org
Pin-Priority: 500
Note: this will replace all your Debian/Ubuntu packages with available packages from the PGDG repository. If you do not want this, skip this step. Update the package lists, and install the pgdg-keyring package to automatically get repository key updates:
apt-get update
apt-get install pgdg-keyring

6 December 2012

Steve Kemp: December 2012 Software Updates

Some brief software updates:
Custodian
This is the monitoring tool I wrote for Bytemark. It still rocks, and has run over 10 million tests without failure.I'd love more outside feedback, even if just to say "documentation needs work".
Slaughter
This is my sysadmin tool for multiple hosts - consider it cfengine-lite, or cfengine-trivial more likely. The 2.x release is finally out, and features pluggable transports. That means your central server can be running HTTP, RSYNC, FTP, or anything you like. 90% of the changes came from or were inspired by Csillag Tamas, to whom I owe a debt of thanks.
Templer
A static-site generator, written in Perl. I use this to generate blogspam.net, and other sites from simple layouts. Tutorial available online.
redis-document-store
A trivial hack which allows using Redis as a schema-less document storage system. Assuming you never delete documents it is simple, transparent, and already in live use for Debian Administration
Random Comment on Templer:
Although I've made extensive notes on common static site generators, and they will be discussed at length in the near future, I do want to highlight one problem common to 90% of them: Symbolic links. For example webgen fails my simple test:
~/hg/websites$ webgen create test.example.com
~/hg/websites$ cd test.example.com/src/
~/hg/websites/test.example.com/src$ mkdir jquery-1.2.3
~/hg/websites/test.example.com/src$ touch jquery-1.2.3/jquery.js
~/hg/websites/test.example.com/src$ ln -s jquery-1.2.3 jquery
~/hg/websites/test.example.com$ webgen
Starting webgen...
...
Finished
~/hg/websites/test.example.com$ ls out/    grep jq
jquery-1.2.3
Here we see creating a symlink to a directory has not produced a matching symlink in the output. Something I use frequently. for example. Some tools mangled symlinked directories, or files, some ignore them completely. Neither is acceptible.

Olivier Berger: A 30 minutes introduction to git

I ve been looking for a set of slides that I could have reused to make a 30 minutes introduction to my colleagues at the computer science department. Finding none that would be suitable (i.e. including graphical examples and not only being the canvas for a workshop), I tried to proceed with making one of my own. I ve reused the document git concepts simplified, whose content seemed quite good, and have converted it to a set of slides. Here s the result, which has been reworked a bit vs. the original git concepts simplified : <iframe allowfullscreen="allowfullscreen" frameborder="0" height="356" marginheight="0" marginwidth="0" mozallowfullscreen="mozallowfullscreen" scrolling="no" src="http://www.slideshare.net/slideshow/embed_code/15517872" style="border:1px solid #CCC;border-width:1px 1px 0;margin-bottom:5px" webkitallowfullscreen="webkitallowfullscreen" width="427"> </iframe>
An introduction to git from olberger
Here s the PDF version. Note that if you prefer a version that aligns more with the original, here s also an older version, in : https://github.com/olberger/git-notes/tree/master/gcs Update: For the lone reader that cannot attend my presentation, I should recommend to read first the HTML version by Sitaram at git concepts simplified, as the slides miss some details (which I ve kept in the beamer notes only), then only refer to my slides in a second time, for some additions. If you want the org-mode source (containing the dot source of the diagrams), contact me. I ll maybe upload the source into some Git repo whan I have enough demand/time ;)

Petter Reinholdtsen: Scripting the Cerebrum/bofhd user administration system using XML-RPC

Where I work at the University of Oslo, we use the Cerebrum user administration system to maintain users, groups, DNS, DHCP, etc. I've known since the system was written that the server is providing an XML-RPC API, but I have never spent time to try to figure out how to use it, as we always use the bofh command line client at work. Until today. I want to script the updating of DNS and DHCP to make it easier to set up virtual machines. Here are a few notes on how to use it with Python. I started by looking at the source of the Java bofh client, to figure out how it connected to the API server. I also googled for python examples on how to use XML-RPC, and found a simple example in the XML-RPC howto. This simple example code show how to connect, get the list of commands (as a JSON dump), and how to get the information about the user currently logged in:
#!/usr/bin/env python
import getpass
import xmlrpclib
server_url = 'https://cerebrum-uio.uio.no:8000';
username = getpass.getuser()
password = getpass.getpass()
server = xmlrpclib.Server(server_url);
#print server.get_commands(sessionid)
sessionid = server.login(username, password)
print server.run_command(sessionid, "user_info", username)
result = server.logout(sessionid)
print result
Armed with this knowledge I can now move forward and script the DNS and DHCP updates I wanted to do.

James Bromberger: Official Debian Images on Amazon Web Services EC2

Please Note: this article is written from my personal perspective as a Debian Developer, and is not the opinion or expression of my employer.
Amazon Web Service s EC2 offers customers a number of Operating Systems to run. There are many Linux Distributions available, however for all this time, there has never been an Official Debian Image or Amazon Machine Image (AMI), created by Debian. For some Debian users this has not been an issue as there are several solutions of creating your own personal AMI. However for the AWS Users who wanted to run a recognised image, it has been a little confusing at times; several Debian AIMs have been made available by other customers, but the source of those images has not been Debian . In October 2012 the AWS Marketplace engaged in discussions with the Debian Project Leader, Stefano Zacchiroli. A group of Debian Developers and the wider community formed to generated a set of AMIs using Anders Ingemann s ec2debian-build-ami script. These AMIs are published in the AWS Marketplace, and you can find the listing here: No fees are collected for Debian for the use of these images via the AWS Marketplace; they are listed here for your convenience. This is the same AMI that you may generate yourself, but this one has been put together by Debian Developers. If you plan to use this AMI, I suggest you read http://wiki.debian.org/Cloud/AmazonEC2Image, and more explicity, SSH as the user admin and then sudo -i to root. Additional details Anders Ingemann and others maintain a GitHub project called ec2debian-build-ami which generates a Debian AMI. This script supports several desired features, an was also updated to add in some new requirements. This means the generated image supports: Debian Stable (Squeeze; 6.0.6 at this point in time) does not contain the cloud-init package, and neither does Debian Testing (Wheezy). A fresh AWS account (ID 379101102735) was used for the initial generation of this image. Any Debian Developer who would like access is welcome to contact me. Minimal charges for the resource utilisation of this account (storage, some EC2 instances for testing) are being absorbed by Amazon for this. Co-ordination of this effort is held on the debian-cloud mailing list. The current Debian stable is 6.0.6 Squeeze , and we re in deep freeze for the Wheezy release. Squeeze has a Xen kernel that works on the Parallel Virtual Machine (PVM) EC2 instance, and hence this is what we support on EC2. (HVM images are a next phase, being headed up by Yasuhiro Akarki <ar@d.o>). Marketplace Listing The process of listing in the AWS Marketplace was conducted as follows: This image went out on the 19th of November 2012. Additional documentation was put into the Wiki at: http://wiki.debian.org/Cloud/AmazonEC2Image/Squeeze A CloudFormation template may help you launch a Debian instance by containing a mapping to the relevent AMI in the region you re using: see the wiki link above. What s Next The goal is to continue stable releases as they come out. Further work is happening to support generation of Wheezy images, and HVM (which may all collapse into one effort with a Linux 3.x kernel in Wheezy). If you re a Debian Developer and would like a login to the AWS account we ve been using, then please drop me a line. Further work to improve this process has come from Marcin Kulisz, who is starting to package ec2debian-build-ami into a Debian: this will complete the circle of the entire stack being in main (one day)! Thanks goes to Stefano, Anders, Charles, and everyone who contributed to this effort. Resources

Dirk Eddelbuettel: RInside 0.2.10

The new maintenance release 0.2.10 of RInside is now on CRAN, including Windows binaries. RInside provides a set of convenience classes which facilitate embedding of R inside of C++ applications and programs, using the classes and functions provided by the Rcpp R and C++ integration package. This release helps with an update to stack checking, required by a recent change in R itself. The NEWS extract below has more details.
Changes in RInside version 0.2.10 (2012-12-05)
  • Adjusted to change in R which requires turning checking of the stack limit off in order to allow for access from multiple threads as in the Wt examples. As there are have been no side-effects, this is enabled by default on all platforms (with the exception of Windows).
  • Added new threads example directory with a simple example based on a Boost mutex example.
  • Disabled two examples (passing an external function down) which do not currently work; external pointer use should still work.
CRANberries also provides a short report with changes from the previous release. More information is on the RInside page. Questions, comments etc should go to the rcpp-devel mailing list off the Rcpp R-Forge page.

4 December 2012

Michal &#268;iha&#345;: PyGtk in virtualenv on Travis

Weblate is using Travis CI for continuous integration. That in turn uses virtualenv to test against different Python version. Unfortunately one of dependencies in Weblate are PyCairo and PyGtk (actually Pango for text rendering) and these does not support installation using distutils. Fortunately this can be workarounded, but it took me some time to make all this properly. Ffirst you need to choose correct versions not as I did mistakenly mixed PyGObject 3 with PyGtk 2. The next step is to manually run configure and make with correct paths to install within virtualenv. For example for Gtk it looks like following:
wget http://ftp.gnome.org/pub/GNOME/sources/pygtk/2.24/pygtk-2.24.0.tar.bz2
tar xf pygtk-2.24.0.tar.bz2
cd pygtk-2.24.0
export PKG_CONFIG_PATH=/home/travis/virtualenv/python$TRAVIS_PYTHON_VERSION/lib/pkgconfig
./configure --prefix=/home/travis/virtualenv/python$TRAVIS_PYTHON_VERSION/
make
make install
The complete Travis configuration file can be found in Weblate's Git repository.

Filed under: English Weblate 0 comments Flattr this!

Russ Allbery: Review: The Borrowers

Review: The Borrowers, by Mary Norton
Series: The Borrowers #1
Illustrator: Beth Krush
Illustrator: Joe Krush
Publisher: Sandpiper
Copyright: 1953
Printing: 1986
ISBN: 0-15-204737-9
Format: Trade paperback
Pages: 180
Arrietty is a 14-year-old girl who lives under the clock. Well, actually, she lives under the kitchen floor, but the entrance to her family's passageway is under the clock, from which they get their name. She sleeps in an old cigar box, her living room is wallpapered with strips cut from letters (and decorated with postage stamps), and she keeps a journal in a miniature book that's half the size she is. Arrietty, and her parents Pod and Homily, are Borrowers. The Borrowers is a classic series of children's books by Mary Norton about miniature people who, at least in this first book, live in the walls and take things. Not big things, but all the little things: the blotter paper, the pins, the stray sock, old postage stamps, empty matchboxes, and the occasional thimble. With these things, they build their furniture, their clothes, and their homes, safely hidden in the walls and other secret places, always wary of the "human beans" (as Arrietty's mother tends to call them). That life is, of course, a remarkable simulation of middle-class, class-conscious British life, which is much of the fun. Arrietty's family lives in the home of an elderly woman who's confined to bed, so it's quiet, safe, and driven entirely by routine. They only have to watch out for the maid and the gardener, and they keep regular schedules. That's just the way the Borrowers like it; predictable and quiet. Which is why they get caught entirely by surprise when a boy shows up, recovering from illness in the country, and spots Pod on one of his borrowing runs. This is technically a re-read, but I read the original series so long ago that I barely remembered it. Not that there is all that much to remember, at least in the way of plot. This first book follows one of the standard plots for children's books of this type (although it follows Arrietty as the viewpoint character rather than the more common choice of the boy): boy discovers hidden creatures, hidden creatures are afraid, boy turns out to want to make friends with them, they strike up a friendship, peril nonetheless ensues. If you've read many children's books, you've probably read this before; The Mouse and the Motorcycle is an obvious example (and, in terms of plot, a somewhat better book). Arrietty is a typical protagonist, her parents are silly (and a bit flighty, particularly her mother), and the boy is, disappointingly, almost a non-entity. But you don't read The Borrowers for the plot. The brilliance of this book is in the descriptions of what they use, how they use it, and how they construct their ingenious homes. Norton captures much of the wonder of other miniaturization stories (the movie Honey, I Shrunk the Kids is an obvious example), but takes it one step further into clever and thoughtful repurposing. The book is full of off-hand descriptions of how everyday objects become different everyday objects for miniature people, giving it the same delight that one gets from descriptions of how model railroad dioramas or hand-made doll houses are constructed. The detailed (if busy) illustrations by the Krushes help tremendously with visualization and make the descriptions even more fun. This is very much a children's book based on an exploration of a single great idea, so don't expect too much in the way of plot or deep characterization. It's also not science fiction; there isn't any realistic extrapolation of culture. The Borrowers all behave like perfectly ordinary English people without any further explanation because, of course, that's what they would do. Re-reading it as an adult, I wished there was a bit more meat to it. But the ingenuity is still a delight. It's great fun to spend a couple of hours imagining how one would construct the miniature accouterments of everyday life. Followed by The Borrowers Afield. Rating: 7 out of 10

2 December 2012

Dirk Eddelbuettel: RQuantLib 0.3.9

A minor feature release RQuantLib 0.3.9 is now on CRAN and in Debian. RQuantLib combines (some of) the quantitative analytics of QuantLib with the R statistical computing environment and language. Bryan Lewis had suggested to enable another pricing engine for American Options in order to get (at least some) Greeks. This is now supported by picking engine="CrankNicolson" as shown in the default example for the AmericanOption function:
R> library(RQuantLib)
R> example(AmericanOption)
AmrcnOR> # simple call with unnamed parameters
AmrcnOR> AmericanOption("call", 100, 100, 0.02, 0.03, 0.5, 0.4)
Concise summary of valuation for AmericanOption 
  value   delta   gamma    vega   theta     rho  divRho 
11.3648      NA      NA      NA      NA      NA      NA 
AmrcnOR> # simple call with some explicit parameters
AmrcnOR> AmericanOption("put", strike=100, volatility=0.4, 100, 0.02, 0.03, 0.5)
Concise summary of valuation for AmericanOption 
  value   delta   gamma    vega   theta     rho  divRho 
10.9174      NA      NA      NA      NA      NA      NA 
AmrcnOR> # simple call with unnamed parameters, using Crank-Nicolons
AmrcnOR> AmericanOption("put", strike=100, volatility=0.4, 100, 0.02, 0.03, 0.5, engine="CrankNicolson")
Concise summary of valuation for AmericanOption 
  value   delta   gamma    vega   theta     rho  divRho 
10.9173 -0.4358  0.0140      NA      NA      NA      NA 
R> 
Thanks to CRANberries, there is also a diff to the previous release 0.3.8. Full changelog details, examples and more details about this package are at my RQuantLib page.

1 December 2012

Steve Kemp: Reassessing what I want from a simple website creation tool

Thanks very much to everybody who commented, both publicly and privately, on my previous entry. To recap: I have three sites that were each generated by slightly different templating software I'd built and tweaked over the years. I was frustrated that the three copies of the generation tool had all drifted and diverged from each other, and was looking to setup a new (static) site. The obvious conclusion was either: I'd cheerfully decided to go down the second route, because life is short. But after having quick reads, then spending several hours investigating likely contenders I kept finding reasons why they weren't suitable. Today I reworked my tool to succesfully generate each of the three sites. That was less annoying than expected, after I'd decided "I'll have to change my templates anyway, when I switch to a real tool". So in the interests of sharing I placed my tool online, and wrote documentation: This is not a U-turn. This is not a commitment to avoid the investigation of real replacements. This is just something I had to do as a cleanup and to make sure I fully understood exactly what my requirements were. In conclusion: My requirements are now absolutely known, fixed, and understood. I still firmly intend and expect to migrate to something by the end of the year. Ideally something that will make tag pages, RSS feeds, and other clever things easy. (Getting rid of literal shell usage in my templates, and unifying the way I auto-generate galleries via file globs was a useful change in its own regard - I always felt slightly dirty..) Lookout for more summaries and reviews of specific tools when I've had the chance to relax and start looking again.

30 November 2012

Richard Hartmann: Release Critical Bug report for Week 48

The UDD bugs interface currently knows about the following release critical bugs: How do we compare to the squeeze cycle ?
Week Squeeze Wheezy Diff
43 284 (213+71) 468 (332+136) +184 (+119/+65)
44 261 (201+60) 408 (265+143) +147 (+64/+83)
45 261 (205+56) 425 (291+134) +164 (+86/+78)
46 271 (200+71) 401 (258+143) +130 (+58/+72)
47 283 (209+74) 366 (221+145) +83 (+12/+71)
48 256 (177+79)
49 256 (180+76)
50 204 (148+56)
51 124 (124+54)
52 115 (78+37)
1 93 (60+33)
2 82 (46+36)
3 25 (15+10)
4 14 (8+6)
5 2 (0+2)
6 release!

29 November 2012

Bernhard R. Link: Gulliver's Travels

After seeing some book descriptions recently on planet debian, let me add some short recommendation, too. Almost everyone has heard about Gulliver's Travels already, so usually only very cursory. For example: did you know the book describes 4 journeys and not only the travel to Lilliput? Given how influential the book has been, that is even more suprising. Words like "endian" or "yahoo" originate from it. My favorite is the third travel, though, especially the acadamy of Lagado, from which I want to share two gems: " His lordship added, 'That he would not, by any further particulars, prevent the pleasure I should certainly take in viewing the grand academy, whither he was resolved I should go.' He only desired me to observe a ruined building, upon the side of a mountain about three miles distant, of which he gave me this account: 'That he had a very convenient mill within half a mile of his house, turned by a current from a large river, and sufficient for his own family, as well as a great number of his tenants; that about seven years ago, a club of those projectors came to him with proposals to destroy this mill, and build another on the side of that mountain, on the long ridge whereof a long canal must be cut, for a repository of water, to be conveyed up by pipes and engines to supply the mill, because the wind and air upon a height agitated the water, and thereby made it fitter for motion, and because the water, descending down a declivity, would turn the mill with half the current of a river whose course is more upon a level.' He said, 'that being then not very well with the court, and pressed by many of his friends, he complied with the proposal; and after employing a hundred men for two years, the work miscarried, the projectors went off, laying the blame entirely upon him, railing at him ever since, and putting others upon the same experiment, with equal assurance of success, as well as equal disappointment.' " "I went into another room, where the walls and ceiling were all hung round with cobwebs, except a narrow passage for the artist to go in and out. At my entrance, he called aloud to me, 'not to disturb his webs.' He lamented 'the fatal mistake the world had been so long in, of using silkworms, while we had such plenty of domestic insects who infinitely excelled the former, because they understood how to weave, as well as spin.' And he proposed further, 'that by employing spiders, the charge of dyeing silks should be wholly saved;' whereof I was fully convinced, when he showed me a vast number of flies most beautifully coloured, wherewith he fed his spiders, assuring us 'that the webs would take a tincture from them; and as he had them of all hues, he hoped to fit everybody s fancy, as soon as he could find proper food for the flies, of certain gums, oils, and other glutinous matter, to give a strength and consistence to the threads.'"

Steve Kemp: Simple website creation tools

I host a number of websites which are mostly static in nature, these are often hand-crafted, but three of them us a slightly hacked up creation of my own. Given a master "template" the file foo.skx gets massaged into foo.html. Sadly I added features randomly, and now I have three template-driven sites which are handled slightly differently. This put me in the position of having to choose between two options: Simplifying my life is good. So I examined a list of static site generators, and a few more found by searching github. Other than doing clever things by knowing which page is "current" I needed to do only minimal magic: Webgen looked like a good fit, but I couldn't get per-page templates to work out. Either they would work, or I would get weird errors about blocks not being known. Webby worked, but I didn't like it. Poole was the next one that I got far down the road with, but it allowed only a single site-wide template. Shame because otherwise I loved its flexibility enought to tolerate writing "macros" in Python. I've still got to test more out, but it is a fun process. I fully intend to adopt an existing tool, and not keep working on my own. Tonight I'm going to look at a few more.

Michal &#268;iha&#345;: Multi part progress in Weblate

Weblate is still evolving and it recently got new feature to allow better overview of translation status - multi part progress bars. The motivation to show more information is to have better overview of the translation. The numbers have been always shown withing Weblate, but making it graphical gives it much better visibility: Multi part progress in Weblate I'm still unsure about colors to use for highlighting different state - the failed checks should not be too different from translated strings as they might contain quite some false positives while fuzzy strings need to be really different as they absolutely need attention from translator. If you have better ideas about color scheme, please let me know! You can see live example at Polish summary at l10n.cihar.com.

Filed under: English Weblate 3 comments Flattr this!

Russ Allbery: Review: American Gods

Review: American Gods, by Neil Gaiman
Publisher: HarperTorch
Copyright: July 2001
Printing: May 2002
ISBN: 0-380-78903-5
Format: Mass market
Pages: 588
At the start of American Gods, Shadow is in prison. He's done three years of a sentence of six and is about to get released on parole. He has a wife and a job waiting for him and every hope of getting back to a good life. None of that happens. The day he's released, he finds out his wife died in a car accident. The job is gone too; it was an offer from his best friend, who died in the same accident. Numb, an ex-con, with no job prospects, he's recruited by a strange man named Wednesday, with the help of a leprechaun named Mad Sweeney, to drive Wednesday around the country, run errands for him, hurt people if absolutely necessary, and hold his vigil if he dies. And that's how Shadow is drawn into the conflict between old gods and young, between the gods that immigrants brought to America from the old world and the flashy new gods of technology and media that America is creating for itself. With native American gods and cultural legends watching around the edges. I first read American Gods shortly after it came out and before I started writing reviews, so this is a re-read. It's a book I'm glad I re-read before reviewing. The first time I read it, I was thoroughly engrossed and thought it was excellent. This time, I was entertained, but surprised at how slowly it moved, how disconnected it sometimes felt, and how little substance there was to the plot. Reading American Gods involves spending lots of time with Shadow, and I think one's reactions to this book will vary a lot based on how much one likes him as a character. Shadow is very withdrawn, very detached from the world, and quietly matter-of-fact in how he goes through his life. He's been through some major shocks and has been severed fairly thoroughly from society by three years in prison, and he doesn't reconnect easily. To some readers, such as myself, this comes across as taciturn competence and quiet observation, two things that I greatly enjoy reading about. I liked Shadow, liked reading between the lines for hints at his feelings, and sympathized a lot with his emotional state. But I can easily see an alternative reading that makes him a cipher or, worse, a blank spot in the book into which the reader has to insert themselves. One has to work at seeing Shadow's character and finding some sympathy for the degree to which he stays separate from the world. American Gods does have a plot, but it's the sort of plot that allows for endless digressions, background stories, and side meetings. In that, it reminded me less of Gaiman's other novels and more of several arcs of Sandman. I think it's worthwhile to think of this less as a novel and more as an extended ramble through the world as Gaiman sees it, full of mythology, stories, quirky people understanding the world in their own ways, and upwellings of the sacred and significant in the most unexpected of places. For example, the holy places of America in American Gods, analogous to the multiple small sacred places and old ruins in Europe, are roadside attractions, via a line of argument that makes much more sense than it has any right to and demonstrates a canny understanding of the expression of American entrepreneurial culture. As with much Gaiman, reading with Wikipedia or another mythological resource at hand is a good idea. Even if you have a good general knowledge of the most common mythology studied in the US, you probably won't recognize many of the figures Gaiman draws on: Czernobog, for example, or Wisakedjak. Part of the fun of this book is seeing Gaiman humanize them and embed them in modern culture and modern concerns while retaining their mythological edge and strangeness. As any reader of Sandman can tell you, this is one of Gaiman's strengths as a writer, and he does it well here. The bits with the undertakers Mr. Ibis and Mr. Jacquel (Thoth and Anubis) are absolutely brilliant and probably the best sections of the book (although Sam's exchange with the Men in Black is a strong contender).
"Back in my day, we had it all set up. You lined up when you died, and you'd answer for your evil deeds and for your good deeds, and if your evil deeds outweighed a feather, we'd feed your soul and your heart to Ammet, the Eater of Souls." "He must have eaten a lot of people." "Not as many as you'd think. It was a really heavy feather. We had it made special. You had to be pretty damn evil to tip the scales on that baby."
Like so much of Gaiman's work, this is the sort of book that contributed a half-dozen quotes to my quote file during my first reading. Gaiman always impresses me by not only managing to portray gods as human without losing the numinous edge, but sustaining that portrayal through very extended contact with the same god. And despite the interlude with the undertakers, the long interlude in small-town life in Lakeside, and the numerous other digressions and meanderings of a book that's not in much of a hurry to get anywhere, there is a plot and a definite conclusion, and Gaiman has some interesting twists in store at the end of that road. There's a lot of philosophical currents going on under the surface, not all of which cohere, but Gaiman does pull a lot of them together. The resolution can be a bit of an anticlimax, but I think it's a fitting one that's true to the nature of the story. I didn't enjoy this as much on the second reading as I did on the first. It didn't feel quite as coherent as I'd remembered it, and the meandering pace occasionally made the story feel stretched. But I do like Shadow, I love spending time in Gaiman's head and with his portrayals of deities, and American Gods is a treasure of great quotes. If you enjoy Gaiman's other work and aren't looking for a lot of plot and forward-moving energy, this is definitely recommended. Rating: 7 out of 10

Next.

Previous.