Search Results: "zed"

16 December 2012

Joachim Breitner: Circle Packing

For an upcoming introductory Haskell talk of mine (next Tuesday in Karlsruhe please come and join if you will) I was looking into data visualization as a example application. With libraries like gloss, getting some nice vector graphics up and running within one talk is quite possible. For some reason, I want to arrange multiple circles of varying size so that they do not overlap and use as little space as possible. My first approach was to use general purpose optimizers such as cmaes, Numeric.GSL.Minimization from the hmatrix package and force-layout. I especially liked the interface of cmaes, which is able to minimize a function of, say, type [(String, Double, Double)] -> Double by automatically finding the values of type Double in the input, as described by Takayuki Muranushi. That would have looked great in the talk. Unfortunately, none of these libraries gave sufficient good results in a reasonable amount of time for this problem. I then reviewed some of the academic literature on the circle packing problem and there were some algorithms proposed, but no simple one and none of the papers had a concise pseudo-code description that I could transfer to Haskell. So eventually I set out to implement a relatively dump and slow greedy algorithm myself: I place one circle after another, starting with the largest, and among the valid positions where a new circle touches two already placed circles, I choose the positions closest to the center of mass. The result, available in the circle-packing package, looks surprisingly well, here visualized using the diagrams library (See the documentation of Optimisation.CirclePacking for the code used to generate that image):
Now showing just this image is not a very good way to demonstrate my code. A few years ago, I might have created a CGI script that would dynamically generate images based on your input. But not in 2012: Since my code is pure Haskell without fancy features like type classes, I can use the fay compiler to generate JavaScript code from it. Add a little Haskell code to interact with the HTML5 canvas and now you can interactively try out circle-packing in your browser. Oh, and while I am talking about neat tricks: You can put vector graphics in your haddock documentation, as I have done for Optimisation.CirclePacking, using the syntax <<<data:image/svg+xml;base64,PD94bWwgdmV...c3ZnPg==>>.

14 December 2012

Russell Coker: Globalisation and Phone Calls

I just watched an interesting TED talk by Pankaj Ghemawat (of which the most important points are summarised in a TED blog post) about the world not being as globalised as people expect [1]. One point is that only 2% of traditional voice phone calling minutes (and ~6% when you include VOIP) are for international calls which is less than most people expect. After reading that it occurred to me that most of the included value in my mobile phone contract goes unused, I pay for extra data transfer and it includes voice calling credit that I don t use. So out of $450 of included calls I typically use much less than $50. $400 of calls to even the most expensive countries is about 100 minutes of talking. So the logical thing to do is to find people in other countries to call. If you are involved in the FOSS community and would like to speak to me then send me an email with your phone number, time zone, and a range of times that are convenient. I won t make any promises about calling you soon (I could use up my monthly credit on a single call), but I will call you eventually. I will also send email to some people I know by email and suggest a chat. Some years ago I did this with people who were involved in SE Linux development and it seemed to help the development of the SE Linux community. Also if anyone in Australia wants to speak to me then that s OK too. While Pankaj s talk inspired me to call people I m not dedicated to calling other countries.

11 December 2012

Francesca Ciceri: Welcome Malatesta, Goodbye Kasbah!

Kasbah, my old laptop, is almost dead. It was my first laptop, an Asus X50C, which worked like a charm and was robust like a tank. But then the screen hinges broke. Time for a new one. So, ladies and gentlemen, I'm glad to present you malatesta. Malatesta is a Lenovo Thinkpad E335. Sadly, while kasbah worked with free drivers, malatesta requires proprietary drivers both for the video card and the wifi. Which sounds a bit ironic for a laptop named after a famous anarchist. The only two problems I encountered with it were related to the video card (00:01.0 VGA compatible controller: Advanced Micro Devices [AMD] nee ATI Device 9808) and the special keys. Malatesta worked well with the kernel of Squeeze, but freezed during the boot with Wheezy. Thankfully, I stumbled upon this blogpost by Blars Blarson suggesting to use the fglrx-driver with Sid. And that solved the problem. Another little odissey was the brightness of the monitor. With kernels < 3.2.0-4-amd64 the thinkpad_acpi didn't recognize my model and I was stuck with a brightness level probably visible from the outer space. Neither xrandr nor xbacklight could help, and I was seriously considering to use sunglasses to work (a slight unusual approach to the brightness problem, I know, but I suffer of migraines and couldn't bear an hyper-bright monitor), when the upgrade to Sid solved the problem. On the bright side (pun intended) this laptop was born without Windows. (Both malatesta and kasbah were shipped with FreeDOS). Now, if you're wondering about the names: my first namescheme was "references to songs by the Clash" (and kasbah is obviously for "Rock the Casbah", with a typo). But it wasn't really scalable. Since last year, I've decided to name my systems after famous anarchists. The server is named after one of my all time favourite anarchists, Buenaventura Durruti.
madamezou@durruti:~$ cat /etc/motd
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Llevamos un mundo nuevo en nuestros corazones;
y ese mundo est  creciendo en este instante
                        Buenaventura Durruti
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
While the new laptop's name comes from Errico Malatesta.
madamezou@malatesta:~$ cat /etc/motd 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
We anarchists do not want to emancipate the people;
We want the people to emancipate themselves.
            Errico Malatesta, L'Agitazione (18 June 1897)
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

8 December 2012

Dirk Eddelbuettel: RcppArmadillo 0.3.6.0

Conrad launched the 3.6 series of Armadillo earlier today with a first 3.6.0 release. So RcppArmadillo, our wrapper for R and Armadillo, is now on CRAN with its corresponding version 0.3.6.0. No R level or interface changes were needed, and the upstream changes are summarized below.
Changes in RcppArmadillo version 0.3.6.0 (2012-12-07)
  • Upgraded to Armadillo release Version 3.6.0 (Piazza del Duomo)
    • faster handling of compound expressions with submatrices and subcubes
    • added support for loading matrices as text files with NaN and Inf elements
    • added stable_sort_index(), which preserves the relative order of elements with equivalent values
    • added handling of sparse matrices by mean(), var(), norm(), abs(), square(), sqrt()
    • added saving and loading of sparse matrices in arma_binary format
Courtesy of CRANberries, there is also a diffstat report for the most recent release As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

1 December 2012

Rapha&#235;l Hertzog: My Free Software Activities in November 2012

This is my monthly summary of my free software related activities. If you re among the people who made a donation to support my work (692.20 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Misc packaging I updated the publican package (a tool for publishing material authored in DocBook XML) with version 3.0, a major new upstream version. As with any important update, it had its share of problems and I created two patches that I sent upstream. I uploaded the package to experimental since we re in freeze. The Debian Administrator s Handbook Since the translation teams have been working for a few months, I wanted to put the result of their work online. I did it and I blogged about it on debian-handbook.info. By the way, we have a Polish translation that just started. This took quite some time because many translators were not well versed with Docbook XML and its structure. So I fixed their mistakes and asked the Weblate developer (Michal Cihar) to implement new checks to avoid those basic XML mistakes. I also added a couple of build scripts to the git repository to make it easier to rebuild translations in multiple formats. I used this opportunity to file a couple of bugs I encountered with Publican (concerning ePub output mainly, and custom brands). I also blogged about our plans to update the book for Wheezy. Roland started to work on it but I did not have the time yet. Debian France The officers (president, treasurer, secretary) have just changed and we had to organize the transition. As the new president, I got administrator access on our Gandi virtual machine (france.debian.net) as well as access to our bank account. I got also got a bunch of administrative papers retracing the history of the association. Carl Chenet (the former president) gave them to me during the mini-debconf that was organized in Paris. Indeed, Sylvestre Ledru and Mehdi Dogguy organized our second mini-debconf Paris and they did it very well. It was a great success with over 100 attendants each of the 2 days it lasted (November 24-25th). Carl managed a merchandising booth that was well stuffed (Luca Capello also brought goodies of Debian.ch) I gave small lightning talk to present the ideas behind my Librement project (it s about funding free software developers). BTW I have not been very good at it, it was only my second lightning talk and I have been a bit too verbose. The talk did not fit in my 5 minutes time slot ;-) Back from the mini-debconf, I have been trying to delegate some projects (like get a real website, improve the work-flow of members management, update our server which was still running Lenny). Julien Cristau was willing to upgrade the server did not exactly knew how to upgrade the kernel (it s a bit special since Gandi manages the kernel on the Xen hypervisor side). So I took care of this part and also did some cleanup (adding a backup with its associated remote disk, tweaking the email configuration). And Julien completed the upgrade on November 30th. Alexandre Delano volunteered to have a try at the website and Emmanuel Bouthenot has been looking a bit to see if there was something better than Galette to handle our members. It looks like we ll stay with Galette but have to take care of upgrading it to a newer version. I also processed the first membership applications and organized a vote to extend the board of administrators (since we have two vacant seats). On Monday, we should be back to 9 administrators. Librement Except for the talk during the mini-debconf, I did not do much on this project. That said I got an answer from the Autorit de Contr le Prudentiel saying that I might be eligible for the exemption case (see discussion of last month) and that I should fill out a form to get a confirmation. I also contacted Tunz.com who might be able to provide the services I need (their E-money manager product in particular). They have the required accreditation as a banking/credit institution and are willing to partner with enterprises who setup platforms where you must manage flows of money between several parties. I m now waiting for details such as the cost of their various services. I expect to have much more to show next month I m working with two developers to implement the first building blocks of all this. Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

29 November 2012

Wouter Verhelst: HELMo talk about Debian

About a month and a half ago, I got an email from someone studying at the HELMo institute for higher eduction in Li ge, Belgium, inviting me to talk at their conference which was held today. In the email, they asked me to talk about "Debian and Open Source", which is such a wide subject that I could talk about it all day; and then they gave me 20 minutes to do it. So I picked "Debian", and focused a bit more on that, only to find out when I saw their schedule that they'd put "Open Source" there instead. Hrm. Oh well. The talk itself went pretty good. I believe I managed a good balance between introducing Debian to those who've not heard of it before, and explaining to those who have, how this Debian thing works (or is supposed to work). The fact that the audience consisted of first- and second-year IT students made this easier (in case you were wondering: the third-year students, who're supposed to graduate at the end of this eyar, organized the thing). I got some interesting questions afterwards (which is always a good sign), and only positive comments as well (which is even better). After me, Lionel Dricot held a talk which was titled "Pirate Party", but which could be summarized as his personal history into the open source world, followed by some parallels between the open source world and the pirate party.That, too, was an interesting talk, even if I'm not sure I'd vote for the pirate party (which isn't even possible in Mechelen, but ignoring that). All in all, a nice use of my time.

Russ Allbery: Review: American Gods

Review: American Gods, by Neil Gaiman
Publisher: HarperTorch
Copyright: July 2001
Printing: May 2002
ISBN: 0-380-78903-5
Format: Mass market
Pages: 588
At the start of American Gods, Shadow is in prison. He's done three years of a sentence of six and is about to get released on parole. He has a wife and a job waiting for him and every hope of getting back to a good life. None of that happens. The day he's released, he finds out his wife died in a car accident. The job is gone too; it was an offer from his best friend, who died in the same accident. Numb, an ex-con, with no job prospects, he's recruited by a strange man named Wednesday, with the help of a leprechaun named Mad Sweeney, to drive Wednesday around the country, run errands for him, hurt people if absolutely necessary, and hold his vigil if he dies. And that's how Shadow is drawn into the conflict between old gods and young, between the gods that immigrants brought to America from the old world and the flashy new gods of technology and media that America is creating for itself. With native American gods and cultural legends watching around the edges. I first read American Gods shortly after it came out and before I started writing reviews, so this is a re-read. It's a book I'm glad I re-read before reviewing. The first time I read it, I was thoroughly engrossed and thought it was excellent. This time, I was entertained, but surprised at how slowly it moved, how disconnected it sometimes felt, and how little substance there was to the plot. Reading American Gods involves spending lots of time with Shadow, and I think one's reactions to this book will vary a lot based on how much one likes him as a character. Shadow is very withdrawn, very detached from the world, and quietly matter-of-fact in how he goes through his life. He's been through some major shocks and has been severed fairly thoroughly from society by three years in prison, and he doesn't reconnect easily. To some readers, such as myself, this comes across as taciturn competence and quiet observation, two things that I greatly enjoy reading about. I liked Shadow, liked reading between the lines for hints at his feelings, and sympathized a lot with his emotional state. But I can easily see an alternative reading that makes him a cipher or, worse, a blank spot in the book into which the reader has to insert themselves. One has to work at seeing Shadow's character and finding some sympathy for the degree to which he stays separate from the world. American Gods does have a plot, but it's the sort of plot that allows for endless digressions, background stories, and side meetings. In that, it reminded me less of Gaiman's other novels and more of several arcs of Sandman. I think it's worthwhile to think of this less as a novel and more as an extended ramble through the world as Gaiman sees it, full of mythology, stories, quirky people understanding the world in their own ways, and upwellings of the sacred and significant in the most unexpected of places. For example, the holy places of America in American Gods, analogous to the multiple small sacred places and old ruins in Europe, are roadside attractions, via a line of argument that makes much more sense than it has any right to and demonstrates a canny understanding of the expression of American entrepreneurial culture. As with much Gaiman, reading with Wikipedia or another mythological resource at hand is a good idea. Even if you have a good general knowledge of the most common mythology studied in the US, you probably won't recognize many of the figures Gaiman draws on: Czernobog, for example, or Wisakedjak. Part of the fun of this book is seeing Gaiman humanize them and embed them in modern culture and modern concerns while retaining their mythological edge and strangeness. As any reader of Sandman can tell you, this is one of Gaiman's strengths as a writer, and he does it well here. The bits with the undertakers Mr. Ibis and Mr. Jacquel (Thoth and Anubis) are absolutely brilliant and probably the best sections of the book (although Sam's exchange with the Men in Black is a strong contender).
"Back in my day, we had it all set up. You lined up when you died, and you'd answer for your evil deeds and for your good deeds, and if your evil deeds outweighed a feather, we'd feed your soul and your heart to Ammet, the Eater of Souls." "He must have eaten a lot of people." "Not as many as you'd think. It was a really heavy feather. We had it made special. You had to be pretty damn evil to tip the scales on that baby."
Like so much of Gaiman's work, this is the sort of book that contributed a half-dozen quotes to my quote file during my first reading. Gaiman always impresses me by not only managing to portray gods as human without losing the numinous edge, but sustaining that portrayal through very extended contact with the same god. And despite the interlude with the undertakers, the long interlude in small-town life in Lakeside, and the numerous other digressions and meanderings of a book that's not in much of a hurry to get anywhere, there is a plot and a definite conclusion, and Gaiman has some interesting twists in store at the end of that road. There's a lot of philosophical currents going on under the surface, not all of which cohere, but Gaiman does pull a lot of them together. The resolution can be a bit of an anticlimax, but I think it's a fitting one that's true to the nature of the story. I didn't enjoy this as much on the second reading as I did on the first. It didn't feel quite as coherent as I'd remembered it, and the meandering pace occasionally made the story feel stretched. But I do like Shadow, I love spending time in Gaiman's head and with his portrayals of deities, and American Gods is a treasure of great quotes. If you enjoy Gaiman's other work and aren't looking for a lot of plot and forward-moving energy, this is definitely recommended. Rating: 7 out of 10

28 November 2012

Alberto Garc a: QEMU and open hardware: SPEC and FMC TDC

Working with open hardware Some weeks ago at LinuxCon EU in Barcelona I talked about how to use QEMU to improve the reliability of device drivers. At Igalia we have been using this for some projects. One of them is the Linux IndustryPack driver. For this project I virtualized two boards: the TEWS TPCI200 PCI carrier and the GE IP-Octal 232 module. This work helped us find some bugs in the device driver and improve its quality. Now, those two boards are examples of products available in the market. But fortunately we can use the same approach to develop for hardware that doesn t exist yet, or is still in a prototype phase. Such is the case of a project we are working on: adding Linux support for this FMC Time-to-digital converter.
FMC TDC
This piece of hardware is designed by CERN and is published under the CERN Open Hardware Licence, which, in their own words is to hardware what the General Public Licence (GPL) is to software . The Open Hardware repository hosts a number of projects that have been published under this license. Why we use QEMU So we are developing the device driver for this hardware, as my colleague Samuel explains in his blog. I m the responsible of virtualizing it using QEMU. There are two main reasons why we want to do this:
  1. Limited availability of the hardware: although the specification is pretty much ready, this is still a prototype. The board is not (yet) commercially available. With virtual hardware, the whole development team can have as many boards as it needs.
  2. Testing: we can test the software against the virtual driver, force all kinds of conditions and scenarios, including the ones that would probably require us to physically damage the board.
While the first point might be the most obvious one, testing the software is actually the one we re more interested in. My colleague Miguel wrote a detailed blog post on how we have been using QEMU to do testing. Writing the virtual hardware Writing a virtual version of a particular piece of hardware for this purpose is not as hard as it might look. First, the point is not to reproduce accurately how the hardware works, but rather how it behaves from the operating system point of view: the hardware is a black box that the OS talks to. Second, it s not necessary to have a complete emulation of the hardware, there s no need to support every single feature, particularly if your software is not going to use it. The emulation can start with the basic functionality and then grow as needed. The FMC TDC, for example, is an FMC card which is in our case connected to a PCIe bridge called SPEC (also available in the Open Hardware repository). We need to emulate both cards in order to have a working system, but the emulation is, at the moment, treating both as if they were just one, which makes it a bit easier to have a prototype and from the device driver point of view doesn t really make a difference. Later the emulation can be split in two as I did with with TPCI200 and IP-Octal 232. This would allow us to support more FMC hardware without having to rewrite the bridging code. There s also code in the emulation to force different kind of scenarios that we are using to test if the driver behaves as expected and handles errors correctly. Those tests include the simulation of input in the any of the lines, simulation of noise, DMA errors, etc.
Tests
And we have written a set of test cases and a continuous integration system, so the driver is automatically tested every time the code is updated. If you want details on this I recommend you again to read Miguel s post.

26 November 2012

Steve Langasek: Upstart in Debian

Good news, everyone! So as of last Sunday, this works on all Linux archs in Debian unstable and gives you a modern version of upstart:
echo 'Yes, do as I say!'   apt-get -o DPkg::options=--force-remove-essential -y --force-yes install upstart
Thanks to the ifupdown, sysvinit, and udev maintainers for their cooperation in getting upstart support in place; to the Debian release team for accomodating the late changes needed for upstart to be supported in wheezy; and to Scott for his past maintenance of upstart in Debian. Benchmarking One of the consequences is that it's now possible to do meaningful head-to-head comparisons of boot speed between sysvinit (with startpar), upstart, and systemd. At one time or another people have tested systemd vs. sysvinit when using bash as /bin/sh, and upstart vs. sysvinit, and systemd vs. sysvinit+startpar, and there are plenty of bootcharts floating around showing results of one init system or another on one distro or another, but I'm not aware of anyone having done a real, fair comparison of the three solutions, changing nothing but the init system. I've done some initial comparisons in a barebones sid VM, and the results are definitely interesting. Sysvinit with startpar (the default in Debian) can boot a stock sid install, with no added services, in somewhere between 3.37 and 3.42 seconds (three runs). That's not a whole lot, but on the other hand this is a system with a single filesystem and no interesting services yet. Is this really as fast as we can boot? No, even this minimal system can boot faster. Testing with upstart shows that upstart can do the same job in between 3.03 and 3.19 seconds (n=3, mean=3.09). This confirms what we'd already seen in Ubuntu, that it makes a difference to boot speed to have filesystem mounting handled by an integrated process that understands the whole system instead of as a group of serialized shell scripts. What about systemd? The same test gives a boot time between 2.32 and 2.85 seconds (n=4, mean=2.48). Interesting; what would make systemd faster than upstart in sid? Well, a quick look at the system shows one possible contributing factor: the rsyslog package in Debian has a systemd unit file, but not an upstart job file. Dropping in the /etc/init/rsyslog.conf from Ubuntu has a noticeable impact, and brings the upstart boot time down nearer to that of systemd (2.78-3.03s, n=5, mean=2.92). Besides telling me that it's time to start spamming Debian maintainers with wishlist bugs asking them to include upstart jobs in their packages, this suggests that the remaining difference in boot time may be due to the outstanding init scripts in rcS.d that are made built-in no-ops by systemd but not (yet) by upstart in Debian (e.g., hwclock, hostname, udev-mtab). (In Ubuntu, /etc/rcS.d has long since been emptied out in favor of upstart jobs in the common case, since the time it takes to get to runlevel=2 is definitely a major issue for boot speed and boot parallelization.) It also gives the lie to the claim that's been made in various places that spawning shells is a major bottleneck for upstart vs. systemd. More study is certainly needed to confirm this, but at least this naive first test suggests that in spite of the purported benefits of hard-coding boot-time policies in C code, upstart with its default degree of runtime configurability is at least in the ballpark of systemd. Indeed, when OpenSuSE switched from upstart to systemd, it seems that something else in the stack managed to nullify any benefit from improving the boot-time performance of apparmor. Contrary to what some would have you believe, systemd is not some kind of silver bullet for boot speed. Upstart, with its boot-time flexibility and its long history of real-world testing in Ubuntu, is a formidable competitor to systemd in the boot speed department - and a solid solution to the many longstanding boot-time ordering bugs in Debian, which still affect users of sysvinit. I've published the bootcharts for the above tests here. Between the fact that Debian's bootchart package logs by default to /var/log/bootchart.tgz (thus overwriting on every boot) and the fact that these tests are in a VM, I haven't bothered to include the raw data, just the bootcharts themselves. The interested reader can probably generate more interesting boot charts of their own anyway - in particular, it will be interesting to see how the different init systems perform with more complicated filesystem layouts, or when booting a less trivial set of services. Other musings The boot charts have been created with the bootchart package rather than with bootchart2. For one thing, it turns out that bootchart2 includes systemd units, not init scripts; so when replacing bootchart with bootchart2, the non-matching init script is left behind and systemd in particular gets terribly confused. This is now reported as Bug #694403. In an amusing twist, while I was experimenting with bootchart2, I also noticed that having systemd installed would slow down booting with other init systems, because systemd installs udev rules which take a noticeable amount of time to run a helper command at boot even though the helper should be a no-op. So if you're doing boot speed testing of other init systems, be sure you don't have systemd on the system at the time!

24 November 2012

Eddy Petri&#537;or: Singing your way to Parliament

Here is a "customized" version of a political ad that I made based on the real one of a guy running in the area where I live right now. I made it as a joke, I don't endorse the guy at all. In fact, I think that I would advise anyone having this guy on the ballot, to vote whomever else.



The candidate is a singer part of a band which had it's heyday around the beginning of the previous decade. His slogan is "Facts not words", but I thought "Rhymes, not words" is more appropriate.

There are two other jokes in here, I hope my Romanian readers will appreciate them.

Russ Allbery: Review: Dune

Review: Dune, by Frank Herbert
Series: Dune #1
Publisher: Ace
Copyright: 1965
Printing: September 1990
ISBN: 0-441-17271-7
Format: Mass market
Pages: 537
The Atreides family, distant cousins to the imperial family, have ruled the planet of Caladan for twenty generations. Caladan is a wet farming world, comfortable and pleasant, but not horribly important. But House Atreides is feuding with House Harkonnen, and, at the start of Dune, that feud maneuvers Duke Leto into giving up his holdings and moving his family to take possession of Arrakis. Arrakis is a desert planet, previously controlled by Baron Harkonnen. It is unrelentingly hostile, home to smugglers and dangerous local desert dwellers called Fremen. But it's also one of the most important planets in the galaxy, since it's the sole origin of the chemical called melange, or spice. Spice permits a limited form of prescience, which allows the navigators of the Spacing Guild to successfully steer ships across the interstellar void. Arrakis's production of spice is what makes interstellar travel, and therefore all of interstellar civilization, possible. Dune is the story of Paul Atreides, son and heir to Duke Leto Atreides. His mother, Lady Jessica, is one of the Bene Gesserit, a secretive order of women devoted to mental and physical discipline and to the long-term genetic improvement of mankind. He is not supposed to exist; Lady Jessica was supposed to only bear a daughter of Leto. But he may be something special, the long-sought (but also dangerous) Kwisatz Haderach who can unite male and female Bene Gesserit powers. The Bene Gesserit take great interest in him from the start of Dune. More surprisingly, so do the Fremen of Arrakis; from the moment he arrives there, he seems to be fulfilling prophecies of theirs that are partly, but not entirely, ones planted by the Bene Gesserit long ago. The feud with the Harkonnens, the unstable place of Arrakis in galactic politics, the dreams of the Fremen and the Imperial ecologist on Arrakis of terraforming, Bene Gesserit plans, Paul's abilities, and the legends of the Fremen all combine in a complex mix of politics, battle, and clashes of culture. Dune is an acknowledged SF masterpiece, one of the best-known classics of the genre. It's usually found in short lists of the best SF novels ever written. It spawned five sequels by Frank Herbert (about which more in a moment), as well as numerous additional sequels and prequels by Kevin J. Anderson and Brian Herbert. It's been adopted for the screen twice, not to mention board games, video games, and numerous other projects. This is my second reading, the first in about twenty years, but the story was still immediately familiar from having seen films and having discussed and read about the universe. This is not science fiction in any strict sense. Dune is science fiction in the same way that Star Wars is: a futuristic gloss on top of power structures inspired by feudalism, heavily mixed with mysticism, mental powers, magic, and implausible but convenient science that creates the story effects the author wants. Both Bene Gesserit powers in general and Paul's abilities in particular are effectively magic. There is some hand-waving explanation of their ability to verbally control other people as taking advantage of specific pitches and intonations that people are vulnerable to, but it's effectively spell-casting (and is a direct inspiration for Jedi mind tricks). All of the mysticism (and there's quite a lot of it in Dune, including race memory, precognition, and even molecular transformation) resembles the Force from Star Wars more than anything scientific. Dune is epic fantasy told on a science fiction stage, complete with a young protagonist coming into his powers and dangerous and sometimes hostile mentors. What Dune gets right, and what has put it so high in the pantheon of great science fiction, is the world building. Herbert sets the story tens of thousands of years into the future of humanity and then effectively projects the feeling of deep history over everything in the novel. This is the kind of book that has appendices with more background information; more to the point, it's the kind of book where you may actually read them out of curiosity. Mankind has a vast interstellar empire (Herbert's universe, like Asimov's Foundation universe, admits no aliens) governed by a system akin to the early British monarchy. An emperor rules in balance with the Great Houses, who meet in a sort of parliament. But against both is a third force: the Spacing Guild, who maintains a monopoly over all interstellar travel. (And the Bene Gesserit form an underground, secretive fourth power base.) Herbert plays with vast swaths of time and great forces of history as well as very good epic fantasy and better than nearly all SF I've read. The detailed world-building is equally good. Nearly all of Dune takes place on the desert planet of Arrakis, which has a lovingly-described ecology and local culture built entirely around scarcity of water. (The details of that ecology are much of the plot and mystery of the book, so I won't spoil them further.) While I doubt the precise details hold up to close scientific scrutiny, this is an obvious precursor to the great ecological stories of later SF, such as Kim Stanley Robinson's Mars trilogy. The details all feel right and hang together in satisfying ways, while also generating the great Sand Worms of Arrakis, a key ingredient in several of the best set pieces in the history of SF. This is the sort of book where the fascinating details and discoveries about the world do as much to keep one turning the pages as the plot, although the plot is also satisfyingly twisty and tense. Unfortunately, Dune doesn't get everything right. The amount of mysticism involved is a bit much, and at times the drug-trip mystical experiences of viewpoint characters turn into excessively purple prose and nearly incomprehensible descriptions. Those mystical experiences also involve race and genetic memory, a concept that's just scientific enough to be unbelievable. A few of the other scentific cheats are also rather blantant; for example, Herbert constructs an elaborate, artificial technology of shielding that seems designed primarily as an excuse to add sword combat to a futuristic story, and I have always struggled to suspend disbelief about the way lasers and shields interact in Dune. The Spacer Guild's monopoly on interstellar travel can be explained; their monopoly on local orbital space, or even the high stratosphere, both vital to allow certain things on Arrakis to remain secret, are much more dubious. Herbert mostly doesn't try to explain these things, and as with Star Wars the less explained the cheats are, the better they work as part of the story. But the technological background doesn't hold up against much examination. Worse, for me, is the general quality of the writing. Herbert does some things very well, such as world-building, and avoids awkward infodumps. Characterization and pacing are both fairly solid; he does a good job with Paul and Jessica in particular, and I've always liked the Fremen. But he wants to put the reader in everyone's head, frequently by giving character thoughts as italicized dialogue, and to enable that he uses a perspective that I always find distracting. Most fiction is written in tight third person. This means that the viewpoint character for any given section of the book is referred to in the third person, like all the other characters, but the reader has special access to their thoughts and emotions. We get to know what they're really thinking and feeling, not just the impressions they give to others, while the non-viewpoint characters are shown only from external appearances and the thoughts of the viewpoint character. Some books hold to the same viewpoint character throughout, but more commonly books move between viewpoint characters at scene breaks to provide more angles on the book's events. First person, in which the story is told by a specific character as if they were telling a story or writing it down, is the most common alternative. Third person objective, in which we don't get any special insight into the internal thoughts of any of the characters, is less common but still unsurprising. Dune does not use any of those perspectives. Instead, Dune uses wandering third-person omniscient, in which we get the inner thoughts and emotions of a character in a scene and then a few lines later the inner thoughts and emotions of a different character. This is the sort of thing that may or may not bug you depending on how much you've read, how deep the expectations of perspective are ingrained, and how much you notice perspective. It drives me nuts. I subconsciously align with the viewpoint character of a section, and pay attention to the ways that authors indicate which character will be the viewpoint character at the start of a scene. Herbert's constant flitting from character to character makes me dizzy. We get the verbatim thoughts of everyone almost indiscriminately, making me feel like I'm randomly hopscotching through the scene. For me, this does two things: it hurts my ability to get engrossed in the story, since I'm constantly thrown out of my normal reading mode when the viewpoint unexpectedly shifts, and it makes the writing feel repetitive. One keeps hearing about the same thing from multiple perspectives, and at times the story bogs down in everyone's internal dialogues rather than showing character reactions and letting the reader draw their own conclusions. I think it tries for a cinematic perspective, but ends up making the story feel muddled. The other flaw, which I didn't notice originally but which leaped out at me during this re-read, is that Herbert's world-building uses quite a few stereotypes. The most notorious, and most widely discussed, is of course the Fremen. Herbert draws heavily on Arab and Islamic culture even beyond the obvious similarities of people living in a harsh, arid climate. He borrows some rather loaded terms and cultural markers, such as jihad, to construct a culture of potential religious fanatics. This is not all bad; the Fremen are clearly portrayed as the good guys, which is a refreshing change from more typical current portrayals of Islam. But it becomes clear that they have aligned their entire culture around influences from outside, and the whole plot of Dune can be fairly characterized as an instance of "what these people need is a white man." Paul (and Kynes before him) joins their culture as well, but Paul becomes a better native than the natives, while simultaneously bringing his outside perspective. It's the sort of plot that is more widely noticed today than it would have been in 1965. Another major example of this, and one that I found more blatant, is that Herbert turns the Harkonnen into hissable, one-sided villains and uses some nasty stereotypes to do it. The insane torturer is consistently and repeatedly described as effeminate, fat is used as a marker of moral inferiority and evil, and the primary villain is homosexual and prefers drugged young male slaves. Here too, this sort of characterization short-cut was more common in 1965, but it's not appealing and makes the (already rather camp) scenes set among the Harkonnen even less enjoyable. Less clear-cut is the way women are handled throughout Dune. I do have to give Herbert some credit, particularly for the era in which he was writing. There are powerful female characters in Dune, including both Jessica and Alia, who have their own independent power and successfully pursue their own agendas throughout. The effectively all-female Bene Gesserit is a major political power in the story and is treated by the other players with respect as well as fear. But it's hard not to also notice the general position of women as subservient to men, not only in the general culture of the Great Houses but also in the more positively-portrayed Fremen culture. Indeed, the subservience of women is even worse in Fremen culture, where they're treated like property and where being killed by a woman is a sign of shame. Again, Herbert deserves some credit for doing better than a lot of 1960s fiction, but the sexism fairy has still been at work here. None of these flaws change the fact that Dune is a masterpiece. Herbert brings together history, world building, ecology, politics, and a compelling coming-of-age story about a messiah figure into a fast-paced, sweeping epic with a thoroughly satisfying conclusion. I think they do make it a flawed masterpiece, but it's still one of those SF novels that everyone should read at least once. Sadly, it's also a masterpiece that I think has suffered from its own success in the form of sequels, prequels, and a ton of supporting material. This is one of the problems that truly excellent world building can lead to. Human history is fractal: any specific detail can be examined in more depth and will usually lead (provided that information is available at all) to even more fascinating detail. The best world building conveys that impression of depth. That's what Herbert achieves here with hints, notes, and asides: the sense that galactic history is a vast ediface with the same fractal complexity as real human history. It makes for a compelling background, but it also inspires people to dig into that background and flesh out all of the details the way that we do with human history. But this doesn't actually work; invented history created by one person simply cannot be fractal in the same way. Human history is endlessly complex because it was generated by the complex interactions of many people. Invented history is an illusion that hints at complexity by building the same surface, but one mind, or even a small number of minds, cannot generate the same depth. The result is that if one digs too deep, one removes that convincing surface and ends up with a mundane, simplistic, and unsatisfyingly fake set of events. I think that's what's happened with all of the supporting material that's been written around Dune since its original publication. Dune is of a piece, a single story that's deeply enjoyable on its own terms and leaves the reader with a satisfying impression of complexity. The systemic excavation of that complexity lessens it and reveals too much of the illusion. Yes, I want to know more about the Butlerian Jihad, but that's the point: the wanting is the sign of succesful crafting of imagined history. Reading the definitive account is more likely to leave me unsatisfied than to lead to the recursive curiosity that human history can create. The sequels to Dune written by Herbert himself are, for me, another matter. Some reviewers level the same criticism at them: that Herbert dives too far into background best left unexplored. But they have the advantage of moving forward, telling more of the story set off by Paul, and the end of Dune is a clear setup for a sequel. One of Paul's goals throughout most of the book has been left unaccomplished. I don't think Herbert dove too deep into his creation; rather, my problem with his sequels (all of which I've read, although it's been some years now) is that he took the story in a direction that I actively disliked and found painful to read. Regardless, the general consensus is that the sequels aren't as good as the original, and while Dune doesn't fully resolve its story, it's complete enough that it's possible to stop here. Stopping is the general recommendation, although I still may re-read and review the sequels at some point. Followed by Dune Messiah. Rating: 8 out of 10

23 November 2012

Benjamin Mako Hill: Cultivated Disinterest in Professional Sports

Like many of my friends, I have treated professional sports with cultivated indifference. But a year and a half ago, I decided to become a football fan. Several years ago, I was at a talk by Michael Albert at MIT where he chastised American intellectuals for what he claimed was cultivated disdain of professional sports. Albert suggested that sports reflect the go-to topic for small talk and building rapport across class and context. But he suggested that almost everybody who used the term "working class struggle" was incapable of making small talk with members of the working class because unlike most working class people (and most people in general) educated people systematically cultivate ignorance in sports. Professional sports are deeply popular. In the US, Sunday Night Football is now the most popular television show among women in its time slot and the third most popular television in America among 18-49 year old women. That it is also the most popular television show in general is old news. There are very few things that anywhere near half of Americans have in common. Interest in football is one of them. An enormous proportion of the US population watches the Superbowl each year. I recognized myself in Albert's critique. So I decided to follow a local team. I picked football because it is the most popular sport in America and because their strong revenue sharing system means that either team has a chance to win any given match. My local team is the New England Patriots and I've watched many of the team's games or highlights over the last season and a half. I've also followed a couple football blogs. A year and half in, I can call myself a football fan. And I've learned a few things in the process:
  1. With a little effort, getting into sports is easy. Although learning the rules of a sport can be complicated, sports are popular because people, in general, find them fun to watch. If you watch a few games with someone who can explain the rules, and if you begin to cheer for a team, you will find yourself getting emotionally invested and excited.
  2. Sports really do, as Albert implied, allow one to build rapport and small talk across society. I used to dread the local cab driver who would try to make small talk by mentioning Tom Brady or the Red Sox. No more! Some of these conversations turn into broader conversations about life and politics.
  3. Interest in sports can expand or shrink to fill the time you're willing to give it. It can mean just glancing through the sports sections of the paper and watching some highlights here or there. Or it can turn into a lifestyle.
  4. It's not all great. Football, like most professional sports, is deeply permeated with advertisements, commercialism, and money. Like other sports, it is also violent. I don't think I could ever get behind a fight sport where the goal is to hurt someone else. The machoness and absence of women in the highest levels of most professional sports bothers me deeply.
I've also tried to think a lot about why I, like most of my friends, avoided sports in the past. Disinterest in sports among academics and the highly educated is, in my experience, far from passive. I've heard people almost compete to explain the depth of their ignorance in sports one doesn't even know the rules, one doesn't own a television, one doesn't know the first thing about the game. I did the same thing myself. Bethany Bryson, a sociologist at JMU has shown that increased education is associated with increased inclusiveness in musical taste (i.e., highly educated people like more types of music) but that these people are most likely to reject music that is highly favored by the least educated people. Her paper's title sums up the attitude: "Anything But Heavy Metal". For highly educated folks, it's a sign of cultivation to be eclectic in one's tastes. But to signal to others that you belong in the intellectual elite, it can pay in cultural capital to dislike things, like sports, that are enormously popular among the least educated parts of society. This ignorance among highly educated people limits our ability to communicate, bond, and build relationships across different segments of society. It limits our ability to engage in conversations and build a common culture that crosses our highly stratified and segmented societies. Sports are not politically or culturally unproblematic. But they provide an easy and enjoyable way to build common ground with our neighbors and fellow citizens that transcend social boundaries.

22 November 2012

Ian Wienand: Upgrading Samsung Series 9 Laptop BIOS without Windows

Having recently acquired a Samsung Series 9 laptop, I'd like to update the BIOS. This is proving rather difficult without Windows. The first thing you download from the Samsung website isn't actually BIOS update at all; it's a Windows application which then goes and finds a BIOS update for your machine. Some time with a Windows VM, tcpdump and you'll end up at http://sbuservice.samsungmobile.com/. I'll leave interested parties to play around at that website, but with a little fiddling you can find the BIOS you're looking for. In this case, it is ITEM_20121024_754_WIN_P08AAH.exe which is the latest version, P08AAH. Unfortunately, this still won't run in any useful way -- in a virtual-machine it errors out saying the battery isn't connected. On previous endeavors such as this, running various incarnations of zip on the .exe file has found interesting components, but not in this case. Not to be deterred, some work with strings will yield the following
$ strings ./ITEM_20121024_754_WIN_P08AAH.exe   grep Copyright
inflate 1.2.7 Copyright 1995-2012 Mark Adler
deflate 1.2.7 Copyright 1995-2012 Jean-loup Gailly and Mark Adler
Now this is interesting, because it tells us that the components aren't using some fancy compression scheme we have no hope of figuring out. Checking RFC1952 we see the following header mentioned
Each member has the following structure:
+---+---+---+---+---+---+---+---+---+---+
 ID1 ID2 CM  FLG      MTIME      XFL OS   (more-->)
+---+---+---+---+---+---+---+---+---+---+
We can easily look for that in the file. Expanding the values, what we're looking for is 0x1f 0x8b 0x08 0x0 0x0 0x0 0x0 0x0 0x0 0x0 which represents the compressed stream (the OS byte is 0xb for NTFS which we can just ignore). A quick hack will get that done and give us the offsets we need
$ ./look-for-headers
Found potential header at 455212
Found potential header at 677136
Found potential header at 1293930
Found potential header at 1294980
Found potential header at 2771193
Found potential header at 2779874
Found potential header at 2789142
Found potential header at 2935778
Found potential header at 3078639
Found potential header at 3083039
Found potential header at 3338552
Found potential header at 3339443
Found potential header at 3352440
Found potential header at 3366757
Found potential header at 3391734
Found potential header at 3444255
Found potential header at 3494372
So, we can feed this into a shell script and extract the streams
FILE=ITEM_20121024_754_WIN_P08AAH.exe
for offset in 455212 677136 1293930 1294980 \
    2771193 2779874 2789142 2935778 3078639 \
    3083039 3338552 3339443 3352440 3366757 \
    3391734 3444255 3494372
do
    echo "create $offset.gz"
    dd if=$FILE  bs=1 skip=$offset   zcat > $offset.data
done
We don't have to worry about the end-point, because zcat will handily stop outputting and ignore junk at the end of the stream. That will yield the following output files:
$ md5sum *
c014e900b02ae39d6e574c63154809b2  1293930.data
982747cd5215c7d51fa1fba255557fef  1294980.data
942c6a8332d5dd06d8f4b2a9cb386ff4  2771193.data
d98d2f80b94f70780b46d1f079a38d93  2779874.data
33ce3174b0e205f58c7cedc61adc6f32  2789142.data
1d6851288f40fa95895b2fadee8a8b82  2935778.data
3a0a9b2fa9b40bdf5721068acde351ce  3078639.data
fc166141d06934e9c7238326f18a4e68  3083039.data
8cf7f86f84afa81363d3cef44b257d50  3338552.data
85e09e18fd495599041e13b4197e1746  3339443.data
0c72004eae0131d52817f23faf8a46a7  3352440.data
26347f93721e7ac49134a17104f56090  3366757.data
26973e301da4948a0016765e40700e8c  3391734.data
c8e4675cda3aa367d523a471bb205fba  3444255.data
7bf5ea753d4cc056b9462a02ac51b160  3494372.data
a2c6562f7f43aaa0e636cc1995b7ee3d  455212.data
d5c91b007dac3a5c6317f8a8833f9304  677136.data
65c0dcd1e7e9f8f87a8e9fb706eb8768  ITEM_20121024_754_WIN_P08AAH.exe
$ file *
1293930.data:                     MS-DOS executable
1294980.data:                     data
2771193.data:                     PE32 executable (native) Intel 80386, for MS Windows
2779874.data:                     PE32+ executable (native) x86-64, for MS Windows
2789142.data:                     PE32 executable (console) Intel 80386 (stripped to external PDB), for MS Windows
2935778.data:                     PE32+ executable (console) x86-64 (stripped to external PDB), for MS Windows
3078639.data:                     MS-DOS executable
3083039.data:                     PE32 executable (GUI) Intel 80386, for MS Windows
3338552.data:                     ASCII text, with CRLF line terminators
3339443.data:                     PE32 executable (native) Intel 80386, for MS Windows
3352440.data:                     PE32+ executable (native) x86-64, for MS Windows
3366757.data:                     PE32 executable (DLL) (GUI) Intel 80386, for MS Windows
3391734.data:                     PE32 executable (GUI) Intel 80386, for MS Windows
3444255.data:                     PE32 executable (GUI) Intel 80386, for MS Windows
3494372.data:                     PE32 executable (DLL) (GUI) Intel 80386, for MS Windows
455212.data:                      PE32 executable (DLL) (console) Intel 80386, for MS Windows
677136.data:                      data
ITEM_20121024_754_WIN_P08AAH.exe: PE32 executable (GUI) Intel 80386, for MS Windows
This is where I get a bit stuck. There is clearly a bunch of WinFlash stuff that Wine kind-of recognises, but so far I haven't managed to figure out how I might go about flashing this from a DOS boot disk. So far I know it's a "Phoenix SecureCore Tiano" based BIOS, and there is some indication that there is a UEFI app to flash the BIOS, but unfortunately I didn't set the laptop up with UEFI (note to anyone setting this up; probably a good idea to do that). I believe that if you have Windows, you can halt the install part-way through and see these files extracted in a temp directory. Possibly if we can put some file-names to the blobs it might help. There may also be a much easier way to do this that I'm completely missing. If anything comes up, I'll update this post; hopefully a future Googlenaut who enjoys BIOS hacking will drop a comment... Update 2012-11-27 I'm unfortunately less optimistic about being able to update this. Certainly the files identified above as MS-DOS executable are EFI binaries; however they are both 32-bit binaries. You can easily get an EFI shell up and running by putting a 64-bit EFI shell binary on a FAT formatted USB disk in the efi/boot/bootx64.efi but it will not run either of those binaries, presumably because they're 32-bit. A 32-bit EFI binary doesn't seem to be recognised for boot, so no luck there. A comment also mentioned some issues with Samsung's and UEFI boot in this bug though it's not clear what models are affected. Another comment mentioned binwalk which also does a nice job of finding the embedded zip streams.
$ strings ./1293930.data
...
RSDS
D:\flash\Source\ENV\src\Efi863\edk\phoenix\edk\Sample\Platform\DUET\ldr16\IA32\oemslp20.pdb
oemslp20.dll
InitializeDriver
2n3u3
$ strings ./3078639.data
en-us;zh-Hans
slp20.marker
slp20.size
slp20.offset
slp20.offset2
read marker fail!
Marker file '%s''s size(%d)!=%d
Not a valided OemMarker!
Update image data error.
marker file is updated.
marker file size too big!
Not a valided OemMarker in ROM.
Update image data error.
SLP marker is reserved.
-PROT
Prot flag is found, mProgFlag is true.
Prot flag is not found, mProgFlag is false.
Please specify input file name.
-PROT
slpFile
OEM Activation 2.0 processing
Failed to Locate Flash Support protocol.
...
D:\flash\Source\ENV\src\Efi863\edk\phoenix\edk\Sample\Platform\DUET\ldr16\IA32\SLP20.pdb

21 November 2012

Dirk Eddelbuettel: Rcpp attributes: A simple example 'making pi'

We introduced Rcpp 0.10.0 with a number of very nice new features a few days ago, and the activity on the rcpp-devel mailing list has been pretty responsive which is awesome. But because few things beat a nice example, this post tries to build some more excitement. We will illustrate how Rcpp attributes makes it really easy to add C++ code to R session, and that that code is as easy to grasp as R code. Our motivating example is everybody's favourite introduction to Monte Carlo simulation: estimating . A common method uses the fact the unit circle has a surface area equal to . We draw two uniform random numbers x and y, each between zero and one. We then check for the distance of the corresponding point (x,y) relative to the origin. If less than one (or equal), it is in the circle (or on it); if more than one it is outside. As the first quadrant is a quarter of a square of area one, the area of the whole circle is -- so our first quadrant approximates over four. The following figure, kindly borrowed from Wikipedia with full attribution and credit, illustrates this: Example of simulating pi Now, a vectorized version (drawing N such pairs at once) of this approach is provided by the following R function.
piR <- function(N)  
    x <- runif(N)
    y <- runif(N)
    d <- sqrt(x^2 + y^2)
    return(4 * sum(d < 1.0) / N)
 
And in C++ we can write almost exactly the same function thanks the Rcpp sugar vectorisation available via Rcpp:
#include <Rcpp.h>
using namespace Rcpp;
// [[Rcpp::export]]
double piSugar(const int N)  
    RNGScope scope;		// ensure RNG gets set/reset
    NumericVector x = runif(N);
    NumericVector y = runif(N);
    NumericVector d = sqrt(x*x + y*y);
    return 4.0 * sum(d < 1.0) / N;
 
Sure, there are small differences: C++ is statically typed, R is not. We need one include file for declaration, and we need one instantiation of the RNGScope object to ensure random number draws remain coordinated between the calling R process and the C++ function calling into its (compiled C-code based) random number generators. That way we even get the exact same draws for the same seed. But the basic approach is identical: draw a vector x and vector y, compute the distance to the origin and then obtain the proportion within the unit circle -- which we scale by four. Same idea, same vectorised implementation in C++. But the real key here is the one short line with the [[Rcpp::export]] attribute. This is all it takes (along with sourceCpp() from Rcpp 0.10.0) to get the C++ code into R. The full example (which assumes the C++ file is saved as piSugar.cpp in the same directory) is now:
#!/usr/bin/r
library(Rcpp)
library(rbenchmark)
piR <- function(N)  
    x <- runif(N)
    y <- runif(N)
    d <- sqrt(x^2 + y^2)
    return(4 * sum(d < 1.0) / N)
 
sourceCpp("piSugar.cpp")
N <- 1e6
set.seed(42)
resR <- piR(N)
set.seed(42)
resCpp <- piSugar(N)
## important: check results are identical with RNG seeded
stopifnot(identical(resR, resCpp))
res <- benchmark(piR(N), piSugar(N), order="relative")
print(res[,1:4])
and it does a few things: set up the R function, source the C++ function (and presto: we have a callable C++ function just like that), compute two simulations given the same seed and ensure they are in fact identical -- and proceed to compare the timing in a benchmarking exercise. That last aspect is not even that important -- we end up being almost-but-not-quite twice as fast on my machine for different values of N. The real takeaway here is the ease with which we can get a C++ function into R --- and the new process completely takes care of passing parameters in, results out, and does the compilation, linking and loading. More details about Rcpp attributes are in the new vignette. Now enjoy the .

Update:One somewhat bad typo fixed.

Update:Corrected one background tag.

19 November 2012

Vasudev Kamath: Weekly Log - 17/23 - 112012

The last week was quite productive as I was on vacation and at home town but sadly I couldn't complete this post so again merging the work with this week but this week ain't much productive as I was tired from journey back and didn't get enough time to recover. So here it goes. Debian Related Upstream Related I raised a pull request #8 on Gubbi fixing the Makefile to more organized and introducing xz compression in it. Additionally I removed distribution specific stuffs from Makefile and made it generic. That's all for the two weeks. This week there will be foss.in and we will be having some Debian specific mini conference, including some Debian basics to newbies and some bug squashing if any :-). So more to report next week, till then Cya.

18 November 2012

Jamie McClelland: New Privacy Arguments are Needed

Another decent article on online privacy was published by the NYT this morning. The article documents the collection and centralized aggregation of consumer data online. It ends with a strong message about consumers becoming the "product" - which echoes what Noam Chomsky and Ben Bagdikian have been saying for decades. The problem with this critique is that it doesn't go far enough. Chomsky and Bagdikian didn't say readers are the product of modern journalism and that's bad because nobody wants to be a product. They critiqued the model because it produces terrible journalism and democracy suffers as a result. Similarly, the online trading of audiences and the invasion of our privacy that results has more important implications than "don't be a sucker" which seems to be the gist of the privacy argument these days. Honestly, I don't stay awake at night worrying that Nike knows I'm looking for a pair of sneakers, or even that they know my income. I do stay awake at night worrying that people are tagging my photo on Facebook, which could allow the New York Police Dept to submit a photo of protesters to Facebook and get a list of names and addresses of the people in the photo. Or it could allow the police to track my movements via existing networks of surveillance cameras by matching my image to my name. Would that require a search warrant? How would that impact my trust in my government to know that my movements are being tracked? Or worse, to know they might be tracked but I'll never know if they are or aren't? Many governments (including the US) have sordid histories of infiltrating political organizations and intentionally sowing dissent. Having access to activists' purchasing habits provides a treasure trove of material for this purpose that previously was labor intensive for cops to collect. It also provides leverage during interrogation, a time of heightened emotional vulnerability where the mere mention of a private detail of one's life can have a powerful impact. How can we prevent law enforcement from having access to these details of our lives? And then there's the dragnet. A central database of consumer habits provides a tantalizing collection of information that could be searched for people who fit a profile of a crime. Is solving the crime at hand worth the invasion of privacy that results from people being investigating only based on their consumer habits? How would we measure the impact an investigation like this would have on our confidence in democracy and government? The corporate advertisers promoting this kind of data collection argue that these problems are the price of progress. However, the ideology behind these technology developments is capitalism, not some intrinsic aspect of the Internet or network communications. Historically, the Internet has developed based on a drive toward de-centralization. Email and web sites, the bedrock of Internet activity, are based on these principles: anyone can add their own email server or web server and everyone can seamlessly access it. In contrast, the logic of capitalism as it relates to the Internet, is one of centralization and aggregating data. Facebook and Google want you to use their platforms for all your online activities so they have as complete a picture as possible of what you like and do. As the New York Times article describes, companies like Rubicon want to track all your movements and aggregate that data for profit. We have a choice. We can carefully evaluate our use of corporate services and consider the implications it has not only on our lives but on society as a whole. And, we can choose to use and support services that promote open standards and interoperability that will respect our personal privacy values. Want a practical example? Friendica is software designed to allow you to post status alerts, photos, videos and other information about yourself in a way similar to Facebook. However, it has one big difference: you can host your account on any server running the software anywhere in the world. You can still "friend" people on different servers, but we don't have to all trust the same organization to host it. Furthermore, Friendica allows you to post once and automatically cross-post to your Twitter, Facebook or other similar services. Sound good? You are welcome to try out an account on the May First/People Link install: https://friends.mayfirst.org/.

17 November 2012

Dirk Eddelbuettel: RcppArmadillo 0.3.4.4

A minor bug-fix release 3.4.4 of Armadillo came out upstream a few days ago. RcppArmadillo, our wrapper for R and Armadillo, is now on CRAN with its corresponding version 0.3.4.4. No R level or interface changes were made and the upstream changes are summarized below.
Changes in RcppArmadillo version 0.3.4.4 (2012-11-15)
  • Upgraded to Armadillo release 3.4.4
    • fix for handling complex numbers by sparse matrices
    • fix for minor memory leak by sparse matrices
Courtesy of CRANberries, there is also a diffstat report for 0.3.4.4 relative to 0.3.4.3 As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

15 November 2012

Martin Pitt: python-dbusmock templates

With python-dbusmock you can provide mocks for arbitrary D-BUS services for your test suites or if you want to reproduce a bug. However, when writing actual tests for gnome-settings-daemon etc. I noticed that it is rather cumbersome to always have to set up the skeleton of common services such as UPower. python-dbusmock 0.2 now introduces the concept of templates which provide those skeletons for common standard services so that your code only needs to set up the particular properties and specific D-BUS objects that you need. These templates can be parameterized for common customizations, and they can provide additional convenience methods on the org.freedesktop.DBus.Mock interface to provide more abstract functionality like add a battery . So if you want to pretend you have one AC and a half-charged battery, you can now simply do
  def setUp(self):
     (self.p_mock, self.obj_upower) = self.spawn_server_template('upower',  )
  def test_ac_bat(self):
     self.obj_upower.AddAC('mock_AC', 'Mock AC')
     self.obj_upower.AddChargingBattery('mock_BAT', 'Mock Battery', 50.0, 1200)
Or, if your code is not in Python, use the CLI/D-BUS interface, like in shell:
  # start a fake system bus
  eval  dbus-launch 
  export DBUS_SYSTEM_BUS_ADDRESS=$DBUS_SESSION_BUS_ADDRESS
  # start mock upower on the fake bus
  python3 -m dbusmock --template upower &
  # add devices
  gdbus call --system -d org.freedesktop.UPower -o /org/freedesktop/UPower \
      -m org.freedesktop.DBus.Mock.AddAC mock_ac 'Mock AC'
  gdbus call --system -d org.freedesktop.UPower -o /org/freedesktop/UPower \
      -m org.freedesktop.DBus.Mock.AddChargingBattery mock_bat 'Mock Bat' 50.0 1200
In both cases upower --dump or gnome-power-statistics will show you the expected devices (of course you need to run that within the environment of the fake $DBUS_SYSTEM_BUS_ADDRESS, or run the mock on the real system bus as root). Iftikhar Ahmad contributed a template for NetworkManager, which allows you to easily set up ethernet and wifi devices and wifi access points. See pydoc3 dbusmock.templates.networkmanager for details and the test cases for how this looks like in practice. I just released python-dbusmock 0.2.1 and uploaded the new version to Debian experimental. I will sync it into Ubuntu Raring in a few hours.

14 November 2012

Tanguy Ortolo: Upcoming signing parties in France

A seal I am organizing two signing parties in France:

Lyon, November 17th This will be an informal key signing party: just come with your ID documents and paper strips with your public key fingerprint. The signing party will occur at 2012-11-17 12:15+01:00 in the Maison Pour Tous in Lyon, France.

Paris, November 25th This will be a formally organized key signing party. If you want to participate, you should follow the indications given in the event wiki page. Basically:
  1. send me your public key;
  2. print some copies of your key fingerprint for people that did not register;
  3. print the attendees list I shall send you before the event;
  4. come to the signing party with all that, ID documents and a pen.
The signing party will occurr at 2012-11-25 11:45+01:00 in the pita school in Le Kremlin-Bic tre, France.

Tips To print strips with your public key fingerprint, you can use the utility gpg-key2ps from the package signing-party. If you are a user of the CAcert.org SSL certification authority, you can come to these signing parties with some copies of the accreditation form: it will be an opportunity to get or give CAcert.org accreditation.

12 November 2012

Stefano Zacchiroli: Debian newcomer experience survey

In recent times we have worked quite a bit to improve the NM process, i.e. the process newcomers go through to become members of the Debian Project. As it happens, I've just read Nathan's recent post on his NM experience and I think it is a perfect example of the joining experience we are trying to offer to all newcomers. But examples, be them positive or negative, are only anecdotal. To evaluate a process one needs actual data and someone analyzing them, ideally with a scientific approach. This is why I'm happy to host below a guest blog post by Kevin Carillo, who is doing a pretty thorough scientific study about how newcomers join a wide range of Free Software projects, including KDE, OpenSUSE, GNOME, and Debian, of course! TL;DR: if you started contributing to Debian after January 2010, there's a survey for you; participating will help us improving the NM process even further. Kevin's guest blog post follows.
Newcomer experience in Debian and other FOSS communities - Survey My name is Kevin Carillo. I am a PhD student currently living in Wellington (New Zealand) and I am doing some research on Free/Open Source Software communities. If you have started contributing to the Debian project after January 2010 (within approximately the last 3 years), I would like to kindly request your help. I am interested in hearing from people who are either technical or non-technical contributors, and who have had either positive or negative newcomer experiences. The purpose of the research is to work out how newcomers to a FOSS community become valued sustainable contributors. The survey is online and will be available until Tuesday, 27 November, 2012. Inspiration from Debian New Member Debian is a successful community that keeps attracting new contributors and that relies on a very unique way to handle the integration of new contributors: the New Member process. The idea behind the NM process is that it is some sort of filtering procedure allowing to only retain the individuals who have the potential to become valued sustainable contributors in Debian. Within Debian, there is a lot of enthusiasm and pride around the NM process as it seems to be functioning pretty well but the question is: Is this really enough to ensure that Debian remains a healthy and growing community? How does it compare to the way newcomers are integrated in other large projects such as KDE, GNOME, or in other non-Linux related communities such as Mozilla? I have to admit that the Debian NM process has been among the main sources of inspiration that made me embark in this research project. I have kept being quite impressed when talking to people who had gone through the process as all of them came out of it with a real passion for the project and love for its community. When reflecting on the reasons why the NM process succeeds, I have a feeling it is some instance of ritualized socialization. In other words, barriers and initiation rituals that require some effort from newcomers, generate members with higher commitment and sense of identification towards the Debian community. What do newcomers really experience? The main assumption that motivated this project is that attracting new members has become crucial for a large majority of FOSS communities but this is not a sufficient condition to ensure the success and prosperity of a project. A proportion of a community's newcomers must contribute to the well-being and growth of the community. Keeping all that in mind, FOSS projects have thus to do a good job at "socializing" their newcomers and turning them into 'good' contributors. Doing a good job here means that FOSS projects shall ensure that they help generate citizenship-like behaviors from newcomers by designing appropriate newcomer programs and procedures. FOSS communities rely on a wide array of initiatives to facilitate the integration of newcomers but it seems like the other side of the coin is less understood: What do newcomers really experience? And how does this influence their contributions and actions within a project? How is this study going to help Debian? The data will help gain insights about the experience of newcomers within the Debian community. In addition, it will allow to understand how to design effective newcomer initiatives to ensure that Debian will remain a successful and healthy community. The dataset will be released under a share-alike ODbL license so that Debian contributors can extract as much value as possible from the data. Since this survey also involves other large FOSS projects such as Mozilla, KDE, Gnome, Ubuntu, Gentoo, OpenSUSE, and NetBSD, it will also be possible to compare practices across projects in order to identify what works from what does not work when facilitating the integration of newcomers. About the survey This survey is anonymous. The raw dataset of everything one fills in the survey will be released under the ODbL. Since all the questions but one are optional, one is free to control the amount of information they are giving away about themselves. I expect the survey to take around 20 minutes of your time. If you know members of the Debian community who you think would be interested in completing it, please do not hesitate to let them know about this research. I will post news about my progress with this research, and the results on my blog. Don't hesitate to contact me. --- Kevin Carillo

Next.

Previous.