Back in early 2012 I had been helping with system administration of a number of Debian/Ubuntu-based machines, and the odd Solaris machine, for a couple of years at $DAYJOB. We had a combination of hand-written scripts, documentation notes that we cut n paste d from during installation, and some locally maintained Debian packages for pulling in dependencies and providing some configuration files. As the number of people and machines involved grew, I realized that I wasn t happy with how these machines were being administrated. If one of these machines would disappear in flames, it would take time (and more importantly, non-trivial manual labor) to get its services up and running again. I wanted a system that could automate the complete configuration of any Unix-like machine. It should require minimal human interaction. I wanted the configuration files to be version controlled. I wanted good security properties. I did not want to rely on a centralized server that would be a single point of failure. It had to be portable and be easy to get to work on new (and very old) platforms. It should be easy to modify a configuration file and get it deployed. I wanted it to be easy to start to use on an existing server. I wanted it to allow for incremental adoption. Surely this must exist, I thought. During January 2012 I evaluated the existing configuration management systems around, like CFEngine, Chef, and Puppet. I don t recall my reasons for rejecting each individual project, but needless to say I did not find what I was looking for. The reasons for rejecting the projects I looked at ranged from centralization concerns (single-point-of-failure central servers), bad security (no OpenPGP signing integration), to the feeling that the projects were too complex and hence fragile. I m sure there were other reasons too. In February I started going back to my original needs and tried to see if I could abstract something from the knowledge that was in all these notes, script snippets and local dpkg packages. I realized that the essence of what I wanted was one shell script per machine, OpenPGP signed, in a Git repository. I could check out that Git repository on every new machine that I wanted to configure, verify the OpenPGP signature of the shell script, and invoke the script. The script would do everything needed to get the machine up into an operational stage again, including package installation and configuration file changes. Since I would usually want to modify configuration files on a system even after its initial installation (hey not everyone is perfect), it was natural to extend this idea to a cron job that did git pull , verified the OpenPGP signature, and ran the script. The script would then have to be a bit more clever and not redo everything every time. Since we had many machines, it was obvious that there would be huge code duplication between scripts. It felt natural to think of splitting up the shell script into a directory with many smaller shell scripts, and invoke each shell script in turn. Think of the /etc/init.d/ hierarchy and how it worked with System V initd. This would allow re-use of useful snippets across several machines. The next realization was that large parts of the shell script would be to create configuration files, such as /etc/network/interfaces. It would be easier to modify the content of those files if they were stored as files in a separate directory, an overlay stored in a sub-directory overlay/, and copied into the file system s hierarchy with rsync. The final realization was that it made some sense to run one set of scripts before rsync ing in the configuration files (to be able to install packages or set things up for the configuration files to make sense), and one set of scripts after the rsync (to perform tasks that require some package to be installed and configured). These set of scripts were called the pre-tasks and post-tasks respectively, and stored in sub-directories called pre-tasks.d/ and post-tasks.d/. I started putting what would become Cosmos together during February 2012. Incidentally, I had been using etckeeper on our machines, and I had been reading its source code, and it greatly inspired the internal design of Cosmos. The git history shows well how the ideas evolved even that Cosmos was initially called Eve but in retrospect I didn t like the religious connotations and there were a couple of rewrites on the way, but on the 28th of February I pushed out version 1.0. It was in total 778 lines of code, with at least 200 of those lines being the license boiler plate at the top of each file. Version 1.0 had a debian/ directory and I built the dpkg file and started to deploy on it some machines. There were a couple of small fixes in the next few days, but development stopped on March 5th 2012. We started to use Cosmos, and converted more and more machines to it, and I quickly also converted all of my home servers to use it. And even my laptops. It took until September 2014 to discover the first bug (the fix is a one-liner). Since then there haven t been any real changes to the source code. It is in daily use today. The README that comes with Cosmos gives a more hands-on approach on using it, which I hope will serve as a starting point if the above introduction sparked some interest. I hope to cover more about how to use Cosmos in a later blog post. Since Cosmos does so little on its own, to make sense of how to use it, you want to see a Git repository with machine models. If you want to see how the Git repository for my own machines looks you can see the sjd-cosmos repository. Don t miss its README at the bottom. In particular, its global/ sub-directory contains some of the foundation, such as OpenPGP key trust handling.

Review: Shady Characters, by Keith Houston

Publisher:	W.W. Norton
Copyright:	2013
ISBN:	0-393-34972-1
Format:	Trade paperback
Pages:	250

Subtitled The Secret Life of Punctuation, Symbols & Other Typographical Marks, Shady Characters is one of those delightfully quirky books that examines the history of something you would not normally connect to history. It's an attempt to document, and in some cases reconstruct, the history of several specific punctuation marks. If you've read and enjoyed Lynn Truss's Eats, Shoots & Leaves, this is a near-perfect complement, focusing on more obscure marks and adding in a more detailed and coherent history. Houston has some interest in the common and quotidian punctuation marks, with chapters on the hyphen, dash, and quotation mark, but he avoids giving a full-chapter treatment to the most obvious periods and commas. (Although one learns quite a bit about them as asides in other histories.) The rest of the book focuses on the newly-popular (the at symbol), the recognizable but less common (the hash mark, the asterisk and dagger symbols used for footnotes, and the ampersand), and the historical but now obscure (the pilcrow or paragraph mark, and the manicule or pointing finger). He even devotes two chapters to unsuccessful invented punctuation: the interrobang and the long, failed history of irony and sarcasm punctuation. And this is an actual history, not just a collection of anecdotes and curious facts. Houston does a great job of keeping the text light, engaging, and readable, but he's serious about his topic. There are many reproductions of ancient documents showing early forms of punctuation, several serious attempts to adjudicate between competing theories of origin, a few well-marked and tentative forays into guesswork, and an open acknowledgment of several areas where we simply don't know. Along the way, the reader gets a deeper feel for the chaos and arbitrary personal stylistic choices of the days of hand-written manuscripts and the transformation of punctuation by technology. So much of what we now use in punctuation was shaped and narrowed by the practicalities of typesetting. And then modern technology revived some punctuation, such as the now-ubiquitous at sign, or the hash mark that graces every telephone touchpad. I think my favorite part of this history was using punctuation as perspective from which to track the changing relationship between people and written material. It's striking how much early punctuation was primarily used for annotations and commentary on the text, as opposed to making the text itself more readable. Much early punctuation was added after the fact, and then slowly was incorporated into the original manuscript, first via recopying and then via intentional authorial choice. Texts started including their own pre-emptive commentary, and brought the corresponding marks along with the notes. And then printing forced a vast simplification and standardization of punctuation conventions. Full compliments to W.W. Norton, as well, for putting the time and effort into making the paper version of this book a beautiful artifact. Punctuation is displayed in red throughout when it is the focus of the text. Endnotes are properly split from footnotes, with asides and digressions presented properly at the bottom of the page, and numbered endnotes properly reserved solely for citations. There is a comprehensive index, list of figures, and a short and curated list of further readings. I'm curious how much of the typesetting care was preserved in the ebook version, and dubious that all of it would have been possible given the current state of ebook formatting. Typesetting, obscure punctuation, and this sort of popular history of something quotidian are all personal interests of mine, so it's unsurprising I found this book delightful. But it's so well-written and engaging that I can recommend it even to people less interested in those topics. The next time you're in the mood for learning more about a corner of human history that few people consider, I recommend Shady Characters to your attention. Rating: 8 out of 10

Review: Magic's Pawn, by Mercedes Lackey

Series:	Last Herald Mage #1
Publisher:	DAW
Copyright:	June 1989
ISBN:	0-88677-352-0
Format:	Mass market
Pages:	349

Continuing my (very slow) re-read of Mercedes Lackey's Valdemar series. I've read all these books before (this series several times, in fact), but long before I started writing reviews. Vanyel is a legendary hero by the time of Talia (the hero of the first Valdemar series, starting with Arrows of the Queen). Talia was reading stories about him at the start of her story: the mage Herald who defended Valdemar against its enemies in the distant past. This is the first book of the trilogy that tells his story, starting (as Lackey's books often do) from rather inauspicious beginnings. Vanyel is the son of a border lord, and about as poorly suited for it as possible. He's small, pretty, entirely uninterested in the hammer-and-tongs sword fighting the arms master wants to teach him, and also gay, not that he has any idea that's even something that exists and has a name. His only ally was his sister, who is now off serving in Valdemar's military. Most of his life is spent hiding, feeling utterly lost and out of place, and wishing he could be a Bard. This is a Valdemar story, so if you're familiar with the series, you know what's coming next. What might come as a surprise is that "next" is quite some time into the story. Vanyel does not get rescued from his situation by a Companion. Instead, his father decides to exile him to the capital under the care of his aunt, who he's only met once and who didn't think highly of him. He expects to be even more miserable, and shuts emotionally down in anticipation. This is one of those books that I remembered as being better than it actually was, and one of the reasons why I enjoyed it less than I expected is that far too much of this book is devoted to describing Vanyel's mental state. This usually involves various elaborate emo analogies (which can be a failing of this series in general), and it's quite hard to maintain sympathy for Vanyel. Lackey gets us to feel for him at the start of the book, when he's really in a bad situation. But once he gets to the capital, he's his own worst enemy, and it's hard to avoid the desire to shake him. I also didn't remember just how much I disliked Tylendel. I'm sure it will come as a surprise to no one that Vanyel eventually meets someone who draws him out of his shell and gives him a reason to want to live. Unfortunately, that person, despite a positive surface impression, is self-obsessed, unstable, and not above manipulating Vanyel into actions that are so obviously catastrophic that it makes one want to yell at the book. I disliked this part of the book so much that even a Companion's choosing was overshadowed. The book does get a bit better after that truly awful middle, but it never hits the emotional stride that other Valdemar books hit. Lackey does introduce the Tayledras, which will be hugely important in later books in this series (and who are some of my favorite characters), but there too I prefer their later appearances. Vanyel's inability to see a good thing when it hits him in the face, punctuated by being occasionally cruel to the people who try to help him, makes it quite hard to enjoy his slow path to becoming a better person. I remember really liking this trilogy, so I think it gets better. But I also remembered liking Vanyel's claiming, and it didn't do much for me on a re-read. I can only recommend this one as a necessary preface to the later books in the trilogy, and expect to need a high tolerance for constant woe-is-me despair. Followed by Magic's Promise. Rating: 5 out of 10

Review: The Three-Body Problem, by Cixin Liu & Ken Liu

Series:	Three-Body Problem #1
Author:	Liu Cixin ( )
Translator:	Ken Liu
Publisher:	Tor
Copyright:	2006, 2014
Printing:	November 2014
ISBN:	1-4668-5344-1
Format:	Kindle
Pages:	399

Liu Cixin ( ) is one of the best-known and most popular SF authors in China, but, due to the paucity of translated SF in the English-speaking world, was largely unknown to English-speaking SF readers. That made this translation by Ken Liu (no relation) highly anticipated: a window into a large world of SF by authors most of us have never read. And Ken Liu is an excellent writer in his own right (see, for instance, "The Literomancer" or "Altogether Elsewhere, Vast Herds of Reindeer"), which was a hopeful sign for a good translation. Indeed, the translation is very good. One can tell that the book comes from a different literary tradition, but primarily from unusual (to US readers, at least) emotional focal points for the characters and lots of Chinese history that the author feels little need to explain. (Ken Liu adds translator notes for the bits most likely to lose non-Chinese audiences.) I found the characters a bit odd and occasionally hard to identify with, but I didn't mind: it added to the feeling of exploring a slightly different literary tradition with different character conventions. Unfortunately, the end of the book went entirely off the rails, at least for me. The Three-Body Problem opens during the Cultural Revolution, with the murder of Ye Wenjie's father, a physics professor, and the suicide of her advisor and friend. Two years later, she's at a work camp in inner Mongolia, cutting trees, numb, keeping her head down. But a traveling journalist smuggles in a secret copy of Silent Spring and gives it to her. The book both hardens her own belief in the evil of humanity and gets her into deeper trouble when she agrees to recopy a draft of a letter to the central leadership about the environmental devastation caused by the logging. Ye Wenjie is forced to choose permanent exile to an experimental radar facility on a remote mountain in Mongolia. Ye Wenjie is vital to the slowly-developing plot of the book, but this is the last the reader sees of her for some time. Part II jumps to present time and to a different protagonist: Wang Miao, a distinguished nanotechnology researcher who is recruited by the PLA and the police into a secret war room. Distinguished scientists are killing themselves in frighteningly large numbers, including one that Wang Miao had a crush on. She left a suicide note before overdosing on sleeping pills, one that says only, cryptically, that physics doesn't exist. The government is desperate to figure out what's going on. The "radar" facility and the suicide of scientists are linked, of course, but it takes much of the book for Wang Miao and his colleagues to piece together how. Key clues come in the form of an odd puzzle virtual reality MMORPG that becomes a popular sensation. In it, players find themselves on an alien world whose suns rise and set at strange and unpredictable intervals: sometimes far too far away and horribly cold, sometimes far too close and destructively hot, and sometimes not seen for long periods of time. The players can somehow dehydrate themselves and the rest of the population to weather the worst seasons, but life is a constant struggle against apparently unpredictable elements. Despite that, players slowly find ways to build civilizations and attempt to predict the strange cycles of heat and cold. It takes the characters a long time to understand what's happening. Readers will probably be much faster, since they have out-of-band information the characters don't. At first, this game seems entirely unrelated to the suicides of the scientists or, for that matter, to Ye Wenjie, who is easy to almost forget. But they all weave together in a way that's almost compelling. I say almost because the motivations of the characters are quite tangled and interesting, but the science is... awful. Those who have read my other book reviews know that I don't insist on hard science in my SF. Usually I'm willing to roll with it, and I can read past even obvious scientific nonsense. But science is critical to the plot of this book; some very specific actions, with a detailed in-book scientific justifications, are the underpinning of the entire plot. And they make utter nonsense of physics. I've rarely had science in a novel destroy my suspension of disbelief this thoroughly. Physics isn't even alone: biology, chemistry, computing, and astronomy are all in for it as Liu reveals the causes of the book's events. I wanted to be engaged by the conflict of ethics and philosophy that drive the plot, and some of that conflict is quite interesting. But every other page would have another colossal pile of scientific nonsense that would throw me out of the book again, usually in the service of a set piece that missed grand and landed on absurdly silly. Be warned that this book also ends on a bit of a cliff-hanger, and there are two more books in the series. The second has been released in a translated edition, but with a different translator, which is disappointing (Ken Liu did an excellent job); the third isn't out yet. If you can get past the awful science, The Three-Body Problem is doing some interesting things. The characters are a bit wooden and forgettable, but the philosophical problems they encounter are ones I don't see often in western SF. This a refreshing change, and I'm interested enough that I'm still considering picking up the sequels. But the science hurts the book so badly. Followed by The Dark Forest. Rating: 6 out of 10

A few days ago, the French equivalent of Hacker News, called "Le Journal du Hacker", interviewed me about my work on OpenStack, my job at Red Hat and my self-published book The Hacker's Guide to Python. I've spent some time translating it into English so you can read it if you don't understand French! I hope you'll enjoy it.

Hi Julien, and thanks for participating in this interview for the Journal du Hacker. For our readers who don't know you, can you introduce you briefly?

You're welcome! My name is Julien, I'm 31 years old, and I live in Paris. I now have been developing free software for around fifteen years. I had the pleasure to work (among other things) on Debian, Emacs and awesome these last years, and more recently on OpenStack. Since a few months now, I work at Red Hat, as a Principal Software Engineer on OpenStack. I am in charge of doing upstream development for that cloud-computing platform, mainly around the Ceilometer, Aodh and Gnocchi projects.

Being myself a system architect, I follow your work in OpenStack since a while. It's uncommon to have the point of view of someone as implied as you are. Can you give us a summary of the state of the project, and then detail your activities in this project?

The OpenStack project has grown and changed a lot since I started 4 years ago. It started as a few projects providing the basics, like Nova (compute), Swift (object storage), Cinder (volume), Keystone (identity) or even Neutron (network) who are basis for a cloud-computing platform, and finally became composed of a lot more projects. For a while, the inclusion of projects was the subject of a strict review from the technical committee. But since a few months, the rules have been relaxed, and we see a lot more projects connected to cloud-computing joining us. As far as I'm concerned, I've started with a few others people the Ceilometer project in 2012, devoted to handling metrics of OpenStack platforms. Our goal is to be able to collect all the metrics and record them to analyze them later. We also have a module providing the ability to trigger actions on threshold crossing (alarm). The project grew in a monolithic way, and in a linear way for the number of contributors, during the first two years. I was the PTL (Project Technical Leader) for a year. This leader position asks for a lot of time for bureaucratic things and people management, so I decided to leave my spot in order to be able to spend more time solving the technical challenges that Ceilometer offered. I've started the Gnocchi project in 2014. The first stable version (1.0.0) was released a few months ago. It's a timeseries database offering a REST API and a strong ability to scale. It was a necessary development to solve the problems tied to the large amount of metrics created by a cloud-computing platform, where tens of thousands of virtual machines have to be metered as often as possible. This project works as a standalone deployment or with the rest of OpenStack. More recently, I've started Aodh, the result of moving out the code and features of Ceilometer related to threshold action triggering (alarming). That's the logical suite to what we started with Gnocchi. It means Ceilometer is to be split into independent modules that can work together with or without OpenStack. It seems to me that the features provided by Ceilometer, Aodh and Gnocchi can also be interesting for operators running more classical infrastructures. That's why I've pushed the projects into that direction, and also to have a more service-oriented architecture (SOA)

I'd like to stop for a moment on Ceilometer. I think that this solution was very expected, especially by the cloud-computing providers using OpenStack for billing resources sold to their customers. I remember reading a blog post where you were talking about the high-speed construction of this brick, and features that were not supposed to be there. Nowadays, with Gnocchi and Aodh, what is the quality of the brick Ceilometer and the programs it relies on?

Indeed, one of the first use-case for Ceilometer was tied to the ability to get metrics to feed a billing tool. That's now a reached goal since we have billing tools for OpenStack using Ceilometer, such as CloudKitty. However, other use-cases appeared rapidly, such as the ability to trigger alarms. This feature was necessary, for example, to implement the auto-scaling feature that Heat needed. At the time, for technical and political reasons, it was not possible to implement this feature in a new project, and the functionality ended up in Ceilometer, since it was using the metrics collected and stored by Ceilometer itself. Though, like I said, this feature is now in its own project, Aodh. The alarm feature is used since a few cycles in production, and the Aodh project brings new features on the table. It allows to trigger threshold actions and is one of the few solutions able to work at high scale with several thousands of alarms. It's impossible to make Nagios run with millions of instances to fetch metrics and triggers alarms. Ceilometer and Aodh can do that easily on a few tens of nodes automatically. On the other side, Ceilometer has been for a long time painted as slow and complicated to use, because its metrics storage system was by default using MongoDB. Clearly, the data structure model picked was not optimal for what the users were doing with the data. That's why I started Gnocchi last year, which is perfectly designed for this use case. It allows linear access time to metrics (O(1) complexity) and fast access time to the resources data via an index. Today, with 3 projects having their own perimeter of features defined and which can work together Ceilometer, Aodh and Gnocchi finally erased the biggest problems and defects of the initial project.

To end with OpenStack, one last question. You're a Python developer for a long time and a fervent user of software testing and test-driven development. Several of your blogs posts point how important their usage are. Can you tell us more about the usage of tests in OpenStack, and the test prerequisites to contribute to OpenStack?

I don't know any project that is as tested on every layer as OpenStack is. At the start of the project, there was a vague test coverage, made of a few unit tests. For each release, a bunch of new features were provided, and you had to keep your fingers crossed to have them working. That's already almost unacceptable. But the big issue was that there was also a lot of regressions, et things that were working were not anymore. It was often corner cases that developers forgot about that stopped working. Then the project decided to change its policy and started to refuse all patches new features or bug fix that would not implement a minimal set of unit tests, proving the patch would work. Quickly, regressions were history, and the number of bugs largely reduced months after months. Then came the functional tests, with the Tempest project, which runs a test battery on a complete OpenStack deployment. OpenStack now possesses a complete test infrastructure, with operators hired full-time to maintain them. The developers have to write the test, and the operators maintain an architecture based on Gerrit, Zuul, and Jenkins, which runs the test battery of each project for each patch sent. Indeed, for each version of a patch sent, a full OpenStack is deployed into a virtual machine, and a battery of thousands of unit and functional tests is run to check that no regressions are possible. To contribute to OpenStack, you need to know how to write a unit test the policy on functional tests is laxer. The tools used are standard Python tools, unittest for the framework and tox to run a virtual environment (venv) and run them. It's also possible to use DevStack to deploy an OpenStack platform on a virtual machine and run functional tests. However, since the project infrastructure also do that when a patch is submitted, it's not mandatory to do that yourself locally.

The tools and tests you write for OpenStack are written in Python, a language which is very popular today. You seem to like it more than you have to, since you wrote a book about it, The Hacker's Guide to Python, that I really enjoyed. Can you explain what brought you to Python, the main strong points you attribute to this language (quickly) and how you went from developer to author?

I stumbled upon Python by chance, around 2005. I don't remember how I hear about it, but I bought a first book to discover it and started toying with that language. At that time, I didn't find any project to contribute to or to start. My first project with Python was rebuildd for Debian in 2007, a bit later. I like Python for its simplicity, its object orientation rather clean, its easiness to be deployed and its rich open source ecosystem. Once you get the basics, it's very easy to evolve and to use it for anything, because the ecosystem makes it easy to find libraries to solve any kind of problem. I became an author by chance, writing blog posts from time to time about Python. I finally realized that after a few years studying Python internals (CPython), I learned a lot of things. While writing a post about the differences between method types in Python which is still one of the most read post on my blog I realized that a lot of things that seemed obvious to me where not for other developers. I wrote that initial post after thousands of hours spent doing code reviews on OpenStack. I, therefore, decided to note all the developers pain points and to write a book about that. A compilation of what years of experience taught me and taught to the other developers I decided to interview in the book.

I've been very interested by the publication of your book, for the subject itself, but also the process you chose. You self-published the book, which seems very relevant nowadays. Is that a choice from the start? Did you look for an editor? Can you tell use more about that?

I've been lucky to find out about others self-published authors, such as Nathan Barry who even wrote a book on that subject, called Authority. That's what convinced me it was possible and gave me hints for that project. I've started to write in August 2013, and I ran the firs interviews with other developers at that time. I started to write the table of contents and then filled the pages with what I knew and what I wanted to share. I manage to finish the book around January 2014. The proof-reading took more time than I expected, so the book was only released in March 2014. I wrote a complete report on that on my blog, where I explain the full process in detail, from writing to launching. I did not look for editors though I've been proposed some. The idea of self-publishing really convince me, so I decided to go on my own, and I have no regret. It's true that you have to wear two hats at the same time and handle a lot more things, but with a minimal audience and some help from the Internet, anything's possible! I've been reached by two editors since then, a Chinese and Korean one. I gave them rights to translate and publish the books in their countries, so you can buy the Chinese and Korean version of the first edition of the book out there. Seeing how successful it was, I decided to launch a second edition in Mai 2015, and it's likely that a third edition will be released in 2016.

Nowadays, you work for Red Hat, a company that represents the success of using Free Software as a commercial business model. This company fascinates a lot in our community. What can you say about your employer from your point of view?

It only has been a year since I joined Red Hat (when they bought eNovance), so my experience is quite recent. Though, Red Hat is really a special company on every level. It's hard to see from the outside how open it is, and how it works. It's really close to and it really looks like an open source project. For more details, you should read The Open Organization, a book wrote by Jim Whitehurst (CEO of Red Hat), which he just published. It describes perfectly how Red Hat works. To summarize, meritocracy and the lack of organization in silos is what makes Red Hat a strong organization and puts them as one of the most innovative company. In the end, I'm lucky enough to be autonomous for the project I work on with my team around OpenStack, and I can spend 100% working upstream and enhance the Python ecosystem.

Like each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In August, 71.50 work hours have been dispatched among 7 paid contributors. Their reports are available:

• Ben Hutchings did 15 hours.
• Guido G nther did 4 hours.
• Mike Gabriel was assigned 8 hours but did not do them. He will catch up in September.
• Rapha l Hertzog did 6.5 hours.
• Santiago Ruano Rinc n did 17 hours.
• Scott Kitterman did 4 hours.
• Thorsten Alteholz did 17 hours.

Evolution of the situation September is stable compared to August (71.50 hours per month) and has not caught up back to the level of July as I hoped. Again it s because 2 sponsors were not able to pay their renewal invoice on time (one of last month paid, but another bigger sponsor failed this month). Those sponsors will continue to support us and I would like to be able to say that things will be back to normal next month, but I can t say it since we have also been informed of the (hopefully temporary) defection of another bronze sponsor that will affect us next month. Fortunately there are also good news, we have 3 new sponsors in the pipe (2 silver, 1 platinum) who shall join the project soon. And Blablacar increased their support from Silver to Gold (from 4h/month to 8h/month). But we still need more support in particular since we would like to commit to support virtualization related packages in Wheezy: that s clearly an objective for us. I recently published the summary of the work session held during DebConf 15 in Heidelberg (video recording). It would be really nice if we could get closer to the goal of funding a full-time position. In terms of security updates waiting to be handled, the situation is close to last month: the dla-needed.txt file lists 18 packages awaiting an update (2 less than last month), the list of open vulnerabilities in Squeeze shows about 30 affected packages in total (8 more than last month). Thanks to our sponsors

• Gold sponsors:
• The Positive Internet (for 15 months already)
• Blablacar (for 14 months already)
• Linode LLC (for 4 months already)
• Silver sponsors:
• David Ayers IntarS Austria (for 15 months already)
• Domeneshop AS (for 14 months already)
• Universit Lille 3 (for 14 months already)
• Trollweb Solutions (for 12 months already)
• Gandi SAS (for 9 months already)
• University of Luxembourg (for 6 months already)
• Rentabiliweb Group (for 4 months already)
• Bronze sponsors:
• Offensive Security (for 15 months already)
• Seznam.cz, a.s. (for 15 months already)
• Evolix (for 14 months already)
• Freeside Internet Service (for 14 months already)
• MyTux (for 14 months already)
• Linuxhotel GmbH (for 12 months already)
• Intevation GmbH (for 11 months already)
• Daevel SARL (for 10 months already)
• FOSSter (for 10 months already)
• Bitfolk LTD (for 9 months already)
• Megaspace Internet Services GmbH (for 9 months already)
• Gree, Inc. (for 8 months already)
• Greenbone Networks GmbH (for 8 months already)
• NUMLOG (for 8 months already)
• WinGo AG (for 7 months already)
• Ecole Centrale de Nantes LHEEA (for 4 months already)
• Sig-I/O

No comment Liked this article? Click here. My blog is Flattr-enabled.

Review: Let's Pretend This Never Happened, by Jenny Lawson

Publisher:	Berkley
Copyright:	2012, 2013
Printing:	March 2013
ISBN:	0-425-26101-8
Format:	Trade paperback
Pages:	366

Let's Pretend This Never Happened, subtitled (A Mostly True Memoir), is the closest that I've ever found to the book form of a stand-up comedy routine. Lawson grew up in rural Texas with a taxidermist father, frequent contact with animals in various forms of distress, an undiagnosed anxiety disorder, and a talent for creatively freaking out about things that never occur to anyone else. But, more importantly, she has a talent for putting down on paper the random thoughts that go through her head so the rest of us can read them. Not to mention excellent comic timing and absolute mastery of the strangely relevant digression. It's always tricky to review comedy. I think tastes differ more wildly in this genre than any other. Things some people find hilarious others will find offensive or just boring. That may be particularly true of Lawson, who, similar to some of the best stand-up comics, specializes in taking off filters and saying all sorts of offensive things that people might think but not say. This kind of comedy is a knife's edge, since it can easily turn into punching down. Lawson avoids this (rather well, in my opinion) by making herself the punch line of most of the jokes. A pretty typical paragraph of the book, so that you know the sort of thing that you're in for:

The following is a series of actual events pulled form my journal that led me to believe that our home was possessed by demons and/or built over an Indian burial ground. (Also, please note that the first part of this chapter actually happens just before the previous chapter, and the last part of it happens just after it. This could be viewed as "clunky and awkward," but I prefer to think of it as "intellectually challenging and chronologically surreal. Like if Memento was a book. About dead dogs and vaginas and puppets made out of squirrel corpses." You can feel free to use that quote if you're reviewing this chapter, or if you're a student and your teacher asks you, "What was the author trying to say here?" That was it. That's what I was trying to say. That and "Use condoms if you're going to have sex, for God's sake. There are a lot of skanks out there." That's not really covered in this book, but it's still good advice.)

That has a little bit of everything this book had for me: Lawson's somewhat surreal worries, the extended digression, a rhythm that's quite compelling once you start reading it, random uncomfortable topics, and the occasional miss that I don't find funny (the last few sentences). It's all mixed together in a slightly breathless rush of narrative momentum. For more samples, Lawson's writing started as a blog and she's still actively blogging, so you can get a good advance sample by reading some of The Bloggess. Her tone there matches the book closely. What makes this book more than only comedy is that Lawson is very open about her struggles with mental illness (anxiety and depression). A lot of the humor comes from "this is the ridiculous nonsense that my brain throws out on a regular basis" and inviting you to laugh along with her, but the undertone is use of humor as a coping mechanism to deal with anxiety spirals. And alongside that coping mechanism is an open-hearted message of "you are not the only person to have completely irrational reactions to the world please laugh along with mine and feel better about yours." Due to that, the best comparison I can make to another book I've read is to Allie Brosh's Hyperbole and a Half. Brosh is more serious in places, more analytical, and a bit better at generalizing to experiences the reader can identify with. (And, of course, more graphical.) Lawson is more madcap, a bit more manic, and focused on absurd situations that don't normally happen to people. I loved this book from beginning to end, and it had me laughing out-loud in multiple places. Despite being a collection of disconnected stories, it has a rhythm and flow that kept me reading. Some books of this kind are best read in small segments with a break between, but I devoured Let's Pretend This Never Happened in large chunks (and had to be careful about reading it in public and laughing too loudly). Check first whether the sense of humor works for you, but if it does, highly recommended. Rating: 9 out of 10

during the last weeks, I spent time mostly with RC bug prevention but I at least managed to also fix a couple of actual RC bugs:

• #777854 src:freecraft: "freecraft: ftbfs with GCC-5"
  add patch from Nicholas Luedtke/HP, QA upload.
• #778025 src:netrek-client-cow: "netrek-client-cow: ftbfs with GCC-5"
  add patch from Thavatchai Makphaibulchoke/HP, upload to DELAYED/2
• #778058 src:percona-xtradb-cluster-galera-2.x: "percona-xtradb-cluster-galera-2.x: ftbfs with GCC-5"
  add patch from alexander balderson/HP, upload to DELAYED/2
• #780206 src:squashfs-tools: "fix ftbfs with GCC 5 on armhf"
  apply patch from Ubuntu/Matthias Klose, upload to DELAYED/2
• #789093 libapache2-authcookie-perl: "libapache2-authcookie-perl: FTBFS: test failures"
  update patch (pkg-perl)
• #790285 src:stdsyslog: "stdsyslog: FTBFS with glibc 2.21 and gcc-5"
  sponsor maintainer upload
• #795202 src:libcatalyst-perl: "FTBFS: t/aggregate/live_engine_request_parameters.t fails"
  upload new upstream release (pkg-perl)
• #796315 src:libflickr-upload-perl: "libflickr-upload-perl: FTBFS: t/failupload.t (Wstat: 65280 Tests: 2 Failed: 0)"
  upload new upstream release (pkg-perl)
• #796537 src:libcgi-simple-perl: "libcgi-simple-perl: Missing Build-Depends on libio-stringy-perl"
  add missing build dependency (pkg-perl)
• #797581 libmojolicious-plugin-i18n-perl: "libmojolicious-plugin-i18n-perl: FTBFS: t/i18n_url_for.t fails"
  upload new upstream release (pkg-perl)
• #797584 libmojolicious-plugin-cgi-perl: "libmojolicious-plugin-cgi-perl: FTBFS: test failures"
  upload new upstream release (pkg-perl)
• #797966 syncevolution-common: "syncevolution-common: uninstallable due to libsynthesis0 dependency"
  sponsor maintainer upload
• #798790 libmoosex-clone-perl: "libmoosex-clone-perl: missing dependency on libnamespace-autoclean-perl"
  update runtime dependencies (pkg-perl)

Xperia X10 My first Android device was The Sony Ericsson Xperia X10i [1]. One of the reasons I chose it was for the large 4 screen, nowadays the desirable phones (the ones that are marketed as premium products) are all bigger than that (the Galaxy S6 is 5.1 ) and even the slightly less expensive phones are bigger. At the moment Aldi is advertising an Android phone with a 4.5 screen for $129. But at the time there was nothing better in the price range that I was willing to pay. I devoted a lot of my first review to the default apps for SMS and Email. Shortly after that I realised that the default email app is never going to be adequate (I now use K9 mail) and the SMS app is barely adequate (but I mostly use instant messaging). I ve got used to the fact that most apps that ship with an Android device are worthless, the camera app and the app to make calls are the only built in apps I regularly use nowadays. In the bug list from my first review the major issue was lack of Wifi tethering which was fixed by an update to Android 2.3. Unfortunately Android 2.3 ran significantly more slowly which decreased the utility of the phone. The construction of the phone is very good. Over the last 2 years the 2 Xperia X10 phones I own have been on loan to various relatives, many of whom aren t really into technology and can t be expected to take good care of things. But they have not failed in any way. Apart from buying new batteries there has been no hardware failure in either phone. While 2 is a small sample size I haven t see any other Android device last nearly as long without problems. Unfortunately I have no reason to believe that Sony has continued to design devices as well. The Xperia X10 phones crash more often than most Android phones with spontaneous reboots being a daily occurrence. While that is worse than any other Android device I ve used it s not much worse. My second review of the Xperia X10 had a section about ways of reducing battery use [2]. Wow, I d forgotten how much that sucked! When I was last using the Xperia X10 the Life360 app that my wife and I use to track each other was taking 15% of the battery, on more recent phones the same app takes about 2%. The design of modern phones seems to be significantly more energy efficient for background tasks and the larger brighter displays use more energy instead. My father is using one of the Xperia phones now, when I give him a better phone to replace it I will have both as emergency Wifi access points. They aren t useful for much else nowadays. Samsung Galaxy S In my first review of the Galaxy S I criticised it for being thin, oddly shaped, and slippery [3]. After using it for a while I found the shape convenient as I could easily determine the bottom of the phone in my pocket and hold it the right way up before looking at it. This is a good feature for a phone that s small enough to rotate in my pocket the Samsung Galaxy Note series of phones is large enough to not rotate in a pocket. In retrospect I think that being slippery isn t a big deal as almost everyone buys a phone case anyway. But it would still be better for use on a desk if the bulge was at the top. I wrote about my Galaxy S failing [4]. Two of my relatives had problems with those phones too. Including a warranty replacement I ve seen 4 of those phones in use and only one worked reliably. The one that worked reliably is now being used by my mother, it s considerably faster than the Xperia X10 because it has more RAM and will probably remain in regular use until it breaks. CyanogenMod I tried using CyanogenMod [5]. The phone became defective 9 months later so even though CyanogenMod is great I don t think I got good value for the amount of time spent installing it. I haven t tried replacing the OS of an Android phone since then. I really wish that they would start manufacturing phones that can have the OS replaced as easily as a PC. Samsung Galaxy S3 and Wireless Charging The Galaxy S3 was the first phone I owned which competes with phones that are currently on sale [6]. A relative bought one at the same time as me and her phone is running well with no problems. But my S3 had some damage to it s USB port which means that the vast majority of USB cables don t charge it (only Samsung cables can be expected to work). After I bought the S3 I bought a Qi wireless phone charging device [7]. One of the reasons for buying that is so if a phone gets a broken USB port then I can still use it. It s ironic that the one phone that had a damaged USB port also failed to work correctly with the Qi card installed. The Qi charger is gathering dust. One significant benefit of the S3 (and most Samsung phones) is that it has a SD socket. I installed a 32G SD card in the S3 and now one of my relatives is happily using it as a media player. Nexus 4 I bought a Nexus 4 [8] for my wife as she needed a better phone but didn t feel like paying for a Galaxy S3. The Nexus 4 is a nice phone in many ways but the lack of storage is a serious problem. At the moment I m only keeping it to use with Google Cardboard, I will lend it to my parents soon. In retrospect I made a mistake buying the Nexus 4. If I had spent a little more money on another Galaxy S3 then I would have had a phone with a longer usage life as well as being able to swap accessories with my wife. The Nexus 4 seems reasonably solid, the back of the case (which is glass) broke on mine after a significant impact but the phone continues to work well. That s a tribute to the construction of the phone and also the Ringke Fusion case [9]. Generally the Nexus 4 is a good phone so I don t regret buying it. I just think that the Galaxy S3 was a better choice. Galaxy Note 2 I got a Samsung Galaxy Note 2 in mid 2013 [10]. In retrospect it was a mistake to buy the Galaxy S3, the Note series is better suited to my use. If I had known how good it is to have a larger phone I d have bought the original Galaxy Note when it was first released. Generally everything is good about the Note 2. While it only has 16G of storage (which isn t much by today s standards) it has an SD socket to allow expansion. It s currently being used by a relative as a small tablet. With a 32G SD card it can fit a lot of movies. Bluetooth Speakers I received Bluetooth speakers in late 2013 [11]. I was very impressed by them but ended up not using them for a while. After they gathered dust for about a year I started using them again recently. While nothing has changed regarding my review of the Hive speakers (which I still like a lot) it seems that my need for such things isn t as great as I thought. One thing that made me start using the Bluetooth speakers again is that my phone case blocks the sound from my latest phone and makes it worse than phone sound usually is. I bought Bluetooth speakers for some relatives as presents, the relatives seemed to appreciate them but I wonder how much they actually use them. Nexus 5 The Nexus 5 [12] is a nice phone. When I first reviewed it there were serious problems with overheating when playing Ingress. I haven t noticed such problems recently so I think that an update to Android might have made it more energy efficient. In that review I was very impressed by the FullHD screen and it made me want a Note 3, at the time I planned to get a Note 3 in the second half of 2014 (which I did). Galaxy Note 3 Almost a year ago I bought the Samsung Galaxy Note 3 [13]. I m quite happy with it at the moment but I don t have enough data for a long term review of it. The only thing to note so far is that in my first review I was unhappy with the USB 3 socket as that made it more difficult to connect a USB cable in the dark. I ve got used to the socket and I can now reliably plug it in at night with ease. I wrote about Rivers jeans being the only brand that can fit a Samsung Galaxy Note series phone in the pocket [14]. The pockets of my jeans have just started wearing out and I think that it s partly due to the fact that I bought a Armourdillo Hybrid case [15] for my Note 3. I ve had the jeans for over 3 years with no noticable wear apart from the pockets starting to wear out after 10 months of using the Armourdillo case. I don t think that the Armourdillo case is bad, but the fact that it has deep grooves and hard plastic causes it to rub more on material when I take the phone out of my pocket. As I check my phone very frequently this causes some serious wear. This isn t necessarily a problem given that a phone costs 20* more than a pair of jeans, if the case was actually needed to save the phone then it would be worth having some jeans wear out. But I don t think I need more protection than a gel case offers. Another problem is that the Armourdillo case is very difficult to remove. This isn t a problem if you don t need access to your phone, IE if you use a phone like the Nexus 5 that doesn t permit changing batteries or SD cards. But if you need to change batteries, SD cards, etc then it s really annoying. My wife seems quite happy with her Armoudillo case but I don t think it was a good choice for me. I m considering abandoning it and getting one of the cheap gel cases. The sound on the Note 3 is awful. I don t know how much of that is due to a limitation in the speaker and how much is due to the case. It s quite OK for phone calls but not much good for music. Tablets I m currently on my third tablet. One was too cheap and nasty so I returned it. Another was still cheap and I hardly ever used it. The third is a Galaxy Note 10 which works really well. I guess the lesson is to buy something worthwhile so you can use it. A tablet that s slower and has less storage than a phone probably isn t going to get used much. Phone Longevity I owned the Xperia X10 for 22 months before getting the Galaxy S3. As that included 9 months of using a Galaxy S I only had 13 months of use out of that phone before lending it to other people. The Galaxy S3 turned out to be a mistake as I replaced it in only 7 months. I had the Note 2 for 15 months before getting the Note 3. I have now had the Note 3 for 11 months and have no plans for a replacement any time soon this is the longest I ve owned an Android phone and been totally satisfied with it. Also I only need to use it for another 4 months to set a record for using an Android phone. The Xperia was free as part of a telco contract. The other phones were somewhere between $500 and $600 each when counting the accessories (case, battery, etc) that I bought with them. So in 4 years and 7 months I ve spent somewhere between $1500 and $1800 on phones plus the cost of the Xperia that was built in to the contract. The Xperia probably cost about the same so I ll assume that I spent $2000 on phones and accessories. This seems like a lot. However that averages out to about $1.20 per day (and hopefully a lot less if my Note 3 lasts another couple of years). I could justify $1.20 per day for either the amount of paid work I do on Android phones or the amount of recreational activities that I perform (the Galaxy S3 was largely purchased for Ingress). Conclusion I think that phone companies will be struggling to maintain sales of high end phones in the future. When I chose the Xperia X10 I knew I was making a compromise, the screen resolution was an obvious limitation on the use of the device (even though it was one of the best devices available). The storage in the Xperia was also a limitation. Now FullHD is the minimum resolution for any sort of high-end device and 32G of storage is small. I think that most people would struggle to observe any improvement over a Nexus 5 or Note 3 at this time. I think that this explains the massive advertising campaign for the Galaxy S6 that is going on at the moment. Samsung can t sell the S6 based on it being better than previous phones because there s not much that they can do to make it obviously better. So they try and sell it for the image.

• [1] http://etbe.coker.com.au/2011/02/18/xperia-x10i-first-experience/
• [2] http://etbe.coker.com.au/2011/04/17/more-about-xperia-x10/
• [3] http://etbe.coker.com.au/2011/11/21/galaxy-xperia-android-network/
• [4] http://etbe.coker.com.au/2012/09/11/back-to-xperia-x10/
• [5] http://etbe.coker.com.au/2011/12/05/cyanogenmod-galaxy-s/
• [6] http://etbe.coker.com.au/2012/12/21/galaxy-s3-power-case/
• [7] http://etbe.coker.com.au/2013/09/20/qi-phone-charging/
• [8] http://etbe.coker.com.au/2013/05/29/nexus-4/
• [9] http://etbe.coker.com.au/2013/08/13/nexus-4-ringke-fusion-case/
• [10] http://etbe.coker.com.au/2013/07/10/samsung-galaxy-note-2/
• [11] http://etbe.coker.com.au/2013/09/23/hive-bluetooth-speakers/
• [12] http://etbe.coker.com.au/2013/12/23/nexus-5/
• [13] http://etbe.coker.com.au/2014/10/31/samsung-galaxy-note-3/
• [14] http://etbe.coker.com.au/2012/11/17/geeky-jeans/
• [15] http://etbe.coker.com.au/2014/03/25/nexus5-armourdillo-hybrid-case/

You may not know it, but wifi is under assault in the USA due to proposed FCC regulations about modifications to devices with modular radios. In short, it would make it illegal for vendors to sell devices with firmware that users can replace. This is of concern to everyone, because Wifi routers are notoriously buggy and insecure. It is also of special concern to amateur radio hobbyists, due to the use of these devices in the Amateur Radio Service (FCC Part 97). I submitted a comment to the FCC about this, which I am pasting in here. This provides a background and summary of the issues for those that are interested. Here it is: My comment has two parts: one, the impact on the Amateur Radio service; and two, the impact on security. Both pertain primarily to the 802.11 ( Wifi ) services typically operating under Part 15. The Amateur Radio Service (FCC part 97) has long been recognized by the FCC and Congress as important to the nation. Through it, amateurs contribute to scientific knowledge, learn skills that bolster the technological competitiveness of the United States, and save lives through their extensive involvement in disaster response. Certain segments of the 2.4GHz and 5GHz Wifi bands authorized under FCC Part 15 also fall under the frequencies available to licensed amateurs under FCC Part 97 [1]. By scrupulously following the Part 97 regulations, many amateur radio operators are taking devices originally designed for Part 15 use and modifying them for legal use under the Part 97 Amateur Radio Service. Although the uses are varied, much effort is being devoted to fault-tolerant mesh networks that provide high-speed multimedia communications in response to a disaster, even without the presence of any traditional infrastructure or Internet backbone. One such effort [2] causes users to replace the firmware on off-the-shelf Part 15 Wifi devices, reconfiguring them for proper authorized use under Part 97. This project has many vital properties, particularly the self-discovery of routes between nodes and self-healing nature of the mesh network. These features are not typically available in the firmware of normal Part 15 devices. It should also be noted that there is presently no vendor of Wifi devices that operate under Part 97 out of the box. The only route available to amateurs is to take Part 15 devices and modify them for Part 97 use. Amateur radio users of these services have been working for years to make sure they do not cause interference to Part 15 users [3]. One such effort takes advantage of the modular radio features of consumer Wifi gear to enable communication on frequencies that are within the Part 97 allocation, but outside (though adjacent) to the Part 15 allocation. For instance, the chart at [1] identifies frequencies such as 2.397GHz or 5.660GHz that will never cause interference to Part 15 users because they lie entirely outside the Part 15 Wifi allocation. If the FCC prevents the ability of consumers to modify the firmware of these devices, the following negative consequences will necessarily follow: 1) The use of high-speed multimedia or mesh networks in the Amateur Radio service will be sharply curtailed, relegated to only outdated hardware. 2) Interference between the Amateur Radio service which may use higher power or antennas with higher gain and Part 15 users will be expanded, because Amateur Radio service users will no longer be able to intentionally select frequencies that avoid Part 15. 3) The culture of inventiveness surrounding wireless communication will be curtailed in an important way. Besides the impact on the Part 97 Amateur Radio Service, I also wish to comment on the impact to end-user security. There have been a terrible slew of high-profile situations where very popular consumer Wifi devices have had incredible security holes. Vendors have often been unwilling to address these issues [4]. Michael Horowitz maintains a website tracking security bugs in consumer wifi routers [5]. Sadly these bugs are both severe and commonplace. Within just the last month, various popular routers have been found vulnerable to remote hacking [6] and platforms for launching Distributed Denial-of-Service (DDoS) attacks [7]. These impacted multiple models from multiple vendors. To make matters worse, most of these issues should have never happened in the first place, and were largely the result of carelessness or cluelessness on the part of manufacturers. Consumers should not be at the mercy of vendors to fix their own networks, nor should they be required to trust unauditable systems. There are many well-regarded efforts to provide better firmware for Wifi devices, which still keep them operating under Part 15 restrictions. One is OpenWRT [8], which supports a wide variety of devices with a system built upon a solid Linux base. Please keep control of our devices in the hands of consumers and amateurs, for the benefit of all. References: [1] https://en.wikipedia.org/wiki/High-speed_multimedia_radio [2] http://www.broadband-hamnet.org/just-starting-read-this.html [3] http://www.qsl.net/kb9mwr/projects/wireless/modify.html [4] http://www.tomsitpro.com/articles/netgear-routers-security-hole,1-2461.html [5] http://routersecurity.org/bugs.php [6] https://www.kb.cert.org/vuls/id/950576 [7] https://www.stateoftheinternet.com/resources-cloud-security-2015-q2-web-security-report.html [8] https://openwrt.org/

Review: Kanban, by David J. Anderson

Publisher:	Blue Hole
Copyright:	2010
ISBN:	0-9845214-0-2
Format:	Trade paperback
Pages:	240

Another belated review, this time of a borrowed book. Which I can now finally return! A delay in the review of this book might be a feature if I had actually used it for, as the subtitle puts it, successful evolutionary change in my technology business. Sadly, I haven't, so it's just late. So, my background: I've done a lot of variations of traditional project management for IT projects (both development and more operational ones), both as a participant and as a project manager. (Although I've never done the latter as a full-time job, and have no desire to do so.) A while back at Stanford, my team adopted Agile, specifically Scrum, so I did a fair bit of research about Scrum including a couple of training courses. Since then, at Dropbox, I've used a few different variations on a vaguely Agile-inspired planning process, although it's not particularly consistent with any one system. I've been hearing about Kanban for a while and have friends who swear by it, but I only had a vague idea of how it worked. That seemed like a good excuse to read a book. And Anderson's book is a good one if, like me, you're looking for a basic introduction. It opens with a basic description and definition, talks about motivation and the expected benefits, and then provides a detailed description of Kanban as a system. The tone is on the theory side, written using the terminology of (I presume) management theory and operations management, areas about which I know almost nothing, but the terminology wasn't so heavy as to make the book hard to read. Anderson goes into lots of detail, and I thought he did a good job of distinguishing between basic principles, optional techniques, and variations that may be appropriate for particular environments. If you're also not familiar, the basic concept of Kanban is to organize work using an in-progress queue. It eschews time-bounded planning entirely in favor of staging work at the start of a sequence of queues and letting the people working on each queue pull from the previous queue when they're ready to take on a new task. As you might guess from that layout, Kanban was originally invented for assembly-line manufacturing (at Toyota). That was one of the problems that I had with it, or at least the presentation in this book: most of my software development doesn't involve finishing one part of something and handing it off to someone else, which made it hard to identify with the pipeline model. Anderson has clearly spent a lot of time working with large-scale programming shops, including outsourced development firms, with dedicated QA and operations roles. This is not at all how Silicon Valley agile development works, so parts of this book felt like missives from a foreign country. That said, the key point of Kanban is not the assembly line but the work limit. One of the defining characteristics of Kanban, at least as Anderson presents it, is that one does not try to estimate work and tile it based on estimates, not even to the extent that Scrum and other Agile methodologies do within the sprint. Instead, each person takes as long as they take on the things they're working on, and pulls a new task when they have free capacity. The system instead puts a limit on how many things they can have in progress at a time. The problem of pipeline stalls is dealt with both via continuous improvement and "swarming" of a problem to unblock the line (since other teams may not be able to work until the block is fixed), and with being careful about the sizing of work items (I'm used to calling them stories) that go in the front end. Predictability, which Scrum uses story sizing and team velocity analysis to try to achieve, is basically statistical. One uses a small number of buckets of sizes of stories, and the whole pipeline will finish some number of work items per unit time, with a variance. The promise made to clients and other teams is that some percentage of the work items will be finished within some time frame from when they enter the system. Most importantly, these are all descriptive properties, determined statistically after the fact, rather than planned properties worked out through story sizing and extensive team discussion. If you, like me, are pretty thoroughly sick of two-hour sprint planning meetings and endless sizing exercises, this is rather appealing. My problem with most work planning systems is that I think they overplan and put too much weight our ability to estimate how long work will take. Kanban is very appealing when viewed through that lens: it gives up on things we're bad at in favor of simple measurement and building a system with enough slack that it can handle work of various different sizes. As mentioned at the top, I haven't had a chance to try it (and I'm not sure it's a good fit with the inter-group planning methods in use at my current workplace), but I came away from this book wanting to do so. If, like me, your experience is mostly with small combined teams or individual programming work, Anderson's examples may seem rather large, rather formal, and bureaucratic. But this is a solid introduction and well worth reading if your only experience with Agile is sprint planning, story writing and sizing, and fast iterations. Rating: 7 out of 10

The Linux kernel keyring is effectively a mechanism to allow shoving blobs of data into the kernel and then setting access controls on them. It's convenient for a couple of reasons: the first is that these blobs are available to the kernel itself (so it can use them for things like NFSv4 authentication or module signing keys), and the second is that once they're locked down there's no way for even root to modify them.

But there's a corner case that can be somewhat confusing here, and it's one that I managed to crash into multiple times when I was implementing some code that works with this. Keys can be "possessed" by a process, and have permissions that are granted to the possessor orthogonally to any permissions granted to the user or group that owns the key. This is important because it allows for the creation of keyrings that are only visible to specific processes - if my userspace keyring manager is using the kernel keyring as a backing store for decrypted material, I don't want any arbitrary process running as me to be able to obtain those keys[1]. As described in keyrings(7), keyrings exist at the session, process and thread levels of granularity.

This is absolutely fine in the normal case, but gets confusing when you start using sudo. sudo by default doesn't create a new login session - when you're working with sudo, you're still working with key posession that's tied to the original user. This makes sense when you consider that you often want applications you run with sudo to have access to the keys that you own, but it becomes a pain when you're trying to work with keys that need to be accessible to a user no matter whether that user owns the login session or not.

I spent a while talking to David Howells about this and he explained the easiest way to handle this. If you do something like the following:
$ sudo keyctl add user testkey testdata @u
a new key will be created and added to UID 0's user keyring (indicated by @u). This is possible because the keyring defaults to 0x3f3f0000 permissions, giving both the possessor and the user read/write access to the keyring. But if you then try to do something like:
$ sudo keyctl setperm 678913344 0x3f3f0000
where 678913344 is the ID of the key we created in the previous command, you'll get permission denied. This is because the default permissions on a key are 0x3f010000, meaning that the possessor has permission to do anything to the key but the user only has permission to view its attributes. The cause of this confusion is that although we have permission to write to UID 0's keyring (because the permissions are 0x3f3f0000), we don't possess it - the only permissions we have for this key are the user ones, and the default state for user permissions on new keys only gives us permission to view the attributes, not change them.

But! There's a way around this. If we instead do:
$ sudo keyctl add user testkey testdata @s
then the key is added to the current session keyring (@s). Because the session keyring belongs to us, we possess any keys within it and so we have permission to modify the permissions further. We can then do:
$ sudo keyctl setperm 678913344 0x3f3f0000
and it works. Hurrah! Except that if we log in as root, we'll be part of another session and won't be able to see that key. Boo. So, after setting the permissions, we should:
$ sudo keyctl link 678913344 @u
which ties it to UID 0's user keyring. Someone who logs in as root will then be able to see the key, as will any processes running as root via sudo. But we probably also want to remove it from the unprivileged user's session keyring, because that's readable/writable by the unprivileged user - they'd be able to revoke the key from underneath us!
$ sudo keyctl unlink 678913344 @s
will achieve this, and now the key is configured appropriately - UID 0 can read, modify and delete the key, other users can't.

This is part of our ongoing work at CoreOS to make rkt more secure. Moving the signing keys into the kernel is the first step towards rkt no longer having to trust the local writable filesystem[2]. Once keys have been enrolled the keyring can be locked down - rkt will then refuse to run any images unless they're signed with one of these keys, and even root will be unable to alter them.

[1] (obviously it should also be impossible to ptrace() my userspace keyring manager)
[2] Part of our Secure Boot work has been the integration of dm-verity into CoreOS. Once deployed this will mean that the /usr partition is cryptographically verified by the kernel at runtime, making it impossible for anybody to modify it underneath the kernel. / remains writable in order to permit local configuration and to act as a data store, and right now rkt stores its trusted keys there.

comments

Like each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In July, 79.50 work hours have been dispatched among 7 paid contributors. Their reports are available:

• Ben Hutchings did 14.75 hours.
• Guido G nther did 8 hours.
• Mike Gabriel did 8 hours.
• Rapha l Hertzog did 15 hours.
• Santiago Ruano Rinc n did 14.75 hours.
• Scott Kitterman did 4 hours.
• Thorsten Alteholz did 15 hours.

Evolution of the situation August has seen a small decrease in terms of sponsored hours (71.50 hours per month) because two sponsors did not pay their renewal invoice on time. That said they reconfirmed their willingness to support us and things should be fixed after the summer. And we should be able to reach our first milestone of funding the equivalent of a half-time position, in particular since a new platinum sponsor might join the project. DebConf 15 happened this month and Debian LTS was featured in a talk and in a work session. Have a look at the video recordings:

• Debian Long Term Support: Past Present and Future (slides)
• Preparing for Wheezy LTS

In terms of security updates waiting to be handled, the situation is better than last month: the dla-needed.txt file lists 20 packages awaiting an update (4 less than last month), the list of open vulnerabilities in Squeeze shows about 22 affected packages in total (11 less than last month). The new LTS frontdesk ensures regular triage of CVE reports and the difference between both counts dropped significantly. That s good! Thanks to our sponsors Thanks to Sig-I/O, a new bronze sponsor, which joins our 35 other sponsors.

• Gold sponsors:
• The Positive Internet (for 14 months already)
• Linode LLC (for 3 months already)
• Silver sponsors:
• David Ayers IntarS Austria (for 14 months already)
• Blablacar (for 13 months already)
• Domeneshop AS (for 13 months already)
• Universit Lille 3 (for 13 months already)
• Trollweb Solutions (for 11 months already)
• Gandi SAS (for 8 months already)
• University of Luxembourg (for 5 months already)
• Rentabiliweb Group (for 4 months already)
• Bronze sponsors:
• Evolix (for 14 months already)
• Offensive Security (for 14 months already)
• Seznam.cz, a.s. (for 14 months already)
• Freeside Internet Service (for 13 months already)
• MyTux (for 13 months already)
• Intevation GmbH (for 11 months already)
• Linuxhotel GmbH (for 11 months already)
• Nantes M tropole (for 11 months already)
• Daevel SARL (for 9 months already)
• FOSSter (for 9 months already)
• Bitfolk LTD (for 8 months already)
• Megaspace Internet Services GmbH (for 8 months already)
• Gree, Inc. (for 7 months already)
• Greenbone Networks GmbH (for 7 months already)
• NUMLOG (for 7 months already)
• WinGo AG (for 7 months already)
• Ecole Centrale de Nantes LHEEA (for 3 months already)
• Sig-I/O

One comment Liked this article? Click here. My blog is Flattr-enabled.

In the last few years I wasn t really maintaining my GPG keys. I ve created a new one (B4E14499) in 2011 during DebConf11, after the older primary one (99E81DA0) became too weak (1024D). I thought that I didn t have enough signatures on the new key and almost lost my place on the debian keyring due to removal on the old one (without adding the new key). Due to my confusion with the key signature, I didn t really take the time to sign other people keys. But that doesn t mean I ignored them completely, as I kept all the information from Debconf11 (yes, 4 years ago) and also the slips I was handed since. Today, I finally took the time to finish the backlog and sign all the keys which are strong enough and still valid (haven t expired / revoked). One less item on the todo list. For those who got my signatures I m sorry for the delay, but better later than never, right ?
Filed under: Debian GNU/Linux Tagged: gpg

<Sigh/> For the second time since I joined Debian, I missed DebConf. This time, mostly for two completely happy reasons (which have, of course, grown up quite significatively): But adding to that very important factor, early-to-mid August is possibly the worst date for me to attend DebConf As that's when we start classes at UNAM, and I really don't want to delegate introducing my class to somebody else (besides it being or not possible). Life has been quite busy here. Besides starting my seventh semester as a teacher (finally being able to present my book on operating systems to the students!), I also took on teaching two modules for a diploma-course on embedded Linux. I'm co- teaching the "User-space Linux" module together with Gabriel Salda a, and will teach the "Boot process" module with Sandino Araico. There are many things that should be improved about the covered program, but we are having good fun working our way on it; Gabriel and I have given two (out of six) 6hr sessions each, and have reached the point where it is actually becoming fun :) As for Debian... I have been doing minimal maintenance work. Answering to some keyring tickets, or taking care of only high-profile important issues. I do not have time to do much. I have been wanting to do some interesting analysis on the evolution of our keyring for several months, but have not had the time to follow it up. Will do at some point, /mehopes. Anyway... This post is probably just because I need to somehow be there at our community's highest social, technical exchange period in the year. That is, there's not much to report, it's only me waving from home!

It's been a while since I talked about Ceilometer and its companions, so I thought I'd go ahead and write a bit about what's going on this side of OpenStack. I'm not going to cover new features and fancy stuff today, but rather a shallow overview of the new project processes we initiated. Ceilometer growing Ceilometer has grown a lot since that time when we started it 3 years ago. It has evolved from a system designed to fetch and store measurements, to a more complex system, with agents, alarms, events, databases, APIs, etc. All those features were needed and asked for by users and operators, but let's be honest, some of them should never have ended up in the Ceilometer code repository, especially not all at the same time. The reality is we picked a pragmatic approach due to the rigidity of the OpenStack Technical Committee in regards to new projects to become OpenStack integrated and, therefore, blessed projects. Ceilometer was actually the first project to be incubated and then integrated. We had to go through the very first issues of that process. Fortunately, now that time has passed, and all those constraints have been relaxed. To me, the OpenStack Foundation is turning into something that looks like the Apache Foundation, and there's, therefore, no need to tie technical solutions to political issues. Indeed, the Big Tent now allows much more flexibility to all of that. Back a year ago, we were afraid to bring Gnocchi into Ceilometer. Was the Technical Committee going to review the project? Was the project going to be in the scope of Ceilometer for the Technical Committee? Now we don't have to ask ourselves those questions, now that we have that freedom, it empowers us to actually do what we think is good in term of technical design without worrying too much about political issues.

Ceilometer development activity

Acknowledging Gnocchi The first step in this new process was to continue working on Gnocchi (a timeserie database and resource indexer designed to overcome historical Ceilometer storage issue) and to decide that it was not the right call to merge it into Ceilometer as some REST API v3, but that it was better to keep it standalone. We managed to get traction to Gnocchi, getting a few contributors and users. We're even seeing talks proposed to the next Tokyo Summit where people leverage Gnocchi, such as "Service of predictive analytics on cost and performance in OpenStack", "Suveil" and "Cutting Edge NFV On OpenStack: Healing and Scaling Distributed Applications". We are also doing some progress on pushing Gnocchi outside of the OpenStack community, as it can be a self-sufficient timeserie and resource database that can be used without any OpenStack interaction. Branching Aodh Rather than continuing to grow Ceilometer, during the last summit we all decided that it was time to reorganize and split Ceilometer into the different components it is made of, leveraging a more service-oriented architecture. The alarm subsystem of Ceilometer being mostly untied to the rest of Ceilometer, we decided it was the first and perfect candidate to do that. I personally engaged into doing the work and created a new repository with only the alarm code from Ceilometer, named Aodh.

Aodh is an Irish word meaning fire. A word picked so it also had some relation to Heat, and because we have some Irish influence around the project .

This made sense for a lot of reason. First because Aodh can now work completely standalone, using either Ceilometer or Gnocchi as a backend or any new plugin you'd write. I love the idea that OpenStack projects can work standalone like Swift does for example without implying any other OpenStack component. I think it's a proof of good design. Secondly, because it allows us to resonate on a smaller chunk of software a reason really under-estimated today in OpenStack. I believe that the size of your software should match a certain ratio to the size of your team. Aodh is, therefore, a new project under the OpenStack Telemetry program (or what remains of OpenStack programs now), alongside Ceilometer and Gnocchi, forked from the original Ceilometer alarm feature. We'll deprecate the latter with the Liberty release, and we'll remove it in the Mitaka release. Lessons learned Actually, moving that code out of Ceilometer (in the case of Aodh), or not merging it in (in the case of Gnocchi) had a few side effects that I admit I think we probably under-estimated back then. Indeed, the code size of Gnocchi or Aodh ended up being much smaller than the entire Ceilometer project Gnocchi is 7 smaller and Aodh 5x smaller than Ceilometer and therefore much more easy to manipulate and to hack on. That allowed us to merge dozens of patches in a few weeks, cleaning-up and enhancing a lot of small things in the code. Those tasks are very much harder in Ceilometer, due to the bigger size of the code base and the small size of our team. By having our small team working on smaller chunks of changes even when it meant actually doing more reviews greatly improved our general velocity and the number of bugs fixed and features implemented. On the more sociological side, I think it gave the team the sensation of finally owning the project. Ceilometer was huge, and it was impossible for people to know every side of it. Now, it's getting possible for people inside a team to cover a much larger portion of those smaller project, which gives them a greater sense of ownership and caring. Which ends up being good for the project quality overall. That also means that we technically decided to have different core teams by project (Ceilometer, Gnocchi, and Aodh) as they all serve different purposes and can all be used standalone or with each others. Meaning we could have contributors completely ignoring other projects. All of that reminds me some discussion I heard about projects such as Glance, trying to fit new features in - some that are really orthogonal to the original purpose. It's now clear to me that having different small components interacting together that can be completely owned and taken care of by a (small) team of contributors is the way to go. People that can therefore trust each others and easily bring new people in, makes a project really incredibly more powerful. Having a project covering a too wide set of features make things more difficult if you don't have enough manpower. This is clearly an issue that big projects inside OpenStack are facing now, such as Neutron or Nova.

Review: The Pyramid Waltz, by Barbara Ann Wright

Series:	Katya and Starbride #1
Publisher:	Bold Strokes
Copyright:	September 2012
ISBN:	1-60282-792-3
Format:	Kindle
Pages:	264

Princess Katya Nar Umbriel is publicly a bored, womanizing, and difficult daughter to the rulers of Farraday. It's all an act, though, with the full knowledge of her parents. As the second child, she's the leader of the Order of Vestra: the equivalent of the Secret Service, devoted to protecting the royal family and, by extension, the kingdom, particularly against magical attacks. Starbride is new to court and entirely out of place. From a northern neighboring country, and far more comfortable in practical clothing than the frilled court dresses that her mother wants her to wear, she has been sent to court to make contacts. Her people are getting the bad side of various trade contracts and desperately need some political maneuvering space of their own. Starbride's best hope for this is to study law in the palace library when she can manage to avoid the other courtiers. But then she and Katya stumble across each other, outside of the roles they're playing, and might have an opportunity for a deeper connection. One that neither of them want to entangle in their personal worries. This is the last of a set of books I picked up while looking for lesbian romance with fantasy or science fiction elements. On the romance front, it's one of the better entries in that set. Both Katya and Starbride are likeable, in large part due to their mutual exasperation with the trappings of the court. (Making the protagonists more serious, thoughtful, and intelligent than the surrounding characters is an old trick, but it works.) Wright has a good ear for banter, particularly the kind when two people of good will are carefully feeling each other out. And despite Katya's need to keep a deep secret from Starbride for some of the book, The Pyramid Waltz mostly avoids irritating communication failures as a plot driver. The fantasy portion and the plot drivers, alas, are weaker. The world building is not exactly bad, but it's just not that interesting. There are a couple of moderately good ideas, in the form of pyramid magic and secret (and dangerous) magical powers that run in the royal family, but they're not well-developed. Pyramid magic turns out to look much like any other generic fantasy magic system, with training scenes that could have come from a Valdemar or Wheel of Time novel (and without as much dramatic tension). And the royal family's secret, while better-developed and integral to the plot, still felt rather generic and one-sided. Maybe that's something Wright develops better in future novels in this series, but that was another problem: the ending of The Pyramid Waltz was rather weak. Partly, I think, this is because the cast is too large and not well-developed. I cared about Katya and Starbird, and to a lesser extent their servants and one of the Order members. (Wright has a moderately interesting bit of worldbuilding about how servants work in Starbride's culture, which I wish we'd seen more of.) But there are a bunch of other Order of Vesta members, Katya's family, and various other bits of history and hinted world views, none of which seemed to get much depth. The ending climax involved a lot of revelations and twists that primarily concerned characters I didn't care about. It lost something in the process. This book is clearly set up for a sequel. There is an ending, but it's not entirely satisfying. Unfortunately, despite liking Katya and Starbird a lot, the rest of the story wasn't compelling enough to make me want to buy it, particularly since the series apparently goes through another three books before reaching a real ending. I enjoyed parts of this book, particularly Katya and Starbird feeling each other out and discovering similarities in their outlook. Katya teasing Starbird, and Starbird teasing herself, over her mother's choice of her clothing was probably the best part. It's not bad for what it's trying to do, but I think it's a bit too generic and not satisfying enough to really recommend. Followed by For Want of a Fiend. Rating: 6 out of 10

A few days ago, fellow Qt/KDE team member Lisandro gave an update on the situation with migration to Plasma 5 in Debian Testing (AKA Stretch). It s changed again. All of Plasma 5 is now in Testing. The upgrade probably won t be entirely smooth, which we ll work on that after the gcc5 transition is done, but it will be much better than the half KDE4 SC half Kf5/Plasma 5 situation we ve had for the last several days. The issues with starting kwin should be resolved once users upgrade to Plasma 5. To use the current kwin with KDE SC 4, you will need to add a symlink from /usr/bin/kwin to /usr/bin/kwin_x11. That will be included in the next upload after gcc5. Systemsettings and plasma-nm now work. In my initial testing, I didn t see anything major that was broken. One user reported an issue with sddm starting automatically, but it worked fine for me. During the upgrade you should get a debconf prompt asking if you want to use kdm or sddm. Pick sddm. When I tried to dist-upgrade, apt wanted to remove task-kde-desktop. I let it remove it and some other packages and then in a second step did apt-get install task-kde-desktop. That pulled it back in successfully along with adding and removing a reasonably large stack of packages. Obviously we need to make that work better before Stretch is released, but as long as you don t restart KDE in between those two steps it should be fine. Lastely, I used apt-get autoremove to clear out a lot of no longer needed KDE4 things (when it asks if you want to stop the running kdm, say no). Here are a few notes on terminology and what I understand of the future plans: What used to be called KDE is now three different things (in part because KDE is now the community of people, not the software): KDE Frameworks 5 (Kf5): This is a group of several dozen small libraries that as a group, roughly equate to what used to be kdelibs. Plasma (Workspaces) 5: This is the desktop that we ve just transitioned to. Applications: These are a mix of kdelibs and Kf5 based applications. Currently in Testing there are some of both and this will evolve over time based on upstream development. As an example, the Kf5 based version of konsole is in Unstable and should transition to Testing shortly. Finally, thanks to Maximiliano Curia (maxy on IRC) for doing virtually all of the packaging of Kf5, Plasma 5, and applications. He did the heavy lifting, the rest of us just nibbled around the edges to keep it moving towards testing.

Hadley is a popular figure, and rightly so as he successfully introduced many newcomers to the wonders offered by R. His approach strikes some of us old greybeards as wrong---I particularly take exception with some of his writing which frequently portrays a particular approach as both the best and only one. Real programming, I think, is often a little more nuanced and aware of tradeoffs which need to be balanced. As a book on another language once popularized: "There is more than one way to do things." But let us leave this discussion for another time. As the reach of the Hadleyverse keeps spreading, we sometimes find ourselves at the receiving end of a cost/benefit tradeoff. That is what this post is about, and it uses a very concrete case I encountered yesterday. As blogged earlier, the RcppZiggurat package was updated. I had not touched it in a year, but Brian Ripley had sent a brief and detailed note concerning something flagged by the Solaris compiler (correctly suggesting I replace fabs() with abs() on integer types). (Allow me to stray from the main story line here for a second to stress just how insane a work load he is carrying, essentially for all of us. R and the R community are so just so indebted to him for all his work---which makes the usual social media banter about him so unfortunate. But that too shall be left for another time.) Upon making the simple fix, and submitting to GitHub the usual Travis CI was triggered. And here is what I saw:
All happy, all green. Previous build a year ago, most recent build yesterday, both passed. But hold on: test time went from 2:54 minutes to 7:47 minutes for an increase of almost five minutes! And I knew that I had not added any new dependencies, or altered any build options. What did happen was that among the dependencies of my package, one had decided to now also depend on ggplot2. Which leads to a chain of sixteen additional packages being loaded besides the four I depend upon---when it used to be just one. And that took five minutes as all those packages are installed from source, and some are big and take a long time to compile. There is however and easy alternative, and for that we have to praise Michael Rutter who looks after a number of things for R on Ubuntu. Among these are the R builds for Ubuntu but also the rrutter PPA as well as the c2d4u PPA. If you have not heard this alphabet soup before, a PPA is a package repository for Ubuntu where anyone (who wants to sign up) can upload (properly setup) source files which are then turned into Ubuntu binaries. With full dependency resolution and all other goodies we have come to expect from the Debian / Ubuntu universe. And Michael uses this facility with great skill and calm to provide us all with Ubuntu binaries for R itself (rebuilding what yours truly uploads into Debian), as well as a number of key packages available via the CRAN mirrors. Less know however is this "c2d4u" which stands for CRAN to Debian for Ubuntu. And this builds on something Charles Blundell once built under my mentorship in a Google Summer of Code. And Michael does a tremdous job covering well over a thousand CRAN source packages---and providing binaries for all. Which we can use for Travis! What all that means is that I could now replace the line

 - ./travis-tool.sh install_r RcppGSL rbenchmark microbenchmark highlight

which implies source builds of the four listed packages and all their dependencies with the following line implying binary installations of already built packages:

 - ./travis-tool.sh install_aptget libgsl0-dev r-cran-rcppgsl r-cran-rbenchmark r-cran-microbenchmark r-cran-highlight

In this particular case I also needed to build a binary package of my RcppGSL package as this one is not (yet) handled by Michael. I happen to have (re-)discovered the beauty of PPAs for Travis earlier this year and revitalized an older and largely dormant launchpad account I had for this PPA of mine. How to build a simple .deb package will also have to left for a future post to keep this more concise. This can be used with the existing r-travis setup---but one needs to use the older, initial variant in order to have the ability to install .deb packages. So in the .travis.yml of RcppZiggurat I just use

before_install:
## PPA for Rcpp and some other packages
- sudo add-apt-repository -y ppa:edd/misc
## r-travis by Craig Citro et al
- curl -OL http://raw.github.com/craigcitro/r-travis/master/scripts/travis-tool.sh
- chmod 755 ./travis-tool.sh
- ./travis-tool.sh bootstrap

to add my own PPA and all is good. If you do not have a PPA, or do not want to create your own packages you can still benefit from the PPAs by Michael and "mix and match" by installing from binary what is available, and from source what is not. Here we were able to use an all-binary approach, so let's see the resulting performance:
Now we are at 1:03 to 1:15 minutes---much better. So to conclude, while the every expanding universe of R packages is fantastic for us as users, it can be seen to be placing a burden on us as developers when installing and testing. Fortunately, the packaging infrastructure built on top of Debian / Ubuntu packages can help and dramatically reduce build (and hence test) times. Learning about PPAs can be a helpful complement to learning about Travis and continued integration. So maybe now I need a new reason to blame Hadley? Well, there is always snake case ...

Follow-up: The post got some pretty immediate feedback shortly after I posted it. Craig Citro pointed out (quite correctly) that I could use r_binary_install which would also install the Ubuntu binaries based on their R packages names. Having built R/CRAN packages for Debian for so long, I am simply more used to the r-cran-* notations, and I think I was also the one contributing install_aptget to r-travis ... Yihui Xie spoke up for the "new" Travis approach deploying containers, caching of packages and explicit whitelists. It was in that very (GH-based) discussion that I started to really lose faith in the new Travis approach as they want use to whitelist each and every package. With 6900 and counting at CRAN I fear this simply does not scale. But different approaches are certainly welcome. I posted my 1:03 to 1:15 minutes result. If the "New School" can do it faster, I'd be all ears.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

A new release 0.12.0 of Rcpp arrived on the CRAN network for GNU R this morning, and I also pushed a Debian package upload. Rcpp has become the most popular way of enhancing GNU R with C++ code. As of today, 423 packages on CRAN depend on Rcpp for making analyses go faster and further. Note that this is 60 more packages since the last release in May! Also, BioConductor adds another 57 packages, and casual searches on GitHub suggests many more. And according to Andrie De Vries, Rcpp has now page rank of one on CRAN as well! And with this release, Rcpp also becomes ready for Big Data, or, as they call it in Texas, Data. Thanks to a lot of work and several pull requests by Qiang Kou, support for R_xlen_t has been added. That means we can now do stunts like

R> library(Rcpp)
R> big <- 2^31-1
R> bigM <- rep(NA, big)
R> bigM2 <- c(bigM, bigM)
R> cppFunction("double getSz(LogicalVector x)   return x.length();  ")
R> getSz(bigM)
[1] 2147483647
R> getSz(bigM2)
[1] 4294967294
R>

where prior versions of Rcpp would just have said

> getSz(bigM2)
Error in getSz(bigM2) :
  long vectors not supported yet: ../../src/include/Rinlinedfuns.h:137
>

which is clearly not Texas-style. Another wellcome change, also thanks to Qiang Kou, adds encoding support for strings. A lot of other things got polished. We are still improving exception handling as we still get the odd curveballs in a corner cases. Matt Dziubinski corrected the var() computation to use the proper two-pass method and added better support for lambda functions in Sugar expression using sapply(), Qiang Kou added more pull requests mostly for string initialization, and Romain added a pull request which made data frame creation a little more robust, and JJ was his usual self in tirelessly looking after all aspects of Rcpp Attributes. As always, you can follow the development via the GitHub repo and particularly the Issue tickets and Pull Requests. And any discussions, questions, ... regarding Rcpp are always welcome at the rcpp-devel mailing list. Last but not least, we are also extremely pleased to annouce that Qiang Kou has joined us in the Rcpp-Core team. We are looking forward to a lot more awesome! See below for a detailed list of changes extracted from the NEWS file.

Changes in Rcpp version 0.12.0 (2015-07-24)

• Changes in Rcpp API:
  • Rcpp_eval() no longer uses R_ToplevelExec when evaluating R expressions; this should resolve errors where calling handlers (e.g. through suppressMessages()) were not properly respected.
  • All internal length variables have been changed from R_len_t to R_xlen_t to support vectors longer than 2^31-1 elements (via pull request 303 by Qiang Kou).
  • The sugar function sapply now supports lambda functions (addressing issue 213 thanks to Matt Dziubinski)
  • The var sugar function now uses a more robust two-pass method, supports complex numbers, with new unit tests added (via pull request 320 by Matt Dziubinski)
  • String constructors now allow encodings (via pull request 310 by Qiang Kou)
  • String objects are preserving the underlying SEXP objects better, and are more careful about initializations (via pull requests 322 and 329 by Qiang Kou)
  • DataFrame constructors are now a little more careful (via pull request 301 by Romain Francois)
  • For R 3.2.0 or newer, Rf_installChar() is used instead of Rf_install(CHAR()) (via pull request 332).
• Changes in Rcpp Attributes:
  • Use more robust method of ensuring unique paths for generated shared libraries.
  • The evalCpp function now also supports the plugins argument.
  • Correctly handle signature termination characters (' ' or ';') contained in quotes.
• Changes in Rcpp Documentation:
  • The Rcpp-FAQ vignette was once again updated with respect to OS X issues and Fortran libraries needed for e.g. RcppArmadillo.
  • The included Rcpp.bib bibtex file (which is also used by other Rcpp* packages) was updated with respect to its CRAN references.

Thanks to CRANberries, you can also look at a diff to the previous release As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads page, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.