Today, my PC did not boot as usual to Debian. BIOS could not find my /dev/sda and was looking for the netboot image. I restarted my PC and got into the BIOS boot setting menu. Hmmm.... my first SDD (/dev/sda) was missing. My second HDD (/dev/sdb) was there. But I did not put the Grub boot-loader there. No wonder it did not boot.

I have a 32GB USB3 stick with the full Debian system. It is not a live CD image USB stick but a HDD formatted and encrypted system. Though it is not the fastest system, it is very light and usable. I plugged it in and powered it up. It booted OK but /dev/sda was still missing. While it booted, I saw "ata1: COMRESET failed (errorno=-16)" . So this ata1 SSD cannot be accessed from BIOS nor Linux. Sigh ...

Looking around the web under the USB stick system, I saw some people were talking that the loose serial ATA cable sometimes causes this message. Since my PC is a laptop, It has no flexible cable but has an on-board connector inside for the SSD.

Hoping my problem is just a bad connection problem, I crack opened the back panel of my PC. The SSD looked fine. I unplugged it from the connector and reinserted back into the connector. After repeating this several times to be sure, I closed the back panel and booted.

It boots as expected into Debian. Looks like everything is fine.
SMART Error Log Version: 1
No Errors Logged
Good.

If you have any boot problem like mine, please reinsert your SSD to the connector like I did before you panic.

Good luck.

Osamu

PS: This Crucial/Micron RealSSD m4/C400/P400 M4-CT256M4SSD2 previously had a problem. A firmware bug made it read-only. The firmware updates fixed my Debian system on this SSD. I could fix this without Win*** OS since the firmware update was on a bootable disk image file.

I was surprised at first to see that a long-standing bug in dspam had been fixed. Until that is, I realised it was from the Debian ftp masters and the reason the bug was closing was that dspam was being removed from the Debian archive. Damn! So, now what? What is a good replacement for dspam that is actually maintained? I don t need anti-virus because mutt just ignores those sorts of things and besides youbankdetails.zip.exe doesn t run too well on Debian. dspam basically used tokens to find common patterns of spam and ham, with you bouncing misses so it learnt from its mistakes. Already got postgrey running for greylisting so its really something that does the bayesan filtering. Some intial comments:

• bogfilter looks interesting and seems the closest thing so far
• cluebringer aka policyd seems like a policy and bld type of spam filter, not bayesan
• I ve heard crm114 is good but hard to use
• spamassasin I used to use this, not sure why I stopped

There really is only me on the mailserver with a pretty light load so no need to worry about efficiencies. Not sure if it matters but my MTA is postfix and I already use procmail for delivery.

To those who don t know, JSCommunicator is a SIP communication tool developed in HTML and JavaScript, currently supporting audio/video calls and SIP SIMPLE instant messaging. We ve recently added i18n internationalization support for English, Portuguese, Spanish, French and now Hebrew. We would like to have support for other languages and it s a pretty straightforward process to add a translation, so even if you do not have a tech background you can contribute! Setup First, create and account in Github. Github is a hosting service for git repositories and open source projects get to use Github for FREE! As a result, many open source project repositories are hosted on Github, including but not limited to JSCommunicator. Once you have created your git account, follow steps 1-4 in the section Setting up git listed in Github s setup tutorial. This will help you install git and configure anything you need. Fork Now fork the JSCommunicator repo. I really like the word fork . Great term. If this is your first fork, know that this will make a copy of the JSCommunicator repo and all its respective code to your Github account. Here s how we go about it:

1. Sign in to Github
2. Browse to https://github.com/JLouback/jscommunicator
3. Somewhere around the upper left corner, you will see a button labeled Fork . Click that.

You should now have your own JSCommunicator repository on your Github account. On your main page select the tab Repositories . It should be the first on the list. The url will be https://github.com/YOUR-USERNAME/jscommunicator. My Github username is JLouback, so my jscommunicator repo can be found at https://github.com/JLouback/jscommunicator. Clone Now you should download all that code to your local machine so you can edit it. On your JSCommunicator page somewhere around the lower right corner you ll see a field entitled HTTPS clone URL: I don t know if this is correct, but you can just copy the URL on your browser too. I do that. Now on your terminal, navigate to the directory you d like to place your repository and enter The current directory should now have a folder named jscommunicator and in it is all the JSCommunicator code. Switch branches The JSCommunicator repo has a branch for the i18n support. A branch is basically a copy of the code that you can modify without changing the original code. It s great for adding new features, once everything is done and tested you can add the new feature to the original code. A more detailed explanation of branches can be found here. On the terminal, enter This will list all the branches in the repository, both local and remote. There should be a branch named remotes/origin/i18n-support . Switch to this branch by entering This will create a local branch named i18n-support with all the contents of the remote i18n-branch in https://github.com/opentelecoms-org/jscommunicator. Create a .properties file Now your jscommunicator directory will have some new files for the internationalization functionality. Among them you should find an INTERNATIONALIZATION_README file with instructions for adding a JSCommunicator translation. It s a less detailed version of this post, but it s worth a read in case I ve missed something here. Go to the directory internationalization . You ll see a few .properties files with different language codes. Choose the language of your preference for the translation base and make a copy of it in the internationalization directory. Your copy should be named Messages_LANGUAGECODE.properties . For example, the language code for german is de , so the german file should be named Messages_de.properties . If you re not sure about the code for your language, please check out this list of ISO 639-1 language codes so you re using the same code as (most) browsers. The .properties file is list of key-value pairs, a word or a few words joined by _ which is the key, the equals sign = then a word or phrase which is the value. Do not change anything to the left of the equals sign. Translate what is on the right of the equals sign. Modify available_languages.xml Go back to the root directory (jscommunicator) and open the file named available_languages.ruby . This .ruby has a series of language elements. At the end of the last </language> add a new language element with your language display name and code, copying the previous languge elements. You actually can put your language element anwhere, as long as it s after the ' and before the , as well as not cutting into any of the other language elements. Your display name should be the name of the language in its native language, for example for a french translation we ll put Fran ais (I think). And the code is the code we used for the .properties file. Here s how it should look: Save your changes and voila! Test your work Once you ve made a .properties file, JSCommunicator will automatically load your translation if you ve set your browser preference to that language. If your browser preference is french, it will load the Messages_fr.properties. If your browser preference is a language we don t have a translation for (let s say german), JScommunicator will load the default Messages.properties file which is in english. The available_languages.ruby is used to build a language selection menu. If you add a language element without a corresponding .properties file, JSCommunicator will throw a JS error and load the default (english) translation. The same happens if you use different language codes in the .properties file name and the available_languages.ruby. The error won t disturb your use of JSCommunicator, it just won t load the language you selected. No harm, no foul. But if you want others to use your translation, do take care to do this correctly. If you read the INTERNATIONALIZATION_README, you will have seen that we need the jquery.i18n.properties.js file that s included in the .html pages. You can download that code here. Be sure to place it in the jscommunicator directory. This is only necessary if you want to see your addition at work, you don t need this file to contribute your translation. There are other 3rd party code dependencies to run JSCommunicator too as you may have noticed. Just creating the .properties file and adding a language element to available_languages.ruby, if done correctly, is enough to make a pull request. Commit Here s the part where we load all your local changes to your remote repository. In the terminal, navigate to the root directory (jscommunicator) and enter Of course, please replace Messages_de.properties with the name of your new .properties file. And mind you, we don t have a german translation yet! Then enter You can enter whatever you d like in the part, it s a good idea to explain what language you are adding. Now we can push the changes: You should now see the changes you made in your Github account page at github.com/YOUR_USER/jscommunicator. The message you put in double quotations ( ) in the commit will be beside each modified file. This push also creates a new branch in your jscommunicator repository, now you will have a master and a i18n-support branch. Pull Now it s time to share your translation with everyone else if you feel so inclined. Your version of JSCommunicator has a new translation but not the official version. First navigate to you jscommunicator repo at github.com/YOUR_USER/jscommunicator and make sure you are on the i18n-branch as that will contain your changes. Click the green button directly to the left of the branch drop down menu shown above. This will create a pull request to add your new code to the official jscommunicator code. Once you click that button, you should see this: Make sure that the base repo (the first on the line above the green Create pull request button) is opentelecoms-org:i18n-support and not opentelecoms-org:master or another branch name. If it is, just click Edit on the right and you ll be able to select the branch from a dropdown like so: If first repo is opentelecoms-org:i18n-support and the second is YOUR_USER:i18n-support and you ve verified (scroll down) that there are 2 files changed which are your new .properties file and the added element to the available_languages.xml, go ahead and press create pull request. It ll open a window with a text box you can add an explanation to with one final Create pull request button. Click that and you re done! You ve kindly contributed with a translation for JSCommunicator!

Today I am very happy to announce the release of AppStream 0.7, the second-largest release (judging by commit number) after 0.6. AppStream 0.7 brings many new features for the specification, adds lots of good stuff to libappstream, introduces a new libappstream-qt library for Qt developers and, as always, fixes some bugs. Unfortunately we broke the API/ABI of libappstream, so please adjust your code accordingly. Apart from that, any other changes are backwards-compatible. So, here is an overview of what s new in AppStream 0.7: Specification changes Distributors may now specify a new <languages/> tag in their distribution XML, providing information about the languages a component supports and the completion-percentage for the language. This allows software-centers to apply smart filtering on applications to highlight the ones which are available in the users native language. A new addon component type was added to represent software which is designed to be used together with a specific other application (think of a Firefox addon or GNOME-Shell extension). Software-center applications can group the addons together with their main application to provide an easy way for users to install additional functionality for existing applications. The <provides/> tag gained a new dbus item-type to expose D-Bus interface names the component provides to the outside world. This means in future it will be possible to search for components providing a specific dbus service: (if you are using the cli tool) A <developer_name/> tag was added to the generic component definition to define the name of the component developer in a human-readable form. Possible values are, for example The KDE Community , GNOME Developers or even the developer s full name. This value can be (optionally) translated and will be displayed in software-centers. An <update_contact/> tag was added to the specification, to provide a convenient way for distributors to reach upstream to talk about changes made to their metadata or issues with the latest software update. This tag was already used by some projects before, and has now been added to the official specification. Timestamps in <release/> tags must now be UNIX epochs, YYYYMMDD is no longer valid (fortunately, everyone is already using UNIX epochs). Last but not least, the <pkgname/> tag is now allowed multiple times per component. We still recommend to create metapackages according to the contents the upstream metadata describes and place the file there. However, in some cases defining one component to be in multiple packages is a short way to make metadata available correctly without excessive package-tuning (which can become difficult if a <provides/> tag needs to be satisfied). As small sidenote: The multiarch path in /usr/share/appdata is now deprecated, because we think that we can live without it (by shipping -data packages per library and using smarter AppStream metadata generators which take advantage of the ability to define multiple <pkgname/> tags) Documentation updates In general, the documentation of the specification has been reworked to be easier to understand and to include less duplication of information. We now use excessive crosslinking to show you the information you need in order to write metadata for your upstream project, or to implement a metadata generator for your distribution. Because the specification needs to define the allowed tags completely and contain as much information as possible, it is not very easy to digest for upstream authors who just want some metadata shipped quickly. In order to help them, we now have Quickstart pages in the documentation, which are rich of examples and contain the most important subset of information you need to write a good metadata file. These quickstart guides already exist for desktop-applications and addons, more will follow in future. We also have an explicit section dealing with the question How do I translate upstream metadata? now. More changes to the docs are planned for the next point releases. You can find the full project documentation at Freedesktop. AppStream GObject library and tools The libappstream library also received lots of changes. The most important one: We switched from using LGPL-3+ to LGPL-2.1+. People who know me know that I love the v3 license family of GPL licenses I like it for tivoization protection, it s explicit compatibility with some important other licenses and cosmetic details, like entities not loosing their right to use the software forever after a license violation. However, a LGPL-3+ library does not mix well with projects licensed under other open source licenses, mainly GPL-2-only projects. I want libappstream to be used by anyone without forcing the project to change its license. For some reason, using the library from proprietary code is easier than using it from a GPL-2-only open source project. The license change was also a popular request of people wanting to use the library, so I made the switch with 0.7. If you want to know more about the LGPL-3 issues, I recommend reading this blogpost by Nikos (GnuTLS). On the code-side, libappstream received a large pile of bugfixes and some internal restructuring. This makes the cache builder about 5% faster (depending on your system and the amount of metadata which needs to be processed) and prepares for future changes (e.g. I plan to obsolete PackageKit s desktop-file-database in the long term). The library also brings back support for legacy AppData files, which it can now read. However, appstream-validate will not validate these files (and kindly ask you to migrate to the new format). The appstream-index tool received some changes, making it s command-line interface a bit more modern. It is also possible now to place the Xapian cache at arbitrary locations, which is a nice feature for developers. Additionally, the testsuite got improved and should now work on systems which do not have metadata installed. Of course, libappstream also implements all features of the new 0.7 specification. With the 0.7 release, some symbols were removed which have been deprecated for a few releases, most notably as_component_get/set_idname, as_database_find_components_by_str, as_component_get/set_homepage and the pkgname property of AsComponent (which is now a string array and called pkgnames ). API level was bumped to 1. Appstream-Qt A Qt library to access AppStream data has been added. So if you want to use AppStream metadata in your Qt application, you can easily do that now without touching any GLib/GObject based code! Special thanks to Sune Vuorela for his nice rework of the Qt library! And that s it with the changes for now! Thanks to everyone who helped making 0.7 ready, being it feedback, contributions to the documentation, translation or coding. You can get the release tarballs at Freedesktop. Have fun!

A common theme over the last few decades was that we could afford to simply sit back and let computer (hardware) engineers take care of increases in computing speed thanks to Moore's law. That same line of thought now frequently points out that we are getting closer and closer to the physical limits of what Moore's law can do for us. So the new best hope is (and has been) parallel processing. Even our smartphones have multiple cores, and most if not all retail PCs now possess two, four or more cores. Real computers, aka somewhat decent servers, can be had with 24, 32 or more cores as well, and all that is before we even consider GPU coprocessors or other upcoming changes. And sometimes our tasks are embarassingly simple as is the case with many data-parallel jobs: we can use higher-level operations such as those offered by the base R package parallel to spawn multiple processing tasks and gather the results. I covered all this in some detail in previous talks on High Performance Computing with R (and you can also consult the Task View on High Performance Computing with R which I edit). But sometimes we can't use data-parallel approaches. Hence we have to redo our algorithms. Which is really hard. R itself has been relying on the (fairly mature) OpenMP standard for some of its operations. Luke Tierney's (awesome) keynote in May at our (sixth) R/Finance conference mentioned some of the issues related to OpenMP. One which matters is that OpenMP works really well on Linux, and either not so well (Windows) or not at all (OS X, due the usual issue with the gcc/clang switch enforced by Applem but the good news is that the OpenMP toolchain is expected to make it to OS X is some more performant form "soon"). R is still expected to make wider use of OpenMP in future versions. Another tool which has been around for a few years, and which can be considered to be equally mature is the Intel Threaded Building Blocks library, or TBB. JJ recently started to wrap this up for use by R. The first approach resulted in a (now superseded, see below) package TBB. But hardware and OS issues bite once again, as the Intel TBB is not really building that well for the Windows toolchain used by R (and based on MinGW). (And yes, there are two more options. But Boost Threads requires linking which precludes easy use as e.g. via our BH package. And C++11 with its threads library (based on Boost Threads) is not yet as widely available as R and Rcpp which means that it is not a real deployment option yet.) Now, JJ, being as awesome as he is, went back to the drawing board and integrated a second threading toolkit: TinyThread++, a small header-only library without further dependencies. Not as feature-rich as Intel Threaded Building Blocks, but at least available everywhere. So a new package RcppParallel, so far only on GitHub, wraps around both TinyThread++ and Intel Threaded Building Blocks and offers a consistent interface available on all platforms used by R. Better still, JJ also authored several pieces demonstrating this new package for the Rcpp Gallery:

• A parallel matrix transformation
• A parallel vector summation
• A parallel inner product
• Parallel Distance Matrix Calculation with RcppParallel

All four are interesting and demonstrate different aspects of parallel computing via RcppParallel. But the last article is key. Based on a question by Jim Bullard, and then written with Jim, it shows how a particular matrix distance metric (which is missing from R) can be implemented in a serial manner in both R, and also via Rcpp. The key implementation, however, uses both Rcpp and RcppParallel and thereby achieves a truly impressive speed gain as the gains from using compiled code (via Rcpp) and from using a parallel algorithm (via RcppParallel) are multiplicative! Between JJ's and my four-core machines the gain was between 200 and 300 fold---which is rather considerable. For kicks, I also used a much bigger machine at work which came in at an even larger speed gain (but gains become clearly sublinear as the number of cores increases; there are however some tuning parameters). So these are exciting times. I am sure there will be lots more to come. For now, head over to the RcppParallel package and start playing. Further contributions to the Rcpp Gallery are not only welcome but strongly encouraged.

Achievement unlocked: I have a new GPG key:

0xF22674467E4AF4A3

pub 4096R/7E4AF4A3 2014-07-13 [caduca: 2016-07-12]
Fingerprint = 445E 3AD0 3690 3F47 E19B 37B2 F226 7446 7E4A F4A3
uid Laura Arjona Reina <laura.arjona@upm.es>
uid Laura Arjona Reina <larjona@fsfe.org>
uid Laura Arjona Reina <larjona99@gmail.com>
sub 3072R/CC706B74 2014-07-13 [expires: 2016-07-12]
sub 3072R/7E51465F 2014-07-13 [expires: 2016-07-12]
sub 4096R/74C23D6E 2014-07-13 [expires: 2016-07-12]

The master key is 4096 bit, stored in a safe place, and 2 subkeys 3072 bit, stored in an FSFE Smartcard (I cannot store 4096 keys there).

I have carefully used the FSFE SmartCard Howto and Creating the perfect GPG keypair by Alex Cabal for strenghtening hash preferences and creating revocation certificate.

It seems everything works as intended. Passphrase is strong and this time I will not forget it. As first celebration, 1/2 lt icecream is waiting for me after dinner :)

People knowing me and around Madrid, please send me an encrypted mail as test or normal communication, and ping me to meet and sign keys :) One more step towards involvement in Debian and free software, controlling my digital life and communications, and becoming familiar with these technologies so I can teach them to my son as the natural way . Yay!
Filed under: My experiences and opinion, Tools Tagged: Contributing to libre software, Debian, encryption, English, Free Software, gpg, libre software, Moving into free software, mswl-cases

So I've recently posted a few links on Twitter, and I see followers clicking them. But also I see random hits. Tonight I posted a link to http://transient.email/, a domain I use for "anonymous" emailing, specifically to see which bots hit the URL. Within two minutes I had 15 visitors the first few of which were:

IP	User-Agent	Request
199.16.156.124	Twitterbot/1.0;	GET /robots.txt
199.16.156.126	Twitterbot/1.0;	GET /robots.txt
54.246.137.243	python-requests/1.2.3 CPython/2.7.2+ Linux/3.0.0-16-virtual	HEAD /
74.112.131.243	Mozilla/5.0 ();	GET /
50.18.102.132	Google-HTTP-Java-Client/1.17.0-rc (gzip)	HEAD /
50.18.102.132	Google-HTTP-Java-Client/1.17.0-rc (gzip)	HEAD /
199.16.156.125	Twitterbot/1.0;	GET /robots.txt
185.20.4.143	Mozilla/5.0 (compatible; TweetmemeBot/3.0; +http://tweetmeme.com/)	GET /
23.227.176.34	MetaURI API/2.0 +metauri.com	GET /
74.6.254.127	Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp);	GET /robots.txt

So what jumps out? The twitterbot makes several requests for /robots.txt, but never actually fetches the page itself which is interesting because there is indeed a prohibition in the supplied /robots.txt file. A surprise was that both Google and Yahoo seem to follow Twitter links in almost real-time. Though the Yahoo site parsed and honoured /robots.txt the Google spider seemed to only make HEAD requests - and never actually look for the content or the robots file. In addition to this a bunch of hosts from the Amazon EC2 space made requests, which was perhaps not a surprise. Some automated processing, and classification, no doubt. Anyway beer. It's been a rough weekend.

Review: Neptune's Brood, by Charles Stross

Series:	Freyaverse #2
Publisher:	Ace
Copyright:	July 2013
ISBN:	1-101-62453-1
Format:	Kindle
Pages:	325

Neptune's Brood is set in the same universe as Saturn's Children, but I wouldn't call it a sequel. It takes place considerably later, after substantial expansion of the robot civilization to the stars, and features entirely different characters (or, if there was overlap, I didn't notice). It also represents a significant shift in tone: while Saturn's Children is clearly a Heinlein pastiche and parody, Neptune's Brood takes its space opera more seriously. There is some situational humor assault auditors, for example but this book is played mostly straight, and I detected little or no Heinlein. This is Stross fleshing out his own space opera concept. This being Stross, that concept is not exactly conventional. This is a space opera about economics. Specifically, it's a space opera about interstellar economics, a debt pyramid, and a very interesting remapping of the continual growth requirements of capitalism to the outward expansion of colonization. The first-person protagonist comes from a "family" (as in Saturn's Children, the concept exists but involves rather more aggressive control of the instantiated "children") of bankers, but she is a forensic accountant and historian who specializes in analysis of financial scams. As you might expect, this is a significant clue about the plot. Neptune's Brood opens with Krina in search of her sister. She is supposed to be an itinerant scholar, moving throughout colonized space to spend some time with various scattered sisters, spreading knowledge and expanding her own. But it's clear from the start of the book that something else is going on, even before an assassin with Krina's face appears on her trail. Unfortunately, it takes roughly a third of the book to learn just what is happening beneath the surface, and most of that time is spent in a pointless interlude on a flying cathedral run by religious fanatics. The religion is a callback to Saturn's Children: robots who are trying to spread original humanity (the Fragiles) to the stars. This mostly doesn't go well, and is going particularly poorly for the ship that Krina works for passage on. But this is a brief gag that I thought went on much too long. The plot happens to Krina for this first section of the book rather than the other way around, little of lasting significance other than some character introductions occurs, and the Church itself, while playing a minor role in the later plot, is not worth the amount of attention that it gets. The best parts of the early book are the interludes in which Krina explains major world concepts to the reader. These are absolutely blatant infodumping, and I'm not sure how Stross gets away with them, but somehow he does, at least for me. They remind me of some of his blog posts, except tighter and fit into an interesting larger structure. Thankfully, once Krina finally arrives on Shin-Tethys, the plot improves considerably. There was a specific moment for me when the book became interesting: when Krina finds her sib's quarters in Shin-Tethys and analyzes what she finds there. It's the first significant thing in the book that she does rather than have done to her or thrust upon her, and she's a much better character when she's making decisions. This is also about the point where Stross starts fully explaining slow money, which is key to both the economics and the plot, and the plot starts to unwind its various mysteries and identify the motives of the players. Even then, Krina suffers from a lack of agency. Only at rare intervals does she get a chance to affect the story. Most of what she did of relevance to this book she did in the past, and while those descriptions of the backstory are interesting, they don't entirely make up for a passive protagonist. Thankfully, the other characters are varied and interesting enough, and the political machinations and cascading revelations captivating enough, that the last part of the book was very satisfying even with Krina on for the ride. This is a Stross novel, so it's full of two-dollar technical words mixed with technobabble. However, it shares with Saturn's Children the recasting of robots as the norm and fleshy humans as the exception, which means much of the technobabble is a straight substitution for our normal babble about meaty bodies and often works as an alienation technique. That makes it a bit more tolerable for me, although I still wished Stross would turn down the manic vocabulary in places. This bothers some people more than others; if you had no trouble with Accelerando, Neptune's Brood will pose no problems. I don't think the first section of this book was successful, but I liked the rest well enough to recommend it. If you like your space opera with a heavy dose of economics, a realistic attitude towards deep space exploration without faster-than-light technology, and a realistic perspective on the hostility of alien planets to Earth life, Neptune's Brood is a good choice. And any book that quotes David Graeber's Debt, and whose author has clearly paid attention to its contents, wins bonus points from me. Rating: 8 out of 10

While I was out at the (immensely impressive and equally enjoyable) useR! 2014 conference at UCLA, Conrad provided a bug-fix release 4.320 of Armadillo, the nifty templated C++ library for linear algebra. I quickly rolled that into RcppArmadillo release 0.4.320.0 which has been on CRAN and in Debian for a good week now. This release fixes some minor things with sparse and dense Eigen solvers (as well as one RNG issue probably of lesser interest to R users deploying the RNGs from R) as shown in the NEWS entry below.

Changes in RcppArmadillo version 0.4.320.0 (2014-07-03)

• Upgraded to Armadillo release Version 4.320 (Daintree Tea Raider)
  • expanded eigs_sym() and eigs_gen() to use an optional tolerance parameter
  • expanded eig_sym() to automatically fall back to standard decomposition method if divide-and-conquer fails
  • automatic installer enables use of C++11 random number generator when using gcc 4.8.3+ in C++11 mode

Courtesy of CRANberries, there is also a diffstat report for the most recent release. As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

As my previous post suggested I'd been running a service for a few years, using Redis as a key-value store. Redis is lovely. If your dataset will fit in RAM. Otherwise it dies hard. Inspired by Memcached, which is a simple key=value store, redis allows for more operations: using sets, using hashes, etc, etc. As it transpires I mostly set keys to values, so it crossed my mind last night an alternative to rewriting the service to use a non-RAM-constrained service might be to juggle redis out and replace it with something else. If it were possible to have a redis-compatible API which secretly stored the data in leveldb, sqlite, or even Berkley DB, then that would solve my problem of RAM-constraints, and also be useful. Looking around there are a few projects in this area nds fork of redis, ssdb, etc. I was hoping to find a Perl Redis::Server module, but sadly nothing exists. I should look at the various node.js stub-servers which exist as they might be easy to hack too. Anyway the short version is that this might be a way forward, the real solution might be to use sqlite or postgres, but that would take a few days work. For the moment the service has been moved to a donated guest and has 2Gb of RAM instead of the paltry 512Mb it was running on previously. Happily the server is installed/maintained by my slaughter tool so reinstalling took about ten minutes - the only hard part was migrating the Redis-contents, and that's trivial thanks to the integrated "slave of" support. (I should write that up regardless though.)

I recently got in contact with Nicolas Arenas Alonso and Nito Martinez from the Quindel group (located in Spain) [1]. Those guys bring forth a software product called TheQVD (The Quality Virtual Desktop) [2]. The project does similar things that X2Go does. In fact, they use NX 3.5 from NoMachine internally like we do in X2Go. Already a year ago, I noticed their activity on TheQVD and thought.. "Ahaaa!?!". Now, a couple of weeks back we received a patch for libxcomp3 that fixes an FTBFS (fails to build from source) for nx-libs-lite against Android [3]. read more

Once upon a time I setup a centralized service for spam-testing blog/forum-comments in real time, that service is BlogSpam.net. This was created because the Debian Administration site was getting hammered with bogus comments, as was my personal blog. Today the unfortunate thing happened, the virtual machine this service was running on ran out of RAM and died - The redis-store that holds all the state has now exceeded the paltry 512Mb allocated to the guest, so OOM killed it. So I'm at an impasse - I either recode it to use MySQL instead of Redis, or something similar to allow the backing store to exceed the RAM-size, or I shut the thing down. There seems to be virtually no liklihood somebody would sponsor a host to run the service, because people just don't pay for this kind of service. I've temporarily given the guest 1Gb of RAM, but that comes at a cost. I've had to shut down my "builder" host - which is used to build Debian packages via pbuilder. Offering an API, for free, which has become increasingly popular and yet equally gets almost zero feedback or "thanks" is a bit of a double-edged sword. Because it has so many users it provides a better service - but equally costs more to run in terms of time, effort, and attention. (And I just realized over the weekend that my Flattr account is full of money (~50 euro) that I can't withdraw - since I deleted my paypal account last year. Ooops.) Meh. Happy news? I'm avoiding the issue of free service indefinitely with the git-based DNS product which was covering costs and now is .. doing better. (20% off your first months bill with coupon "20PERCENT".)

Back in March, I uploaded a Debian s version of lcdproc with a unique feature: user and maintainer configurations are merged during package upgrade: user customizations and developers enhancements are both preserved in the new configuration file. (See this blog for more details). This avoids tedious edition of the configuration LCDd.conf file after every upgrade of lcdproc package. At the beginning of June, a new version of lcdproc (0.5.7-1) was uploaded. This triggered another round of automatic upgrades on user s systems. According to the popcon rise of libconfig-model-lcdproc-perl, about 100 people have upgraded lcdproc on their system. Since automatic upgrade has an opt-out feature, one cannot say for sure that 100 people are actually using automatic upgrade, but I bet a fair portion are them are. So far, only one people has complained: a bug report was filed about the many dependencies brought by libconfig-model-lcdproc-perl. The next challenge for lcdproc configuration upgrade is brought by a bug reported on Ubuntu: the device file provided by imon kernel module is a moving target: The device file created by the kernel can be /dev/lcd0 or /dev/lcd1 or even /dev/lcd2. Static configuration files and moving target don t mix well. The obvious solution is to provide a udev rule so that a symbolic link is created from a fixed location (/dev/lcd-imon) to the moving target. Once the udev rule is installed, the user only has to update LCDd.conf file to use the symlink as imon device file and we re done. But, wait The whole point of automatic configuration upgrade is to spare the user this kind of trouble: the upgrade must be completely automatic. Moreover, the upgrade must work in all cases: whether udev is available (Linux) or not. If udev is not available, the value present in the configuration file must be preserved. To know whether udev is available, the upgrade tool (aka cme) will check whether the file provided by udev (/dev/lcd-imon) is present or not. This will be done by lcdproc postinst script (which is run automatically at the end of lcdproc upgrade). Which means that the new udev rule must also be
activated in the postinst script before the upgrade is done. In other words, the next version of lcdproc (0.5.7-2) will:

• Install a new udev rule to provide lcd-imon symbolic link
• Activate this rule in lcdproc postinst script before upgrading the configuration (note to udev experts: yes, the udev rule is activated with --action=change option)
• Upgrade the configuration by running cme migrate in lcdproc postinst script.

In the lcdproc configuration model installed by libconfig-model-lcdproc-perl, the imon device parameter is enhanced so that running cme check lcdproc or cme migrate lcdproc issues a warning if /dev/lcd-imon exists and if imon driver is not configured to use it. This way, the next installation of lcdproc will deliver a fix for imon and cme will fix user s configuration file without requiring user input. The last point is admittedly bad marketing as users will not be aware of the magic performed by Config::Model Oh well In the previous section, I ve briefly mentioned that imon_device parameter is enhanced in lcdproc configuration model. If you re not already bored, let s lift the hood and see what kind of enhancements was added. Let s peek in lcdproc configuration file, LCDd.conf file which is used to generate lcdproc configuration model. You may remember that the formal description of all LCDd.conf parameters and their properties is generated from LCDd.conf to provide lcdproc configuration model. The comments in LCDd.conf follow a convention so that most properties of the parameters can be extracted from the comments. In the example below, the comments show that NewFirmware is a boolean value expressed as yes or no, the latter being the default :

# Set the firmware version (New means >= 2.0) [default: no; legal: yes, no]
NewFirmware=no

Back to the moving target. In LCDd.conf, imon device file parameter is declared this way:

# Select the output device to use
Device=/dev/lcd0

This means that device is a string where the default value is /dev/lcd0. Which is wrong once the special udev rule provided with Debian packages is activated. With this rule, the default value must be /dev/lcd-imon. To fix this problem, a special comment is added in the Debian version of LCDd.conf to tune further the properties of the device parameter:

# select the device to use
#  %
#   default~
#   compute
#     use_eval=1
#     formula="my $l = '/dev/lcd-imon'; -e $l ? $l : '/dev/lcd0';"
#     allow_override=1 -
#   warn_if:not_lcd_imon
#     code="my $l = '/dev/lcd-imon';defined $_ and -e $l and $_ ne $l ;"
#     msg="imon device does not use /dev/lcd-imon link."
#     fix="$_ = undef;"
#   warn_unless:found_device_file
#     code="defined $_ ? -e : 1"
#     msg="missing imon device file"
#     fix="$_ = undef;"
#   - % 
Device=/dev/lcd0

This special comment between % and % follows the syntax of Config::Model::Loader. A small configuration model is declared there to enhance the model generated from LCDd.conf file. Here are the main parts:

• default~ suppress the default value of the device parameter declared in the original LCDd.conf (i.e. /dev/ldcd0 )
• compute and the 3 lines below computes a default value for the device file. Since use_eval is true, the formula is evaluated as Perl code. This code will return /dev/lcd-imon if this file is found. Otherwise, /dev/lcd0 is returned. Hence, either /dev/lcd-imon or /dev/lcd0 will be used a as default value. allow_override=1 lets the user override this computed value
• warn_if and the 3 lines below test the configured device file with the Perl instructions provided by the code parameter. There, the device value is available in the $_ variable. This code will return true if /dev/lcd-imon exists and if the configured device does not use it. This will trigger a warning that will show the specified message.
• Similarly warn_unless and the 3 lines below warns the user if the configured device file is not found.

In both warn_unless and warn_if parts, the fix code snippet is run when by the command cme fix lcdproc and is used to repair the warning condition. In this case, the fix consists in resetting the device configuration value so the computed value above can be used. cme fix lcdproc is triggered during package post install script installed by dh_cme_upgrade. Come to think of it, generating a configuration model from a configuration file can probably be applied to other projects: for instance, php.ini and kdmrc are also shipped with detailed comments. May be I should make a more generic model generator from the example used to generate lcdproc model Well, I will do it if people show interest. Not in the form yeah, that would be cool , but in the form, yes, I will use your work to generate a configuration model for project [...] . I ll let you fill the blank ;-)
Tagged: Config::Model, configuration, debian, lcdproc, Perl, upgrade

For my local backup regimen I use flexbackup to create a full backup twice a year and differential/incremental backups on a weekly/monthly basis. I then upload these to a new amazon S3 bucket for each half year (so each bucket corresponds to the a full backup plus the associated differentials and incrementals). I then set the bucket's lifecycle to archive to glacier (cheaper offline storage) from the month after that half year has ended (reducing costs) and to delete it a year after the half ends. It used to be possible to do this via the S3 web interface but the absolute date based options seem to have been removed in favour of time since last update, which is not what I want. However the UI will still display such lifecycles if they are configured and directs you to the REST API to set them up. I had a look around but couldn't any existing CLI tools to do this directly but I figured it must be possible with curl. A little bit of reading later I found that it was possible but it involved some faff calculating signatures etc. Luckily EricW has written Amazon S3 Authentication Tool for Curl (AKA s3curl) which automates the majority of that faff. The tool is "New BSD" licensed according to that page or Apache 2.0 license according to the included LICENSE file and code comments. Setup Following the included README setup ~/.s3curl containing your id and secret key (I called mine personal which I then use below). Getting the existing lifecycle Retrieving an existing lifecycle is pretty easy. For the bucket which I used for the first half of 2014:

$ s3curl --id=personal -- --silent http://$bucket.s3.amazonaws.com/?lifecycle   xmllint --format -
<?xml version="1.0" encoding="UTF-8"?>
<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Rule>
    <ID>Archive and Expire</ID>
    <Prefix/>
    <Status>Enabled</Status>
    <Transition>
      <Date>2014-07-31T00:00:00.000Z</Date>
      <StorageClass>GLACIER</StorageClass>
    </Transition>
    <Expiration>
      <Date>2015-01-31T00:00:00.000Z</Date>
    </Expiration>
  </Rule>
</LifecycleConfiguration>

See GET Bucket Lifecycle for details of the XML. Setting a new lifecycle The desired configuration needs to be written to a file. For example to set the lifecycle for the bucket I'm going to use for the second half of 2014:

$ cat s3.lifecycle
<LifecycleConfiguration>
  <Rule>
    <ID>Archive and Expire</ID>
    <Prefix/>
    <Status>Enabled</Status>
    <Transition>
      <Date>2015-01-31T00:00:00.000Z</Date>
      <StorageClass>GLACIER</StorageClass>
    </Transition>
    <Expiration>
      <Date>2015-07-31T00:00:00.000Z</Date>
    </Expiration>
  </Rule>
</LifecycleConfiguration>
$ s3curl --id=personal --put s3.lifecycle --calculateContentMd5 -- http://$bucket.s3.amazonaws.com/?lifecycle

See PUT Bucket Lifecycle for details of the XML.

When I first started using computers a word processor was a program that edited text. The most common and affordable printers were dot-matrix and people who wanted good quality printing used daisy wheel printers. Text from a word processor was sent to a printer a letter at a time. The options for fancy printing were bold and italic (for dot-matrix), underlines, and the use of spaces to justify text. It really wasn t much good if you wanted to include pictures, graphs, or tables. But if you just wanted to write some text it worked really well. When you were editing text it was typical that the entire screen (25 rows of 80 columns) would be filled with the text you were writing. Some word processors used 2 or 3 lines at the top or bottom of the screen to display status information. Some time after that desktop publishing (DTP) programs became available. Initially most people had no interest in them because of the lack of suitable printers, the early LASER printers were very expensive and the graphics mode of dot matrix printers was slow to print and gave fairly low quality. Printing graphics on a cheap dot matrix printer using the thin continuous paper usually resulted in damaging the paper a bad result that wasn t worth the effort. When LASER and Inkjet printers started to become common word processing programs started getting many more features and basically took over from desktop publishing programs. This made them slower and more cumbersome to use. For example Star Office/OpenOffice/LibreOffice has distinguished itself by remaining equally slow as it transitioned from running on an OS/2 system with 16M of RAM in the early 90 s to a Linux system with 256M of RAM in the late 90 s to a Linux system with 1G of RAM in more recent times. It s nice that with the development of PCs that have AMD64 CPUs and 4G+ of RAM we have finally managed to increase PC power faster than LibreOffice can consume it. But it would be nicer if they could optimise for the common cases. LibreOffice isn t the only culprit, it seems that every word processor that has been in continual development for that period of time has had the same feature bloat. The DTP features that made word processing programs so much slower also required more menus to control them. So instead of just having text on the screen with maybe a couple of lines for status we have a menu bar at the top followed by a couple of lines of toolbars , then a line showing how much width of the screen is used for margins. At the bottom of the screen there s a search bar and a status bar. Screen Layout By definition the operation of a DTP program will be based around the size of the paper to be used. The default for this is A4 (or Letter in the US) in a portrait layout (higher than it is wide). The cheapest (and therefore most common) monitors in use are designed for displaying wide-screen 16:9 ratio movies. So we have images of A4 paper with a width:height ratio of 0.707:1 displayed on a wide-screen monitor with a 1.777:1 ratio. This means that only about 40% of the screen space would be used if you don t zoom in (but if you zoom in then you can t see many rows of text on the screen). One of the stupid ways this is used is by companies that send around word processing documents when plain text files would do, so everyone who reads the document uses a small portion of the screen space and a large portion of the email bandwidth. Note that this problem of wasted screen space isn t specific to DTP programs. When I use the Google Keep website [1] to edit notes on my PC they take up a small fraction of the screen space (about 1/3 screen width and 80% screen height) for no good reason. Keep displays about 70 characters per line and 36 lines per page. Really every program that allows editing moderate amounts of text should allow more than 80 characters per line if the screen is large enough and as many lines as fit on the screen. One way to alleviate the screen waste on DTP programs is to use a landscape layout for the paper. This is something that all modern printers support (AFAIK the only printers you can buy nowadays are LASER and ink-jet and it s just a big image that gets sent to the printer). I tried to do this with LibreOffice but couldn t figure out how. I m sure that someone will comment and tell me I m stupid for missing it, but I think that when someone with my experience of computers can t easily figure out how to perform what should be a simple task then it s unreasonably difficult for the vast majority of computer users who just want to print a document. When trying to work out how to use landscape layout in LibreOffice I discovered the Web Layout option in the View menu which allows all the screen space to be used for text (apart from the menu bar, tool bars, etc). That also means that there are no page breaks! That means I can use LibreOffice to just write text, take advantage of the spelling and grammar correcting features, and only have screen space wasted by the tool bars and menus etc. I never worked out how to get Google Docs to use a landscape document or a single webpage view. That s especially disappointing given that the proportion of documents that are printed from Google Docs is probably much lower than most word processing or DTP programs. What I Want What I d like to have is a word processing program that s suitable for writing draft blog posts and magazine articles. For blog posts most of the formatting is done by the blog software and for magazine articles the editorial policy demands plain text in most situations, so there s no possible benefit of DTP features. The ability to edit a document on an Android phone and on a Linux PC is a good feature. While the size of a phone screen limits what can be done it does allow jotting down ideas and correcting mistakes. I previously wrote about using Google Keep on a phone for lecture notes [2]. It seems that the practical ability of Keep to edit notes on a PC is about limited to the notes for a 45 minute lecture. So while Keep works well for that task it won t do well for anything bigger unless Google make some changes. Google Docs is quite good for editing medium size documents on a phone if you use the Android app. Given the limitations of the device size and input capabilities it works really well. But it s not much good for use on a PC. I ve seen a positive review of One Note from Microsoft [3]. But apart from the fact that it s from Microsoft (with all the issues that involves) there s the issue of requiring another account. Using an Android phone requires a Gmail account (in practice for almost all possible uses if not in theory) so there s no need to get an extra account for Google Keep or Docs. What would be ideal is an Android editor that could talk to a cloud service that I run (maybe using WebDAV) and which could use the same data as a Linux-X11 application. Any suggestions?

• [1] http://keep.google.com/
• [2] http://etbe.coker.com.au/2014/04/19/phone-based-lectures/
• [3] http://www.onenote.com/

Maybe do you remember, last year I mentored a Google Summer of code whose aim was to replace our well known Package Tracking System with something more modern, usable by derivatives and more easily hackable. The result of this project is a new Django-based software called Distro Tracker. With the help of the Debian System Administrators, it s now setup on tracker.debian.org! This service is also managed by the Debian QA team, it s deployed in /srv/tracker.debian.org/ (on ticharich.debian.org, a VM) if you want to verify something on the live installation. It runs under the qa user (so members of the qa-core group can administer it). That said you can reproduce the setup on your workstation quite easily, just by checking out the git repository and applying this change: Speaking of contributing, the documentation includes a Contributing section to get you up and running, ready to do your first contribution! Now go use this new service and report any issue against the new tracker.debian.org pseudo-package (BTW tracker.debian.org knows about pseudo-packages, example here). There are many small things that need to be fixed/improved, if you know Python/Django and would like to start contributing to Debian, here s your chance!

10 comments Liked this article? Click here. My blog is Flattr-enabled.

This release fixes an embarassing bug that caused the server to often abort sending information back to the client if the server-side program exited while the server was still sending data. It also fixes a more minor bug in which children of the server were spawned with SIGPIPE ignored instead of set to the default handling behavior. The major new feature in this release is a new ACL type, localgroup, contributed by Remi Ferrand. When this ACL type is used, the remote principal is converted to a local username using the normal Kerberos krb5_aname_to_localname call, and then the server checks whether that local username is a member of a local (nsswitch) group. This release also prefers calloc and reallocarray to malloc and realloc with calculated sizes across all code, and adds version and compatibility information to all manual pages. You can get the latest release from the remctl distribution page.

Review: Lockstep, by Karl Schroeder

Publisher:	Tor
Copyright:	March 2014
ISBN:	0-7653-3726-6
Format:	Hardcover
Pages:	351

Toby McGonigal's family fled an Earth dominated by trillionaires and vicious class conflict, and attempt, instead, a very risky and precarious settlement on a trans-Neptunian object. It's the last unclaimed but marginally habitable space left in the solar system, and securing their claim requires constant bureaucratic hoop-jumping. That's what sent Toby on a solo mission to a distant moon of their home to claim it and solidify their title. But, while in cold sleep, his craft hits a small chunk of rock, and he wakes up near another world: cold, silent, dead... but apparently with lifeless cities. Another trip through suspended animation in a desperate attempt to conserve resources against the distant hope of rescue has him awakening in a society both utterly foreign and yet strangely familiar. Schroeder is one of the better big idea writers in science fiction, but I found Lockstep to be a mixed bag. He does a surprisingly good job with the core conceit of the novel (more on that in a moment), even though it's a tricky one to make believable. Surrounding that, though, are a lot of less convincing bits that I kept having to not think about too hard, such as Toby crossing the path of another trans-Neptunian object after the accident (space is really, really big and really, really empty), or the implications of later revelations about the time scales involved in parts of the plot. Some parts of the world building, even if scientifically plausible, struck me as sloppy; for example, a religion plays a prominent role in the plot, but the nature of that religion was not particularly believable, nor was its interaction with the plot climax. I won't go into details about the religion, since it's a major plot point, but the short version is that religions tend to mature from the concrete to the abstract, not the other way around. That said, the core conceit is both surprising and considerably better-defended in the book than I thought it could be. The world into which Toby awakes is the world of the locksteps: a society that uses suspended animation in a universal and coordinated way to build a functional society on the far outskirts of the solar system. Humans emerge for some short period of time, like a month, and build, trade, interact, and consume. They welcome ships traveling from other trans-Neptunian worlds and prepare for their own journeys. And then they go into suspended animation for an extended period 15 years and 30 years are common choices while robots slowly gather more resources and energy, and repair and replace what's consumed in the month of active life. Ships travel with passengers in suspended animation, allowing the vast distances between these cold worlds to be reached during the sleep period. And, since all members of the lockstep sleep and wake on the same schedule, there is no wrenching desynchronization with the surrounding society during travel. One may spend thirty years in transit, but no time passes for anyone else in the lockstep while you're traveling either. A world can effectively trade with all other worlds within a thirty year travel radius without noticing the elapsed time. This is a technological system that at first doesn't sound like it would work, but Schroeder does a great job defending it and chasing down implications. The lockstep civilization serves as a sort of anchor and time capsule separate from the more frantic pace of the so-called fast worlds. The trans-Neptunian lockstep worlds are remote enough and poor enough to not attract too much unwanted attention, and are thus tolerated by fast societies that may have more available resources. (Although, to be sure, automated robot defenses have to carry a lot of weight here given how helpless the locksteps are during a sleep cycle.) I'm not sure I completely bought the sociology, but it works well enough to carry the weight of the story, and I've never seen a science fiction construction that uses the common technology of suspended animation in quite this way before. Schroeder sets up some nice stabilizing tensions between resources and time, adds some believable political reasons for this fragile society to survive, and uses some of its obvious political vulnerabilities as story drivers. The plot, unfortunately, isn't as good as the big idea, and particularly suffers at the start of the book. Toby is a teenager without much experience (Lockstep is marketed as young adult), and is immediately thrown into a strange and quickly hostile environment. This means that he spends the first half of the book reacting to a blizzard of new information, and the plot tends to revert to a tour of Schroeder's constructed world. Toby is also a bit of a cipher and a bit of an everyman, without many feelings or opinions beyond the obvious feelings that any teenage boy would go through in this situation. That makes the sense of a world tour even stronger. This flaw does not persist through the whole book. Toby does eventually start making decisions and doing things, some of the supporting characters add additional depth, and I found the unwinding of the plot surprisingly satisfying. The nature of time in this world lets Schroeder have both epic sweep and personal connection at the same time, which lets him pull off some neat contrasts between the personal and the political. I also liked Toby's gradual piecing together of what actually happened while he was asleep, both in the construction of society as a whole and in the personal conflicts between people he knew well. Some of the ease of grand manipulation seemed dubious to me, but, in Schroeder's defense, people do develop a reverence for and stories about things that have lasted a very long time, and Schroeder's setup gives him quite a lot to work with in that department. So, a mixed bag. The core concept is thought-provoking and up to Schroeder's usual standards. The surrounding world-building isn't as much, and I think Schroeder reaches for some too-easy explanations and still underplays just how many disruptive things can happen over the span of time that this book covers. The characterization I found weak and unsatisfying for much of the book, but it gets moderately better in the end. There's a bit too much tour, and a bit too much world exploration instead of plot, but it's a fascinating world and I still enjoyed the tour. With stronger characters and a few fewer dubious supporting pillars in the world background, I think this would have been an excellent book. As is, it's an enjoyable novel set firmly in the big idea and deep future end of science fiction, and I'm always happy to see more stories like that. It's not the best that Schroeder has done, but I still recommend it. Rating: 7 out of 10