So SteamOS has been released. While that's marginally interesting in and as of itself, there are two observations to be made:

1. Microsoft, Nintendo, and Sony will feel an impact. More functionality on cheaper hardware which can easily by upgraded; I bet quite a few managers are not happy, at the moment.
2. While the stand-alone Linux Steam client was initially targeted at Ubuntu, SteamOS is based on Debian Wheezy.

Actual Linux (not Android) for the end user The first one means more Linux installations. In the living room. On a machine that children are really focussed on and will want to play with, quite literally. The next logical step is for people who play games to install SteamOS on their other machines; desktops, laptops, everywhere they want to game. This could really be the tipping point where the average adolescent computer enthusiast does not need to reboot into Linux to fool around with, but the other way round: To need to boot to Windows for a few select legacy applications which don't run on the FLOSS variant of Wine like Office or Photoshop. And once this momentum starts to shift, other software vendors will follow the money trail. Could 2014 finally be.... the year of Linux on the desktop...? Debian vs Debian-based The latter one is also really interesting... Obviously, I don't know why Valve decided to go down this road, but there are several reasons which come to mind:

• No need for the extra bloat
• They wanted to avoid the tie-in with another for-profit entity
• Unhappiness with some technical decisions made by UbuntuCanonical
• Lack of faith in the long-term governance of Ubuntu

What we are left with is a major player entering the ring of Linux for end-users and choosing Debian over Ubuntu. Hopefully, improvements to the base system will be fed back upstream, enabling all Debian-based distributions to profit easily, not only Ubuntu-based ones. I am willing to bet that two years ago, SteamOS would have been based on Ubuntu, not Debian. Recently, there's been a lot of backlash over various decisions which Canonical forced onto Ubuntu and it will be interesting to see how this plays out in the long run... I will be interesting to see how much pain Linux Mint and Kubuntu will endure. Forecast Users All in all, we are looking at a massive influx of new users into the Debian ecosystem. How massive? 65 million registered users massive. 7 million concurrent users at once, 1.2 million users actively playing the top 100 games at the same time massive. This is huge. Contributors In time, a substantial part of that userbase will switch over one or more of their machines over to SteamOS. The tinkerers among them will realize they can install plain Debian and install Steam as a package. The hackers among those will start to improve upon their systems; and what better way to do that then to go upstream? If even a tiny fraction of users makes it this far, the count of actively involved contributors with Debian will skyrocket if we let them join. Raspbian and some other not-quite-ideal decisions come to mind. Vendors Commercial software vendors need to stay profitable. Thus, they are forced to support distributions which promise enough paying users. In the past, this meant mainly SuSE and Red Hat; they had commercial backers, went through certifications, etc. In the recent past, this also meant Ubuntu. All of a sudden, Debian stable has a potential market of tens of millions of average computer users and computer enthusiasts. A lot of whom will want to continue to use their OS of choice at work, as well. Oh boy...

Cleaning up X server warnings So I was sitting in the Narita airport with a couple of other free software developers merging X server patches. One of the developers was looking over my shoulder while the X server was building and casually commented on the number of warnings generated by the compiler. I felt like I had invited someone into my house without cleaning for months embarrassed and ashamed that we d let the code devolve into this state. Of course, we ve got excuses the X server code base is one of the oldest pieces of regularly used free software in existence. It was started before ANSI-C was codified. No function prototypes, no const , no void * , no enums or stdint.h. There may be a few developers out there who remember those days (fondly, of course), but I think most of us are glad that our favorite systems language has gained a lot of compile-time checking in the last 25 years. We ve spent time in the past adding function prototypes and cleaning up other warnings, but there was never a point at which the X server compiled without any warnings. More recently, we ve added a pile of new warning flags when compiling the X server which only served to increase the number of warnings dramatically. The current situation With the master branch of the X server and released versions of the dependencies, we generate 1047 warnings in the default build. -Wcast-qual considered chatty The GCC flag, -Wcast-qual, complains when you cast a pointer and change the const qualifier status. A very common thing for the X server to do is declare pointers as const to mark them as immutable once assigned. Often, the relevant data is actually constructed once at startup in allocated memory and stored to the data structure. During server reset, that needs to be freed, but free doesn t take a const pointer, so we cast to (void *), which -Wcast-qual then complains about. Loudly. Of the 1047 warnings, 380 of them are generated by this one warning flag. I ve gone ahead and just disabled it in util/macros for now. String constants are a pain The X server uses string constants to initialize defaults for font paths, configuration options, font names along with a host of other things. These end up getting stored in variables that can also take allocated storage. I ve gone ahead and declared the relevant objects as const and then fixed the code to suit. I don t have a count of the number of warnings these changes fixed; they were scattered across dozens of X server directories, and I was fixing one directory at a time, but probably more than half of the remaining warnings were of this form. And a host of other warnings Fixing the rest of the warnings was mostly a matter of stepping through them one at a time and actually adjusting the code. Shadowed declarations, unused values, redundant declarations and missing printf attributes were probably the bulk of them though. Changes to external modules Instead of just hacking the X server code, I ve created patches for other modules where necessary to fix the problems in the right place.

• proto/fontsproto. Declares FontPathElement names as const char *
• mesa/drm. Adds printf attribute to the debug_print function
• util/macros. Removes -Wcast-qual from the default warning set.

Getting the bits In case it wasn t clear, the X server build now generates zero warnings on my machine. I m hoping that this will also be true for other people. Patches are available at:

xserver - git://people.freedesktop.org/~keithp/xserver warning-fixes
fontsproto - git://people.freedesktop.org/~keithp/fontsproto fontsproto-next
mesa/drm - git://people.freedesktop.org/~keithp/drm warning-fixes
util/macros - already upstream on master

Keeping our house clean Of course, these patches are all waiting until 1.15 ships so that we don t accidentally break something important. However, once they re merged, I ll be bouncing any patches which generate warnings on my system, and if other people find warnings when they build, I ll expect them to send patches as well. Now to go collect the tea cups in my office and get them washed along with the breakfast dishes so I won t be embarrassed if some free software developers show up for lunch today.

Fingerprints and Key IDsOpenPGPv4 fingerprints are made from an SHA-1 digest over the key's public key material, creation date, and some boilerplate. SHA-1 digests are 160 bits in length. The "long key ID" of a key is the last 64 bits of the key's fingerprint. The "short key ID" of a key is the last 32 bits of the key's fingerprint. You can see both of the key IDs as a hash in and of themselves, as "32-bit truncated SHA-1" is a sort of hash (albeit not a cryptographically secure one). I'm arguing here that short Key IDs and long Key IDs are actually useless, and we should stop using them entirely where we can do so. We certainly should not be exposing normal human users to them. (Note that I am not arguing that OpenPGP v4 fingerprints themselves are cryptographically insecure. I do not believe that there are any serious cryptographic risks currently associated with OpenPGP v4 fingerprints. This post is about Key IDs specifically, not fingerprints.)Key IDs have serious problemsAsheesh pointed out two years ago that OpenPGP short key IDs are bad because they are trivial to replicate. This is called a preimage attack against the short key ID (which is just a truncated fingerprint). Today, David Leon Gil demonstrated that a collision attack against the long key ID is also trivial. A collision attack differs from a preimage attack in that the attacker gets to generate two different things that both have the same digest. Collision attacks are easier than preimage attacks because of the birthday paradox. dlg's colliding keys are not a surprise, but hopefully the explicit demonstration can serve as a wakeup call to help us improve our infrastructure. So this is not a way to spoof a specific target's long key ID on its own. But it indicates that it's more of a worry than most people tend to think about or plan for. And remember that for a search space as small as 64-bits (the long key ID), if you want to find a pre-image against any one of 2k keys, your search is actually only in a (64-k)-bit space to find a single pre-image. The particularly bad news: gpg doesn't cope well with the two keys that have the same long key ID:

0 dkg@alice:~$ gpg --import x
gpg: key B8EBE1AF: public key "9E669861368BCA0BE42DAF7DDDA252EBB8EBE1AF" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
0 dkg@alice:~$ gpg --import y
gpg: key B8EBE1AF: doesn't match our copy
gpg: Total number processed: 1
2 dkg@alice:~$ 

This probably also means that caff (from the signing-party package) will also choke when trying to deal with these two keys. I'm sure there are other OpenPGP-related tools that will fail in the face of two keys with matching 64-bit key IDs. We should not use Key IDs I am more convinced than ever that key IDs (both short and long) are actively problematic to real-world use of OpenPGP. We want two things from a key management framework: unforgability, and human-intelligible handles. Key IDs fail at both.

• Fingerprints are unforgable (as much as SHA-1's preimage resistance, anyway -- that's a separate discussion), but they aren't human-intelligible.
• User IDs are human-intelligible, and they are unforgable if we can rely on a robust keysigning network.
• Key IDs (both short and long) are neither human-intelligible nor unforgable (regardless of existence of a keysigning network), so they are the worst of all possible worlds.

So reasonable tools should not expose either short or long key IDs to users, or use them internally if they can avoid them. They do not have any properties we want, and in the worst case, they actively mislead people or lead them into harm. What reasonable tool should do that? How to replace Key IDs If we're not going to use Key IDs, what should we do instead? For anything human-facing, we should be using human-intelligible things like user IDs and creation dates. These are trivial to forge, but people can relate to them. This is better than offering the user something that is also trivial to forge, but that people cannot relate to. The job of any key management UI should be to interpret the cryptographic assurances provided by the certifications and present that to the user in a comprehensible way. For anything not human-facing (e.g. key management data storage, etc), we should be using the full key itself. We'll also want to store the full fingerprint as an index, since that is used for communication and key exchange (e.g. on calling cards). There remain parts of the spec (e.g. PK-ESK, Issuer subpackets) that make some use of the long key ID in ways that provide some measure of convenience but no real cryptographic security. We should fix the spec to stop using those, and either remove them entirely, or replace them with the full fingerprints. These fixes are not as urgent as the user-facing changes or the critical internal indexing fixes, though. Key IDs are not useful. We should stop using them.Tags: collision, crypto, gpg, openpgp, pgp, security

I am happy to announce that OASIS v0.4.0 has just been released. OASIS is a tool to help OCaml developers to integrate configure, build and install systems in their projects. It should help to create standard entry points in the source code build system, allowing external tools to analyse projects easily. This tool is freely inspired by Cabal which is the same kind of tool for Haskell. You can find the new release here and the changelog here. More information about OASIS in general on the OASIS website. I have recently resumed my work on OASIS and this will be hopefully the new version that will lead to quicker iteration in the development of OASIS. The development process was slowdown by the fact, that I feared introducing new fields in _oasis or regression. This was a pain and I decided to change my development model. Features The most important step is the introduction of AlphaFeatures and BetaFeatures fields. They allow to introduce pieces of code that will only be activated if certain features are listed in those fields. It should help to be always ready to release. The features also cover other aspect like flag_tests and flag_docs which has been introduced in OASIS v0.3.0. In fact the features API is now used to introduce all enhancement while keeping backward compatibility with regard to OASISFormat. Rather than defining a ~since_version:0.3 for fields we use a feature that handle the maturity level of the feature. When I feel a specific feature is ready to ship, I just change the InDev Alpha to InDev Beta and then SinceVersion 0.4. On the long term, when we won't support anymore a version of OASIS that existed before the SinceVersion, the feature will always be true and I will fully integrate it in the code. The only constraint around features is: if you use AlphaFeatures or BetaFeatures field, you must use the latest OASISFormat... Features section in the manual. Example of features available:

• section_object: allow to create object (.cmo/.cmx) in _oasis
• pure_interface: an OCamlbuild feature that allows to handle .mli without a .ml file

Automate Another topic is automation of testing releases. For OASIS v0.3.0, I ran tests on all platforms manually, late in the development of v0.3.0 and it was painful to fix. So I have decided to setup a Jenkins instance that automate testing on Linux. On the long term, I plan to also setup a Mac OS X builder and start looking at Windows as well. This should help me catch errors early and be able to fix them quickly. However, for v0.4.0 I have decided to just release what I have and which has mainly be tested on Linux. The point here is to quickly release and iterate, rather than wait for perfection. Hopefully end user testing will allow to quickly discover new bugs. Time boxed release In the coming months, I will try to do time boxed releases. I will try to release version of OASIS every 15th of the month. The point here is to try to iterate faster and avoid long delay between release. See you in 1 month for the next release.

A new Armadillo release 3.930 came out a few days ago, with a very nice set of changes (see below). I rolled this into RcppArmadillo 0.3.930.0. However, one of these changes revealed that R shipped only the standard SVD for complex-valued matrices, and not the more performant divide-and-conquer approach. So in R builds using the default built-in Lapack, at least one CRAN package no longer built. After some back and forth, Conrad put some branching in the library to fall back to the standard SVD, and I added a built-time configuration test for an appropriate preprocessor directive used by the fallback code. This is now on which is now on CRAN and in Debian as RcppArmadillorelease 0.3.930.1, and Conrad will probably update the Armadillo page as well (though the fix is only needed with R's builtin Rlapack). Also of note is that R Core already added the missing Fortran routine zgesdd to R 3.1.0 (aka "R-devel") so this issue goes away with the next release. Also of note, I wrote up a short Rcpp Gallery post illustrating the performance gains available from divide-and-conquer SVD. The complete list of changes is below.

Changes in RcppArmadillo version 0.3.930.1 (2013-12-09)

• Upgraded to Armadillo release Version 3.930.1
  • Armadillo falls back to standard complex svd if the more performant divide-and-conquer variant is unavailable
• Added detection for Lapack library and distinguish between R's own version (withhout zgesdd) and system Lapack; a preprocessor define is set accordingly

Changes in RcppArmadillo version 0.3.930.0 (2013-12-06)

• Upgraded to Armadillo release Version 3.930 ("Dragon's Back")
  • added divide-and-conquer variant of svd_econ(), for faster SVD
  • added divide-and-conquer variant of pinv(), for faster pseudo-inverse
  • added element-wise variants of min() and max()
  • added size() based specifications of submatrix view sizes
  • added randi() for generating matrices with random integer values
  • added more intuitive specification of sort direction in sort() and sort_index()
  • added more intuitive specification of method in det(), .i(), inv() and solve()
  • added more precise timer for the wall_clock class when using C++11
• New unit tests for complex matrices and vectors

Courtesy of CRANberries, there is also a diffstat report for the most recent release As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

I ve come up with some new favorites this season. The boys and Laura were around for all three, and I am happy to report there were many kitchen smiles over these! From-Scratch Hot Chocolate There s something about hot chocolate made from scratch, with chocolate melted into milk, instead of a powder stirred in. It takes quite a bit more time, and probably has more calories, but it is quite delicious. The key to a delicious result where milk is concerned is to take things slow and keep stirring. You don t want the chocolate to scorch at the bottom of the pan. Heating up the milk before the chocolate should help things mix in more easily as well.

• Begin with 3 cups milk and 1 cup heavy whipping cream. Heat slowly over moderate to low heat, stirring periodically. Once you see bubbles start to form around the edges, it is plenty hot (or even a bit more hot than it needs to be).
• Add one cup of semisweet chocolate chips, 1 teaspoon sugar, and 1/2 tsp vanilla extract.
• Stir constantly until all the chocolate is melted and well mixed. There will still be some small bits of chocolate within, but if it is all done slowly like this, the chocolate should be pretty well melted.

The basis for this recipe was here, and it called for 2 cups milk and 2 cups half-and-half. I trust my heavy whipping cream was fine! <grin> There are also some other variations on that site. This nearly made my little cast iron kettle overflow, so next time I made a 3/4 recipe. Hot Spiced Cider We put up a Christmas tree yesterday, so I thought hot spiced cider would be perfect for the occasion. I went searching for recipes, and many of them called for cloves (which have to be sifted out later or put in a spice bag). I wasn t going to have time to delay two boys from setting up a Christmas tree long enough for that, so I found this basic recipe to work well. However, I, as usual, made some modifications ;-)

• Warm 4 cups apple cider (not juice, as the recipe suggests) in a pot.
• Add 1/2 tsp cinnamon
• Add 1/4 tsp nutmeg or allspice (I used allspice because I was mysteriously out of nutmeg, but will probably use nutmeg next time)
• Add 1 tbsp brown sugar
• Stir constantly until sufficiently dissolved. Pour immediately before drinking, as the contents will tend to separate.

Mmmmm . yum . Turkey or Chicken Noodle Soup The annual what to do with all that leftover turkey quest strikes again. I like chicken noodle soup, so why not a turkey noodle soup done the same way? Here s what I used, roughly, in my large 6-quart cast iron cooking pot (aka Dutch oven ):

• 9 cups chicken/turkey broth. Your own if you have it, or the canned variety works. Or make your own with boullion if you have it.
• 2 chopped yellow onions. (I added half a chopped red onion as well because I had it sitting around. Nobody complained, but 2.5 onions was a little much.)
• 4 tsp fresh basil or 1 tsp dried basil
• 4 tsp fresh oregano or 1 tsp dried oregano
• 1/2 tsp pepper
• 1/2 tsp salt
• 1 tsp beef bouillon
• 2 bay leaves
• 20 oz frozen mixed vegetables (I d probably add more than that next year; this wasn t quite enough)
• Plenty of wide egg noodles. The recipe I used called for 1 cup, which was laughably inadequate. I just dumped until it looked right, and then the package was almost empty so I dumped the rest in too.
• 4 cups cooked turkey or chicken, cubed (a kitchen scissors makes quick work of that)
• Two 14.5-oz cans diced tomatoes (do not drain)

Start with the broth, onion, basil, oregano, pepper, and bay leaf. Heat up the mixture and add the vegetables. Bring it to boiling, then add the uncooked noodles. Return to boiling, then reduce heat, cover, and simmer for 8 minutes. Add the turkey or chicken and diced tomatoes, and simmer until hot enough to serve. The nice thing about soups is that they freeze well and make great winter leftovers. This recipe makes quite a lot of soup; you may wish to halve it. This recipe was adapted from one in a Better Homes & Gardens cookbook.

There has been a lot of discussion recently about the concept of a Basic Income (Wikipedia), largely due to the efforts to change the Swiss constitution to provide a Basic Income [1]. The concept of a Basic Income is that residents get a fixed payment without having to be sick, disabled, looking for work, or eligible for other forms of social security. A Basic Income wouldn t replace all other forms of social security, one of the most obvious examples is that sick people will often need money for medical care in additional to living expenses. Also I believe that it shouldn t be means tested in any way. I think that one of the problems with current payment schemes is that there are complex eligibility criteria which require effort for the applicant and for government agencies to prevent accidental or fraudulent over-payment. The tax rates could be raised slightly to make it revenue neutral. Newstart In Australia the main form of social security for unemployed people at the moment is called Newstart [2]. Currently Newstart payments range from a maximum of $501 per fortnight for a single person ($13,026 per annum), to a maximum of $699.90 per fortnight for someone who is a carer. The Newstart payments start to decrease if the recipient earns more than $62 per fortnight. The minimum wage in Australia is $16.37 per hour for permanent work or $20.30 for casual work [3]. So if someone works for more than 3 hours at a casual rate (and I can t imagine 4 hours a fortnight being anything other than casual) then their Newstart payments will decrease. The payment decreases are fairly significant, for every dollar that is earned about 50 cents will be deducted from the payments. That s a great incentive to either avoid opportunities to do part-time work or to do cash-only work that s outside the tax system. The most obvious way of implementing a Basic Income would be to replace Newstart. Then anyone who is in that situation would be free to just not get a job which would be OK IMHO as people who don t want to work probably wouldn t do a good job if the government forced them to get a job. People who are unemployed who want to work could work as much as they want and scale up according to what their employer asks and how much money they need. Currently the full-time minimum wage is $622.20 per week (I m not sure exactly how they get that from $16.37). That s almost 2.5* the Newstart allowance for a single person (but less than twice the Newstart allowance for a carer). While Newstart (and the other forms of social security) don t provide a great income, it seems that the difference between Newstart and the minimum wage isn t that great particularly when you consider that working involves some expenses for travel etc. There doesn t seem to be a great financial incentive for someone to leave Newstart and get a minimum wage job. People Who Want Social Security Some people think it s great to get government payments while others find it embarrassing to need such payments and won t necessarily apply if they are eligible. I think that the current system of forcing people to apply for social security is a way of discouraging people who find themselves unexpectedly in a difficult situation but doesn t discourage people who are happy not to work. This seems to effectively reduce the incidence of payments to the people who most tax-payers would regard as the most worthy recipients. Economics Charles Stross wrote about some ideas related to this [4]. He suggests that as the workforce participation has been steadily reducing due to technology we should move to a social model that isn t based around working to live but working to buy luxuries that aren t covered by the Basic Income. One of the many economic changes related to a Basic Income is that the minimum wage could be smaller than it might otherwise be. For example if the minimum wage was decreased by the same amount that the Basic Income provided then the minimum income would remain the same while employers would pay less, this would affect the viability of certain types of contract work web sites if they were subject to minimum wage laws (currently they just ignore the minimum wage laws by paying based on job completion instead of hours worked). I don t think that the minimum wage should decrease that much though, currently employers are able to run viable businesses with the minimum wage laws and I don t think that a Basic Income should be used as a way of helping corporations avoid paying their employees. If we had a Basic Income then there s many ways that it could be used to stabilise the economy. If people could pay their rent even if they lost their job then a down-turn in one area of the economy wouldn t immediately affect other areas. Also if rent payments were deducted automatically from an account used to receive the Basic Income then landlords would be more likely to rent to poor people as they could be guaranteed to receive rent payments (it would be easy to have a contractual agreement for rent to take priority and have bank computers enforce that). The Implementation Problem I don t think that my idea would have any significant negative effects. It wouldn t decrease government revenues if tax was adjusted accordingly. It wouldn t make people stop working as people who don t want to work already avoid it. It would help people who are out of work to get work by reducing the barriers to entry in terms of paperwork and of unreasonable cuts to Newstart making it bad value to take part time work. I think that the big problem with implementing it is people who want to prevent poor people from having opportunities. They want to reduce social security and minimum wages even though such changes will in the long run only give less tax revenue and greater expense in law enforcement. It seems rather ironic that such hostility often comes from people at the low end of the middle class whos jobs are most likely to be at risk from new technology. As on-going technological development reduces the number of workers that are required to keep things running we need to have some form of payment to the people who aren t doing enough work to survive. A decent Basic Income is a much better option than giving Newstart payments and forcing a significant portion of the population into a degrading search for jobs that don t exist. As that s the inevitable future I think we should make political changes to deal with it sooner rather than later. However a Basic Income might be implemented now it s surely going to be a lot better than what might happen if we wait until the majority of the population are unemployed before doing something about it.

• [1] http://tinyurl.com/mwemen5
  
• [2] http://www.humanservices.gov.au/customer/services/centrelink/newstart-allowance
• [3] http://www.fairwork.gov.au/PAY/NATIONAL-MINIMUM-WAGE/pages/default.aspx
• [4] http://tinyurl.com/mjq3dwl

I spent yesterday at the very enjoyable Big Data Summit held at the University of Illinois Research Park at the edge of the University of Illinois at Urbana-Champaign. campus. My (short) presentation was part of a panel session on R and Big Data which Doug Simpson of the UIUC Statistics department had put together very well. We heard from a vendor / technology provider with Christopher Nguyen from Adatao talking about their "Big R", from industry with Andy Stevens talking about a number of some real-life challenges with big data at John Deere, from academia with Jonathon Greenberg talking about R and HPC for geospatial research and I added a few short comments and links about R, HPC and Rcpp. My few slides are now up on my talks / presentations page. Overall, a good day with a number of interesting presentations and of course a number of engaging hallway discussions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

digest version 0.6.4 is now on CRAN and in Debian. This is a pure maintenance release which should help with a build issue affecting users on Solaris. CRANberries provides the usual summary of changes to version 0.6.3. Our package is available via the R-Forge page leading to svn and tarball access, my digest page, the local directory here as well as via Debian and its mirrors.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

I ve did something I surely will (financially) regret, next year, and designated the Neo900 to be the successor to my PocketPC, due to the latter having only 64 MiB RAM and Geocaching applications being quite hungry. It s got a lovely hardware keyboard, a pen display like the PocketPC (as opposed to the wishy-washy displays that Android and iPhone have), not only GPS but also GLONASS, fully free software with mostly free firmware (I m okay with that, mostly), a Ctrl key (useful in ssh and locally and my text editor; ^I is Tab, so it s useful in shell, too), WLAN, UMTS (I don t think I need LTE and would rather it have the more RAM), USB host (OTG), and lots of other nice features. In short, it s a tinkerable device: one I can not only hack at, but also hack on. Since I use a dumbphone for mobile phone anyway (pro: separate battery from the toy PocketPC/Smartphone we re talking two+ weeks of battery time when using it here, and easier use and less bugs, and a reliable fallback when I tinker too much ), this is perfect for me. I m reposting this in the wlog mostly because it s an interesting technical and OSS project, and because if 1000 people want one it will get less expensive for all of us (while here shameless plug any sponsors willing to contribute some EUR so I don t ruin myself with this, in exchange for services of some kind?). I ll probably run Debian on it (unless it goes systemd), maybe in a chroot if the native OS has functionality needed that I can t simply put into packages; they say Maemo has much better power management, but considering most use will have GPS, GLONASS and backlight on, battery isn t going to last long anyway or maybe even native I ve been wanting to know what this freesmartphone stuff my m68k (Atari VM) buildd has been happily compiling, anyway and some sort of Geocaching application (ideally a cross between something online, CacheWolf and an offline OSM (with most of Europe, but uninteresting tags stripped) and possibly access to the GS Live API but nevertheless supporting TC, NC, OC, gpsgames too), and my usual mksh(1), GNU screen, jupp(1), lynx(1), ssh(1) toolchain.) Delivery is expected for mid to end of 2014, but once it s there I ll keep you informed

In early October, I had an opportunity to talk about Rcpp and RcppArmadillo at the Statistical Computing Seminar at the Booth School of Business at the University of Chicago. And then two weeks ago, I had an invitation to talk at the Center for Research Methods and Data Analysis at the University of Kansas where I covered similar material as well as ongoing work on the RcppZiggurat package (for which I should have an updated version soon). Slides from both talks are now at the top of my talks / presentations page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Replacing my phone My first Android phone was a ZTE Blade (sold as Orange San Francisco here in the UK). It originally shipped with Android 2.1 but was upgradable to 2.3 thanks to CyanogenMod (or other unofficial mods). However there's little sign of an upgrade to 4.x; the apps I want to use are pushing the limits of its CPU, RAM and internal storage; and I've never got very good at typing on a soft keyboard. So it seemed like time to get a newer phone with a hard keyboard (and still with a SD slot). After a little research with the CyanogenMod compatibility list and some time looking at reviews, I settled on the Samsung Galaxy S Relay 4G, which was exclusive to T-Mobile USA but is being resold through eBay. It shipped with Android 4.0 but I immediately installed CyanogenMod 10.2 (Android 4.3). This was a bit of an adventure as I bought the Blade with CM already installed and wasn't familiar with the multiple steps that were necessary. Copying my data The last step, and the real subject of this entry, was to move my data across. Much of this was on a SD card which I could simply plug into the Relay. The internal files had to be backed up onto this card and then restored, using the recovery environment (ClockworkMod) on each phone. After rebooting the Relay into the full Android system, my apps and settings were mostly present but I was immediately confronted with a series of error dialogs reporting that 'Unfortunately, Dialler has stopped' - and the same for 'Clock' and 'the process android.process.acore' (whatever that is). What went wrong There are thankfully more detailed error logs in the filesystem, under /data/system/dropbox (this is for logging at the Java level; if a process is terminated by a fatal signal it's logged to /data/tombstones). There are several ways to get at them, but I used recovery mode and adb to get a root shell where I could easily read and write files as necessary. The Contacts (aka People) database upgrade fails with an SQLiteException, apparently because it doesn't account for upgrading from the schema used in 2.3:

Process: android.process.acore
Flags: 0x883e45
Package: com.android.providers.applications v18 (4.3.1-f963020e38)
Package: com.android.providers.contacts v18 (4.3.1-f963020e38)
Package: com.android.providers.userdictionary v18 (4.3.1-f963020e38)
Build: samsung/apexqtmo/apexqtmo:4.1.2/JZO54K/T699UVBMC5:user/release-keys
android.database.sqlite.SQLiteException: no such column: phonebook_label (code 1): , while compiling: UPDATE raw_contacts SET display_name_source=?,display_name=?,display_name_alt=?,phonetic_name=?,phonetic_name_style=?,sort_key=?,phonebook_label=?,phonebook_bucket=?,sort_key_alt=?,phonebook_label_alt=?,phonebook_bucket_alt=? WHERE _id=?
...
        at com.android.providers.contacts.ContactsDatabaseHelper.updateRawContactDisplayName(ContactsDatabaseHelper.java:5344)
        at com.android.providers.contacts.ContactsDatabaseHelper.upgradeToVersion504(ContactsDatabaseHelper.java:3580)
        at com.android.providers.contacts.ContactsDatabaseHelper.onUpgrade(ContactsDatabaseHelper.java:2261)
...

Clock fails somewhat similarly though it apparently didn't try to upgrade:

Process: com.android.deskclock
Flags: 0xc8be45
Package: com.android.deskclock v203 (2.0.3)
Build: samsung/apexqtmo/apexqtmo:4.1.2/JZO54K/T699UVBMC5:user/release-keys
android.database.sqlite.SQLiteException: no such column: incvol (code 1): , while compiling: SELECT _id, hour, minutes, daysofweek, alarmtime, enabled, vibrate, message, alert, incvol, profile FROM alarms WHERE (enabled=1)
...
        at com.android.deskclock.AlarmProvider.query(AlarmProvider.java:73)
        at android.content.ContentProvider.query(ContentProvider.java:744)
        at android.content.ContentProvider$Transport.query(ContentProvider.java:199)
        at android.content.ContentResolver.query(ContentResolver.java:414)
        at android.content.ContentResolver.query(ContentResolver.java:357)
        at com.android.deskclock.Alarms.getFilteredAlarmsCursor(Alarms.java:165)
        at com.android.deskclock.Alarms.calculateNextAlert(Alarms.java:344)
        at com.android.deskclock.Alarms.setNextAlert(Alarms.java:417)
        at com.android.deskclock.Alarms.saveSnoozeAlert(Alarms.java:510)
        at com.android.deskclock.AlarmInitReceiver$1.run(AlarmInitReceiver.java:49)
...

The media player fails with a NullPointerException:

Process: com.andrew.apollo:main
Flags: 0x98be65
Package: com.andrew.apollo v2 (1.1)
Build: samsung/apexqtmo/apexqtmo:4.1.2/JZO54K/T699UVBMC5:user/release-keys
java.lang.NullPointerException
        at com.andrew.apollo.MusicPlaybackService.stop(MusicPlaybackService.java:878)
        at com.andrew.apollo.MusicPlaybackService.openCurrentAndMaybeNext(MusicPlaybackService.java:1048)
        at com.andrew.apollo.MusicPlaybackService.openCurrentAndNext(MusicPlaybackService.java:1031)
        at com.andrew.apollo.MusicPlaybackService.access$1500(MusicPlaybackService.java:74)
        at com.andrew.apollo.MusicPlaybackService$MusicPlayerHandler.handleMessage(MusicPlaybackService.java:2337)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at android.os.Looper.loop(Looper.java:137)
        at android.os.HandlerThread.run(HandlerThread.java:61)

Fixing the problem Maybe I could have worked out how to upgrade the relevant databases myself, but I went for simpler solutions. The clock settings are easy to re-enter and most of the media player state is regenerated by scanning files on the SD card. So I just deleted those on the Relay:

  rm -rf /data/data/com.android.deskclock
  rm -rf /data/data/com.android.providers.media

The contacts were what I really cared about, and there are actually specific menu items in Contacts to export and import those (using VCF format), side-stepping the database upgrade. So I did:

1. Insert SD card in Blade
2. Open Contacts and export to VCF (I forget where this is in the menus but it was easy to find)
3. Remove broken Contacts database on Relay:
   rm -rf /data/data/com.android.providers.contacts
4. Insert SD card in Relay
5. Move exported contacts to internal storage:
   mv /storage/sdcard1/*.vcf /storage/emulated/legacy
6. Open People and tap the menu key, 'Import/export', 'Import from storage', then the filename

This may not include all data, and in particular it doesn't seem to include attached photos. But that was good enough for me.

This is my monthly summary of my free software related activities. If you re among the people who made a donation to support my work (44.52 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. The Debian Administrator s Handbook Wheezy update completed. Roland and I completed the update of the Debian Administrator s Handbook for Debian 7 Wheezy. We still have some proofreading work to do but you can already enjoy the result here: http://debian-handbook.info/browse/wheezy/ Feel free to report back any problem that you discover. You can also submit us patches ready to apply if you want to go one step further. Publican contributions. The book is generated with publican and I maintain its Debian package. This month I got a release critical bug because it stopped working it turns out that the problem lied in libxml-treebuilder-perl and I thus reassigned #728885 while providing a tentative patch to the upstream author. After a few days without action from the pkg-perl team, and after having received a FTBFS bug on debian-handbook (of course publican was broken in unstable!), I prepared a fixed package myself and I uploaded it (I m still part of the pkg-perl team although I m inactive). Since I used publican heavily this month, I filed two tickets in its bugzilla. I requested a new feature in #1034836 (the possibility to keep around the former string for fuzzy strings to update), and I reported a problem with the handling of \n in PO files in #1036150. Debian France Galette update. I updated the galette package and its paypal plugin, and I deployed those on france.debian.net. It had some fixes for the reminder mails sent to members. Bylaws update. I also resumed my work on preparing new bylaws for Debian France. Sylvestre Ledru came up with a draft (with the help of a lawyer) a few months ago and I m reviewing/improving them now. The main goal is to clarify that Debian France is meant to be a Trusted Organization for the Debian project. Debian France Shop. We had the idea since a few months already but Sylvestre did the leg work to open a Debian France shop with the help of EnVenteLibre. I asked our members to prepare some CSS that better match the Debian colors and this should be fixed in a few days. The first goodies will also start to appear shortly, just in time for Christmas! Misc Debian work Distro Tracker. In the continuation of the Google Summer of Code, I asked the DSA team to setup a new virtual machine to host tracker.debian.org, an instance of Distro Tracker, the rewritten Package Tracking System. They have done their part of the job (except the mail setup), it s now waiting on me to find some time to complete some cleanups and deploy the thing. WordPress. I packaged wordpress 3.7.1 and sent a call for help on debian-mentors. I got 3 replies, I gave them some initial direction but I haven t heard back anything since. WordPress 3.8 is expected in a few days, hopefully one of the new volunteers will take care of preparing the next update. Dpkg regressions. I haven t done anything for multiple months but at least I keep running the git version of dpkg and I detected two regressions. Good to have them squashed before the upcoming 1.17.2 upload to unstable. PTS fix. I fixed some warnings that the PTS code started generating since the upgrade of its host to wheezy. They were generating some annoying backscatter mails to users of the pts@qa.debian.org bot. Ruby security update. I helped the ruby team to prepare the required security updates of ruby1.8 and ruby1.9.1 (see #730178 and #730189). This work was sponsored by Kali/Offensive Security. Smartcard setup. I bought 2 OpenPGP smartcards with a reader and I moved all my private keys on those devices (one card with the master key for signature/certification to be kept at home, one card for daily/mobile usage with the subkeys for encryption/signature/authentication). My laptop s harddrive doesn t contain any private key anymore. I have kept the required offline backup in a safe place, but in the end, my private keys are much harder to steal. I should write down my findings in another article Thanks See you next month for a new summary of my activities.

2 comments Liked this article? Click here. My blog is Flattr-enabled.

Right on the heels of release 0.2.1 of RcppCNPy, a new version 0.2.2 is now on CRAN. RcppCNPy uses the CNPY library by Carl Rogers to provide R with easy read and write access to NumPy files. The reason for the new version that I had experimented with a different way to test endianness (as needed for the NumPy file headers) but accidentally sent an interim tarball to CRAN which still wanted to include endian.h promptly breaking Windows builds. So now we do something even simpler and just rely on the (even more complete) test for endianness when R is built, which prevents all sorts of complications for us and builds everywhere (with thanks to Brian Ripley for the suggestion). While we were at it, we also added a new unit test. Full changes are listed below.

Changes in version 0.2.2 (2013-11-29)

• Switched to using the result from the compile-time configuration for R to determine big or little endian (as needed for the NPy headers)
• Added a new test (and test validation result file) for a complete save-reload cycle and comparison

CRANberries also provides a diffstat report for 0.2.2 relative to 0.2.1. As always, feedback is welcome and the rcpp-devel mailing list off the R-Forge page for Rcpp is the best place to start a discussion.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Shanley wrote an insightful article about microagressions and management [1]. It s interesting to read that and think of past work experiences, even the best managers do it. Bill Stone gave an inspiring TED talk about exploring huge caves, autonamous probes to explore underground lakes (which can be used on Europa) and building a refuelling station on the Moon [2]. Simon Lewis gave an interesting TED talk about consciousness and the technology needed to help him recover from injuries sustained in a serious car crash [3]. Paul Wayper wrote an interesting article about reforming the patent system [4]. He also notes that the patent system is claimed to be protecting the mythical home inventor when it s really about patent trolls (and ex-inventors who work for them). This is similar to the way that ex-musicians work for organisations that promote extreme copyright legislation. Amanda Palmer gave an interesting TED talk about asking for donations/assistance, and the interactions between musicians and the audience [5]. Some part of this are NSFW. Hans Rakers wrote a useful post about how to solve a Dovecot problem with too many files open [6]. His solution was for a Red Hat based system, for Debian you can do the same but by editing /etc/init.d/dovecot. The use of the /proc/N/limits file was interesting, I ve never had a cause to deliberately use that file before. Krebs on Security has an interesting article about Android malware being used to defeat SMS systems to prevent bank fraud [7]. Apparently an infected PC will instruct the user to install an Android app to complete the process. Rick Falkvinge wrote an interesting article about how to apply basic economics terminology to so-called Intellectual Property [8]. Matthew Garrett wrote an interesting post about the way that Ubuntu gets a better result than Debian and Fedora because it has clear fixed goals [9]. He states that many people regard Fedora as a playground to produce a range of niche derivatives , probably a large portion of the Fedora and Debian developers consider this a feature not a bug. Ming Thein wrote an interesting article about the demise of the DSLR [10]. Bruce Schneier wrote an interesting post on the detention of David Miranda by the British authorities [11]. It s mostly speculation as to why they would do such a thing (which seems to go against their own best interests) and whether the NSA even knows which documents Edward Snowden copied. Jaclyn Friedman wrote an interesting article on Mens Rights Movements (MRAs) and how they are bad for MEN as well as for women [12]. Rodney S. Tucker wrote an insightful article for the IEEE about the NBN [13]. Basically the Liberal party are going to spend most of the tax money needed for a full NBN but get a significantly less than the full benefit. Lauren Drell wrote an interesting article for Mashable about TellSpec, a portable spectrometer that communicates with an Android phone to analyse food for allergens [14]. I guess this will stop schools from banning phones. Katie McDonough wrote an interesting article for Salon about the Pope s statements about the problems with unchecked capitalism [15]. His ideas are really nothing new to anyone who has read the Bible and read the news. It seems to me that the most newsworthy part of this is that most Christian leaders don t make similar statements. Daniel Leidert wrote an interesting post about power saving when running Debian on a HP Microserver [16]. Most of it is relevant to other AMD64 hardware too, I ll have to investigate the PCIE ASPM and spin down options on some of my systems that are mostly idle.

• [1] https://medium.com/about-work/65d4740f7a2f
• [2] http://www.ted.com/talks/bill_stone_explores_the_earth_and_space.html
• [3] http://tinyurl.com/6zgcb3f
• [4] http://mabula.net/tbfw/2013/08/01
• [5] http://www.ted.com/talks/amanda_palmer_the_art_of_asking.html
• [6] http://tinyurl.com/k3xz9a8
• [7] http://krebsonsecurity.com/2013/08/a-closer-look-perkele-android-malware-kit/
• [8] http://tinyurl.com/orf9gds
• [9] http://mjg59.dreamwidth.org/27517.html
• [10] http://blog.mingthein.com/2013/08/24/the-demise-of-the-dslr/
• [11] https://www.schneier.com/blog/archives/2013/08/detaining_david.html
• [12] http://prospect.org/article/good-mens-rights-movement-hard-find
• [13] http://tinyurl.com/q49b9fa
• [14] http://mashable.com/2013/11/26/tellspec/
• [15] http://tinyurl.com/nosxkj2
• [16] http://www.wgdd.de/2013/08/hp-n54l-microserver-energy-efficiency.html

Axel just said that PDiffs are really useful for stable because it changes seldom. The truth is that there are no PDiffs for stable. I consider this a bug because you need to download a huge blob on every point release even though only few stanzas changed.

The real problem of PDiffs is the count that needs to be applied. We run "dinstall" much more often now than we used to and we cannot currently skip diffs in the middle when looking at a dak-maintained repository. It would be more useful if you could fetch a single diff from some revision to the current state and apply this instead of those (download, apply, hash check)+ cycles. 56 available index diffs are also not particularly helpful in this regard. They cover 14 days with four dinstalls each, but it takes a very long time to apply them, even on modern machines.

I know that there was ongoing work improving PDiffs that wanted to use git to generate the intermediate diffs. The current algorithm used in dak does not require old versions except one to be kept, but keeping everything is the far other end of the spectrum where you have a git repository that potentially gets huge unless you rewrite it. It should be possible to just implement a rolling ring buffer of Packages files to diff against, reprepro already generates the PDiffs in the right way, the Index format is flexible enough to support this. So someone would just need to make that bit of work in dak.

This is one thing I have wanted to do for a long time: configure Mutt to encrypt all messages sent to addresses for which I have a valid public key. Well, here is an awk-based script to generate that configuration. (Yes, I know, a similar script was already written. But I did not see it at first, and I find mine more readable. :-) )Code

#! /bin/sh
set -e
# Specific hooks set global configuration values for Mutt, which are not
# reset for new messages, so a default hook is needed to set the default
# value (do not try encrypting all messages).
printf "send-hook . unset crypt_autoencrypt\n\n"
cat <<EOF
# This group contains all known PGP users, and can be used to define
# hooks, for instance to encrypt all messages sent to PGP users, or to
# add a specific signature to promote PGP usage to non-PGP users.
EOF
# Generate a machine-readable list of known keys
gpg2 --batch --with-colons --list-keys \
      awk -- '
        BEGIN  
            FS = ":"
            # A flag that indicates whether or not we are in a public
            # key with usable encryption capability
            usable = 0
         
        # Detect beginning of public keys with usable encryption
        # capability
        $1 == "pub" && $12 ~ /E/   usable = 1  
        # Detect beginning of public keys without usable encryption
        # capability
        $1 == "pub" && $12 !~ /E/   usable = 0  
        
        # Only consider user IDs from public keys with usable encryption
        # capability and with regular, marginal, full or ultimate
        # validity (n, m, f or u in the second field)
        usable && $1 == "uid" && $2 ~ /[nmfu]/ && $10 ~ /@/  
            # Remove everything before and after the email address in
            # brackets
            sub(/^[^<]*</, "", $10)
            sub(/>[^>]*$/, "", $10)
            # Finally print the email address
            print $10
         ' \
      sort -u \
       
        # Group addresses by ten to reduce the number of Mutt commands
        finished=0
        while [ $finished -ne 1 ]
        do
            addresses=""
            for i in 0 1 2 3 4 5 6 7 8 9
            do
                if read address
                then
                    addresses="$addresses $address"
                else
                    finished=1
                    break
                fi
            done
            # Add PGP users' addresses to a pgp-users group
            printf "group -group pgp-users -addr %s\n" \
                "$addresses"
        done
     
# Build a hook to try encrypting messages sent to PGP users
printf "\n# Encrypt all messages sent to known PGP users\n"
printf "send-hook \"%%C pgp-users\" set crypt_autoencrypt\n" \
printf "\n# Add a specific signature to promote PGP usage to non-PGP users\n"
printf "#send-hook \"!%%C pgp-users\" set signature=~/.signature.use-pgp\n"

Usage Script usage This script generates uses GnuPG to print a list of all known keys, takes only valid ones, and uses them to build a series of configuration instructions for Mutt, that define hooks that are triggered when you write a message to them. This configuration has to be written to your Mutt configuration file:

$ bin/gen-pgp-hooks.sh >> ~/.muttrc

Or better (far better, in fact), to a dedicated configuration file you will refer to in your main one:

$ bin/gen-pgp-hooks.sh > ~/.muttrc.crypt-recipients
$ echo "source ~/.muttrc.crypt-recipients" >> ~/.muttrc

You may want to adapt this script to it directly writes to that file rather than having to redirect it. Also, remember to run it when you refresh your public keyring or add new keys to it. Mutt usage These hooks will make Mutt try to encrypt all messages you send to at least one known PGP user. That means it will try it even for messages you send to a PGP user and to non-PGP ones. In these cases, it will not find keys for all the recipient, and ask you which key to use; as you have none, you will have to quit that prompt by typing ^G, and manually disable encryption by accessing the PGP options menu with the key p.

Back on Black (Friday) Event Altus Metrum is pleased to announce our Back on Black (Friday) event! For the first time since the Black Forest fire in June, we re re-opening our web store this weekend with a host of new and classic Altus Metrum products, including a special pre-order discount on our latest-and-greatest flight computer design, TeleMega. This weekend only, Friday, 29 November 2013 through Monday, 2 December, 2013, the first 40 TeleMega direct orders placed through our web store will receive a special $50 pre-order discount (regular $400, now only $350!). TeleMega is an advanced flight computer with 9-axis IMU, 6 pyro channels, uBlox Max 7Q GPS and 40mW telemetry system. We designed TeleMega to be the ideal flight computer for sustainers and other complex projects. TeleMega production is currently in process, and we expect to be ready to ship in mid-December. Pre-order now and we won t charge you until we ship. Learn more about TeleMega at: http://altusmetrum.org/TeleMega/ We are also pleased to announce that TeleBT is back in stock. Priced at $150, TeleBT is our latest ground station that connects to your laptop over USB or your Android device over BlueTooth. Learn more about TeleBT at http://altusmetrum.org/TeleBT/ Another new product we re thrilled to announce is EasyMini! Priced at only $80, EasyMini is a two-channel flight computer with built-in data logging and USB data download. Like our more advanced flight computers, EasyMini is loaded with sophisticated electronics and firmware, designed to be very simple to use yet capable enough for high performance airframes. Perfect as a first flight computer, EasyMini is also great as a backup deployment controller in complext projects. Learn more about EasyMini at: http://altusmetrum.org/EasyMini/ Also in stock for immediate shipment is MicroPeak, our 1.9 gram recording altimeter available for $50. The MicroPeak USB adapter, also $50, has been improved to make data downloading a snap. Read more about these at: http://altusmetrum.org/MicroPeak http://altusmetrum.org/MicroPeakUSB You can learn more about these and all our other Altus Metrum products at http://altusmetrum.org. The special discount on TeleMega pre-orders is available only on orders placed directly through Bdale s web store at http://shop.gag.com Thank you all for your support of Altus Metrum during 2013. It s been a rough year, but we re having a great time updating our existing products and designing new stuff! We look forward to returning products like TeleMetrum and TeleMini to the market soon, and plan to introduce even more new products soon.