This is a minor bug fix release of the Perl POD translators for text and man pages. It fixes a warning about use of uninitialized variables when run on a Perl module in the current directory, cleans up a confusing warning during the Perl core build, and fixes a long-standing bug in turning off italic font in =item tags in a C<> block. You can get the latest release form the podlators distribution page.

Conrad announced a new Armadillo release 6.600.4 yesterday. This followed some work we had done for thorough pre-release checking with reverse dependencies tests for the over 200 CRAN packages using it and resulted in one more squashed bug. This releasehas been folded into RcppArmadillo 0.6.600.4.0 which arrived on CRAN and Debian earlier today. Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. Changes in this release are as follows:

Changes in RcppArmadillo version 0.6.600.4.0 (2016-03-15)

• Upgraded to Armadillo 6.600.4 (Catabolic Amalgamator)
  • expanded sum(), mean(), min(), max() to handle cubes
  • expanded Cube class to handle arbitrarily sized empty cubes (eg. 0x5x2)
  • added shift() for circular shifts of elements
  • added sqrtmat() for finding the square root of a matrix
  • fix for gmm_diag when using Mahalanobis distance
• The configure script now reflects the full LAPACK fallback offered by R 3.3.0 or later (PR #81)

Courtesy of CRANberries, there is also a diffstat report for this release. As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Last week was quite equally split between refactoring and bug fixing. As we're getting closer to 4.6.0 release more people give feedback on it and some corner case issues are being discovered. However the biggest challenge was strange bug that was reported long time ago - it affected only Italian translation on Windows server. After lot of debugging, I've realized that trying to encode Cardinalit </td> to json (using json_encode) causes this problem. Obviously there have to be more conditions met as this string only doesn't cause it, however removing it or placing x between and < fixes the problem. In the end I've workarounded it in our code and hopefully I will get to create proper bugreport for php so that it can be fixed upstream as well. The refactoring was mostly focused on encoding conversion part which is now all embedded in the Encoding class. It was quite simplified and should also speed up pages a bit as it does the initialization only when needed not on every phpMyAdmin's page. Handled issues:

• #12075 Info: Blank Login Page and URL-Query-Sorting
• #12084 1064 error when importing a wordpress sql-file
• #11705 Error code: 200 when loading table's structure (Windows 10)
• #12081 Space Usage alternating rows wrong when overhead is present
• #12083 mb_ereg_replace function was removed
• #12040 Update common.css.php
• #12065 INT column with default value and comment exported faulty
• #12077 phpmyadmin export invalid - Missing ENGINE/CHARSET/COLLATE
• #12076 phpmyadmin export invalid - COMMENT AS
• #11805 issue with mysql 5.5 sql_mode=TRADITIONAL with PMA 4.5.3.1
• #12072 database copy and export creating bad SQL
• #3 Typo in readme

Filed under: English phpMyAdmin 0 comments

So, a customer uses a FortiNet VPN gateway. Because I have perfectly fine IPsec software already installed, the only thing missing are appropriate settings. As they use IKEv1 in aggressive mode, there is not much of an error reply if you get any of them wrong. So, here's a StrongSwan setup that works for me:

conn fortinet
        left=%any
        leftauth=psk
        leftid=""
        leftauth2=xauth
        xauth_identity="your username"
        leftsourceip=%config
        right=gateway IP address
        rightsubnet=VPN subnet
        rightauth=psk
        keyexchange=ikev1
        aggressive=yes
        ike=aes128-sha1-modp1536!
        esp=aes128-sha1-modp1536!
        auto=add

Not sure if that can be optimized further by getting the VPN subnet through mode_config as well, but I'm basically happy with the current state. In addition to that, you need the PSK and XAUTH secrets, obviously.

cat hello.c
#include <stdio.h>
int main(void)  
    printf("Hello World!\n");
    return 0;
 

A simple, standard, C program. Compiling and running it shouldn't produce any surprises... except when it does.

./hello
bash: ./hello: No such file or directory

The first time I got that, it was quite confusing. So, strace to the rescue -- I thought. But nope:

strace ./hello
execve("./hello", ["./hello"], [/* 14 vars */]) = -1 ENOENT (No such file or directory)

(output trimmed) No luck there. What's going on? No, it's not some race condition whereby the file is being removed. A simple ls -l will show it's there, and that it's executable. So what? When I first encountered this, I was at a loss. Eventually, after searching for several hours, I figured it out and filed it under "surprising things with an easy fix". And didn't think of it anymore for a while, because once you understand what's going on, it's not that complicated. But I recently realized that it's not that obvious, and I've met several people who were at a loss when encountering this, and who didn't figure it out. So, here goes: If you tell the kernel to run some application (i.e., if you run one of the exec system calls), it will open the binary and try to figure out what kind of application it's dealing with. It may be a script with a shebang line (in which case the kernel calls the appropriate interpreter), or it may be an ELF binary, or whatever. If it is an ELF binary, the kernel checks if the architecture of the binary matches the CPU we're running on. If that's the case, it will just execute the instructions in the binary. If it's not an ELF binary, or if the architecture doesn't match, it will fall back on some other mechanism (e.g., the binfmt_misc subsystem could have some emulator set up to run binaries for the architecture in question, or may have been set up to run java on jar files, etc). Eventually, if all else fails, the kernel will return an error:

./hello: cannot execute binary file: Exec format error

"Exec format error" is the error message for ENOEXEC, the error code which the kernel returns if it determines that it cannot run the given binary. This makes it fairly obvious what's wrong, and why. Now assume the kernel is biarch -- that is, it runs on an architecture which can run binaries for two CPU ISAs; e.g., this may be the case for an x86-64 machine, where the CPU can run binaries for that architecture as well as binaries for the i386 architecture (and its 32-bit decendants) without emulation. If the kernel has the option to run 32-bit x86 binaries enabled at compile time (which most binary distributions do, these days), then running i386 ELF binaries is possible, in theory. As far as the kernel is concerned, at least. So, the kernel maps the i386 binary into memory, and jumps to the binary's entry point. And here is where it gets interesting: When the binary in question uses shared libraries, the kernel not only needs to open this binary itself, but also the runtime dynamic linker (RTDL). It is then the job of the RTDL to map the shared libraries into memory for the process to use, before jumping to its code. But what if the RTDL isn't installed? Well, then the kernel won't find it. The RTDL is just this file, so that means it will produce an ENOENT error code -- the error message for which is "No such file or directory". And there you have it. The solution is simple, and the explanation too; but still, even so, it can be a bit baffling: the system tells you "the file isn't there", without telling you which file it's missing. Since you passed it only one file, it's reasonable for you to believe the missing file is the binary you're asking the system to execute. But usually, that's not the problem: the problem is that the file containing the RTDL is not installed, which is just a result of you not having enabled multiarch. Solution:

dpkg --add-architecture <target architecture>
apt update
apt install libc6:<target architecture>

Obviously you might need to add some more libraries, too, but that's not usually a problem.

I was privileged enough to get access to the alpha preview of Keybase.io. Last year, the project raised 11M$ in venture capital and they seem to be attacking the hard problems, so it is a serious project, worth considering for any cryptography hacker like me. This was obviously discussed elsewhere previously but since Keybase has evolved a bit since then, I figured it would be worth adding my grain of salt.

The elite alpha problem My first problem with Keybase was of course that I wasn't in the elite club of invited people. Somehow, I finally got in but the problem remains for the everyone else. In my case, it was the Keybase filesystem announcement that brought my attention back to the project, although I have yet to test that itself - yet another invite-only problem (need to email chris@keybase.io to get access, I guess). Still, 10GB of storage might be interesting for some people, but considering the number of hurdles or "friction", as they say, that needs to be jumped over to get access, I am not sure how much traction this will get for now. The project has been in alpha for a while now (more than a year) with no announced public beta just yet, which is way slower than similarly challenging projects with less capital. Let's encrypt, as a comparison, was publicly announced in 2014 and by the end of 2015, the public beta was opened.

Private key leakage My second major concern is that Keybase does worrisome things to private key material. If you tell it to import your regular GPG keys, it will copy them into the keybase keyring, which is already a problem: now you have two places where you store sensitive secrets. But worse, Keybase offers you to upload your private key material on the server, without clearly and explicitely exposing the potential risks that means to the user. The key, presumably, is encrypted (at least that is what I can parse from the Keybase privacy policy). Yet this is still a concern: a determined attacker (e.g. the government or another malicious entity) could force Keybase to give up the encrypted private key material and install sniffers that would reveal the passphrase that would allow decryption and impersonation of the related identity. That is a huge deal. This is what caused Hushmail's demise in 2007: they had similar promises about how they had a "secure" email (in this case) service based on OpenPGP, but since they key was stored on the server, they only had to serve a malicious applet to victims and were able to give cleartext data to the US government (12 CDs!) under a Mutual Legal Assistance Treaty with Canada. In Keybase, When I generate a new key, it asks me if I want to upload the key to their servers, and "Yes" is chosen as a default, so it is clearly a policy that is encouraged. This is presumably to enable browser-based crypto, yet this could be done by storing keys within the browser, not on the server. The mere possibility that the client can meddle with secret key material in such a way is a problem. Even if uploading secret keys to the server is optional, the fact that it is a possibility widens the attack surface on the client significantly. GPG has been working on isolating private key operations to a different process (gpg-agent) or smart cards, SSH has privilege seperation. The idea of uploading secret key material online is so beyond current practices that it should bring the project to a full stop already. My private key material is pretty sensitive. I am a Debian developer: if someone gets access to that key, they can upload arbitrary code for thousands of packages, affecting thousands if not millions of users. That is a huge deal, and I am not ready to give that power to an arbitrary third party that I know little about.

Privacy policy issues Furthermore, the Keybase privacy policy explicitely states that "Logs of this information may persist for an indefinite period", which include, and I quote:

• Your computer s IP address
• Your preferences and settings (time zone, language, privacy preferences, application preferences, etc.)
• The URL of the site that referred you to the Service
• The buttons and controls you clicked on (if any) within the Service
• How long you used the Service and which parts and features you used
• Session times and lengths

For a company working to improve user's privacy, this is a pretty bad policy. It exposes Keybase users to undue surveillance, on a lot of private information that is not necessary to operate the service. I haven't actually read the remaining of the policy, as I was already scared as hell and figure I would just log off for now and try not to add any extra personal information in that already growing pile. Let's mention, while I'm here, that the keybase commandline client has this odd way of working that it forks a daemon in the background all the time, even to run just keybase status. One has to wonder if this thing is included in the surveillance data above, which would make the keybase client an amazing surveillance device in itself. To stop all this, you need:

keybase logout
keybase ctl stop

Not quite obvious...

Usability issues Which brings me to a series of usability issues I have found while working with Keybase. Having worked on usability myself in the Monkeysphere project, I understand how hard those problems can be. But right now, the fact is that keybase is only a commandline client, with some web-based sugar sprinkled on top. I recognize the huge efforts that have been done to make the user experience (UX) as easy as possible through the commandline, but this is definitely not "grand public" material yet. Even as a OpenPGP hacker, Keybase can get confusing. Since it has its own separate keyring from GPG, things that used to be obvious before are suddenly new things to learn. For example, I originally generated a key for anarcat@debian.org in keybase, thinking it would be a separate thing from my current GPG identity. Well, that ended up creating a new key in my GPG keyring, duplicating my existing identity, with no way for me to tell this was a Keybase copy. This could have ended up on the keyservers and confuse anyone looking for my PGP key.

Revoking keys So I revoked that key, which, in itself, is not such an obvious process either. I first had to "drop" the key:

keybase pgp drop '010143572d4e438f457a7447c9758804cc7be44c1ee2b7915c3904567d0a3fb5cf590a'

To find that magic number, I had to go on the website and click through the PGP key to end up on the revoke link which told me which magic string to put. The web UI was telling me I could use keybase status to get that information, but I couldn't find that in the output here:

$ keybase status
Username:      anarcat
Logged in:     no
Device:
    name:      angela
    ID:        070f6a13f2a66afbb463f49dadfd4518
    status:    active
Keybase keys:  unlocked
Session:       anarcat [salt only]
KBFS:
    status:    not running
    version:
    log:       /home/anarcat/.cache/keybase/keybase.kbfs.log
Service:
    status:    running
    version:   1.0.14-1
    log:       /home/anarcat/.cache/keybase/keybase.service.log
Platform Information:
    OS:        linux
    Runtime:   go1.5.3
    Arch:      amd64
Client:
    version:   1.0.14-1
Desktop app:
    status:    not running
    version:
    log:       /home/anarcat/.cache/keybase/Keybase.app.log
Config path:   /home/anarcat/.config/keybase/config.json
Default user:  anarcat
Other users:
command-line client:
    keybase status [pid: 14562, version: 1.0.14-1]

So keybase drop "deletes" the key from the keybase client (and presumably the server). One problem here is that we have three different verbs to mean the same thing: in some places, the website says "deleted" (in the web graph), the commandline client uses "drop" and everyone else using PGP in the world uses "revoke". So what is it? Also note that just dropping that key is not enough to eradicate that key: you still need to generate and import a revocation certificate on GPG's side as well, for that key. This makes sense, because you may have imported that key and you may not want to destroy it when you remove it from keybase, but there could be a little better guessing and UX here as well.

Password usage Then another problem is that, when you signup on the commandline, it asks you to choose a passphrase. It's not clear at all from the UI what that passphrase is for. Is it to protect the private key material? To login the website? Both? It turns out it is actually to login to the website and the Keybase API. It could also be the key to the private key material, but I haven't quite figured that out yet. Similarly, the "paper devices" are a little confusing to me as well. The registration process is fairly insistent that I need to write a series of secret words down on a piece of paper and put it my wallet. This is based on the idea that you can use that key to recover your account and setup new devices. However, it is not made clear at all how those credentials should be protected or used. What if I lose my wallet? What do I do then? Is it okay if i write it on the back of my laptop instead? I know it sounds like stupid questions to crypto geek, but it's definitely stuff people will do, and education, at all levels, is necessary here.

The existing web of trust Finally, I found it surprisingly counter-intuitive to sign keys with keybase. I was assuming the whole point of the thing was to expand the web of trust, but it seems their mission lies somewhere else: there is no facility to sign keys directly in keybase. You sign statements on various social sites (Github, Twitter, Reddit...) but doing traditional key exchanges seems to be off limits. My obvious use case was to sign my existing key material with my newly generated anarcat@keybase.io key, so that keybase would become another way of verifying my identity. Well, this is not possible with keybase directly - you still have to go through the regular:

gpg --sign-key DEADBEEF

Which is really unfortunate, because some craaaazy people like me insist on doing in person, actually secure key exchanges sometimes, and the tools to do this right now (mainly gpg, caff and monkeysign) really need a lot of love and help. Keybase could have helped a lot in pushing those initiatives forward and bring new ways of doing secure key exchanges, instead of only lowering the standards on cryptographic identity.

Usual proprietary startup rant I can't help but finish by noticing that the whole basic problem with Keybase is more fundamental than a few usability issues, which can all be fixed fairly easily. Keybase.IO is a web-based service that is entirely proprietary. The client is freely downloadable and free software - but the server side is not. This makes us rely on a central point of failure and the goodwill of those operators to not only keep the service running (for free?) indefinitely, but to protect us against all attackers, state-run or otherwise. This is definitely against basic free software principles and the open architecture of the internet federation, well embodied in the Franklin street statement. Online services should be decentralized and open like the OpenPGP keyservers have always been, period.

Conclusion I am curious to see where Keybase brings us, as a community. So far, I am concerned about centralization and devaluation of the web of trust as cryptographic system. I do not believe that trusting multiple corporate social networking sites bring much benefit to our security, although it does improve usability significantly and is certainly better than TOFU, so that part of the project is definitely interesting, especially since it allows leveraging classical internet protocols like DNS and HTTPS. Key exchange is a critical problem that still hasn't properly been resolved in the cryptographic community. Most efforts at building communication tools (say like Signal or Telegram) mostly ignore the problem and expect people to read up strings of numbers or rely on synchronous communications (see the Axolotl rachet for Signal's take on it). Keybase's attempts at fixing this are great, but needs much more work to actually resolve the PKI issues in a significant way. More fundamentally, the practice of storing keys on the servers should just stop: it is a definite no go for me, and a classic crypto mistake that has bitten way too many people. In my mind, it discredits the whole project. The point of OpenPGP is end to end encryption across devices: storing the keys on a server breaks that apart, and doesn't improve much on the existing HTTPS security systems already in place on the web. It is especially a problem given the new waves of attacks on cryptography from western governments, from the UK to California... Until those issues are resolved, I can't get myself to recommend Keybase to anyone just yet.

Post-scriptum Note to Keybase developers: I have considered filing issues regarding the above, but unfortunately, I was unable to filter through the gigantic github issues list for duplicates, and didn't have time to file detailed reports for everything. I apologize for bypassing those usual conventions. If you want to try out Keybase to make your own opinion, I do have 4 invites I can give away, but I will do so only if you need to test specific issues. Otherwise, I note that there is a Request For Package for the Debian package to become official, that Debian developers (or Keybase developers!) may be interested in completing.

After almost six months of development Weblate 2.5 has been released. It brings lot of improvements and it's quite hard to point few ones. The most important ones include support for Python 3, reports generators, placeables highlighting, extended keyboard shortcuts, configurable dashboard or group based ACLs. Full list of changes for 2.5:

• Fixed automatic translation for project owners.
• Improved performance of commit and push operations.
• New management command to add suggestions from command line.
• Added support for merging comments on file upload.
• Added support for some GNU extensions to C printf format.
• Documentation improvements.
• Added support for generating translator credits.
• Added support for generating contributor stats.
• Site wide search can search only in one language.
• Improve quality checks for Armenian.
• Support for starting translation components without existing translations.
• Support for adding new transations in Qt TS.
• Improved support for translating PHP files.
• Performance improvements for quality checks.
• Fixed sitewide search for failing checks.
• Added option to specify source language.
• Improved support for XLIFF files.
• Extended list of options for import_project.
• Improved targeting for whiteboard messages.
• Support for automatic translation across projects.
• Optimized fulltext search index.
• Added management command for auto translation.
• Added placeables highlighting.
• Added keyboard shortcuts for placeables, checks and machine translations.
• Improved translation locking.
• Added quality check for AngularJS interpolation.
• Added extensive group based ACLs.
• Clarified terminology on strings needing review (formerly fuzzy).
• Clarified terminology on strings needing action and not translated strings.
• Support for Python 3.
• Dropped support for Django 1.7.
• Dropped dependency on msginit for creating new Gettext po files.
• Added configurable dashboard views.
• Improved notifications on parse erorrs.
• Added option to import components with duplicate name to import_project.
• Improved support for translating PHP files
• Added XLIFF export for dictionary.
• Added XLIFF and Gettext PO export for all translations.
• Documentation improvements.
• Added support for configurable automatic group assignments.
• Improved adding of new translations.

If you are upgrading from older version, please follow our upgrading instructions. You can find more information about Weblate on https://weblate.org, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Weblate is also being used https://hosted.weblate.org/ as official translating service for phpMyAdmin, OsmAnd, Aptoide, FreedomBox, Weblate itself and many other projects. Should you be looking for hosting of translations for your project, I'm happy to host them for you or help with setting it up on your infrastructure. Further development of Weblate would not be possible without people providing donations, thanks to everybody who have helped so far! The roadmap for next release is just being prepared, you can influence this by expressing support for individual issues either by comments or by providing bounty for them.

Filed under: English Weblate 0 comments

The last week was mostly spent on bugfixing and cleanup after security releases. Hopefully the amount of security reports will go down now. Most of the bug fixes were in the SQL parser which influences quite a lot of parts of phpMyAdmin. It is responsible for splitting queries on import, generating queries for export or linting the queries as users type them. Additionally Debian packages were also updated, for both unstable and testing and for Ubuntu PPA. Handled issues:

• #12067 Adding "JSON" option to dropdown
• #12047 Filtering databases on databases listing
• #12052 Filtering databases on databases listing, Issue #12047
• #12063 duplicate a table occur an error,for uft-8?
• #12064 Cross-site scripting (XSS) vulnerability in phpMyAdmin Version 4.5.4.1
• #11776 SQL Linter Problems
• #12025 Import reports false SQL error with MariaDB
• #12045 unrecognized keyword left in where clause #11975 (REMAINS UNSOLVED)
• #12041 Missing indexes and constratins in export
• #12028 "ALL" keyword not recognized
• #12054 MySQLDump .sql import in v4.5.5 fails becuase of escaped characters
• #12053 upload-release error
• #12055 Parse eror with 4.6.0-rc1 and master
• #12056 Invalid data stored in $_SESSION[' PMA_token '] if openssl_random_pseudo_bytes() fails
• #12015 create-release errors
• #12048 SQL parser doesn't honor vendor config
• #12024 Better icon for table hiding
• #12032 Icons added and code changed to display icons #12024
• #12037 Fix parse git data without gz support (bug 12030)
• #12030 Do not try to parse git data without gz support
• #12044 Fix example in test/README.rst
• #10 1.1 XSS in Static analysis of SQL query [PMASA-2016-10]
• #11 1.2 XSS in "Edit inline" of SQL query [PMASA-2016-11]
• #16 1.6 XSS Via HOST header [PMASA-2016-11]
• #17 1.7 XSS in file_echo.php by mime sniffing text/plain (only in old IE <= 8 & old Safari on windows) [PMASA-2016-11]
• #19 2. insecure CURL SSL Settings [PMASA-2016-13]
• #37 Fix DROP VIEW statement is not constructed properly by the parser, Issue #36
• #39 Fails to parser CREATE TABLE
• #36 DROP VIEW statement is not constructed properly by the parser
• #38 Recognize ALL when used with WHERE clause

Filed under: Debian English phpMyAdmin 0 comments

As most distributions have switched to systemd as default "init", some people have asked why we actually keep sysvinit around, as it's old and crusty, and systemd can do so much more. My answer is: systemd and sysvinit solve entirely different problems, and neither can ever replace the other fully. The big argument in favour of systemd is integration: when you keep everything in one process, it is easy to make sure that components talk to each other. That is nice, but kind of misses the point: integration could be done via IPC as well, and systemd uses a lot of it to separate privileges anyway. The tighter integration comes from something else: the configuration format. Where sysvinit uses a lot of shell scripts ("imperative" format), systemd uses explicit keywords ("descriptive" format), and this is the important design decision here: descriptive formats cannot do anything unanticipated, neither good nor bad. This is important for UI programmers: if there is a button that says "if I close the lid switch, suspend the laptop", then this needs to be connected to functionality that behaves this way, and this connection needs to work in both directions, so the user will find the active setting there when opening the dialog. So, we need to limit ourselves to a closed set of choices, so the UI people can prepare translations. This is the design tradeoff systemd makes: better integration in exchange for less flexibility. Of course, it is simple to allow "custom" actions everywhere, but that starts to break the "well-integrated" behaviour. There is an old blog post from Joey Hess about how to implement a simple alarm clock, using systemd, and this shows both how powerful and how limited the system is. On one hand, the single line WakeSystem=true takes care of the entire dependency chain attached to making sure the system is actually on -- when the system is going to suspend mode, some program needs to determine that there are scheduled jobs that should wake up the system, determined which of these is the next one, program the wakeup time into the hardware and only then allow the suspend to continue. On the other hand, the descriptive framework already breaks down in this simple example, because the default behaviour of the system is to go to sleep if the lid switch is closed, and the easiest way to fix this is to tell systemd to ignore the lid switch while the program is running. The IPC permissions disallow scheduled user jobs from setting this, so a "custom" action is taken that sets up the "ignore the lid switch" policy, changes to an unprivileged user, and runs the actual job. So, it is possible to get back flexibility, but this is traded for integration: while the job is running, any functionality that the "suspend laptop when the lid is closed" button had, is broken. For something simple as a switch on a personal laptop that breaks if the user configures something that is non-standard, that is acceptable. However, the keywords in the service unit files, the timer files, etc. are also part of a descriptive framework, and the services using them expect that the things they promise are kept by the framework, so there is a limit to how many exceptions you can grant. SystemV init really comes from the other direction here. Nothing is promised to init scripts but a vague notion that they will be run with sufficient privileges. There is no dependency tracking that ensures that certain services will always start before others, just priorities. You can add a dependency system like insserv if you like and make sure that all the requirements for it to work (i.e. services declaring their dependencies) are given, and there is a safe fallback of not parallelizing so we can always play it safe. Because there is so little promised to scripts, writing them becomes a bit tedious -- all of them have a large case statement where they parse the command line, all of them need to reimplement dropping privileges and finding the PID of a running process. There are lots of helpers for the more common things, like the LSB standard functions for colorful and matching console output, and Debian's start-stop-daemon for privilege and PID file handling, but in the end it remains each script's responsibility. Systemd will never be able to provide the same flexibility to admins while at the same time keeping the promises made in the descriptive language, and sysvinit will never reach the same amount of easy integration. I think it is fairly safe to predict both of these things, and I'd even go a step further: if any one of them tried, I'd ask the project managers what they were thinking. The only way to keep systems as complex as these running is to limit the scope of the project. With systemd, the complexity is kept in the internals, and it is important for manageability that all possible interactions are well-defined. That is a huge issue when writing adventure games, where you need to define interaction rules for each player action in combination with each object, and each pair of objects that can be combined with the "Use" action. Where adventure games would default to "This doesn't seem to work." or "I cannot use this.", this would not be a helpful error message when we're trying to boot, so we really need to cut down on actions and object types here. Sysvinit, on the other hand, is both more extreme in limiting the scope to very few actions (restart, wait, once) and only one object type (process that can be started and may occasionally terminate) in the main program, and a lot more open in what scripts outside the main program are allowed to do. The prime example for this is that the majority of configuration takes place outside the inittab even -- a script is responsible for the interface that services are registered by dropping init scripts into specific directories, and scripts can also create additional interfaces, but none of this is intrinsic to the init system itself. Which system is the right choice for your system is just that: your choice. You would choose a starting point, and customize what is necessary for your computer to do what you want it to do. Here's the thing: most users will be entirely happy with fully uncustomized systemd. It will suspend your laptop if you close the lid, and even give your download manager veto power. I fully support Debian's decision to use systemd as the default init for new installs, because it makes sense to use the default that is good for the vast majority of users. However, my opening statement still stands: systemd can never support all use cases, because it is constrained by its design. Trying to change that would turn it into an unmaintainable mess, and the authors are right in rejecting this. There are use cases that define the project, and use cases that are out of scope. This is what we need sysvinit for: the bits that the systemd project managers rightfully reject, but that are someone's use case. Telling these users to get lost would be extremely bad style for the "universal operating system." For example, if someone needs a service that asks a database server for a list of virtual machines to start, runs each in its private network namespace that is named after replacing part of the virtual machine name with the last two octets of the host machine's IP address, then binds a virtual Ethernet device into the network namespace and sets up a NAT rule that allows devices to talk to the public Internet and a firewall rule to stop VMs from talking to each other. Such a beast would live outside of systemd's world view. You can easily start it, but systemd would not know how to monitor it (as long as there is some process still running, is that a good sign), not know how to shut down one instance, not know how to map processes to instances and so on. SystemV init has the advantage here that none of these things are defined by the system, and as long as my init script has some code to handle "status", I can easily check up on my processes with the same interface I'd use for everything else. This requires me to have more knowledge of how the init system works, that is true, however I'd also venture that the learning curve for sysvinit is shallower. If you know shell scripting, you can understand what init scripts do, without memorizing special rules that govern how certain keywords interact, and keeping track of changes to these rules as the feature set is expanded. That said, I'm looking at becoming the new maintainer of the sysvinit package so we can continue to ship it and give our users a choice, but I'd definitely need help, as I only have a few systems left that won't run properly with systemd because of its nasty habit to keep lots of code in memory where sysvinit would unload it after use (by letting the shell process end), and these don't make good test systems. The overall plan is to set up a CI system that

• tests sysvinit -> systemd transition
• tests systemd -> sysvinit transition
• tests debootstrap with systemd
• tests debootstrap with sysvinit
• tests whether service start works for a list of services, so we can catch regressions
• scans packages that have systemd unit files for init scripts, and vice versa

Ideally, I'd like to reach a state where we can actually ensure that the default configuration is somewhat similar regardless of which init system we use, and that it is possible to install with either (this is a nasty regression in jessie: bootstrapping a foreign-architecture system broke). This will take time, and step one is probably looking at bugs first -- I'm grateful for any help I can get here, primarily I'd like to hear about regressions via bug reports. tl;dr: as a distribution, we need both systemd and sysvinit. If you suggest that systemd can replace sysvinit, I will probably understand that you think that the systemd maintainers don't know project management. I'm reluctantly looking at becoming the official maintainer because I think it is important to keep around.

Review: The Lost Steersman, by Rosemary Kirstein

Series:	Steerswomen #3
Publisher:	Rosemary Kirstein
Copyright:	2003, 2014
Printing:	2014
ISBN:	0-9913546-2-1
Format:	Kindle
Pages:	432

This is the third book in the Steerswomen series and a direct follow-up to the events of The Outskirter's Secret. It does, marvel of marvels, feature an in-character summary of the events of the series to date! I do love when authors do this; it helps immensely if you come back to a series after a bit of a break between books. But this whole series is so good, and the emotional tone and development of Rowan as a character is so strong, that I recommend against starting here. After the events of the last book, Rowan has returned to the Inner Lands. She's sent her report back to the Archives, but stopped at the Annex in Alemeth. This is an auxiliary library that should have copies of the journals and other research that Rowan wants to search, and stopping there saves substantial travel time. However, she finds the steerswoman who was custodian of the Annex is deceased and the Annex is, from Rowan's perspective, a mess. Nothing is organized, the books aren't properly cared-for, and Mira's interactions with the townsfolk were far different than Rowan's natural attitude. The start of this book was a surprising shift. After the large-scale revelations at the end of The Outskirter's Secret, and the sense of escalating danger, Rowan's return to small-town life in the Inner Lands comes as a shock. That's true for both the reader and for Rowan, and the parallels make it a remarkably effective bit of writing. At the beginning of the story, the reader is already familiar with Rowan (at least if you've read the previous books) and how she thinks of being a steerswoman. Rowan is very much on edge and in a hurry given what's going on in the broader world. But the town is used to Mira: a gregarious socializer who cared far more about town gossip and her role as coordinator of it than she cared about most of her steerswoman duties. (At least as seen from Rowan's perspective. By the end of the book, we have a few hints that something else might be going on, but the damage to the books at least feels unforgivable.) Rowan is resented and even disliked at first, particularly by Steffie and Gwen who did most of the chores at the Annex and were closest to Mira. One of the reasons why I love this series so much is that Kirstein has a gift for characterization. Rowan (and Bel, who largely doesn't appear in this book) are brilliant characters, but it's not just them. At the start of this book, the reader tends to share Rowan's opinion of the town: a sort of half-bemused, half-exasperated indifference. Even as the characters start to grow on one, it seems like a backwater and a diversion from the larger story. But it becomes clear that Rowan is very on-edge from her experience in the Outskirts, that she's underestimated the relevance of Mira to at least the town's happiness, and she's greatly underestimated the ability of the townsfolk to help her. Steffie, in particular, is a wonderful character; by the end of the book, he had become one of my favorite people in the series so far. He doesn't think he's particularly smart, and his life before Rowan is very simple, but there are depths to him that no one, including him, expected. There is a plot here, apart from small town politics and Rowan's slow relaxation. (Although those were so compelling that I'm not sure I would have minded if that were the entire book.) The lost steersman of the title is an old student friend of Rowan that she unexpectedly meets in town, a former steersman who quit the order and refused to explain why. Rowan, of course, cannot resist trying to fix this situation. The second plot driver is a dangerous invasion of Outskirts monsters into the town. Those who have read the previous books will have some immediate guesses as to why this might be, and Rowan does as well. But there's more going on than it might first appear. This book is not entirely a diversion. It returns to the main plot of the series by the end of the book, and we learn much more about Rowan's world. But, somewhat surprisingly, that was my least favorite part of the book. It has some nice bits of exploration and puzzle-solving, and Rowan is always a delight to spend time with. But the last section of the book is similar to many other genre books I've read before well-written, to be sure, but not as unique. It also features a rather long section following a character who is severely physically ill, which is something I always find very hard to read (a personal quirk). But there's a lot of meat here for the broader plot, and I have no idea what will happen in the next book. The part of The Lost Steersman that I'm going to remember, though, are the town bits, up through the arrival of Zenna (another delightful character who adds even more variety to Kirstein's presentation of steerswomen). Kirstein is remarkably good at mixing small-town characters with the scientific investigation of the steerswomen and letting them bounce off of each other to reveal more about the character of both. If it weren't for the end of the book, which bothered me for partly idiosyncratic reasons, I think this would have been my favorite book of the series. Followed by The Language of Power, and be warned that this book ends on something close to a cliffhanger. Rating: 8 out of 10

This year s Debian Ruby team sprint started today here at Curitiba. Everyone arrived fine, and we started working at the meeting room we have booked for the week at Curitiba campus of the Federal Technical University of Paran . The room is at the Department of Business and Community Relations , what makes a lot of sense! :-) The day started with a quick setup, with a simple 8-port switch, and a couple of power strips. It tooks us a few minutes to figure what was blocked or not on the corporate network, and almost everyone who needs connections that are usually blocked in such environments already had their VPN setups so we were able to get started right after that. We are taking notes live on mozilla s piblic etherpad site Today we accomplished quite a lot:

• analyzed the pending issues for the ruby2.3 transition issues, categorizing the missing packages into needs a binNMU now , needs a binNMU after switching the default to ruby2.3 , and broken .
• ruby-defaults uploaded to experimental switching to ruby2.3 as default, and dropping support for ruby2.2
• ruby-gsl fixed to build against GSL 2.x (was blocking ruby2.3 transition)
• #816253: ruby-fast-gettext: fix FTBFS issue and import a new upstream
• ruby-aws: several issues fixed
• ruby-binding-of-caller: fixed rubygems-integration
• fixed ruby-kakasi-ffi (again)
• made ruby-blockenspiel arch:any again
• ruby-fast-gettext 1.0.0-1 (fix ftbfs 816253)
• ruby-aws-sdk: new upstream, debcheck fix and several bumps
• ruby-fcgi dropped transitional packages + refreshed packaging with -w
• ruby-sshkit 1.8.0
• ruby-beautify: new upstream and few minor fixes
• asciidoctor (new version sponsored)
• capistrano 3.4.0 (ftbfs #795724, #802734)
• #816254: ruby-packetfu: FTBFS: expected NameError with uninitialized constant PacketFu::FakePacket , got #
• updated rake to 10.5.0; making it not include -I/usr/lib/ruby/vendor_ruby when running tests
• triaged an closed open bugs on ruby-httpclient that do not apply anymore.
• updated ruby-httpclient to get rid of warnings in apt-listbugs under Ruby 2.3
• ruby-packetfu (fix ftbfs #816254) by @kanashiro
• investigated extension/rdepends build failing with ruby2.3-only
• applied upstream patch to ruby to fix extension ftbfs when extension uses c++
• ruby-albino 1.3.3-4 (#813644)
• basic user-level testing using ruby2.3 as default:
  • chef WORKS mostly; it seems ohai segfaults some times
  • rails autopkgtest FIXED Could not find gem binding_of_caller (>= 0.7.2) , which is required by gem web-console (~> 2.0) , in any of the sources.
  • vagrant WORKS
  • redmine autopkgtest WORKS
  • apt-listbugs FIXED warnings (e.g. try apt-listbugs list $pkg ); caused by ruby-httpclient
  • nanoc WORKS
• capistrano 3.4.0-1 (#795724, #802734) by @sbadia + @terceiro

This article starts a series of blogs on my ventures into Android land after having used iOS devices for 6 years. I have never been a strong believer in the Apple ecosystem, and my trials to convince myself to MacBooks and MacPros always ended with installing Linux on them, so I guess I wasn t really the ideal target for iOS. Thus, my iOS devices were also permanently in jailbroken state, otherwise I would feel amputated. But as much as I disliked the lock-in and closed environment of the iOS world, it was from the user perspective surprisingly well done and smooth. So it was with a certain level of tension when I finally switched to Linux^WAndroid. If you don t want to read on, here the preliminary conclusion: Why didn t I do it earlier! But before we go into details, let me start with my background: History of my devices Originally I was a big opponent of smart phones and preferred the Unix-way, one device for one thing. So I had a normal phone and (various) Palm devices (Tungsten X, Tungsten C, and above all my beloved Handera TRGpro). I loved the Palm world and considered it superior to the then smart phone world, until I came to Japan, where the challenge of a proper input method for Japanese and proper Japanese support posed a big hurdle. The Palm devices had a stick and written input fields, but Japanese input was practically impossible and a huge pain. Searching for a word in Japanese was more hurdle then looking it up in a printed dictionary. In addition, I needed a phone in Japan, so I plunged into the smart phone world and got myself a iPhone (3g). What a world did open for me: easy typing of Japanese, dictionaries, on-the-fly translation, woooow! And above all, I discovered my most beloved and till now one of my most important programs: Flashcards Deluxe. Thanks, and I have to say to 80% thanks to this program my Japanese learning speed has accelerated considerably. There is nothing more important for me than getting drilled in a systematic way. But I derail, anyway, having Flashcard Deluxe on the iPhone within rather short time I had about 10000 or more flashcards created, and moving on to a different architecture (Android) was for quite some time practically impossible without loosing years of statistics and learning, so I renewed my contract after 2 years together with a iPhone 4s. Another two years passed, and these years brought an Android version of Flashcards Deluxe, as well as Dropbox syncing, so I had no excuse anymore to remain in iOS land, wouldn t it be for an iPhone 5s that was passed to me near the end of my forth year, so I again extended the contract for two years. Finally, after 6 years of iPhone devices, this January I finally decided it is time to switch to Android. After lots of thinking, comparing, and requesting advice from good friends with more experience in the smart phone market I went for a Google Nexus 6p. Google Nexus 6p I will not repeat the specs of this phone as they are widely available on the net. My original plan was a Samsung S6, but after consultation with an expert I decided for an original Google phone for better security support. That left me with the option between a Nexus 5x and 6p, and due to prize differences (prizes of mobiles are ridiculously strange in Japan) I went for the 6p instead of the 5x. One point that made the decision for this slight too big device easy was the fact that it uses a great AMOLED display. Moving the data Since I was using Google Calendar and Google Contacts already on the iPhone, moving to the Android phone was far less a hassle than I thought. My contacts and events showed up without a hiccup. Most of the usual apps are nowadays available on both iOS and Android, so the most difficult thing was remembering all the passwords to log into the applications again (SNS like G+, FB, Twitter etc). The same is more or less true for messengers of all kinds (Line, WhatsApp, Threema, etc), but here one is advised to check with the respective web site first to make sure one does not loose all of the important data. Line for example is a stupid ***** that deletes all previous chats on the old phone and does not make them available on the new one. WhatsApp can be converted with a special conversion program. Threema, too allows for transfer of ids. Move of applications After that came the hunt for replacement applications for those that are not available as is on Android: Mail At first like probably everyone I used the shipped GMail program. It might be good for Google Mail accounts, but for anything else it is just a real pain. Thus, I have searched a bit and finally settled (for now) for K-9 Mail: it is open source, open development, feature rich, and more a hackers type email program, perfectly suited to me. There is a commercial variant called K@-Mail that says that it improves the user interface and some usability items as well as features, but I didn t see much of an advantage over the original version (which is completely free) and in fact some of my accounts didn t work at all. So I remain with K9 Mail and I think this is a good decision. Calendar Managing Calendars is one of the most important task for me. I have been a fervent supporter of DateBk4, DateBk 5, and DateBk 6 on the original Palm series, and when I left the Palm World it was with great pain that I had to loose DateBk. Not only because it was a simply fantastic calendar program that allowed me to keep track of all my climbing routes, festivities, in a much more advanced way than any other Calendering application, but also because the programmer of the DateBk series is running the Dewar Wildlife Trust, a Gorilla rescue group and a lot of the money he makes from the app sales is going to rescue Gorillas. With the switch to iOS this option was gone, and I first used the built-in calendar application (which is so weak) and later and for long time Pocket Informant Pro. This is a very good program and probably the only one that can compete with DateBk with respect to functionality and usefulness. During the time of me being locked in in iOS I realized that the world has moved on and a new version of DateBk series for Java was developed, called Pimlical. First only available on Windows, it became later available also on Android and Linux, too. The following screen shot puts Pimlical on the left, and Pocket Informant on the right. I will write a more detailed comparison in future, in short: PInformant is more streamlined and polished, Pimlical has more configuration option. Practically everything can be adjusted to one s need, and in addition there is also a Desktop application that sync either with Google and the phone, or only with the phone if you want to live off the grid. So nowadays on Android I have both Pocket Informant Pro as well as Pimlical, but after a short time I have now switched practically exclusively to Pimlical. Notes Here there is pain HUGE PAIN!!! iOS has an excellent applications for notes, called simply Notebooks. This little pearl was my work horse for everything (more or less) memorable. From poems and song texts to bus time tables, from PDF to GIFs, from MarkDown to HTML, everything could be saved into Notebooks, displayed, edited, ordered. And above all it had automatic background sync with Dropbox. So I could drop new files into the respective sub-folder of my Dropbox folder and could be sure I have the files available on my phone when I leave for a trip. And there is a huge bag of features that I haven t even tapped into! Android is unfortunately not on the list of supported architectures of Notebooks. So I searched far and wide, and without any success. There are all kind of notes, flash colors, overly simple, fast and slow, stylish and plain, but none of them did even provide half of the features of Notebooks. None, not even half. I still hope I might find the ultimate notes application, or even better would be an Android version of the original Notebooks application (but this is not high on the developers todo list), but for now I am in despair The Rest As I said, most apps are nowadays available on both platforms, so there is not much more to do than download the respective Android app and log in again. That worked very nice across practically all apps. Things I don t like (i.e., which are broken!) on Android Although a very convenient system and perfectly made to fit my taste, there are some things that are a huge pain (and a big shame on Google to not being able to fix that for long time!): Japanese fonts when the device is in English interface language In case you are a foreigner living in Japan and want your Android phone in English, but still read emails, news, etc in Japanese, then Android provides you with the worst, namely Chinese fonts: This is a well known problem and I have blogged about fixing the very same problem on Linux (Debian), and the solution is a simple reshuffling in the fontconfig configuration files. There is even an application for it in the Google store, Kanji Fix, but it needs a rooted device (which I haven t done till now my failure!). I can only hope that Google fixes this completely stupid problem in a future version. The Me problem Another of these beasty problems: The Android Contacts application has an entry for Me , which unfortunately, no idea why, cannot be linked with my normal me in the list of contacts. There are reports all over the Internet, strange suggestions, and no real solution. Again, a simple thing that should work but doesn t. Invisible Images folder in MTP mode A more annoying problem is that the camera folder under Photos does not show up when connecting the device in MTP mode to my computer, and as consequence me being unable to copy photos from the device to my computer. The solution I am using at the moment is moving the photos with a file manager to a new folder which is visible during MTP communcation, and copy the photos from there. But this, too, should be something trivial, but alas, despite a lot of posts on the internet I couldn t find a proper solution. Google Music As written somewhere else, Google Music has switched from 5 star system to up/down system, which is a huge pain and PITA. Things I do like (or I discovered) on Android There are some things I haven t been used/tried on iOS they might be possible which I really like: Yubikey Neo support I will write about this in a different blog, but nowadays I have my GPG keys on an hardware token (Yubikey Neo) and the application OpenKeychain on Android works nicely with both K9 Mail and via NFC with my Yubikey. That is a great tool! Bluetooth streaming Bluetooth on iOS devices was always a bit broken for me, so connecting my phone to my old car radio I needed radio transmitter that was connected to the cable port of the iPhone. With Android I use a Bluetooth Radio device (receives data via bluetooth, and sends music out via radio waves for a car stereo to receive them). Now if my monthly data limit wouldn t be that low Debian on Android Yes, you can have a full Debian system running in your terminal on Android. There are several applications providing this feature, and I am rather surprised how smooth it works. Conclusion My preliminary conclusion is that the switch to Android at this time was perfectly timed, and from the technological side I should have done much earlier. In future blogs I will discuss particular instances of this transition in more details. If you have any suggestion for me, in particular for a good notes taking application, please let me know!

As the flow of incoming bugs for upcoming 4.6.0 has slowed down a bit it was more time for code cleanups and related tasks. But it's also time where potential Google Summer of Code students come to our organization and want to get involved. On the cleanup side the biggest was change to remove embedded PHP libraries which are available on Packagist from our Git and use Composer to manage the dependencies. This change will happen in 4.7.0, so it's still some time ahead, but it's already in our master branch. There still some third party libraries which we use and can not be installed using Composer, so we keep these for now. Besides the usual bug fixing stuff, I've noticed that we lack issues which can be easily understood and fixed by potential Google Summer of Code. We require them to get involved before the program starts, so that we can see they are capable of useful contributions and also to see how they behave if asked for patch improvements. To fix this deficit we're prepared few small cleanup or refactoring tasks, where the students can show their skills. All handled issues:

• #11980 Fix #11979 DECLARE not accepted as valid SQL
• #11609 Errors when Analyzer fails
• #11974 debug event in phpmyadmin for day and month (error)
• #11975 unrecognized keyword left in where clause
• #11973 Tracker won't track some queries executed from SQL tab
• #11907 Update query shows twice
• #11850 CSV import to new table broken
• #11958 Bug Fix #11907
• #11470 Impossible to cancel "Drop files here" overlay
• #6146 Syslog authentication requests / brute-force protection
• #6261 All-Languages version via composer/packagist.org
• #11947 Notice in sql.lib
• #11669 Can't edit rows if table name has one upper case char (OS X)
• #11968 Syntax-errored ALTER TABLE is exported
• #11967 Warning on loading corrupted XML
• #11959 Uncaught ReferenceError: isStorageSupported is not defined
• #22 Make INI settings configurable via environment variables

Filed under: English phpMyAdmin 2 comments

As you may be aware I have a bit of a problem with Single Board Computers in that I have a lot of them. Keeping them organised has turned into a bit of a problem.

I designed clip cases for many of these systems giving me a higher storage density on my rack shelves and built a power supply to reduce the cabling complexity. These helped but I still ended up with a cluttered shelf full of SBC.

I decided I would make a rack enclosure to hold the SBC, I was limited to material I could easily CNC machine which limited me to acrylic plastics or wood.

Initially I started with the idea of housing the individual boards in a toast rack arrangement. This would mean that the enclosure would have to be at least 2U high to fit the boards all the existing cases would have to be discarded. This approach was dropped when the drawbacks of having no flexibility and only being able to fit the units that were selected at design time became apparent (connector cutouts and mounting hole placement.

Instead I changed course to try and incorporate the existing cases which already solved the differing connector and mounting placement problem and gave me a uniform size to consider. Once I had this approach the design came pretty quickly. I used a tube girder construction 1U in height to get as much strength as possible from the 3mm acrylic plastic I would use.

The design was simply laser cut from sheet stock and fastened together with M3 nut and bolts. Once I corrected the initial design errors (I managed to get almost every important dimension wrong on the first attempt) the result was a success.

The prototype is a variety of colours because makespace ran out of suitably sized clear acrylic stock but the colouring has no effect on the result other than aesthetical. The structure gives a great deal of rigidity and there is no sagging or warping, indeed testing on the prototype got to almost 50Kg loading without a failure (one end clamped and the other end loaded at 350mm distance)

I added some simple rotating latches at the front which keep the modules held in place and allow units to be removed quickly if necessary.

Overall this project was successful and I can now pack five SBC per U neatly. It does limit me to using systems cased in my "slimline" designs (68x30x97mm) which currently means the Raspberry Pi B+ style and the Orange Pi PC.

Once small drawback is access to I/O and power connectors. These need to be right angled and must be unplugged before unit removal which can be a little fiddly. Perhaps a toast rack design of cases would have given easier connector access but I am happy with this trade off of space for density.

As usual the design files are freely available, perhaps they could be useful as a basis for other laser cut rack enclosure designs.

Debian Long Term Support (LTS) This is my third month working on Debian LTS, started by Raphael Hertzog at Freexian. This month was my first month working on the frontdesk duty and did a good bunch of triage. I also performed one upload and reviewed a few security issues.

Frontdesk duties I spent some time trying to get familiar with the frontdesk duty. I still need to document a bit of what I learned, which did involve asking around for parts of the process. The following issues were triaged:

• roundcube in squeeze was happily not vulnerable to CVE-2015-8794 and CVE-2015-8793, as the code affected was not present. roundcube is also not shipped with jessie but the backport is vulnerable
• the php-openid vulnerability was actually just a code sample, a bug report comment clarified all of CVE-2016-2049
• ffmpeg issues were closed, as it is not supported in squeeze
• libxml2 was marked as needing work (CVE-2016-2073)
• asterisk was triaged for all distros before i found out it is also unsupported in squeeze (CVEs coming up, AST-2016-001, AST-2016-001, AST-2016-001)
• libebml and libmatroska were marked as unsupported, although an upload of debian-security-support will be necessary to complete that work (bug #814557 filed)

Uploads and reviews I only ended up doing one upload, of the chrony package (CVE-2016-1567), thanks to the maintainer which provided the patch. I tried my best trying to sort through the issues with tiff (CVE-2015-8668 and CVE-2015-7554), which didn't have obvious fixes available. OpenSUSE seem to have patches, but it is really hard to find them through their issue trackers, which were timing out on me. Hopefully someone else can pick that one up. I also tried and failed to reproduce the cpio issue (CVE-2016-2037), which, at the time, didn't have a patch for a fix. This ended up being solved and Santiago took up the upload. I finally spent some time trying to untangle the mess that is libraw, or more precisely, all the packages that embed dcraw code instead of linking against libraw. Now I really feel even more strongly for the Debian policy section 4.13 which states that Debian packages should not ship with copies of other packages code. It made it really hard to figure out which packages were vulnerable, especially because it was hard to figure out which versions of libraw/dcraw were actually vulnerable to the bug, but also just plain figure out which packages were copying code from libraw. I wish I had found out about secure-testing/data/embedded-code-copies earlier... Still, it was interesting to get familiar with codesearch.debian.net to try to find copies of the vulnerable code, which was not working so well. Kudos to darktable 2.0 for getting rid of their embedded copy of libraw, by the way - it made it completely not vulnerable to the issue, the versions in stretch and sid not having the code at all and older versions having non-vulnerable copies of the code.

Issues with VMs again I still had problems running a squeeze VM - not wanting to use virtualbox because of the overhead, I got lost for a bit trying to use libvirt and KVM. A bunch of issues crept up: using virt-manager would just fail on startup with an error saying interface mtu value is improper, which is a very unhelpful error message (what is a proper value??) - and, for the record, the MTU on eth0 and wlan0 is the fairly standard 1500, while lo is at 65536 bytes, nothing unusual there as far as I know. Then the next problem was actually running a VM - I still somewhat expected to be able to boot off a chroot, something I should definitely forget about it seems like (boot loader missing? not sure). I ended up calling virt-install with the live ISO image I was previously using:

virt-install --virt-type kvm --name squeeze-amd64 --memory 512 --cdrom ~/iso/Debian/cdimage.debian.org_mirror_cdimage_archive_6.0.10_live_amd64_iso_hybrid_debian_live_6.0.10_amd64_gnome_desktop.iso --disk size=4 --os-variant debiansqueeze

At least now I have an installed squeeze VM, something I didn't get to do in Virtualbox (mostly because I didn't want to wait through the install, because it was so slow). Finally, I still have trouble getting a commandline console on the VM: somehow, running virtsh console squeeze-amd64 doesn't give me a login terminal, and worse, it actually freezes the terminal that I can actually get on virt-viewer squeeze-amd64, which definitely sounds like a bug. I documented a bit more of that setup in the Debian wiki KVM page so hopefully this will be useful for others.

Other free software work I continued my work on improving timetracking with ledger in my ledger-timetracking git repository, which now got a place on the new plaintextaccounting.org website, which acts as a portal for ledger-like software projects and documentation.

Darktable 2.0 I had the pleasure of trying the new Darktable 2.0 release, which only recently entered Debian. I built a backport for jessie, which works beautifully: much faster thumbnail rendering, no dropping of history when switching views... The new features are great, but I also appreciate how they are being very conservative in their approach. Darktable is great software: I may have trouble approaching the results other are having with lightroom and snapseed, but those are proprietary software that I can't use anyways. I also suspect that I just don't have enough of a clue of what I'm doing to get the results I need in Darktable. Maybe with hand-holding, one day, I will surpass the results I get with the JPEGs from my Canon camera. Until then, I turned off RAW exports in my camera to try and control the explosion of disk use I saw since I got that camera:

41M     2004
363M    2005
937M    2006
2,2G    2007
894M    2008
800M    2009
1,8G    2010
1,4G    2011
9,8G    2012
31G     2013
26G     2014
9,8G    2015

The drop in 2015 is mostly due to me taking less pictures in the last year, for some reason...

Markdown mode hacks I ended up writing some elisp for the markdown mode. It seems I am always writing links like [text](link) which seems more natural at first, but then the formatting looks messier, as paragraph wrapping is all off because of the long URLs. So I always ended up converting those links, which was a painful series of keystrokes. So I made a macro, and while I'm a it, why not rewrite it as a lisp function. Twice. Then I was told by the markdown-mode.el developers that they had already fixed that (in the 2.1 version, not in Debian jessie) and that the C-c C-a r key binding actually recognized existing links and conveniently converted them. I documented my adventures in bug #94, but it seems I wrote this code for nothing else than re-learning Emacs lisp, which was actually quite fun.

More emacs hacking Another thing I always wasted time doing by and is "rename file and buffer". Often, you visit a file but it's named wrong. My most common case is a .txt file that i rename to .mdwn. I would then have to do:

M-x rename-file <ret> newfile
M-x rename-buffer <ret> newfile
C-x C-s <ret> newfile

Really annoying. Turns out that set-visited-file-name actually does most of the job, but doesn't actually rename the file, which is really silly. So I wrote this small function instead:

(defun rename-file-and-buffer (newfname)
  "combine rename-file and rename-buffer
set-visited-file-name does most of the job, but unfortunately
doesn't actually rename the file. rename-file does that, but
doesn't rename the buffer. rename-buffer only renames the buffer,
which is pretty pointless.
only operates on current buffer because set-visited-file-name
also does so and we don't bother doing excursions around.
"
  (interactive "GRename file and bufer: ")
  (let ((oldfname (buffer-file-name)))
    (set-visited-file-name newfname nil t)
    (rename-file oldfname newfname)
    )
  )

Not bound to any key, really trivial, but doing this without that function is really non-trivial, especially since set-visited-file-name needs special arguments to not mark the file as modified.

IRC packages updates I updated the Sopel IRC bot package to the latest release, 6.3.0. They have finally switched to Requests, but apart from that, no change was necessary. I am glad to finally see SNI support working everywhere in the bot! I also update the Charydbis IRC server package to the latest 3.5.0 stable release. This release is great news, as I was able to remove 5 of the 7 patches I was dragging along the Debian package. The previous Charybdis stable release was over 3 years old, as 3.4.2 was released in (December) 2012! I spend a good chunk of time making the package reproducible. I filed a bug upstream and eventually made a patch to make it possible to hardcode a build timestamp, which seems to have been the only detectable change in the reproducible build infrastructure. Charybdis had been FTBS for a while in sid now, and the upload should fix that as well. Unfortunately, Charybdis still doesn't build with hardening flags - but hopefully a future update of the package should fix that. It is probably because CFLAGS are not passed around properly. There's really interesting stuff going on in the IRC world. Even though IRC is one of the oldest protocols still in operation (1988, even before the Web, but after SMTP and the even more venerable FTP), it is still being actively developed, with a working group drafting multiple extensions to the IRCv3 protocol defined in RFC 1459. For example, IRCv3.3 includes a Strict Transport Security extension, which tries to ensure users use encrypted channels as much as possible, through warnings and STARTTLS support. Charybdis goes even further by proposing a reversal of the +S ("secure channel" flag) where all channels are secure by default, and you need to deliberately mark a channel as insecure with the +U flag if you actually want to allow users on an clear-text connection to join the channel. A transition mechanism is also proposed.

Miscellaneous bug reports En vrac... I fell face-first in this amazing game that is endless-sky. I made a small pull request on the documentation, a bug report and a feature request. I forwarded a bug report, originally filed against monkeysign, to the pyqrencode maintainers. I filed a usability bug against tails-installer, which just entered Debian, mostly usability issues. I discovered the fim image viewer, which re-entered Debian recently. It seemed perfect to adjust my photos-import workflow, so I added it to my script, to be able to review photos prior to importing them into Darktable and git-annex.

The first major version of the scalable timeserie database I work on, Gnocchi was a released a few months ago. In this first iteration, it took a rather naive approach to data storage. We had little ideas about if and how our distributed back-ends were going to be heavily used, so we stuck to the code of the first proof-of-concept written a couple of years ago. Recently we got more feedbacks from our users, ran a few benchmarks. That gave us enough feedback to start investigating in improving our storage strategy. Data split Up to Gnocchi 1.3, all data for a single metric are stored in a single gigantic file per aggregation method (min, max, average ). This means that the file can grow to several megabytes in size, which make it slow to manipulate. For the next version of Gnocchi, our first work has been to rework that storage and split the data into smaller parts. The diagram above shows how data are organized inside Gnocchi. Until version 1.3, there would have been only one file for each aggregation methods. In the upcoming 2.0 version, Gnocchi will split all these data into smaller parts, where each data split is stored in a file/object. This allows to manipulate smaller pieces of data and to increase the parallelism of the CRUD operations on the back-end leading to large speed improvement. In order to split timeseries into several chunks, Gnocchi defines a maximum number of N points to keep per chunk, to limit their maximum size. It then defines a hash function that produces a non-unique key for any timestamp. It makes it easy to find in which chunk any timestamp should be stored or retrieved. Data compression Up to Gnocchi 1.3, the data stored for each metric is simply serialized using msgpack, a fast and small serialization format. Though, this format does not provide any compression. That means that storing data points needs 8 bytes for a timestamp (64 bits timestamp with nanosecond precision) and 8 bytes for a value (64 bits double-precision floating-point), plus some overhead (extra information and msgpack itself). After looking around on how to compress all these measures, I stumbled upon a paper from some Facebook engineers called about Gorilla, their in-memory timeserie database, entitled "Gorilla: A Fast, Scalable, In-Memory Time Series Database". For reference, part of this encoding is also used by InfluxDB in its new storage engine. The first technique I implemented is easy enough, and it's inspired from delta-of-delta encoding. Instead of storing each timestamp for each data point, and since all the data points are aggregated on a regular interval, we transpose points to be the time difference divided by the interval. For example, the suite of timestamps timestamps = [41230, 41235, 41240, 41250, 41255] is encoded into timestamps = [41230, 1, 1, 2, 1], interval = 5. This allows regular compression algorithms to reduce the size of the integer list using run-length encoding. To actually compress the values, I tried two different algorithms:

• [LZ4](https://en.wikipedia.org/wiki/LZ4_(compression_algorithm), a fast compression/decompression algorithm
• The XOR based compression scheme described in the Gorilla paper mentioned above that I had to implement myself. For reference, it also exists a Go implementation in go-tsz.

I then benchmarked these solutions: The XOR algorithm implemented in Python is pretty slow, compared to LZ4. Truth is that python-lz4 is fully implemented in C, which makes it fast. I've profiled my XOR implementation in Python, to discover that one operation took 20 % of the time: count_lead_and_trail_zeroes, which is in charge of counting the number of leading and trailing zeroes in a binary number. I tried 2 Python implementations of the same algorithm (and submitted them to my friend and Python developer Victor Stinner by the way). The first version using string search with .index() is 10 faster than the second one that only do integer computation. Ah, Python As Victor explained, each Python operation is slow and there's a lot in the second version, whereas .index() is implemented in C and really well optimized and only needs 2 Python operations. Finally, I ended up optimizing that code by leveraging cffi to use directly ffsll() and flsll(). That decreased the run-time of count_lead_and_trail_zeroes by 45 %, making the entire XOR compression code speed increased by a small 7 %. This is not enough to catch up with LZ4 speed. At this stage, the only solution to achieve a high-speed would probably to go with a full C implementation. Considering the compression ratio of the different algorithms, they are pretty much identical. The worst case scenario (random values) for LZ4 compress down to 9 bytes per data point, whereas XOR can go down to 7.38 bytes per data point. In general XOR encoding beats LZ4 by 15 %, except for cases where all values are 0 or 1. However, LZ4 is faster than XOR by a factor of 4 -70 depending on cases. That means that we'll use LZ4 for data compression in Gnocchi 2.0. It's possible that we could achieve as fast compression/decompression algorithm, but I don't think it's worth the effort right now it'd represent a lot of code to write and to maintain.

Johannes Bechberger, while working on his Bachelor s thesis supervised by my colleague Andreas Zwinkau, has developed a performance benchmark runner and results visualizer called temci , and used GHC as a guinea pig. You can read his elaborate analysis on his blog. This is particularly interesting given recent discussions about GHC itself becoming slower and slower, as for example observed by Johannes Waldmann and Anthony Cowley. Johannes Bechberger s take-away is that, at least for the programs at hand (which were taken from the The Computer Language Benchmarks Game, there are hardly any changes worth mentioning, as most of the observed effects are less than a standard deviation and hence insignificant. He tries hard to distill some useful conclusions from the data; the one he finds are:

• Compile time does not vary significantly.
• The compiler flag -O2 indeed results in faster code than -O.
• With -O (but not -O2), GHC 8.0.1 is better than GHC 7.0.1. Maybe some optimizations were promoted to -O?

If you are interested, please head over to Johannes s post and look at the gory details of the analysis and give him feedback on that. Also, maybe his tool temci is something you want to try out? Personally, I find it dissatisfying to learn so little from so much work, but as he writes: It s so easy to lie with statistics. , and I might add lie to yourself , e.g. by ignoring good advise about standard deviations and significance. I m sure my tool gipeda (which powers perf.haskell.org) is guilty of that sin. Maybe a different selection of test programs would yield more insight; the benchmark s games programs are too small and hand-optimized, the nofib programs are plain old and the fibon collection has bitrotted. I would love to see a curated, collection of real-world programs, bundled with all dependencies and frozen to allow meaningful comparisons, but updated to a new, clearly marked revision, on a maybe bi-yearly basis maybe Haskell-SPEC-2016 if that were not a trademark infringement.

Since fourteen years, I have been photographing digitally and putting the pictures on my webpage. Back then, online privacy was not a big deal, but things have changed, and I had to at least mildly protect the innocent. In particular, I wanted to prevent search engines from accessing some of my pictures. As I did not want my friends and family having to create an account and remember a password, I set up an OpenID based scheme five years ago. This way, they could use any of their OpenID enabled account, e.g. their Google Mail account, to log in, without disclosing any data to me. As my photo album consists of just static files, I created two copies on the server: the real one with everything, and a bunch of symbolic links representing the publicly visible parts. I then used mod_auth_openid to prevent access to the protected files, unless the users logged in. I never got around of actually limiting who could log in, so strangers were still able to see all photos, but at least search engine spiders were locked out. But, very unfortunately, OpenID did never really catch on, Google even stopped being a provider, and other promising decentralized authentication schemes like Mozilla Persona are also phased out. So I needed an alternative. A very simply scheme would be a single password that my friends and family can get from me and that unlocks the pictures. I could have done that using HTTP Auth, but that is not very user-friendly, and the login does not persist (at least not without the help of the browser). Instead, I wanted something that involves a simple HTTP form. But I also wanted to avoid server-side programming, for performance and security reasons. I love serving static files whenever it is feasible. Then I found that mod_rewrite, Apache s all-around-tool for URL rewriting and request mangling, supports reading and writing cookies! So I came up with a scheme that implements the whole login logic in the Apache server configuration. I d like to describe this setup here, in case someone finds it inspiring. I created a login.html with a simple HTML form:

<form method="GET" action="/bilder/login.html">
 <div style="text-align:center">
  <input name="password" placeholder="Password" />
  <button type="submit">Sign-In</button>
 </div>
</form>

It sends the user to the same page again, putting the password into the query string, hence the method="GET" mod_rewrite unfortunately cannot read the parameters of a POST request. The Apache configuration is as follows:

RewriteMap public "dbm:/var/www/joachim-breitner.de/bilder/publicfiles.dbm"
<Directory /var/www/joachim-breitner.de/bilder>
 RewriteEngine On
 # This is a GET request, trying to set a password.
 RewriteCond % QUERY_STRING  password=correcthorsebatterystaple
 RewriteRule ^login.html /bilder/loggedin.html [L,R,QSD,CO=bilderhp:correcthorsebatterystaple:www.joachim-breitner.de:2000000:/bilder]
 # This is a GET request, trying to set a wrong password.
 RewriteCond % QUERY_STRING  password=
 RewriteRule ^login.html /bilder/notloggedin.html [L,R,QSD]
 # No point in loggin in if there is already the right password
 RewriteCond % HTTP:Cookie  bilderhp=correcthorsebatterystaple
 RewriteRule ^login.html /bilder/loggedin.html [L,R]
 # If protected file is requested, check for cookie.
 # If no cookie present, redirect pictures to replacement picture
 RewriteCond % HTTP:Cookie  !bilderhp=correcthorsebatterystaple
 RewriteCond $ public:$0 private  private
 RewriteRule ^.*\.(png jpg)$ /bilder/pleaselogin.png [L]
 RewriteCond % HTTP:Cookie  !bilderhp=correcthorsebatterystaple
 RewriteCond $ public:$0 private  private
 RewriteRule ^.+$ /bilder/login.html [L,R]
</Directory>

The publicfiles.dbm file is generated from a text file with lines like

login.html.en 1
login.html.de 1
pleaselogin.png 1
thumbs/20030920165701_thumb.jpg 1
thumbs/20080813225123_thumb.jpg 1
...

using

/usr/sbin/httxt2dbm -i publicfiles.txt -o publicfiles.dbm

and whitelists all files that are visible without login. Make sure it contains the login page, otherwise you ll get a redirect circle. The other directives in the above configure fulfill these tasks:

• If the password (correcthorsebatterystaple) is in the query string, the server redirects the user to a logged-in-page that tells him that the login was successful and instructs him to reload the photo album. It also sets a cookie that will last very long -- after all, I want this to be convenient for my visitors. The query string parsing is not very strict (e.g. a password of correcthorsebatterystaplexkcdrules would also work), but that s ok.
• The next request detects an attempt to set a password. It must be wrong (otherwise the first rule would have matched), so we redirect the user to a variant of the login page that tells him so.
• If the user tries to access the login page with a valid cookie, just log him in.
• The next two rules implement the actual protection. If there no valid cookie and the accessed file is not whitelisted, then access is forbidden. For requests to images, we do an internal redirect to a placeholder image, while for everything else we redirect the user to the login page.

And that s it! No resource-hogging web frameworks, not security-dubious scripting languages, and a dead-simple way to authenticate. Oh, and if you believe you know me well enough to be allowed to see all photos: The real password is not correcthorsebatterystaple; just ask me what it is.

I've been in favor of using TLS and encryption for as much as possible for a while, but I never wanted to pay money to the certificate cartel. I'd been using certificates from CAcert, but they're not recognized by most browsers, so it felt rude to redirect everything to TLS with one of their certificates. Finally, the EFF and others put together Let's Encrypt with free, browser-recognized certificates and even a really solid automatic renewal system. That's perfect, and also eliminated my last excuse to go do the work, so now all of my personal web sites use TLS and HTTPS by default and redirect to the encrypted version of the web site. And better yet, all the certificates should just renew themselves automatically, meaning one less thing I have to keep track of and deal with periodically. Many thanks to Wouter Verhelst for his short summary of how to get the Let's Encrypt client to work properly from the command line without doing all the other stuff it wants to do in order to make things easier for less sophisticated users. Also useful was the SSL Labs server test to make sure I got the modern TLS configuration right. (All my sites should now be an A. I decided to not cut off support for Internet Explorer older than version 11 yet. I imported copies of the Debian packages needed for installation of the Let's Encrypt package on Debian jessie that weren't already in Debian backports into my personal Debian repository for my own convenience, but they're also there for anyone else. Oh, that reminds me: this also affects the archives.eyrie.org APT repository (the one linked above), so if any of you were using that, you'll now need to install apt-transport-https and might want to change the URL to use HTTPS.