Some things I don't really understand reading in German media

• Suddenly the electoral system becomes a legitimacy problem. While it has never been a problem for any of the previous decisions of the Catalunyan regional government suddenly a "only 48% of people voted for the government" results in the decisions being illegitimate? This is also a property of many governments (Greece and the US president being obvious examples but also the German Bundestag can have a majority government without the majority of votes). Is this just the media trying to find something they can blame on "the other side"?
• How can you ever possibly excuse violence against people peacefully and non-violently doing whatever they're doing. Sure this referendum was considered illegal (and it may be legitimate to ignore the result, or legal prosecution of the initiators) but how can that ever possibly be an excuse for half a population peacefully doing whatever they are about to do? How can you possibly claim that "both sides are to blame" for the violence? "Die Zeit" seems to be the only one with an somewhat convincing argument ("Deciding to press on despite the obviously happening violence") while "Welt", "Spiegel" and "S ddeutsche" all trying to blame the regional government for the violence with as much of an argument as asking people to do something illegal in a totally peaceful way. Possibly an argument for legal consequences, sure -- but for violence?

Too bad I didn't keep the links / articles from Sunday night.

A short while ago, FAUST participated in this year's CSAW qualification and -- as usual -- I was working on the Crypto challenges again. The first puzzle I worked on was called "Another Xor" -- and, while there are quite some write ups already our solution was somewhat different (maybe even the intended solution given how nice things worked out) and certainly interesting. The challenge provides a cipher-text. It's essentially a stream cipher with key repeated to generate the key stream. The plain-text was plain + key + checksum.

p = this is a plaintextThis is the keyfa5d46a2a2dcdeb83e0241ee2c0437f7
k = This is the keyThis is the keyThis is the keyThis is the keyThis i

Key length Our first step was figuring out the key length. Let's assume for now the key was This is the key. Notice that the key is also part of the plain-text and we know something about its location -- it ends at 32 characters from the back. If we only take a look at the encrypted key it should have the following structure:

p' = This is the key
k' = he keyThis is t

The thing to notice here is that every character in the Key appears both in the plain-text and key stream sequence. And the cipher-text is the XOR ( ) of both. Therefore XOR over the cipher-text sequence encrypting the key should equal 0 ( (p') (k') = 0). So remove the last 32 characters and find all suffixes that result in a XOR of 0. Fortunately there is exactly one such suffix (there could be multiple) and therefore we know the key size: 67. To put it in code, this basically is the function we implemented for this:

def calculate(ciphertextcandidate):
    accumulator = 0
    for char in ciphertextcandidate:
        accumulator = accumulator ^ char

Which, for the matching plain-text and key-stream fragments is equal (due to the XOR encryption) to

def calculate(plainfragment, keyfragment):
    accumulator = 0
    for i in range(len(plainfragment):
        accumulator = accumulator ^ (plainfragment[i] ^ keyfragment[i])

Now XOR lets us nicely reorder this to

def calculate(plainfragment, keyfragment):
    accumulator = 0
    for i in range(len(plainfragment):
        accumulator = accumulator ^ (plainfragment[i] ^
                                     keyfragment[(i + 6) % len(plainfragment)])

And, as plainfragment[i] and keyfragment[(i + 6) % len(plainfragment)] are equal for the plain-text range encoding the key this becomes

def calculate(plainfragment, keyfragment):
    accumulator = 0
    for i in range(len(plainfragment):
        accumulator = accumulator ^ 0

Or simply 0 if the guess of the cipher-text range is correct. Key recovery Now the nice thing to notice is that the length of the key (67) is a prime (and 38, the plain-text length, is a generator). As a result, we only need to guess one byte of the key: Assume you know one byte of the key (and the position). Now you can use that one byte of the key to decrypt the next byte of the key (using the area where the key is part of the plain-text). Due to the primeness of the key length this allows recovery of the full key. Finally you can either print all 256 options and look for the one that looks reasonable or you can verify the md5sum which will give you the one valid solution, flag sti11_us3_da_x0r_for_my_s3cratz . Code

cipher = b"'L\x10\x12\x1a\x01\x00I[P-U\x1cU\x7f\x0b\x083X]\x1b'\x03\x0bR(\x04\r7SI\n\x1c\x02T\x15\x05\x15%EQ\x18\x00\x19\x11SJ\x00RV\n\x14YO\x0b\x1eI\n\x01\x0cE\x14A\x1e\x07\x00\x14aZ\x18\x1b\x02R\x1bX\x03\x05\x17\x00\x02\x07K\n\x1aLAM\x1f\x1d\x17\x1d\x00\x15\x1b\x1d\x0fH\x0eI\x1e\x02I\x01\x0c\x15\x00P\x11\\PXPCB\x03B\x13TBL\x11PC\x0b^\tM\x14IW\x08\rDD%FC"
def keycover(guess):
    key = dict()
    pos = 38
    key[38] = guess
    for i in range(67):
        newpos = (pos % 67) + 38
        key[newpos] = xor(cipher[pos:], key[pos])
        pos = newpos
    try:
        return b''.join([ key[i] for i in range(38, 105, 1) ])
    except:
        return b'test'
for guess in range(256):
    keycand = keycover(bytes([guess]))
    plaincand = xor(cipher, repeat(keycand, len(cipher)))
    if md5(plaincand[:-32]).hexdigest().encode() == plaincand[-32:]:
        print(keycand, plaincand)

Seems it is now almost a decade since I migrated from Thunderbird to GNUS. And GNUS is an awesome mail program that I still rather like. However GNUS is also heavily quirky. It's essentially single-threaded and synchronous which means you either have to wait for the "IMAP check for new mails" to finish or you have to C-g abort it if you want the user interface to work; You have to wait for the "Move mail" to complete (which can take a while -- especially with dovecot-antispam training the filter) before you can continue working. It has it's funny way around TLS and certificate validation. And it seems to hang from time to time until it is C-g interrupted. So when I set up my new desktop machine I decided to try something else. My first try was claws-mail which seems OK but totally fails in the asynchronous area. While the GUI stays reactive, all actions that require IMAP interactions become incredibly slow when a background IMAP refresh is running. I do have quite some mailboxes and waiting the 5+ minutes after opening claws or whenever it decides to do a refresh is just to much. Now my last try has been Kmail -- also driven by the idea of having a more integrated setup with CalDAV and CardDAV around and similar goodies. And Kmail really compares nicely to claws in many ways. After all, I can use it while it's doing its things in the background. However the KDE folks seem to have dropped all support for the \recent IMAP flag which I heavily rely on. I do -- after all -- keep a GNUS like workflow where all unread mail (ref \seen) needs to still be acted upon which means there can easily be quite a few unread messages when I'm busy at the moment and just having a quick look at the new (ref \recent) mail to see if there's something super-urgent is essential. So I'm now looking for useful suggestions for a mail program (ideally with desktop integration) with the following essential features:

• It stays usable at all times -- which means smarter queuing than claws -- so foreground actions are not delayed by any background task the mail program might be up to and tasks like moving mail are handled in the background.
• Decent support for filtering. Apart from some basic stuff I need shortcut filtering for \recent mail.
• Option to hide \seen mail (and ideally hide all folders that only contain \seen mail). Hopefully toggle-able by some hotkey. "Age in days" would be an acceptable approximation, but Kmail doesn't seem to allow that in search (it's available as a filter though).

This weekend was ASIS Quals weekend again. And just like last year they have quite a lot of nice crypto-related puzzles which are fun to solve (and not "the same as every ctf"). Actually Secured OTP Server is pretty much the same as the First OTP Server (actually it's a "fixed" version to enforce the intended attack). However the template phrase now starts with enough stars to prevent simple root.:

def gen_otps():
    template_phrase = '*************** Welcome, dear customer, the secret passphrase for today is: '
    OTP_1 = template_phrase + gen_passphrase(18)
    OTP_2 = template_phrase + gen_passphrase(18)
    otp_1 = bytes_to_long(OTP_1)
    otp_2 = bytes_to_long(OTP_2)
    nbit, e = 2048, 3
    privkey = RSA.generate(nbit, e = e)
    pubkey  = privkey.publickey().exportKey()
    n = getattr(privkey.key, 'n')
    r = otp_2 - otp_1
    if r < 0:
        r = -r
    IMP = n - r**(e**2)
    if IMP > 0:
        c_1 = pow(otp_1, e, n)
        c_2 = pow(otp_2, e, n)
    return pubkey, OTP_1[-18:], OTP_2[-18:], c_1, c_2

Now let A = template * 2^(18*8), B = passphrase. This results in OTP = A + B. c therefore is (A+B)^3 mod n == A^3 + 3A^2b + 3AB^2 + B^3. Notice that only B^3 is larger than N and is statically known. Therefore we can calculate A^3 // N and add that to c to "undo" the modulo operation. With that it's only iroot and long_to_bytes to the solution. Note that we're talking about OTP and C here. The code actually produced two OTP and C values but you can use either one just fine.

#!/usr/bin/python3
import sys
from util import bytes_to_long
from gmpy2 import iroot
PREFIX = b'*************** Welcome, dear customer, the secret passphrase for today is: '
OTPbase = bytes_to_long(PREFIX + b'\x00' * 18)
N = 27990886688403106156886965929373472780889297823794580465068327683395428917362065615739951108259750066435069668684573174325731274170995250924795407965212988361462373732974161447634230854196410219114860784487233470335168426228481911440564783725621653286383831270780196463991259147093068328414348781344702123357674899863389442417020336086993549312395661361400479571900883022046732515264355119081391467082453786314312161949246102368333523674765325492285740191982756488086280405915565444751334123879989607088707099191056578977164106743480580290273650405587226976754077483115441525080890390557890622557458363028198676980513
WRAPPINGS = (OTPbase ** 3) // N
C = 13094996712007124344470117620331768168185106904388859938604066108465461324834973803666594501350900379061600358157727804618756203188081640756273094533547432660678049428176040512041763322083599542634138737945137753879630587019478835634179440093707008313841275705670232461560481682247853853414820158909864021171009368832781090330881410994954019971742796971725232022238997115648269445491368963695366241477101714073751712571563044945769609486276590337268791325927670563621008906770405196742606813034486998852494456372962791608053890663313231907163444106882221102735242733933067370757085585830451536661157788688695854436646
x = N * WRAPPINGS + C
val, _ = iroot(x, 3)
bstr = "%x" % int(val)
for i in range(0, len(bstr) // 2):
    sys.stdout.write(chr(int(bstr[2*i:2*i+2], 16)))
print()

As web search engines and IRC seems to be of no help, maybe someone here has a helpful idea. I have some service written in python that comes with a .service file for systemd. I now want to build&install a working service file from the software's setup.py. I can override the build/build_py commands of setuptools, however that way I still lack knowledge wrt. the bindir/prefix where my service script will be installed. Solution Turns out, if you override the install command (not the install_data!), you will have self.root and self.install_scripts (and lots of other self.install_*). As a result, you can read the template and write the desired output file after calling super's run method. The fix was inspired by GateOne (which, however doesn't get the --root parameter right, you need to strip self.root from the beginning of the path to actually make that work as intended). As suggested on IRC, the snippet (and my software) no use pkg-config to get at the systemd path as well. This is a nice improvement orthogonal to the original problem. The implementation here follows bley.

def systemd_unit_path():
    try:
        command = ["pkg-config", "--variable=systemdsystemunitdir", "systemd"]
        path = subprocess.check_output(command, stderr=subprocess.STDOUT)
        return path.decode().replace('\n', '')
    except (subprocess.CalledProcessError, OSError):
        return "/lib/systemd/system"
class my_install(install):
    _servicefiles = [
        'foo/bar.service',
        ]
    def run(self):
        install.run(self)
        if not self.dry_run:
            bindir = self.install_scripts
            if bindir.startswith(self.root):
                bindir = bindir[len(self.root):]
            systemddir = "%s%s" % (self.root, systemd_unit_path())
            for servicefile in self._servicefiles:
                service = os.path.split(servicefile)[1]
                self.announce("Creating %s" % os.path.join(systemddir, service),
                              level=2)
                with open(servicefile) as servicefd:
                    servicedata = servicefd.read()
                with open(os.path.join(systemddir, service), "w") as servicefd:
                    servicefd.write(servicedata.replace("%BINDIR%", bindir))

Comments, suggestions and improvements, of course, welcome!

Back in August, I was looking for a Homeserver replacement. During FrOSCon I was then reminded of the Turris Omnia project by NIC.cz. The basic SoC (Marvel Armada 38x) seemed to be nice hand have decent mainline support (and, with the turris, users interested in keeping it working). Only I don't want any WIFI and I wasn't sure the standard case would be all that usefully. Fortunately, there's also a simple board available with the same SoC called ClearFog and so I got one of these (the Base version). With shipping and the SSD (the only 2242 M.2 SSD with 250 GiB I could find, a ADATA SP600) it slightly exceeds the budget but well. When installing the machine, the obvious goal was to use mainline FOSS components only if possible. Fortunately there's mainline kernel support for the device as well as mainline U-Boot. First attempts to boot from a micro SD card did not work out at all, both with mainline U-Boot and the vendor version though. Turns out the eMMC version of the board does not support any micro SD cards at all, a fact that is documented but others failed to notice as well. U-Boot As the board does not come with any loader on eMMC and booting directly from M.2 requires removing some resistors from the board, the easiest way is using UART for booting. The vendor wiki has some shell script wrapping an included C fragment to feed U-Boot to the device but all that is really needed is U-Boot's kwboot utility. For some reason the SPL didn't properly detect UART booting on my device (wrong magic number) but patching the if (in arch-mvebu's spl.c) and always assume UART boot is an easy way around. The plan then was to boot a Debian armhf rootfs with a defconfig kernel from USB stick. and install U-Boot and the rootfs to eMMC from within that system. Unfortunately U-Boot seems to be unable to talk to the USB3 port so no kernel loading from there. One could probably make UART loading work but switching between screen for serial console and xmodem seemed somewhat fragile and I never got it working. However ethernet can be made to work, though you need to set eth1addr to eth3addr (or just the right one of these) in U-Boot, saveenv and reboot. After that TFTP works (but is somewhat slow). eMMC There's one last step required to allow U-Boot and Linux to access the eMMC. eMMC is wired to the same PINs as the SD card would be. However the SD card has an additional indicator pin showing whether a card is present. You might be lucky inserting a dummy card into the slot or go the clean route and remove the pin specification from the device tree.

--- a/arch/arm/dts/armada-388-clearfog.dts
+++ b/arch/arm/dts/armada-388-clearfog.dts
@@ -306,7 +307,6 @@
                        sdhci@d8000  
                                bus-width = <4>;
-                               cd-gpios = <&gpio0 20 GPIO_ACTIVE_LOW>;
                                no-1-8-v;
                                pinctrl-0 = <&clearfog_sdhci_pins
                                             &clearfog_sdhci_cd_pins>;

Next Up is flashing the U-Boot to eMMC. This seems to work with the vendor U-Boot but proves to be tricky with mainline. The fun part boils down to the fact that the boot firmware reads the first block from eMMC, but the second from SD card. If you write the mainline U-Boot, which was written and tested for SD card, to eMMC the SPL will try to load the main U-Boot starting from it's second sector from flash -- obviously resulting in garbage. This one took me several tries to figure out and made me read most of the SPL code for the device. The fix however is trivial (apart from the question on how to support all different variants from one codebase, which I'll leave to the U-Boot developers):

--- a/include/configs/clearfog.h
+++ b/include/configs/clearfog.h
@@ -143,8 +143,7 @@
 #define CONFIG_SPL_LIBDISK_SUPPORT
 #define CONFIG_SYS_MMC_U_BOOT_OFFS             (160 << 10)
 #define CONFIG_SYS_U_BOOT_OFFS                 CONFIG_SYS_MMC_U_BOOT_OFFS
-#define CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_SECTOR        ((CONFIG_SYS_U_BOOT_OFFS / 512)\
-                                                + 1)
+#define CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_SECTOR        (CONFIG_SYS_U_BOOT_OFFS / 512)
 #define CONFIG_SYS_U_BOOT_MAX_SIZE_SECTORS     ((512 << 10) / 512) /* 512KiB */
 #ifdef CONFIG_SPL_BUILD
 #define CONFIG_FIXED_SDHCI_ALIGNED_BUFFER      0x00180000      /* in SDRAM */

Linux Now we have a System booting from eMMC with mainline U-Boot (which is a most welcome speedup compared to the UART and TFTP combination from the beginning). Getting to fine-tune linux on the device -- we want to install the armmp Debian kernel and have it work. As all the drivers are build as modules for that kernel this also means initrd support. Funnily U-Boots bootz allows booting a plain vmlinux kernel but I couldn't get it to boot a plain initrd. Passing a uImage initrd and a normal kernel however works pretty well. Back when I first tried there were some modules missing and ethernet didn't work with the PHY driver built as a module. In the meantime the PHY problem was fixed in the Debian kernel and almost all modules already added. Ben then only added the USB3 module on my suggestion and as a result, unstable's armhf armmp kernel should work perfectly well on the device (you still need to patch the device tree similar to the patch above). Still missing is an updated flash-kernel to automatically generate the initrd uImage which is work in progress but got stalled until I fixed the U-Boot on eMMC problem and everything should be fine -- maybe get debian u-boot builds for that board. Pro versus Base The main difference so far between the Pro and the Base version of the ClearFog is the switch chip which is included on the Pro. The Base instead "just" has two gigabit ethernet ports and a SFP. Both, linux' and U-Boot's device tree are intended for the Pro version which makes on of the ethernet ports unusable (it tries to find the switch behind the ethernet port which isn't there). To get both ports working (or the one you settled on earlier) there's a second patch to the device tree (my version might be sub-optimal but works), U-Boot -- the linux-kernel version is a trivial adaption:

--- a/arch/arm/dts/armada-388-clearfog.dts
+++ b/arch/arm/dts/armada-388-clearfog.dts
@@ -89,13 +89,10 @@
                internal-regs  
                        ethernet@30000  
                                mac-address = [00 50 43 02 02 02];
+                               managed = "in-band-status";
+                               phy = <&phy1>;
                                phy-mode = "sgmii";
                                status = "okay";
-
-                               fixed-link  
-                                       speed = <1000>;
-                                       full-duplex;
-                                ;
                         ;
                        ethernet@34000  
@@ -227,6 +224,10 @@
                                pinctrl-0 = <&mdio_pins>;
                                pinctrl-names = "default";
+                               phy1: ethernet-phy@1   /* Marvell 88E1512 */
+                                    reg = <1>;
+                                ;
+
                                phy_dedicated: ethernet-phy@0  
                                        /*
                                         * Annoyingly, the marvell phy driver
@@ -386,62 +386,6 @@
                tx-fault-gpio = <&expander0 13 GPIO_ACTIVE_HIGH>;
         ;
-       dsa@0  
-               compatible = "marvell,dsa";
-               dsa,ethernet = <&eth1>;
-               dsa,mii-bus = <&mdio>;
-               pinctrl-0 = <&clearfog_dsa0_clk_pins &clearfog_dsa0_pins>;
-               pinctrl-names = "default";
-               #address-cells = <2>;
-               #size-cells = <0>;
-
-               switch@0  
-                       #address-cells = <1>;
-                       #size-cells = <0>;
-                       reg = <4 0>;
-
-                       port@0  
-                               reg = <0>;
-                               label = "lan1";
-                        ;
-
-                       port@1  
-                               reg = <1>;
-                               label = "lan2";
-                        ;
-
-                       port@2  
-                               reg = <2>;
-                               label = "lan3";
-                        ;
-
-                       port@3  
-                               reg = <3>;
-                               label = "lan4";
-                        ;
-
-                       port@4  
-                               reg = <4>;
-                               label = "lan5";
-                        ;
-
-                       port@5  
-                               reg = <5>;
-                               label = "cpu";
-                        ;
-
-                       port@6  
-                               /* 88E1512 external phy */
-                               reg = <6>;
-                               label = "lan6";
-                               fixed-link  
-                                       speed = <1000>;
-                                       full-duplex;
-                                ;
-                        ;
-                ;
-        ;
-
        gpio-keys  
                compatible = "gpio-keys";
                pinctrl-0 = <&rear_button_pins>;

Conclusion Apart from the mess with eMMC this seems to be a pretty nice device. It's now happily running with a M.2 SSD providing enough storage for now and still has a mSATA/mPCIe plug left for future journeys. It seems to be drawing around 5.5 Watts with SSD and one Ethernet connected while mostly idle and can feed around 500 Mb/s from disk over an encrypted ethernet connection which is, I guess, not too bad. My plans now include helping to finish flash-kernel support, creating a nice case and probably get it deployed. I might bring it to FOSDEM first though. Working on it was really quite some fun (apart from the frustrating parts finding the one-block-offset ..) and people were really helpful. Big thanks here to Debian's arm folks, Ben Hutchings the kernel maintainer and U-Boot upstream (especially Tom Rini and Stefan Roese)

At this year's FrOSCon I repeted my presentation on DNSSEC. In the audience, there was the suggestion of a lack of proper monitoring plugins for a DANE and DNSSEC infrastructure that was easily available. As I already had some personal tools around and some spare time to burn I've just started a repository with some useful tools. It's available on my website and has mirrors on Gitlab and Github. I intent to keep this repository up-to-date with my personal requirements (which also means adding a xmpp check soon) and am happy to take any contributions (either by mail or as "pull requests" on one of the two mirrors). It currently has smtp (both ssmtp and starttls) and https support as well as support for checking valid DNSSEC configuration of a zone. While working on it it turned out some things can be complicated. My language of choice was python3 (if only because the ssl library has improved since 2.7 a lot), however ldns and unbound in Debian lack python3 support in their bindings. This seems fixable as the source in Debian is buildable and useable with python3 so it just needs packaging adjustments. Funnily the ldns module, which is only needed for check_dnssec, in debian is currently buggy for python2 and python3 and ldns' python3 support is somewhat lacking so I spent several hours hunting SWIG problems.

Almost exactly six years ago I bought one of these Fuloong 6064 mini PCs. The machine has been working great ever since both collecting my mail and acting as an IMAP server as well as providing public services -- it's also keyserver.siccegge.de. However jessie is supposed to be the last Debian release supporting the hardware and the system's rather slow and lacks memory. This is especially noticeable with IMAP spam filter training and mail indexing. Therefore I'm looking for some nice replacement -- preferably non-x86 again (no technical reasons). My requirements are pretty simple:

• Works with vanilla stretch (and stretch kernel)
• Still works with Debian stable six years from now
• Faster (single-core performance, 2-4 cores would be nice as well), currently it's a 900MHz super-scalar, out-of-order MIPS64 CPU
• Consumes less power
• SATA port
• Preferably fanless
• Maximum same price range, around 200 EUR including case and shipping

Now I'd consider one of these ARM boards and get it a nice case but they seem all to either fail in terms of SATA or not being faster at all (and one needs to go for outdated hardware to stand a chance of mainline kernel support). If anyone knows something nice and non-x86 I'll happily take suggestions.

Without further words:

% for i in $(seq 1 90) ; do doveadm mailbox status messages debian.buildd.archive.2011.05   column -t ;  doveadm deduplicate mailbox debian.buildd.archive.2011.05 ; done
debian.buildd.archive.2011.05  messages=8094
debian.buildd.archive.2011.05  messages=7939
debian.buildd.archive.2011.05  messages=7816
debian.buildd.archive.2011.05  messages=7698
debian.buildd.archive.2011.05  messages=7610
debian.buildd.archive.2011.05  messages=7529
debian.buildd.archive.2011.05  messages=7455
debian.buildd.archive.2011.05  messages=7375
debian.buildd.archive.2011.05  messages=7294
debian.buildd.archive.2011.05  messages=7215
debian.buildd.archive.2011.05  messages=7136
debian.buildd.archive.2011.05  messages=7032
debian.buildd.archive.2011.05  messages=6941
debian.buildd.archive.2011.05  messages=6839
debian.buildd.archive.2011.05  messages=6721
debian.buildd.archive.2011.05  messages=6631
debian.buildd.archive.2011.05  messages=6553
debian.buildd.archive.2011.05  messages=6476
debian.buildd.archive.2011.05  messages=6388
debian.buildd.archive.2011.05  messages=6301
debian.buildd.archive.2011.05  messages=6211
debian.buildd.archive.2011.05  messages=6140
debian.buildd.archive.2011.05  messages=6056
debian.buildd.archive.2011.05  messages=6007
debian.buildd.archive.2011.05  messages=5955
debian.buildd.archive.2011.05  messages=5887
debian.buildd.archive.2011.05  messages=5826
debian.buildd.archive.2011.05  messages=5752
debian.buildd.archive.2011.05  messages=5706
debian.buildd.archive.2011.05  messages=5657
debian.buildd.archive.2011.05  messages=5612
debian.buildd.archive.2011.05  messages=5570
debian.buildd.archive.2011.05  messages=5523
debian.buildd.archive.2011.05  messages=5474
debian.buildd.archive.2011.05  messages=5422
debian.buildd.archive.2011.05  messages=5382
debian.buildd.archive.2011.05  messages=5343
debian.buildd.archive.2011.05  messages=5308
debian.buildd.archive.2011.05  messages=5256
debian.buildd.archive.2011.05  messages=5221
debian.buildd.archive.2011.05  messages=5168
debian.buildd.archive.2011.05  messages=5133
debian.buildd.archive.2011.05  messages=5092
debian.buildd.archive.2011.05  messages=5058
debian.buildd.archive.2011.05  messages=5030
debian.buildd.archive.2011.05  messages=4994
debian.buildd.archive.2011.05  messages=4964
debian.buildd.archive.2011.05  messages=4935
debian.buildd.archive.2011.05  messages=4900
debian.buildd.archive.2011.05  messages=4868
debian.buildd.archive.2011.05  messages=4838
debian.buildd.archive.2011.05  messages=4811
debian.buildd.archive.2011.05  messages=4778
debian.buildd.archive.2011.05  messages=4748
debian.buildd.archive.2011.05  messages=4722
debian.buildd.archive.2011.05  messages=4686
debian.buildd.archive.2011.05  messages=4661
debian.buildd.archive.2011.05  messages=4637
debian.buildd.archive.2011.05  messages=4613
debian.buildd.archive.2011.05  messages=4593
debian.buildd.archive.2011.05  messages=4570
debian.buildd.archive.2011.05  messages=4554
debian.buildd.archive.2011.05  messages=4536
debian.buildd.archive.2011.05  messages=4520
debian.buildd.archive.2011.05  messages=4500
debian.buildd.archive.2011.05  messages=4481
debian.buildd.archive.2011.05  messages=4466
debian.buildd.archive.2011.05  messages=4445
debian.buildd.archive.2011.05  messages=4430
debian.buildd.archive.2011.05  messages=4417
debian.buildd.archive.2011.05  messages=4405
debian.buildd.archive.2011.05  messages=4390
debian.buildd.archive.2011.05  messages=4376
debian.buildd.archive.2011.05  messages=4366
debian.buildd.archive.2011.05  messages=4360
debian.buildd.archive.2011.05  messages=4350
debian.buildd.archive.2011.05  messages=4336
debian.buildd.archive.2011.05  messages=4329
debian.buildd.archive.2011.05  messages=4320
debian.buildd.archive.2011.05  messages=4315
debian.buildd.archive.2011.05  messages=4312
debian.buildd.archive.2011.05  messages=4311
debian.buildd.archive.2011.05  messages=4309
debian.buildd.archive.2011.05  messages=4308
debian.buildd.archive.2011.05  messages=4308
debian.buildd.archive.2011.05  messages=4308
debian.buildd.archive.2011.05  messages=4308
debian.buildd.archive.2011.05  messages=4308
debian.buildd.archive.2011.05  messages=4308
debian.buildd.archive.2011.05  messages=4308

As of a few minutes ago, the old weblog on christoph-egger.org is past. I've added redirects for all the entries to the new one at weblog.siccegge.de.if you find any dead links please contact me so I can fix it up. Note that comments are gone. I'll try to include the already present comments on the new blog some time in the future. Not sure if I will ever add a comment function again (though chronicle seems to have some support for that)

I just released sbuild 0.66.0-1 into unstable. It fixes a whopping 30 bugs! Thus, I'd like to use this platform to:

• kindly ask all sbuild users to report any new bugs introduced with this release
• give a big thank you to everybody who supplied the patches that made fixing this many bugs possible (in alphabetical order): Aurelien Jarno, Christian Kastner, Christoph Egger, Colin Watson, Dima Kogan, Guillem Jover, Luca Falavigna, Maria Valentina Marin Rordrigues, Miguel A. Col n V lez, Paul Tagliamonte

And a super big thank you to Roger Leigh who, despite having resigned from Debian, was always available to give extremely helpful hints, tips, opinion and guidance with respect to sbuild development. Thank you! Here is a list of the major changes since the last release:

• add option --arch-all-only to build arch:all packages
• environment variable SBUILD_CONFIG allows to specify a custom configuration file
• add option --build-path to set a deterministic build path
• fix crossbuild dependency resolution
• add option --extra-repository-key for extra apt keys
• add option --build-dep-resolver=aspcud for aspcud based resolver
• allow complex commands as sbuild hooks
• add now external command %SBUILD_SHELL produces an interactive shell
• add options --build-deps-failed-commands, --build-failed-commands and --anything-failed-commands for more hooks

So moving, away from awesome Core-Backbone provided internet at home and to Kabel Deutschland madness. Missing routes for several hours and noone even seems to notice, no mail address to properly report issues. Quality Network.

traceroute to sks-keyservers.net (37.191.220.247), 30 hops max, 60 byte packets
 1  192.168.0.1 (192.168.0.1)  11.033 ms * *
 2  83-169-183-122-isp.superkabel.de (83.169.183.122)  28.413 ms  74.792 ms  75.376 ms
 3  ip5886ea49.dynamic.kabel-deutschland.de (88.134.234.73)  77.083 ms  77.671 ms  79.283 ms
 4  ip5886ee61.dynamic.kabel-deutschland.de (88.134.238.97)  79.833 ms ip5886ed7c.dynamic.kabel-deutschland.de (88.134.237.124)  81.416 ms  83.348 ms
 5  ip5886ca25.dynamic.kabel-deutschland.de (88.134.202.37)  87.592 ms  88.101 ms *
 6  * * *
 7  * * *
 8  * * *
[...]

Needless to say, of course, that it still works perfectly fine from the old network.

Slighthly delayed, but here are the stats for week 5 of the DUCK challenge:

• Emmanuel Bourg uploaded gant
• Marco d'Itri uploaded libsystemd-dummy
• Christoph Egger uploaded sbcl
• Sandro Tosi uploaded basemap
• Hilko Bengen uploaded libbde
• Bas Couwenberg uploaded gpx2shp
• Michael Stapelberg uploaded dh-make-golang
• Jackson Doak uploaded pyicu
• Peter Pentchev uploaded stdsyslog
• Gianfranco Costamagna uploaded cld2

So we had 10 packages fixed and uploaded by 10 different uploaders. A big "Thank You" to you!! Since the start of this challenge, a total of 59 packages, were fixed. Here is a quick overview:

	Week 1	Week 2	Week 3	Week 4	Week 5	Week 6	Week 7
# Packages	10	15	10	14	10	-	-
Total	10	25	35	49	59	-	-

The list of the fixed and updated packages is availabe here. I will try to update this ~daily. If I missed one of your uploads, please drop me a line. Only 2 more weeks to DebConf15 so please get involved: The DUCK Challenge is running until end of DebConf15! Pevious articles are here: Week 1, Week 2, Week 3, Week 4.

The problem One problem to solve when doing web authentication has always been one identity provider, so you don't have to remember which username (or email address) you used for that bugtracker you used three years ago or that website. And tie it to one login ideally. Five years ago this problem seemed to be basically solved. There was OpenID and while it may not have been great it worked. You could have your own provider, your institution (university, company, foss project, ..) could have one and you could use your university-provided ID for all university stuff. Today's state Looking at the problem again today and the situation seems to have changed. To the worse. A lot. People are actively removing OpenID support. There seemed to be a replacement with, at least, proper design goals: Mozilla's persona. However this seems to be a dead end, no-one (almost) actually supports it. Then there is what people call OAuth2. However there does not seem to be such a thing as OAuth2 at all, at least not for logging into websites. So for example phabricator supports 12 different OAuth2 systems. That includes Google, Facebook, Twitter, Amazon Github and a whole bunch of other services. Each with a different implementation in the webapp of course. And of course you can not just have your university/company/.. provide an OAuth2 service for you to use -- you would need to write yet another adapter on the (foreign) website to talk to your implementation and your provider. And the strange thing, people seem to still consider OAuth2 a replacement for OpenID while it does not even provide the core functionality of the older system. Plus there does not seem to be any awareness of that all together. Other features Now of course, OpenID is not (and never was) the ultimate answer to the web authentication problem. The most obvious problem being user tracking. Your identity provider will see every website you log into, will see when you log into it and even be able to log into that website with your credentials. Of course, this problem is fully inherited by OAuth2. And in contrast to OpenID you can no longer run your own provider whom you can fully trust and who already knows about your surfing habits (because it's actually you already). Mozilla's persona might have solved that, they at least intended to. But, again, persona seems quite dead.

logind hangs If you just updated systemd and ssh to that host seems to hang, that's just a known Bug (Debian Bug). Don't panic. Wait for the logind timeout and restart logind. restart and stop;start One thing that confused me several times and still confuses people is systemctl restart doing more than systemctl stop ; systemctl start. You will notice the difference once you have a failed service. A restart will try to start the service again. Both stop and start however will just ignore it. Rumors have it this has changed post jessie however. sysvinit-wrapped services and stop While there are certainly bugs with sysvinit services in general (I found myself several times without a local resolver as unbound failed to be started, haven't managed to debug further), the stop behavior of wrapped services is just broken. systemctl stop will block until the sysv initscript finished. It will even note the result of the action in its state. However systemctl will return with exitcode 0 and not output anything on stdout/stderr. This has been reported as Debian Bug. zsh helper I found the following zshrc snipped quite helpful in dealing with non-reported systemctl failures. On root shells it will display a list of failed services as part of the prompt. This will give proper feedback whether your systemctl stop failed, it will give feedback if you still have type=simple services and if the sysv-init script or wrapper is broken.

precmd ()  
    if [[ $UID == 0 && $+commands[systemctl] != 0 ]]
    then
      use_systemd=true
      systemd_failed=" systemctl --state=failed   grep failed   cut -d \  -f 2   tr '\n' ' ' "
    fi
 
if [[ $UID == 0 && $+commands[systemctl] != 0 ]]
then
  PROMPT=$'% $fg[red]>>  $systemd_failed$reset_color% \n'
else
  PROMPT=""
fi
PROMPT+=whateveryourpromptis

zsh completion Speaking of zsh, there's one problem that bothers me a lot and I don't have any solution for. Tab-completing the service name for service is blazing fast. Tab-completing the service name for systemctl restart takes ages. People traced down to truckloads of dbus communication for the completion but no further fix is known (to me). type=simple services As described in length by Lucas Nussbaum type=simple services are actively harmful. Proper type=forking daemons are strictly superior (they provide feedback of finished initialization and success thereof) and type=notify services are so simple there's no excuse for not using them even for private one-off hacks. Even if you're language doesn't provide libsystemd-daemon bindings:

(defun sd-notify (event-string)
  (let ((socket (make-instance 'sb-bsd-sockets:local-socket :type :datagram))
        (name (posix-getenv "NOTIFY_SOCKET"))
        (bufferlen (length event-string)))
    (when name
      (sb-bsd-sockets:socket-connect socket name)
      (sb-bsd-sockets:socket-send socket event-string bufferlen))))

This is a stable API guaranteed to not break in the future and implemented in less than ten lines of code with just basic socket functions. And if your language has support it becomes actually trivial:

    try:
        import systemd
        systemd.daemon.notify("READY=1")
    except ImportError:
        pass

Note that in both cases there is no drawback at all on systemd-free setups. It has the overhead of checking the process' environment for NOTIFY_SOCKET or for the systemd package and behaves like a simple service otherwise. Actually the idea of separating the technical aspect (daemonizing) from the semantic aspect of signalizing "initialization finished, everything's fine" is a pretty good idea and hopefully has the potential to reduce the number of services signalizing the "everything's fine" too early. It could even be ported to non-systemd init systems easily given the API.

logind hangs If you just updated systemd and ssh to that host seems to hang, that's just a known Bug (Debian Bug #770135). Don't panic. Wait for the logind timeout and restart logind. restart and stop;start One thing that confused me several times and still confuses people is systemctl restart doing more than systemctl stop ; systemctl start. You will notice the difference once you have a failed service. A restart will try to start the service again. Both stop and start however will just ignore it. Rumors have it this has changed post jessie however. sysvinit-wrapped services and stop While there are certainly bugs with sysvinit services in general (I found myself several times without a local resolver as unbound failed to be started, haven't managed to debug further), the stop behavior of wrapped services is just broken. systemctl stop will block until the sysv initscript finished. It will even note the result of the action in its state. However systemctl will return with exitcode 0 and not output anything on stdout/stderr. This has been reported as Debian Bug #792045. zsh helper I found the following zshrc snipped quite helpful in dealing with non-reported systemctl failures. On root shells it will display a list of failed services as part of the prompt. This will give proper feedback whether your systemctl stop failed, it will give feedback if you still have type=simple services and if the sysv-init script or wrapper is broken.

precmd ()  
    if [[ $UID == 0 && $+commands[systemctl] != 0 ]]
    then
      use_systemd=true
      systemd_failed=" systemctl --state=failed   grep failed   cut -d \  -f 2   tr '\n' ' ' "
    fi
 
if [[ $UID == 0 && $+commands[systemctl] != 0 ]]
then
  PROMPT=$'% $fg[red]>>  $systemd_failed$reset_color% \n'
else
  PROMPT=""
fi
PROMPT+=whateveryourpromptis

zsh completion Speaking of zsh, there's one problem that bothers me a lot and I don't have any solution for. Tab-completing the service name for service is blazing fast. Tab-completing the service name for systemctl restart takes ages. People traced down to truckloads of dbus communication for the completion but no further fix is known (to me). type=simple services As described in length by Lucas Nussbaum type=simple services are actively harmful. Proper type=forking daemons are strictly superior (they provide feedback of finished initialization and success thereof) and type=notify services are so simple there's no excuse for not using them even for private one-off hacks. Even if you're language doesn't provide libsystemd-daemon bindings:

(defun sd-notify (event-string)
  (let ((socket (make-instance 'sb-bsd-sockets:local-socket :type :datagram))
        (name (posix-getenv "NOTIFY_SOCKET"))
        (bufferlen (length event-string)))
    (when name
      (sb-bsd-sockets:socket-connect socket name)
      (sb-bsd-sockets:socket-send socket event-string bufferlen))))

This is a stable API guaranteed to not break in the future and implemented in less than ten lines of code with just basic socket functions. And if your language has support it becomes actually trivial:

    try:
        import systemd
        systemd.daemon.notify("READY=1")
    except ImportError:
        pass

Note that in both cases there is no drawback at all on systemd-free setups. It has the overhead of checking the process' environment for NOTIFY_SOCKET or for the systemd package and behaves like a simple service otherwise. Actually the idea of separating the technical aspect (daemonizing) from the semantic aspect of signalizing "initialization finished, everything's fine" is a pretty good idea and hopefully has the potential to reduce the number of services signalizing the "everything's fine" too early. It could even be ported to non-systemd init systems easily given the API.

TinyTiny-RSS has some nice failure modes. And upstream support forums aren't really helpfull so when you search for your current problem, chances are that there is one mention of it on the web, in the forum, and the only thing happening there is people making fun of the reporter. Anyway. This installation has seen lots of error messages from the updater in the last several months:

Warning: Fatal error, unknown preferences key: ALLOW_DUPLICATE_POSTS (owner: 3) in /srv/www/tt-rss.faui2k9.de/root/classes/db/prefs.php on line 108
Warning: Fatal error, unknown preferences key: ALLOW_DUPLICATE_POSTS (owner: 3) in /srv/www/tt-rss.faui2k9.de/root/classes/db/prefs.php on line 108
Warning: Fatal error, unknown preferences key: ALLOW_DUPLICATE_POSTS (owner: 3) in /srv/www/tt-rss.faui2k9.de/root/classes/db/prefs.php on line 108
Warning: Fatal error, unknown preferences key: ALLOW_DUPLICATE_POSTS (owner: 3) in /srv/www/tt-rss.faui2k9.de/root/classes/db/prefs.php on line 108
  

And, more recently, the android app stopped working with ERROR:JSON Parse failed.. Turns out both things are related. First thing I noticed was changing preferences in the web panel stopped working until you use the reset to Defaults option and then changed settings. Plugging wireshark in between showed what was going on (Note: API was displayed as enabled in Preferences/Preferences):

HTTP/1.1 200 OK
Server: nginx/1.8.0
Date: Thu, 06 Aug 2015 11:00:31 GMT
Content-Type: text/json
Transfer-Encoding: chunked
Connection: keep-alive
X-Powered-By: PHP/5.4.43
Content-Language: auto
Set-Cookie: [...]
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Api-Content-Length: 234
ea


Warning:  Fatal error, unknown preferences key: ENABLE_API_ACCESS (owner: 2) in /srv/www/tt-rss.faui2k9.de/root/classes/db/prefs.php on line 108

 "seq":0,"status":1,"content": "error":"API_DISABLED" 
0
  

Solution for fixing the Android app (and the logspam on the way as well) seems to be to reset the preferences and then configure tt-rss again (In the webapp, not in the android thing!). Also silences tt-rss update_daemon as well, yay! One last thing: someone out there who wants to explain to me how to fix

Fatal error: Query INSERT INTO ttrss_enclosures
                                                        (content_url, content_type, title, duration, post_id, width, height) VALUES
                                                        ('https://2.gravatar.com/avatar/e6d6ceb7764252af8da058e30cd8cb5f?s=96&d=identicon&r=G', '', '', '', '0', 0, 0) failed: ERROR:  insert or update on table "ttrss_enclosures" violates foreign key constraint "ttrss_enclosures_post_id_fkey"
DETAIL:  Key (post_id)=(0) is not present in table "ttrss_entries". in /srv/www/tt-rss.faui2k9.de/root/classes/db/pgsql.php on line 46
  

TinyTiny-RSS has some nice failure modes. And upstream support forums aren't really helpfull so when you search for your current problem, chances are that there is one mention of it on the web, in the forum, and the only thing happening there is people making fun of the reporter. Anyway. This installation has seen lots of error messages from the updater in the last several months:

Warning: Fatal error, unknown preferences key: ALLOW_DUPLICATE_POSTS (owner: 3) in /srv/www/tt-rss.faui2k9.de/root/classes/db/prefs.php on line 108
Warning: Fatal error, unknown preferences key: ALLOW_DUPLICATE_POSTS (owner: 3) in /srv/www/tt-rss.faui2k9.de/root/classes/db/prefs.php on line 108
Warning: Fatal error, unknown preferences key: ALLOW_DUPLICATE_POSTS (owner: 3) in /srv/www/tt-rss.faui2k9.de/root/classes/db/prefs.php on line 108
Warning: Fatal error, unknown preferences key: ALLOW_DUPLICATE_POSTS (owner: 3) in /srv/www/tt-rss.faui2k9.de/root/classes/db/prefs.php on line 108
  

And, more recently, the android app stopped working with ERROR:JSON Parse failed.. Turns out both things are related. First thing I noticed was changing preferences in the web panel stopped working until you use the reset to Defaults option and then changed settings. Plugging wireshark in between showed what was going on (Note: API was displayed as enabled in Preferences/Preferences):

HTTP/1.1 200 OK
Server: nginx/1.8.0
Date: Thu, 06 Aug 2015 11:00:31 GMT
Content-Type: text/json
Transfer-Encoding: chunked
Connection: keep-alive
X-Powered-By: PHP/5.4.43
Content-Language: auto
Set-Cookie: [...]
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Api-Content-Length: 234
ea


Warning:  Fatal error, unknown preferences key: ENABLE_API_ACCESS (owner: 2) in /srv/www/tt-rss.faui2k9.de/root/classes/db/prefs.php on line 108

 "seq":0,"status":1,"content": "error":"API_DISABLED" 
0
  

Solution for fixing the Android app (and the logspam on the way as well) seems to be to reset the preferences and then configure tt-rss again (In the webapp, not in the android thing!). Also silences tt-rss update_daemon as well, yay! One last thing: someone out there who wants to explain to me how to fix

Fatal error: Query INSERT INTO ttrss_enclosures
                                                        (content_url, content_type, title, duration, post_id, width, height) VALUES
                                                        ('https://2.gravatar.com/avatar/e6d6ceb7764252af8da058e30cd8cb5f?s=96&d=identicon&r=G', '', '', '', '0', 0, 0) failed: ERROR:  insert or update on table "ttrss_enclosures" violates foreign key constraint "ttrss_enclosures_post_id_fkey"
DETAIL:  Key (post_id)=(0) is not present in table "ttrss_entries". in /srv/www/tt-rss.faui2k9.de/root/classes/db/pgsql.php on line 46
  

TinyTiny-RSS has some nice failure modes. And upstream support forums aren't really helpfull so when you search for your current problem, chances are that there is one mention of it on the web, in the forum, and the only thing happening there is people making fun of the reporter. Anyway. This installation has seen lots of error messages from the updater in the last several months:

Warning: Fatal error, unknown preferences key: ALLOW_DUPLICATE_POSTS (owner: 3) in /srv/www/tt-rss.faui2k9.de/root/classes/db/prefs.php on line 108
Warning: Fatal error, unknown preferences key: ALLOW_DUPLICATE_POSTS (owner: 3) in /srv/www/tt-rss.faui2k9.de/root/classes/db/prefs.php on line 108
Warning: Fatal error, unknown preferences key: ALLOW_DUPLICATE_POSTS (owner: 3) in /srv/www/tt-rss.faui2k9.de/root/classes/db/prefs.php on line 108
Warning: Fatal error, unknown preferences key: ALLOW_DUPLICATE_POSTS (owner: 3) in /srv/www/tt-rss.faui2k9.de/root/classes/db/prefs.php on line 108
  

And, more recently, the android app stopped working with ERROR:JSON Parse failed.. Turns out both things are related. First thing I noticed was changing preferences in the web panel stopped working until you use the reset to Defaults option and then changed settings. Plugging wireshark in between showed what was going on (Note: API was displayed as enabled in Preferences/Preferences):

HTTP/1.1 200 OK
Server: nginx/1.8.0
Date: Thu, 06 Aug 2015 11:00:31 GMT
Content-Type: text/json
Transfer-Encoding: chunked
Connection: keep-alive
X-Powered-By: PHP/5.4.43
Content-Language: auto
Set-Cookie: [...]
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Api-Content-Length: 234
ea


Warning:  Fatal error, unknown preferences key: ENABLE_API_ACCESS (owner: 2) in /srv/www/tt-rss.faui2k9.de/root/classes/db/prefs.php on line 108

 "seq":0,"status":1,"content": "error":"API_DISABLED" 
0
  

Solution for fixing the Android app (and the logspam on the way as well) seems to be to reset the preferences and then configure tt-rss again (In the webapp, not in the android thing!). Also silences tt-rss update_daemon as well, yay! One last thing: someone out there who wants to explain to me how to fix

Fatal error: Query INSERT INTO ttrss_enclosures
                                                        (content_url, content_type, title, duration, post_id, width, height) VALUES
                                                        ('https://2.gravatar.com/avatar/e6d6ceb7764252af8da058e30cd8cb5f?s=96&d=identicon&r=G', '', '', '', '0', 0, 0) failed: ERROR:  insert or update on table "ttrss_enclosures" violates foreign key constraint "ttrss_enclosures_post_id_fkey"
DETAIL:  Key (post_id)=(0) is not present in table "ttrss_entries". in /srv/www/tt-rss.faui2k9.de/root/classes/db/pgsql.php on line 46
  

Mostly a mental note as I've reinvented this the second time now. If you just quickly want to share some org-mode notes with some non-Emacs-users the built-in HTML export comes handy. However it has one Problem: All source syntax highlighting is derived from your current theme. Which of course is a bad idea if your editor has a dark background (say Emacs.reverseVideo: on). The same if your terminal's background color is dark. Running Emacs in batch mode and still getting colorful code formatting seems to be an unsolved problem. All that can be found on the Internet suggests adding a dark background to your HTML export (at least to the code blocks). Or maybe use an external style-sheet. Both not exactly the thing if you just want to scp snippets of HTML somewhere to share. However there's a working hack:

#!/usr/bin/make -f
%.html: %.org
	xvfb-run urxvt +rv -e emacs -nw --visit $< --funcall org-html-export-to-html --kill >/dev/null

So use a terminal you can easily force into light-background-mode (like urxvt +rv) so the emacs -nw runs in light-background-mode and wrap the thing in xvfb-run so you can properly run this over ssh (and don't get annoying windows pop up and disappear again when typing make)