Search Results: "karl"

17 November 2023

Jonathan Dowland: denver luna

picture of the denver luna record on a turntable
I haven't done one of these in a while! Denver Luna is the latest single from Underworld, here on a pink 12" vinyl. The notable thing about this release was it was preceded by an "acapella" mix, consisting of just Karl Hyde's vocals: albeit treated and layered. Personally I prefer the "main" single mix, which calls back to their biggest hits. The vinyl also features an instrumental take, which is currently unavailable in any other formats. The previous single (presumably both from a forthcoming album) was and the colour red. In this crazy world we live in, this was limited to 1,000 copies. Flippers have sold 4 on eBay already, at between 55 and 75.

16 November 2022

Antoine Beaupr : Wayland: i3 to Sway migration

I started migrating my graphical workstations to Wayland, specifically migrating from i3 to Sway. This is mostly to address serious graphics bugs in the latest Framwork laptop, but also something I felt was inevitable. The current status is that I've been able to convert my i3 configuration to Sway, and adapt my systemd startup sequence to the new environment. Screen sharing only works with Pipewire, so I also did that migration, which basically requires an upgrade to Debian bookworm to get a nice enough Pipewire release. I'm testing Wayland on my laptop, but I'm not using it as a daily driver because I first need to upgrade to Debian bookworm on my main workstation. Most irritants have been solved one way or the other. My main problem with Wayland right now is that I spent a frigging week doing the conversion: it's exciting and new, but it basically sucked the life out of all my other projects and it's distracting, and I want it to stop. The rest of this page documents why I made the switch, how it happened, and what's left to do. Hopefully it will keep you from spending as much time as I did in fixing this. TL;DR: Wayland is mostly ready. Main blockers you might find are that you need to do manual configurations, DisplayLink (multiple monitors on a single cable) doesn't work in Sway, HDR and color management are still in development. I had to install the following packages:
apt install \
    brightnessctl \
    foot \
    gammastep \
    gdm3 \
    grim slurp \
    pipewire-pulse \
    sway \
    swayidle \
    swaylock \
    wdisplays \
    wev \
    wireplumber \
    wlr-randr \
    xdg-desktop-portal-wlr
And did some of tweaks in my $HOME, mostly dealing with my esoteric systemd startup sequence, which you won't have to deal with if you are not a fan.

Why switch? I originally held back from migrating to Wayland: it seemed like a complicated endeavor hardly worth the cost. It also didn't seem actually ready. But after reading this blurb on LWN, I decided to at least document the situation here. The actual quote that convinced me it might be worth it was:
It s amazing. I have never experienced gaming on Linux that looked this smooth in my life.
... I'm not a gamer, but I do care about latency. The longer version is worth a read as well. The point here is not to bash one side or the other, or even do a thorough comparison. I start with the premise that Xorg is likely going away in the future and that I will need to adapt some day. In fact, the last major Xorg release (21.1, October 2021) is rumored to be the last ("just like the previous release...", that said, minor releases are still coming out, e.g. 21.1.4). Indeed, it seems even core Xorg people have moved on to developing Wayland, or at least Xwayland, which was spun off it its own source tree. X, or at least Xorg, in in maintenance mode and has been for years. Granted, the X Window System is getting close to forty years old at this point: it got us amazingly far for something that was designed around the time the first graphical interface. Since Mac and (especially?) Windows released theirs, they have rebuilt their graphical backends numerous times, but UNIX derivatives have stuck on Xorg this entire time, which is a testament to the design and reliability of X. (Or our incapacity at developing meaningful architectural change across the entire ecosystem, take your pick I guess.) What pushed me over the edge is that I had some pretty bad driver crashes with Xorg while screen sharing under Firefox, in Debian bookworm (around November 2022). The symptom would be that the UI would completely crash, reverting to a text-only console, while Firefox would keep running, audio and everything still working. People could still see my screen, but I couldn't, of course, let alone interact with it. All processes still running, including Xorg. (And no, sorry, I haven't reported that bug, maybe I should have, and it's actually possible it comes up again in Wayland, of course. But at first, screen sharing didn't work of course, so it's coming a much further way. After making screen sharing work, though, the bug didn't occur again, so I consider this a Xorg-specific problem until further notice.) There were also frustrating glitches in the UI, in general. I actually had to setup a compositor alongside i3 to make things bearable at all. Video playback in a window was laggy, sluggish, and out of sync. Wayland fixed all of this.

Wayland equivalents This section documents each tool I have picked as an alternative to the current Xorg tool I am using for the task at hand. It also touches on other alternatives and how the tool was configured. Note that this list is based on the series of tools I use in desktop. TODO: update desktop with the following when done, possibly moving old configs to a ?xorg archive.

Window manager: i3 sway This seems like kind of a no-brainer. Sway is around, it's feature-complete, and it's in Debian. I'm a bit worried about the "Drew DeVault community", to be honest. There's a certain aggressiveness in the community I don't like so much; at least an open hostility towards more modern UNIX tools like containers and systemd that make it hard to do my work while interacting with that community. I'm also concern about the lack of unit tests and user manual for Sway. The i3 window manager has been designed by a fellow (ex-)Debian developer I have a lot of respect for (Michael Stapelberg), partly because of i3 itself, but also working with him on other projects. Beyond the characters, i3 has a user guide, a code of conduct, and lots more documentation. It has a test suite. Sway has... manual pages, with the homepage just telling users to use man -k sway to find what they need. I don't think we need that kind of elitism in our communities, to put this bluntly. But let's put that aside: Sway is still a no-brainer. It's the easiest thing to migrate to, because it's mostly compatible with i3. I had to immediately fix those resources to get a minimal session going:
i3 Sway note
set_from_resources set no support for X resources, naturally
new_window pixel 1 default_border pixel 1 actually supported in i3 as well
That's it. All of the other changes I had to do (and there were actually a lot) were all Wayland-specific changes, not Sway-specific changes. For example, use brightnessctl instead of xbacklight to change the backlight levels. See a copy of my full sway/config for details. Other options include:
  • dwl: tiling, minimalist, dwm for Wayland, not in Debian
  • Hyprland: tiling, fancy animations, not in Debian
  • Qtile: tiling, extensible, in Python, not in Debian (1015267)
  • river: Zig, stackable, tagging, not in Debian (1006593)
  • velox: inspired by xmonad and dwm, not in Debian
  • vivarium: inspired by xmonad, not in Debian

Status bar: py3status waybar I have invested quite a bit of effort in setting up my status bar with py3status. It supports Sway directly, and did not actually require any change when migrating to Wayland. Unfortunately, I had trouble making nm-applet work. Based on this nm-applet.service, I found that you need to pass --indicator for it to show up at all. In theory, tray icon support was merged in 1.5, but in practice there are still several limitations, like icons not clickable. Also, on startup, nm-applet --indicator triggers this error in the Sway logs:
nov 11 22:34:12 angela sway[298938]: 00:49:42.325 [INFO] [swaybar/tray/host.c:24] Registering Status Notifier Item ':1.47/org/ayatana/NotificationItem/nm_applet'
nov 11 22:34:12 angela sway[298938]: 00:49:42.327 [ERROR] [swaybar/tray/item.c:127] :1.47/org/ayatana/NotificationItem/nm_applet IconPixmap: No such property  IconPixmap 
nov 11 22:34:12 angela sway[298938]: 00:49:42.327 [ERROR] [swaybar/tray/item.c:127] :1.47/org/ayatana/NotificationItem/nm_applet AttentionIconPixmap: No such property  AttentionIconPixmap 
nov 11 22:34:12 angela sway[298938]: 00:49:42.327 [ERROR] [swaybar/tray/item.c:127] :1.47/org/ayatana/NotificationItem/nm_applet ItemIsMenu: No such property  ItemIsMenu 
nov 11 22:36:10 angela sway[313419]: info: fcft.c:838: /usr/share/fonts/truetype/dejavu/DejaVuSans.ttf: size=24.00pt/32px, dpi=96.00
... but that seems innocuous. The tray icon displays but is not clickable. Note that there is currently (November 2022) a pull request to hook up a "Tray D-Bus Menu" which, according to Reddit might fix this, or at least be somewhat relevant. If you don't see the icon, check the bar.tray_output property in the Sway config, try: tray_output *. The non-working tray was the biggest irritant in my migration. I have used nmtui to connect to new Wifi hotspots or change connection settings, but that doesn't support actions like "turn off WiFi". I eventually fixed this by switching from py3status to waybar, which was another yak horde shaving session, but ultimately, it worked.

Web browser: Firefox Firefox has had support for Wayland for a while now, with the team enabling it by default in nightlies around January 2022. It's actually not easy to figure out the state of the port, the meta bug report is still open and it's huge: it currently (Sept 2022) depends on 76 open bugs, it was opened twelve (2010) years ago, and it's still getting daily updates (mostly linking to other tickets). Firefox 106 presumably shipped with "Better screen sharing for Windows and Linux Wayland users", but I couldn't quite figure out what those were. TL;DR: echo MOZ_ENABLE_WAYLAND=1 >> ~/.config/environment.d/firefox.conf && apt install xdg-desktop-portal-wlr

How to enable it Firefox depends on this silly variable to start correctly under Wayland (otherwise it starts inside Xwayland and looks fuzzy and fails to screen share):
MOZ_ENABLE_WAYLAND=1 firefox
To make the change permanent, many recipes recommend adding this to an environment startup script:
if [ "$XDG_SESSION_TYPE" == "wayland" ]; then
    export MOZ_ENABLE_WAYLAND=1
fi
At least that's the theory. In practice, Sway doesn't actually run any startup shell script, so that can't possibly work. Furthermore, XDG_SESSION_TYPE is not actually set when starting Sway from gdm3 which I find really confusing, and I'm not the only one. So the above trick doesn't actually work, even if the environment (XDG_SESSION_TYPE) is set correctly, because we don't have conditionals in environment.d(5). (Note that systemd.environment-generator(7) do support running arbitrary commands to generate environment, but for some some do not support user-specific configuration files... Even then it may be a solution to have a conditional MOZ_ENABLE_WAYLAND environment, but I'm not sure it would work because ordering between those two isn't clear: maybe the XDG_SESSION_TYPE wouldn't be set just yet...) At first, I made this ridiculous script to workaround those issues. Really, it seems to me Firefox should just parse the XDG_SESSION_TYPE variable here... but then I realized that Firefox works fine in Xorg when the MOZ_ENABLE_WAYLAND is set. So now I just set that variable in environment.d and It Just Works :
MOZ_ENABLE_WAYLAND=1

Screen sharing Out of the box, screen sharing doesn't work until you install xdg-desktop-portal-wlr or similar (e.g. xdg-desktop-portal-gnome on GNOME). I had to reboot for the change to take effect. Without those tools, it shows the usual permission prompt with "Use operating system settings" as the only choice, but when we accept... nothing happens. After installing the portals, it actualyl works, and works well! This was tested in Debian bookworm/testing with Firefox ESR 102 and Firefox 106. Major caveat: we can only share a full screen, we can't currently share just a window. The major upside to that is that, by default, it streams only one output which is actually what I want most of the time! See the screencast compatibility for more information on what is supposed to work. This is actually a huge improvement over the situation in Xorg, where Firefox can only share a window or all monitors, which led me to use Chromium a lot for video-conferencing. With this change, in other words, I will not need Chromium for anything anymore, whoohoo! If slurp, wofi, or bemenu are installed, one of them will be used to pick the monitor to share, which effectively acts as some minimal security measure. See xdg-desktop-portal-wlr(1) for how to configure that.

Side note: Chrome fails to share a full screen I was still using Google Chrome (or, more accurately, Debian's Chromium package) for some videoconferencing. It's mainly because Chromium was the only browser which will allow me to share only one of my two monitors, which is extremely useful. To start chrome with the Wayland backend, you need to use:
chromium  -enable-features=UseOzonePlatform -ozone-platform=wayland
If it shows an ugly gray border, check the Use system title bar and borders setting. It can do some screensharing. Sharing a window and a tab seems to work, but sharing a full screen doesn't: it's all black. Maybe not ready for prime time. And since Firefox can do what I need under Wayland now, I will not need to fight with Chromium to work under Wayland:
apt purge chromium
Note that a similar fix was necessary for Signal Desktop, see this commit. Basically you need to figure out a way to pass those same flags to signal:
--enable-features=WaylandWindowDecorations --ozone-platform-hint=auto

Email: notmuch See Emacs, below.

File manager: thunar Unchanged.

News: feed2exec, gnus See Email, above, or Emacs in Editor, below.

Editor: Emacs okay-ish Emacs is being actively ported to Wayland. According to this LWN article, the first (partial, to Cairo) port was done in 2014 and a working port (to GTK3) was completed in 2021, but wasn't merged until late 2021. That is: after Emacs 28 was released (April 2022). So we'll probably need to wait for Emacs 29 to have native Wayland support in Emacs, which, in turn, is unlikely to arrive in time for the Debian bookworm freeze. There are, however, unofficial builds for both Emacs 28 and 29 provided by spwhitton which may provide native Wayland support. I tested the snapshot packages and they do not quite work well enough. First off, they completely take over the builtin Emacs they hijack the $PATH in /etc! and certain things are simply not working in my setup. For example, this hook never gets ran on startup:
(add-hook 'after-init-hook 'server-start t) 
Still, like many X11 applications, Emacs mostly works fine under Xwayland. The clipboard works as expected, for example. Scaling is a bit of an issue: fonts look fuzzy. I have heard anecdotal evidence of hard lockups with Emacs running under Xwayland as well, but haven't experienced any problem so far. I did experience a Wayland crash with the snapshot version however. TODO: look again at Wayland in Emacs 29.

Backups: borg Mostly irrelevant, as I do not use a GUI.

Color theme: srcery, redshift gammastep I am keeping Srcery as a color theme, in general. Redshift is another story: it has no support for Wayland out of the box, but it's apparently possible to apply a hack on the TTY before starting Wayland, with:
redshift -m drm -PO 3000
This tip is from the arch wiki which also has other suggestions for Wayland-based alternatives. Both KDE and GNOME have their own "red shifters", and for wlroots-based compositors, they (currently, Sept. 2022) list the following alternatives: I configured gammastep with a simple gammastep.service file associated with the sway-session.target.

Display manager: lightdm gdm3 Switched because lightdm failed to start sway:
nov 16 16:41:43 angela sway[843121]: 00:00:00.002 [ERROR] [wlr] [libseat] [common/terminal.c:162] Could not open target tty: Permission denied
Possible alternatives:

Terminal: xterm foot One of the biggest question mark in this transition was what to do about Xterm. After writing two articles about terminal emulators as a professional journalist, decades of working on the terminal, and probably using dozens of different terminal emulators, I'm still not happy with any of them. This is such a big topic that I actually have an entire blog post specifically about this. For starters, using xterm under Xwayland works well enough, although the font scaling makes things look a bit too fuzzy. I have also tried foot: it ... just works! Fonts are much crisper than Xterm and Emacs. URLs are not clickable but the URL selector (control-shift-u) is just plain awesome (think "vimperator" for the terminal). There's cool hack to jump between prompts. Copy-paste works. True colors work. The word-wrapping is excellent: it doesn't lose one byte. Emojis are nicely sized and colored. Font resize works. There's even scroll back search (control-shift-r). Foot went from a question mark to being a reason to switch to Wayland, just for this little goodie, which says a lot about the quality of that software. The selection clicks are a not quite what I would expect though. In rxvt and others, you have the following patterns:
  • single click: reset selection, or drag to select
  • double: select word
  • triple: select quotes or line
  • quadruple: select line
I particularly find the "select quotes" bit useful. It seems like foot just supports double and triple clicks, with word and line selected. You can select a rectangle with control,. It correctly extends the selection word-wise with right click if double-click was first used. One major problem with Foot is that it's a new terminal, with its own termcap entry. Support for foot was added to ncurses in the 20210731 release, which was shipped after the current Debian stable release (Debian bullseye, which ships 6.2+20201114-2). A workaround for this problem is to install the foot-terminfo package on the remote host, which is available in Debian stable. This should eventually resolve itself, as Debian bookworm has a newer version. Note that some corrections were also shipped in the 20211113 release, but that is also shipped in Debian bookworm. That said, I am almost certain I will have to revert back to xterm under Xwayland at some point in the future. Back when I was using GNOME Terminal, it would mostly work for everything until I had to use the serial console on a (HP ProCurve) network switch, which have a fancy TUI that was basically unusable there. I fully expect such problems with foot, or any other terminal than xterm, for that matter. The foot wiki has good troubleshooting instructions as well. Update: I did find one tiny thing to improve with foot, and it's the default logging level which I found pretty verbose. After discussing it with the maintainer on IRC, I submitted this patch to tweak it, which I described like this on Mastodon:
today's reason why i will go to hell when i die (TRWIWGTHWID?): a 600-word, 63 lines commit log for a one line change: https://codeberg.org/dnkl/foot/pulls/1215
It's Friday.

Launcher: rofi rofi?? rofi does not support Wayland. There was a rather disgraceful battle in the pull request that led to the creation of a fork (lbonn/rofi), so it's unclear how that will turn out. Given how relatively trivial problem space is, there is of course a profusion of options:
Tool In Debian Notes
alfred yes general launcher/assistant tool
bemenu yes, bookworm+ inspired by dmenu
cerebro no Javascript ... uh... thing
dmenu-wl no fork of dmenu, straight port to Wayland
Fuzzel ITP 982140 dmenu/drun replacement, app icon overlay
gmenu no drun replacement, with app icons
kickoff no dmenu/run replacement, fuzzy search, "snappy", history, copy-paste, Rust
krunner yes KDE's runner
mauncher no dmenu/drun replacement, math
nwg-launchers no dmenu/drun replacement, JSON config, app icons, nwg-shell project
Onagre no rofi/alfred inspired, multiple plugins, Rust
menu no dmenu/drun rewrite
Rofi (lbonn's fork) no see above
sirula no .desktop based app launcher
Ulauncher ITP 949358 generic launcher like Onagre/rofi/alfred, might be overkill
tofi yes, bookworm+ dmenu/drun replacement, C
wmenu no fork of dmenu-wl, but mostly a rewrite
Wofi yes dmenu/drun replacement, not actively maintained
yofi no dmenu/drun replacement, Rust
The above list comes partly from https://arewewaylandyet.com/ and awesome-wayland. It is likely incomplete. I have read some good things about bemenu, fuzzel, and wofi. A particularly tricky option is that my rofi password management depends on xdotool for some operations. At first, I thought this was just going to be (thankfully?) impossible, because we actually like the idea that one app cannot send keystrokes to another. But it seems there are actually alternatives to this, like wtype or ydotool, the latter which requires root access. wl-ime-type does that through the input-method-unstable-v2 protocol (sample emoji picker, but is not packaged in Debian. As it turns out, wtype just works as expected, and fixing this was basically a two-line patch. Another alternative, not in Debian, is wofi-pass. The other problem is that I actually heavily modified rofi. I use "modis" which are not actually implemented in wofi or tofi, so I'm left with reinventing those wheels from scratch or using the rofi + wayland fork... It's really too bad that fork isn't being reintegrated... For now, I'm actually still using rofi under Xwayland. The main downside is that fonts are fuzzy, but it otherwise just works. Note that wlogout could be a partial replacement (just for the "power menu").

Image viewers: geeqie ? I'm not very happy with geeqie in the first place, and I suspect the Wayland switch will just make add impossible things on top of the things I already find irritating (Geeqie doesn't support copy-pasting images). In practice, Geeqie doesn't seem to work so well under Wayland. The fonts are fuzzy and the thumbnail preview just doesn't work anymore (filed as Debian bug 1024092). It seems it also has problems with scaling. Alternatives: See also this list and that list for other list of image viewers, not necessarily ported to Wayland. TODO: pick an alternative to geeqie, nomacs would be gorgeous if it wouldn't be basically abandoned upstream (no release since 2020), has an unpatched CVE-2020-23884 since July 2020, does bad vendoring, and is in bad shape in Debian (4 minor releases behind). So for now I'm still grumpily using Geeqie.

Media player: mpv, gmpc / sublime This is basically unchanged. mpv seems to work fine under Wayland, better than Xorg on my new laptop (as mentioned in the introduction), and that before the version which improves Wayland support significantly, by bringing native Pipewire support and DMA-BUF support. gmpc is more of a problem, mainly because it is abandoned. See 2022-08-22-gmpc-alternatives for the full discussion, one of the alternatives there will likely support Wayland. Finally, I might just switch to sublime-music instead... In any case, not many changes here, thankfully.

Screensaver: xsecurelock swaylock I was previously using xss-lock and xsecurelock as a screensaver, with xscreensaver "hacks" as a backend for xsecurelock. The basic screensaver in Sway seems to be built with swayidle and swaylock. It's interesting because it's the same "split" design as xss-lock and xsecurelock. That, unfortunately, does not include the fancy "hacks" provided by xscreensaver, and that is unlikely to be implemented upstream. Other alternatives include gtklock and waylock (zig), which do not solve that problem either. It looks like swaylock-plugin, a swaylock fork, which at least attempts to solve this problem, although not directly using the real xscreensaver hacks. swaylock-effects is another attempt at this, but it only adds more effects, it doesn't delegate the image display. Other than that, maybe it's time to just let go of those funky animations and just let swaylock do it's thing, which is display a static image or just a black screen, which is fine by me. In the end, I am just using swayidle with a configuration based on the systemd integration wiki page but with additional tweaks from this service, see the resulting swayidle.service file. Interestingly, damjan also has a service for swaylock itself, although it's not clear to me what its purpose is...

Screenshot: maim grim, pubpaste I'm a heavy user of maim (and a package uploader in Debian). It looks like the direct replacement to maim (and slop) is grim (and slurp). There's also swappy which goes on top of grim and allows preview/edit of the resulting image, nice touch (not in Debian though). See also awesome-wayland screenshots for other alternatives: there are many, including X11 tools like Flameshot that also support Wayland. One key problem here was that I have my own screenshot / pastebin software which will needed an update for Wayland as well. That, thankfully, meant actually cleaning up a lot of horrible code that involved calling xterm and xmessage for user interaction. Now, pubpaste uses GTK for prompts and looks much better. (And before anyone freaks out, I already had to use GTK for proper clipboard support, so this isn't much of a stretch...)

Screen recorder: simplescreenrecorder wf-recorder In Xorg, I have used both peek or simplescreenrecorder for screen recordings. The former will work in Wayland, but has no sound support. The latter has a fork with Wayland support but it is limited and buggy ("doesn't support recording area selection and has issues with multiple screens"). It looks like wf-recorder will just do everything correctly out of the box, including audio support (with --audio, duh). It's also packaged in Debian. One has to wonder how this works while keeping the "between app security" that Wayland promises, however... Would installing such a program make my system less secure? Many other options are available, see the awesome Wayland screencasting list.

RSI: workrave nothing? Workrave has no support for Wayland. activity watch is a time tracker alternative, but is not a RSI watcher. KDE has rsiwatcher, but that's a bit too much on the heavy side for my taste. SafeEyes looks like an alternative at first, but it has many issues under Wayland (escape doesn't work, idle doesn't work, it just doesn't work really). timekpr-next could be an alternative as well, and has support for Wayland. I am also considering just abandoning workrave, even if I stick with Xorg, because it apparently introduces significant latency in the input pipeline. And besides, I've developed a pretty unhealthy alert fatigue with Workrave. I have used the program for so long that my fingers know exactly where to click to dismiss those warnings very effectively. It makes my work just more irritating, and doesn't fix the fundamental problem I have with computers.

Other apps This is a constantly changing list, of course. There's a bit of a "death by a thousand cuts" in migrating to Wayland because you realize how many things you were using are tightly bound to X.
  • .Xresources - just say goodbye to that old resource system, it was used, in my case, only for rofi, xterm, and ... Xboard!?
  • keyboard layout switcher: built-in to Sway since 2017 (PR 1505, 1.5rc2+), requires a small configuration change, see this answer as well, looks something like this command:
     swaymsg input 0:0:X11_keyboard xkb_layout de
    
    or using this config:
     input *  
         xkb_layout "ca,us"
         xkb_options "grp:sclk_toggle"
      
    
    That works refreshingly well, even better than in Xorg, I must say. swaykbdd is an alternative that supports per-window layouts (in Debian).
  • wallpaper: currently using feh, will need a replacement, TODO: figure out something that does, like feh, a random shuffle. swaybg just loads a single image, duh. oguri might be a solution, but unmaintained, used here, not in Debian. wallutils is another option, also not in Debian. For now I just don't have a wallpaper, the background is a solid gray, which is better than Xorg's default (which is whatever crap was left around a buffer by the previous collection of programs, basically)
  • notifications: currently using dunst in some places, which works well in both Xorg and Wayland, not a blocker, salut a possible alternative (not in Debian), damjan uses mako. TODO: install dunst everywhere
  • notification area: I had trouble making nm-applet work. based on this nm-applet.service, I found that you need to pass --indicator. In theory, tray icon support was merged in 1.5, but in practice there are still several limitations, like icons not clickable. On startup, nm-applet --indicator triggers this error in the Sway logs:
     nov 11 22:34:12 angela sway[298938]: 00:49:42.325 [INFO] [swaybar/tray/host.c:24] Registering Status Notifier Item ':1.47/org/ayatana/NotificationItem/nm_applet'
     nov 11 22:34:12 angela sway[298938]: 00:49:42.327 [ERROR] [swaybar/tray/item.c:127] :1.47/org/ayatana/NotificationItem/nm_applet IconPixmap: No such property  IconPixmap 
     nov 11 22:34:12 angela sway[298938]: 00:49:42.327 [ERROR] [swaybar/tray/item.c:127] :1.47/org/ayatana/NotificationItem/nm_applet AttentionIconPixmap: No such property  AttentionIconPixmap 
     nov 11 22:34:12 angela sway[298938]: 00:49:42.327 [ERROR] [swaybar/tray/item.c:127] :1.47/org/ayatana/NotificationItem/nm_applet ItemIsMenu: No such property  ItemIsMenu 
     nov 11 22:36:10 angela sway[313419]: info: fcft.c:838: /usr/share/fonts/truetype/dejavu/DejaVuSans.ttf: size=24.00pt/32px, dpi=96.00
    
    ... but it seems innocuous. The tray icon displays but, as stated above, is not clickable. If you don't see the icon, check the bar.tray_output property in the Sway config, try: tray_output *. Note that there is currently (November 2022) a pull request to hook up a "Tray D-Bus Menu" which, according to Reddit might fix this, or at least be somewhat relevant. This was the biggest irritant in my migration. I have used nmtui to connect to new Wifi hotspots or change connection settings, but that doesn't support actions like "turn off WiFi". I eventually fixed this by switching from py3status to waybar.
  • window switcher: in i3 I was using this bespoke i3-focus script, which doesn't work under Sway, swayr an option, not in Debian. So I put together this other bespoke hack from multiple sources, which works.
  • PDF viewer: currently using atril (which supports Wayland), could also just switch to zatura/mupdf permanently, see also calibre for a discussion on document viewers
See also this list of useful addons and this other list for other app alternatives.

More X11 / Wayland equivalents For all the tools above, it's not exactly clear what options exist in Wayland, or when they do, which one should be used. But for some basic tools, it seems the options are actually quite clear. If that's the case, they should be listed here:
X11 Wayland In Debian
arandr wdisplays yes
autorandr kanshi yes
xdotool wtype yes
xev wev yes
xlsclients swaymsg -t get_tree yes
xrandr wlr-randr yes
lswt is a more direct replacement for xlsclients but is not packaged in Debian. See also: Note that arandr and autorandr are not directly part of X. arewewaylandyet.com refers to a few alternatives. We suggest wdisplays and kanshi above (see also this service file) but wallutils can also do the autorandr stuff, apparently, and nwg-displays can do the arandr part. Neither are packaged in Debian yet. So I have tried wdisplays and it Just Works, and well. The UI even looks better and more usable than arandr, so another clean win from Wayland here. TODO: test kanshi as a autorandr replacement

Other issues

systemd integration I've had trouble getting session startup to work. This is partly because I had a kind of funky system to start my session in the first place. I used to have my whole session started from .xsession like this:
#!/bin/sh
. ~/.shenv
systemctl --user import-environment
exec systemctl --user start --wait xsession.target
But obviously, the xsession.target is not started by the Sway session. It seems to just start a default.target, which is really not what we want because we want to associate the services directly with the graphical-session.target, so that they don't start when logging in over (say) SSH. damjan on #debian-systemd showed me his sway-setup which features systemd integration. It involves starting a different session in a completely new .desktop file. That work was submitted upstream but refused on the grounds that "I'd rather not give a preference to any particular init system." Another PR was abandoned because "restarting sway does not makes sense: that kills everything". The work was therefore moved to the wiki. So. Not a great situation. The upstream wiki systemd integration suggests starting the systemd target from within Sway, which has all sorts of problems:
  • you don't get Sway logs anywhere
  • control groups are all messed up
I have done a lot of work trying to figure this out, but I remember that starting systemd from Sway didn't actually work for me: my previously configured systemd units didn't correctly start, and especially not with the right $PATH and environment. So I went down that rabbit hole and managed to correctly configure Sway to be started from the systemd --user session. I have partly followed the wiki but also picked ideas from damjan's sway-setup and xdbob's sway-services. Another option is uwsm (not in Debian). This is the config I have in .config/systemd/user/: I have also configured those services, but that's somewhat optional: You will also need at least part of my sway/config, which sends the systemd notification (because, no, Sway doesn't support any sort of readiness notification, that would be too easy). And you might like to see my swayidle-config while you're there. Finally, you need to hook this up somehow to the login manager. This is typically done with a desktop file, so drop sway-session.desktop in /usr/share/wayland-sessions and sway-user-service somewhere in your $PATH (typically /usr/bin/sway-user-service). The session then looks something like this:
$ systemd-cgls   head -101
Control group /:
-.slice
 user.slice (#472)
    user.invocation_id: bc405c6341de4e93a545bde6d7abbeec
    trusted.invocation_id: bc405c6341de4e93a545bde6d7abbeec
   user-1000.slice (#10072)
      user.invocation_id: 08f40f5c4bcd4fd6adfd27bec24e4827
      trusted.invocation_id: 08f40f5c4bcd4fd6adfd27bec24e4827
     user@1000.service   (#10156)
        user.delegate: 1
        trusted.delegate: 1
        user.invocation_id: 76bed72a1ffb41dca9bfda7bb174ef6b
        trusted.invocation_id: 76bed72a1ffb41dca9bfda7bb174ef6b
       session.slice (#10282)
         xdg-document-portal.service (#12248)
           9533 /usr/libexec/xdg-document-portal
           9542 fusermount3 -o rw,nosuid,nodev,fsname=portal,auto_unmount,subt 
         xdg-desktop-portal.service (#12211)
           9529 /usr/libexec/xdg-desktop-portal
         pipewire-pulse.service (#10778)
           6002 /usr/bin/pipewire-pulse
         wireplumber.service (#10519)
           5944 /usr/bin/wireplumber
         gvfs-daemon.service (#10667)
           5960 /usr/libexec/gvfsd
         gvfs-udisks2-volume-monitor.service (#10852)
           6021 /usr/libexec/gvfs-udisks2-volume-monitor
         at-spi-dbus-bus.service (#11481)
           6210 /usr/libexec/at-spi-bus-launcher
           6216 /usr/bin/dbus-daemon --config-file=/usr/share/defaults/at-spi2 
           6450 /usr/libexec/at-spi2-registryd --use-gnome-session
         pipewire.service (#10403)
           5940 /usr/bin/pipewire
         dbus.service (#10593)
           5946 /usr/bin/dbus-daemon --session --address=systemd: --nofork --n 
       background.slice (#10324)
         tracker-miner-fs-3.service (#10741)
           6001 /usr/libexec/tracker-miner-fs-3
       app.slice (#10240)
         xdg-permission-store.service (#12285)
           9536 /usr/libexec/xdg-permission-store
         gammastep.service (#11370)
           6197 gammastep
         dunst.service (#11958)
           7460 /usr/bin/dunst
         wterminal.service (#13980)
           69100 foot --title pop-up
           69101 /bin/bash
           77660 sudo systemd-cgls
           77661 head -101
           77662 wl-copy
           77663 sudo systemd-cgls
           77664 systemd-cgls
         syncthing.service (#11995)
           7529 /usr/bin/syncthing -no-browser -no-restart -logflags=0 --verbo 
           7537 /usr/bin/syncthing -no-browser -no-restart -logflags=0 --verbo 
         dconf.service (#10704)
           5967 /usr/libexec/dconf-service
         gnome-keyring-daemon.service (#10630)
           5951 /usr/bin/gnome-keyring-daemon --foreground --components=pkcs11 
         gcr-ssh-agent.service (#10963)
           6035 /usr/libexec/gcr-ssh-agent /run/user/1000/gcr
         swayidle.service (#11444)
           6199 /usr/bin/swayidle -w
         nm-applet.service (#11407)
           6198 /usr/bin/nm-applet --indicator
         wcolortaillog.service (#11518)
           6226 foot colortaillog
           6228 /bin/sh /home/anarcat/bin/colortaillog
           6230 sudo journalctl -f
           6233 ccze -m ansi
           6235 sudo journalctl -f
           6236 journalctl -f
         afuse.service (#10889)
           6051 /usr/bin/afuse -o mount_template=sshfs -o transform_symlinks - 
         gpg-agent.service (#13547)
           51662 /usr/bin/gpg-agent --supervised
           51719 scdaemon --multi-server
         emacs.service (#10926)
            6034 /usr/bin/emacs --fg-daemon
           33203 /usr/bin/aspell -a -m -d en --encoding=utf-8
         xdg-desktop-portal-gtk.service (#12322)
           9546 /usr/libexec/xdg-desktop-portal-gtk
         xdg-desktop-portal-wlr.service (#12359)
           9555 /usr/libexec/xdg-desktop-portal-wlr
         sway.service (#11037)
           6037 /usr/bin/sway
           6181 swaybar -b bar-0
           6209 py3status
           6309 /usr/bin/i3status -c /tmp/py3status_oy4ntfnq
           6969 Xwayland :0 -rootless -terminate -core -listen 29 -listen 30 - 
       init.scope (#10198)
         5909 /lib/systemd/systemd --user
         5911 (sd-pam)
     session-7.scope (#10440)
       5895 gdm-session-worker [pam/gdm-password]
       6028 /usr/libexec/gdm-wayland-session --register-session sway-user-serv 
[...]
I think that's pretty neat.

Environment propagation At first, my terminals and rofi didn't have the right $PATH, which broke a lot of my workflow. It's hard to tell exactly how Wayland gets started or where to inject environment. This discussion suggests a few alternatives and this Debian bug report discusses this issue as well. I eventually picked environment.d(5) since I already manage my user session with systemd, and it fixes a bunch of other problems. I used to have a .shenv that I had to manually source everywhere. The only problem with that approach is that it doesn't support conditionals, but that's something that's rarely needed.

Pipewire This is a whole topic onto itself, but migrating to Wayland also involves using Pipewire if you want screen sharing to work. You can actually keep using Pulseaudio for audio, that said, but that migration is actually something I've wanted to do anyways: Pipewire's design seems much better than Pulseaudio, as it folds in JACK features which allows for pretty neat tricks. (Which I should probably show in a separate post, because this one is getting rather long.) I first tried this migration in Debian bullseye, and it didn't work very well. Ardour would fail to export tracks and I would get into weird situations where streams would just drop mid-way. A particularly funny incident is when I was in a meeting and I couldn't hear my colleagues speak anymore (but they could) and I went on blabbering on my own for a solid 5 minutes until I realized what was going on. By then, people had tried numerous ways of letting me know that something was off, including (apparently) coughing, saying "hello?", chat messages, IRC, and so on, until they just gave up and left. I suspect that was also a Pipewire bug, but it could also have been that I muted the tab by error, as I recently learned that clicking on the little tiny speaker icon on a tab mutes that tab. Since the tab itself can get pretty small when you have lots of them, it's actually quite frequently that I mistakenly mute tabs. Anyways. Point is: I already knew how to make the migration, and I had already documented how to make the change in Puppet. It's basically:
apt install pipewire pipewire-audio-client-libraries pipewire-pulse wireplumber 
Then, as a regular user:
systemctl --user daemon-reload
systemctl --user --now disable pulseaudio.service pulseaudio.socket
systemctl --user --now enable pipewire pipewire-pulse
systemctl --user mask pulseaudio
An optional (but key, IMHO) configuration you should also make is to "switch on connect", which will make your Bluetooth or USB headset automatically be the default route for audio, when connected. In ~/.config/pipewire/pipewire-pulse.conf.d/autoconnect.conf:
context.exec = [
      path = "pactl"        args = "load-module module-always-sink"  
      path = "pactl"        args = "load-module module-switch-on-connect"  
    #  path = "/usr/bin/sh"  args = "~/.config/pipewire/default.pw"  
]
See the excellent as usual Arch wiki page about Pipewire for that trick and more information about Pipewire. Note that you must not put the file in ~/.config/pipewire/pipewire.conf (or pipewire-pulse.conf, maybe) directly, as that will break your setup. If you want to add to that file, first copy the template from /usr/share/pipewire/pipewire-pulse.conf first. So far I'm happy with Pipewire in bookworm, but I've heard mixed reports from it. I have high hopes it will become the standard media server for Linux in the coming months or years, which is great because I've been (rather boldly, I admit) on the record saying I don't like PulseAudio. Rereading this now, I feel it might have been a little unfair, as "over-engineered and tries to do too many things at once" applies probably even more to Pipewire than PulseAudio (since it also handles video dispatching). That said, I think Pipewire took the right approach by implementing existing interfaces like Pulseaudio and JACK. That way we're not adding a third (or fourth?) way of doing audio in Linux; we're just making the server better.

Keypress drops Sometimes I lose keyboard presses. This correlates with the following warning from Sway:
d c 06 10:36:31 curie sway[343384]: 23:32:14.034 [ERROR] [wlr] [libinput] event5  - SONiX USB Keyboard: client bug: event processing lagging behind by 37ms, your system is too slow 
... and corresponds to an open bug report in Sway. It seems the "system is too slow" should really be "your compositor is too slow" which seems to be the case here on this older system (curie). It doesn't happen often, but it does happen, particularly when a bunch of busy processes start in parallel (in my case: a linter running inside a container and notmuch new). The proposed fix for this in Sway is to gain real time privileges and add the CAP_SYS_NICE capability to the binary. We'll see how that goes in Debian once 1.8 gets released and shipped.

Improvements over i3

Tiling improvements There's a lot of improvements Sway could bring over using plain i3. There are pretty neat auto-tilers that could replicate the configurations I used to have in Xmonad or Awesome, see:

Display latency tweaks TODO: You can tweak the display latency in wlroots compositors with the max_render_time parameter, possibly getting lower latency than X11 in the end.

Sound/brightness changes notifications TODO: Avizo can display a pop-up to give feedback on volume and brightness changes. Not in Debian. Other alternatives include SwayOSD and sway-nc, also not in Debian.

Debugging tricks The xeyes (in the x11-apps package) will run in Wayland, and can actually be used to easily see if a given window is also in Wayland. If the "eyes" follow the cursor, the app is actually running in xwayland, so not natively in Wayland. Another way to see what is using Wayland in Sway is with the command:
swaymsg -t get_tree

Other documentation

Conclusion In general, this took me a long time, but it mostly works. The tray icon situation is pretty frustrating, but there's a workaround and I have high hopes it will eventually fix itself. I'm also actually worried about the DisplayLink support because I eventually want to be using this, but hopefully that's another thing that will hopefully fix itself before I need it.

A word on the security model I'm kind of worried about all the hacks that have been added to Wayland just to make things work. Pretty much everywhere we need to, we punched a hole in the security model: Wikipedia describes the security properties of Wayland as it "isolates the input and output of every window, achieving confidentiality, integrity and availability for both." I'm not sure those are actually realized in the actual implementation, because of all those holes punched in the design, at least in Sway. For example, apparently the GNOME compositor doesn't have the virtual-keyboard protocol, but they do have (another?!) text input protocol. Wayland does offer a better basis to implement such a system, however. It feels like the Linux applications security model lacks critical decision points in the UI, like the user approving "yes, this application can share my screen now". Applications themselves might have some of those prompts, but it's not mandatory, and that is worrisome.

20 April 2022

Russell Coker: Android Without Play

A while ago I was given a few reasonably high-end Android phones to give away. I gave two very nice phones to someone who looks after refugees so a couple of refugee families could make video calls to relatives. The third phone is a Huawei Nova 7i [1] which doesn t have the Google Play Store. The Nova 7i is a ridiculously powerful computer (8G of RAM in a phone!!!) but without the Google Play Store it s not much use to the average phone user. It has the HuaWei App Gallery which isn t as bad as most of the proprietary app stores of small players in the Android world, it has SnapChat, TikTok, Telegram, Alibaba, WeChat, and Grays auction (an app I didn t even know existed) along with many others. It also links to ApkPure (apparently a 3rd party app installer that obtains APK files for major commercial apps) for Facebook among others. The ApkPure thing might be Huawei outsourcing the violation of Facebook terms of service. For the moment I ve decided to only use free software on this phone and use my old phone for non-free stuff (Facebook, LinkedIn, etc). The eventual aim is that I can only carry a phone with free software for normal use and carry a second phone if I m active on LinkedIn or something. My recollection is that when I first got the phone (almost 2 years ago) it didn t have such a range of apps. The first thing to install was f-droid [2] as the app repository. F-droid has a repository of thousands of free software Android apps as well as some apps that are slightly less free which are tagged appropriately. You can install the F-Droid app from the web site. As an aside I had to go to settings and enable force old index format to get the list of packages, I don t know why as other phones had worked without it. Here are the F-Droid apps I installed: Future Plans The current main things I m missing are a calendar, a contact list, and a shared note taking system (like Google Keep). For calendaring and a contact list the CalDAV and CardDAV protocols seem best. The most common implementation on the server side appears to be DAViCal [5]. The Nextcloud system supports CalDAV, CardDAV, web editing of notes and documents (including LibreOffice if you install that plugin) [6]. But it is huge and demands write access to all it s own code (bad for security), and it s not packaged for Debian. Also in my tests it gave me an error 401 when I tried to authenticate to it from the Android Nextcloud client. I ve seen a positive review about Radicale, a simple CalDAV and CardDAV server that doesn t need a database [7]. I prefer the Unix philosophy of keeping things simple with file storage unless there s a real need for anything else. I don t think that anything I ever do with calendaring will require the PostgreSQL database that DAViCal uses. I ll give Radicale a go for CalDAV and CardDAV, but I still need something for shared notes (shopping lists etc). Suggestions welcome. Current Status Lack of a contacts list is a major loss of functionality in a phone. I could store contacts in the phone memory or on the SIM, but I would still have to get all my old contacts in there and also getting something half working reduces motivation for getting it working properly. Lack of a calendar is also a problem, again I could work around that by exporting all my Google calendars as iCal URLs but I d rather get it working correctly. The lack of shared notes may be a harder problem to solve given the failure of Nextcloud. For that I would consider just having the keep.google.com web site always open in Mozilla at least in the short term. At the moment I require two phones, my new Android phone without Google and the old one for my contacts list etc. Hopefully in a week or so I ll have my new phone doing contacts, calendaring, and notes. Then my old phone will just be for proprietary apps which I don t need most of the time and I can leave it at home when I don t need that sort of thing.

7 February 2021

Chris Lamb: Favourite books of 2020

I won't reveal precisely how many books I read in 2020, but it was definitely an improvement on 74 in 2019, 53 in 2018 and 50 in 2017. But not only did I read more in a quantitative sense, the quality seemed higher as well. There were certainly fewer disappointments: given its cultural resonance, I was nonplussed by Nick Hornby's Fever Pitch and whilst Ian Fleming's The Man with the Golden Gun was a little thin (again, given the obvious influence of the Bond franchise) the booked lacked 'thinness' in a way that made it interesting to critique. The weakest novel I read this year was probably J. M. Berger's Optimal, but even this hybrid of Ready Player One late-period Black Mirror wasn't that cringeworthy, all things considered. Alas, graphic novels continue to not quite be my thing, I'm afraid. I perhaps experienced more disappointments in the non-fiction section. Paul Bloom's Against Empathy was frustrating, particularly in that it expended unnecessary energy battling its misleading title and accepted terminology, and it could so easily have been an 20-minute video essay instead). (Elsewhere in the social sciences, David and Goliath will likely be the last Malcolm Gladwell book I voluntarily read.) After so many positive citations, I was also more than a little underwhelmed by Shoshana Zuboff's The Age of Surveillance Capitalism, and after Ryan Holiday's many engaging reboots of Stoic philosophy, his Conspiracy (on Peter Thiel and Hulk Hogan taking on Gawker) was slightly wide of the mark for me. Anyway, here follows a selection of my favourites from 2020, in no particular order:

Fiction Wolf Hall & Bring Up the Bodies & The Mirror and the Light Hilary Mantel During the early weeks of 2020, I re-read the first two parts of Hilary Mantel's Thomas Cromwell trilogy in time for the March release of The Mirror and the Light. I had actually spent the last few years eagerly following any news of the final instalment, feigning outrage whenever Mantel appeared to be spending time on other projects. Wolf Hall turned out to be an even better book than I remembered, and when The Mirror and the Light finally landed at midnight on 5th March, I began in earnest the next morning. Note that date carefully; this was early 2020, and the book swiftly became something of a heavy-handed allegory about the world at the time. That is to say and without claiming that I am Monsieur Cromuel in any meaningful sense it was an uneasy experience to be reading about a man whose confident grasp on his world, friends and life was slipping beyond his control, and at least in Cromwell's case, was heading inexorably towards its denouement. The final instalment in Mantel's trilogy is not perfect, and despite my love of her writing I would concur with the judges who decided against awarding her a third Booker Prize. For instance, there is something of the longueur that readers dislike in the second novel, although this might not be entirely Mantel's fault after all, the rise of the "ugly" Anne of Cleves and laborious trade negotiations for an uninspiring mineral (this is no Herbertian 'spice') will never match the court intrigues of Anne Boleyn, Jane Seymour and that man for all seasons, Thomas More. Still, I am already looking forward to returning to the verbal sparring between King Henry and Cromwell when I read the entire trilogy once again, tentatively planned for 2022.

The Fault in Our Stars John Green I came across John Green's The Fault in Our Stars via a fantastic video by Lindsay Ellis discussing Roland Barthes famous 1967 essay on authorial intent. However, I might have eventually come across The Fault in Our Stars regardless, not because of Green's status as an internet celebrity of sorts but because I'm a complete sucker for this kind of emotionally-manipulative bildungsroman, likely due to reading Philip Pullman's His Dark Materials a few too many times in my teens. Although its title is taken from Shakespeare's Julius Caesar, The Fault in Our Stars is actually more Romeo & Juliet. Hazel, a 16-year-old cancer patient falls in love with Gus, an equally ill teen from her cancer support group. Hazel and Gus share the same acerbic (and distinctly unteenage) wit and a love of books, centred around Hazel's obsession of An Imperial Affliction, a novel by the meta-fictional author Peter Van Houten. Through a kind of American version of Jim'll Fix It, Gus and Hazel go and visit Van Houten in Amsterdam. I'm afraid it's even cheesier than I'm describing it. Yet just as there is a time and a place for Michelin stars and Haribo Starmix, there's surely a place for this kind of well-constructed but altogether maudlin literature. One test for emotionally manipulative works like this is how well it can mask its internal contradictions while Green's story focuses on the universalities of love, fate and the shortness of life (as do almost all of his works, it seems), The Fault in Our Stars manages to hide, for example, that this is an exceedingly favourable treatment of terminal illness that is only possible for the better off. The 2014 film adaptation does somewhat worse in peddling this fantasy (and has a much weaker treatment of the relationship between the teens' parents too, an underappreciated subtlety of the book). The novel, however, is pretty slick stuff, and it is difficult to fault it for what it is. For some comparison, I later read Green's Looking for Alaska and Paper Towns which, as I mention, tug at many of the same strings, but they don't come together nearly as well as The Fault in Our Stars. James Joyce claimed that "sentimentality is unearned emotion", and in this respect, The Fault in Our Stars really does earn it.

The Plague Albert Camus P. D. James' The Children of Men, George Orwell's Nineteen Eighty-Four, Arthur Koestler's Darkness at Noon ... dystopian fiction was already a theme of my reading in 2020, so given world events it was an inevitability that I would end up with Camus's novel about a plague that swept through the Algerian city of Oran. Is The Plague an allegory about the Nazi occupation of France during World War Two? Where are all the female characters? Where are the Arab ones? Since its original publication in 1947, there's been so much written about The Plague that it's hard to say anything new today. Nevertheless, I was taken aback by how well it captured so much of the nuance of 2020. Whilst we were saying just how 'unprecedented' these times were, it was eerie how a novel written in the 1940s could accurately how many of us were feeling well over seventy years on later: the attitudes of the people; the confident declarations from the institutions; the misaligned conversations that led to accidental misunderstandings. The disconnected lovers. The only thing that perhaps did not work for me in The Plague was the 'character' of the church. Although I could appreciate most of the allusion and metaphor, it was difficult for me to relate to the significance of Father Paneloux, particularly regarding his change of view on the doctrinal implications of the virus, and spoiler alert that he finally died of a "doubtful case" of the disease, beyond the idea that Paneloux's beliefs are in themselves "doubtful". Answers on a postcard, perhaps. The Plague even seemed to predict how we, at least speaking of the UK, would react when the waves of the virus waxed and waned as well:
The disease stiffened and carried off three or four patients who were expected to recover. These were the unfortunates of the plague, those whom it killed when hope was high
It somehow captured the nostalgic yearning for high-definition videos of cities and public transport; one character even visits the completely deserted railway station in Oman simply to read the timetables on the wall.

Tinker, Tailor, Soldier, Spy John le Carr There's absolutely none of the Mad Men glamour of James Bond in John le Carr 's icy world of Cold War spies:
Small, podgy, and at best middle-aged, Smiley was by appearance one of London's meek who do not inherit the earth. His legs were short, his gait anything but agile, his dress costly, ill-fitting, and extremely wet.
Almost a direct rebuttal to Ian Fleming's 007, Tinker, Tailor has broken-down cars, bad clothes, women with their own internal and external lives (!), pathetically primitive gadgets, and (contra Mad Men) hangovers that significantly longer than ten minutes. In fact, the main aspect that the mostly excellent 2011 film adaption doesn't really capture is the smoggy and run-down nature of 1970s London this is not your proto-Cool Britannia of Austin Powers or GTA:1969, the city is truly 'gritty' in the sense there is a thin film of dirt and grime on every surface imaginable. Another angle that the film cannot capture well is just how purposefully the novel does not mention the United States. Despite the US obviously being the dominant power, the British vacillate between pretending it doesn't exist or implying its irrelevance to the matter at hand. This is no mistake on Le Carr 's part, as careful readers are rewarded by finding this denial of US hegemony in metaphor throughout --pace Ian Fleming, there is no obvious Felix Leiter to loudly throw money at the problem or a Sheriff Pepper to serve as cartoon racist for the Brits to feel superior about. By contrast, I recall that a clever allusion to "dusty teabags" is subtly mirrored a few paragraphs later with a reference to the installation of a coffee machine in the office, likely symbolic of the omnipresent and unavoidable influence of America. (The officer class convince themselves that coffee is a European import.) Indeed, Le Carr communicates a feeling of being surrounded on all sides by the peeling wallpaper of Empire. Oftentimes, the writing style matches the graceless and inelegance of the world it depicts. The sentences are dense and you find your brain performing a fair amount of mid-flight sentence reconstruction, reparsing clauses, commas and conjunctions to interpret Le Carr 's intended meaning. In fact, in his eulogy-cum-analysis of Le Carr 's writing style, William Boyd, himself a ventrioquilist of Ian Fleming, named this intentional technique 'staccato'. Like the musical term, I suspect the effect of this literary staccato is as much about the impact it makes on a sentence as the imperceptible space it generates after it. Lastly, the large cast in this sprawling novel is completely believable, all the way from the Russian spymaster Karla to minor schoolboy Roach the latter possibly a stand-in for Le Carr himself. I got through the 500-odd pages in just a few days, somehow managing to hold the almost-absurdly complicated plot in my head. This is one of those classic books of the genre that made me wonder why I had not got around to it before.

The Nickel Boys Colson Whitehead According to the judges who awarded it the Pulitzer Prize for Fiction, The Nickel Boys is "a devastating exploration of abuse at a reform school in Jim Crow-era Florida" that serves as a "powerful tale of human perseverance, dignity and redemption". But whilst there is plenty of this perseverance and dignity on display, I found little redemption in this deeply cynical novel. It could almost be read as a follow-up book to Whitehead's popular The Underground Railroad, which itself won the Pulitzer Prize in 2017. Indeed, each book focuses on a young protagonist who might be euphemistically referred to as 'downtrodden'. But The Nickel Boys is not only far darker in tone, it feels much closer and more connected to us today. Perhaps this is unsurprising, given that it is based on the story of the Dozier School in northern Florida which operated for over a century before its long history of institutional abuse and racism was exposed a 2012 investigation. Nevertheless, if you liked the social commentary in The Underground Railroad, then there is much more of that in The Nickel Boys:
Perhaps his life might have veered elsewhere if the US government had opened the country to colored advancement like they opened the army. But it was one thing to allow someone to kill for you and another to let him live next door.
Sardonic aper us of this kind are pretty relentless throughout the book, but it never tips its hand too far into on nihilism, especially when some of the visual metaphors are often first-rate: "An American flag sighed on a pole" is one I can easily recall from memory. In general though, The Nickel Boys is not only more world-weary in tenor than his previous novel, the United States it describes seems almost too beaten down to have the energy conjure up the Swiftian magical realism that prevented The Underground Railroad from being overly lachrymose. Indeed, even we Whitehead transports us a present-day New York City, we can't indulge in another kind of fantasy, the one where America has solved its problems:
The Daily News review described the [Manhattan restaurant] as nouveau Southern, "down-home plates with a twist." What was the twist that it was soul food made by white people?
It might be overly reductionist to connect Whitehead's tonal downshift with the racial justice movements of the past few years, but whatever the reason, we've ended up with a hard-hitting, crushing and frankly excellent book.

True Grit & No Country for Old Men Charles Portis & Cormac McCarthy It's one of the most tedious cliches to claim the book is better than the film, but these two books are of such high quality that even the Coen Brothers at their best cannot transcend them. I'm grouping these books together here though, not because their respective adaptations will exemplify some of the best cinema of the 21st century, but because of their superb treatment of language. Take the use of dialogue. Cormac McCarthy famously does not use any punctuation "I believe in periods, in capitals, in the occasional comma, and that's it" but the conversations in No Country for Old Men together feel familiar and commonplace, despite being relayed through this unconventional technique. In lesser hands, McCarthy's written-out Texan drawl would be the novelistic equivalent of white rap or Jar Jar Binks, but not only is the effect entirely gripping, it helps you to believe you are physically present in the many intimate and domestic conversations that hold this book together. Perhaps the cinematic familiarity helps, as you can almost hear Tommy Lee Jones' voice as Sheriff Bell from the opening page to the last. Charles Portis' True Grit excels in its dialogue too, but in this book it is not so much in how it flows (although that is delightful in its own way) but in how forthright and sardonic Maddie Ross is:
"Earlier tonight I gave some thought to stealing a kiss from you, though you are very young, and sick and unattractive to boot, but now I am of a mind to give you five or six good licks with my belt." "One would be as unpleasant as the other."
Perhaps this should be unsurprising. Maddie, a fourteen-year-old girl from Yell County, Arkansas, can barely fire her father's heavy pistol, so she can only has words to wield as her weapon. Anyway, it's not just me who treasures this book. In her encomium that presages most modern editions, Donna Tartt of The Secret History fame traces the novels origins through Huckleberry Finn, praising its elegance and economy: "The plot of True Grit is uncomplicated and as pure in its way as one of the Canterbury Tales". I've read any Chaucer, but I am inclined to agree. Tartt also recalls that True Grit vanished almost entirely from the public eye after the release of John Wayne's flimsy cinematic vehicle in 1969 this earlier film was, Tartt believes, "good enough, but doesn't do the book justice". As it happens, reading a book with its big screen adaptation as a chaser has been a minor theme of my 2020, including P. D. James' The Children of Men, Kazuo Ishiguro's Never Let Me Go, Patricia Highsmith's Strangers on a Train, James Ellroy's The Black Dahlia, John Green's The Fault in Our Stars, John le Carr 's Tinker, Tailor Soldier, Spy and even a staged production of Charles Dicken's A Christmas Carol streamed from The Old Vic. For an autodidact with no academic background in literature or cinema, I've been finding this an effective and enjoyable means of getting closer to these fine books and films it is precisely where they deviate (or perhaps where they are deficient) that offers a means by which one can see how they were constructed. I've also found that adaptations can also tell you a lot about the culture in which they were made: take the 'straightwashing' in the film version of Strangers on a Train (1951) compared to the original novel, for example. It is certainly true that adaptions rarely (as Tartt put it) "do the book justice", but she might be also right to alight on a legal metaphor, for as the saying goes, to judge a movie in comparison to the book is to do both a disservice.

The Glass Hotel Emily St. John Mandel In The Glass Hotel, Mandel somehow pulls off the impossible; writing a loose roman- -clef on Bernie Madoff, a Ponzi scheme and the ephemeral nature of finance capital that is tranquil and shimmeringly beautiful. Indeed, don't get the wrong idea about the subject matter; this is no over over-caffeinated The Big Short, as The Glass Hotel is less about a Madoff or coked-up financebros but the fragile unreality of the late 2010s, a time which was, as we indeed discovered in 2020, one event away from almost shattering completely. Mandel's prose has that translucent, phantom quality to it where the chapters slip through your fingers when you try to grasp at them, and the plot is like a ghost ship that that slips silently, like the Mary Celeste, onto the Canadian water next to which the eponymous 'Glass Hotel' resides. Indeed, not unlike The Overlook Hotel, the novel so overflows with symbolism so that even the title needs to evoke the idea of impermanence permanently living in a hotel might serve as a house, but it won't provide a home. It's risky to generalise about such things post-2016, but the whole story sits in that the infinitesimally small distance between perception and reality, a self-constructed culture that is not so much 'post truth' but between them. There's something to consider in almost every character too. Take the stand-in for Bernie Madoff: no caricature of Wall Street out of a 1920s political cartoon or Brechtian satire, Jonathan Alkaitis has none of the oleaginous sleaze of a Dominic Strauss-Kahn, the cold sociopathy of a Marcus Halberstam nor the well-exercised sinuses of, say, Jordan Belford. Alkaitis is dare I say it? eminently likeable, and the book is all the better for it. Even the C-level characters have something to say: Enrico, trivially escaping from the regulators (who are pathetically late to the fraud without Mandel ever telling us explicitly), is daydreaming about the girlfriend he abandoned in New York: "He wished he'd realised he loved her before he left". What was in his previous life that prevented him from doing so? Perhaps he was never in love at all, or is love itself just as transient as the imaginary money in all those bank accounts? Maybe he fell in love just as he crossed safely into Mexico? When, precisely, do we fall in love anyway? I went on to read Mandel's Last Night in Montreal, an early work where you can feel her reaching for that other-worldly quality that she so masterfully achieves in The Glass Hotel. Her f ted Station Eleven is on my must-read list for 2021. "What is truth?" asked Pontius Pilate. Not even Mandel cannot give us the answer, but this will certainly do for now.

Running the Light Sam Tallent Although it trades in all of the clich s and stereotypes of the stand-up comedian (the triumvirate of drink, drugs and divorce), Sam Tallent's debut novel depicts an extremely convincing fictional account of a touring road comic. The comedian Doug Stanhope (who himself released a fairly decent No Encore for the Donkey memoir in 2020) hyped Sam's book relentlessly on his podcast during lockdown... and justifiably so. I ripped through Running the Light in a few short hours, the only disappointment being that I can't seem to find videos online of Sam that come anywhere close to match up to his writing style. If you liked the rollercoaster energy of Paul Beatty's The Sellout, the cynicism of George Carlin and the car-crash invertibility of final season Breaking Bad, check this great book out.

Non-fiction Inside Story Martin Amis This was my first introduction to Martin Amis's work after hearing that his "novelised autobiography" contained a fair amount about Christopher Hitchens, an author with whom I had a one of those rather clich d parasocial relationship with in the early days of YouTube. (Hey, it could have been much worse.) Amis calls his book a "novelised autobiography", and just as much has been made of its quasi-fictional nature as the many diversions into didactic writing advice that betwixt each chapter: "Not content with being a novel, this book also wants to tell you how to write novels", complained Tim Adams in The Guardian. I suspect that reviewers who grew up with Martin since his debut book in 1973 rolled their eyes at yet another demonstration of his manifest cleverness, but as my first exposure to Amis's gift of observation, I confess that I was thought it was actually kinda clever. Try, for example, "it remains a maddening truth that both sexual success and sexual failure are steeply self-perpetuating" or "a hospital gym is a contradiction like a young Conservative", etc. Then again, perhaps I was experiencing a form of nostalgia for a pre-Gamergate YouTube, when everything in the world was a lot simpler... or at least things could be solved by articulate gentlemen who honed their art of rhetoric at the Oxford Union. I went on to read Martin's first novel, The Rachel Papers (is it 'arrogance' if you are, indeed, that confident?), as well as his 1997 Night Train. I plan to read more of him in the future.

The Collected Essays, Journalism and Letters: Volume 1 & Volume 2 & Volume 3 & Volume 4 George Orwell These deceptively bulky four volumes contain all of George Orwell's essays, reviews and correspondence, from his teenage letters sent to local newspapers to notes to his literary executor on his deathbed in 1950. Reading this was part of a larger, multi-year project of mine to cover the entirety of his output. By including this here, however, I'm not recommending that you read everything that came out of Orwell's typewriter. The letters to friends and publishers will only be interesting to biographers or hardcore fans (although I would recommend Dorian Lynskey's The Ministry of Truth: A Biography of George Orwell's 1984 first). Furthermore, many of his book reviews will be of little interest today. Still, some insights can be gleaned; if there is any inconsistency in this huge corpus is that his best work is almost 'too' good and too impactful, making his merely-average writing appear like hackwork. There are some gems that don't make the usual essay collections too, and some of Orwell's most astute social commentary came out of series of articles he wrote for the left-leaning newspaper Tribune, related in many ways to the US Jacobin. You can also see some of his most famous ideas start to take shape years if not decades before they appear in his novels in these prototype blog posts. I also read Dennis Glover's novelised account of the writing of Nineteen-Eighty Four called The Last Man in Europe, and I plan to re-read some of Orwell's earlier novels during 2021 too, including A Clergyman's Daughter and his 'antebellum' Coming Up for Air that he wrote just before the Second World War; his most under-rated novel in my estimation. As it happens, and with the exception of the US and Spain, copyright in the works published in his lifetime ends on 1st January 2021. Make of that what you will.

Capitalist Realism & Chavs: The Demonisation of the Working Class Mark Fisher & Owen Jones These two books are not natural companions to one another and there is likely much that Jones and Fisher would vehemently disagree on, but I am pairing these books together here because they represent the best of the 'political' books I read in 2020. Mark Fisher was a dedicated leftist whose first book, Capitalist Realism, marked an important contribution to political philosophy in the UK. However, since his suicide in early 2017, the currency of his writing has markedly risen, and Fisher is now frequently referenced due to his belief that the prevalence of mental health conditions in modern life is a side-effect of various material conditions, rather than a natural or unalterable fact "like weather". (Of course, our 'weather' is being increasingly determined by a combination of politics, economics and petrochemistry than pure randomness.) Still, Fisher wrote on all manner of topics, from the 2012 London Olympics and "weird and eerie" electronic music that yearns for a lost future that will never arrive, possibly prefiguring or influencing the Fallout video game series. Saying that, I suspect Fisher will resonate better with a UK audience more than one across the Atlantic, not necessarily because he was minded to write about the parochial politics and culture of Britain, but because his writing often carries some exasperation at the suppression of class in favour of identity-oriented politics, a viewpoint not entirely prevalent in the United States outside of, say, Tour F. Reed or the late Michael Brooks. (Indeed, Fisher is likely best known in the US as the author of his controversial 2013 essay, Exiting the Vampire Castle, but that does not figure greatly in this book). Regardless, Capitalist Realism is an insightful, damning and deeply unoptimistic book, best enjoyed in the warm sunshine I found it an ironic compliment that I had quoted so many paragraphs that my Kindle's copy protection routines prevented me from clipping any further. Owen Jones needs no introduction to anyone who regularly reads a British newspaper, especially since 2015 where he unofficially served as a proxy and punching bag for expressing frustrations with the then-Labour leader, Jeremy Corbyn. However, as the subtitle of Jones' 2012 book suggests, Chavs attempts to reveal the "demonisation of the working class" in post-financial crisis Britain. Indeed, the timing of the book is central to Jones' analysis, specifically that the stereotype of the "chav" is used by government and the media as a convenient figleaf to avoid meaningful engagement with economic and social problems on an austerity ridden island. (I'm not quite sure what the US equivalent to 'chav' might be. Perhaps Florida Man without the implications of mental health.) Anyway, Jones certainly has a point. From Vicky Pollard to the attacks on Jade Goody, there is an ignorance and prejudice at the heart of the 'chav' backlash, and that would be bad enough even if it was not being co-opted or criminalised for ideological ends. Elsewhere in political science, I also caught Michael Brooks' Against the Web and David Graeber's Bullshit Jobs, although they are not quite methodical enough to recommend here. However, Graeber's award-winning Debt: The First 5000 Years will be read in 2021. Matt Taibbi's Hate Inc: Why Today's Media Makes Us Despise One Another is worth a brief mention here though, but its sprawling nature felt very much like I was reading a set of Substack articles loosely edited together. And, indeed, I was.

The Golden Thread: The Story of Writing Ewan Clayton A recommendation from a dear friend, Ewan Clayton's The Golden Thread is a journey through the long history of the writing from the Dawn of Man to present day. Whether you are a linguist, a graphic designer, a visual artist, a typographer, an archaeologist or 'just' a reader, there is probably something in here for you. I was already dipping my quill into calligraphy this year so I suspect I would have liked this book in any case, but highlights would definitely include the changing role of writing due to the influence of textual forms in the workplace as well as digression on ergonomic desks employed by monks and scribes in the Middle Ages. A lot of books by otherwise-sensible authors overstretch themselves when they write about computers or other technology from the Information Age, at best resulting in bizarre non-sequiturs and dangerously Panglossian viewpoints at worst. But Clayton surprised me by writing extremely cogently and accurate on the role of text in this new and unpredictable era. After finishing it I realised why for a number of years, Clayton was a consultant for the legendary Xerox PARC where he worked in a group focusing on documents and contemporary communications whilst his colleagues were busy inventing the graphical user interface, laser printing, text editors and the computer mouse.

New Dark Age & Radical Technologies: The Design of Everyday Life James Bridle & Adam Greenfield I struggled to describe these two books to friends, so I doubt I will suddenly do a better job here. Allow me to quote from Will Self's review of James Bridle's New Dark Age in the Guardian:
We're accustomed to worrying about AI systems being built that will either "go rogue" and attack us, or succeed us in a bizarre evolution of, um, evolution what we didn't reckon on is the sheer inscrutability of these manufactured minds. And minds is not a misnomer. How else should we think about the neural network Google has built so its translator can model the interrelation of all words in all languages, in a kind of three-dimensional "semantic space"?
New Dark Age also turns its attention to the weird, algorithmically-derived products offered for sale on Amazon as well as the disturbing and abusive videos that are automatically uploaded by bots to YouTube. It should, by rights, be a mess of disparate ideas and concerns, but Bridle has a flair for introducing topics which reveals he comes to computer science from another discipline altogether; indeed, on a four-part series he made for Radio 4, he's primarily referred to as "an artist". Whilst New Dark Age has rather abstract section topics, Adam Greenfield's Radical Technologies is a rather different book altogether. Each chapter dissects one of the so-called 'radical' technologies that condition the choices available to us, asking how do they work, what challenges do they present to us and who ultimately benefits from their adoption. Greenfield takes his scalpel to smartphones, machine learning, cryptocurrencies, artificial intelligence, etc., and I don't think it would be unfair to say that starts and ends with a cynical point of view. He is no reactionary Luddite, though, and this is both informed and extremely well-explained, and it also lacks the lazy, affected and Private Eye-like cynicism of, say, Attack of the 50 Foot Blockchain. The books aren't a natural pair, for Bridle's writing contains quite a bit of air in places, ironically mimics the very 'clouds' he inveighs against. Greenfield's book, by contrast, as little air and much lower pH value. Still, it was more than refreshing to read two technology books that do not limit themselves to platitudinal booleans, be those dangerously naive (e.g. Kevin Kelly's The Inevitable) or relentlessly nihilistic (Shoshana Zuboff's The Age of Surveillance Capitalism). Sure, they are both anti-technology screeds, but they tend to make arguments about systems of power rather than specific companies and avoid being too anti-'Big Tech' through a narrower, Silicon Valley obsessed lens for that (dipping into some other 2020 reading of mine) I might suggest Wendy Liu's Abolish Silicon Valley or Scott Galloway's The Four. Still, both books are superlatively written. In fact, Adam Greenfield has some of the best non-fiction writing around, both in terms of how he can explain complicated concepts (particularly the smart contract mechanism of the Ethereum cryptocurrency) as well as in the extremely finely-crafted sentences I often felt that the writing style almost had no need to be that poetic, and I particularly enjoyed his fictional scenarios at the end of the book.

The Algebra of Happiness & Indistractable: How to Control Your Attention and Choose Your Life Scott Galloway & Nir Eyal A cocktail of insight, informality and abrasiveness makes NYU Professor Scott Galloway uncannily appealing to guys around my age. Although Galloway definitely has his own wisdom and experience, similar to Joe Rogan I suspect that a crucial part of Galloway's appeal is that you feel you are learning right alongside him. Thankfully, 'Prof G' is far less err problematic than Rogan (Galloway is more of a well-meaning, spirited centrist), although he, too, has some pretty awful takes at time. This is a shame, because removed from the whirlwind of social media he can be really quite considered, such as in this long-form interview with Stephanie Ruhle. In fact, it is this kind of sentiment that he captured in his 2019 Algebra of Happiness. When I look over my highlighted sections, it's clear that it's rather schmaltzy out of context ("Things you hate become just inconveniences in the presence of people you love..."), but his one-two punch of cynicism and saccharine ("Ask somebody who purchased a home in 2007 if their 'American Dream' came true...") is weirdly effective, especially when he uses his own family experiences as part of his story:
A better proxy for your life isn't your first home, but your last. Where you draw your last breath is more meaningful, as it's a reflection of your success and, more important, the number of people who care about your well-being. Your first house signals the meaningful your future and possibility. Your last home signals the profound the people who love you. Where you die, and who is around you at the end, is a strong signal of your success or failure in life.
Nir Eyal's Indistractable, however, is a totally different kind of 'self-help' book. The important background story is that Eyal was the author of the widely-read Hooked which turned into a secular Bible of so-called 'addictive design'. (If you've ever been cornered by a techbro wielding a Wikipedia-thin knowledge of B. F. Skinner's behaviourist psychology and how it can get you to click 'Like' more often, it ultimately came from Hooked.) However, Eyal's latest effort is actually an extended mea culpa for his previous sin and he offers both high and low-level palliative advice on how to avoid falling for the tricks he so studiously espoused before. I suppose we should be thankful to capitalism for selling both cause and cure. Speaking of markets, there appears to be a growing appetite for books in this 'anti-distraction' category, and whilst I cannot claim to have done an exhausting study of this nascent field, Indistractable argues its points well without relying on accurate-but-dry "studies show..." or, worse, Gladwellian gotchas. My main criticism, however, would be that Eyal doesn't acknowledge the limits of a self-help approach to this problem; it seems that many of the issues he outlines are an inescapable part of the alienation in modern Western society, and the only way one can really avoid distraction is to move up the income ladder or move out to a 500-acre ranch.

1 July 2020

Utkarsh Gupta: FOSS Activites in June 2020

Here s my (ninth) monthly update about the activities I ve done in the F/L/OSS world.

Debian
This was my 16th month of contributing to Debian. I became a DM in late March last year and a DD last Christmas! \o/ This month was a little intense. I did a lot of different kinds of things in Debian this month. Whilst most of my time went on doing security stuff, I also sponsored a bunch of packages. Here are the following things I did this month:

Uploads and bug fixes:

Other $things:
  • Hosted Ruby team meeting. Logs here.
  • Mentoring for newcomers.
  • FTP Trainee reviewing.
  • Moderation of -project mailing list.
  • Sponsored ruby-ast for Abraham, libexif for Hugh, djangorestframework-gis and karlseguin-ccache for Nilesh, and twig-extensions, twig-i18n-extension, and mariadb-mysql-kbs for William.

GSoC Phase 1, Part 2! Last month, I got selected as a Google Summer of Code student for Debian again! \o/
I am working on the Upstream-Downstream Cooperation in Ruby project. The first half of the first month is blogged here, titled, GSoC Phase 1.
Also, I log daily updates at gsocwithutkarsh2102.tk. Whilst the daily updates are available at the above site^, I ll breakdown the important parts of the later half of the first month here:
  • Documented the first cop, GemspecGit via PR #2.
  • Made an initial release, v0.1.0!
  • Spread the word/usage about this tool/library via adding them in the official RuboCop docs.
  • We had our third weekly meeting where we discussed the next steps and the things that are supposed to be done for the next set of cops.
  • Wrote more tests so as to cover different aspects of the GemspecGit cop.
  • Opened PR #4 for the next Cop, RequireRelativeToLib.
  • Introduced rubocop-packaging to the outer world and requested other upstream projects to use it! It is being used by 6 other projects already
  • Had our fourth weekly meeting where we pair-programmed (and I sucked :P) and figured out a way to make the second cop work.
  • Found a bug, reported at issue #5 and raised PR #6 to fix it.
  • And finally, people loved the library/tool (and it s outcome):



    (for those who don t know, @bbatsov is the author of RuboCop, @lienvdsteen is an amazing fullstack engineer at GitLab, and @pboling is the author of some awesome Ruby tools and libraries!)

Debian LTS
Debian Long Term Support (LTS) is a project to extend the lifetime of all Debian stable releases to (at least) 5 years. Debian LTS is not handled by the Debian security team, but by a separate group of volunteers and companies interested in making it a success. This was my ninth month as a Debian LTS paid contributor. I was assigned 30.00 hours and worked on the following things:

CVE Fixes and Announcements:

Other LTS Work:
  • Triaged sympa, apache2, qemu, and coturn.
  • Add fix for CVE-2020-0198/libexif.
  • Requested CVE for bug#60251 against apache2 and prodded further.
  • Raised issue #947 against sympa reporting an incomplete patch for CVE-2020-10936. More discussions internally.
  • Created the LTS Survey on the self-hosted LimeSurvey instance.
  • Attended the third LTS meeting. Logs here.
  • General discussion on LTS private and public mailing list.

Other(s)
Sometimes it gets hard to categorize work/things into a particular category.
That s why I am writing all of those things inside this category.
This includes two sub-categories and they are as follows.

Personal: This month I did the following things:
  • Wrote and published v0.1.0 of rubocop-packaging on RubyGems!
    It s open-sourced and the repository is here.
    Bug reports and pull requests are welcomed!
  • Integrated a tiny (yet a powerful) hack to align images in markdown for my blog.
    Commit here.
  • Released v0.4.0 of batalert on RubyGems!

Open Source: Again, this contains all the things that I couldn t categorize earlier.
Opened several issues and PRs:
Thank you for sticking along for so long :) Until next time.
:wq for today.

12 May 2020

Jacob Adams: Roman Finger Counting

I recently wrote a final paper on the history of written numerals. In the process, I discovered this fascinating tidbit that didn t really fit in my paper, but I wanted to put it somewhere. So I m writing about it here. If I were to ask you to count as high as you could on your fingers you d probably get up to 10 before running out of fingers. You can t count any higher than the number of fingers you have, right? The Romans could! They used a place-value system, combined with various gestures to count all the way up to 9,999 on two hands.

The System Finger Counting (Note that in this diagram 60 is, in fact, wrong, and this picture swaps the hundreds and the thousands.) We ll start with the units. The last three fingers of the left hand, middle, ring, and pinkie, are used to form them. Zero is formed with an open hand, the opposite of the finger counting we re used to. One is formed by bending the middle joint of the pinkie, two by including the ring finger and three by including the middle finger, all at the middle joint. You ll want to keep all these bends fairly loose, as otherwise these numbers can get quite uncomfortable. For four, you extend your pinkie again, for five, also raise your ring finger, and for six, you raise your middle finger as well, but then lower your ring finger. For seven you bend your pinkie at the bottom joint, for eight adding your ring finger, and for nine, including your middle finger. This mirrors what you did for one, two and three, but bending the finger at the bottom joint now instead. This leaves your thumb and index finger for the tens. For ten, touch the nail of your index finger to the inside of your top thumb joint. For twenty, put your thumb between your index and middle fingers. For thirty, touch the nails of your thumb and index fingers. For forty, bend your index finger slightly towards your palm and place your thumb between the middle and top knuckle of your index finger. For fifty, place your thumb against your palm. For sixty, leave your thumb where it is and wrap your index finger around it (the diagram above is wrong). For seventy, move your thumb so that the nail touches between the middle and top knuckle of your index finger. For eighty, flip your thumb so that the bottom of it now touches the spot between the middle and top knuckle of your index finger. For ninety, touch the nail of your index finger to your bottom thumb joint. The hundreds and thousands use the same positions on the right hand, with the units being the thousands and the tens being the hundreds. One account, from which the picture above comes, swaps these two, but the first account we have uses this ordering. Combining all these symbols, you can count all the way to 9,999 yourself on just two hands. Try it!

History

The Venerable Bede The first written record of this system comes from the Venerable Bede, an English Benedictine monk who died in 735. He wrote De computo vel loquela digitorum, On Calculating and Speaking with the Fingers, as the introduction to a larger work on chronology, De temporum ratione. (The primary calculation done by monks at the time was calculating the date of Easter, the first Sunday after the first full moon of spring). He also includes numbers from 10,000 to 1,000,000, but its unknown if these were inventions of the author and were likely rarely used regardless. They require moving your hands to various positions on your body, as illustrated below, from Jacob Leupold s Theatrum Arilhmetico-Geometricum, published in 1727: Finger Counting with Large Numbers

The Romans If Bede was the first to write it, how do we know that it came from Roman times? It s referenced in many Roman writings, including this bit from the Roman satirist Juvenal who died in 130:
Felix nimirum qui tot per saecula mortem distulit atque suos iam dextera computat annos. Happy is he who so many times over the years has cheated death And now reckons his age on the right hand.
Because of course the right hand is where one counts hundreds! There s also this Roman riddle:
Nunc mihi iam credas fieri quod posse negatur: octo tenes manibus, si me monstrante magistro sublatis septem reliqui tibi sex remanebunt. Now you shall believe what you would deny could be done: In your hands you hold eight, as my teacher once taught; Take away seven, and six still remain.
If you form eight with this system and then remove the symbol for seven, you get the symbol for six!

Sources My source for this blog post is Paul Broneer s 1969 English translation of Karl Menninger s Zahlwort und Ziffer (Number Words and Number Symbols).

8 March 2020

Ulrike Uhlig: Implementing feedback into our work culture

Everywhere I worked in the past, the only feedback that was asked of employees was during a yearly evaluation meeting. These meetings always felt to me like talking to Santa Claus and his Knecht Ruprecht. I was asked: Were you a good employee last year? If yes, we might give you a raise. If no, admit all your mistakes now, even if we already know everything, ho ho ho. And don't you talk about your feelings, or your well-being, or say anything about the organization's (invisible) hierarchies, otherwise we will put you on the "naughty list", and that's it with candy. The yearly evaluation set aside, there was no other place to give feedback (except by escalating a matter by involving the Labour Court, if you happen to work in France, or going on strike, also mostly part of French culture). Feedback allows to reflect on work processes, to situate oneself, and to get closure. How surprised was I when, some years ago, I received an email from a collaborator asking me: kindly for just few paragraphs (doesn t have to be anything long) to hear from you about the process, your work, challenges you had, or anything else you want to mention there.. Wow!
This simple email allowed me to reflect about: How do we get to a feedback culture? How do we get from German Christmas folklore, protestant work ethics, and the deeply rooted principles of disciplining and punishing to a feedback culture on eye level? It sounds a bit like going from the dark ages to a really cool science fiction utopia with universal peace, telepathy, and magic between all sentient beings on all inhabited planets in the cosmos at least that's how I imagined it as a child, just like some of my heroes did: the cosmonaut girl who saves Earth, the boy who talks to space flowers that give him the capacity to fly, and the little onion who fights for justice (the Italian author was so popular on our side of the iron curtain that a soviet astronomer named a minor planet after him. His wife meanwhile immortalized Karl Marx.) and some romantic part of me hangs on to these ideas. Feedback is not always easy to hear and to give. I-Statements Giving and receiving feedback is hard in a culture where people learnt that when they made a mistake they won't get candy. Or that they have to constantly please other people because they are not worthy by themselves. This can lead to people putting mistakes on one another. Every sentence that starts with You are has the potential of creating a lot of hurt, and anger. Have you heard of I-Statements? They have very powerfully changed my world view, as they shift from accusation to ownership of feelings. So instead of telling someone Your writing style is impossible! You really need to change the way you write., with an I-Statement one could say I have a hard time understanding that part of the text. I-Statements make cooperation possible. Listening actively Feedback is not about being right or wrong, it's first of all about being able to see how another person has experienced a situation. Active listening is a tool that helps with understanding. It might seem easy, but needs quite some practice and a safe space. One part of active listening is to restate what you hear the other person say (by mirroring, or paraphrasing), to make sure you understood, and make sure they know you understood what they were trying to say. You can practise this: in a circle of three people, have one person tell how they experienced a (possibly conflictual) situation, have one person do the active listening, and the third person observing in order to give feedback to the active listener about how they did. Then switch roles, for example clockwise, until everyone has had every role. Encouraging continuous feedback A working feedback culture does not take place only once a year. It needs to be a continuous process and therefore implemented in meetings, teams, eventually on the level of a project. Making clear: Who can I talk to if I experience an issue? is not different than telling developers and users where and how they can report a bug, or request a feature. A safe space to express feedback is key. Encouraging multiple feedback channels Some people might feel less empowered or more vulnerable over a channel than others. Make sure to have different channels for receiving feedback such as email, a point on each meeting agenda, a one-to-one meeting, or a poll. Giving and receiving feedback on eye level In a workplace that does not have a working feedback culture, feedback is easily perceived as policing. If your feedback process consists of asking people to upload a form to a cloud server every 3 months, and you notice that some people don't do it, you could ask yourself if there is an issue with how your colleagues perceive giving feedback in your organization. Do you meet your colleagues on eye level when it comes to feedback? Do you take feedback seriously and act on it? How do you deal with unpleasant feedback? How do you react when colleagues don't meet your expectations? Can people participate in the feedback process within their paid work time? Did everybody understand what the feedback process is about? Don't jump to conclusions Humans are problem solving animals. When someone comes to us with a problem, the first thing we want to do is to solve it, to help them. But sometimes this is uncalled for, it can be disempowering, or prevent people from acquiring competences themselves, and it can even break people's boundaries. So instead of asking What can I do for you?, try asking What do you need right now? People will often reply something that you did not expect at all. Acting on feedback Make sure you have a process to collect feedback (possibly anonymized) and to regularly evaluate if the organization needs to implement changes to thrive. Conclusion I stumbled upon Hans-Christian Dany's critique of feedback again recently, therefore I need to make it clear: I'm not interested in improving capitalist work culture by using cybernetic principles of self-regulation through feedback. Instead, I am interested in improving cooperation between people who work either individually or in organizations on eye level. In this framework, I see feedback processes as profoundly anti-capitalist methods to improve cooperation while working towards common good. Implementing these ideas should be doable: there are organizations who provide feedback training for example. This document, initially aiming at people in cooperatives, gives many insights on communication skills and feedback, the agile and UX worlds do feedback "retrospectives". and otherwise I'll have to go and write science fiction stories for children myself.

2 October 2017

Lars Wirzenius: Attracting contributors to a new project

How do you attract contributors to a new free software project? I'm in the very early stages of a new personal project. It is irrelevant for this blog post what the new project actually is. Instead, I am thinking about the following question:
Do I want the project to be mainly for myself, and maybe a handful of others, or do I want to try to make it a more generally useful, possibly even a well-known, popular project? In other words, do I want to just solve a specific problem I have or try to solve it for a large group of people?
If it's a personal project, I'm all set. I can just start writing code. (In fact, I have.) If it's the latter, I'll need to attract contributions from others, and how do I do that? I asked that question on Twitter and Mastodon and got several suggestions. This is a summary of those, with some editorialising from me. I don't know if these things are all correct, or that they're enough to grow a successful, popular project. Karl Foger'l seminal book Producing Open Source Software should also be mentioned.

28 July 2017

Joachim Breitner: How is coinduction the dual of induction?

Earlier today, I demonstrated how to work with coinduction in the theorem provers Isabelle, Coq and Agda, with a very simple example. This reminded me of a discussion I had in Karlsruhe with my then colleague Denis Lohner: If coinduction is the dual of induction, why do the induction principles look so different? I like what we observed there, so I d like to share this. The following is mostly based on my naive understanding of coinduction based on what I observe in the implementation in Isabelle. I am sure that a different, more categorial presentation of datatypes (as initial resp. terminal objects in some category of algebras) makes the duality more obvious, but that does not necessarily help the working Isabelle user who wants to make sense of coninduction.

Inductive lists I will use the usual polymorphic list data type as an example. So on the one hand, we have normal, finite inductive lists:
datatype 'a list = nil   cons (hd : 'a) (tl : "'a list")
with the well-known induction principle that many of my readers know by heart (syntax slightly un-isabellized):
P nil   ( x xs. P xs   P (cons x xs))     xs. P xs

Coinductive lists In contrast, if we define our lists coinductively to get possibly infinite, Haskell-style lists, by writing
codatatype 'a llist = lnil   lcons (hd : 'a)  (tl : "'a llist")
we get the following coinduction principle:
(  xs ys.
    R xs ys'   (xs = lnil) = (ys = lnil)  
               (xs   lnil   ys'   lnil  
	         hd xs = hd ys   R (tl xs) (tl ys)))  
  (  xs ys. R xs ys   xs = ys)
This is less scary that it looks at first. It tell you if you give me a relation R between lists which implies that either both lists are empty or both lists are nonempty, and furthermore if both are non-empty, that they have the same head and tails related by R, then any two lists related by R are actually equal. If you think of the infinte list as a series of states of a computer program, then this is nothing else than a bisimulation. So we have two proof principles, both of which make intuitive sense. But how are they related? They look very different! In one, we have a predicate P, in the other a relation R, to point out just one difference.

Relation induction To see how they are dual to each other, we have to recognize that both these theorems are actually specializations of a more general (co)induction principle. The datatype declaration automatically creates a relator:
rel_list :: ('a   'b   bool)   'a list   'b list   bool
The definition of rel_list R xs ys is that xs and ys have the same shape (i.e. length), and that the corresponding elements are pairwise related by R. You might have defined this relation yourself at some time, and if so, you probably introduced it as an inductive predicate. So it is not surprising that the following induction principle characterizes this relation:
Q nil nil  
( x xs y ys. R x y   Q xs ys   Q (cons x xs) (cons y ys))  
( xs ys   rel_list R xs ys   Q xs ys)
Note how how similar this lemma is in shape to the normal induction for lists above! And indeed, if we choose Q xs ys (P xs xs = ys) and R x y (x = y), then we obtain exactly that. In that sense, the relation induction is a generalization of the normal induction.

Relation coinduction The same observation can be made in the coinductive world. Here, as well, the codatatype declaration introduces a function
rel_llist :: ('a   'b   bool)   'a llist   'b llist   bool
which relates lists of the same shape with related elements only that this one also relates infinite lists, and therefore is a coinductive relation. The corresponding rule for proof by coinduction is not surprising and should remind you of bisimulation, too:
( xs ys.
    R xs ys   (xs = lnil) = (ys = lnil)  
              (xs   lnil   ys   lnil  
	        Q (hd xs) (hd ys)   R (tl xs) (tl ys)))  
(  xs ys   R xs ys   rel_llist Q xs ys)
It is even more obvious that this is a generalization of the standard coinduction principle shown above: Just instantiate Q with equality, which turns rel_llist Q into equality on the lists, and you have the theorem above.

The duality With our induction and coinduction principle generalized to relations, suddenly a duality emerges: If you turn around the implication in the conclusion of one you get the conclusion of the other one. This is an example of cosomething is something with arrows reversed . But what about the premise(s) of the rules? What happens if we turn around the arrow here? Although slighty less immediate, it turns out that they are the same as well. To see that, we start with the premise of the coinduction rule, reverse the implication and then show that to be equivalent to the two premises of the induction rule:
( xs ys.
    R xs ys   (xs = lnil) = (ys = lnil)  
              (xs   lnil   ys   lnil  
	        Q (hd xs) (hd ys)   R (tl xs) (tl ys)))
=   case analysis (the other two cases are vacuously true)  
  ( xs ys.
    xs = lnil   ys = lnil  
    R xs ys   (xs = lnil) = (ys = lnil)  
              (xs   lnil   ys   lnil  
	        Q (hd xs) (hd ys)   R (tl xs) (tl ys)))
  ( xs ys.
    xs   lnil   ys   lnil
    R xs ys   (xs = lnil) = (ys = lnil)  
              (xs   lnil   ys   lnil  
	        Q (hd xs) (hd ys)   R (tl xs) (tl ys)))
=   simplification  
  ( xs ys.  xs = lnil   ys = lnil   R xs ys
  ( x xs y ys.  R (cons x xs) (cons y ys)   (Q x y   R xs ys))
=   more rewriting  
  R nil nil
  ( x xs y ys. Q x y   R xs ys   R (cons x xs) (cons y ys))

Conclusion The coinduction rule is not the direct dual of the induction rule, but both are specializations of more general, relational proof methods, where the duality is clearly present. More generally, this little excursion shows that it is often beneficial to think of types less as sets, and more as relations this way of thinking is surprisingly fruitful, and led to proofs of parametricity and free theorems and other nice things.

28 February 2017

Chris Lamb: Free software activities in February 2017

Here is my monthly update covering what I have been doing in the free software world (previous month):
Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users. The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced either maliciously or accidentally during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. (I have been awarded a grant from the Core Infrastructure Initiative to fund my work in this area.) This month I:
I also made the following changes to our tooling:
diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.

  • New features:
    • Add a machine-readable JSON output format. (Closes: #850791).
    • Add an --exclude option. (Closes: #854783).
    • Show results from debugging packages last. (Closes: #820427).
    • Extract archive members using an auto-incrementing integer avoiding the need to sanitise filenames. (Closes: #854723).
    • Apply --max-report-size to --text output. (Closes: #851147).
    • Specify <html lang="en"> in the HTML output. (re. #849411).
  • Bug fixes:
    • Fix errors when comparing directories with non-directories. (Closes: #835641).
    • Device and RPM fallback comparisons require xxd. (Closes: #854593).
    • Fix tests that call xxd on Debian Jessie due to change of output format. (Closes: #855239).
    • Add missing Recommends for comparators. (Closes: #854655).
    • Importing submodules (ie. parent.child) will attempt to import parent. (Closes: #854670).
    • Correct logic of module_exists ensuring we correctly skip the debian.deb822 tests when python3-debian is not installed. (Closes: #854745).
    • Clean all temporary files in the signal handler thread instead of attempting to pass the exception back to the main thread. (Closes: #852013).
    • Fix behaviour of setting report maximums to zero (ie. no limit).
  • Optimisations:
    • Don't uselessly run xxd(1) on non-directories.
    • No need to track libarchive directory locations.
    • Optimise create_limited_print_func.
  • Tests:
    • When comparing two empty directories, ensure that the mtime of the directory is consistent to avoid non-deterministic failures.
    • Ensure we can at least import the "deb_fallback" and "rpm_fallback" modules.
    • Add test for symlink differing in destination.
    • Add tests for --progress, --status-fd and profiling output options as well as the Deb Changes,Buildinfo,Dsc and RPM fallback comparisons.
    • Add get_data and @skip_unless_module_exists test helpers.
    • Mark impossible-to-reach code to improve test coverage.

buildinfo.debian.net

buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.

  • Drop raw_text fields now as we've moved these to Amazon S3.
  • Drop storage of Installed-Build-Depends and subsequently-orphaned Binary package instances to recover diskspace.

strip-nondeterminism

strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.

  • Print log entry when fixing a file. (Closes: #777239).
  • Run our entire testsuite in autopkgtests, not just the first test. (Closes: #852517).
  • Don't test for stat(2)'s blksize and block attributes. (Closes: #854937).
  • Use error() from Dh_Lib.pm over "manual" die().


Debian
Debian LTS

This month I have been paid to work 13 hours on Debian Long Term Support (LTS). In that time I did the following:
  • "Frontdesk" duties, triaging CVEs, etc.
  • Issued DLA 817-1 for libphp-phpmailer, correcting a local file disclosure vulnerability where insufficient parsing of HTML messages could potentially be used by attacker to read a local file.
  • Issued DLA 826-1 for wireshark which fixes a denial of service vulnerability in wireshark, where a malformed NATO Ground Moving Target Indicator Format ("STANAG 4607") capture file could cause a memory exhausion/infinite loop.

Uploads
  • python-django (1:1.11~beta1-1) New upstream beta release.
  • redis (3:3.2.8-1) New upstream release.
  • gunicorn (19.6.0-11) Use $ misc:Pre-Depends to populate Pre-Depends for dpkg-maintscript-helper.
  • dh-virtualenv (1.0-1~bpo8+1) Upload to jessie-backports.

I sponsored the following uploads: I also performed the following QA uploads:
  • dh-kpatches (0.99.36+nmu4) Make kernel kernel builds reproducible.
Finally, I made the following non-maintainer uploads:
  • cpio (2.12+dfsg-3) Remove rmt.8.gz to prevent a piuparts error.
  • dot-forward (1:0.71-2.2) Correct a FTBFS; we don't install anything to /usr/sbin, so use GNU Make's $(wildcard ..) over the shell's own * expansion.


FTP Team

As a Debian FTP assistant I ACCEPTed 116 packages: autobahn-cpp, automat, bglibs, bitlbee, bmusb, bullet, case, certspotter, checkit-tiff, dash-el, dash-functional-el, debian-reference, el-x, elisp-bug-hunter, emacs-git-messenger, emacs-which-key, examl, genwqe-user, giac, golang-github-cloudflare-cfssl, golang-github-docker-goamz, golang-github-docker-libnetwork, golang-github-go-openapi-spec, golang-github-google-certificate-transparency, golang-github-karlseguin-ccache, golang-github-karlseguin-expect, golang-github-nebulouslabs-bolt, gpiozero, gsequencer, jel, libconfig-mvp-slicer-perl, libcrush, libdist-zilla-config-slicer-perl, libdist-zilla-role-pluginbundle-pluginremover-perl, libevent, libfunction-parameters-perl, libopenshot, libpod-weaver-section-generatesection-perl, libpodofo, libprelude, libprotocol-http2-perl, libscout, libsmali-1-java, libtest-abortable-perl, linux, linux-grsec, linux-signed, lockdown, lrslib, lua-curses, lua-torch-cutorch, mariadb-10.1, mini-buildd, mkchromecast, mocker-el, node-arr-exclude, node-brorand, node-buffer-xor, node-caller, node-duplexer3, node-ieee754, node-is-finite, node-lowercase-keys, node-minimalistic-assert, node-os-browserify, node-p-finally, node-parse-ms, node-plur, node-prepend-http, node-safe-buffer, node-text-table, node-time-zone, node-tty-browserify, node-widest-line, npd6, openoverlayrouter, pandoc-citeproc-preamble, pydenticon, pyicloud, pyroute2, pytest-qt, pytest-xvfb, python-biomaj3, python-canonicaljson, python-cgcloud, python-gffutils, python-h5netcdf, python-imageio, python-kaptan, python-libtmux, python-pybedtools, python-pyflow, python-scrapy, python-scrapy-djangoitem, python-signedjson, python-unpaddedbase64, python-xarray, qcumber, r-cran-urltools, radiant, repo, rmlint, ruby-googleauth, ruby-os, shutilwhich, sia, six, slimit, sphinx-celery, subuser, swarmkit, tmuxp, tpm2-tools, vine, wala & x265. I additionally filed 8 RC bugs against packages that had incomplete debian/copyright files against: checkit-tiff, dash-el, dash-functional-el, libcrush, libopenshot, mkchromecast, pytest-qt & x265.

6 June 2016

Norbert Preining: TeX Live 2016 released

After long months of testing and waiting for the DVD production, we have released TeX Live 2016 today! texlive2016 Detailed changes can be found here, the most important ones are: CTAN mirrors are working on getting the latest releases, but in a day or two all the servers should be updated. Thanks to all the developers, package writers, documentation writers, testers, and especially Karl Berry for his perfect organization. Now get the champagne and write some nice documents!

20 April 2016

Norbert Preining: GnuPG notes: subkeys, yubikey, gpg1 vs gpg2

Switching from one GnuPG master key to the usage of subkeys was long on my list of things I wanted to do, but never came around. With the advent of a YubiKey NEO in my pocket I finally took the plunge: reading through lots of web pages (and adding one here for confusion), trying to understand the procedures, and above all, understanding my own requirements! gpg-subkeys-yubi To sum up a long story, it was worth the plunge, and all over the security level of my working environment has improved considerable. While the advantages of subkeys are well documented (e.g., Debian Wiki), at the end of the day I was like probably many Debian Developers having one master key that was used for every action: mail decryption and signing, signing of uploads, etc. Traveling a lot I always felt uncomfortable. Despite a lengthy passphrase, I still didn t want my master key to get into wrong hands in case the laptop got stolen. Furthermore, I had my master key on several computers (work, laptop, mail server), which didn t help a lot either. With all this, I started to compile a list of requirements/objectives I wanted to have: Warning Before we start a word of caution make backups, best is to make backups at every stage. You don t want that an erroneous operations wipes out your precious keys without a backup! Preparation In the following I will assume that MASTERKEY environment variable contains the id of the master key to be converted. Furthermore, I have followed some of the advice here, so key ids will be shown in long format. Let us start with the current situation:
$ gpg -K $MASTERKEY
sec   4096R/0x6CACA448860CDC13 2010-09-14
      Key fingerprint = F7D8 A928 26E3 16A1 9FA0  ACF0 6CAC A448 860C DC13
uid                            Norbert Preining <norbert@preining.info>
uid                            Norbert Preining <preining@logic.at>
uid                            Norbert Preining <preining@debian.org>
uid                            Norbert Preining <preining@jaist.ac.jp>
ssb   4096R/0xD1D2BD14810F62B3 2010-09-14
In the following we will go through the following steps: Yubikey SmartCard setup There are several guides out there, but I will in very near future write one about using the NEO for various usage scenaria including GPG keys. Edit the current key The following can be done in one session or in different sessions, the screen logs are after starting with:
$ gpg --expert --edit-key $MASTERKEY
add expiry date Having an expiry date on your key serves two purposes: If you loose it, it will solve itself automatically, and furthermore, you are forced to deal with the key and refresh your gpg knowledge at least once a year. That are two perfect reasons to set expiry to one year. The following log selects each key in turn and sets its expiry date.
$ gpg --expert --edit-key $MASTERKEY
gpg (GnuPG) 1.4.20; Copyright (C) 2015 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
 
Secret key is available.
 
pub  4096R/0x6CACA448860CDC13  created: 2010-09-14  expires: never       usage: SC  
                               trust: ultimate      validity: ultimate
sub  4096R/0xD1D2BD14810F62B3  created: 2010-09-14  expires: never       usage: E   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
 
gpg> expire
Changing expiration time for the primary key.
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0) 1y
Key expires at Mon 06 Feb 2017 08:09:16 PM JST
Is this correct? (y/N) y
 
You need a passphrase to unlock the secret key for
user: "Norbert Preining <norbert@preining.info>"
4096-bit RSA key, ID 0x6CACA448860CDC13, created 2010-09-14
 
Enter passphrase:
 
pub  4096R/0x6CACA448860CDC13  created: 2010-09-14  expires: 2017-02-06  usage: SC  
                               trust: ultimate      validity: ultimate
sub  4096R/0xD1D2BD14810F62B3  created: 2010-09-14  expires: never       usage: E   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
 
gpg> key 1
 
pub  4096R/0x6CACA448860CDC13  created: 2010-09-14  expires: 2017-02-06  usage: SC  
                               trust: ultimate      validity: ultimate
sub* 4096R/0xD1D2BD14810F62B3  created: 2010-09-14  expires: never       usage: E   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
 
gpg> expire
Changing expiration time for a subkey.
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0) 1y
Key expires at Mon 06 Feb 2017 08:09:27 PM JST
Is this correct? (y/N) y
 
You need a passphrase to unlock the secret key for
user: "Norbert Preining <norbert@preining.info>"
4096-bit RSA key, ID 0x6CACA448860CDC13, created 2010-09-14
 
Enter passphrase:
 
pub  4096R/0x6CACA448860CDC13  created: 2010-09-14  expires: 2017-02-06  usage: SC  
                               trust: ultimate      validity: ultimate
sub* 4096R/0xD1D2BD14810F62B3  created: 2010-09-14  expires: 2017-02-06  usage: E   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
Add a photo Not strictly necessary, but an interesting feature. gpg suggests 240 288, I resized a photo of my head, greyscaled it, and optimized it with jpegoptim -s -m40 my-photo.jpg. The parameter 40 is the quality, I played around a bit to find the best balance between size and quality. The size should not be too big as the photo will be part of the key!
gpg> addphoto
 
Pick an image to use for your photo ID.  The image must be a JPEG file.
Remember that the image is stored within your public key.  If you use a
very large picture, your key will become very large as well!
Keeping the image close to 240x288 is a good size to use.
 
Enter JPEG filename for photo ID: GPG/norbert-head.jpg
Is this photo correct (y/N/q)? y
 
You need a passphrase to unlock the secret key for
user: "Norbert Preining <norbert@preining.info>"
4096-bit RSA key, ID 0x6CACA448860CDC13, created 2010-09-14
 
Enter passphrase:
 
pub  4096R/0x6CACA448860CDC13  created: 2010-09-14  expires: 2017-02-06  usage: SC  
                               trust: ultimate      validity: ultimate
sub* 4096R/0xD1D2BD14810F62B3  created: 2010-09-14  expires: 2017-02-06  usage: E   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ unknown] (5)  [jpeg image of size 4185]
Add subkeys of 2048bit for signing/encryption/authentication Now comes the interesting part, adding three subkeys: one for signing, one for encrypting, and one for authentication. The one for signing is the one you will use for signing your uploads to Debian as well as emails. The authentication key will later be used to provide ssh authentication. Note that you have to use the --expert expert option to edit-key (as shown above), otherwise gpg does not allow to do this. As I want to move the subkeys to the Yubikey NEO, a keysize of 2048bits is necessary. First for the signing:
gpg> addkey
Key is protected.
 
You need a passphrase to unlock the secret key for
user: "Norbert Preining <norbert@preining.info>"
4096-bit RSA key, ID 0x6CACA448860CDC13, created 2010-09-14
 
Enter passphrase:
 
Please select what kind of key you want:
   (3) DSA (sign only)
   (4) RSA (sign only)
   (5) Elgamal (encrypt only)
   (6) RSA (encrypt only)
   (7) DSA (set your own capabilities)
   (8) RSA (set your own capabilities)
Your selection? 4
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 
Requested keysize is 2048 bits
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0) 1y
Key expires at Mon 06 Feb 2017 08:10:06 PM JST
Is this correct? (y/N) y
Really create? (y/N) y
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
....+++++
..........+++++
 
pub  4096R/0x6CACA448860CDC13  created: 2010-09-14  expires: 2017-02-06  usage: SC  
                               trust: ultimate      validity: ultimate
sub* 4096R/0xD1D2BD14810F62B3  created: 2010-09-14  expires: 2017-02-06  usage: E   
sub  2048R/0xEC00B8DAD32266AA  created: 2016-02-07  expires: 2017-02-06  usage: S   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ unknown] (5)  [jpeg image of size 4185]
Now the same for encryption key:
gpg> addkey
Key is protected.
 
You need a passphrase to unlock the secret key for
user: "Norbert Preining <norbert@preining.info>"
4096-bit RSA key, ID 0x6CACA448860CDC13, created 2010-09-14
 
Enter passphrase:
 
Please select what kind of key you want:
   (3) DSA (sign only)
   (4) RSA (sign only)
   (5) Elgamal (encrypt only)
   (6) RSA (encrypt only)
   (7) DSA (set your own capabilities)
   (8) RSA (set your own capabilities)
Your selection? 6
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 
Requested keysize is 2048 bits
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0) 1y
Key expires at Mon 06 Feb 2017 08:10:20 PM JST
Is this correct? (y/N) y
Really create? (y/N) y
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
..+++++
........+++++
 
pub  4096R/0x6CACA448860CDC13  created: 2010-09-14  expires: 2017-02-06  usage: SC  
                               trust: ultimate      validity: ultimate
sub* 4096R/0xD1D2BD14810F62B3  created: 2010-09-14  expires: 2017-02-06  usage: E   
sub  2048R/0xEC00B8DAD32266AA  created: 2016-02-07  expires: 2017-02-06  usage: S   
sub  2048R/0xBF361ED434425B4C  created: 2016-02-07  expires: 2017-02-06  usage: E   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ unknown] (5)  [jpeg image of size 4185]
Finally for the authentication key. Note that only here the --expert is necessary! We use (8) RSA (set your own capabilities) and then toggle sign and encryption capabilities off, and authentication on.
gpg> addkey
Key is protected.
 
You need a passphrase to unlock the secret key for
user: "Norbert Preining <norbert@preining.info>"
4096-bit RSA key, ID 0x6CACA448860CDC13, created 2010-09-14
 
Enter passphrase:
 
Please select what kind of key you want:
   (3) DSA (sign only)
   (4) RSA (sign only)
   (5) Elgamal (encrypt only)
   (6) RSA (encrypt only)
   (7) DSA (set your own capabilities)
   (8) RSA (set your own capabilities)
Your selection? 8
 
Possible actions for a RSA key: Sign Encrypt Authenticate 
Current allowed actions: Sign Encrypt 
 
   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished
 
Your selection? s
 
Possible actions for a RSA key: Sign Encrypt Authenticate 
Current allowed actions: Encrypt 
 
   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished
 
Your selection? e
 
Possible actions for a RSA key: Sign Encrypt Authenticate 
Current allowed actions: 
 
   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished
 
Your selection? a
 
Possible actions for a RSA key: Sign Encrypt Authenticate 
Current allowed actions: Authenticate 
 
   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished
 
Your selection? q
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 
Requested keysize is 2048 bits
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0) 1y
Key expires at Mon 06 Feb 2017 08:10:34 PM JST
Is this correct? (y/N) y
Really create? (y/N) y
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
......+++++
+++++
 
pub  4096R/0x6CACA448860CDC13  created: 2010-09-14  expires: 2017-02-06  usage: SC  
                               trust: ultimate      validity: ultimate
sub* 4096R/0xD1D2BD14810F62B3  created: 2010-09-14  expires: 2017-02-06  usage: E   
sub  2048R/0xEC00B8DAD32266AA  created: 2016-02-07  expires: 2017-02-06  usage: S   
sub  2048R/0xBF361ED434425B4C  created: 2016-02-07  expires: 2017-02-06  usage: E   
sub  2048R/0x9C7CA4E294F04D49  created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ unknown] (5)  [jpeg image of size 4185]
 
gpg> save
Check the current status Good point to take a break and inspect the current status. We should have one main key and three subkeys, all with expiry dates of 1 year ahead, and a photo also attached to the key:
$ gpg --expert --edit-key $MASTERKEY
gpg (GnuPG) 1.4.20; Copyright (C) 2015 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
 
Secret key is available.
 
gpg: checking the trustdb
gpg: public key 0x0FC3EC02FBBB8AB1 is 58138 seconds newer than the signature
gpg: 3 marginal(s) needed, 1 complete(s) needed, classic trust model
gpg: depth: 0  valid:   2  signed:  28  trust: 0-, 0q, 0n, 0m, 0f, 2u
gpg: depth: 1  valid:  28  signed:  41  trust: 28-, 0q, 0n, 0m, 0f, 0u
gpg: next trustdb check due at 2016-11-02
pub  4096R/0x6CACA448860CDC13  created: 2010-09-14  expires: 2017-02-06  usage: SC  
                               trust: ultimate      validity: ultimate
sub  4096R/0xD1D2BD14810F62B3  created: 2010-09-14  expires: 2017-02-06  usage: E   
sub  2048R/0xEC00B8DAD32266AA  created: 2016-02-07  expires: 2017-02-06  usage: S   
sub  2048R/0xBF361ED434425B4C  created: 2016-02-07  expires: 2017-02-06  usage: E   
sub  2048R/0x9C7CA4E294F04D49  created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ultimate] (5)  [jpeg image of size 4185]
 
gpg>
Create revocation certificate In case something happens, like all your backups are burned, your computers are destroyed, or all data stolen by the NSA, it is a good idea to have an old fashioned paper print out of a revocation certificate which allows you to revoke the key even if you are not in possession of it. This should be printed out and kept in a safe place.
$ gpg --gen-revoke $MASTERKEY > GPG/revoke-certificate-$MASTERKEY.txt
 
sec  4096R/0x6CACA448860CDC13 2010-09-14 Norbert Preining <norbert@preining.info>
 
Create a revocation certificate for this key? (y/N) y
Please select the reason for the revocation:
  0 = No reason specified
  1 = Key has been compromised
  2 = Key is superseded
  3 = Key is no longer used
  Q = Cancel
(Probably you want to select 1 here)
Your decision? 1
Enter an optional description; end it with an empty line:
> 
Reason for revocation: Key has been compromised
(No description given)
Is this okay? (y/N) y
 
You need a passphrase to unlock the secret key for
user: "Norbert Preining <norbert@preining.info>"
4096-bit RSA key, ID 0x6CACA448860CDC13, created 2010-09-14
 
Enter passphrase:
 
ASCII armored output forced.
Revocation certificate created.
Please move it to a medium which you can hide away; if the NSA or KGB or Mossad gets access to this certificate, they can use it to make your key unusable. It is smart to print this certificate and store it away, just in case your media become unreadable. Create gpg 2.1 structure There are currently three versions of gpg available: classic (version 1) which is one static binary, perfect for servers or scripting tasks; stable (version 2.0) which is the modularized version supporting OpenPGP, S/MIME, and Secure Shell; and finally modern (version 2.1 and up) with enhanced features like support for Elliptic Curve cryptography. Debian currently ships version 1 as standard, and also the modern version (but there are traces in experimental of a pending transition). The newer versions of GnuPG are modularized and use an agent. For the following we need to kill any running instance of gpg-agent.
$ killall gpg-agent
After that a simple call to gpg2 to list the secret keys will convert the layout to the new standard:
$ gpg2 -K $MASTERKEY
gpg: keyserver option 'ca-cert-file' is obsolete; please use 'hkp-cacert' in dirmngr.conf
gpg: starting migration from earlier GnuPG versions
gpg: porting secret keys from '/home/norbert/.gnupg/secring.gpg' to gpg-agent
gpg: key 0xD2BF4AA309C5B094: secret key imported
gpg: key 0x6CACA448860CDC13: secret key imported
gpg: migration succeeded
sec   rsa4096/0x6CACA448860CDC13 2010-09-14 [SC] [expires: 2017-02-06]
      Key fingerprint = F7D8 A928 26E3 16A1 9FA0  ACF0 6CAC A448 860C DC13
uid                   [ultimate] Norbert Preining <norbert@preining.info>
uid                   [ultimate] Norbert Preining <preining@logic.at>
uid                   [ultimate] Norbert Preining <preining@debian.org>
uid                   [ultimate] Norbert Preining <preining@jaist.ac.jp>
uid                   [ultimate] [jpeg image of size 4185]
ssb   rsa4096/0xD1D2BD14810F62B3 2010-09-14 [E] [expires: 2017-02-06]
ssb   rsa2048/0xEC00B8DAD32266AA 2016-02-07 [S] [expires: 2017-02-06]
ssb   rsa2048/0xBF361ED434425B4C 2016-02-07 [E] [expires: 2017-02-06]
ssb   rsa2048/0x9C7CA4E294F04D49 2016-02-07 [A] [expires: 2017-02-06]
After this there will be new files/directories in the .gnupg directory, in particular: .gnupg/private-keys-v1.d/ which contains the private keys. Creating backup Now your .gnupg directory contains still all the keys, available for gpg1 and gpg2.1. You MUST MAKE A BACKUP NOW!!! on at least 3 USB sticks and maybe some other offline media. Keep them in a safe place, better in different and safe places, you will need them for extending the expiry date, signing other keys, etc. Warning concerning USB and vfat file systems gpg >= 2.1 requires gpg-agent which in turn needs a socket. If you have the backup on an USB drive (most often with vfat file system), you need to redirect the socket, as vfat does not support sockets! Edit /USBSTICK/gnupghome/S.gpg-agent and enter there
%Assuan%
socket=/dev/shm/S.gpg-agent
After that the socket will be created in /dev/shm/ instead and invoking gpg with gpg2 --homedir /USBSTICK/gnupghome will work. You have done your backups, right? Move sub keys to card As I mentioned, I want to have no keys on my laptop which I carry around to strange countries, instead I want to have them all on a Yubikey NEO. I will describe the setup and usage in details soon, but mention here only how to move the keys to the card. This requires a finished setup including change of pins. Note that when using gpg2 to move the keys to the card, the local copies are actually deleted, but only for the gpg2(.1) files. The gpg1 secret keys are still all in place.
$ gpg2 --edit-key $MASTERKEY
gpg (GnuPG) 2.1.11; Copyright (C) 2016 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
 
Secret key is available.
 
sec  rsa4096/0x6CACA448860CDC13
     created: 2010-09-14  expires: 2017-02-06  usage: SC  
     trust: ultimate      validity: ultimate
ssb  rsa4096/0xD1D2BD14810F62B3
     created: 2010-09-14  expires: 2017-02-06  usage: E   
ssb  rsa2048/0xEC00B8DAD32266AA
     created: 2016-02-07  expires: 2017-02-06  usage: S   
ssb  rsa2048/0xBF361ED434425B4C
     created: 2016-02-07  expires: 2017-02-06  usage: E   
ssb  rsa2048/0x9C7CA4E294F04D49
     created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ultimate] (5)  [jpeg image of size 4185]
 
gpg> key 2
 
sec  rsa4096/0x6CACA448860CDC13
     created: 2010-09-14  expires: 2017-02-06  usage: SC  
     trust: ultimate      validity: ultimate
ssb  rsa4096/0xD1D2BD14810F62B3
     created: 2010-09-14  expires: 2017-02-06  usage: E   
ssb* rsa2048/0xEC00B8DAD32266AA
     created: 2016-02-07  expires: 2017-02-06  usage: S   
ssb  rsa2048/0xBF361ED434425B4C
     created: 2016-02-07  expires: 2017-02-06  usage: E   
ssb  rsa2048/0x9C7CA4E294F04D49
     created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ultimate] (5)  [jpeg image of size 4185]
 
gpg> keytocard
Please select where to store the key:
   (1) Signature key
   (3) Authentication key
Your selection? 1
 
sec  rsa4096/0x6CACA448860CDC13
     created: 2010-09-14  expires: 2017-02-06  usage: SC  
     trust: ultimate      validity: ultimate
ssb  rsa4096/0xD1D2BD14810F62B3
     created: 2010-09-14  expires: 2017-02-06  usage: E   
ssb* rsa2048/0xEC00B8DAD32266AA
     created: 2016-02-07  expires: 2017-02-06  usage: S   
ssb  rsa2048/0xBF361ED434425B4C
     created: 2016-02-07  expires: 2017-02-06  usage: E   
ssb  rsa2048/0x9C7CA4E294F04D49
     created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ultimate] (5)  [jpeg image of size 4185]
 
gpg> key 2
 
sec  rsa4096/0x6CACA448860CDC13
     created: 2010-09-14  expires: 2017-02-06  usage: SC  
     trust: ultimate      validity: ultimate
ssb  rsa4096/0xD1D2BD14810F62B3
     created: 2010-09-14  expires: 2017-02-06  usage: E   
ssb  rsa2048/0xEC00B8DAD32266AA
     created: 2016-02-07  expires: 2017-02-06  usage: S   
ssb  rsa2048/0xBF361ED434425B4C
     created: 2016-02-07  expires: 2017-02-06  usage: E   
ssb  rsa2048/0x9C7CA4E294F04D49
     created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ultimate] (5)  [jpeg image of size 4185]
 
gpg> key 3
 
sec  rsa4096/0x6CACA448860CDC13
     created: 2010-09-14  expires: 2017-02-06  usage: SC  
     trust: ultimate      validity: ultimate
ssb  rsa4096/0xD1D2BD14810F62B3
     created: 2010-09-14  expires: 2017-02-06  usage: E   
ssb  rsa2048/0xEC00B8DAD32266AA
     created: 2016-02-07  expires: 2017-02-06  usage: S   
ssb* rsa2048/0xBF361ED434425B4C
     created: 2016-02-07  expires: 2017-02-06  usage: E   
ssb  rsa2048/0x9C7CA4E294F04D49
     created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ultimate] (5)  [jpeg image of size 4185]
 
gpg> keytocard
Please select where to store the key:
   (2) Encryption key
Your selection? 2
 
sec  rsa4096/0x6CACA448860CDC13
     created: 2010-09-14  expires: 2017-02-06  usage: SC  
     trust: ultimate      validity: ultimate
ssb  rsa4096/0xD1D2BD14810F62B3
     created: 2010-09-14  expires: 2017-02-06  usage: E   
ssb  rsa2048/0xEC00B8DAD32266AA
     created: 2016-02-07  expires: 2017-02-06  usage: S   
ssb* rsa2048/0xBF361ED434425B4C
     created: 2016-02-07  expires: 2017-02-06  usage: E   
ssb  rsa2048/0x9C7CA4E294F04D49
     created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ultimate] (5)  [jpeg image of size 4185]
 
gpg> key 3
 
sec  rsa4096/0x6CACA448860CDC13
     created: 2010-09-14  expires: 2017-02-06  usage: SC  
     trust: ultimate      validity: ultimate
ssb  rsa4096/0xD1D2BD14810F62B3
     created: 2010-09-14  expires: 2017-02-06  usage: E   
ssb  rsa2048/0xEC00B8DAD32266AA
     created: 2016-02-07  expires: 2017-02-06  usage: S   
ssb  rsa2048/0xBF361ED434425B4C
     created: 2016-02-07  expires: 2017-02-06  usage: E   
ssb  rsa2048/0x9C7CA4E294F04D49
     created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ultimate] (5)  [jpeg image of size 4185]
 
gpg> key 4
 
sec  rsa4096/0x6CACA448860CDC13
     created: 2010-09-14  expires: 2017-02-06  usage: SC  
     trust: ultimate      validity: ultimate
ssb  rsa4096/0xD1D2BD14810F62B3
     created: 2010-09-14  expires: 2017-02-06  usage: E   
ssb  rsa2048/0xEC00B8DAD32266AA
     created: 2016-02-07  expires: 2017-02-06  usage: S   
ssb  rsa2048/0xBF361ED434425B4C
     created: 2016-02-07  expires: 2017-02-06  usage: E   
ssb* rsa2048/0x9C7CA4E294F04D49
     created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ultimate] (5)  [jpeg image of size 4185]
 
gpg> keytocard
Please select where to store the key:
   (3) Authentication key
Your selection? 3
 
sec  rsa4096/0x6CACA448860CDC13
     created: 2010-09-14  expires: 2017-02-06  usage: SC  
     trust: ultimate      validity: ultimate
ssb  rsa4096/0xD1D2BD14810F62B3
     created: 2010-09-14  expires: 2017-02-06  usage: E   
ssb  rsa2048/0xEC00B8DAD32266AA
     created: 2016-02-07  expires: 2017-02-06  usage: S   
ssb  rsa2048/0xBF361ED434425B4C
     created: 2016-02-07  expires: 2017-02-06  usage: E   
ssb* rsa2048/0x9C7CA4E294F04D49
     created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ultimate] (5)  [jpeg image of size 4185]
 
gpg> key 4
 
sec  rsa4096/0x6CACA448860CDC13
     created: 2010-09-14  expires: 2017-02-06  usage: SC  
     trust: ultimate      validity: ultimate
ssb  rsa4096/0xD1D2BD14810F62B3
     created: 2010-09-14  expires: 2017-02-06  usage: E   
ssb  rsa2048/0xEC00B8DAD32266AA
     created: 2016-02-07  expires: 2017-02-06  usage: S   
ssb  rsa2048/0xBF361ED434425B4C
     created: 2016-02-07  expires: 2017-02-06  usage: E   
ssb  rsa2048/0x9C7CA4E294F04D49
     created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ultimate] (5)  [jpeg image of size 4185]
 
gpg> save
Note the repetition of selecting and deselecting keys. Current status After this procedure we are now in the following situation: In gpg words it looks like this:
$ gpg2 -K $MASTERKEY
gpg: keyserver option 'ca-cert-file' is obsolete; please use 'hkp-cacert' in dirmngr.conf
sec   rsa4096/0x6CACA448860CDC13 2010-09-14 [SC] [expires: 2017-02-06]
      Key fingerprint = F7D8 A928 26E3 16A1 9FA0  ACF0 6CAC A448 860C DC13
uid                   [ultimate] Norbert Preining <norbert@preining.info>
uid                   [ultimate] Norbert Preining <preining@logic.at>
uid                   [ultimate] Norbert Preining <preining@debian.org>
uid                   [ultimate] Norbert Preining <preining@jaist.ac.jp>
uid                   [ultimate] [jpeg image of size 4185]
ssb   rsa4096/0xD1D2BD14810F62B3 2010-09-14 [E] [expires: 2017-02-06]
ssb>  rsa2048/0xEC00B8DAD32266AA 2016-02-07 [S] [expires: 2017-02-06]
ssb>  rsa2048/0xBF361ED434425B4C 2016-02-07 [E] [expires: 2017-02-06]
ssb>  rsa2048/0x9C7CA4E294F04D49 2016-02-07 [A] [expires: 2017-02-06]
 
$ gpg -K $MASTERKEY
sec   4096R/0x6CACA448860CDC13 2010-09-14 [expires: 2017-02-06]
      Key fingerprint = F7D8 A928 26E3 16A1 9FA0  ACF0 6CAC A448 860C DC13
uid                            Norbert Preining <norbert@preining.info>
uid                            Norbert Preining <preining@logic.at>
uid                            Norbert Preining <preining@debian.org>
uid                            Norbert Preining <preining@jaist.ac.jp>
uid                            [jpeg image of size 4185]
ssb   4096R/0xD1D2BD14810F62B3 2010-09-14 [expires: 2017-02-06]
ssb   2048R/0xEC00B8DAD32266AA 2016-02-07 [expires: 2017-02-06]
ssb   2048R/0xBF361ED434425B4C 2016-02-07 [expires: 2017-02-06]
ssb   2048R/0x9C7CA4E294F04D49 2016-02-07 [expires: 2017-02-06]
 
$ gpg2 --card-status
 
....
Name of cardholder: Norbert Preining
....
PIN retry counter : 3 3 3
Signature counter : 0
Signature key ....: 5871 F824 2DCC 3660 2362  BE7D EC00 B8DA D322 66AA
      created ....: 2016-02-07 11:10:06
Encryption key....: 2501 195C 90AB F4D2 3DEA  A303 BF36 1ED4 3442 5B4C
      created ....: 2016-02-07 11:10:20
Authentication key: 9CFB 3775 C164 0E99 F0C8  014C 9C7C A4E2 94F0 4D49
      created ....: 2016-02-07 11:10:34
General key info..: sub  rsa2048/0xEC00B8DAD32266AA 2016-02-07 Norbert Preining <norbert@preining.info>
sec   rsa4096/0x6CACA448860CDC13  created: 2010-09-14  expires: 2017-02-06
ssb   rsa4096/0xD1D2BD14810F62B3  created: 2010-09-14  expires: 2017-02-06
ssb>  rsa2048/0xEC00B8DAD32266AA  created: 2016-02-07  expires: 2017-02-06
                                  card-no: 0006 03645719
ssb>  rsa2048/0xBF361ED434425B4C  created: 2016-02-07  expires: 2017-02-06
                                  card-no: 0006 03645719
ssb>  rsa2048/0x9C7CA4E294F04D49  created: 2016-02-07  expires: 2017-02-06
                                  card-no: 0006 03645719
$
Remove private master keys You are sure that you have a working backup? Did you try it with gpg --homedir ...? Only if you are really sure, continue. We are now removing the master key from both the gpg2 and gpg1 setup. removal for gpg2 gpg2 keeps the private keys in ~/.gnupg/private-keys-v1.d/KEYGRIP.key and the KEYGRIP can be found by adding --with-keygrip to the key listing. Be sure to delete the correct file, the one related to the master key.
$ gpg2 --with-keygrip --list-key $MASTERKEY
pub   rsa4096/0x6CACA448860CDC13 2010-09-14 [SC] [expires: 2017-02-06]
      Key fingerprint = F7D8 A928 26E3 16A1 9FA0  ACF0 6CAC A448 860C DC13
      Keygrip = 9DC1E90703856C1DE0EAC970CED7ABF5EE5EF79D
uid                   [ultimate] Norbert Preining <norbert@preining.info>
uid                   [ultimate] Norbert Preining <preining@logic.at>
uid                   [ultimate] Norbert Preining <preining@debian.org>
uid                   [ultimate] Norbert Preining <preining@jaist.ac.jp>
uid                   [ultimate] [jpeg image of size 4185]
sub   rsa4096/0xD1D2BD14810F62B3 2010-09-14 [E] [expires: 2017-02-06]
      Keygrip = 4B8FF57434DD989243666377376903281D861596
sub   rsa2048/0xEC00B8DAD32266AA 2016-02-07 [S] [expires: 2017-02-06]
      Keygrip = 39B14EF1392F2F251863A87AE4D44CE502755C39
sub   rsa2048/0xBF361ED434425B4C 2016-02-07 [E] [expires: 2017-02-06]
      Keygrip = E41C8DDB2A22976AE0DA8D7D11F586EA793203EA
sub   rsa2048/0x9C7CA4E294F04D49 2016-02-07 [A] [expires: 2017-02-06]
      Keygrip = A337DE390143074C6DBFEA64224359B9859B02FC
 
$ rm ~/.gnupg/private-keys-v1.d/9DC1E90703856C1DE0EAC970CED7ABF5EE5EF79D.key
$
After that the missing key is shown in gpg2 -K with an additional # meaning that the key is not available:
$ gpg2 -K $MASTERKEY
sec#  rsa4096/0x6CACA448860CDC13 2010-09-14 [SC] [expires: 2017-02-06]
...
removal for gpg1 Up to gpg v2.0 there is no simple way to delete only one part of the key. We export the subkeys, delete the private key, and reimport the subkeys:
$ gpg --output secret-subkeys --export-secret-subkeys $MASTERKEY
 
$ gpg --delete-secret-keys $MASTERKEY
 
sec  4096R/0x6CACA448860CDC13 2010-09-14 Norbert Preining <norbert@preining.info>
 
Delete this key from the keyring? (y/N) y
This is a secret key! - really delete? (y/N) y
 
$ gpg --import secret-subkeys
gpg: key 0x6CACA448860CDC13: secret key imported
gpg: key 0x6CACA448860CDC13: "Norbert Preining <norbert@preining.info>" not changed
gpg: Total number processed: 1
gpg:              unchanged: 1
gpg:       secret keys read: 1
gpg:   secret keys imported: 1
 
$
Current status We are basically at the stage we wanted to achieve: For gpg2.1 only the old encryption key is available, the master key is not, and the other sub keys are moved to the yubikey:
$ gpg2 -K $MASTERKEY
sec#  rsa4096/0x6CACA448860CDC13 2010-09-14 [SC] [expires: 2017-02-06]
      Key fingerprint = F7D8 A928 26E3 16A1 9FA0  ACF0 6CAC A448 860C DC13
uid                   [ultimate] Norbert Preining <norbert@preining.info>
uid                   [ultimate] Norbert Preining <preining@logic.at>
uid                   [ultimate] Norbert Preining <preining@debian.org>
uid                   [ultimate] Norbert Preining <preining@jaist.ac.jp>
uid                   [ultimate] [jpeg image of size 4185]
ssb   rsa4096/0xD1D2BD14810F62B3 2010-09-14 [E] [expires: 2017-02-06]
ssb>  rsa2048/0xEC00B8DAD32266AA 2016-02-07 [S] [expires: 2017-02-06]
ssb>  rsa2048/0xBF361ED434425B4C 2016-02-07 [E] [expires: 2017-02-06]
ssb>  rsa2048/0x9C7CA4E294F04D49 2016-02-07 [A] [expires: 2017-02-06]
$
And for gpg <= 2.0 the old encryption key and the sub keys are available, but the master key is not:
$ gpg -K $MASTERKEY
sec#  4096R/0x6CACA448860CDC13 2010-09-14 [expires: 2017-02-06]
      Key fingerprint = F7D8 A928 26E3 16A1 9FA0  ACF0 6CAC A448 860C DC13
uid                            Norbert Preining <norbert@preining.info>
uid                            Norbert Preining <preining@logic.at>
uid                            Norbert Preining <preining@debian.org>
uid                            Norbert Preining <preining@jaist.ac.jp>
uid                            [jpeg image of size 4185]
ssb   4096R/0xD1D2BD14810F62B3 2010-09-14 [expires: 2017-02-06]
ssb   2048R/0xEC00B8DAD32266AA 2016-02-07 [expires: 2017-02-06]
ssb   2048R/0xBF361ED434425B4C 2016-02-07 [expires: 2017-02-06]
ssb   2048R/0x9C7CA4E294F04D49 2016-02-07 [expires: 2017-02-06]
 
$
Split the .gnupg directory for mail server and laptop As mentioned, I want to have a gpg1 version available at the server where I read my emails, and be able to sign/encrypt emails there, while on my laptop no secret key is available. Thus I prepare two gnupg directories. For the mailserver the gpg2 specific files are removed:
$ cp -a .gnupg .gnupg-mail
$ cd .gnupg-mail
$ rm -rf private-keys-v1.d/ pubring.gpg~ reader_0.status
$ rm -rf S.gpg-agent* S.scdaemon .gpg-v21-migrated
On my laptop, where I did all this operation, I remove the gpg1 files, namely the outdated secring.gpg:
$ cd $HOME/.gnupg
$ rm secring.gpg
As a last step I move the .gnupg-mail directory to my mail server. Once could *expire* the old encryption key, but for now I leave it as is. Upload keys to keyservers If you are a Debian Developer, a simple update of your master key will suffice:
gpg --keyserver hkp://keyring.debian.org --send-key YOURMASTERKEYID
Note that the update from the keyring server to the actual Debian keyring takes up to one month. Until that time either do not upload anything, or use the (offline) master key for signing. After your key has been updated in the Debian keyring, signatures made with the signing subkey will be accepted for uploading to Debian. It might be also a good idea to upload your new keys to some keyservers like:
gpg --keyserver hkp://pool.sks-keyservers.net --send-key $MASTERKEY
Now you an also fix the configuration file skew between gpg1 and gpg2. Further remark I am currently trying to use the authentication key from my Yubikey NEO as ssh key, but bugs (see #795368 and #818969) prohibit it at the moment. Raphael Herzog gave a possible fix by killing the gpg-agent and restarting it with gpg-agent --daemon from an X terminal, and I can confirm that this worked. After one year before the key expires I need to extend the key validity for another year. For this you need the offline master key. I will describe the process when it becomes necessary. Reading list The following web sites have been useful in collecting the necessary information:
  1. https://iain.learmonth.me/yubikey-neo-gpg/
  2. https://iain.learmonth.me/yubikey-udev/
  3. http://blog.josefsson.org/2014/06/23/offline-gnupg-master-key-and-subkeys-on-yubikey-neo-smartcard/
  4. https://wiki.debian.org/Subkeys
  5. https://jclement.ca/articles/2015/gpg-smartcard/ as modernized version of (3)
  6. https://www.esev.com/blog/post/2015-01-pgp-ssh-key-on-yubikey-neo/ similar style, with ssh and gnome-keyring infos
  7. http://karlgrz.com/2fa-gpg-ssh-keys-with-pass-and-yubikey-neo/ also good reading
  8. https://help.riseup.net/en/security/message-security/openpgp/best-practices good and concise advise on gpg practices
My writing is mostly based on (5) with additions from (4). Please let me know of any errors, improvements, and fixes. I hope this walk-through might help others in the same situation.

10 April 2016

Russ Allbery: Largish haul

Let's see if I can scrounge through all of my now-organized directories of ebooks and figure out what I haven't recorded here yet. At least the paper books make that relatively easy, since I don't shelve them until I post them. (Yeah, yeah, I should actually make a database.) Hugh Aldersey-Williams Periodic Tales (nonfiction)
Sandra Ulbrich Almazan SF Women A-Z (nonfiction)
Radley Balko Rise of the Warrior Cop (nonfiction)
Peter V. Brett The Warded Man (sff)
Lois McMaster Bujold Gentleman Jole and the Red Queen (sff)
Fred Clark The Anti-Christ Handbook Vol. 2 (nonfiction)
Dave Duncan West of January (sff)
Karl Fogel Producing Open Source Software (nonfiction)
Philip Gourevitch We Wish to Inform You That Tomorrow We Will Be Killed With Our Families (nonfiction)
Andrew Groen Empires of EVE (nonfiction)
John Harris @ Play (nonfiction)
David Hellman & Tevis Thompson Second Quest (graphic novel)
M.C.A. Hogarth Earthrise (sff)
S.L. Huang An Examination of Collegial Dynamics... (sff)
S.L. Huang & Kurt Hunt Up and Coming (sff anthology)
Kameron Hurley Infidel (sff)
Kevin Jackson-Mead & J. Robinson Wheeler IF Theory Reader (nonfiction)
Rosemary Kirstein The Lost Steersman (sff)
Rosemary Kirstein The Language of Power (sff)
Merritt Kopas Videogames for Humans (nonfiction)
Alisa Krasnostein & Alexandra Pierce (ed.) Letters to Tiptree (nonfiction)
Mathew Kumar Exp. Negatives (nonfiction)
Ken Liu The Grace of Kings (sff)
Susan MacGregor The Tattooed Witch (sff)
Helen Marshall Gifts for the One Who Comes After (sff collection)
Jack McDevitt Coming Home (sff)
Seanan McGuire A Red-Rose Chain (sff)
Seanan McGuire Velveteen vs. The Multiverse (sff)
Seanan McGuire The Winter Long (sff)
Marc Miller Agent of the Imperium (sff)
Randal Munroe Thing Explainer (graphic nonfiction)
Marguerite Reed Archangel (sff)
J.K. Rowling Harry Potter: The Complete Collection (sff)
K.J. Russell Tides of Possibility (sff anthology)
Robert J. Sawyer Starplex (sff)
Bruce Schneier Secrets & Lies (nonfiction)
Mike Selinker (ed.) The Kobold Game to Board Game Design (nonfiction)
Douglas Smith Chimerascope (sff collection)
Jonathan Strahan Fearsome Journeys (sff anthology)
Nick Suttner Shadow of the Colossus (nonfiction)
Aaron Swartz The Boy Who Could Change the World (essays)
Caitlin Sweet The Pattern Scars (sff)
John Szczepaniak The Untold History of Japanese Game Developers I (nonfiction)
John Szczepaniak The Untold History of Japanese Game Developers II (nonfiction)
Jeffrey Toobin The Run of His Life (nonfiction)
Hayden Trenholm Blood and Water (sff anthology)
Coen Teulings & Richard Baldwin (ed.) Secular Stagnation (nonfiction)
Ursula Vernon Book of the Wombat 2015 (graphic nonfiction)
Ursula Vernon Digger (graphic novel) Phew, that was a ton of stuff. A bunch of these were from two large StoryBundle bundles, which is a great source of cheap DRM-free ebooks, although still rather hit and miss. There's a lot of just fairly random stuff that's been accumulating for a while, even though I've not had a chance to read very much. Vacation upcoming, which will be a nice time to catch up on reading.

2 July 2015

Petter Reinholdtsen: MakerCon Nordic videos now available on Frikanalen

Last oktober I was involved on behalf of NUUG with recording the talks at MakerCon Nordic, a conference for the Maker movement. Since then it has been the plan to publish the recordings on Frikanalen, which finally happened the last few days. A few talks are missing because the speakers asked the organizers to not publish them, but most of the talks are available. The talks are being broadcasted on RiksTV channel 50 and using multicast on Uninett, as well as being available from the Frikanalen web site. The unedited recordings are available on Youtube too. This is the list of talks available at the moment. Visit the Frikanalen video pages to view them. Part of the reason this took so long was that the scripts NUUG had to prepare a recording for publication were five years old and no longer worked with the current video processing tools (command line argument changes). In addition, we needed better audio normalization, which sent me on a detour to package bs1770gain for Debian. Now this is in place and it became a lot easier to publish NUUG videos on Frikanalen.

15 June 2015

Lunar: Reproducible builds: week 7 in Stretch cycle

What happened about the reproducible builds effort for this week: Presentations On June 7th, Reiner Herrmann presented the project at the Gulaschprogrammiernacht 15 in Karlsruhe, Germany. Video and audio recordings in German are available, and so are the slides in English. Toolchain fixes Daniel Kahn Gillmor's report on help2man started a discussion with Brendan O'Dea and Ximin Luo about standardizing a common environment variable that would provide a replacement for an embedded build date. After various proposals and research by Ximin about date handling in several programming languages, the best solution seems to define SOURCE_DATE_EPOCH with a value suitable for gmtime(3).
  1. Martin Borgert wondered if Sphinx could be changed in a way that would avoid having to tweak debian/rules in packages using it to produce HTML documentation.
Daniel Kahn Gillmor opened a new report about icont producing unreproducible binaries. Packages fixed The following 32 packages became reproducible due to changes in their build dependencies: agda, alex, c2hs, clutter-1.0, colorediffs-extension, cpphs, darcs-monitor, dispmua, haskell-curl, haskell-glfw, haskell-glib, haskell-gluraw, haskell-glut, haskell-gnutls, haskell-gsasl, haskell-hfuse, haskell-hledger-interest, haskell-hslua, haskell-hsqml, haskell-hssyck, haskell-libxml-sax, haskell-openglraw, haskell-readline, haskell-terminfo, haskell-x11, jarjar-maven-plugin, kxml2, libcgi-struct-xs-perl, libobject-id-perl, maven-docck-plugin, parboiled, pegdown. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which did not make their way to the archive yet: reproducible.debian.net A new variation to better notice when a package captures the environment has been introduced. (h01ger) The test on Debian packages works by building the package twice in a short time frame. But sometimes, a mirror push can happen between the first and the second build, resulting in a package built in a different build environment. This situation is now properly detected and will run a third build automatically. (h01ger) OpenWrt, the distribution specialized in embedded devices like small routers, is now being tested for reproducibility. The situation looks very good for their packages which seems mostly affected by timestamps in the tarball. System images will require more work on debbindiff to be better understood. (h01ger) debbindiff development Reiner Herrmann added support for decompling Java .class file and .ipk package files (used by OpenWrt). This is now available in version 22 released on 2015-06-14. Documentation update Stephen Kitt documented the new --insert-timestamp available since binutils-mingw-w64 version 6.2 available to insert a ready-made date in PE binaries built with mingw-w64. Package reviews 195 obsolete reviews have been removed, 65 added and 126 updated this week. New identified issues: Misc. Holger Levsen reported an issue with the locales-all package that Provides: locales but is actually missing some of the files provided by locales. Coreboot upstream has been quick to react after the announcement of the tests set up the week before. Patrick Georgi has fixed all issues in a couple of days and all Coreboot images are now reproducible (without a payload). SeaBIOS is one of the most frequently used payload on PC hardware and can now be made reproducible too. Paul Kocialkowski wrote to the mailing list asking for help on getting U-Boot tested for reproducibility. Lunar had a chat with maintainers of Open Build Service to better understand the difference between their system and what we are doing for Debian.

15 April 2015

Joachim Breitner: Talk and Article on Monads for Reverse Engineering

In a recent project of mine, a tool to analyze and create files for the Ravensburger Tiptoi pen, I used two interesting Monads with good results: As that s quite neat, I write a blog post for the German blog funktionale-programmierung.de about it, and also held a talk at the Karlsruhe functional programmers group. If you know some German, enjoy; if not, wait until I have a reason to hold the talk in English. (As a matter of fact, I did hold the talk in English, but only spontaneously, so the text is in German only so far.)

13 April 2015

Dirk Eddelbuettel: inline 0.3.14

The inline package facilitates writing code in-line in simple string expressions or short files. The package is mature and in maintenance mode: Rcpp used it greatly for several years but then moved on to Rcpp Attributes so we have a much limited need for extensions to inline. But we now have a new inline version 0.3.14. It brings both a few minor code updates since the last release in 2013, but also new extensions to both support Fortran better (for several flavours including f95) and to make working with dynamic library files easier. These were contributed by long-time R author Karline Soetaert who thereby became a package co-author. Also, the package moved to GitHub sometime last year and now lives in this repo. See below for a detailed list of changes extracted from the NEWS file.
Changes in inline version 0.3.14 (2015-04-11)
  • Removed call to Rcpp::RcppLdFlags() which is no longer needed
  • With move of repository to GitHub, added a .travis.yml file and corresponding entry in .Rbuildignore
  • Replaced calls to require() with calls to requireNamespace(); also updated one call
  • Much improved support for Fortran and Fortran95 thanks to Karline Soetaert who became a package co-author
  • New helper functions writeDynLib and readDynLib as well as new methods print and code (also by Karline)
Courtesy of CRANberries, there is a comparison to the previous release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

6 November 2014

Dirk Eddelbuettel: RPushbullet 0.1.1

A minor bugfix release 0.1.1 of the RPushbullet package (interfacing the neat Pushbullet service) landed on CRAN yesterday morning. It cleans up a small issue related to the ability to transfer files between devices via the Pushbullet service where the ability to select a (non-default) target device has now been restored. With that, allow me to borrow one excellent use case of RPushbullet from the blog of Karl Broman: how to use RPushbullet for error notifications from R:
options(error = function()  
    library(RPushbullet)
    pbPost("note", "Error", geterrmessage())
 )
This is very clever: should an error occur, you get immediate notification in browser or on your phone. Left as an exercise for the reader is to combine this with the equally excellent rfoaas package (github cran) to get appropriately colourful error messages... More details about the package are at the RPushbullet webpage and the RPushbullet GitHub repo. Courtesy of CRANberries, there is also a diffstat report for this release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

26 September 2014

Dirk Eddelbuettel: R and Docker

r and docker talk picture by @mediafly Earlier this evening I gave a short talk about R and Docker at the September Meetup of the Docker Chicago group. Thanks to Karl Grzeszczak for setting the meeting, and for providing a pretty thorough intro talk regarding CoreOS and Docker. My slides are now up on my presentations page.

2 July 2014

Russ Allbery: Review: Lockstep

Review: Lockstep, by Karl Schroeder
Publisher: Tor
Copyright: March 2014
ISBN: 0-7653-3726-6
Format: Hardcover
Pages: 351
Toby McGonigal's family fled an Earth dominated by trillionaires and vicious class conflict, and attempt, instead, a very risky and precarious settlement on a trans-Neptunian object. It's the last unclaimed but marginally habitable space left in the solar system, and securing their claim requires constant bureaucratic hoop-jumping. That's what sent Toby on a solo mission to a distant moon of their home to claim it and solidify their title. But, while in cold sleep, his craft hits a small chunk of rock, and he wakes up near another world: cold, silent, dead... but apparently with lifeless cities. Another trip through suspended animation in a desperate attempt to conserve resources against the distant hope of rescue has him awakening in a society both utterly foreign and yet strangely familiar. Schroeder is one of the better big idea writers in science fiction, but I found Lockstep to be a mixed bag. He does a surprisingly good job with the core conceit of the novel (more on that in a moment), even though it's a tricky one to make believable. Surrounding that, though, are a lot of less convincing bits that I kept having to not think about too hard, such as Toby crossing the path of another trans-Neptunian object after the accident (space is really, really big and really, really empty), or the implications of later revelations about the time scales involved in parts of the plot. Some parts of the world building, even if scientifically plausible, struck me as sloppy; for example, a religion plays a prominent role in the plot, but the nature of that religion was not particularly believable, nor was its interaction with the plot climax. I won't go into details about the religion, since it's a major plot point, but the short version is that religions tend to mature from the concrete to the abstract, not the other way around. That said, the core conceit is both surprising and considerably better-defended in the book than I thought it could be. The world into which Toby awakes is the world of the locksteps: a society that uses suspended animation in a universal and coordinated way to build a functional society on the far outskirts of the solar system. Humans emerge for some short period of time, like a month, and build, trade, interact, and consume. They welcome ships traveling from other trans-Neptunian worlds and prepare for their own journeys. And then they go into suspended animation for an extended period 15 years and 30 years are common choices while robots slowly gather more resources and energy, and repair and replace what's consumed in the month of active life. Ships travel with passengers in suspended animation, allowing the vast distances between these cold worlds to be reached during the sleep period. And, since all members of the lockstep sleep and wake on the same schedule, there is no wrenching desynchronization with the surrounding society during travel. One may spend thirty years in transit, but no time passes for anyone else in the lockstep while you're traveling either. A world can effectively trade with all other worlds within a thirty year travel radius without noticing the elapsed time. This is a technological system that at first doesn't sound like it would work, but Schroeder does a great job defending it and chasing down implications. The lockstep civilization serves as a sort of anchor and time capsule separate from the more frantic pace of the so-called fast worlds. The trans-Neptunian lockstep worlds are remote enough and poor enough to not attract too much unwanted attention, and are thus tolerated by fast societies that may have more available resources. (Although, to be sure, automated robot defenses have to carry a lot of weight here given how helpless the locksteps are during a sleep cycle.) I'm not sure I completely bought the sociology, but it works well enough to carry the weight of the story, and I've never seen a science fiction construction that uses the common technology of suspended animation in quite this way before. Schroeder sets up some nice stabilizing tensions between resources and time, adds some believable political reasons for this fragile society to survive, and uses some of its obvious political vulnerabilities as story drivers. The plot, unfortunately, isn't as good as the big idea, and particularly suffers at the start of the book. Toby is a teenager without much experience (Lockstep is marketed as young adult), and is immediately thrown into a strange and quickly hostile environment. This means that he spends the first half of the book reacting to a blizzard of new information, and the plot tends to revert to a tour of Schroeder's constructed world. Toby is also a bit of a cipher and a bit of an everyman, without many feelings or opinions beyond the obvious feelings that any teenage boy would go through in this situation. That makes the sense of a world tour even stronger. This flaw does not persist through the whole book. Toby does eventually start making decisions and doing things, some of the supporting characters add additional depth, and I found the unwinding of the plot surprisingly satisfying. The nature of time in this world lets Schroeder have both epic sweep and personal connection at the same time, which lets him pull off some neat contrasts between the personal and the political. I also liked Toby's gradual piecing together of what actually happened while he was asleep, both in the construction of society as a whole and in the personal conflicts between people he knew well. Some of the ease of grand manipulation seemed dubious to me, but, in Schroeder's defense, people do develop a reverence for and stories about things that have lasted a very long time, and Schroeder's setup gives him quite a lot to work with in that department. So, a mixed bag. The core concept is thought-provoking and up to Schroeder's usual standards. The surrounding world-building isn't as much, and I think Schroeder reaches for some too-easy explanations and still underplays just how many disruptive things can happen over the span of time that this book covers. The characterization I found weak and unsatisfying for much of the book, but it gets moderately better in the end. There's a bit too much tour, and a bit too much world exploration instead of plot, but it's a fascinating world and I still enjoyed the tour. With stronger characters and a few fewer dubious supporting pillars in the world background, I think this would have been an excellent book. As is, it's an enjoyable novel set firmly in the big idea and deep future end of science fiction, and I'm always happy to see more stories like that. It's not the best that Schroeder has done, but I still recommend it. Rating: 7 out of 10

Next.