Search Results: "Axel Beckert"

19 September 2022

Axel Beckert: wApua 0.06.4 released

I today released version 0.06.4 of my WAP WML browser wApua and also uploaded that release to Debian Unstable. It s a bugfix release and the first upstream release since 2017. It fixes the recognition of WAP WML pages with more recent DTD location URLs ending in .dtd instead of .xml (and some other small difference). No idea when these URLs changed, but I assume they have been changed to look more like the URLs of other DTDs. The old URLs of the DTD still work, but more recent WAP pages (yes, they do exist :-) seem to use the new DTD URLs, so there was a need to recognise them instead of throwing an annoying warning. Thanks to Lian Begett for the bug report!

15 February 2021

Axel Beckert: Starting a GNU Screen session via SSH's ~/.ssh/config

This is more or less a followup to this blog posting of mine about AutoSSH and GNU Screen from nearly ten years ago which by the way is still valid and still the way, I use SSH and GNU Screen. Recently a friend asked me how to automatically start or reconnect to a GNU Screen session directly via OpenSSH s configuration file. Here s how to do it: Add an entry to ~/.ssh/config similar to this one:
Host screen_on_server
    Hostname server.example.org
    RequestTTY yes
    RemoteCommand screen -RD
and then just call ssh screen_on_server and you ll get connected to an existing screen session if present, otherwise you ll a new new one. Should work with tmux, too, but might need commandlien different options.

12 October 2020

Axel Beckert: Git related shell aliases I commonly use

Hope this might be an inspiration to use these or similar aliases as well.

27 June 2020

Russell Coker: Links June 2020

Bruce Schneier wrote an informative post about Zoom security problems [1]. He recommends Jitsi which has a Debian package of their software and it s free software. Axel Beckert wrote an interesting post about keyboards with small numbers of keys, as few as 28 [2]. It s not something I d ever want to use, but interesting to read from a computer science and design perspective. The Guardian has a disturbing article explaining why we might never get a good Covid19 vaccine [3]. If that happens it will change our society for years if not decades to come. Matt Palmer wrote an informative blog post about private key redaction [4]. I learned a lot from that. Probably the simplest summary is that you should never publish sensitive data unless you are certain that all that you are publishing is suitable, if you don t understand it then you don t know if it s suitable to be published! This article by Umair Haque on eand.co has some interesting points about how Freedom is interpreted in the US [5]. This article by Umair Haque on eand.co has some good points about how messed up the US is economically [6]. I think that his analysis is seriously let down by omitting the savings that could be made by amending the US healthcare system without serious changes (EG by controlling drug prices) and by reducing the scale of the US military (there will never be another war like WW2 because any large scale war will be nuclear). If the US government could significantly cut spending in a couple of major areas they could then put the money towards fixing some of the structural problems and bootstrapping a first-world economic system. The American Conservatrive has an insightful article Seven Reasons Police Brutality is Systemic Not Anecdotal [7]. Scientific American has an informative article about how genetic engineering could be used to make a Covid-19 vaccine [8]. Rike wrote an insightful post about How Language Changes Our Concepts [9]. They cover the differences between the French, German, and English languages based on gender and on how the language limits thoughts. Then conclude with the need to remove terms like master/slave and blacklist/whitelist from our software, with a focus on Debian but it s applicable to all software. Gunnar Wolf also wrote an insightful post On Masters and Slaves, Whitelists and Blacklists [10], they started with why some people might not understand the importance of the issue and then explained some ways of addressing it. The list of suggested terms includes Primary-secondary, Leader-follower, and some other terms which have slightly different meanings and allow more precision in describing the computer science concepts used. We can be more precise when describing computer science while also not using terms that marginalise some groups of people, it s a win-win! Both Rike and Gunnar were responding to a LWN article about the plans to move away from Master/Slave and Blacklist/Whitelist in the Linux kernel [11]. One of the noteworthy points in the LWN article is that there are about 70,000 instances of words that need to be changed in the Linux kernel so this isn t going to happen immediately. But it will happen eventually which is a good thing.

30 March 2020

Axel Beckert: How do you type on a keyboard with only 46 or even 28 keys?

Some of you might have noticed that I m into keyboards since a few years ago into mechanical keyboards to be precise. Preface It basically started with the Swiss Mechanical Keyboard Meetup (whose website I started later on) was held in the hackerspace of the CCCZH. I mostly used TKL keyboards (i.e. keyboards with just the for me useless number block missing) and tried to get my hands on more keyboards with Trackpoints (but failed so far). At some point a year or two ago, I looking into smaller keyboards for having a mechanical keyboard with me when travelling. I first bought a Vortex Core at Candykeys. The size was nice and especially having all layers labelled on the keys was helpful, but nevertheless I soon noticed that the smaller the keyboards get, the more important is, that they re properly programmable. The Vortex Core is programmable, but not the keys in the bottom right corner which are exactly the keys I wanted to change to get a cursor block down there. (Later I found out that there are possibilities to get this done, either with an alternative firmware and a hack of it or desoldering all switches and mounting an alternative PCB called Atom47.) 40% Keyboards So at some point I ordered a MiniVan keyboard from The Van Keyboards (MiniVan keyboards will soon be available again at The Key Dot Company), here shown with GMK Paperwork (also bought from and designed by The Van Keyboards):
The MiniVan PCBs are fully programmable with the free and open source firmware QMK and started to use that more and more instead of bigger keyboards. Layers With the MiniVan I learned the concepts of layers. Layers are similar to what many laptop keyboards do with the Fn key and to some extent also what the German standard layout does with the AltGr key: Layers are basically alternative key maps you can switch with a special key (often called Fn , Fn1 , Fn2 , etc., or especially if there are two additional layers Raise and Lower ). There are several concepts how these layers can be reached with these keys: My MiniVan Layout For the MiniVan, two additional layers suffice easily, but since I have a few characters on multiple layers and also have mouse control and media keys crammed in there, I have three additional layers on my MiniVan keyboards:

TRNS means transparent, i.e. use the settings from lower layers.
I also use a feature that allows me to mind different actions to a key depending if I just tap the key or if I hold it. Some also call this tap dance . This is especially very popular on the usually rather huge spacebar. There, the term SpaceFn has been coined, probably after this discussion on Geekhack. I use this for all my layer switching keys: With this layout I can type English texts as fast as I can type them on a standard or TKL layout. German umlauts are a bit more difficult because it requires 4 to 6 key presses per umlaut as I use the Compose key functionality (mapped to the Menu key between the spacebars and the cursor block. So to type an on my MiniVan, I have to:
  1. press and release Menu (i.e. Compose); then
  2. press and hold either Shift-Spacebar (i.e. Shift-Fn1) or Slash (i.e. Fn2), then
  3. press N for a double quote (i.e. Shift-Fn1-N or Fn2-N) and then release all keys, and finally
  4. press and release the base character for the umlaut, in this case Shift-A.
And now just use these concepts and reduce the amount of keys to 28: 30% and Sub-30% Keyboards In late 2019 I stumbled upon a nice little keyboard kit shop on Etsy which I (and probably most other people in the mechanical keyboard scene) didn t take into account for looking for keyboards called WorldspawnsKeebs. They offer mostly kits for keyboards of 40% size and below, most of them rather simple and not expensive. For about 30 you get a complete sub-30% keyboard kit (without switches and keycaps though, but that very common for keyboard kits as it leaves the choice of switches and key caps to you) named Alpha28 consisting of a minimal Acrylic case and a PCB and electronics set. This Alpha28 keyboard is btw. fully open source as the source code, (i.e. design files) for the hardware are published under a free license (MIT license) on GitHub. And here s how my Alpha28 looks like with GMK Mitolet (part of the GMK Pulse group-buy) key caps:
So we only have character keys, Enter (labelled Data as there was no 1u Enter key with that row profile in that key cap set; I ll also call it Data for the rest of this posting) and a small spacebar, not even modifier keys. The Default Alpha28 Layout The original key layout by the developer of the Alpha28 used the spacbar as Shift on hold and as space if just tapped, and the Data key switches always to the next layer, i.e. it switches the layer permanently on tap and not just on hold. This way that key rotates through all layers. In all other layers, V switches back to the default layer. I assume that the modifiers on the second layer are also on tap and apply to the next other normal key. This has the advantage that you don t have to bend your fingers for some key combos, but you have to remember on which layer you are at the moment. (IIRC QMK allows you to show that via LEDs or similar.) Kinda just like vi. My Alpha28 Layout But maybe because I m more an Emacs person, I dislike remembering states myself and don t bind bending my fingers. So I decided to develop my own layout using tap-or-hold and only doing layer switches by holding down keys:

A triangle means that the settings from lower layers are used, N/A means the key does nothing.
It might not be very obvious, but on the default layer, all keys in the bottom row and most keys on the row ends have tap-or-hold configurations. Basic ideasBottom row if holdOther rows if holdHow the keys are divided into layersUsing the Alpha28 This layout works surprisingly well for me. Only for Minus, Equal, Single Quote and Semicolon I still often have to think or try if they re on Layer 1 or 2 as on my 40%s (MiniVan, Zlant, etc.) I have them all on layer 1 (and in general one layer less over all). And for really seldom used keys like Insert, PrintScreen, ScrollLock or Pause, I might have to consult my own documentation. They re somewhere in the middle of the keyboard, either on layer 1, 2, or 3. ;-) And of course, typing umlauts takes even two keys more per umlaut as on the MiniVan since on the one hand Menu is not on the default layer and on the other hand, I don t have this nice shifted number row and actually have to also press Shift to get a double quote. So to type an on my Alpha, I have to:
  1. press and release Space-F (i.e. Fn1-F) for Menu (i.e. Compose); then
  2. press and hold A-Spacebar-L (i.e. Shift-Fn1-L) for getting a double quote, then
  3. press and release the base character for the umlaut, in this case L-A for Shift-A (because we can t use A for Shift as I can t hold a key and then press it again :-).
Conclusion If the characters on upper layers are not labelled like on the Vortex Core, i.e. especially on all self-made layouts, typing is a bit like playing that old children s game Memory: as soon as you remember (or your muscle memory knows) where some special characters are, typing gets faster. Otherwise, you start with trial and error or look the documentation. Or give up. ;-) Nevertheless, typing on a sub-30% keyboard like the Alpha28 is much more difficult and slower than on a 40% keyboard like the MiniVan. So the Alpha28 very likely won t become my daily driver while the MiniVan defacto is my already my daily driver. But I like these kind of challenges as others like the game Memory . So I ordered three more 30% and sub-30% keyboard kits and WorldspawnsKeebs for soldering on the upcoming weekend during the COVID19 lockdown: And if I at some point want to try to type with even fewer keys, I ll try a Butterstick keyboard with just 20 keys. It s a chorded keyboard where you have to press multiple keys at the same time to get one charcter: So to get an A from the missing middle row, you have to press Q and Z simultaneously, to get Escape, press Q and W simultaneously, to get Control, press Q, W, Z and X simultaneously, etc. And if that s not even enough, I already bought a keyboard kit named Ginny (or Ginni, the developer can t seem to decide) with just 10 keys from an acquaintance. Couldn t resist when offered his surplus kits. :-) It uses the ASETNIOP layout which was initially developed for on-screen keyboards on tablets.

26 March 2020

Axel Beckert: Pictures in pure HTML with chafa and aha

I recently stumbled upon chafa, a tool to display pictures, especially color pictures on your ANSI text terminal, e.g. inside an xterm. And I occasionally use aha, the Ansi HTML Adapter to convert a colorful terminal content into HTML to show off terminal screenshots without the requirement of a picture so that it also works in e.g. text browsers or for blinds. Combining chafa and aha: Examples A moment ago I had the thought what would happen if I feed the output of chafa into aha and expected nothing really usable. But I was surprised by the quality of the outcome.
looks like this after chafa -w 9 -c full -s 160x50 DSCN4692.jpg aha -n:
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
                                                                                                                                     
Checking the Look in Text Browsers It even looks not that bad in elinks as far as I know the only text browser which supports CSS and styles:
In Lynx and Links 2, the text composing the image is displayed only in black and white, but you at least can recognise the edges in the picture:
Same Functionality in One Tool? I knew there was a tool which did this in one step. Seems to have been png2html. Tried to play around with it, too, but neither really understood how to use it (seems to require a text file for the characters to be used why?) nor did I really got it working. It always ran until I aborted it and it never filled the target file with any content. Additionally, png2html insists on one character per pixel, requiring to first properly resize the image before converting to HTML. The Keyboard in the Pictures Oh, and btw., the displayed keyboard is my Zlant. The Zlant is a 40% uniform staggered mechanical keyboard. Currently, only Zlant PCBs are available at 1UP Keyboards (USA), i.e. no complete kits. It is shown with the SA Vilebloom key cap set, currently available at MechSupply (UK).

11 March 2020

Axel Beckert: Backup over Tor with BackupPC

I have a Raspberry Pi at my parents home. They have internet access via some ISP using Carrier Grade NAT (CGN). Hence their home router is not reachable via IPv4 from the outside, they do have IPv6 and the devices can also be made accessible via IPv6 via the local router. Did that, was able to access my Raspberry Pi over IPv6 and SSH from the outside. So doing backup of that Raspberry Pi with BackupPC from the outside was a walk in the park. Unfortunately the IPv6 prefix seems to change occasionally and the router only allows to configure explicit IPv6 addresses in firewall rules so after a prefix change the configured rules no more match the devices IPv6 addresses. Meh. So there were multiple possibilities to work around these restrictions and access a devices behind the router: Enabling a Tor Hidden Service for port 22 is a no-brainer and was done most quickly (actually it already was in place as I already suspected that an IPv6 prefix change might happen) and I so far was too lazy to replace it with something more proper. But my backup was relying on direct SSH access via IPv6. So I needed to get that working over Tor, too. Here s what was needed for the host named sherpa (named after the Fiberfab Sherpa) to be backed up via Tor: That s it basically. Of course you also need to have the SSH public host key in the .ssh/known_hosts file also for the .onion hostname. And the Tor Hidden Service needs to be configured on the target device. But that s left as exercise for the reader. There s a lot of documentation about that on the internet, including slides and video recordings of talks and live demos I gave about this topic in German. Ah, and in case you might think that s unfair and misuse of the resources of the Tor Project: No, I explicitly asked and they said more or less any additional traffic helps to make it more difficult to analyse Tor traffic or to track Tor users and is hence welcome. Addendum: The last direct full backup of that Raspberry Pi (5.5 GB) took around 32 minutes. The first full backup over Tor (8.7 GB) took 341 minutes. Seems much slower, but there might be other factors as well: Most backups which ran last night were running at only 0.85 MB/s to 1 MB/s, probably because too many backups were running in parallel after a recent backup server downtime with file system check the backup server was probably the bottleneck. The backup of the Raspberry Pi over Tor ran at 0.42 MB/s, so about half the speed of the other backups. (Will probably add some more notes if I have more statistics over time.)

21 June 2017

Reproducible builds folks: Reproducible Builds: week 112 in Stretch cycle

Here's what happened in the Reproducible Builds effort between Sunday June 11 and Saturday June 17 2017: Upcoming events Upstream patches and bugs filed Reviews of unreproducible packages 1 package review has been added, 19 have been updated and 2 have been removed in this week, adding to our knowledge about identified issues. Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by: diffoscope development tests.reproducible-builds.org As you might have noticed, Debian stretch was released last week. Since then, Mattia and Holger renamed our testing suite to stretch and added a buster suite so that we keep our historic results for stretch visible and can continue our development work as usual. In this sense, happy hacking on buster; may it become the best Debian release ever and hopefully the first reproducible one! Axel Beckert is currently in the process of setting up eight LeMaker HiKey960 boards. These boards were sponsored by Hewlett Packard Enterprise and will be hosted by the SOSETH students association at ETH Zurich. Thanks to everyone involved here and also thanks to Martin Michlmayr and Steve Geary who initiated getting these boards to us. Misc. This week's edition was written by Chris Lamb, Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

3 May 2017

Axel Beckert: Upcoming Hacker/FLOSS Events in Switzerland: Debian BSP, LPD, Hackerfunk 10th Anniversary, ZeTeCo

There are quite some events and dates ahead for hackers, makers, debianers and hackerfunk listeners: Crowdfunding for ZeTeCo Camp in July Ends in Two Days! You might have heard of the ZeTeCo Camp near Schaffhausen in July. If you want ot come or at least support that event, please contribute to their crowdfunding campaign. They have more than 90% of their goal funded and there s less only about two days left to reach their funding goal. If it doesn t get funded in time, the event may be be on a knife edge. Debian Bug Squashing Party in Zurich this Weekend One week before the More-than-a-BSP at the Mozilla office in Paris there will also be a Debian Bug Squashing Party (BSP) in Z rich at the CCCZH Hackerspace R schtibach . We ll start on Friday, the 5th of May 2017 in the late afternoon, probably around 4pm or 5pm, and will end on Sunday, the 7th of May 2017 also in the late afternoon. Please add yourself to the according section on the BSP s wiki page if you want to join us to squash the hopefully not that many left over bugs in testing. Unfortunately we didn t notice two date clashes when we set the date for the BSP during the Annual General Meeting (AGM) of the Debian.ch Association earlier this year: Linux Presentation Day this Saturday, 6th of May 2017 On the one hand there will be the Swiss Edition of the Linux Presentation Day (LPD) on Saturday, the 6th of May 2017 including the LPD in Z rich. The latter will take place in the same building as the BSP, just on a different floor: The BSP will be at floor B3 in the CCCZH Hackerspace and the LPD will be on the ground floor at Revamp-IT in the former ZKB foyer. 10 Years Hackerfunk: Show on 6th, Party on 13th of May 2017 And on the other hand, Venty s and my (German dialect) radio show and podcast Hackerfunk will have it s 10th anniversary show also on that Saturday. So I ll vanish from BSP for a few hours on Saturday evening for broadcasting this very special Hackerfunk episode on Radio Radius. But since Venty and me didn t want to make yet another big event at R schtibach on the same weekend, we ll do the Hackerfunk 10th Anniversary Party one weekend later on Saturday the 13th of May 2017 also at the CCCZH Hackerspace R schtibach . A separate announcement on https://www.hackerfunk.ch/ (also in the RSS feed there) will follow.

28 March 2017

Axel Beckert: System Tray Icon to Monitor a Linux Software RAID Locally

About a year ago I bought a new workstation computer for myself at home. It s a Tuxedo XUX_Cube which is advertised as gaming PC. But I ordered a slightly atypical non-gamer configuration: Of course the box runs Debian. To be more precise, it runs Debian Sid with sysvinit-core as init system and i3 as window manager. As I usually have no monitoring clients on my laptops and private workstations, I rather often felt the urge to do a cat /proc/mdstat on that box. So at some point I wanted something like smart-notifier, but for Linux Software (MD) RAIDs. And since I found nothing, I did what Open Source guys usually do in such cases: I wrote it myself of course in Perl and called it systray-mdstat. First I wondered about which build system would be most suitable for that task, but in the end I once again went with Dist::Zilla for the upstream build system and hence dh-dist-zilla for the Debian packaging. Ideas for the actual implementation were taken from Wouter s fdpowermon for the systray icon framework in Perl and Myon s mdstat Xymon plugin for an already proven logic to parse /proc/mdstat. (Both, Wouter and Myon have stated in a GnuPG-signed e-mail that I copied less code than would validate their copyrights, so I was able to license it under a single license, namely GNU GPL version 3.) As of now, systray-mdstat is also available as package in Debian Unstable. It won t make it to Stretch as its first line of code has been written after the soft-freeze for Stretch was already in place.

Axel Beckert: Maintaining Debian Packages of Perl Modules with dh-dist-zilla

Maintaining Debian packages of Perl modules usually can be done with the common git-buildpackage (aka gbp) workflow with its three git branches master (or debian), upstream and pristine-tar: This also works more or less fine for Perl modules, where the Debian package maintainer is also the upstream developer. In that case mostly the upstream branch is used (and then maybe called master while the Debian packaging branch is then called debian). But the files needed for a proper so called CPAN distribution of a Perl module often contain redundant information (version numbers, required modules, etc.) which needs to be maintained. And for that, many people prefer Don t Repeat Yourself (DRY) as a principle. Dist::Zilla One nice and common tool for that is Dist::Zilla or short dzil. It generates most redundant but required data out of a central source, e.g. Dist::Zilla s dist.ini or the contained .pm files, etc. dzil build creates tar ball which contains all files necessary by CPAN. But now we have a dilemma: Debian expects those generated files inside the upstream branch while the files are only generated from other files in that branch. There are multiple solutions, but all of them involve committing generated files to the git repository: librun-parts-perl aka Run::Parts (a Perl wrapper around and a pure-perl implementation of Debian s run-parts tool) was initially maintained in the latter way. But especially in cases where we just need a Perl module packaged as .deb without uploading it to CPAN (e.g. project-internal modules), this is a tedious workflow and overkill. It would be much nicer if debhelper would just call dzil to generate all the stuff it needs to build the package. dh-dist-zilla Well, you can do that now, at least with Debian Jessie. This is what dh-dist-zilla does: It is a debhelper sequence plugin which calls dzil build and dzil clean in the right moment and takes care that all dh_auto_* commands look in the directory with the generated files instead of the rather clean project root directory. To use dh-dist-zilla, you just need to add a build-dependency on it and the Dist::Zilla plugins you use, and add --with dist-zilla to your minimal dh-style debian/rules file:
#!/usr/bin/make -f
%:
	dh $@ --with dist-zilla
That s it. With regards to workflow and git branches, you may still want to use separate branches for upstream work and debian work, and you may want to continue to use pristine-tar, but you don t have to commit generated files to git anymore and you can maintain a clean master branch with nearly no redundancy. And if you need to generate to final upstream tar ball for you debian package, just call dh get-orig-source or maybe easier to use with tab completion dh_dist_zilla_origtar. This is how the librun-parts-perl package is maintained nowadays. There s otherwise not much difference to the old, classically maintained versions. More DRY Next step in the DRY evolution is to reduce redundancies between upstream (Dist::Zilla based) packaging and the Debian packaging. There are a few tools available, partially brand new, partially not yet packaged: I wouldn t be surprised if there s more to come in this area. P.S.: I actually started this blog posting in September 2014 and never finished it until now. Had to kick out some already outdated again stuff, but also could add some more recent things.

21 July 2016

Reproducible builds folks: Reproducible builds: week 62 in Stretch cycle

What happened in the Reproducible Builds effort between June 26th and July 2nd 2016: Read on to find out why we're lagging some weeks behind ! GSoC and Outreachy updates Toolchain fixes With the doxygen upload we are now down to only 2 modified packages in our repository: dpkg and rdfind. Weekly reports delay and the future of statistics To catch up with our backlog of weekly reports we have decided to skip some of the statistics for this week. We might publish them in a future report, or we might switch to a format where we summarize them more (and which we can create (even) more automatically), we'll see. We are doing these weekly statistics because we believe it's appropriate and useful to credit people's work and make it more visible. What do you think? We would love to hear your thoughts on this matter! Do you read these statistics? Somewhat? Actually, thanks to the power of notmuch, Holger came up with what you can see below, so what's missing for this week are the uploads fixing irreprodubilities. Which we really would like to show for the reasons stated above and because we really really need these uploads to happen ;-) But then we also like to confirm the bugs are really gone, which (atm) requires manual checking, and to look for the words "reproducible" and "deterministic" (and spelling variations) in debian/changelogs of all uploads, to spot reproducible work not tracked via the BTS. And we still need to catch up on the backlog of weekly reports. Bugs submitted with reproducible usertags It seems DebCamp in Cape Town was hugely successful and made some people get a lot of work done: 61 bugs have been filed with reproducible builds usertags and 60 of them had patches: Package reviews 437 new reviews have been added (though most of them were just linking the bug, "only" 56 new issues in packages were found), an unknown number has been been updated and 60 have been removed in this week, adding to our knowledge about identified issues. 4 new issue types have been found: Weekly QA work 98 FTBFS bugs have been reported by Chris Lamb and Santiago Vila. diffoscope development strip-nondeterminism development tests.reproducible-builds.org Misc. This week's edition was written by Mattia Rizzolo, Reiner Herrmann, Ceridwen and Holger Levsen and reviewed by a bunch of Reproducible builds folks on IRC.

18 June 2016

Manuel A. Fernandez Montecelo: More work on aptitude

The last few months have been a bit of a crazy period of ups and downs, with a tempest of events beneath the apparent and deceivingly calm surface waters of being unemployed (still at it). The daily grind Chief activities are, of course, those related to the daily grind of job-hunting, sending applications, and preparing and attending interviews. It is demoralising when one searches for many days or weeks without seeing anything suitable for one's skills or interests, or other more general life expectations. And it takes a lot of time and effort to put one's best in the applications for positions that one is really, really, interested in. And even for the ones which are meh, for a variety of reasons (e.g. one is not very suitable for what the offer demands). After that, not being invited to interviews (or doing very badly at them) is bad, of course, but quick and not very painful. A swift, merciful end to the process. But it's all the more draining when waiting for many weeks when not a few months with the uncertainty of not knowing if one is going to be lucky enough to be summoned for an interview; harbouring some hope one has to appear enthusiastic in the interviews, after all , while trying to keep it contained lest it grows too much ; then in the interview hearing good words and some praises, and feeling the impression that one will fit in, that one did nicely and that chances are good letting the hope grow again ; start to think about life changes that the job will require to make a quick decision should the offer finally arrives ; perhaps make some choices and compromises based on the uncertain result; then wait for a week or two after the interview to know the result... ... only to end up being unsuccessful. All the effort and hopes finally get squashed with a cold, short email or automatic response, or more often than not, complete radio silence from prospective employers, as an end to a multi-month-long process. An emotional roller coaster [1], which happened to me several times in the last few months. All in a day's work The months of preparing and waiting for a new job often imply an impasse that puts many other things that one cares about on hold, and one makes plans that will never come to pass. All in a day's (half-year's?) work of an unemployed poor soul. But not all is bad. This period was also a busy time doing some plans about life, mid- and long-term; the usual and some really unusual! family events; visits to and from friends, old and new; attending nice little local Debian gatherings or the bigger gathering of Debian SunCamp2016, and other work for side projects or for other events that will happen soon... And amidst all that, I managed to get some work done on aptitude. Two pictures worth (less than) a thousand bugs To be precise, worth 709 bugs 488 bugs in the first graph, plus 221 in the second. In 2015-11-15 (link to the post Work on aptitude): aptitude BTS Graph, 2015-11-15 In 2016-06-18: aptitude BTS Graph, 2016-06-18 Numbers The BTS numbers for aptitude right now are: Highlights Beyond graphs and stats, I am specially happy about two achievements in the last year:
  1. To have aptitude working today, first and foremost Apart from the abandon that suffered in previous years, I mean specifically the critical step of getting it through the troubles of the last summer, with the GCC-5/C++11 transition in parallel with a transition of the Boost library (explained in more detail in Work on aptitude). Without that, possibly aptitude would not have survived until today.
  2. Improvements to the suggestions of the resolver In the version 0.8, there were a lot of changes related with improving the order of the suggestions from the resolver, when it finds conflicts or other problems with the planned actions. Historically, but specially in the last few years, there have been many complaints about the nonsensical or dangerous suggestions from the resolver. The first solution offered by the resolver was very often regarded as highly undesirable (for example, removal of many packages), and preferable solutions like upgrades of one or only a handful of packages being offered only after many removals; and keeps only offered as last resort.
Perhaps these changes don't get a lot of attention, given that in the first case it's just to keep working (with few people realising that it could have collapsed on the spot, if left unattended), and the second can probably go unnoticed because it just works or it started to work more smoothly doesn't get as much immediate attention as it suddenly broke! . Still, I wanted to mention them, because I am quite proud of those. Thanks Even if I put a lot of work on aptitude in the last year, the results of the graph and numbers have not been solely achieved by me. Special thanks go to Axel Beckert (abe / XTaran) and the apt team, David Kalnischkies and Julian Andres Klode who, despite the claim in that page, does not mostly work python-apt anymore... but also in the main tools. They help with fixing some of the issues directly, or changing things in apt that benefit aptitude, testing changes, triaging bugs or commenting on them, patiently explaining to me why something in libapt doesn't do what I think it does, and good company in general. Not the least, for holding impromptu BTS group therapy / support meetings, for those cases when prolonged exposure to BTS activity starts to induce very bad feelings. Thanks also to people who sent their translation updates, notified about corrections, sent or tested patches, submitted bugs, or tried to help in other ways. Change logs for details. Notes [1] ^ It's even an example in the Cambridge Dictionaries Online website, for the entry of roller coaster:
He was on an emotional roller coaster for a while when he lost his job.

30 May 2016

Reproducible builds folks: Reproducible builds: week 57 in Stretch cycle

What happened in the Reproducible Builds effort between May 22nd and May 28th 2016: Media coverage Documentation update Toolchain fixes Packages fixed The following 18 packages have become reproducible due to changes in their build dependencies: canl-c configshell dbus-java dune-common frobby frown installation-guide jexcelapi libjsyntaxpane-java malaga octave-ocs paje.app pd-boids pfstools r-cran-rniftilib scscp-imcce snort vim-addon-manager The following packages have become reproducible after being fixed: Some uploads have fixed some reproducibility issues, but not all of them: Patches submitted that have not made their way to the archive yet: Package reviews 123 reviews have been added, 57 have been updated and 135 have been removed in this week. 21 FTBFS bugs have been reported by Chris Lamb and Santiago Vila. strip-nondeterminism development tests.reproducible-builds.org Misc. This week's edition was written by Reiner Herrmann and Holger Levsen and reviewed by a bunch of Reproducible builds folks on IRC.

17 May 2016

Reproducible builds folks: Reproducible builds: week 55 in Stretch cycle

What happened in the Reproducible Builds effort between May 8th and May 14th 2016: Documentation updates Toolchain fixes Packages fixed The following 28 packages have become newly reproducible due to changes in their build dependencies: actor-framework ask asterisk-prompt-fr-armelle asterisk-prompt-fr-proformatique coccinelle cwebx d-itg device-tree-compiler flann fortunes-es idlastro jabref konclude latexdiff libint minlog modplugtools mummer mwrap mxallowd mysql-mmm ocaml-atd ocamlviz postbooks pycorrfit pyscanfcs python-pcs weka The following 9 packages had older versions which were reproducible, and their latest versions are now reproducible again due to changes in their build dependencies: csync2 dune-common dune-localfunctions libcommons-jxpath-java libcommons-logging-java libstax-java libyanfs-java python-daemon yacas The following packages have become newly reproducible after being fixed: The following packages had older versions which were reproducible, and their latest versions are now reproducible again after being fixed: Some uploads have fixed some reproducibility issues, but not all of them: Patches submitted that have not made their way to the archive yet: Package reviews 344 reviews have been added, 125 have been updated and 20 have been removed in this week. 14 FTBFS bugs have been reported by Chris Lamb. tests.reproducible-builds.org Misc. Dan Kegel sent a mail to report about his experiments with a reproducible dpkg PPA for Ubuntu. According to him sudo add-apt-repository ppa:dank/dpkg && sudo apt-get update && sudo apt-get install dpkg should be enough to get reproducible builds on Ubuntu 16.04. This week's edition was written by Ximin Luo and Holger Levsen and reviewed by a bunch of Reproducible builds folks on IRC.

10 May 2016

Reproducible builds folks: Reproducible builds: week 54 in Stretch cycle

What happened in the Reproducible Builds effort between May 1st and May 7th 2016: Media coverage There has been a surprising tweet last week: "Props to @FiloSottile for his nifty gvt golang tool. We're using it to get reproducible builds for a Zika & West Nile monitoring project." and to our surprise Kenn confirmed privately that he indeed meant "reproducible builds" as in "bit by bit identical builds". Wow. We're looking forward to learn more details about this; for now we just know that they are doing this for software quality reasons basically. Two of the four GSoC and Outreachy participants for Reproducible builds posted their introductions to Planet Debian: Toolchain fixes and other upstream developments dpkg 1.18.5 was uploaded fixing two bugs relevant to us: This upload made it necessary to rebase our dpkg on the version on sid again, which Niko Tyni and Lunar promptly did. Then a few days later 1.18.6 was released to fix a regression in the previous upload, and Niko promptly updated our patched version again. Following this Niko Tyni found #823428: "dpkg: many packages affected by dpkg-source: error: source package uses only weak checksums". Alexis Bienven e worked on tex related packages and SOURCE_DATE_EPOCH: Emmanuel Bourg uploaded jflex/1.4.3+dfsg-2, which removes timestamps from generated files. Packages fixed The following 285 packages have become reproducible due to changes in their build dependencies (mostly from GCC honouring SOURCE_DATE_EPOCH, see the previous week report): 0ad abiword abcm2ps acedb acpica-unix actiona alliance amarok amideco amsynth anjuta aolserver4-nsmysql aolserver4-nsopenssl aolserver4-nssqlite3 apbs aqsis aria2 ascd ascii2binary atheme-services audacity autodocksuite avis awardeco bacula ballerburg bb berusky berusky2 bindechexascii binkd boinc boost1.58 boost1.60 bwctl cairo-dock cd-hit cenon.app chipw ckermit clp clustalo cmatrix coinor-cbc commons-pool cppformat crashmail crrcsim csvimp cyphesis-cpp dact dar darcs darkradiant dcap dia distcc dolphin-emu drumkv1 dtach dune-localfunctions dvbsnoop dvbstreamer eclib ed2k-hash edfbrowser efax-gtk efax exonerate f-irc fakepop fbb filezilla fityk flasm flightgear fluxbox fmit fossil freedink-dfarc freehdl freemedforms-project freeplayer freeradius fxload gdb-arm-none-eabi geany-plugins geany geda-gaf gfm gif2png giflib gifticlib glaurung glusterfs gnokii gnubiff gnugk goaccess gocr goldencheetah gom gopchop gosmore gpsim gputils grcompiler grisbi gtkpod gvpe hardlink haskell-github hashrat hatari herculesstudio hpcc hypre i2util incron infiniband-diags infon ips iptotal ipv6calc iqtree jabber-muc jama jamnntpd janino jcharts joy2key jpilot jumpnbump jvim kanatest kbuild kchmviewer konclude krename kscope kvpnc latexdiff lcrack leocad libace-perl libcaca libcgicc libdap libdbi-drivers libewf libjlayer-java libkcompactdisc liblscp libmp3spi-java libpwiz librecad libspin-java libuninum libzypp lightdm-gtk-greeter lighttpd linpac lookup lz4 lzop maitreya meshlab mgetty mhwaveedit minbif minc-tools moc mrtrix mscompress msort mudlet multiwatch mysecureshell nifticlib nkf noblenote nqc numactl numad octave-optim omega-rpg open-cobol openmama openmprtl openrpt opensm openvpn openvswitch owx pads parsinsert pcb pd-hcs pd-hexloader pd-hid pd-libdir pear-channels pgn-extract phnxdeco php-amqp php-apcu-bc php-apcu php-solr pidgin-librvp plan plymouth pnscan pocketsphinx polygraph portaudio19 postbooks-updater postbooks powertop previsat progressivemauve puredata-import pycurl qjackctl qmidinet qsampler qsopt-ex qsynth qtractor quassel quelcom quickplot qxgedit ratpoison rlpr robojournal samplv1 sanlock saods9 schism scorched3d scummvm-tools sdlbasic sgrep simh sinfo sip-tester sludge sniffit sox spd speex stimfit swarm-cluster synfig synthv1 syslog-ng tart tessa theseus thunar-vcs-plugin ticcutils tickr tilp2 timbl timblserver tkgate transtermhp tstools tvoe ucarp ultracopier undbx uni2ascii uniutils universalindentgui util-vserver uudeview vfu virtualjaguar vmpk voms voxbo vpcs wipe x264 xcfa xfrisk xmorph xmount xyscan yacas yasm z88dk zeal zsync zynaddsubfx Last week the 1000th bug usertagged "reproducible" was fixed! This means roughly 2 bugs per day since 2015-01-01. Kudos and huge thanks to everyone involved! Please also note: FTBFS packages have not been counted here and there are still 600 open bugs with reproducible patches provided. Please help bringing that number down to 0! The following packages have become reproducible after being fixed: Some uploads have fixed some reproducibility issues, but not all of them: Uploads which fix reproducibility issues, but currently FTBFS: Patches submitted that have not made their way to the archive yet: Package reviews 54 reviews have been added, 6 have been updated and 44 have been removed in this week. 18 FTBFS bugs have been reported by Chris Lamb, James Cowgill and Niko Tyni. diffoscope development Thanks to Mattia, diffoscope 52~bpo8+1 is available in jessie-backports now. tests.reproducible-builds.org Misc. This week's edition was written by Reiner Herrmann, Holger Levsen and Mattia Rizzolo and reviewed by a bunch of Reproducible builds folks on IRC. Mattia also wrote a small ikiwiki macro for this blog to ease linking reproducible issues, packages in the package tracker and bugs in the Debian BTS.

26 April 2016

Niels Thykier: Putting Debian packages in labelled boxes

Lintian 2.5.44 was released the other day and (to most) the most significant bug fix was probably that Lintian learned about Policy 3.9.8. I would like to thank Axel Beckert for doing that. Notably it also made me update the test suite so to make future policy releases less painful. For others, it might be the fact that Lintian now accepts (valid) versioned provides (which seemed prudent now that Britney accepts them as well). Newcomers might appreciate that we are giving a much more sensible warning when they have extra spaces in their changelog sign off line (rather than pretending it is an improper NMU). But I digress What I am here to talk about is that Lintian 2.5.44 started classifying packages based on various facts or properties , we can determine. Therefore: Here are some of the labelled boxes your packages will be put into[0]: The tags themselves are (as mentioned) mere classifications and their primary purpose is to classify or measure certain properties. With them any body can download the data set and come with some bold statement about Debian packages (hopefully without relying too much on lies, damned lies and statistics ). Lets try that immediately! In the next release, we will also add tracking of auto-generated snippets from dh_*-tools. Currently unversioned, but I hope to add versioning to that so we can find and rebuild packages that have been built with buggy autoscripts (like #788098) If you want to see the classification tags for your package, please run lintian with like this:
# Add classification tags
$ lintian -L +classification <pkg-or-changes>
# Or if you want only classification tags$ lintian -L =classification <pkg-or-changes>
Please keep in mind that classification tags ( C ) are not issues in themselves. Lintian is simply attempting to add a visible indicator about a given fact or property in the package nothing more, nothing less. Future work help (read: patches) welcome: [0] Mind you, the reporting framework s handling of these tags could certainly be improved. [1] Please note how it distinguishes 1.0 into native and non-native based on whether the package has a diff.gz. Presumably that can be exploited somehow [2] Disclaimer: At the time of writing, only ~80% of the archive have been processed. This is computed as: NS / (NS + WS), where NS and WS are the number of unique packages with the tags no-ctrl-scripts and ctrl-script respectively. [3] or maybe not, but we got two packages classified as using both CDBS and the dh-sequencer. I have not looked at it in detail. For the curious: libmecab-java and ctioga2.
Filed under: Debian, Lintian

21 February 2016

Lunar: Reproducible builds: week 43 in Stretch cycle

What happened in the reproducible builds effort between February 14th and February 20th 2016:

Toolchain fixes Yaroslav Halchenko uploaded cython/0.23.4+git4-g7eed8d8-1 which makes its output deterministic. Original patch by Chris Lamb. Didier Raboud uploaded pyppd/1.0.2-3 to experimental which now serialize PPD deterministically. Lunar submitted two patches for lcms to add a way for clients to set the creation date/time in profile headers and initialize all bytes when writing named colors.

Packages fixed The following packages have become reproducible due to changes in their build dependencies: dbconfig-common, dctrl-tools, dvdwizard, ekg2, expeyes, galternatives, gpodder, icewm, latex-mk, libiio, lives, navit, po4a, tasksel, tilda, vdr-plugin-infosatepg, xaos. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: Unknown status:
  • tomcat7/7.0.68-1 by Emmanuel Bourg (test suite fails in test environment).
Patches submitted which have not made their way to the archive yet:
  • #814840 on tor by Petter Reinholdtsen: use the UTC timezone when calling asciidoc.
  • #815082 on arachne-pnr by Dhole: use the C locale to format the changelog date.
  • #815192 on manpages-de by Reiner Herrmann: tell grep to always treat the input as text so that it works with non-UTF-8 locales.
  • #815193 on razorqt by Reiner Herrmann: tell grep to always treat the input as text so that it works with non-UTF-8 locales.
  • #815250 on jacal by Reiner Herrmann: use the C locale to format the build date.
  • #815252 on colord by Lunar: remove extra timestamps when generating CMF and spectra and implement support for SOURCE_DATE_EPOCH.

reproducible.debian.net Two new package sets have been added: freedombox and freedombox_build-depends. (h01ger)

diffoscope development diffoscope version 49 was released on February 17th. It continues to improve handling of debug symbols for ELF files. Their content will now be compared separately to make them more readable. The search for matching debug packages is more efficient by looking only for .deb files in the same parent directory. Alongside more bug fixes, support for ICC profiles has been added, and libarchive is now also used to read metadata for ar archives.

strip-nondeterminism development Reiner Herrmann added support to normalize Gettext .mo files.

Package reviews 170 reviews have been removed, 172 added and 54 updated in the previous week. 34 new FTBFS bugs have been opened by Chris Lamb, h01ger and Reiner Herrmann. New issues added this week: lxqt_translate_desktop_binary_file_matched_under_certain_locales, timestamps_in_manpages_generated_by_autogen. Improvements to the prebuilder script: avoid ccache, skip disorderfs hook if device nodes cannot be created, compatibility with grsec trusted path execution (Reiner Herrmann), code cleanup (Esa Peuha).

Misc. Steven Chamberlain highlighted reproducibility problems due to differences in how Linux and FreeBSD handle permissions for symlinks. Some possible ways forward have been discussed on the reproducible-builds mailing list. Bernhard M. Wiedemann reported on some reproducibility tests made on OpenSuse mentioning the growing support for SOURCE_DATE_EPOCH. If you are eligible for Outreachy or Google Summer of Code, consider spending the summer working on reproducible builds!

17 January 2016

Lunar: Reproducible builds: week 38 in Stretch cycle

What happened in the reproducible builds effort between January 10th and January 16th:

Toolchain fixes Benjamin Drung uploaded mozilla-devscripts/0.43 which sorts the file list in preferences files. Original patch by Reiner Herrmann. Lunar submitted an updated patch series to make timestamps in packages created by dpkg deterministic. To ensure that the mtimes in data.tar are reproducible, with the patches, dpkg-deb uses the --clamp-mtime option added in tar/1.28-1 when available. An updated package has been uploaded to the experimental repository. This removed the need for a modified debhelper as all required changes for reproducibility have been merged or are now covered by dpkg.

Packages fixed The following packages have become reproducible due to changes in their build dependencies: angband-doc, bible-kjv, cgoban, gnugo, pachi, wmpuzzle, wmweather, wmwork, xfaces, xnecview, xscavenger, xtrlock, virt-top. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: Untested changes:

reproducible.debian.net Once again, Vagrant Cascadian is providing another armhf build system, allowing to run 6 more armhf builder jobs, right there. (h01ger) Stop requiring a modified debhelper and adapt to the latest dpkg experimental version by providing a predetermined identifier for the .buildinfo filename. (Mattia Rizzolo, h01ger) New X.509 certificates were set up for jenkins.debian.net and reproducible.debian.net using Let's Encrypt!. Thanks to GlobalSign for providing certificates for the last year free of charge. (h01ger)

Package reviews 131 reviews have been removed, 85 added and 32 updated in the previous week. FTBFS issues filled: 29. Thanks to Chris Lamb, Mattia Rizzolo, and Niko Tyni. New issue identified: timestamps_in_manpages_added_by_golang_cobra.

Misc. Most of the minutes from the meetings held in Athens in December 2015 are now available to the public.

2 January 2016

Lunar: Reproducible builds: week 33 in Stretch cycle

What happened in the reproducible builds effort between December 6th and December 12th: Toolchain fixes Reiner Herrmann rebased our experimental version of doxygen on version 1.8.9.1-6. Chris Lamb submitted a patch to make the manpages generated by ruby-ronn reproducible by using the locale-agnostic %Y-%m-%d for the dates. Daniel Kahn Gillmor took another shot at the issue of source path captured in DWARF symbols. A patch has been sent for review by GCC upstream to add the ability to read an environment variable with -fdebug-prefix-map. Packages fixed The following 24 packages have become reproducible due to changes in their build dependencies: gkeyfile-sharp, gprbuild, graphmonkey, gthumb, haskell-yi-language, ion, jackson-databind, jackson-dataformat-smile, jackson-dataformat-xml, jnr-ffi, libcommons-net-java, libproxy, maven-shared-utils, monodevelop-database, mydumper, ndesk-dbus, nini, notify-sharp, pixz, protozero, python-rtslib-fb, slurm-llnl, taglib-sharp, tomboy-latex. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: These uploads might have fixed reproducibility issues but could not be tested yet: Patches submitted which have not made their way to the archive yet: reproducible.debian.net Files created with diffoscope now have diffoscope in their name instead debbindiff. (h01ger) Hostnames of first and second build node are now recorded and shown in the build history. (Mattia Rizzolo) Exchanges have started with F-Droid developers to better understand what would be required to test F-Droid applications. (h01ger) A first small set of Fedora 23 packages is now also being tested while development on a new framework for testing RPMs in general has begun. A new Jenkins job has been added to set up to mock, the build system used by Fedora. Another new job takes care of testing RPMs from Fedora 23 on x86_64. So far only 151 packages from the buildsys-build group are tested (currently all unreproducible), but the plan is to build all 17,000 source packages in Fedora 23 and rawhide. The page presenting the results should also soon be improved. (h01ger, Dhiru Kholia) For Arch Linux, all 2223 packages from the extra repository will also be tested from now on. Packages in extra" are tested every four weeks, while those from core every week. Statistics are now displayed alongside the results. (h01ger) jenkins.debian.net has been updated to jenkins-job-builder version 1.3.0. Many job configurations have been simplified and refactored using features of the new version. This was another milestone for the jenkins.debian.org migration. (Phil Hands, h01ger) diffoscope development Chris Lamb announced try.diffoscope.org: an online service that runs diffoscope on user provided files. Screenshot of try.diffoscope.org Improvements are welcome. The application is licensed under the AGPLv3. On diffoscope itself, most pending patches have now been merged. Expect a release soon! Most of the code implementing parallel processing has been polished. Sadly, unpacking archive is CPU-bound in most cases, so the current thread-only implementation does not offer much gain on big packages. More work is still require to also add concurrent processes. Documentation update Ximin Luo has started to write a specification for buildinfo files that could become a larger platform than the limited set of features that were thought so far for Debian .buildinfo. Package reviews 113 reviews have been removed, 111 added and 56 updated in the previous week. 42 new FTBFS bugs were opened by Chris Lamb and Niko Tyni. New issues identified this week: timestamps_in_documentation_generated_by_docbook_dbtimestamp, timestamps_in_sym_l_files_generated_by_malaga, timestamps_in_edj_files_generated_by_edje_cc. Misc. Chris Lamb presented reproducible builds at skroutz.gr.

Next.