Yesterday was our monthly Debian meeting. We do one every month for Tokyo and Kansai combined, because it's online and no reason to split for now. I presented what I learnt about nodejs packaging in Debian. This time, I started using Emacs for presentation, presenting PDF file. This week I switched most of my login shells to emacsclient, and experimenting. It's interesting how much things break, and how much I depended on .bashrc and .profile being loaded. But most things work and I don't need most things outside of emacs...

I was going OK with Thunderbird and enigmail(though it have many problems). Normally I go through changelogs before updating packages and rarely do a complete upgrage of my machine. Couple of days ago I did a complete upgrade of system which updated my Thunderbird to latest version and throwing of enigmail plugin for using their native openPGP support. There is a blog from Mozilla which I should ve read earlier. Thunderbird s builtin openPGP functionality is still in experimental, atleast not ready for my workflow. I could ve downgrade to version 68. But I chose to move to my secondary MUA, mutt. I was using mutt for emails and newsletters that I check twice in a year a so. So I started configuring mutt to handle my big mailboxes. It took three evenings to configure mutt to my workflow. Though the basic setup can be done in less than an hour it is the small nitpicks consumed much of my time. Currently I have isync to pull and keep mails offline. Mutt to read, msmtp to send, abook as the email address book and urlview to see the links in mail. I am still learning notmuch and virtual mailbox ways to filter. There are ton of articles out there to configure mutt and all related things to it. But I find certain configs very hard to get. So I will write down those.

• As a long Thunderbird user, I still want my mailbox index to look in a certain way. I used;

   set date_format="%d/%m/%y %I:%M %p"
   set index_format="%3C   %Z %?X?@& ? %-22.17L %-5.50s %> %-22.20D"
  

  This gives following order - serial number, various flags (if there is an attachment, it will shows @ ), sender, subject and to extreme right there will be date and time (12h local) of mail
• If use nano to write mails you can use

  set editor="nano --syntax=email -r 70 -b"
  

  to get 70 char length and mail related syntax highlighting
• mutt has a way to make new mail notification. The new_mail_command can be used to execute a custom script upon new mail, such as running notify-send. There are many standalone mail notifier such as mailnag, mail notification etc. It all just felt bulky for me. So I ended up making these.

  #!/bin/sh
  accounts="$(awk '/^Channel/  print $2 ' "$MBSYNCRC")"
  for account in $accounts; do
       acc="$(echo "$account"   sed "s/.*\///")"
       new=$(find "$MAILBOX/$acc/Inbox/new/" -type f -newer "$MUTT/.mailsynclastrun" 2> /dev/null)
       newcount=$(echo "$new"   sed '/^\s*$/d'   wc -l)
       if [ "$newcount" -gt "0" ]; then
               for file in $new; do
               # Extract subject and sender from mail.
               from=$(awk '/^From: / && ++n ==1,/^\<.*\>:/' "$file"   perl -CS -MEncode -ne 'print decode("MIME-Header", $_)'   awk '  $1=""; if (NF>=3)$NF=""; print $0  '   sed 's/^[[:blank:]]*[\"'\''\<]*//;s/[\"'\''\>]*[[:blank:]]*$//')
               subject=$(awk '/^Subject: / && ++n == 1,/^\<.*\>: / && ++i == 2' "$file"   head -n 1   perl -CS -MEncode -ne 'print decode("MIME-Header", $_)'   sed 's/^Subject: //'   sed 's/^ [[:blank:]]*[\"'\''\<]*//;s/[\"'\''\>]*[[:blank:]]*$//'   tr -d '\n')
               displays="$(pgrep -a Xorg   grep -wo "[0-9]*:[0-9]\+")"
                       for x in $displays; do
                               export DISPLAY=$x
                               notify-send -i $MUTT/mutt.png -t 5000 "$account received new message:" "$from: <i>$subject</i>"
                       done
               done
       fi
  done
  touch "$MUTT/.mailsynclastrun" 
  

  And hooked to mail_new_command. This code snippet is from lukesmith s mutt-wizard, I barely modified to meet my need. Currently it only look in to Inbox . I need to modify to check other mailbox folders in future.

So far, everything going okay.

Cons

• some times mbsync throws EOF and secret key not found error.
• searching is still a pain in mutt
• nano s spell checker also check things which I am replying to.

More to come Well for now I moved mail from part of the Thunderbird. But Thunderbird was more than a MUA to me. It was my RSS reader, calendar and to-do list manager. I will write more about those once I make a complete transition.

It's been 20 years since I became a Debian Developer. Lots of fun things happened, and I think fondly of the team. I am no longer active for the past 10 years due to family reasons, and it's surprising that I have been inactive for that long. I still use Debian, and I still participate in the local Debian meetings.

The Rcpp team is proud to announce release 1.0.6 of Rcpp which arrived at CRAN earlier today, and has been uploaded to Debian too. Windows and macOS builds should appear at CRAN in the next few days. This marks the first release on the new six-months cycle announced with release 1.0.5 in July. As reminder, interim dev or rc releases will often be available in the Rcpp drat repo; this cycle there were four. Rcpp has become the most popular way of enhancing R with C or C++ code. As of today, 2174 packages on CRAN depend on Rcpp for making analytical code go faster and further (which is an 8.5% increase just since the last release), along with 207 in BioConductor. This release features six different pull requests from five different contributors, mostly fixing fairly small corner cases, plus some minor polish on documentation and continuous integration. Before releasing we once again made numerous reverse dependency checks none of which revealed any issues. So the passage at CRAN was pretty quick despite the large dependency footprint, and we are once again grateful for all the work the CRAN maintainers do.

Changes in Rcpp patch release version 1.0.6 (2021-01-14)

• Changes in Rcpp API:
  • Replace remaining few uses of EXTPTR_PTR with R_ExternalPtrAddr (Kevin in #1098 fixing #1097).
  • Add push_back and push_front for DataFrame (Walter Somerville in #1099 fixing #1094).
  • Remove a misleading-to-wrong comment (Mattias Ellert in #1109 cleaning up after #1049).
  • Address a sanitizer report by initializing two private bool variables (Benjamin Christoffersen in #1113).
  • External pointer finalizer toggle default values were corrected to true (Dirk in #1115).
• Changes in Rcpp Documentation:
  • Several URLs were updated to https and/or new addresses (Dirk).
• Changes in Rcpp Deployment:
  • Added GitHub Actions CI using the same container-based setup used previously, and also carried code coverage over (Dirk in #1128).
• Changes in Rcpp support functions:
  • Rcpp.package.skeleton() avoids warning from R. (Dirk)

Thanks to my CRANberries, you can also look at a diff to the previous release. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. Bugs reports are welcome at the GitHub issue tracker as well (where one can also search among open or closed issues); questions are also welcome under rcpp tag at StackOverflow which also allows searching among the (currently) 2616 previous questions. If you like this or other open-source work I do, you can sponsor me at GitHub. My sincere thanks to my current sponsors for me keeping me caffeinated.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Mainly to recall what happened last year and to give thoughts and plan for the upcoming year(s) I m once again revisiting my previous year (previous editions: 2019, 2018, 2017, 2016, 2015, 2014, 2013 + 2012). Due to the Coronavirus disease (COVID-19) pandemic, 2020 was special for several reasons, but overall I consider myself and my family privileged and am very grateful for that. In terms of IT events, I planned to attend Grazer Linuxdays and DebConf in Haifa/Israel. Sadly Grazer Linuxdays didn t take place at all, and DebConf took place online instead (which I didn t really participate in for several reasons). I took part in the well organized DENOG12 + ATNOG 2020/1 online meetings. I still organize our monthly Security Treff Graz (STG) meetups, and for half of the year, those meetings took place online (which worked OK-ish overall IMO). Only at the beginning of 2020, I managed to play Badminton (still playing in the highest available training class (in german: Kader ) at the University of Graz / Universit ts-Sportinstitut, USI). For the rest of the year except for ~2 weeks in October or so the sessions couldn t occur. Plenty of concerts I planned to attend were cancelled for obvious reasons, including the ones I would have played myself. But I managed to attend Jazz Redoute 2020 Dom im Berg, Martin Grubinger in Musikverein Graz and Emiliano Sampaio s Mega Mereneu Project at WIST Moserhofgasse (all before the corona situation kicked in). The concert from Ton Feinig & RTV Slovenia Big Band occurred under strict regulations in Summer, as well as Elektra Opera by Richard Strau in a very special setting (only one piano player instead of the orchestra because of a Corona case in the orchestra) in Autumn. At the beginning of 2020, I also visited Literaturshow Roboter mit Senf at Literaturhaus Graz. The lack of concerts and rehearsals also severely impacted my playing the drums (including at HTU BigBand Graz), which pretty much didn t take place. :( Grml-wise we managed to publish release 2020.06, codename Ausgehfuahangl. Regarding jenkins-debian-glue I tried to clarify its state and received some really lovely feedback. I consider 2020 as the year where I dropped regular usage of Jabber (so far my accounts still exist, but I m no longer regularly online and am not sure for how much longer I ll keep my accounts alive as such). Business-wise it was our seventh year of business with SynPro Solutions GmbH. No big news but steady and ongoing work with my other business duties Grml Solutions and Grml-Forensic. As usual, I shared childcare with my wife. Due to the corona situation, my wife got a new working schedule, which shuffled around our schedule a bit on Mondays + Tuesdays. Still, we managed to handle the homeschooling/distance learning quite well. Currently we re sitting in the third lockdown, and yet another round of homeschooling/distance learning is going on those days (let s see how long ). I counted 112 actual school days in all of 2020 for our older daughter with only 68 school days since our first lockdown on 16th of March, whereas we had 213(!) press conferences by our Austrian government in 2020. (Further rants about the situation in Austria snipped.) Book reading-wise I managed to complete 60 books (see Mein Lesejahr 2020 ). Once again, I noticed that what felt like good days for me always included reading books, so I ll try to keep my reading pace for 2021. I ll also continue with my hobbies Buying Books and Reading Books , to get worse at Tsundoku. Hoping for vaccination and a more normal 2021, Schwuppdiwupp!

Before and during FOSDEM 2020, I agreed with the people (developers, supporters, managers) of the UBports Foundation to package the Unity8 Operating Environment for Debian. Since 27th Feb 2020, Unity8 has now become Lomiri. Things got delayed a little recently as my main developer contact on the upstream side was on sick leave for a while. Fortunately, he has now fully recovered and work is getting back on track. Recent Uploads to Debian related to Lomiri Over the past 3 months I worked on the following bits and pieces regarding Lomiri in Debian:

• Work on DEB package qtmir.
• Revisit upstream status of lomiri-ui-toolkit (and remind upstream on open issues to be resolved).
• Debug googletest / cmake-extras build failures in qtmir DEB package.
• Prepare MR!2 f r cmake-extras upstream, upload cmake-extras 1.5-3 to Debian unstable.
• Upload qtmir to unstable/NEW, submit patches upstream.
• Upload to unstable as NEW: ayatana-indicator-bluetooth.
• Upload to unstable: qtmir 0.6.1-2.
• Upload to unstable: cmake-extras 1.5-5.
• More debugging on googletest / cmake-extras.
• Qt5.15 transition in Debian, various uploads.
• Upload unstable: cmake-extras 1.5-6.
• Debug and fix libqtdbustest / libqtdbusmock failures in Debian.
• Get in touch with RAOF (on IRC, from the Mir Team) on possible fixes of Mir FTBFS in Debian (after a Mesa upload).
• Upload to unstable: mir 1.8.0+dfsg1-11 (fix FTBFS).
• Fix dbus-cpp autopkgtests.
• Fix Mir FTBFS after Boost 1.74 transition.
• Fix FTBFS in unit tests in lomiri-url-dispatcher.
• Drop autopilot dependency from DEB package qtmir.
• Heads-Up Meeting with Marius Gripsgard (discussion next steps).

The next projects / packages ahead are lomiri-ui-toolkit, integrating changes required for Lomiri into Ayatana Indicators and then moving on with several other, smaller packages. Credits Many big thanks go to Marius and Dalton for their work on the UBports project and being always available for questions, feedback, etc. Thanks to Florian Leeber for being my point of contact for topcis regarding my cooperation with the UBports Foundation.

I'm sorry to announce that I gave up on the Fairphone series and switched to a Google Phone (Pixel 4a) running CalyxOS.

Problems in fairy land My fairphone2, even if it is less than two years old, is having major problems:

• from time to time, the screen flickers and loses "touch" until I squeeze it back together
• the camera similarly disconnects regularly
• even when it works, the camera is... pretty bad: low light is basically unusable, it's slow and grainy
• the battery can barely keep up for one day
• the cellular coverage is very poor, in Canada: I lose signal at the grocery store and in the middle of my house...

Some of those problems are known: the Fairphone 2 is old now. It was probably old even when I got it. But I can't help but feel a little sad to let it go: the entire point of that device was to make it easy to fix. But alas, because it's sold only in Europe, local stores don't carry replacement parts. To be fair, Fairphone did offer to fix the device, but with a 2 weeks turnaround, I had to get another phone anyways. I did actually try to buy a fairphone3, from Clove. But they did some crazy validation routine. By email, they asked me to provide a photo copy of a driver's license and the credit card, arguing they need to do this to combat fraud. I found that totally unacceptable and asked them to cancel my order. And because I'm not sure the FP3 will fix the coverage issues, I decided to just give up on Fairphone until they officially ship to the Americas.

Do no evil, do not pass go, do not collect 200$ So I got a Google phone, specifically a Pixel 4a. It's a nice device, all small and shiny, but it's "plasticky" - I would have prefered metal, but it seems you need to pay much, much more to get that (in the Pixel 5). In any case, it's certainly a better form factor than the Fairphone 2: even though the screen is bigger, the device itself is actually smaller and thinner, which feels great. The OLED screen is beautiful, awesome contrast and everything, and preliminary tests show that the camera is much better than the one on the Fairphone 2. (The be fair, again, that is another thing the FP3 improved significantly. And that is with the stock Camera app from CalyxOS/AOSP, so not as good as the Google Camera app, which does AI stuff.)

CalyxOS: success The Pixel 4a not not supported by LineageOS: it seems every time I pick a device in that list, I manage to miss the right device by one (I bought a Samsung S9 before, which is also unsupported, even though the S8 is). But thankfully, it is supported by CalyxOS. That install was a breeze: I was hesitant in playing again with installing a custom Android firmware on a phone after fighting with this quite a bit in the past (e.g. htc-one-s, lg-g3-d852). But it turns out their install instructions, mostly using a AOSP alliance device-flasher works absolutely great. It assumes you know about the commandline, and it does require to basically curl sudo (because you need to download their binary and run it as root), but it Just. Works. It reminded me of how great it was to get the Fairphone with TWRP preinstalled... Oh, and kudos to the people in #calyxos on Freenode: awesome tech support, super nice folks. An amazing improvement over the ambiance in #lineageos!

Migrating data Unfortunately, migrating the data was the usual pain in the back. This should improve the next time I do this: CalyxOS ships with seedvault, a secure backup system for Android 10 (or 9?) and later which backs up everything (including settings!) with encryption. Apparently it works great, and CalyxOS is also working on a migration system to switch phones. But, obviously, I couldn't use that on the Fairphone 2 running Android 7... So I had to, again, improvised. The first step was to install Syncthing, to have an easy way to copy data around. That's easily done through F-Droid, already bundled with CalyxOS (including the privileged extension!). Pair the devices and boom, a magic portal to copy stuff over. The other early step I took was to copy apps over using the F-Droid "find nearby" functionality. It's a bit quirky, but really helps in copying a bunch of APKs over. Then I setup a temporary keepassxc password vault on the Syncthing share so that I could easily copy-paste passwords into apps. I used to do this in a text file in Syncthing, but copy-pasting in the text file is much harder than in KeePassDX. (I just picked one, maybe KeePassDroid is better? I don't know.) Do keep a copy of the URL of the service to reduce typing as well. Then the following apps required special tweaks:

• AntennaPod has an import/export feature: export on one end, into the Syncthing share, then import on the other. then go to the queue and select all episodes and download
• the Signal "chat backup" does copy the secret key around, so you don't get the "security number change" warning (even if it prompts you to re-register) - external devices need to be relinked though
• AnkiDroid, DSub, Nextcloud, and Wallabag required copy-pasting passwords

I tried to sync contacts with DAVx5 but that didn't work so well: the account was setup correctly, but contacts didn't show up. There's probably just this one thing I need to do to fix this, but since I don't really need sync'd contact, it was easier to export a VCF file to Syncthing and import again.

Known problems One problem with CalyxOS I found is that the fragile little microg tweaks didn't seem to work well enough for Signal. That was unexpected so they encouraged me to file that as a bug. The other "issue" is that the bootloader is locked, which makes it impossible to have "root" on the device. That's rather unfortunate: I often need root to debug things on Android. In particular, it made it difficult to restore data from OSMand (see below). But I guess that most things just work out of the box now, so I don't really need it and appreciate the extra security. Locking the bootloader means full cryptographic verification of the phone, so that's a good feature to have! OSMand still doesn't have a good import/export story. I ended up sharing the Android/data/net.osmand.plus/files directory and importing waypoints, favorites and tracks by hand. Even though maps are actually in there, it's not possible for Syncthing to write directly to the same directory on the new phone, "thanks" to the new permission system in Android which forbids this kind of inter-app messing around. Tracks are particularly a problem: my older OSMand setup had all those folders neatly sorting those tracks by month. This makes it really annoying to track every file manually and copy it over. I have mostly given up on that for now, unfortunately. And I'll still need to reconfigure profiles and maps and everything by hand. Sigh. I guess that's a good clearinghouse for my old tracks I never use... Update: turns out setting storage to "shared" fixed the issue, see comments below!

Conclusion Overall, CalyxOS seems like a good Android firmware. The install is smooth and the resulting install seems solid. The above problems are mostly annoyances and I'm very happy with the experience so far, although I've only been using it for a few hours so this is very preliminary.

First of all, let me all wish you a happy new year, with all my wishes of health and succes. I sincerely hope this year will be simpler for most people as last year ! For the first post of the year, I wanted to show you how to take advantage of Ruby, the programming language embedded in QSoas, to make various things, like:

• creating a column with the sum of Y values;
• extending values that are present only in a few lines;
• renaming datasets using a pattern.

Summing the values in a column When using commands that take formulas (Ruby code), like apply-formula, the code is run for every single point, for which all the values are updated. In particulier, the state of the previous point is not known. However, it is possible to store values in what is called global variables, whose name start with an $ sign. Using this, we can keep track of the previous values. For instance, to create a new column with the sum of the y values, one can use the following approach:

QSoas> eval $sum=0
QSoas> apply-formula /extra-columns=1 $sum+=y;y2=$sum

The first line initializes the variable to 0, before we start summing, and the code in the second line is run for each dataset row, in order. For the first row, for instance, $sum is initially 0 (from the eval line); after the execution of the code, it is now the first value of y. After the second row, the second value of y is added, and so on. The image below shows the resulting y2 when used on:

QSoas> generate-dataset -1 1 x

Extending values in a column Another use of the global variables is to add "missing" data. For instance, let's imagine that a files given the variation of current over time as the potential is changed, but the potential is only changed stepwise and only indicated when it changes:

## time	current	potential
0	0.1	0.5
1	0.2
2	0.3
3	0.2
4	1.2	0.6
5	1.3
...

If you need to have the values everywhere, for instance if you need to split on their values, you could also use a global variable, taking advantage of the fact that missing values are represented by QSoas using "Not A Number" values, which can be detected using the Ruby function nan?:

QSoas> apply-formula "if y2.nan?; then y2=$value; else $value=y2;end"

Note the need of quotes because there are spaces in the ruby code. If the value of y2 is NaN, that is it is missing, then it is taken from the global variable $value else $value is set the current value of y2. Hence, the values are propagated down:

## time	current	potential
0	0.1	0.5
1	0.2	0.5
2	0.3	0.5
3	0.2	0.5
4	1.2	0.6
5	1.3	0.6
...

Of course, this doesn't work if the first value of y2 is missing.

Renaming using a pattern The command save-datasets can be used to save a whole series of datasets to the disk. It can also rename them on the fly, and, using the /mode=rename option, does only the renaming part, without saving. You can make full use of meta-data (see also a first post here)for renaming. The full power is unlocked using the /expression= option. For instance, for renaming the last 5 datasets (so numbers 0 to 4) using a scheme based on the value of their pH meta-data, you can use the following code:

QSoas> save-datasets /mode=rename /expression='"dataset-# $meta.pH "' 0..4

The double quotes are cumbersome but necessary, since the outer quotes (') prevent the inner ones (") to be removed and the inner quotes are here to indicate to Ruby that we are dealing with text. The bit inside # ... is interpreted by Ruby as Ruby code; here it is $meta.pH, the value of the "pH" meta-data. Finally the 0..4 specifies the datasets to work with. So theses datasets will change name to become dataset-7 for pH 7, etc...

About QSoas QSoas is a powerful open source data analysis program that focuses on flexibility and powerful fitting capacities. It is released under the GNU General Public License. It is described in Fourmond, Anal. Chem., 2016, 88 (10), pp 5050 5052. Current version is 3.0. You can download its source code there (or clone from the GitHub repository) and compile it yourself, or buy precompiled versions for MacOS and Windows there.

Note: this is another article in my series on asynchronous communication in Linux with UUCP and NNCP. In the previous installment on store-and-forward backups, I mentioned how easy it is to do with ZFS, and some of the tools that can be used to do it without ZFS. A lot of those tools are a bit less robust, so we need some sort of store-and-forward mechanism to verify backups. To be sure, verifying backups is good with ANY scheme, and this could be used with ZFS backups also. So let s say you have a shiny new backup scheme in place, and you d like to verify that it s working correctly. To do that, you need to compare the source directory tree on machine A with the backed-up directory tree on machine B. Assuming a conventional setup, here are some ways you might consider to do that:

• Just copy everything from machine A to machine B and compare locally
• Or copy everything from machine A to a USB drive, plug that into machine B, and compare locally
• Use rsync in dry-run mode and see if it complains about anything

The first two options are not particularly practical for large datasets, though I note that the second is compatible with airgapping. Using rsync requires both systems to be online at the same time to perform the comparison. What would be really nice here is a tool that would write out lots of information about the files on a system: their names, sizes, last modified dates, maybe even sha256sum and other data. This file would be far smaller than the directory tree itself, would compress nicely, and could be easily shipped to an airgapped system via NNCP, UUCP, a USB drive, or something similar. Tool choices It turns out there are already quite a few tools in Debian (and other Free operating systems) to do this, and half of them are named mtree (though, of course, not all mtrees are compatible with each other.) We ll look at some of the options here. I ve made a simple test directory for illustration purposes with these commands:

mkdir test
cd test
echo hi > hi
ln -s hi there
ln hi foo
touch empty
mkdir emptydir
mkdir somethingdir
cd somethingdir
ln -s ../there

I then also used touch to set all files to a consistent timestamp for illustration purposes. Tool option: getfacl (Debian package: acl) This comes with the acl package, but can be used with other than ACL purposes. Unfortunately, it doesn t come with a tool to directly compare its output with a filesystem (setfacl, for instance, can apply the permissions listed but won t compare.) It ignores symlinks and doesn t show sizes or dates, so is ineffective for our purposes. Example output:

$ getfacl --numeric -R test
...
# file: test/hi
# owner: 1000
# group: 1000
user::rw-
group::r--
other::r--
...

Tool option: fmtree, the FreeBSD mtree (Debian package: freebsd-buildutils) fmtree can prepare a specification based on a directory tree, and compare a directory tree to that specification. The comparison also is aware of files that exist in a directory tree but not in the specification. The specification format is a bit on the odd side, but works well enough with fmtree. Here s a sample output with defaults:

$ fmtree -c -p test
...
# .
/set type=file uid=1000 gid=1000 mode=0644 nlink=1
.               type=dir mode=0755 nlink=4 time=1610421833.000000000
    empty       size=0 time=1610421833.000000000
    foo         nlink=2 size=3 time=1610421833.000000000
    hi          nlink=2 size=3 time=1610421833.000000000
    there       type=link mode=0777 time=1610421833.000000000 link=hi
... skipping ...
# ./somethingdir
/set type=file uid=1000 gid=1000 mode=0777 nlink=1
somethingdir    type=dir mode=0755 nlink=2 time=1610421833.000000000
    there       type=link time=1610421833.000000000 link=../there
# ./somethingdir
..
..

You might be wondering here what it does about special characters, and the answer is that it has octal escapes, so it is 8-bit clean. To compare, you can save the output of fmtree to a file, then run like this:

cd test
fmtree < ../test.fmtree

If there is no output, then the trees are identical. Change something and you get a line of of output explaining each difference. You can also use fmtree -U to change things like modification dates to match the specification. fmtree also supports quite a few optional keywords you can add with -K. They include things like file flags, user/group names, various tipes of hashes, and so forth. I'll note that none of the options can let you determine which files are hardlinked together. Here's an excerpt with -K sha256digest added:

    empty       size=0 time=1610421833.000000000 \
                sha256digest=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
    foo         nlink=2 size=3 time=1610421833.000000000 \
                sha256digest=98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4

If you include a sha256digest in the spec, then when you verify it with fmtree, the verification will also include the sha256digest. Obviously fmtree -U can't correct a mismatch there, but of course it will detect and report it. Tool option: mtree, the NetBSD mtree (Debian package: mtree-netbsd) mtree produces (by default) output very similar to fmtree. With minor differences (such as the name of the sha256digest in the output), the discussion above about fmtree also applies to mtree. There are some differences, and the most notable is that mtree adds a -C option which reads a spec and converts it to a "format that's easier to parse with various tools." Here's an example:

$ mtree -c -K sha256digest -p test   mtree -C
. type=dir uid=1000 gid=1000 mode=0755 nlink=4 time=1610421833.0 flags=none 
./empty type=file uid=1000 gid=1000 mode=0644 nlink=1 size=0 time=1610421833.0 flags=none 
./foo type=file uid=1000 gid=1000 mode=0644 nlink=2 size=3 time=1610421833.0 flags=none 
./hi type=file uid=1000 gid=1000 mode=0644 nlink=2 size=3 time=1610421833.0 flags=none 
./there type=link uid=1000 gid=1000 mode=0777 nlink=1 link=hi time=1610421833.0 flags=none 
./emptydir type=dir uid=1000 gid=1000 mode=0755 nlink=2 time=1610421833.0 flags=none 
./somethingdir type=dir uid=1000 gid=1000 mode=0755 nlink=2 time=1610421833.0 flags=none 
./somethingdir/there type=link uid=1000 gid=1000 mode=0777 nlink=1 link=../there time=1610421833.0 flags=none 

Most definitely an improvement in both space and convenience, while still retaining the relevant information. Note that if you want the sha256digest in the formatted output, you need to pass the -K to both mtree invocations. I could have done that here, but it is easier to read without it. mtree can verify a specification in either format. Given what I'm about to show you about bsdtar, this should illustrate why I bothered to package mtree-netbsd for Debian. Unlike fmtree, the mtree -U command will not adjust modification times based on the spec, but it will report on differences. Tool option: bsdtar (Debian package: libarchive-tools) bsdtar is a fascinating program that can work with many formats other than just tar files. Among the formats it supports is is the NetBSD mtree "pleasant" format (mtree -C compatible). bsdtar can also convert between the formats it supports. So, put this together: bsdtar can convert a tar file to an mtree specification without extracting the tar file. bsdtar can also use an mtree specification to override the permissions on files going into tar -c, so it is a way to prepare a tar file with things owned by root without resorting to tools like fakeroot. Let's look at how this can work:

$ cd test
$ bsdtar --numeric -cf - --format=mtree .
#mtree
. time=1610472086.318593729 mode=755 gid=1000 uid=1000 type=dir
./empty time=1610421833.0 mode=644 gid=1000 uid=1000 type=file size=0
./foo nlink=2 time=1610421833.0 mode=644 gid=1000 uid=1000 type=file size=3
./hi nlink=2 time=1610421833.0 mode=644 gid=1000 uid=1000 type=file size=3
./ormat\075mtree time=1610472086.318593729 mode=644 gid=1000 uid=1000 type=file size=5632
./there time=1610421833.0 mode=777 gid=1000 uid=1000 type=link link=hi
./emptydir time=1610421833.0 mode=755 gid=1000 uid=1000 type=dir
./somethingdir time=1610421833.0 mode=755 gid=1000 uid=1000 type=dir
./somethingdir/there time=1610421833.0 mode=777 gid=1000 uid=1000 type=link link=../there

You can use mtree -U to verify that as before. With the --options mtree: set, you can also add hashes and similar to the bsdtar output. Since bsdtar can use input from tar, pax, cpio, zip, iso9660, 7z, etc., this capability can be used to create verification of the files inside quite a few different formats. You can convert with bsdtar -cf output.mtree --format=mtree @input.tar. There are some foibles with directly using these converted files with mtree -U, but usually minor changes will get it there. Side mention: stat(1) (Debian package: coreutils) This tool isn't included because it won't operate recursively, but is a tool in the similar toolbox. Putting It Together I will still be developing a complete non-ZFS backup system for NNCP (or UUCP) in a future post. But in the meantime, here are some ideas you can reflect on:

• Let's say your backup scheme involves sending a full backup every night. On the source system, you could pipe the generated tar file through something like tee >(bsdtar -cf bcakup.mtree @-) to generate an mtree file in-band while generating the tar file. This mtree file could be shipped over for verification.
• Perhaps your backup scheme involves sending incremental backup data via rdup or even ZFS, but you would like to periodically verify that everything is good -- that an incremental didn't miss something. Something like mtree -K sha256 -c -x -p / mtree -C -K sha256 would let you accomplish that.

I will further develop at least one of these ideas in a future post. Bonus: cross-tool comparisons In my mtree-netbsd packaging, I added tests like this to compare between tools:

fmtree -c -K $(MTREE_KEYWORDS)   mtree
mtree -c -K $(MTREE_KEYWORDS)   sed -e 's/\(md5\ sha1\ sha256\ sha384\ sha512\)=/\1digest=/' -e 's/rmd160=/ripemd160digest=/'   fmtree
bsdtar -cf - --options 'mtree:uname,gname,md5,sha1,sha256,sha384,sha512,device,flags,gid,link,mode,nlink,size,time,uid,type,uname' --format mtree .   mtree

The promise of free software is a near-future utopia, built on democratized technology. This future is just and it is beautiful, full of opportunity and fulfillment for everyone everywhere. We can create the things we dream about when we let our minds wander into the places they want to. We can be with the people we want and need to be, when we want and need to. This is currently possible with the technology we have today, but it s availability is limited by the reality of the world we live in the injustice, the inequity, the inequality. Technology runs the world, but it does not serve the interests of most of us. In order to create a better world, our technology must be transparent, accountable, trustworthy. It must be just. It must be free. The job of the free software movement is to demonstrate that this world is possible by living its values now: justice, equity, equality. We build them into our technology, and we build technology that make it possible for these values to exist in the world. At the Free Software Foundation, we liked to say that we used all free software because it was important to show that we could. You can do anything with free software, so we did everything with it. We demonstrated the importance of unions for tech workers and non-profit workers by having one. We organized collectively and protected our rights for the sake of ourselves and one another. We had non-negotiable salaries, based on responsibility level and position. That didn t mean we worked in an office free from the systemic problems that plague workplaces everywhere, but we were able to think about them differently. Things were this way because of Richard Stallman but I view his influence on these things as negative rather than positive. He was a cause that forced these outcomes, rather than being supportive of the desires and needs of others. Rather than indulge in gossip or stories, I would like to jump to the idea that he was supposed to have been deplatformed in October 2019. In resigning from his position as president of the FSF, he certainly lost some of his ability to reach audiences. However, Richard still gives talks. The FSF continues to use his image and rhetoric in their own messaging and materials. They gave him time to speak at their annual conference in 2020. He maintains leadership in the GNU project and otherwise within the FSF sphere. The people who empowered him for so many years are still in charge. Richard, and the continued respect and space he is given, is not the only problem. It represents a bigger problem. Sexism and racism (among others) run rampant in the community. This happens because of bad actors and, more significantly, by the complacency of organizations, projects, and individuals afraid of losing contributors, respect, or funding. In a sector that has so much money and so many resources, women are still being paid less than men; we deny people opportunities to learn and grow in the name of immediate results; people who aren t men, who aren t white, are abused and harassed; people are mentally and emotionally taken advantage of, and we are coerced into burn out and giving up our lives for these companies and projects and we are paid for tolerating all of this by being told we re doing a good job or making a difference. But we re not making a difference. We re perpetuating the worst of the status quo that we should be fighting against. We must not continue. We cannot. We need to live our ideals as they are, and take the natural next steps in their evolution. We cannot have a world of just technology when we live in a world of exclusion; we cannot have free software if we continue to allow, tolerate, and laud the worst of us. I ve been in and around free software for seventeen years. Nearly every part of it I ve participated in has members and leadership that benefit from allowing and encouraging the continuation of maleficence and systemic oppression. We must purge ourselves of these things of sexism, racism, injustice, and the people who continue and enable it. There is no space to argue over whether a comment was transphobic if it hurt a trans person then it is transphobic and it is unacceptable. Racism is a global problem and we must be anti-racist or we are complicit. Sexism is present and all men benefit from it, even if they don t want to. These are free software issues. These are things that plague software, and these are things software reinforces within our societies. If a technology is exclusionary, it does not work. If a community is exclusionary, it must be fixed or thrown away. There is no middle ground here. There is no compromise. Without doing this, without taking the hard, painful steps to actually live the promise of user freedom and everything it requires and entails, our work is pointless and free software will fail. I don t think it s too late for there to be a radical change the radical change that allows us to create the utopia we want to see in the world. We must do that by acknowledging that just technology leads to a just society, and that a just society allows us to make just technology. We must do that by living within the principles that guide this future now. I don t know what will happen if things don t change soon. I recently saw someone comment that change doesn t happens unless one person is willing to sacrifice everything to make that change, to lead and inspire others to play small parts. This is unreasonable to ask of or expect from someone. I ve been burning myself out to meet other people s expectations for seventeen years, and I can t keep doing it. Of course I am not alone, and I am not the only one working on and occupied by these problems. More people must step up, not just for my sake, but for the sake of all of us, the work free software needs to do, and the future I dream about.

After a lot of hard work by its maintainer Alexandre Viau and others, the decentralized communication platform Jami (earlier known as Ring), managed to get its latest version into Debian Testing. Several of its dependencies has caused build and propagation problems, which all seem to be solved now. In addition to the fact that Jami is decentralized, similar to how bittorrent is decentralized, I first of all like how it is not connected to external IDs like phone numbers. This allow me to set up computers to send me notifications using Jami without having to find get a phone number for each computer. Automatic notification via Jami is also made trivial thanks to the provided client side API (as a DBus service). Here is my bourne shell script demonstrating how to let any system send a message to any Jami address. It will create a new identity before sending the message, if no Jami identity exist already:

#!/bin/sh
#
# Usage: $0  
#
# Send  to , create local jami account if
# missing.
#
# License: GPL v2 or later at your choice
# Author: Petter Reinholdtsen
if [ -z "$HOME" ] ; then
    echo "error: missing \$HOME, required for dbus to work"
    exit 1
fi
# First, get dbus running if not already running
DBUSLAUNCH=/usr/bin/dbus-launch
PIDFILE=/run/asterisk/dbus-session.pid
if [ -e $PIDFILE ] ; then
    . $PIDFILE
    if ! kill -0 $DBUS_SESSION_BUS_PID 2>/dev/null ; then
        unset DBUS_SESSION_BUS_ADDRESS
    fi
fi
if [ -z "$DBUS_SESSION_BUS_ADDRESS" ] && [ -x "$DBUSLAUNCH" ]; then
    DBUS_SESSION_BUS_ADDRESS="unix:path=$HOME/.dbus"
    dbus-daemon --session --address="$DBUS_SESSION_BUS_ADDRESS" --nofork --nopidfile --syslog-only < /dev/null > /dev/null 2>&1 3>&1 &
    DBUS_SESSION_BUS_PID=$!
    (
        echo DBUS_SESSION_BUS_PID=$DBUS_SESSION_BUS_PID
        echo DBUS_SESSION_BUS_ADDRESS=\""$DBUS_SESSION_BUS_ADDRESS"\"
        echo export DBUS_SESSION_BUS_ADDRESS
    ) > $PIDFILE
    . $PIDFILE
fi &
dringop()  
    part="$1"; shift
    op="$1"; shift
    dbus-send --session \
        --dest="cx.ring.Ring" /cx/ring/Ring/$part cx.ring.Ring.$part.$op $*
 
dringopreply()  
    part="$1"; shift
    op="$1"; shift
    dbus-send --session --print-reply \
        --dest="cx.ring.Ring" /cx/ring/Ring/$part cx.ring.Ring.$part.$op $*
 
firstaccount()  
    dringopreply ConfigurationManager getAccountList   \
      grep string   awk -F'"' ' print $2 '   head -n 1
 
account=$(firstaccount)
if [ -z "$account" ] ; then
    echo "Missing local account, trying to create it"
    dringop ConfigurationManager addAccount \
      dict:string:string:"Account.type","RING","Account.videoEnabled","false"
    account=$(firstaccount)
    if [ -z "$account" ] ; then
        echo "unable to create local account"
        exit 1
    fi
fi
# Not using dringopreply to ensure $2 can contain spaces
dbus-send --print-reply --session \
  --dest=cx.ring.Ring \
  /cx/ring/Ring/ConfigurationManager \
  cx.ring.Ring.ConfigurationManager.sendTextMessage \
  string:"$account" string:"$1" \
  dict:string:string:"text/plain","$2" 

If you want to check it out yourself, visit the the Jami system project page to learn more, and install the latest Jami client from Debian Unstable or Testing. As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

In the comments on my post about Load Average Monitoring [1] an anonymous person recommended that I investigate PSI. As an aside, why do I get so many great comments anonymously? Don t people want to get credit for having good ideas and learning about new technology before others? PSI is the Pressure Stall Information subsystem for Linux that is included in kernels 4.20 and above, if you want to use it in Debian then you need a kernel from Testing or Unstable (Bullseye has kernel 4.19). The place to start reading about PSI is the main Facebook page about it, it was originally developed at Facebook [2]. I am a little confused by the actual numbers I get out of PSI, while for the load average I can often see where they come from (EG have 2 processes each taking 100% of a core and the load average will be about 2) it s difficult to work out where the PSI numbers come from. For my own use I decided to treat them as unscaled numbers that just indicate problems, higher number is worse and not worry too much about what the number really means. With the cgroup2 interface which is supported by the version of systemd in Testing (and which has been included in Debian backports for Buster) you get PSI files for each cgroup. I ve just uploaded version 1.3.5-2 of etbemon (package mon) to Debian/Unstable which displays the cgroups with PSI numbers greater than 0.5% when the load average test fails.

System CPU Pressure: avg10=0.87 avg60=0.99 avg300=1.00 total=20556310510
/system.slice avg10=0.86 avg60=0.92 avg300=0.97 total=18238772699
/system.slice/system-tor.slice avg10=0.85 avg60=0.69 avg300=0.60 total=11996599996
/system.slice/system-tor.slice/tor@default.service avg10=0.83 avg60=0.69 avg300=0.59 total=5358485146
System IO Pressure: avg10=18.30 avg60=35.85 avg300=42.85 total=310383148314
 full avg10=13.95 avg60=27.72 avg300=33.60 total=216001337513
/system.slice avg10=2.78 avg60=3.86 avg300=5.74 total=51574347007
/system.slice full avg10=1.87 avg60=2.87 avg300=4.36 total=35513103577
/system.slice/mariadb.service avg10=1.33 avg60=3.07 avg300=3.68 total=2559016514
/system.slice/mariadb.service full avg10=1.29 avg60=3.01 avg300=3.61 total=2508485595
/system.slice/matrix-synapse.service avg10=2.74 avg60=3.92 avg300=4.95 total=20466738903
/system.slice/matrix-synapse.service full avg10=2.74 avg60=3.92 avg300=4.95 total=20435187166

Above is an extract from the output of the loadaverage check. It shows that tor is a major user of CPU time (the VM runs a ToR relay node and has close to 100% of one core devoted to that task). It also shows that Mariadb and Matrix are the main users of disk IO. When I installed Matrix the Debian package told me that using SQLite would give lower performance than MySQL, but that didn t seem like a big deal as the server only has a few users. Maybe I should move Matrix to the Mariadb instance. to improve overall system performance. So far I have not written any code to display the memory PSI files. I don t have a lack of RAM on systems I run at the moment and don t have a good test case for this. I welcome patches from people who have the ability to test this and get some benefit from it. We are probably about 6 months away from a new release of Debian and this is probably the last thing I need to do to make etbemon ready for that.

• [1] https://etbe.coker.com.au/2020/02/02/load-average-monitoring/
• [2] https://facebookmicrosites.github.io/psi/docs/overview.html

RISC-V is the latest RISC architecture that s become popular. It is the 5th RISC architecture from the University of California Berkeley. It seems to be a competitor to ARM due to not having license fees or restrictions on alterations to the architecture (something you have to pay extra for when using ARM). RISC-V seems the most popular architecture to implement in FPGA. When I first tried to run RISC-V under QEMU it didn t work, which was probably due to running Debian/Unstable on my QEMU/KVM system and there being QEMU bugs in Unstable at the time. I have just tried it again and got it working. The Debian Wiki page about RISC-V is pretty good [1]. The instructions there got it going for me. One thing I wasted some time on before reading that page was trying to get a netinst CD image, which is what I usually do for setting up a VM. Apparently there isn t RISC-V hardware that boots from a CD/DVD so there isn t a Debian netinst CD image. But debootstrap can install directly from the Debian web server (something I ve never wanted to do in the past) and that gave me a successful installation. Here are the commands I used to setup the base image:

apt-get install debootstrap qemu-user-static binfmt-support debian-ports-archive-keyring
debootstrap --arch=riscv64 --keyring /usr/share/keyrings/debian-ports-archive-keyring.gpg --include=debian-ports-archive-keyring unstable /mnt/tmp http://deb.debian.org/debian-ports

I first tried running RISC-V Qemu on Buster, but even ls didn t work properly and the installation failed.

chroot /mnt/tmp bin/bash
# ls -ld .
/usr/bin/ls: cannot access '.': Function not implemented

When I ran it on Unstable ls works but strace doesn t work in a chroot, this gave enough functionality to complete the installation.

chroot /mnt/tmp bin/bash
# strace ls -l
/usr/bin/strace: test_ptrace_get_syscall_info: PTRACE_TRACEME: Function not implemented
/usr/bin/strace: ptrace(PTRACE_TRACEME, ...): Function not implemented
/usr/bin/strace: PTRACE_SETOPTIONS: Function not implemented
/usr/bin/strace: detach: waitpid(1602629): No child processes
/usr/bin/strace: Process 1602629 detached

When running the VM the operation was noticably slower than the emulation of PPC64 and S/390x which both ran at an apparently normal speed. When running on a server with equivalent speed CPU a ssh login was obviously slower due to the CPU time taken for encryption, a ssh connection from a system on the same LAN took 6 seconds to connect. I presume that because RISC-V is a newer architecture there hasn t been as much effort made on optimising the Qemu emulation and that a future version of Qemu will be faster. But I don t think that Debian/Bullseye will give good Qemu performance for RISC-V, probably more changes are needed than can happen before the freeze. Maybe a version of Qemu with better RISC-V performance can be uploaded to backports some time after Bullseye is released. Here s the Qemu command I use to run RISC-V emulation:

qemu-system-riscv64 -machine virt -device virtio-blk-device,drive=hd0 -drive file=/vmstore/riscv,format=raw,id=hd0 -device virtio-blk-device,drive=hd1 -drive file=/vmswap/riscv,format=raw,id=hd1 -m 1024 -kernel /boot/riscv/vmlinux-5.10.0-1-riscv64 -initrd /boot/riscv/initrd.img-5.10.0-1-riscv64 -nographic -append net.ifnames=0 noresume security=selinux root=/dev/vda ro -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-device,rng=rng0 -device virtio-net-device,netdev=net0,mac=02:02:00:00:01:03 -netdev tap,id=net0,helper=/usr/lib/qemu/qemu-bridge-helper

Currently the program /usr/sbin/sefcontext_compile from the selinux-utils package needs execmem access on RISC-V while it doesn t on any other architecture I have tested. I don t know why and support for debugging such things seems to be in early stages of development, for example the execstack program doesn t work on RISC-V now. RISC-V emulation in Unstable seems adequate for people who are serious about RISC-V development. But if you want to just try a different architecture then PPC64 and S/390 will work better.

• [1] https://wiki.debian.org/RISC-V

It is undeniable that banning Donald Trump from Facebook, Twitter, and similar sites is a benefit for the moment. It may well save lives, perhaps lots of lives. But it raises quite a few troubling issues. First, as EFF points out, these platforms have privileged speakers with power, especially politicians, over regular users. For years now, it has been obvious to everyone that Donald Trump has been violating policies on both platforms, and yet they did little or nothing about it. The result we saw last week was entirely forseeable and indeed, WAS forseen, including by elements in those companies themselves. (ACLU also raises some good points) Contrast that with how others get treated. Facebook, two days after the coup attempt, banned Benjamin Wittes, apparently because he mentioned an Atlantic article opposed to nutcase conspiracy theories. The EFF has also documented many more egregious examples: taking down documentation of war crimes, childbirth images, black activists showing the racist messages they received, women discussing online harassment, etc. The list goes on; YouTube, for instance, has often been promoting far-right violent videos while removing peaceful LGBTQ ones. In short, have we simply achieved legal censorship by outsourcing it to dominant corporations? It is worth pausing at this point to recognize two important princples: First, that we do not see it as right to compel speech. Secondly, that there exist communications channels and other services that nobody is calling on to suspend Donald Trump. Let s dive into those a little bit. There have been no prominent calls for AT&T, Verizon, Gmail, or whomever provides Trump and his campaign with cell phones or email to suspend their service to him. Moreover, the gas stations that fuel his vehicles and the airports that service his plane continue to provide those services, and nobody has seriously questioned that, either. Even his Apple phone that he uses to post to Twitter remains, as far as I know, fully active. Secondly, imagine you were starting up a small web forum focused on raising tomato plants. It is, and should be, well within your rights to keep tomato-haters out, as well as people that have no interest in tomatoes but would rather talk about rutabagas, politics, or Mars. If you are going to host a forum about tomatoes, you have the right to keep it a forum about tomatoes; you cannot be forced to distribute someone else s speech. Likewise in traditional media, a newspaper cannot be forced to print every letter to the editor in full. In law, there is a notion of a common carrier, that provides services to the general public without discrimination. Phone companies and ISPs fall under this. Facebook, Twitter, and tomato sites don t. But consider what happens if Facebook bans you. You might be using Facebook-owned Whatsapp to communicate with family and friends, and suddenly find yourself unable to ask someone to pick you up. Or your treasured family photos might be in Facebook-owned Instagram, lost forever. It s not just Facebook; similar things happen with Google, locking people out of their phones and laptops, their emails, even their photos. Is it right that Facebook and Google aren t regulated as common carriers? Perhaps, or perhaps we need some line of demarcation between their speech-to-the-public services (Facebook timeline posts, YouTube) and private communication (Whatsapp, Gmail). It s a thorny issue; should government be regulating speech instead? That s also fraught. So is corporate control. Decentralization Helps Dramatically With email, you get to pick your email provider (yes, there are two or three big ones, but still plenty of others). Each email provider will have its own set of things it considers acceptable, and its own set of other servers and accounts it s willing to exchange mail with. (It is extremely common for mail providers to choose not to accept mail from various other mail servers based on ISP, IP address, reputation, and so forth.) What if we could do something like that for Twitter and Facebook? Let you join whatever instance you like. Maybe one instance is all about art and they don t talk about politics. Or another is all about Free Software and they don t have advertising. And then there are plenty of open instances that accept anything that s respectful. And, like email, people of one server can interact with those using another just as easily as if they were using the same one. Well, this isn t hypothetical; it already exists in the Fediverse. The most common option is Mastodon, and it so happens that a month ago I wrote about its benefits for other reasons, and included some links on getting started. There is no reason that we must all let our online speech be controlled by companies with a profit motive to keep hate speech on their platforms. There is no reason that we must all have a single set of rules, or accept strong corporate or government control, either. The quality of conversation on Mastodon is far higher than either Twitter or Facebook; decentralization works and it s here today.

Boost is a very large and comprehensive set of (peer-reviewed) libraries for the C++ programming language, containing well over 100 individual libraries. The BH package provides a sizeable subset of header-only libraries for use by R. Version 1.75.0 of Boost was released in December, right on schedule with their April, August and December releases. I now try to follow these releases at a lower (annual) cadence and prepared BH 1.75.0-0 in mid-December. Extensive reverse-depends checks revealed a need for changes in a handful of packages whose maintainers I contacted then. With one exception, everybody responded in kind and brought updated packages to CRAN which permitted us to upload the package there two days ago. And thanks to this planned and coordinated upload, the package is now available on CRAN a mere two days later. My thanks to the maintainers of these packages for helping it along; this prompt responses really are appreciated. The version on CRAN is the same as the one the drat announced in this tweet asking for testing help. If you installed that version, you are still current as no changes were required since December and CRAN now contains same file. This release adds one new library: Boost Beast, an http and websocket library built on top of Boost Asio. Other changes are highlighed below.

Changes in version 1.75.0-0 (2020-12-12)

• Removed file NAMESPACE as the package has neither R code, nor a shared library to load
• The file LICENSE_1_0.txt is now included (as requested in #73)
• Added new beast library (as requested in #74)
• Upgraded to Boost 1.75.0 (#75)

Via CRANberries, there is a diffstat report relative to the previous release. Its final line is quite impressive: 3485 files changed, 100838 insertions(+), 84890 deletions(-). Wow. Comments and suggestions about BH are welcome via the issue tracker at the GitHub repo. If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

weeklyOSM posts lots of interesting links. Here are some beautiful views of OpenStreetMap edits and renderings:

• OpenWhateverMap :: An Open Awsumnez Map: A quilt-like showcase of OpenStreetMap renderings
• Show Me The Way: shows ways as they are added to OpenStreetMap in (near) realtime
• OSM In Realtime: simple visualization of the changes made to OpenStreetMap in (near) real-time
• WHODIDIT: OpenStreetMap Changeset Analyzer which can also generate an RSS feed for an area, so one can stay on top of what happens to the bits of the map one cares about

Background I usually don t mind ads, as not as they not very intrusive. I get that the current media model is basically ad-funded, and that unless I want to pay $1/month or so to 50 web sites, I have to accept ads, so I don t run an ad-blocker. Sure, sometimes are annoying (hey YT, mid-roll ads are borderline), but I ve also seen many good ads, as in interesting or funny even. Well, I don t think I ever bought anything as direct result from ads, so I don t know how useful ads are for the companies, but hey, what do I care. Except there a few ad networks that run what I would say are basically revolting ads. Things I don t want to ever accidentally see while eating, or things that are really make you go WTF? Maybe you know them, maybe you don t, but I guess there are people who don t know how to clean their ears, or people for whom a fast 7 day weight loss routine actually works. Thankfully, most of the time I don t browse sites which use this networks, but randomly they do leak to even sites I do browse. If I m not very stressed already, I can ignore them, otherwise they really, really annoy me. Case in point, I was on Slashdot, and because I was logged on and recently had mod points, the right side column had a check-box disable ads . That sidebar had some relatively meaningful ads, like a VPN subscription (not that I would use it, but it is a tech thing), or even a book about Kali Linux, etc. etc. So I click the disable ads , and the right column goes away. I scroll down happily, only to be met, at the bottom, by the best way to clean your ear , the most 50 useless planes ever built (which had a drawing of something that was for sure never ever built outside of in movies), you won t believe how this child actor looks today , etc.

Solving the problem The above really, really pissed me off, so I went to search how to block ad network . To my surprise, the fix was not that simple, for standard users at least.

Method 1: hosts file The hosts file is reasonable as it is relatively cross-platform (Linux and Windows and Mac, I think), but how the heck do you edit hosts on your phone? And furthermore, it has some significant downsides. First, /etc/hosts lists individual hosts, so for an entire ad network, the example I had had two screens of host names. This is really unmaintainable, since rotating host names, or having a gazillion of them is trivial. Second, it cannot return negative answers. I.e. you have to give each of those hosts a valid IPv4/IPv6, and have something either reply with 404 or another 4xx response, or not listen on port 80/443. Too annoying. And finally, it s a client-side solution, so one would have to replicate it across all clients in a home, and keep it in sync.

Method 2: ad-blockers I dislike ad-blockers on principle, since they need wide permissions on all pages, but it is a recommended solution. However, to my surprise, one finds threads saying ad-blocker foo has whitelisted ad network bar, at which point you re WTF? Why do I use an ad-blocker if they get paid by the lowest of the ad networks to show the ads? And again, it s a client-side solution, and one would have to deploy it across N clients, and keep them in sync, etc.

Method 3: HTTP proxy blocking To my surprise, I didn t find this mentioned in a quick internet search. Well, HTTP proxies have long gone the way of the dodo due to HTTPs everywhere , and while one can still use them even with HTTPS, it s not that convenient:

• you need to tunnel all traffic through them, which might result in bottlenecks (especially for media playing/maybe video-conference/etc.).
• or even worse, there might be protocol issues/incompatibilities due to 100% tunneling.
• running a proxy opens up some potential security issues on the internal network, so you need to harden the proxy as well, and maintain it.
• you need to configure all clients to know about the proxy (via DHCP or manually), which might or might not work well, since it s client-dependent.
• you can only block at CONNECT level (host name), and you have to build up regexes for the host name.

On the good side, the actual blocking configuration is centralised, and the only distributed configuration is pointing the clients through the proxy. While I used to run a proxy back in HTTP times, the gains were significant back them (media elements caching, downloads caching, all with a slow pipe, etc.), but today is not worth it, so I ve stopped and won t bring a proxy back just for this.

Method 4: DNS resolver filtering After thinking through all the options, I thought - hey, a caching/recursive DNS resolver is what most people with a local network run, right? How difficult is to block at resolver level? and oh my, it is so trivial, for some resolvers at least. And yes, I didn t know about this a week ago

Response Policy Zones Now, of course, there is a standard for this, called Response Policy Zone, and which is supported across multiple resolvers. There are many tutorials on how to use RPZs to configure things, some of them quite detailed - e.g. this one, or a simple/more straightforward one here. The upstream BIND documentation also explains things quite well here, so you can go that route as well. It looks a bit hairy for me thought, but it works, and since it is a standard, it can be more easily deployed. There are many discussions on the internet about how to configure RPZs, how to not even resolve the names (if you re going to return something explicitly/statically), etc. so there are docs, but again it seems a bit overdone.

Resolver hooks There s another way too, if your resolver allows scripting. For example, the PowerDNS resolver allow Lua scripting, and has a relatively simple API at least, to me it looks way, way simpler than the RPZ equivalent. After 20 minutes of reading the docs, I ended up with this, IMO trivial, solution (in a file named e.g. rules.lua):

ads = newDS()
ads:add( 'evilads.com', 'evilads.well-known-cdn.com', 'moreads.net' )

function preresolve(dq)
  if ads:check(dq.qname) then
    dq.rcode = pdns.NXDOMAIN
    return true;
  end
  return false;
end

and that s it. Well, enable it/load the file in the configuration, but nothing else. Syntax is pretty straightforward, matching by suffix here, and if you need more complex stuff, you can of course do it; it s just Lua and a simple API. I don t see any immediate equivalent in Bind, so there s that, but if you can use PowerDNS, then the above solution seems simple for simple cases, and could be extended if needed (not sure in which cases). The only other thing one needs to do is to serve the local/custom resolver to all clients, whether desktop or mobile, and that s it. DNS server is bread-and-butter in DHCP, so better support than proxy, and once the host name has been (mis)resolved, nothing is involved anymore in the communication path. True, your name server might get higher CPU usage, but for home network, this should not be a problem. Can this filtering method (either RPZ or hooks) be worked around by ad networks? Sure, like anything. But changing the base domain is not fun. DNSSEC might break it (note Bind RPZ can be configure to ignore DNSSEC), but I m more worried about DNS-over-HTTPS, which I thought initially it s done for the user, but now I m not so sure anymore. Not being in control even of your own DNS resolver seems evil , but what do I know.

Combined authoritative + recursive solution This solution was provided by Guillem Jover, who uses unbound, which is a combined authoritative name server and recursive resolver in one, and dnsmasq (which is even more things, I think):

For my LANs I use unbound, and then block this kind of thing in /etc/unbound/unbound.conf.d/block.conf, with stuff like:

server:
 local-zone: adsite.example.com refuse

But then for things that are mobile, and might get out of the LAN, such as laptops, I also block with dnsmasq in /etc/dnsmasq.d/block.conf, with stuff like:

 address=/adsite.example.com/

I still use ublock-origin to block stuff at the browser level, though, for yet an extra layer of noise suppression. :)

Thanks for the info!

Happy browsing! 10 lines of Lua, and now for sure I m going to get even fatter without the this natural method will melt your belly fat in 7 days information. Or I will just throw away banana peels without knowing what I could do with hem. After a few days, I asked myself but ads are not so bad, why did I and then realised that yes, ads are not so bad anymore. And Slashdot actually loads faster So, happy browsing!

Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language and is widely used by (currently) 802 other packages on CRAN. This release was needed because we use the Matrix package for some (optional) tests related to sparse matrices, and a small and subtle change and refinement in the recent 1.3.0 release of Matrix required us to make an update for the testing. Nothing has changed in how we set up, or operate on, sparse matrices. My thanks to Binxiang and Martin Maechler for feedback and suggestion on the initial fix both Binxiang and I set up independently. At the same time we upgrade some package internals related to continuous integration (for that, also see my blog post and video from earlier this week). Lastly Conrad sent in a one-line upstream fix for dealing with NaN in sign(). The full set of changes follows.

Changes in RcppArmadillo version 0.10.1.2.2 (2021-01-08)

• Correct one unit test for Matrix 1.3.0-caused changed (Binxiang in #319 and Dirk in #322).
• Suppress one further warning from Matrix (Dirk)
• Apply an upstream NaN correction (Conrad in #321)
• Added GitHub Actions CI using run.sh from r-ci (Dirk)

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

If you want to set up debian.net subdomain, you may setup E-mail server on your own. But if there is not afford to setup it by yourself, you need external E-mail server. In this article, I'll explain how to use external mailbox service - Sakura Mailbox service. Prerequisite

• Owner of debian.net subdomain (fabre.debian.net)
• Have an account of Sakura mailbox service

www.sakura.ne.jp I've chosen Sakura mailbox service because of maintainance cost (87yen/month) Configure dnsZoneEntry Set dnZoneEntry, it means that gpg --clearsign and send to changes@db.debian.org. Here is an example for fabre.debian.net.

fabre IN A 140.227.125.189
mail.fabre IN A 49.212.207.226
fabre IN MX 10 mail.fabre.debian.net.

140.227.125.189 is Web server (https://fabre.debian.net) and 49.212.207.226 is E-mail server (Sakura mailbox service) for my case. Note that It varies for your case. DebianDotNet - Debian Wiki Configure Sakura mailbox service Set fabre.debian.net subdomain without transfer in Domain/SSL menu. Then create each account for @fabre.debian.net. After a while, you can use xxx@fabre.debian.net E-mail account. Yay!