Here s my (twelfth) monthly update about the activities I ve done in the F/L/OSS world.
Debian
This was my 21st month of contributing to Debian.
I became a DM in late March last year and a DD last Christmas! \o/
I ve been busy with my undergraduation stuff but I still squeezed out some time for the regular Debian work.
Here are the following things I did in Debian this month:
Sponsored trace-cmd for Sudip, ruby-asset-sync for Nilesh, and mariadb-mysql-kbs for William.
RuboCop::Packaging - Helping the Debian Ruby team! \o/
This Google Summer of Code, I worked on writing a linter that could flag offenses for lines of code
that are very troublesome for Debian maintainers while trying to package and maintain Ruby libraries and applications!
Whilst the GSoC period is over, I ve been working on improving that tool and have extended that linter to now auto-correct these offenses
by itself! \o/
You can now just use the -A flag and you re done! Boom! The ultimate game-changer!
Here s a quick demo for this feature:
A few quick updates on RuboCop::Packaging:
Has 4 cops, solving 4 different issues.
3 of them support auto-correction. Just use the -A flag.
I ve also spent a considerable amount of time in raising awareness about this and in more general sense, about downstream maintenance.
As a result, I raised a bunch of PRs which got really good response. I got all of the 20 PRs merged upstream,
fixing these issues.
Debian (E)LTS
Debian Long Term Support (LTS) is a project to extend the lifetime of all Debian stable releases to (at least) 5 years. Debian LTS is not handled by the Debian security team, but by a separate group of volunteers and companies interested in making it a success.
And Debian Extended LTS (ELTS) is its sister project, extending support to the Jessie release (+2 years after LTS support).
This was my twelfth month as a Debian LTS and third month as a Debian ELTS paid contributor.
I was assigned 19.75 hours for LTS and 15.00 hours for ELTS and worked on the following things:
(for LTS, I over-worked for 11 hours last month on the survey so only had 8.75 hours this month!)
LTS CVE Fixes and Announcements:
Issued DLA 2362-1, fixing CVE-2020-11984, for uwsgi.
For Debian 9 Stretch, these problems have been fixed in version 2.0.14+20161117-3+deb9u3.
Issued DLA 2363-1, fixing CVE-2020-17446, for asyncpg.
For Debian 9 Stretch, these problems have been fixed in version 0.8.4-1+deb9u1.
Issued ELA 274-1, fixing CVE-2020-11984, for uwsgi.
For Debian 8 Jessie, these problems have been fixed in version 2.0.7-1+deb8u3.
Issued ELA 275-1, fixing CVE-2020-14363, for libx11.
For Debian 8 Jessie, these problems have been fixed in version 2:1.6.2-3+deb8u4.
Issued ELA 278-1, fixing CVE-2020-8184, for ruby-rack.
For Debian 8 Jessie, these problems have been fixed in version 1.5.2-3+deb8u4.
Also worked on updating the version of clamAV from v0.101.5 to v0.102.4.
This was a bit tricky package to work on since it involved an ABI/API change and was more or less a transition.
Super thanks to Emilio for his invaluable help and him taking over the package, finishing, and uploading it in the end.
Other (E)LTS Work:
Front-desk duty from 31-08 to 06-09 and from 28-09 onward for both LTS and ELTS.
I keep forgetting how to make presentations. I had a list of tools in
a wiki from a previous job, but that's now private and I don't see why
I shouldn't share this (even if for myself!).
So here it is. What's your favorite presentation tool?
Tips
if you have some text to present, outline keywords so that you
can present your subject without reading every word
ideally, don't read from your slides - they are there to help
people follow, not for people to read
even better: make your slides pretty with only a few words, or
don't make slides at all
I'm currently using Pandoc with PDF input (with a trip through LaTeX)
for most slides, because PDFs are more reliable and portable than web
pages. I've also used Libreoffice, Pinpoint, and S5 (through RST) in
the past. I miss Pinpoint, too bad that it died.
Some of my presentations are available in my GitLab.com account:
Other options
Another option I have seriously considered is just generate a series
of images with good resolution, hopefully matching the resolution (or
at least aspect ratio) of the output device. Then you flip through a
series of images one by one. In that case, any of those image viewers
(not an exhaustive list) would work:
Update: it turns out I already wrote a somewhat similar thing when I
did a recent presentation. If you're into rants, you might enjoy the
README file accompanying the Kubecon rant presentation. TL;DR:
"makes me want to scream" and "yet another unsolved problem space,
sigh" (refering to "display images full-screen" specifically).
Running new hardware is always fun.
The problems are endless.
The solutions not so much.
So I've got a brand new ThinkPad X13 AMD.
It features an AMD Ryzen 5 PRO 4650U, 16GB of RAM and a 256GB NVME SSD.
The internal type identifier is 20UF.
It runs the latest firmware as of today with version 1.09.
So far I found two problems with it:
It refuses to boot my Debian image with Secure Boot enabled.
It produces ACPI errors on every key press on the internal keyboard.
Disable Secure Boot
The system silently fails to boot a signed shim and grub from an USB thumb drive.
I used on of the Debian Cloud images, which should properly work in this setup and do on my other systems.
The only fix I found was to disable Secure Boot alltogether.
Select Linux in firmware
Running a Linux 5.8 with default firmware setting produces ACPI errors on each key press.
ACPI BIOS Error (bug): Could not resolve symbol [\_SB.PCI0.GPP3], AE_NOT_FOUND (20200528/psargs-330)ACPI Error: Aborting method \_SB.GPIO._EVT due to previous error (AE_NOT_FOUND) (20200528/psparse-529)
This can be "fixed" by setting a strategic setting inside the firmware: Config > Power > Sleep State to Linux
Here is my monthly update covering what I have been doing in the free software world during September 2020 (previous month):
Updated my tickle-me-email library (which implements Getting Things Done (GTD)-like behaviours in IMAP inboxes) to support adding 'TODO' entries as read or unread based on a runtime configuration parameter. [...]
As part of my role of being the assistant Secretary of the Open Source Initiative and a board director of Software in the Public Interest, I attended their respective monthly meetings and participated in various licensing and other discussions occurring on the internet as well as the usual internal discussions, etc. I participated in the OSI's inaugural State of the Source conference and began the 'onboarding' of a new project to SPI.
Reproducible Builds
One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. However, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into ostensibly secure software during the various compilation and distribution processes.
The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.
The project is proud to be a member project of the Software Freedom Conservancy. Conservancy acts as a corporate umbrella allowing projects to operate as non-profit initiatives without managing their own corporate structure. If you like the work of the Conservancy or the Reproducible Builds project, please consider becoming an official supporter.
This month, I:
Also include the general news in our RSS feed [...] and drop including weekly reports from the RSS feed (they are never shown now that we have over 10 items) [...].
Update ordering and location of various news and links to tarballs, etc. [...][...][...]
Make the (long) demonstration GIF less prominent. [...]
Debian
Lintian
For Lintian, the static analysis tool for Debian packages, I uploaded versions 2.93.0, 2.94.0, 2.95.0 & 2.96.0 (not counting uploads to the backports repositories), as well as:
Frontdesk duties, responding to user/developer questions, reviewing others' packages, attending our monthly IRC meeting, participating in mailing list discussions, etc.
Issued DLA 2368-1 for grunt to fix an arbitrary code execution
vulnerability due to the unsafe loading of YAML documents.
Issued DLA 2370-1 and ELA-281-1 for the python-pip Python package installer to fix a directory traversal attack where arbitrary local files (eg. /root/.ssh/authorized_keys) could be overridden.
Issued DLA 2372-1 and ELA-282-1 to prevent a denial of service attack in libproxy, a library to make applications HTTP proxy-aware.
Issued DLA 2374-1 for the gnome-shell component of the GNOME desktop. In certain configurations, when logging out of an account, the password box from the login dialog could reappear with the password visible in plaintext.
Issued DLA 2380-1 for ruby-gon, a library to send/convert data to
Javascript from Ruby applications, to prevent cross-site scripting (XSS) vulnerabilities.
You can find out more about the project via the following video:
Uploads
Before and during FOSDEM 2020, I agreed with the people (developers, supporters, managers) of the UBports Foundation to package the Unity8 Operating Environment for Debian. Since 27th Feb 2020, Unity8 has now become Lomiri.
Recent Uploads to Debian related to Lomiri
Over the past 4 months I worked on the following bits and pieces regarding Lomiri in Debian:
Work on lomiri-app-launch (Debian packaging, upstream work, upload to Debian)
Fork lomiri-url-dispatcher from url-dispatcher (upstream work)
Upload lomiri-url-dispatcher to Debian
Fork out suru-icon-theme and make it its own upstream project
Package and upload suru-icon-theme to Debian
First glance at lomiri-ui-toolkit (currently FTBFS, needs to be revisited)
Update of Mir (1.7.0 -> 1.8.0) in Debian
Fix net-cpp FTBFS in Debian
Fix FTBFS in gsettings-qt.
Fix FTBFS in mir (support of binary-only and arch-indep-only builds)
Coordinate with Marius Gripsgard and Robert Tari on shift over from Ubuntu Indicator to Ayatana Indicators
Upload ayatana-indicator-* (and libraries) to Debian (new upstream releases)
Package and upload to Debian: qmenumodel (still in Debian's NEW queue)
Package and upload to Debian: ayatana-indicator-sound
Symbol-Updates (various packages) for non-standard architectures
Fix FTBFS of qtpim-opensource-src in Debian since Qt5.14 had landed in unstable
Fix FTBFS on non-standard architectures of qtsystems, qtpim and qtfeedback
Fix wlcs in Debian (for non-standard architectures), more Symbol-Updates (esp. for the mir DEB package)
Symbol-Updates (mir, fix upstream tinkering with debian/libmiral3.symbols)
Fix FTBFS in lomiri-url-dispatcher against Debian unstable, file merge request upstream
Upstream release of qtmir 0.6.1 (via merge request)
Improve check_whitespace.py script as used in lomiri-api to ignore debian/ subfolder
Upstream release of lomiri-api 0.1.1 and upload to Debian unstable.
The next two big projects / packages ahead are lomiri-ui-toolkit and qtmir.
Credits
Many big thanks go to Marius and Dalton for their work on the UBports project and being always available for questions, feedback, etc.
Thanks to Ratchanan Srirattanamet for providing some of his time for debugging some non-thread safe unit tests (currently unsure, what package we actually looked at...).
Thanks for Florian Leeber for being my point of contact for topcis regarding my cooperation with the UBports Foundation.
Previous Posts about my Debian UBports Team Efforts
When building route filters with bgpq4 or bgpq3, the speed of
rr.ntt.net or whois.radb.net can be a bottleneck. Updating many
filters may take several tens of minutes, depending on the load:
$time bgpq4 -h whois.radb.net AS-HURRICANE wc -l
9098691.96s user 0.15s system 2% cpu 1:17.64 total$time bgpq4 -h rr.ntt.net AS-HURRICANE wc -l
9278651.86s user 0.08s system 12% cpu 14.098 total
A possible solution is to run your own IRRd instance in your
network, mirroring the main routing registries. A close
alternative is to bundle IRRd with all the data in a
ready-to-use Docker image. This also has the advantage of easy
integration into a Docker-based CI/CD pipeline.
$ git clone https://github.com/vincentbernat/irrd-legacy.git -b blade/master
$cd irrd-legacy
$ docker build . -t irrd-snapshot:latest
[ ]Successfully built 58c3e83a1d18Successfully tagged irrd-snapshot:latest$ docker container run --rm --detach --publish=43:43 irrd-snapshot
4879cfe7413075a0c217089dcac91ed356424c6b88808d8fcb01dc00eafcc8c7$time bgpq4 -h localhost AS-HURRICANE wc -l
9041371.72s user 0.11s system 96% cpu 1.881 total
assembling the final container with the result of the two previous stages.
The second stage fetches the databases used by rr.ntt.net:
NTTCOM, RADB, RIPE, ALTDB, BELL, LEVEL3, RGNET, APNIC, JPIRR, ARIN,
BBOI, TC, AFRINIC, ARIN-WHOIS, and REGISTROBR. However, it misses
RPKI.2 Feel free to adapt!
The image can be scheduled to be rebuilt daily or weekly, depending on
your needs. The repository includes a .gitlab-ci.yaml
file automating the build and triggering the compilation
of all filters by your CI/CD upon success.
Instead of using the latest version of IRRd, the image
relies on an older version that does not require a PostgreSQL
instance and uses flat files instead.
Unlike the others, the RPKI database is built from the
published RPKI ROAs. They can be retrieved with rpki-client and
transformed into RPSL objects to be imported in IRRd.
I always had the feeling that my server, hosted by Hetzner, somehow has a slow internet connection. Then, I did put it on the distance between Finland and Japan, and didn t care too much. Until yesterday my server stopped reacting to pings/ssh, and needed a hard reset. It turned out that the server was running fine, only that the ethernet card did hang. Hetzner support answered promptly and directed me to this web page, which described a change in the kernel concerning fragmentation offloading, and suggested the following configuration to regain connection speed:
ethtool -K <interface> tso off gso off
And to my surprise, this simple thing did wonder, and the connection speed improved dramatically, even from Japan (something like factor 10 in large rsync transfers). I have added this incantation to system cron tab and run it every hour, just to be sure that even after a reboot it is fine.
If you have bad connection speed with this kind of ethernet card, give it a try.
It is known that Debian Developer can setup *.debian.net.
wiki.debian.org
When Debian Developer had retired, actual DNS entry is removed, but dnsZoneEntry: field
is kept on LDAP (db.debian.org)
So you can not reuse *.debian.net if retired Debian Developer owns your prefered subdomain already.
I've posted question about this current undocumented specification.
lists.debian.org
Internet is split into five regional Internet registry: AFRINIC,
ARIN, APNIC, LACNIC and RIPE. Each RIR maintains an Internet Routing
Registry. An IRR allows one to publish information about the
routing of Internet number resources.1 Operators use this to
determine the owner of an IP address and to construct and maintain
routing filters. To ensure your routes are widely accepted, it is
important to keep the prefixes you announce up-to-date in an IRR.
There are two common tools to query this database: whois and
bgpq4. The first one allows you to do a query with the WHOIS
protocol:
There is no module available on Ansible Galaxy to manage these
objects. Each IRR has different ways of being updated. Some RIRs
propose an API but some don t. If we restrict ourselves to RIPE, ARIN
and APNIC, the only common method to update objects is email
updates, authenticated with a password or a GPG signature.2
Let s write a custom Ansible module for this purpose!
Prerequisites
The source file should be a set of objects to sync using the RPSL
language. This would be the same content you would send manually by
email. All objects should be managed by the same maintainer, which is
also provided as a parameter.
Signing and sending the result is not the responsibility of this
module. You need two additional tasks for this purpose:
You also need to authorize the PGP keys used to sign the updates by
creating a key-cert object and adding it as a valid authentication
method for the corresponding mntner object:
key-cert:PGPKEY-A791AAABcertif:-----BEGIN PGP PUBLIC KEY BLOCK-----certif:certif:mQGNBF8TLY8BDADEwP3a6/vRhEERBIaPUAFnr23zKCNt5YhWRZyt50mKq1RmQBBY[]certif:-----END PGP PUBLIC KEY BLOCK-----mnt-by:fr-blade-1-mntsource:RIPEmntner:fr-blade-1-mnt[]auth:PGPKEY-A791AAABmnt-by:fr-blade-1-mntsource:RIPE
Module definition
Starting from the skeleton described in the previous article, we define the module:
Getting existing objects
To grab existing objects, we use the whois command to retrieve all
the objects from the provided maintainer.
# Per-IRR variations:# - whois serverwhois='ARIN':'rr.arin.net','RIPE':'whois.ripe.net','APNIC':'whois.apnic.net'# - whois optionsoptions='ARIN':['-r'],'RIPE':['-BrG'],'APNIC':['-BrG']# - objects excluded from synchronizationexcluded=["domain"]ifirr=="ARIN":# ARIN does not return these objectsexcluded.extend(["key-cert","mntner",])# Grab existing objectsargs=["-h",whois[irr],"-s",irr,*options[irr],"-i","mnt-by",module.params['mntner']]proc=subprocess.run("whois",*args,capture_output=True)ifproc.returncode!=0:raiseAnsibleError(f"unable to query whois: args")output=proc.stdout.decode('ascii')got=extract(output,excluded)
The first part of the code setup some IRR-specific constants: the
server to query, the options to provide to the whois command and the
objects to exclude from synchronization. The second part invokes the
whois command, requesting all objects whose mnt-by field is the
provided maintainer. Here is an example of output:
The result is passed to the extract() function. It parses and
normalizes the results into a dictionary mapping object names to
objects. We store the result in the got variable.
defextract(raw,excluded):"""Extract objects."""# First step, remove comments and unwanted linesobjects="\n".join([objforobjinraw.split("\n")ifnotobj.startswith(("#","%",))])# Second step, split objectsobjects=[RPSLObject(obj.strip())forobjinre.split(r"\n\n+",objects)ifobj.strip()andnotobj.startswith(tuple(f"x:"forxinexcluded))]# Last step, put objects in a dictobjects=repr(obj):objforobjinobjectsreturnobjects
RPSLObject() is a class enabling normalization and comparison of
objects. Look at the module code for more details.
Returning updates
The module does not have a side effect. If there is a difference, we
return the updates to send by email. We choose to include all wanted
objects in the updates (contained in the source variable) and let
the IRR ignore unmodified objects. We also append the objects to be
deleted by adding a delete: attribute to each them them.
# We send all source objects and deleted objects.deleted_mark=f"'delete:':16deleted by CMDB"deleted="\n\n".join([f"got[k].raw\ndeleted_mark"forkingotifknotinwanted])result['objects']=f"source\n\ndeleted"module.exit_json(**result)
The complete code is available on GitHub. The module
supports both --diff and --check flags. It does not return
anything if no change is detected. It can work with APNIC, RIPE and
ARIN. It is not perfect: it may not detect some changes,3 it
is not able to modify objects not owned by the provided
maintainer4 and some attributes cannot be modified, requiring
to manually delete and recreate the updated object.5 However,
this module should automate 95% of your needs.
Other IRRs exist without being attached to a RIR. The most
notable one is RADb.
ARIN is phasing out this method in favor of IRR-online.
RIPE has an API available, but email updates are still
supported and not planned to be deprecated. APNIC plans to expose
an API.
For ARIN, we cannot query key-cert and mntner objects
and therefore we cannot detect changes in them. It is also not
possible to detect changes to the auth mechanisms of a mntner
object.
APNIC do not assign top-level objects to the maintainer
associated with the owner.
Changing the status of an inetnum object requires
deleting and recreating the object.
Update 2020-09-30: The post has created considerable movement, and a PR request by Fedora developers to rebase cjs onto more current gjs and libmozjs78 is being tested. I have uploaded packages of cinnamon and cjs to experimental based on these patches (400 files, about 50000 lines of code touched, not what I normally like to have in a debian patch) and would appreciate testing and feedback.
I have been more or less maintaining Cinnamon now for quite some time, but using it only sporadically due to my switch to KDE/Plasma. Currently, Cinnamon s cjs package depends on mozjs52, which also is probably going to be orphaned soon. This will precipitate a lot of changes, not the least being Cinnamon being removed from Debian/testing.
I have pinged upstream several times, without much success. So for now the future looks bleak for cinnamon in Debian. If there are interested developers (Debian or not), please get in touch with me, or directly try to update cjs to mozjs78.
Italy during the cold war has always been in too strategic a position, and with
too strong a left wing movement, not to get the CIA involved.
Here are a few stories of coup d' tat and other kinds of efforts to manipulate
Italian politics:
For almost a decade, I ve been very slowly making progress on a multicast IPTV
system. Recently I ve made a significant leap forward in this project, and I
wanted to write a little on the topic so I ll have something to look at when I
pick this up next. I was aspiring to have a useable system by the end of today,
but for a couple of reasons, it wasn t possible.
When I started thinking about this project, it was still common to watch
broadcast television. Over time the design of this system has been changing as
new technologies have become available. Multicast IP is probably the only
constant, although I m now looking at IPv6 rather than IPv4.
Initially, I d been looking at DVB-T PCI cards. USB
devices have become common
and are available cheaply. There are also DVB-T
hats available for the
Raspberry Pi. I m now looking at a combination of Raspberry Pi hats and USB
devices with one of each on a couple of Pis.
Two Raspberry Pis with DVB hats installed, TV antenna sockets showing
The Raspberry Pi devices will run
DVBlast, an open-source DVB
demultiplexer and streaming server. Each of the tuners will be tuned to a
different transponder giving me the ability to stream any combination of
available channels simultaneously. This is everything that would be needed to
watch TV on PCs on the home network with VLC.
I ve not yet worked out if Kodi will accept multicast streams as a TV source,
but I do know that Tvheadend will. Tvheadend can also
act as a PVR to record programmes for later playback so is useful even if the
multicast streams can be viewed directly.
So how far did I get? I have built two Raspberry Pis in cases with the DVB-T
hats on. They need to sit in the lounge as that s where the antenna comes down
from the roof. There s no wired network connection in the lounge. I planned to
use an OpenBSD box as a gateway, bridging the wireless network to a wired
network.
Two problems quickly emerged. The first being that the wireless card I had
purchased only supported 2.4GHz, no 5GHz, and I have enough noise from
neighbours that the throughput rate and packet loss are unacceptable.
The second problem is that I had forgotten the problems with bridging wireless
networks. To create a bridge, you need to be able to spoof the MAC
addresses of wired
devices on the wireless interface, but this can only be done when the wireless
interface is in access point mode.
So when I come back to this, I will have to look at routing rather than
bridging to work around the MAC address issue, and I ll also be on the lookout
for a cheap OpenBSD supported mini-PCIe wireless card that can do 5GHz.
Two years ago, I held the CIS194 minicourse on Haskell at the University of Pennsylvania. In that installment of the course, I changed the first four weeks to teach the basics of Haskell using the online Haskell environment CodeWorld, and lead the students towards implementing the game Sokoban.
As it is customary for CIS194, I put my lecture notes and exercises online, and this has been used as a learning resources by people from all over the world. But since I have left the University of Pennsylvania, I lost the ability to update the text, and as the CodeWorld API has evolved, some of the examples and exercises no longer work.
Some recent complains about that, in bug reports against CodeWorld and in unrealistically flattering tweets ( Shame, this was the best Haskell course ever!!! ) motivated me to extract that material and turn it into an updated stand-alone tutorial that I can host myself.
So if you feel like learning Haskell without worrying about local installation, and while creating a reasonably fun game, head over to https://haskell-via-sokoban.nomeata.de/ and get started! Improvements can now also be contributed at https://github.com/nomeata/haskell-via-sokoban.
Credits go to Brent Yorgey, Richard Eisenberg and Noam Zilberstein, who held the previous installments of the course, and Chris Smith for creating the CodeWorld environment.
A new release 0.2.0 of pkgKitten just hit on CRAN today, or about eleven months after the previous release.
This release brings support for tinytest by having pkgKitten::kitten() automagically call tinytest::puppy() if the latter package is installed (and the user did not opt out of calling it). So your newly created minimal package now also uses a wonderful yet tiny testing framework. We also added a new documentation site using the previously tweeted-about wrapper for Material for MkDocs I really dig. And last but not least we switched to BSPM-based Continued Integration (which I wrote about yesterday in R4 #30) and fixed one bug regarding the default NAMESPACE file.
Changes in version 0.2.0 (2020-09-27)
Continuous Integration uses the updated BSPM-based script on Travis and with GitHub Actions (Dirk in #11 plus earlier commits).
A new default NAMESPACE file is now installed (Dirk in #12).
A package documentation website was added (Dirk in #13).
Call tinytest::puppy if installed and not opted out (Dirk in #14).
Winding down slightly - we've worked our way through most of the images and testing. Schweer tested all of the Debian Edu/Skolelinux images for which many thanksSledge, RattusRattus, Isy and I have been working pretty much solidly for 10 3/4 hours. There's still some images to build - mips, mipsel and s390x but these are all images that we don't have hardware to test on particularly. Another good and useful day - bits and pieces done throughout. NOTE: There appear to have been some security updates since the main release this morning so, as ever, it's worth updating machines on a regular basis.Waiting for the final images to finish building so that we can check the archive for completeness and then publish to the media mirrors. All the best until next time: thanks as ever to Sledge for his invaluable help. See you again in a couple of months in all likelihood. A much smaller release: some time in the next month we hope to be able to build and test an Alpha release for Bullseye. Bullseye is likely to be released somewhere round the middle of next year so we'll have additional Buster stable point releases in the meantime.
Nothing particularly new or unexpected: there's a point release happening at some point this weekend for Debian stable. Usual rules apply: if you've already got a system current and up to date, there's not much to do but the base files version will change at some point to reflect 10.6 when you next update. If you have media from 10.5, you may not _have_ to go and get media this weekend but it's always useful to get new media in due course. There's an updated kernel and an ABI bump. You _will_ need to reboot at some time to use the new kernel image. This point release will contain security fixes, consequent changes etc. as usual - it is always good and useful to keep machines up to date.Working with the CD team to eventually test, build and release CD / DVD images and media as and when files gradually become available. As ever, this may take 12-16 hours. As ever, I'll post some blog entries as we go.Currently "sitting in Cambridge" via video link with Sledge, RattusRattus and Isy who are all involved in the testing and we'll have a great day, as ever.
I ran into filesystem corruption
(ext4) on the root partition of my
backup server
which caused it to go into read-only mode. Since it's the root partition,
it's not possible to unmount it and repair it while it's running. Normally I
would boot from an Ubuntu live CD / USB
stick, but in this case
the machine is using the
mipsel architecture and
so that's not an option.
Repair using a USB enclosure
I had to pull the shutdown the server and then pull the SSD drive out. I
then moved it to an external USB enclosure and connected it to my laptop.
I started with an automatic filesystem repair:
fsck.ext4 -pf /dev/sde2
which failed for some reason and so I moved to an interactive repair:
fsck.ext4 -f /dev/sde2
Once all of the errors were fixed, I ran a full surface scan to update the
list of bad blocks:
fsck.ext4 -c /dev/sde2
Finally, I forced another check to make sure that everything was fixed at
the filesystem level:
fsck.ext4 -f /dev/sde2
Fix invalid alternate GPT
The other thing I noticed is this messge in my dmesg log:
scsi 8:0:0:0: Direct-Access KINGSTON SA400S37120 SBFK PQ: 0 ANSI: 6
sd 8:0:0:0: Attached scsi generic sg4 type 0
sd 8:0:0:0: [sde] 234441644 512-byte logical blocks: (120 GB/112 GiB)
sd 8:0:0:0: [sde] Write Protect is off
sd 8:0:0:0: [sde] Mode Sense: 31 00 00 00
sd 8:0:0:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sd 8:0:0:0: [sde] Optimal transfer size 33553920 bytes
Alternate GPT is invalid, using primary GPT.
sde: sde1 sde2
I therefore checked to see if the partition table looked fine and got the
following:
$ fdisk -l /dev/sde
GPT PMBR size mismatch (234441643 != 234441647) will be corrected by write.
The backup GPT table is not on the end of the device. This problem will be corrected by write.
Disk /dev/sde: 111.8 GiB, 120034123776 bytes, 234441648 sectors
Disk model: KINGSTON SA400S3
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 799CD830-526B-42CE-8EE7-8C94EF098D46
Device Start End Sectors Size Type
/dev/sde1 2048 8390655 8388608 4G Linux swap
/dev/sde2 8390656 234441614 226050959 107.8G Linux filesystem
It turns out that all I had to do, since only the backup / alternate GPT
partition table was corrupt and the primary one was fine, was to re-write
the partition table:
$ fdisk /dev/sde
Welcome to fdisk (util-linux 2.33.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
GPT PMBR size mismatch (234441643 != 234441647) will be corrected by write.
The backup GPT table is not on the end of the device. This problem will be corrected by write.
Command (m for help): w
The partition table has been altered.
Syncing disks.
Run SMART checks
Since I still didn't know what caused the filesystem corruption in the first
place, I decided to do one last check:
SMART errors.
I couldn't do this via the USB enclosure since the SMART commands aren't
forwarded to the drive and so I popped the drive back into the backup
server and booted it up.
First, I checked whether any SMART errors had been reported using
smartmontools:
smartctl -a /dev/sda
That didn't show any errors and so I kicked off an extended test:
smartctl -t long /dev/sda
which ran for 30 minutes and then passed without any errors.
The mystery remains unsolved.
Welcome to the 30th post in the rarified R recommendation resources series or R4 for short. The last post introduced BSPM. In the four weeks since, we have worked some more on BSPM to bring it to the point where it is ready for use with continuous integration. Building on this, it is now used inside the run.sh script that driven our CI use for many years (via the r-travis repo).
Which we actually use right now on three different platforms:
All three use the exact same script facilitating this, and run a matrix over Linux and macOS. You read this right: one CI setup that is portable and which you can take to your CI provider of choice. No lock-in or tie-in. Use what works, change at will. Or run on all three if you like burning extra cycles.
This is already used by handful of my repos as well as by at least two repos of friends also deploying r-travis. How does it work? In a nutshell we are
downloading run.sh via curl and changing its mode;
running run.sh bootstrap which sets the operating system default:
on Linux we use Ubuntu,
add two PPAs repos for R itself and over 4600 r-cran-* binaries,
and enable BSPM to use these from install.packages()
on macOS we use the standard setup also used on Travis, GitHub Actions and elsewhere;
this provides us with fast, reliable, easy, and portable access to binaries on two OSs under dependency resolution;
running run.sh install_deps to install just the requireded Depends:, Imports: and LinkingTo:
running run.sh tests to build the tarball and test it via R CMD check --as-cran.
There are several customizations that are possible via environment variables
additional PPAs or drat repos can be added to offer even more package choice;
alternatively one could run run.sh install_all to also install Suggests:;
optionally one could run run.sh install_r pkgA pkgB ... to install packages explicitly listed;
optionally one could also run run.sh install_aptget r-cran-pkga r-cran-pkgb otherpackage to add more Ubuntu binaries.
We find this setup compelling. The scheme is simple: there really is just one shell script behind it which can also be downloaded and altered. The scheme is also portable as we can (as shown) rotate between CI provides. The scheme is also more flexible: in case of debugging needs one can simply run the script on a local Docker or VM instance. Lastly, the scheme moves away from single points of failure or breakage.
Currently the script uses only BSPM as I had the hunch that it would a) work and b) be compelling. Adding support for RSPM would be equally easy, but I have no immediate need to do so. Adding BioConductor installation may be next. That is easy when BioConductor uses r-release; it may be little more challenging under r-devel to but it should work too. Stay tuned.
In the meantime, if the above sounds compelling, give run.sh from r-travis a go!
If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.
We're working our way through some of the CD/DVD/Blu-Ray media images, doing test installs, noting failures and so on. It's repetitive work but vital if we're going to provide some assurance that folk can install from the images we make. There's always the few things that catch us out and there's always something to note for next time. Schweer has joined us and is busy chasing down debian-edu/Skolelinux installs from Germany. We're getting there, one way and another, and significantly ahead of where we were last time around when the gremlins got in and delayed us. All good :)
Participating in the Debian media testing on debian-cd. One of my colleagues has just spent time to sort out an email issue having spent a couple of hours with me the other night. I now have good, working email for the first time in years - I can't value that highly enough.Sledge, RattusRattus, Isy and myself are all engaged in testing various CD images. At the same time, they're debugging a new application to save us from wiki problems when we do this - and we're also able to use a video link which is really handy to chat backwards and forwards and means I can sit virtually in Cambridge :)Lots of backchat and messages flying backwards and forwards - couldn't wish for a better way to spend an afternoon with friends.
This Google Summer of Code, I worked on writing a linter that could flag offenses for lines of code
that are very troublesome for Debian maintainers while trying to package and maintain Ruby libraries and applications!
Whilst the GSoC period is over, I ve been working on improving that tool and have extended that linter to now auto-correct these offenses
by itself! \o/
A few quick updates on RuboCop::Packaging:
Here is my monthly update covering what I have been doing in the free software world during September 2020 (
Before and during FOSDEM 2020, I agreed with the people (developers, supporters, managers) of the UBports Foundation to package the Unity8 Operating Environment for Debian. Since 27th Feb 2020, Unity8 has now become Lomiri.
Recent Uploads to Debian related to Lomiri
Over the past 4 months I worked on the following bits and pieces regarding Lomiri in Debian:
I always had the feeling that my server, hosted by Hetzner, somehow has a slow internet connection. Then, I did put it on the distance between Finland and Japan, and didn t care too much. Until yesterday my server stopped reacting to pings/ssh, and needed a hard reset. It turned out that the server was running fine, only that the ethernet card did hang. Hetzner support answered promptly and directed me to this 
For almost a decade, I ve been very slowly making progress on a multicast IPTV
system. Recently I ve made a significant leap forward in this project, and I
wanted to write a little on the topic so I ll have something to look at when I
pick this up next. I was aspiring to have a useable system by the end of today,
but for a couple of reasons, it wasn t possible.
When I started thinking about this project, it was still common to watch
broadcast television. Over time the design of this system has been changing as
new technologies have become available. Multicast IP is probably the only
constant, although I m now looking at IPv6 rather than IPv4.
Initially, I d been looking at DVB-T PCI cards. 
Two Raspberry Pis with DVB hats installed, TV antenna sockets showing
Two years ago, I held the 
A new release 0.2.0 of