This post was supposed to be called “death with bcachefs”, but it sounded a
bit too dramatic. :-) Evidently bcachefs-tools in Debian is finally getting
an update (although in experimental), so that's good. Meanwhile, one of my
multi-device filesystems died a horrible death, and since I had backups,
I didn't ask for its fix to be prioritized—fsck still is unable to repair it
and I don't use bcachefs on that machine anymore. But the other one still
lives fairly happily.
Hanging around #bcachefs on IRC tells me that indeed, this thing is still
quite experimental. Some of the killer features (like proper compression)
don't perform as well as they should yet. Large rewrites are still happening.
People are still reporting quite weird bugs that are being triaged and
mostly fixed (although if you can't reproduce them, you're pretty much
hosed). But it's a fun ride. Again: Have backups. They saved me. :-)
It’s been around 6 months since the GNOME Foundation was joined by our new Executive Director, Holly Million, and the board and I wanted to update members on the Foundation’s current status and some exciting upcoming changes.
Finances
As you may be aware, the GNOME Foundation has operated at a deficit (nonprofit speak for a loss – ie spending more than we’ve been raising each year) for over three years, essentially running the Foundation on reserves from some substantial donations received 4-5 years ago. The Foundation has a reserves policy which specifies a minimum amount of money we have to keep in our accounts. This is so that if there is a significant interruption to our usual income, we can preserve our core operations while we work on new funding sources. We’ve now “hit the buffers” of this reserves policy, meaning the Board can’t approve any more deficit budgets – to keep spending at the same level we must increase our income.
One of the board’s top priorities in hiring Holly was therefore her experience in communications and fundraising, and building broader and more diverse support for our mission and work. Her goals since joining – as well as building her familiarity with the community and project – have been to set up better financial controls and reporting, develop a strategic plan, and start fundraising. You may have noticed the Foundation being more cautious with spending this year, because Holly prepared a break-even budget for the Board to approve in October, so that we can steady the ship while we prepare and launch our new fundraising initiatives.
Strategy & Fundraising
The biggest prerequisite for fundraising is a clear strategy – we need to explain what we’re doing and why it’s important, and use that to convince people to support our plans. I’m very pleased to report that Holly has been working hard on this and meeting with many stakeholders across the community, and has prepared a detailed and insightful five year strategic plan. The plan defines the areas where the Foundation will prioritise, develop and fund initiatives to support and grow the GNOME project and community. The board has approved a draft version of this plan, and over the coming weeks Holly and the Foundation team will be sharing this plan and running a consultation process to gather feedback input from GNOME foundation and community members.
In parallel, Holly has been working on a fundraising plan to stabilise the Foundation, growing our revenue and ability to deliver on these plans. We will be launching a variety of fundraising activities over the coming months, including a development fund for people to directly support GNOME development, working with professional grant writers and managers to apply for government and private foundation funding opportunities, and building better communications to explain the importance of our work to corporate and individual donors.
Board Development
Another observation that Holly had since joining was that we had, by general nonprofit standards, a very small board of just 7 directors. While we do have some committees which have (very much appreciated!) volunteers from outside the board, our officers are usually appointed from within the board, and many board members end up serving on multiple committees and wearing several hats. It also means the number of perspectives on the board is limited and less representative of the diverse contributors and users that make up the GNOME community.
Holly has been working with the board and the governance committee to reduce how much we ask from individual board members, and improve representation from the community within the Foundation’s governance. Firstly, the board has decided to increase its size from 7 to 9 members, effective from the upcoming elections this May & June, allowing more voices to be heard within the board discussions. After that, we’re going to be working on opening up the board to more participants, creating non-voting officer seats to represent certain regions or interests from across the community, and take part in committees and board meetings. These new non-voting roles are likely to be appointed with some kind of application process, and we’ll share details about these roles and how to be considered for them as we refine our plans over the coming year.
Elections
We’re really excited to develop and share these plans and increase the ways that people can get involved in shaping the Foundation’s strategy and how we raise and spend money to support and grow the GNOME community. This brings me to my final point, which is that we’re in the run up to the annual board elections which take place in the run up to GUADEC. Because of the expansion of the board, and four directors coming to the end of their terms, we’ll be electing 6 seats this election. It’s really important to Holly and the board that we use this opportunity to bring some new voices to the table, leading by example in growing and better representing our community.
Allan wrote in the past about what the board does and what’s expected from directors. As you can see we’re working hard on reducing what we ask from each individual board member by increasing the number of directors, and bringing additional members in to committees and non-voting roles. If you’re interested in seeing more diverse backgrounds and perspectives represented on the board, I would strongly encourage you consider standing for election and reach out to a board member to discuss their experience.
Thanks for reading! Until next time.
Best Wishes, Rob President, GNOME Foundation
(also posted to GNOME Discourse, please head there if you have any questions or comments)
I wrote a blog post The Shape of Computers [1] exploring ideas of how computers might evolve and how we can use them. One of the devices I mentioned was the Humane AI Pin, which has just been the recipient of one of the biggest roast reviews I’ve ever seen [2], good work Marques Brownlee! As an aside I was once given a product to review which didn’t work nearly as well as I think it should have worked so I sent an email to the developers saying “sorry this product failed to work well so I can’t say anything good about it” and didn’t publish a review.
One of the first things that caught my attention in the review is the note that the AI Pin doesn’t connect to your phone. I think that everything should connect to everything else as a usability feature. For security we don’t want so much connecting and it’s quite reasonable to turn off various connections at appropriate times for security, the Librem5 is an example of how this can be done with hardware switches to disable Wifi etc. But to just not have connectivity is bad.
The next noteworthy thing is the external battery which also acts as a magnetic attachment from inside your shirt. So I guess it’s using wireless charging through your shirt. A magnetically attached external battery would be a great feature for a phone, you could quickly swap a discharged battery for a fresh one and keep using it. When I tried to make the PinePhonePro my daily driver [3] I gave up and charging was one of the main reasons. One thing I learned from my experiment with the PinePhonePro is that the ratio of charge time to discharge time is sometimes more important than battery life and being able to quickly swap batteries without rebooting is a way of solving that. The reviewer of the AI Pin complains later in the video about battery life which seems to be partly due to wireless charging from the detachable battery and partly due to being physically small. It seems the “phablet” form factor is the smallest viable personal computer at this time.
The review glosses over what could be the regarded as the 2 worst issues of the device. It does everything via the cloud (where “the cloud” means “a computer owned by someone I probably shouldn’t trust”) and it records everything. Strange that it’s not getting the hate the Google Glass got.
The user interface based on laser projection of menus on the palm of your hand is an interesting concept. I’d rather have a Bluetooth attached tablet or something for operations that can’t be conveniently done with voice. The reviewer harshly criticises the laser projection interface later in the video, maybe technology isn’t yet adequate to implement this properly.
The first criticism of the device in the “review” part of the video is of the time taken to answer questions, especially when Internet connectivity is poor. His question “who designed the Washington Monument” took 8 seconds to start answering it in his demonstration. I asked the Alpaca LLM the same question running on 4 cores of a E5-2696 and it took 10 seconds to start answering and then printed the words at about speaking speed. So if we had a free software based AI device for this purpose it shouldn’t be difficult to get local LLM computation with less delay than the Humane device by simply providing more compute power than 4 cores of a E5-2696v3. How does a 32 core 1.05GHz Mali G72 from 2017 (as used in the Galaxy Note 9) compare to 4 cores of a 2.3GHz Intel CPU from 2015? Passmark says that Intel CPU can do 48GFlop with all 18 cores so 4 cores can presumably do about 10GFlop which seems less than the claimed 20-32GFlop of the Mali G72. It seems that with the right software even older Android phones could give adequate performance for a local LLM. The Alpaca model I’m testing with takes 4.2G of RAM to run which is usable in a Note 9 with 8G of RAM or a Pixel 8 Pro with 12G. A Pixel 8 Pro could have 4.2G of RAM reserved for a LLM and still have as much RAM for other purposes as my main laptop as of a few months ago. I consider the speed of Alpaca on my workstation to be acceptable but not great. If we can get FOSS phones running a LLM at that speed then I think it would be great for a first version – we can always rely on newer and faster hardware becoming available.
Marques notes that the cause of some of the problems is likely due to a desire to make it a separate powerful product in the future and that if they gave it phone connectivity in the start they would have to remove that later on. I think that the real problem is that the profit motive is incompatible with good design. They want to have a product that’s stand-alone and justifies the purchase price plus subscription and that means not making it a “phone accessory”. While I think that the best thing for the user is to allow it to talk to a phone, a PC, a car, and anything else the user wants. He compares it to the Apple Vision Pro which has the same issue of trying to be a stand-alone computer but not being properly capable of it.
One of the benefits that Marques cites for the AI Pin is the ability to capture voice notes. Dictaphones have been around for over 100 years and very few people have bought them, not even in the 80s when they became cheap. While almost everyone can occasionally benefit from being able to make a note of an idea when it’s not convenient to write it down there are few people who need it enough to carry a separate device, not even if that device is tiny. But a phone as a general purpose computing device with microphone can easily be adapted to such things. One possibility would be to program a phone to start a voice note when the volume up and down buttons are pressed at the same time or when some other condition is met. Another possibility is to have a phone have a hotkey function that varies by what you are doing, EG if bushwalking have the hotkey be to take a photo or if on a flight have it be taking a voice note. On the Mobile Apps page on the Debian wiki I created a section for categories of apps that I think we need [4]. In that section I added the following list:
Voice input for dictation
Voice assistant like Google/Apple
Voice output
Full operation for visually impaired people
One thing I really like about the AI Pin is that it has the potential to become a really good computing and personal assistant device for visually impaired people funded by people with full vision who want to legally control a computer while driving etc. I have some concerns about the potential uses of the AI Pin while driving (as Marques stated an aim to do), but if it replaces the use of regular phones while driving it will make things less bad.
Marques concludes his video by warning against buying a product based on the promise of what it can be in future. I bought the Librem5 on exactly that promise, the difference is that I have the source and the ability to help make the promise come true. My aim is to spend thousands of dollars on test hardware and thousands of hours of development time to help make FOSS phones a product that most people can use at low price with little effort.
Another interesting review of the pin is by Mrwhostheboss [5], one of his examples is of asking the pin for advice about a chair but without him knowing the pin selected a different chair in the room. He compares this to using Google’s apps on a phone and seeing which item the app has selected. He also said that he doesn’t want to make an order based on speech he wants to review a page of information about it. I suspect that the design of the pin had too much input from people accustomed to asking a corporate travel office to find them a flight and not enough from people who look through the details of the results of flight booking services trying to save an extra $20. Some people might say “if you need to save $20 on a flight then a $24/month subscription computing service isn’t for you”, I reject that argument. I can afford lots of computing services because I try to get the best deal on every moderately expensive thing I pay for. Another point that Mrwhostheboss makes is regarding secret SMS, you probably wouldn’t want to speak a SMS you are sending to your SO while waiting for a train. He makes it clear that changing between phone and pin while sharing resources (IE not having a separate phone number and separate data store) is a desired feature.
The most insightful point Mrwhostheboss made was when he suggested that if the pin had come out before the smartphone then things might have all gone differently, but now anything that’s developed has to be based around the expectations of phone use. This is something we need to keep in mind when developing FOSS software, there’s lots of different ways that things could be done but we need to meet the expectations of users if we want our software to be used by many people.
I previously wrote a blog post titled Considering Convergence [1] about the possible ways of using a phone as a laptop. While I still believe what I wrote there I’m now considering the possibility of ease of movement of work in progress as a way of addressing some of the same issues.
Currently the expected use is that if you have web pages open on Chrome on Android it’s possible to instruct Chrome on the desktop to open the same page if both instances of Chrome are signed in to the same GMail account. It’s also possible to view the Chrome history with CTRL-H, select “tabs from other devices” and load things that were loaded on other devices some time ago. This is very minimal support for moving work between devices and I think we can do better.
Firstly for web browsing the Chrome functionality is barely adequate. It requires having a heavyweight login process on all browsers that includes sharing stored passwords etc which isn’t desirable. There are many cases where moving work is desired without sharing such things, one example is using a personal device to research something for work. Also the Chrome method of sending web pages is slow and unreliable and the viewing history method gets all closed tabs when the common case is “get the currently open tabs from one browser window” without wanting the dozens of web pages that turned out not to be interesting and were closed. This could be done with browser plugins to allow functionality similar to KDE Connect for sending tabs and also the option of emailing a list of URLs or a JSON file that could be processed by a browser plugin on the receiving end. I can send email between my home and work addresses faster than the Chrome share to another device function can send a URL.
For documents we need a way of transferring files. One possibility is to go the Chromebook route and have it all stored on the web. This means that you rely on a web based document editing system and the FOSS versions are difficult to manage. Using Google Docs or Sharepoint for everything is not something I consider an acceptable option. Also for laptop use being able to run without Internet access is a good thing.
There are a range of distributed filesystems that have been used for various purposes. I don’t think any of them cater to the use case of having a phone/laptop and a desktop PC (or maybe multiple PCs) using the same files.
For a technical user it would be an option to have a script that connects to a peer system (IE another computer with the same accounts and access control decisions) and rsync a directory of working files and the shell history, and then opens a shell with the HISTFILE variable, current directory, and optionally some user environment variables set to match. But this wouldn’t be the most convenient thing even for technical users.
For programs that are integrated into the desktop environment it’s possible for them to be restarted on login if they were active when the user logged out. The session tracking for that has about 1/4 the functionality needed for requesting a list of open files from the application, closing the application, transferring the files, and opening it somewhere else. I think that this would be a good feature to add to the XDG setup.
The model of having programs and data attached to one computer or one network server that terminals of some sort connect to worked well when computers were big and expensive. But computers continue to get smaller and cheaper so we need to think of a document based use of computers to allow things to be easily transferred as convenient. With convenience being important so the hacks of rsync scripts that can work for technical users won’t work for most people.
A new minor release 0.4.22 of RQuantLib
arrived at CRAN earlier today,
and has been uploaded to Debian.
QuantLib is a rather
comprehensice free/open-source library for quantitative
finance. RQuantLib
connects (some parts of) it to the R environment and language, and has
been part of CRAN for more than
twenty years (!!) as it was one of the first packages I uploaded
there.
This release of RQuantLib
updates to QuantLib version 1.34
which was just released yesterday, and deprecates use of an access point
/ type for price/yield conversion for bonds. We also made two minor
earlier changes.
Changes in RQuantLib version 0.4.22 (2024-04-25)
Small code cleanup removing duplicate R code
Small improvements to C++ compilation flags
Robustify internal version comparison to accommodate RC
releases
Nine days ago, I started migrating orphaned Debian packages with no
version control system listed in debian/control of the source to git.
At the time there were 438 such packages. Now there are 391,
according to the UDD. In reality it is slightly less, as there is a
delay between uploads and UDD updates. In the nine days since, I have
thus been able to work my way through ten percent of the packages. I
am starting to run out of steam, and hope someone else will also help
brushing some dust of these packages. Here is a recipe how to do it.
I start by picking a random package by querying the UDD for a list of
10 random packages from the set of remaining packages:
PGPASSWORD="udd-mirror" psql --port=5432 --host=udd-mirror.debian.net \
--username=udd-mirror udd -c "select source from sources \
where release = 'sid' and (vcs_url ilike '%anonscm.debian.org%' \
OR vcs_browser ilike '%anonscm.debian.org%' or vcs_url IS NULL \
OR vcs_browser IS NULL) AND maintainer ilike '%packages@qa.debian.org%' \
order by random() limit 10;"
Next, I visit http://salsa.debian.org/debian and search for the
package name, to ensure no git repository already exist. If it does,
I clone it and try to get it to an uploadable state, and add the Vcs-*
entries in d/control to make the repository more widely known. These
packages are a minority, so I will not cover that use case here.
For packages without an existing git repository, I run the
following script debian-snap-to-salsa to prepare a git
repository with the existing packaging.
#!/bin/sh
#
# See also https://bugs.debian.org/804722#31
set -e
# Move to this Standards-Version.
SV_LATEST=4.7.0
PKG="$1"
if [ -z "$PKG" ]; then
echo "usage: $0 "
exit 1
fi
if [ -e "${PKG}-salsa" ]; then
echo "error: ${PKG}-salsa already exist, aborting."
exit 1
fi
if [ -z "ALLOWFAILURE" ] ; then
ALLOWFAILURE=false
fi
# Fetch every snapshotted source package. Manually loop until all
# transfers succeed, as 'gbp import-dscs --debsnap' do not fail on
# download failures.
until debsnap --force -v $PKG || $ALLOWFAILURE ; do sleep 1; done
mkdir ${PKG}-salsa; cd ${PKG}-salsa
git init
# Specify branches to override any debian/gbp.conf file present in the
# source package.
gbp import-dscs --debian-branch=master --upstream-branch=upstream \
--pristine-tar ../source-$PKG/*.dsc
# Add Vcs pointing to Salsa Debian project (must be manually created
# and pushed to).
if ! grep -q ^Vcs- debian/control ; then
awk "BEGIN { s=1 } /^\$/ { if (s==1) { print \"Vcs-Browser: https://salsa.debian.org/debian/$PKG\"; print \"Vcs-Git: https://salsa.debian.org/debian/$PKG.git\" }; s=0 } { print }" < debian/control > debian/control.new && mv debian/control.new debian/control
git commit -m "Updated vcs in d/control to Salsa." debian/control
fi
# Tell gbp to enforce the use of pristine-tar.
inifile +inifile debian/gbp.conf +create +section DEFAULT +key pristine-tar +value True
git add debian/gbp.conf
git commit -m "Added d/gbp.conf to enforce the use of pristine-tar." debian/gbp.conf
# Update to latest Standards-Version.
SV="$(grep ^Standards-Version: debian/control|awk '{print $2}')"
if [ $SV_LATEST != $SV ]; then
sed -i "s/\(Standards-Version: \)\(.*\)/\1$SV_LATEST/" debian/control
git commit -m "Updated Standards-Version from $SV to $SV_LATEST." debian/control
fi
if grep -q pkg-config debian/control; then
sed -i s/pkg-config/pkgconf/ debian/control
git commit -m "Replaced obsolete pkg-config build dependency with pkgconf." debian/control
fi
if grep -q libncurses5-dev debian/control; then
sed -i s/libncurses5-dev/libncurses-dev/ debian/control
git commit -m "Replaced obsolete libncurses5-dev build dependency with libncurses-dev." debian/control
fi
Some times the debsnap script fail to download some of the versions.
In those cases I investigate, and if I decide the failing versions
will not be missed, I call it using ALLOWFAILURE=true to ignore the
problem and create the git repository anyway.
With the git repository in place, I do a test build (gbp
buildpackage) to ensure the build is actually working. If it does not
I pick a different package, or if the build failure is trivial to fix,
I fix it before continuing. At this stage I revisit
http://salsa.debian.org/debian and create the project under this group
for the package. I then follow the instructions to publish the local
git repository. Here is from a recent example:
With a working build, I have a look at the build rules if I want to
remove some more dust. I normally try to move to debhelper compat
level 13, which involves removing debian/compat and modifying
debian/control to build depend on debhelper-compat (=13). I also test
with 'Rules-Requires-Root: no' in debian/control and verify in
debian/rules that hardening is enabled, and include all of these if
the package still build. If it fail to build with level 13, I try
with 12, 11, 10 and so on until I find a level where it build, as I do
not want to spend a lot of time fixing build issues.
Some times, when I feel inspired, I make sure debian/copyright is
converted to the machine readable format, often by starting with
'debhelper -cc' and then cleaning up the autogenerated content until
it matches realities. If I feel like it, I might also clean up
non-dh-based debian/rules files to use the short style dh build
rules.
Once I have removed all the dust I care to process for the package,
I run 'gbp dch' to generate a debian/changelog entry based on the
commits done so far, run 'dch -r' to switch from 'UNRELEASED' to
'unstable' and get an editor to make sure the 'QA upload' marker is in
place and that all long commit descriptions are wrapped into sensible
lengths, run 'debcommit --release -a' to commit and tag the new
debian/changelog entry, run 'debuild -S' to build a source only
package, and 'dput ../perl-byacc_2.0-10_source.changes' to do the
upload. During the entire process, and many times per step, I run
'debuild' to verify the changes done still work. I also some times
verify the set of built files using 'find debian' to see if I can spot
any problems (like no file in usr/bin any more or empty package). I
also try to fix all lintian issues reported at the end of each
'debuild' run.
If I find Debian specific patches, I try to ensure their metadata
is fairly up to date and some times I even try to reach out to
upstream, to make the upstream project aware of the patches. Most of
my emails bounce, so the success rate is low. For projects with no
Homepage entry in debian/control I try to track down one, and for
packages with no debian/watch file I try to create one. But at least
for some of the packages I have been unable to find a functioning
upstream, and must skip both of these.
If I could handle ten percent in nine days, twenty people could
complete the rest in less then five days. I use approximately twenty
minutes per package, when I have twenty minutes spare time to spend.
Perhaps you got twenty minutes to spare too?
As usual, if you use Bitcoin and want to show your support of my
activities, please send Bitcoin donations to my address
15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.
With local recursive DNS and a 5G modem in place the next thing was to work on some sort of automatic failover when the primary FTTP connection failed. My wife works from home too and I sometimes travel so I wanted to make sure things didn’t require me to be around to kick them into switch the link in use.
First, let’s talk about what I didn’t do. One choice to try and ensure as seamless a failover as possible would be to get a VM somewhere out there. I’d then run Wireguard tunnels over both the FTTP + 5G links to the VM, and run some sort of routing protocol (RIP, OSPF?) over the links. Set preferences such that the FTTP is preferred, NAT v4 to the VM IP, and choose somewhere that gave me a v6 range I could just use directly.
This has the advantage that I’m actively checking link quality to the outside work, rather than just to the next hop. It also means, if the failover detection is fast enough, that existing sessions stay up rather than needing re-established.
The downsides are increased complexity, adding another point of potential failure (the VM + provider), the impact on connection quality (even with a decent endpoint it’s an extra hop and latency), and finally the increased cost involved.
I can cope with having to reconnect my SSH sessions in the event of a failure, and I’d rather be sure I can make full use of the FTTP connection, so I didn’t go this route. I chose to rely on local link failure detection to provide the signal for failover, and a set of policy routing on top of that to make things a bit more seamless.
Local link failure turns out to be fairly easy. My FTTP is a PPPoE configuration, so in /etc/ppp/peers/aquiss I have:
Which gives me a failover of ~ 5s if the link goes down.
I’m operating the 5G modem in “bridge” rather than “router” mode, which means I get the actual IP from the 5G network via DHCP. The DHCP lease the modem hands out is under a minute, and in the event of a network failure it only hands out a 192.168.254.x IP to talk to its web interface. As the 5G modem is the last resort path I choose not to do anything special with this, but the information is at least there if I need it.
To allow both interfaces to be up and the FTTP to be preferred I’m simply using route metrics. For the PPP configuration that’s:
There’s a wrinkle in that pppd will not replace an existing default route, so I’ve created /etc/ppp/ip-up.d/default-route to ensure it’s added:
#!/bin/bash
[ "$PPP_IFACE" = "pppoe-wan" ] || exit 0
# Ensure we add a default route; pppd will not do so if we have
# a lower pref route out the 5G modem
ip route add default dev pppoe-wan metric 100 || true
Additionally, in /etc/dhcp/dhclient.conf I’ve disabled asking for any server details (DNS, NTP, etc) - I have internal setups for the servers I want, and don’t want to be trying to select things over the 5G link by default.
However, what I do want is to be able to access the 5G modem web interface and explicitly route some traffic out that link (e.g. so I can add it to my smokeping tests). For that I need some source based routing.
First step, add a 5g table to /etc/iproute2/rt_tables:
16 5g
Then I ended up with the following in /etc/dhcp/dhclient-exit-hooks.d/modem-interface-route, which is more complex than I’d like but seems to do what I want:
#!/bin/sh
case "$reason" in
BOUND|RENEW|REBIND|REBOOT)
# Check if we've actually changed IP address
if [ -z "$old_ip_address" ] ||
[ "$old_ip_address" != "$new_ip_address" ] ||
[ "$reason" = "BOUND" ] || [ "$reason" = "REBOOT" ]; then
if [ ! -z "$old_ip_address" ]; then
ip rule del from $old_ip_address lookup 5g
fi
ip rule add from $new_ip_address lookup 5g
ip route add default dev sfp.31 table 5g || true
ip route add 192.168.254.1 dev sfp.31 2>/dev/null || true
fi
;;
EXPIRE)
if [ ! -z "$old_ip_address" ]; then
ip rule del from $old_ip_address lookup 5g
fi
;;
*)
;;
esac
What does all that aim to do? We want to ensure traffic directed to the 5G WAN address goes out the 5G modem, so I can SSH into it even when the main link is up. So we add a rule directing traffic from that IP to hit the 5g routing table, and a default route in that table which uses the 5G link. There’s no configuration for the FTTP connection in that table, so if the 5G link is down the traffic gets dropped, which is what we want. We also configure 192.168.254.1 to go out the link to the modem, as that’s where the web interface lives.
I also have a curl callout (curl --interface sfp.31 … to ensure it goes out the 5G link) after the routes are configured to set dynamic DNS with Mythic Beasts, which helps with knowing where to connect back to. I seem to see IP address changes on the 5G link every couple of days at least.
Additionally, I have an entry in the interfaces configuration carving out the top set of the netblock my smokeping server is in:
up ip rule add from 192.0.2.224/27 lookup 5g
My smokeping /etc/smokeping/config.d/Probes file then looks like:
which allows me to use probe = FPing5G for targets to test them over the 5G link.
That mostly covers the functionality I want for a backup link. There’s one piece that isn’t quite solved, however, IPv6, which can wait for another post.
With the work that has been done in the debian-installer/netcfg merge-proposal !9 it is possible to install a standard Debian system, using the normal Debian-Installer (d-i) mini.iso images, that will come pre-installed with Netplan and all network configuration structured in /etc/netplan/.
In this write-up I’d like to run you through a list of commands for experiencing the Netplan enabled installation process first-hand. For now, we’ll be using a custom ISO image, while waiting for the above-mentioned merge-proposal to be landed. Furthermore, as the Debian archive is going through major transitions builds of the “unstable” branch of d-i don’t currently work. So I implemented a small backport, producing updated netcfg and netcfg-static for Bookworm, which can be used as localudebs/ during the d-i build.
Let’s start with preparing a working directory and installing the software dependencies for our virtualized Debian system:
Next we’ll prepare a VM, by copying the EFI firmware files, preparing some persistent EFIVARs file, to boot from FS0:\EFI\debian\grubx64.efi, and create a virtual disk for our machine:
Finally, let’s launch the installer using a custom preseed.cfg file, that will automatically install Netplan for us in the target system. A minimal preseed file could look like this:
For this demo, we’re installing the full netplan.io package (incl. Python CLI), as the netplan-generator package was not yet split out as an independent binary in the Bookworm cycle. You can choose the preseed file from a set of different variants to test the different configurations:
We’re using the custom linux kernel and initrd.gz here to be able to pass the PRESEED_URL as a parameter to the kernel’s cmdline directly. Launching this VM should bring up the normal debian-installer in its netboot/gtk form:
Now you can click through the normal Debian-Installer process, using mostly default settings. Optionally, you could play around with the networking settings, to see how those get translated to /etc/netplan/ in the target system.
After you confirmed your partitioning changes, the base system gets installed. I suggest not to select any additional components, like desktop environments, to speed up the process.
During the final step of the installation (finish-install.d/55netcfg-copy-config) d-i will detect that Netplan was installed in the target system (due to the preseed file provided) and opt to write its network configuration to /etc/netplan/ instead of /etc/network/interfaces or /etc/NetworkManager/system-connections/.
Done! After the installation finished you can reboot into your virgin Debian Bookworm system.
To do that, quit the current Qemu process, by pressing Ctrl+C and make sure to copy over the EFIVARS.fd file that was written by grub during the installation, so Qemu can find the new system. Then reboot into the new system, not using the mini.iso image any more:
Finally, you can play around with your Netplan enabled Debian system! As you will find, /etc/network/interfaces exists but is empty, it could still be used (optionally/additionally). Netplan was configured in /etc/netplan/ according to the settings given during the d-i installation process.
In our case we also installed the Netplan CLI, so we can play around with some of its features, like netplan status:
Thank you for following along the Netplan enabled Debian installation process and happy hacking! If you want to learn more join the discussion at Salsa:installer-team/netcfg and find us at GitHub:netplan.
Nation is a stand-alone young adult fantasy novel. It was
published in the gap between Discworld novels Making Money and Unseen
Academicals.
Nation starts with a plague. The Russian influenza has ravaged
Britain, including the royal family. The next in line to the throne is
off on a remote island and must be retrieved and crowned as soon as
possible, or an obscure provision in Magna Carta will cause no end of
trouble. The Cutty Wren is sent on this mission, carrying the
Gentlemen of Last Resort.
Then comes the tsunami.
In the midst of fire raining from the sky and a wave like no one has ever
seen, Captain Roberts tied himself to the wheel of the Sweet Judy
and steered it as best he could, straight into an island. The sole
survivor of the shipwreck: one Ermintrude Fanshaw, daughter of the
governor of some British island possessions. Oh, and a parrot.
Mau was on the Boys' Island when the tsunami came, going through his rite
of passage into manhood. He was to return to the Nation the next morning
and receive his tattoos and his adult soul. He survived in a canoe. No
one else in the Nation did.
Terry Pratchett considered Nation to be his best book. It is not
his best book, at least in my opinion; it's firmly below the top tier of
Discworld novels, let alone Night Watch.
It is, however, an interesting and enjoyable book that tackles gods and
religion with a sledgehammer rather than a knife.
It's also very, very dark and utterly depressing at the start, despite a
few glimmers of Pratchett's humor. Mau is the main protagonist at first,
and the book opens with everyone he cares about dying. This is the place
where I thought Pratchett diverged the most from his Discworld style: in
Discworld, I think most of that would have been off-screen, but here we
follow Mau through the realization, the devastation, the disassociation,
the burials at sea, the thoughts of suicide, and the complete upheaval of
everything he thought he was or was about to become. I found the start of
this book difficult to get through. The immediate transition into
potentially tragic misunderstandings between Mau and Daphne (as Ermintrude
names herself once there is no one to tell her not to) didn't help.
As I got farther into the book, though, I warmed to it. The best parts
early on are Daphne's baffled but scientific attempts to understand Mau's
culture and her place in it. More survivors arrive, and they start to
assemble a community, anchored in large part by Mau's stubborn
determination to do what's right even though he's lost all of his
moorings. That community eventually re-establishes contact with the rest
of the world and the opening plot about the British monarchy, but not
before Daphne has been changed profoundly by being part of it.
I think Pratchett worked hard at keeping Mau's culture at the center of
the story. It's notable that the community that reforms over the course
of the book essentially follows the patterns of Mau's lost Nation and
incorporates Daphne into it, rather than (as is so often the case) the
other way around. The plot itself is fiercely anti-colonial in a way that
mostly worked. Still, though, it's a quasi-Pacific-island culture written
by a white British man, and I had some qualms.
Pratchett quite rightfully makes it clear in the afterward that this is an
alternate world and Mau's culture is not a real Pacific island culture.
However, that also means that its starkly gender-essentialist nature was a
free choice, rather than one based on some specific culture, and I found
that choice somewhat off-putting. The religious rituals are all gendered,
the dwelling places are gendered, and one's entire life course in Mau's
world seems based on binary classification as a man or a woman. Based on
Pratchett's other books, I assume this was more an unfortunate default
than a deliberate choice, but it's still a choice he could have avoided.
The end of this book wrestles directly with the relative worth of Mau's
culture versus that of the British. I liked most of this, but the twists
that Pratchett adds to avoid the colonialist results we saw in our world
stumble partly into the trap of making Mau's culture valuable by British
standards. (I'm being a bit vague here to avoid spoilers.) I think it is
very hard to base this book on a different set of priorities and still
bring the largely UK, US, and western European audience along, so I don't
blame Pratchett for failing to do it, but I'm a bit sad that the world
still revolved around a British axis.
This felt quite similar to Discworld to me in its overall sensibilities,
but with the roles of moral philosophy and humor reversed. Discworld
novels usually start with some larger-than-life characters and an absurd
plot, and then the moral philosophy sneaks up behind you when you're not
looking and hits you over the head. Nation starts with the moral
philosophy: Mau wrestles with his gods and the problem of evil in a way
that reminded me of Job, except with a far different pantheon and rather
less tolerance for divine excuses on the part of the protagonist. It's
the humor, instead, that sneaks up on you and makes you laugh when the
plot is a bit too much. But the mix arrives at much the same place: the
absurd hand-in-hand with the profound, and all seen from an angle that
makes it a bit easier to understand.
I'm not sure I would recommend Nation as a good place to start with
Pratchett. I felt like I benefited from having read a lot of Discworld
to build up my willingness to trust where Pratchett was going. But it has
the quality of writing of late Discworld without the (arguable) need to
read 25 books to understand all of the backstory. Regardless,
recommended, and you'll never hear Twinkle Twinkle Little Star in
quite the same way again.
The XKCD comic Code Quality [1] inspired me to test out emoji in source. I really should have done this years ago when that XKCD was first published.
The following code compiles in gcc and runs in the way that anyone who wants to write such code would want it to run. The hover text in the XKCD comic is correct. You could have a style guide for such programming, store error messages in the doctor and nurse emoji for example.
#include <stdio.h>
int main()
{
int 😇 = 1, 😈 = 2;
printf("😇=%d, 😈=%d\n", 😇, 😈);
return 0;
}
To get this to display correctly in Debian you need to install the fonts-noto-color-emoji package (used by the KDE emoji picker that runs when you press Windows-. among other things) and restart programs that use emoji. The Konsole terminal emulator will probably need it’s profile settings changed to work with this if you ran Konsole before installing fonts-noto-color-emoji. The Kitty terminal emulator works if you restart it after installing fonts-noto-color-emoji.
This web page gives a list of HTML codes for emoji [2]. If I start writing real code with emoji variable names then I’ll have to update my source to HTML conversion script (which handles <>" and repeated spaces) to convert emoji.
I spent a couple of hours on this and I think it’s worth it. I have filed several Debian bug reports about improvements needed to issues related to emoji.
To resolve that you could upgrade to SE Linux, but the other option is to create a file named /etc/apparmor.d/bwrap with the following contents:
abi <abi/4.0>,
include <tunables/global>
profile bwrap /usr/bin/bwrap flags=(unconfined) {
userns,
# Site-specific additions and overrides. See local/README for details.
include if exists <local/bwrap>
}
Version 3.3 brings in new features, including reverse Laplace transforms and fits, pH fits, commands for picking points from a dataset, averaging points with the same X value, or perform singular value decomposition.
In addition to these new features, many previous commands were improved, like the addition of a bandcut filter in FFT filtering, better handling of the loading of files produced by QSoas itself, and a button to interrupt the processing of scripts.
There are a lot of other new features, improvements and so on, look for the full list there.
About QSoas
QSoas is a powerful open source data analysis program that focuses on flexibility and powerful fitting capacities. It is released under the GNU General Public License. It is described in Fourmond, Anal. Chem., 2016, 88 (10), pp 5050–5052. Current version is 3.3. You can download for free its source code or precompiled versions for MacOS and Windows there. Alternatively, you can clone from the GitHub repository.
The Stars, Like Dust is usually listed as the first book in
Asimov's lesser-known Galactic Empire Trilogy since it takes place before
Pebble in the Sky. Pebble in the
Sky was published first, though, so I count it as the second book. It is
very early science fiction with a few mystery overtones.
Buying books produces about 5% of the pleasure of reading them while
taking much less than 5% of the time. There was a time in my life when I
thoroughly enjoyed methodically working through a used book store, list in
hand, tracking down cheap copies to fill in holes in series. This means
that I own a lot of books that I thought at some point that I would want
to read but never got around to, often because, at the time, I was feeling
completionist about some series or piece of world-building. From time to
time, I get the urge to try to read some of them.
Sometimes this is a poor use of my time.
The Galactic Empire series is from Asimov's first science fiction period,
after the Foundation series but contemporaneous with their
collection into novels. They're set long, long before Foundation,
but after humans have inhabited numerous star systems and Earth has become
something of a backwater. That process is just starting in The
Stars, Like Dust: Earth is still somewhere where an upper-class son might
be sent for an education, but it has been devastated by nuclear wars and
is well on its way to becoming an inward-looking relic on the edge of
galactic society.
Biron Farrill is the son of the Lord Rancher of Widemos, a wealthy noble
whose world is one of those conquered by the Tyranni. In many other SF
novels, the Tyranni would be an alien race; here, it's a hierarchical and
authoritarian human civilization. The book opens with Biron discovering a
radiation bomb planted in his dorm room. Shortly after, he learns that
his father had been arrested. One of his fellow students claims to be on
Biron's side against the Tyranni and gives him false papers to travel to
Rhodia, a wealthy world run by a Tyranni sycophant.
Like most books of this era, The Stars, Like Dust is a short novel
full of plot twists. Unlike some of its contemporaries, it's not devoid
of characterization, but I might have liked it better if it were. Biron
behaves like an obnoxious teenager when he's not being an arrogant ass.
There is a female character who does a few plot-relevant things and at no
point is sexually assaulted, so I'll give Asimov that much, but the gender
stereotypes are ironclad and there is an entire subplot focused on what I
can only describe as seduction via petty jealousy.
The writing... well, let me quote a typical passage:
There was no way of telling when the threshold would be reached.
Perhaps not for hours, and perhaps the next moment. Biron remained
standing helplessly, flashlight held loosely in his damp hands. Half
an hour before, the visiphone had awakened him, and he had been at
peace then. Now he knew he was going to die.
Biron didn't want to die, but he was penned in hopelessly, and there
was no place to hide.
Needless to say, Biron doesn't die. Even if your tolerance for pulp
melodrama is high, 192 small-print pages of this sort of thing is
wearying.
Like a lot of Asimov plots, The Stars, Like Dust has some of the
shape of a mystery novel. Biron, with the aid of some newfound companions
on Rhodia, learns of a secret rebellion against the Tyranni and attempts
to track down its base to join them. There are false leads, disguised
identities, clues that are difficult to interpret, and similar classic
mystery trappings, all covered with a patina of early 1950s imaginary
science. To me, it felt constructed and artificial in ways that made the
strings Asimov was pulling obvious. I don't know if someone who likes
mystery construction would feel differently about it.
The worst part of the plot thankfully doesn't come up much. We learn
early in the story that Biron was on Earth to search for a long-lost
document believed to be vital to defeating the Tyranni. The nature of
that document is revealed on the final page, so I won't spoil it, but if
you try to think of the stupidest possible document someone could have
built this plot around, I suspect you will only need one guess. (In
Asimov's defense, he blamed Galaxy editor H.L. Gold for persuading
him to include this plot, and disavowed it a few years later.)
The Stars, Like Dust is one of the worst books I have ever read.
The characters are overwrought, the politics are slapdash and build on
broad stereotypes, the romantic subplot is dire and plays out mainly via
Biron egregiously manipulating his petulant love interest, and the writing
is annoying. Sometimes pulp fiction makes up for those common flaws
through larger-than-life feats of daring, sweeping visions of future
societies, and ever-escalating stakes. There is little to none of that
here. Asimov instead provides tedious political maneuvering among a class
of elitist bankers and land owners who consider themselves natural
leaders. The only places where the power structures of this future
government make sense are where Asimov blatantly steals them from either
the Roman Empire or the Doge of Venice.
The one thing this book has going for it — the thing, apart from
bloody-minded completionism, that kept me reading — is that the technology
is hilariously weird in that way that only 1940s and 1950s science fiction
can be. The characters have access to communication via some sort of
interstellar telepathy (messages coded to a specific person's "brain
waves") and can travel between stars through hyperspace jumps, but each
jump is manually calculated by referring to the pilot's (paper!) volumes of
the Standard Galactic Ephemeris. Communication between ships (via
"etheric radio") requires manually aiming a radio beam at the area in
space where one thinks the other ship is. It's an unintentionally
entertaining combination of technology that now looks absurdly primitive
and science that is so advanced and hand-waved that it's obviously made
up.
I also have to give Asimov some points for using spherical coordinates.
It's a small thing, but the coordinate systems in most SF novels and TV
shows are obviously not fit for purpose.
I spent about a month and a half of this year barely reading, and while
some of that is because I finally tackled a few projects I'd been putting
off for years, a lot of it was because of this book. It was only 192
pages, and I'm still curious about the glue between Asimov's
Foundation and Robot series, both of which I devoured as a
teenager. But every time I picked it up to finally finish it and start
another book, I made it about ten pages and then couldn't take any more.
Learn from my error: don't try this at home, or at least give up if the
same thing starts happening to you.
I am upstream and Debian package maintainer of
python-debianbts, which is a Python library that allows for
querying Debian’s Bug Tracking System (BTS). python-debianbts is used by
reportbug, the standard tool to report bugs in Debian, and therefore the glue
between the reportbug and the BTS.
debbugs, the software that powers Debian’s BTS, provides a SOAP
interface for querying the BTS. Unfortunately, SOAP is not a very popular
protocol anymore, and I’m facing the second migration to another underlying
SOAP library as they continue to become unmaintained over time. Zeep, the
library I’m currently considering, requires a WSDL file in order to work
with a SOAP service, however, debbugs does not provide one. Since I’m not
familiar with WSDL, I need help from someone who can create a WSDL file for
debbugs, so I can migrate python-debianbts away from pysimplesoap to zeep.
How did we get here?
Back in the olden days, reportbug was querying the BTS by parsing its HTML
output. While this worked, it tightly coupled the user-facing
presentation of the BTS with critical functionality of the bug reporting tool.
The setup was fragile, prone to breakage, and did not allow changing anything
in the BTS frontend for fear of breaking reportbug itself.
In 2007, I started to work on reportbug-ng, a user-friendly alternative
to reportbug, targeted at users not comfortable using the command line. Early
on, I decided to use the BTS’ SOAP interface instead of parsing HTML like
reportbug did. 2008, I extracted the code that dealt with the BTS into a
separate Python library, and after some collaboration with the reportbug
maintainers, reportbug adopted python-debianbts in 2011 and has used it ever
since.
2015, I was working on porting python-debianbts to Python 3.
During that process, it turned out that its major dependency, SoapPy was pretty
much unmaintained for years and blocking the Python3 transition. Thanks to the
help of Gaetano Guerriero, who ported python-debianbts to
pysimplesoap, the migration was unblocked and could proceed.
In 2024, almost ten years later, pysimplesoap seems to be unmaintained as well,
and I have to look again for alternatives. The most promising one right now
seems to be zeep. Unfortunately, zeep requires a WSDL file for working with
a SOAP service, which debbugs does not provide.
How can you help?
reportbug (and thus python-debianbts) is used by thousands of users and I have
a certain responsibility to keep things working properly. Since I simply don’t
know enough about WSDL to create such a file for debbugs myself, I’m looking
for someone who can help me with this task.
If you’re familiar with SOAP, WSDL and optionally debbugs, please get in
touch with me. I don’t speak Pearl, so I’m not
really able to read debbugs code, but I do know some things about the SOAP
requests and replies due to my work on python-debianbts, so I’m sure we can
work something out.
There is a WSDL file for a debbugs version used by GNU, but I
don’t think it’s official and it currently does not work with zeep. It may be a
good starting point, though.
The future of debbugs’ API
While we can probably continue to support debbugs’ SOAP interface for a while,
I don’t think it’s very sustainable in the long run. A simpler, well documented
REST API that returns JSON seems more appropriate nowadays. The queries and
replies that debbugs currently supports are simple enough to design a REST API
with JSON around it. The benefit would be less complex libraries on the client
side and probably easier maintainability on the server side as well. debbugs’
maintainer seemed to be in agreement with this idea back in
2018. I created an attempt to define a new API
(HTML render), but somehow we got stuck and no progress has been
made since then. I’m still happy to help shaping such an API for debbugs, but I
can’t really implement anything in debbugs itself, as it is written in Perl,
which I’m not familiar with.
Time really flies when you are really busy you have fun! Our Montréal
Debian User Group met on Sunday March 31st and I only just found the time to
write our report :)
This time around, 9 of us we met at EfficiOS's offices1 to
chat, hang out and work on Debian and other stuff!
Here is what we did:
pollo:
did some clerical work for the DebConf videoteam
tried to book a plane ticket for DC24
triaged #1067620 (dependency problem with whipper)
Here are pictures of the event. Well, one picture (thanks Tassia!) of the event
itself and another one of the crisp Italian lager I drank at the bar after the
event :)
Maintainers, amongst other things, of the great LTTng. ↩
The diffoscope maintainers are pleased to announce the release of diffoscope
version 265. This version includes the following changes:
[ Chris Lamb ]
* Ensure that tests with ">=" version constraints actually print the
corresponding tool name. (Closes: reproducible-builds/diffoscope#370)
* Prevent odt2txt tests from always being skipped due to an impossibly new
version requirement. (Closes: reproducible-builds/diffoscope#369)
* Avoid nested parens-in-parens when printing "skipping…" messages
in the testsuite.
Having setup recursive DNS it was time to actually sort out a backup internet connection. I live in a Virgin Media area, but I still haven’t forgiven them for my terrible Virgin experiences when moving here. Plus it involves a bigger contractual commitment. There are no altnets locally (though I’m watching youfibre who have already rolled out in a few Belfast exchanges), so I decided to go for a 5G modem. That gives some flexibility, and is a bit easier to get up and running.
I started by purchasing a ZTE MC7010. This had the advantage of being reasonably cheap off eBay, not having any wifi functionality I would just have to disable (it’s going to plug it into the same router the FTTP connection terminates on), being outdoor mountable should I decide to go that way, and, finally, being powered via PoE.
For now this device sits on the window sill in my study, which is at the top of the house. I printed a table stand for it which mostly does the job (though not as well with a normal, rather than flat, network cable). The router lives downstairs, so I’ve extended a dedicated VLAN through the study switch, down to the core switch and out to the router. The PoE study switch can only do GigE, not 2.5Gb/s, but at present that’s far from the limiting factor on the speed of the connection.
The device is 3 branded, and, as it happens, I’ve ended up with a 3 SIM in it. Up until recently my personal phone was with them, but they’ve kicked me off Go Roam, so I’ve moved. Going with 3 for the backup connection provides some slight extra measure of resiliency; we now have devices on all 4 major UK networks in the house. The SIM is a preloaded data only SIM good for a year; I don’t expect to use all of the data allowance, but I didn’t want to have to worry about unexpected excess charges.
Performance turns out to be disappointing; I end up locking the device to 4G as the 5G signal is marginal - leaving it enabled results in constantly switching between 4G + 5G and a significant extra latency. The smokeping graph below shows a brief period where I removed the 4G lock and allowed 5G:
(There’s a handy zte.js script to allow doing this from the device web interface.)
I get about 10Mb/s sustained downloads out of it. EE/Vodafone did not lead to significantly better results, so for now I’m accepting it is what it is. I tried relocating the device to another part of the house (a little tricky while still providing switch-based PoE, but I have an injector), without much improvement. Equally pinning the 4G to certain bands provided a short term improvement (I got up to 40-50Mb/s sustained), but not reliably so.
This is disappointing, but if it turns out to be a problem I can look at mounting it externally. I also assume as 5G is gradually rolled out further things will naturally improve, but that might be wishful thinking on my part.
Rather than wait until my main link had a problem I decided to try a day working over the 5G connection. I spend a lot of my time either in browser based apps or accessing remote systems via SSH, so I’m reasonably sensitive to a jittery or otherwise flaky connection. I picked a day that I did not have any meetings planned, but as it happened I ended up with an adhoc video call arranged. I’m pleased to say that it all worked just fine; definitely noticeable as slower than the FTTP connection (to be expected), but all workable and even the video call was fine (at least from my end). Looking at the traffic graph shows the expected ~ 10Mb/s peak (actually a little higher, and looking at the FTTP stats for previous days not out of keeping with what we see there), and you can just about see the ~ 3Mb/s symmetric use by the video call at 2pm:
The test run also helped iron out the fact that the content filter was still enabled on the SIM, but that was easily resolved.
Joachim Breitner wrote about a Convenient sandboxed development environment and thus reminded me to blog about MicroVM. I’ve toyed around with it a little but not yet seriously used it as I’m currently not coding.
MicroVM is a nix based project to configure and run minimal VMs. It can mount and thus reuse the hosts nix store inside the VM and thus has a very small disk footprint. I use MicroVM on a debian system using the nix package manager.
The MicroVM author uses the project to host production services. Otherwise I consider it also a nice way to learn about NixOS after having started with the nix package manager and before making the big step to NixOS as my main system.
The guests root filesystem is a tmpdir, so one must explicitly define folders that should be mounted from the host and thus be persistent across VM reboots.
I defined the VM as a nix flake since this is how I started from the MicroVM projects example:
{
description = "Haskell dev MicroVM";
inputs.impermanence.url = "github:nix-community/impermanence";
inputs.microvm.url = "github:astro/microvm.nix";
inputs.microvm.inputs.nixpkgs.follows = "nixpkgs";
outputs = { self, impermanence, microvm, nixpkgs }:
let
persistencePath = "/persistent";
system = "x86_64-linux";
user = "thk";
vmname = "haskell";
nixosConfiguration = nixpkgs.lib.nixosSystem {
inherit system;
modules = [
microvm.nixosModules.microvm
impermanence.nixosModules.impermanence
({pkgs, ... }: {
environment.persistence.${persistencePath} = {
hideMounts = true;
users.${user} = {
directories = [
"git" ".stack"
];
};
};
environment.sessionVariables = {
TERM = "screen-256color";
};
environment.systemPackages = with pkgs; [
ghc
git
(haskell-language-server.override { supportedGhcVersions = [ "94" ]; })
htop
stack
tmux
tree
vcsh
zsh
];
fileSystems.${persistencePath}.neededForBoot = nixpkgs.lib.mkForce true;
microvm = {
forwardPorts = [
{ from = "host"; host.port = 2222; guest.port = 22; }
{ from = "guest"; host.port = 5432; guest.port = 5432; } # postgresql
];
hypervisor = "qemu";
interfaces = [
{ type = "user"; id = "usernet"; mac = "00:00:00:00:00:02"; }
];
mem = 4096;
shares = [ {
# use "virtiofs" for MicroVMs that are started by systemd
proto = "9p";
tag = "ro-store";
# a host's /nix/store will be picked up so that no
# squashfs/erofs will be built for it.
source = "/nix/store";
mountPoint = "/nix/.ro-store";
} {
proto = "virtiofs";
tag = "persistent";
source = "~/.local/share/microvm/vms/${vmname}/persistent";
mountPoint = persistencePath;
socket = "/run/user/1000/microvm-${vmname}-persistent";
}
];
socket = "/run/user/1000/microvm-control.socket";
vcpu = 3;
volumes = [];
writableStoreOverlay = "/nix/.rwstore";
};
networking.hostName = vmname;
nix.enable = true;
nix.nixPath = ["nixpkgs=${builtins.storePath <nixpkgs>}"];
nix.settings = {
extra-experimental-features = ["nix-command" "flakes"];
trusted-users = [user];
};
security.sudo = {
enable = true;
wheelNeedsPassword = false;
};
services.getty.autologinUser = user;
services.openssh = {
enable = true;
};
system.stateVersion = "24.11";
systemd.services.loadnixdb = {
description = "import hosts nix database";
path = [pkgs.nix];
wantedBy = ["multi-user.target"];
requires = ["nix-daemon.service"];
script = "cat ${persistencePath}/nix-store-db-dump|nix-store --load-db";
};
time.timeZone = nixpkgs.lib.mkDefault "Europe/Berlin";
users.users.${user} = {
extraGroups = [ "wheel" "video" ];
group = "user";
isNormalUser = true;
openssh.authorizedKeys.keys = [
"ssh-rsa REDACTED"
];
password = "";
};
users.users.root.password = "";
users.groups.user = {};
})
];
};
in {
packages.${system}.default = nixosConfiguration.config.microvm.declaredRunner;
};
}
I start the microVM with a templated systemd user service:
The above service definition creates a dump of the hosts nix store db so that it can be imported in the guest. This is necessary so that the guest can actually use what is available in /nix/store. There is an effort for an overlayed nix store that would be preferable to this hack.
Finally the microvm is started inside a tmux session named “microvm”. This way I can use the VM with SSH or through the console and also access the qemu console.
And for completeness the virtiofsd service:
[Unit]
Description=serve host persistent folder for dev VM
AssertPathIsDirectory=%h/.local/share/microvm/vms/%i/persistent
[Service]
ExecStart=%h/.local/state/nix/profile/bin/virtiofsd \
--socket-path=${XDG_RUNTIME_DIR}/microvm-%i-persistent \
--shared-dir=%h/.local/share/microvm/vms/%i/persistent \
--gid-map :995:%G:1: \
--uid-map :1000:%U:1:
The speaker however did not mention, thattherehavealreadybeenmanyattempts at building distributed search engines. So why do I think that such an attempt could finally succeed?
More people are searching for alternatives to Google.
Mainstream hard discs are incredibly big.
Mainstream internet connection is incredibly fast.
Google is bleeding talent.
Most of the building blocks are available as free software.
“Success” depends on your definition…
My definition of success is:
A mildly technical computer user (able to install software) has access to a search engine that provides them with superior search results compared to Google for at least a few predefined areas of interest.
The exact algorithm used by Google Search to rank websites is a secret even to most Googlers. Still it is clear, that it relies heavily on big data: billions of queries, a comprehensive web index and user behaviour data. - All this is not available to us.
A distributed search engine however can instead rely on user input. Every admin of one node seeds the node ranking with their personal selection of trusted sites. They connect their node with nodes of people they trust. This results in a web of (transitive) trust much like pgp.
For comparison, imagine you are searching for something in a world without computers: You ask the people around you. They probably forward your question to their peers.
I already had a look at YaCy. It is active, somewhat usable and has a friendly maintainer. Unfortunately I consider the codebase to show its age. It takes a lot of time for a newcomer to find their way around and it contains a lot of cruft. Nevertheless, YaCy is a good example that a decentralized search software can be done even by a small team or just one person.
I myself started working on a software in Haskell and keep my notes here: Populus:DezInV. Since I’m learning Haskell along the way, there is nothing there to see yet. Additionally I took a yak shaving break to learn nix.
By the way: DuckDuckGo is not the alternative. And while I would encourage you to also try Yandex for a second opinion, I don’t consider this a solution.
install software that might not be available in Debian
install software without root access
declare software necessary for a user’s environment inside $HOME/.config
Especially the last point nagged me every time I set up a new Debian installation. My emacs configuration and my Desktop setup expects certain software to be installed.
Please be aware that I’m a beginner with nix and that my config might not follow best practice. Additionally many nix users are already using the new flakes feature of nix that I’m still learning about.
So I’ve got this file at .config/nixpkgs/config.nix1:
You can see that I install nix with nix. This gives me a newer version than the one available in Debian stable. However, the nix-daemon still runs as the older binary from Debian. My dirty hack is to put this override in /etc/systemd/system/nix-daemon.service.d/override.conf:
Unseen Academicals is the 37th Discworld novel and includes many of
the long-standing Ankh-Morpork cast, but mostly as supporting characters.
The main characters are a new (and delightful) bunch with their own
concerns. You arguably could start reading here if you really wanted to,
although you would risk spoiling several previous books (most notably
Thud!) and will miss some references
that depend on familiarity with the cast.
The Unseen University is, like most institutions of its sort, funded by an
endowment that allows the wizards to focus on the pure life of the mind
(or the stomach). Much to their dismay, they have just discovered that an
endowment that amounts to most of their food budget requires that they
field a football team.
Glenda runs the night kitchen at the Unseen University. Given the deep
and abiding love that wizards have for food, there is both a main kitchen
and a night kitchen. The main kitchen is more prestigious, but the night
kitchen is responsible for making pies, something that Glenda is quietly
but exceptionally good at.
Juliet is Glenda's new employee. She is exceptionally beautiful, not very
bright, and a working class girl of the Ankh-Morpork streets down to her
bones. Trevor Likely is a candle dribbler, responsible for assisting the
Candle Knave in refreshing the endless university candles and ensuring
that their wax is properly dribbled, although he pushes most of that work
off onto the infallibly polite and oddly intelligent Mr. Nutt.
Glenda, Trev, and Juliet are the sort of people who populate the great
city of Ankh-Morpork. While the people everyone has heard of have
political crises, adventures, and book plots, they keep institutions like
the Unseen University running. They read romance novels, go to the
football games, and nurse long-standing rivalries. They do not expect the
high mucky-mucks to enter their world, let alone mess with their game.
I approached Unseen Academicals with trepidation because I normally
don't get along as well with the Discworld wizard books. I need not have
worried; Pratchett realized that the wizards would work better as
supporting characters and instead turns the main plot (or at least most of
it; more on that later) over to the servants. This was a brilliant
decision. The setup of this book is some of the best of Discworld up to
this point.
Trev is a streetwise rogue with an uncanny knack for kicking around a can
that he developed after being forbidden to play football by his dear old
mum. He falls for Juliet even though their families support different
football teams, so you may think that a Romeo and Juliet spoof is
coming. There are a few gestures of one, but Pratchett deftly avoids the
pitfalls and predictability and instead makes Juliet one of the best
characters in the book by playing directly against type. She is one of
the characters that Pratchett is so astonishingly good at, the ones that
are so thoroughly themselves that they transcend the stories they're put
into.
The heart of this book, though, is Glenda.
Glenda enjoyed her job. She didn't have a career; they were for people
who could not hold down jobs.
She is the kind of person who knows where she fits in the world and likes
what she does and is happy to stay there until she decides something isn't
right, and then she changes the world through the power of common sense
morality, righteous indignation, and sheer stubborn persistence.
Discworld is full of complex and subtle characters fencing with each
other, but there are few things I have enjoyed more than Glenda being a
determinedly good person. Vetinari of course recognizes and respects (and
uses) that inner core immediately.
Unfortunately, as great as the setup and characters are, Unseen
Academicals falls apart a bit at the end. I was eagerly reading the
story, wondering what Pratchett was going to weave out of the stories of
these individuals, and then it partly turned into yet another wizard book.
Pratchett pulled another of his deus ex machina tricks for the
climax in a way that I found unsatisfying and contrary to the tone of the
rest of the story, and while the characters do get reasonable endings, it
lacked the oomph I was hoping for. Rincewind is as determinedly one-note
as ever, the wizards do all the standard wizard things, and the plot just
isn't that interesting.
I liked Mr. Nutt a great deal in the first part of the book, and I wish he
could have kept that edge of enigmatic competence and unflappableness.
Pratchett wanted to tell a different story that involved more angst and
self-doubt, and while I appreciate that story, I found it less engaging
and a bit more melodramatic than I was hoping for. Mr. Nutt's reactions
in the last half of the book were believable and fit his background, but
that was part of the problem: he slotted back into an archetype that I
thought Pratchett was going to twist and upend.
Mr. Nutt does, at least, get a fantastic closing line, and as usual there
are a lot of great asides and quotes along the way, including possibly the
sharpest and most biting Vetinari speech of the entire series.
The Patrician took a sip of his beer. "I have told this to few
people, gentlemen, and I suspect never will again, but one day when I
was a young boy on holiday in Uberwald I was walking along the bank of
a stream when I saw a mother otter with her cubs. A very endearing
sight, I'm sure you will agree, and even as I watched, the mother
otter dived into the water and came up with a plump salmon, which she
subdued and dragged on to a half-submerged log. As she ate it, while
of course it was still alive, the body split and I remember to this
day the sweet pinkness of its roes as they spilled out, much to the
delight of the baby otters who scrambled over themselves to feed on
the delicacy. One of nature's wonders, gentlemen: mother and children
dining on mother and children. And that's when I first learned about
evil. It is built into the very nature of the universe. Every world
spins in pain. If there is any kind of supreme being, I told myself,
it is up to all of us to become his moral superior."
My dissatisfaction with the ending prevents Unseen Academicals
from rising to the level of Night Watch,
and it's a bit more uneven than the best books of the series. Still,
though, this is great stuff; recommended to anyone who is reading the
series.
Followed in publication order by I Shall Wear Midnight.
I am happy to report that
the megactl package,
useful to fetch RAID status when using the LSI Megaraid controller,
now is available in Debian. It passed NEW a few days ago, and is now
available in
unstable, and probably showing up in testing in a weeks time. The
new version should provide Appstream hardware mapping and should
integrate nicely with isenkram.
As usual, if you use Bitcoin and want to show your support of my
activities, please send Bitcoin donations to my address
15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.
Armadillo is a powerful
and expressive C++ template library for linear algebra and scientific
computing. It aims towards a good balance between speed and ease of use,
has a syntax deliberately close to Matlab, and is useful for algorithm
development directly in C++, or quick conversion of research code into
production environments. RcppArmadillo
integrates this library with the R environment and language–and is
widely used by (currently) 1135 other packages on CRAN, downloaded 33.7 million
times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint
/ vignette) by Conrad and myself has been cited 579 times according
to Google Scholar.
Yesterday’s release accommodates reticulate by
suspending a single test that now ‘croaks’ creating a reverse-dependency
issue for that package. No other changes were made.
The set of changes since the last CRAN release follows.
Changes
in RcppArmadillo version 0.12.8.2.1 (2024-04-15)
One-char bug fix release commenting out one test that upsets reticulate when accessing a scipy sparse matrix
I have mailed to a Debian bug on allegro4.4 describing my reasoning regarding the allegro libraries – in short, allegro4.4 is pretty much dead upstream, and my interest was basically to keep alex4 (which is cool) in Debian, but since it migrated to non-free, my interest in allegro4.4 has waned. So, if anybody would like to still see allegro4.4 in Debian, please step up now and help out. Since it is dead upstream, my reasoning is that it is better to remove it from Debian if no maintainer who wants to help steps up.
Previously Tobias Hansen has helped out, but now it is 8 (!) years since his last upload of either package. (Please don’t interpret this as judgement, I am very happy for the help he has provided and all the work he has done on the packages).
Allegro5 is another deal – still active upstream, and I have kept it up to date in Debian, and while I have held the latest upload a short while because of the time_t transition, it will come sooner or later – There I am also waiting on a final decision on this bug from upstream. Other than that allegro 5 is in a very good state, and I will keep maintaining it as long as I can. But help would of course be appreciated on allegro5 too.
There are several packages in Debian without a associated git
repository with the packaging history. This is unfortunate and it
would be nice if more of these would do so. Quote a lot of these are
without a maintainer, ie listed as maintained by the
'Debian
QA Group' place holder. In fact, 438 packages have this property
according to UDD (SELECT source FROM sources WHERE release = 'sid'
AND (vcs_url ilike '%anonscm.debian.org%' OR vcs_browser ilike
'%anonscm.debian.org%' or vcs_url IS NULL OR vcs_browser IS NULL) AND
maintainer ilike '%packages@qa.debian.org%';). Such packages can
be updated without much coordination by any Debian developer, as they
are considered orphaned.
To try to improve the situation and reduce the number of packages
without associated git repository, I started a few days ago to search
out candiates and provide them with a git repository under the
'debian' collaborative Salsa project. I started with the packages
pointing to obsolete Alioth git repositories, and am now working my
way across the ones completely without git references. In addition to
updating the Vcs-* debian/control fields, I try to update
Standards-Version, debhelper compat level, simplify d/rules, switch to
Rules-Requires-Root: no and fix lintian issues reported. I only
implement those that are trivial to fix, to avoid spending too much
time on each orphaned package. So far my experience is that it take
aproximately 20 minutes to convert a package without any git
references, and a lot more for packages with existing git repositories
incompatible with git-buildpackages.
So far I have converted 10 packages, and I will keep going until I
run out of steam. As should be clear from the numbers, there is
enough packages remaining for more people to do the same without
stepping on each others toes. I find it useful to start by searching
for a git repo already on salsa, as I find that some times a git repo
has already been created, but no new version is uploaded to Debian
yet. In those cases I start with the existing git repository. I
convert to the git-buildpackage+pristine-tar workflow, and ensure a
debian/gbp.conf file with "pristine-tar=True" is added early, to avoid
uploading a orig.tar.gz with the wrong checksum by mistake. Did that
three times in the begin before I remembered my mistake.
So, if you are a Debian Developer and got some spare time, perhaps
considering migrating some orphaned packages to git?
As usual, if you use Bitcoin and want to show your support of my
activities, please send Bitcoin donations to my address
15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.
With the release of Libntlm version 1.8 the release tarball can be reproduced on several distributions. We also publish a signed minimal source-only tarball, produced by git-archive which is the same format used by Savannah, Codeberg, GitLab, GitHub and others. Reproducibility of both tarballs are tested continuously for regressions on GitLab through a CI/CD pipeline. If that wasn’t enough to excite you, the Debian packages of Libntlm are now built from the reproducible minimal source-only tarball. The resulting binaries are reproducible on several architectures.
What does that even mean? Why should you care? How you can do the same for your project? What are the open issues? Read on, dear reader…
Let’s look at how a maintainer release some software, and how a user can reproduce the released artifacts from the source code. Libntlm provides a shared library written in C and uses GNU Make, GNU Autoconf, GNU Automake, GNU Libtool and gnulib for build management, but these ideas should apply to most project and build system. The following illustrate the steps a maintainer would take to prepare a release:
git clone https://gitlab.com/gsasl/libntlm.git
cd libntlm
git checkout v1.8
./bootstrap
./configure
make distcheck
gpg -b libntlm-1.8.tar.gz
The generated files libntlm-1.8.tar.gz and libntlm-1.8.tar.gz.sig are published, and users download and use them. This is how the GNU project have been doing releases since the late 1980’s. That is a testament to how successful this pattern has been! These tarballs contain source code and some generated files, typically shell scripts generated by autoconf, makefile templates generated by automake, documentation in formats like Info, HTML, or PDF. Rarely do they contain binary object code, but historically that happened.
The XZUtils incident illustrate that tarballs with files that are not included in the git archive offer an opportunity to disguise malicious backdoors. I blogged earlier how to mitigate this risk by using signed minimal source-only tarballs.
The risk of hiding malware is not the only motivation to publish signed minimal source-only tarballs. With pre-generated content in tarballs, there is a risk that GNU/Linux distributions such as Trisquel, Guix, Debian/Ubuntu or Fedora ship generated files coming from the tarball into the binary *.deb or *.rpm package file. Typically the person packaging the upstream project never realized that some installed artifacts was not re-built through a typical autoconf -fi && ./configure && make install sequence, and never wrote the code to rebuild everything. This can also happen if the build rules are written but are buggy, shipping the old artifact. When a security problem is found, this can lead to time-consuming situations, as it may be that patching the relevant source code and rebuilding the package is not sufficient: the vulnerable generated object from the tarball would be shipped into the binary package instead of a rebuilt artifact. For architecture-specific binaries this rarely happens, since object code is usually not included in tarballs — although for 10+ years I shipped the binary Java JAR file in the GNU Libidn release tarball, until I stopped shipping it. For interpreted languages and especially for generated content such as HTML, PDF, shell scripts this happens more than you would like.
Publishing minimal source-only tarballs enable easier auditing of a project’s code, to avoid the need to read through all generated files looking for malicious content. I have taken care to generate the source-only minimal tarball using git-archive. This is the same format that GitLab, GitHub etc offer for the automated download links on git tags. The minimal source-only tarballs can thus serve as a way to audit GitLab and GitHub download material! Consider if/when hosting sites like GitLab or GitHub has a security incident that cause generated tarballs to include a backdoor that is not present in the git repository. If people rely on the tag download artifact without verifying the maintainer PGP signature using GnuPG, this can lead to similar backdoor scenarios that we had for XZUtils but originated with the hosting provider instead of the release manager. This is even more concerning, since this attack can be mounted for some selected IP address that you want to target and not on everyone, thereby making it harder to discover.
With all that discussion and rationale out of the way, let’s return to the release process. I have added another step here:
make srcdist
gpg -b libntlm-1.8-src.tar.gz
Now the release is ready. I publish these four files in the Libntlm’s Savannah Download area, but they can be uploaded to a GitLab/GitHub release area as well. These are the SHA256 checksums I got after building the tarballs on my Trisquel 11 aramo laptop:
So how can you reproduce my artifacts? Here is how to reproduce them in a Ubuntu 22.04 container:
podman run -it --rm ubuntu:22.04
apt-get update
apt-get install -y --no-install-recommends autoconf automake libtool make git ca-certificates
git clone https://gitlab.com/gsasl/libntlm.git
cd libntlm
git checkout v1.8
./bootstrap
./configure
make dist srcdist
sha256sum libntlm-*.tar.gz
You should see the exact same SHA256 checksum values. Hooray!
This works because Trisquel 11 and Ubuntu 22.04 uses the same version of git, autoconf, automake, and libtool. These tools do not guarantee the same output content for all versions, similar to how GNU GCC does not generate the same binary output for all versions. So there is still some delicate version pairing needed.
Ideally, the artifacts should be possible to reproduce from the release artifacts themselves, and not only directly from git. It is possible to reproduce the full tarball in a AlmaLinux 8 container – replace almalinux:8 with rockylinux:8 if you prefer RockyLinux:
podman run -it --rm almalinux:8
dnf update -y
dnf install -y make wget gcc
wget https://download.savannah.nongnu.org/releases/libntlm/libntlm-1.8.tar.gz
tar xfa libntlm-1.8.tar.gz
cd libntlm-1.8
./configure
make dist
sha256sum libntlm-1.8.tar.gz
The source-only minimal tarball can be regenerated on Debian 11:
podman run -it --rm debian:11
apt-get update
apt-get install -y --no-install-recommends make git ca-certificates
git clone https://gitlab.com/gsasl/libntlm.git
cd libntlm
git checkout v1.8
make -f cfg.mk srcdist
sha256sum libntlm-1.8-src.tar.gz
As the Magnus Opus or chef-d’œuvre, let’s recreate the full tarball directly from the minimal source-only tarball on Trisquel 11 – replace docker.io/kpengboy/trisquel:11.0 with ubuntu:22.04 if you prefer.
podman run -it --rm docker.io/kpengboy/trisquel:11.0
apt-get update
apt-get install -y --no-install-recommends autoconf automake libtool make wget git ca-certificates
wget https://download.savannah.nongnu.org/releases/libntlm/libntlm-1.8-src.tar.gz
tar xfa libntlm-1.8-src.tar.gz
cd libntlm-v1.8
./bootstrap
./configure
make dist
sha256sum libntlm-1.8.tar.gz
Yay! You should now have great confidence in that the release artifacts correspond to what’s in version control and also to what the maintainer intended to release. Your remaining job is to audit the source code for vulnerabilities, including the source code of the dependencies used in the build. You no longer have to worry about auditing the release artifacts.
I find it somewhat amusing that the build infrastructure for Libntlm is now in a significantly better place than the code itself. Libntlm is written in old C style with plenty of string manipulation and uses broken cryptographic algorithms such as MD4 and single-DES. Remember folks: solving supply chain security issues has no bearing on what kind of code you eventually run. A clean gun can still shoot you in the foot.
Side note on naming: GitLab exports tarballs with pathnames libntlm-v1.8/ (i.e.., PROJECT-TAG/) and I’ve adopted the same pathnames, which means my libntlm-1.8-src.tar.gz tarballs are bit-by-bit identical to GitLab’s exports and you can verify this with tools like diffoscope. GitLab name the tarball libntlm-v1.8.tar.gz (i.e., PROJECT-TAG.ARCHIVE) which I find too similar to the libntlm-1.8.tar.gz that we also publish. GitHub uses the same git archive style, but unfortunately they have logic that removes the ‘v’ in the pathname so you will get a tarball with pathname libntlm-1.8/ instead of libntlm-v1.8/ that GitLab and I use. The content of the tarball is bit-by-bit identical, but the pathname and archive differs. Codeberg (running Forgejo) uses another approach: the tarball is called libntlm-v1.8.tar.gz (after the tag) just like GitLab, but the pathname inside the archive is libntlm/, otherwise the produced archive is bit-by-bit identical including timestamps. Savannah’s CGIT interface uses archive name libntlm-1.8.tar.gz with pathname libntlm-1.8/, but otherwise file content is identical. Savannah’s GitWeb interface provides snapshot links that are named after the git commit (e.g., libntlm-a812c2ca.tar.gz with libntlm-a812c2ca/) and I cannot find any tag-based download links at all. Overall, we are so close to get SHA256 checksum to match, but fail on pathname within the archive. I’ve chosen to be compatible with GitLab regarding the content of tarballs but not on archive naming. From a simplicity point of view, it would be nice if everyone used PROJECT-TAG.ARCHIVE for the archive filename and PROJECT-TAG/ for the pathname within the archive. This aspect will probably need more discussion.
Side note on git archive output: It seems different versions of git archive produce different results for the same repository. The version of git in Debian 11, Trisquel 11 and Ubuntu 22.04 behave the same. The version of git in Debian 12, AlmaLinux/RockyLinux 8/9, Alpine, ArchLinux, macOS homebrew, and upcoming Ubuntu 24.04 behave in another way. Hopefully this will not change that often, but this would invalidate reproducibility of these tarballs in the future, forcing you to use an old git release to reproduce the source-only tarball. Alas, GitLab and most other sites appears to be using modern git so the download tarballs from them would not match my tarballs – even though the content would.
Side note on ChangeLog: ChangeLog files were traditionally manually curated files with version history for a package. In recent years, several projects moved to dynamically generate them from git history (using tools like git2cl or gitlog-to-changelog). This has consequences for reproducibility of tarballs: you need to have the entire git history available! The gitlog-to-changelog tool also output different outputs depending on the time zone of the person using it, which arguable is a simple bug that can be fixed. However this entire approach is incompatible with rebuilding the full tarball from the minimal source-only tarball. It seems Libntlm’s ChangeLog file died on the surgery table here.
So how would a distribution build these minimal source-only tarballs? I happen to help on the libntlm package in Debian. It has historically used the generated tarballs as the source code to build from. This means that code coming from gnulib is vendored in the tarball. When a security problem is discovered in gnulib code, the security team needs to patch all packages that include that vendored code and rebuild them, instead of merely patching the gnulib package and rebuild all packages that rely on that particular code. To change this, the Debian libntlm package needs to Build-Depends on Debian’s gnulib package. But there was one problem: similar to most projects that use gnulib, Libntlm depend on a particular git commit of gnulib, and Debian only ship one commit. There is no coordination about which commit to use. I have adopted gnulib in Debian, and add a git bundle to the *_all.deb binary package so that projects that rely on gnulib can pick whatever commit they need. This allow an no-network GNULIB_URL and GNULIB_REVISION approach when running Libntlm’s ./bootstrap with the Debian gnulib package installed. Otherwise libntlm would pick up whatever latest version of gnulib that Debian happened to have in the gnulib package, which is not what the Libntlm maintainer intended to be used, and can lead to all sorts of version mismatches (and consequently security problems) over time. Libntlm in Debian is developed and tested on Salsa and there is continuous integration testing of it as well, thanks to the Salsa CI team.
Side note on git bundles: unfortunately there appears to be no reproducible way to export a git repository into one or more files. So one unfortunate consequence of all this work is that the gnulib *.orig.tar.gz tarball in Debian is not reproducible any more. I have tried to get Git bundles to be reproducible but I never got it to work — see my notes in gnulib’s debian/README.source on this aspect. Of course, source tarball reproducibility has nothing to do with binary reproducibility of gnulib in Debian itself, fortunately.
One open question is how to deal with the increased build dependencies that is triggered by this approach. Some people are surprised by this but I don’t see how to get around it: if you depend on source code for tools in another package to build your package, it is a bad idea to hide that dependency. We’ve done it for a long time through vendored code in non-minimal tarballs. Libntlm isn’t the most critical project from a bootstrapping perspective, so adding git and gnulib as Build-Depends to it will probably be fine. However, consider if this pattern was used for other packages that uses gnulib such as coreutils, gzip, tar, bison etc (all are using gnulib) then they would all Build-Depends on git and gnulib. Cross-building those packages for a new architecture will therefor require git on that architecture first, which gets circular quick. The dependency on gnulib is real so I don’t see that going away, and gnulib is a Architecture:all package. However, the dependency on git is merely a consequence of how the Debian gnulib package chose to make all gnulib git commits available to projects: through a git bundle. There are other ways to do this that doesn’t require the git tool to extract the necessary files, but none that I found practical — ideas welcome!
Finally some brief notes on how this was implemented. Enabling bootstrappable source-only minimal tarballs via gnulib’s ./bootstrap is achieved by using the GNULIB_REVISION mechanism, locking down the gnulib commit used. I have always disliked git submodules because they add extra steps and has complicated interaction with CI/CD. The reason why I gave up git submodules now is because the particular commit to use is not recorded in the git archive output when git submodules is used. So the particular gnulib commit has to be mentioned explicitly in some source code that goes into the git archive tarball. Colin Watson added the GNULIB_REVISION approach to ./bootstrap back in 2018, and now it no longer made sense to continue to use a gnulib git submodule. One alternative is to use ./bootstrap with --gnulib-srcdir or --gnulib-refdir if there is some practical problem with the GNULIB_URL towards a git bundle the GNULIB_REVISION in bootstrap.conf.
git archive --prefix=libntlm-v1.8/ -o libntlm-v1.8.tar.gz HEAD
Making the make dist generated tarball reproducible can be more complicated, however for Libntlm it was sufficient to make sure the modification times of all files were set deterministically to the timestamp of the last commit in the git repository. Interestingly there seems to be a couple of different ways to accomplish this, Guix doesn’t support minimal source-only tarballs but rely on a .tarball-timestamp file inside the tarball. Paul Eggert explained what TZDB is using some time ago. The approach I’m using now is fairly similar to the one I suggested over a year ago. If there are problems because all files in the tarball now use the same modification time, there is a solution by Bruno Haible that could be implemented.
Side note on git tags: Some people may wonder why not verify a signed git tag instead of verifying a signed tarball of the git archive. Currently most git repositories uses SHA-1 for git commit identities, but SHA-1 is not a secure hash function. While current SHA-1 attacks can be detected and mitigated, there are fundamental doubts that a git SHA-1 commit identity uniquely refers to the same content that was intended. Verifying a git tag will never offer the same assurance, since a git tag can be moved or re-signed at any time. Verifying a git commit is better but then we need to trust SHA-1. Migrating git to SHA-256 would resolve this aspect, but most hosting sites such as GitLab and GitHub does not support this yet. There are other advantages to using signed tarballs instead of signed git commits or git tags as well, e.g., tar.gz can be a deterministically reproducible persistent stable offline storage format but .git sub-directory trees or git bundles do not offer this property.
Doing continous testing of all this is critical to make sure things don’t regress. Libntlm’s pipeline definition now produce the generated libntlm-*.tar.gz tarballs and a checksum as a build artifact. Then I added the 000-reproducability job which compares the checksums and fails on mismatches. You can read its delicate output in the job for the v1.8 release. Right now we insists that builds on Trisquel 11 match Ubuntu 22.04, that PureOS 10 builds match Debian 11 builds, that AlmaLinux 8 builds match RockyLinux 8 builds, and AlmaLinux 9 builds match RockyLinux 9 builds. As you can see in pipeline job output, not all platforms lead to the same tarballs, but hopefully this state can be improved over time. There is also partial reproducibility, where the full tarball is reproducible across two distributions but not the minimal tarball, or vice versa.
If this way of working plays out well, I hope to implement it in other projects too.
Years ago, at what I think I remember was DebConf 15, I hacked for a while
on debhelper to
write build-ids to debian binary control files,
so that the build-id (more specifically, the ELF note
.note.gnu.build-id) wound up in the Debian apt archive metadata.
I’ve always thought this was super cool, and seeing as how Michael Stapelberg
blogged
some great pointers around the ecosystem, including the fancy new debuginfod
service, and the
find-dbgsym-packages
helper, which uses these same headers, I don’t think I’m the only one.
At work I’ve been using a lot of rust,
specifically, async rust using tokio. To try and work on
my style, and to dig deeper into the how and why of the decisions made in these
frameworks, I’ve decided to hack up a project that I’ve wanted to do ever
since 2015 – write a debug filesystem. Let’s get to it.
Back to the Future
Time to admit something. I really love Plan 9. It’s
just so good. So many ideas from Plan 9 are just so prescient, and everything
just feels right. Not just right like, feels good – like, correct. The
bit that I’ve always liked the most is 9p, the network protocol for serving
a filesystem over a network. This leads to all sorts of fun programs, like the
Plan 9 ftp client being a 9p server – you mount the ftp server and access
files like any other files. It’s kinda like if fuse were more fully a part
of how the operating system worked, but fuse is all running client-side. With
9p there’s a single client, and different servers that you can connect to,
which may be backed by a hard drive, remote resources over something like SFTP, FTP, HTTP or even purely synthetic.
The interesting (maybe sad?) part here is that 9p wound up outliving Plan 9
in terms of adoption – 9p is in all sorts of places folks don’t usually expect.
For instance, the Windows Subsystem for Linux uses the 9p protocol to share
files between Windows and Linux. ChromeOS uses it to share files with Crostini,
and qemu uses 9p (virtio-p9) to share files between guest and host. If you’re
noticing a pattern here, you’d be right; for some reason 9p is the go-to protocol
to exchange files between hypervisor and guest. Why? I have no idea, except maybe
due to being designed well, simple to implement, and it’s a lot easier to validate the data being shared
and validate security boundaries. Simplicity has its value.
As a result, there’s a lot of lingering 9p support kicking around. Turns out
Linux can even handle mounting 9p filesystems out of the box. This means that I
can deploy a filesystem to my LAN or my localhost by running a process on top
of a computer that needs nothing special, and mount it over the network on an
unmodified machine – unlike fuse, where you’d need client-specific software
to run in order to mount the directory. For instance, let’s mount a 9p
filesystem running on my localhost machine, serving requests on 127.0.0.1:564
(tcp) that goes by the name “mountpointname” to /mnt.
Linux will mount away, and attach to the filesystem as the root user, and by default,
attach to that mountpoint again for each local user that attempts to use
it. Nifty, right? I think so. The server is able
to keep track of per-user access and authorization
along with the host OS.
WHEREIN I STYX WITH IT
Since I wanted to push myself a bit more with rust and tokio specifically,
I opted to implement the whole stack myself, without third party libraries on
the critical path where I could avoid it. The 9p protocol (sometimes called
Styx, the original name for it) is incredibly simple. It’s a series of client
to server requests, which receive a server to client response. These are,
respectively, “T” messages, which transmit a request to the server, which
trigger an “R” message in response (Reply messages). These messages are
TLV payload
with a very straight forward structure – so straight forward, in fact, that I
was able to implement a working server off nothing more than a handful of man
pages.
Later on after the basics worked, I found a more complete
spec page
that contains more information about the
unix specific variant
that I opted to use (9P2000.u rather than 9P2000) due to the level
of Linux specific support for the 9P2000.u variant over the 9P2000
protocol.
MR ROBOTO
The backend stack over at zoo is rust and tokio
running i/o for an HTTP and WebRTC server. I figured I’d pick something
fairly similar to write my filesystem with, since 9P can be implemented
on basically anything with I/O. That means tokio tcp server bits, which
construct and use a 9p server, which has an idiomatic Rusty API that
partially abstracts the raw R and T messages, but not so much as to
cause issues with hiding implementation possibilities. At each abstraction
level, there’s an escape hatch – allowing someone to implement any of
the layers if required. I called this framework
arigato which can be found over on
docs.rs and
crates.io.
/// Simplified version of the arigato File trait; this isn't actually
/// the same trait; there's some small cosmetic differences. The
/// actual trait can be found at:
///
/// https://docs.rs/arigato/latest/arigato/server/trait.File.html
trait File {
/// OpenFile is the type returned by this File via an Open call.
typeOpenFile: OpenFile;
/// Return the 9p Qid for this file. A file is the same if the Qid is
/// the same. A Qid contains information about the mode of the file,
/// version of the file, and a unique 64 bit identifier.
fnqid(&self) -> Qid;
/// Construct the 9p Stat struct with metadata about a file.
async fnstat(&self) -> FileResult<Stat>;
/// Attempt to update the file metadata.
async fnwstat(&mut self, s: &Stat) -> FileResult<()>;
/// Traverse the filesystem tree.
async fnwalk(&self, path: &[&str]) -> FileResult<(Option<Self>, Vec<Self>)>;
/// Request that a file's reference be removed from the file tree.
async fnunlink(&mut self) -> FileResult<()>;
/// Create a file at a specific location in the file tree.
async fncreate(
&mut self,
name: &str,
perm: u16,
ty: FileType,
mode: OpenMode,
extension: &str,
) -> FileResult<Self>;
/// Open the File, returning a handle to the open file, which handles
/// file i/o. This is split into a second type since it is genuinely
/// unrelated -- and the fact that a file is Open or Closed can be
/// handled by the `arigato` server for us.
async fnopen(&mut self, mode: OpenMode) -> FileResult<Self::OpenFile>;
}
/// Simplified version of the arigato OpenFile trait; this isn't actually
/// the same trait; there's some small cosmetic differences. The
/// actual trait can be found at:
///
/// https://docs.rs/arigato/latest/arigato/server/trait.OpenFile.html
trait OpenFile {
/// iounit to report for this file. The iounit reported is used for Read
/// or Write operations to signal, if non-zero, the maximum size that is
/// guaranteed to be transferred atomically.
fniounit(&self) -> u32;
/// Read some number of bytes up to `buf.len()` from the provided
/// `offset` of the underlying file. The number of bytes read is
/// returned.
async fnread_at(
&mut self,
buf: &mut [u8],
offset: u64,
) -> FileResult<u32>;
/// Write some number of bytes up to `buf.len()` from the provided
/// `offset` of the underlying file. The number of bytes written
/// is returned.
fnwrite_at(
&mut self,
buf: &mut [u8],
offset: u64,
) -> FileResult<u32>;
}
Thanks, decade ago paultag!
Let’s do it! Let’s use arigato to implement a 9p filesystem we’ll call
debugfs that will serve all the debug
files shipped according to the Packages metadata from the apt archive. We’ll
fetch the Packages file and construct a filesystem based on the reported
Build-Id entries. For those who don’t know much about how an apt repo
works, here’s the 2-second crash course on what we’re doing. The first is to
fetch the Packages file, which is specific to a binary architecture (such as
amd64, arm64 or riscv64). That architecture is specific to a
component (such as main, contrib or non-free). That component is
specific to a suite, such as stable, unstable or any of its aliases
(bullseye, bookworm, etc). Let’s take a look at the Packages.xz file for
the unstable-debugsuite, maincomponent, for all amd64 binaries.
This will return the Debian-style
rfc2822-like headers,
which is an export of the metadata contained inside each .deb file which
apt (or other tools that can use the apt repo format) use to fetch
information about debs. Let’s take a look at the debug headers for the
netlabel-tools package in unstable – which is a package named
netlabel-tools-dbgsym in unstable-debug.
So here, we can parse the package headers in the Packages.xz file, and store,
for each Build-Id, the Filename where we can fetch the .deb at. Each
.deb contains a number of files – but we’re only really interested in the
files inside the .deb located at or under /usr/lib/debug/.build-id/,
which you can find in debugfs under
rfc822.rs. It’s
crude, and very single-purpose, but I’m feeling a bit lazy.
Who needs dpkg?!
For folks who haven’t seen it yet, a .deb file is a special type of
.ar file, that contains (usually)
three files inside – debian-binary, control.tar.xz and data.tar.xz.
The core of an .ar file is a fixed size (60 byte) entry header,
followed by the specified size number of bytes.
[8 byte .ar file magic]
[60 byte entry header]
[N bytes of data]
[60 byte entry header]
[N bytes of data]
[60 byte entry header]
[N bytes of data]
...
First up was to implement a basic ar parser in
ar.rs. Before we get
into using it to parse a deb, as a quick diversion, let’s break apart a .deb
file by hand – something that is a bit of a rite of passage (or at least it
used to be? I’m getting old) during the Debian nm (new member) process, to take
a look at where exactly the .debug file lives inside the .deb file.
$ ar x netlabel-tools-dbgsym_0.30.0-1+b1_amd64.deb
$ ls
control.tar.xz debian-binary
data.tar.xz netlabel-tools-dbgsym_0.30.0-1+b1_amd64.deb
$ tar --list -f data.tar.xz | grep '.debug$'
./usr/lib/debug/.build-id/e5/9f81f6573dadd5d95a6e4474d9388ab2777e2a.debug
Since we know quite a bit about the structure of a .deb file, and I had to
implement support from scratch anyway, I opted to implement a (very!) basic
debfile parser using HTTP Range requests. HTTP Range requests, if supported by
the server (denoted by a accept-ranges: bytes HTTP header in response to an
HTTP HEAD request to that file) means that we can add a header such as
range: bytes=8-68 to specifically request that the returned GET body be the
byte range provided (in the above case, the bytes starting from byte offset 8
until byte offset 68). This means we can fetch just the ar file entry from
the .deb file until we get to the file inside the .deb we are interested in
(in our case, the data.tar.xz file) – at which point we can request the body
of that file with a final range request. I wound up writing a struct to
handle a read_at-style API surface in
hrange.rs, which
we can pair with ar.rs above and start to find our data in the .deb remotely
without downloading and unpacking the .deb at all.
After we have the body of the data.tar.xz coming back through the HTTP
response, we get to pipe it through an xz decompressor (this kinda sucked in
Rust, since a tokioAsyncRead is not the same as an http Body response is
not the same as std::io::Read, is not the same as an async (or sync)
Iterator is not the same as what the xz2 crate expects; leading me to read
blocks of data to a buffer and stuff them through the decoder by looping over
the buffer for each lzma2 packet in a loop), and tarfile parser (similarly
troublesome). From there we get to iterate over all entries in the tarfile,
stopping when we reach our file of interest. Since we can’t seek, but gdb
needs to, we’ll pull it out of the stream into a Cursor<Vec<u8>> in-memory
and pass a handle to it back to the user.
I was originally hoping to avoid transferring the whole tar file over the
network (and therefore also reading the whole debug file into ram, which
objectively sucks), but quickly hit issues with figuring out a way around
seeking around an xz file. What’s interesting is xz has a great primitive
to solve this specific problem (specifically, use a block size that allows you
to seek to the block as close to your desired seek position just before it,
only discarding at most block size - 1 bytes), but data.tar.xz files
generated by dpkg appear to have a single mega-huge block for the whole file.
I don’t know why I would have expected any different, in retrospect. That means
that this now devolves into the base case of “How do I seek around an lzma2
compressed data stream”; which is a lot more complex of a question.
Thankfully, notoriously brilliant tianon was
nice enough to introduce me to Jon Johnson
who did something super similar – adapted a technique to seek inside a
compressed gzip file, which lets his service
oci.dag.dev
seek through Docker container images super fast based on some prior work
such as soci-snapshotter, gztool, and
zran.c.
He also pulled this party trick off for apk based distros
over at apk.dag.dev, which seems apropos.
Jon was nice enough to publish a lot of his work on this specifically in a
central place under the name “targz”
on his GitHub, which has been a ton of fun to read through.
The gist is that, by dumping the decompressor’s state (window of previous
bytes, in-memory data derived from the last N-1 bytes) at specific
“checkpoints” along with the compressed data stream offset in bytes and
decompressed offset in bytes, one can seek to that checkpoint in the compressed
stream and pick up where you left off – creating a similar “block” mechanism
against the wishes of gzip. It means you’d need to do an O(n) run over the
file, but every request after that will be sped up according to the number
of checkpoints you’ve taken.
Given the complexity of xz and lzma2, I don’t think this is possible
for me at the moment – especially given most of the files I’ll be requesting
will not be loaded from again – especially when I can “just” cache the debug
header by Build-Id. I want to implement this (because I’m generally curious
and Jon has a way of getting someone excited about compression schemes, which
is not a sentence I thought I’d ever say out loud), but for now I’m going to
move on without this optimization. Such a shame, since it kills a lot of the
work that went into seeking around the .deb file in the first place, given
the debian-binary and control.tar.gz members are so small.
The Good
First, the good news right? It works! That’s pretty cool. I’m positive
my younger self would be amused and happy to see this working; as is
current day paultag. Let’s take debugfs out for a spin! First, we need
to mount the filesystem. It even works on an entirely unmodified, stock
Debian box on my LAN, which is huge. Let’s take it for a spin:
And, let’s prove to ourselves that this actually mounted before we go
trying to use it:
$ mount | grep build-id
192.168.0.2 on /usr/lib/debug/.build-id type 9p (rw,relatime,aname=unstable-debug,access=user,trans=tcp,version=9p2000.u,port=564)
Slick. We’ve got an open connection to the server, where our host
will keep a connection alive as root, attached to the filesystem provided
in aname. Let’s take a look at it.
$ ls /usr/lib/debug/.build-id/
00 0d 1a 27 34 41 4e 5b 68 75 82 8E 9b a8 b5 c2 CE db e7 f3
01 0e 1b 28 35 42 4f 5c 69 76 83 8f 9c a9 b6 c3 cf dc E7 f4
02 0f 1c 29 36 43 50 5d 6a 77 84 90 9d aa b7 c4 d0 dd e8 f5
03 10 1d 2a 37 44 51 5e 6b 78 85 91 9e ab b8 c5 d1 de e9 f6
04 11 1e 2b 38 45 52 5f 6c 79 86 92 9f ac b9 c6 d2 df ea f7
05 12 1f 2c 39 46 53 60 6d 7a 87 93 a0 ad ba c7 d3 e0 eb f8
06 13 20 2d 3a 47 54 61 6e 7b 88 94 a1 ae bb c8 d4 e1 ec f9
07 14 21 2e 3b 48 55 62 6f 7c 89 95 a2 af bc c9 d5 e2 ed fa
08 15 22 2f 3c 49 56 63 70 7d 8a 96 a3 b0 bd ca d6 e3 ee fb
09 16 23 30 3d 4a 57 64 71 7e 8b 97 a4 b1 be cb d7 e4 ef fc
0a 17 24 31 3e 4b 58 65 72 7f 8c 98 a5 b2 bf cc d8 E4 f0 fd
0b 18 25 32 3f 4c 59 66 73 80 8d 99 a6 b3 c0 cd d9 e5 f1 fe
0c 19 26 33 40 4d 5a 67 74 81 8e 9a a7 b4 c1 ce da e6 f2 ff
Outstanding. Let’s try using gdb to debug a binary that was provided by
the Debian archive, and see if it’ll load the ELF by build-id from the
right .deb in the unstable-debug suite:
$ gdb -q /usr/sbin/netlabelctl
Reading symbols from /usr/sbin/netlabelctl...
Reading symbols from /usr/lib/debug/.build-id/e5/9f81f6573dadd5d95a6e4474d9388ab2777e2a.debug...
(gdb)
Yes! Yes it will!
$ file /usr/lib/debug/.build-id/e5/9f81f6573dadd5d95a6e4474d9388ab2777e2a.debug
/usr/lib/debug/.build-id/e5/9f81f6573dadd5d95a6e4474d9388ab2777e2a.debug: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter *empty*, BuildID[sha1]=e59f81f6573dadd5d95a6e4474d9388ab2777e2a, for GNU/Linux 3.2.0, with debug_info, not stripped
The Bad
Linux’s support for 9p is mainline, which is great, but it’s not robust.
Network issues or server restarts will wedge the mountpoint (Linux can’t
reconnect when the tcp connection breaks), and things that work fine on local
filesystems get translated in a way that causes a lot of network chatter – for
instance, just due to the way the syscalls are translated, doing an ls, will
result in a stat call for each file in the directory, even though linux had
just got a stat entry for every file while it was resolving directory names.
On top of that, Linux will serialize all I/O with the server, so there’s no
concurrent requests for file information, writes, or reads pending at the same
time to the server; and read and write throughput will degrade as latency
increases due to increasing round-trip time, even though there are offsets
included in the read and write calls. It works well enough, but is
frustrating to run up against, since there’s not a lot you can do server-side
to help with this beyond implementing the 9P2000.L variant (which, maybe is
worth it).
The Ugly
Unfortunately, we don’t know the file size(s) until we’ve actually opened the
underlying tar file and found the correct member, so for most files, we don’t
know the real size to report when getting a stat. We can’t parse the tarfiles
for every stat call, since that’d make ls even slower (bummer). Only
hiccup is that when I report a filesize of zero, gdb throws a bit of a
fit; let’s try with a size of 0 to start:
$ ls -lah /usr/lib/debug/.build-id/e5/9f81f6573dadd5d95a6e4474d9388ab2777e2a.debug
-r--r--r-- 1 root root 0 Dec 31 1969 /usr/lib/debug/.build-id/e5/9f81f6573dadd5d95a6e4474d9388ab2777e2a.debug
$ gdb -q /usr/sbin/netlabelctl
Reading symbols from /usr/sbin/netlabelctl...
Reading symbols from /usr/lib/debug/.build-id/e5/9f81f6573dadd5d95a6e4474d9388ab2777e2a.debug...
warning: Discarding section .note.gnu.build-id which has a section size (24) larger than the file size [in module /usr/lib/debug/.build-id/e5/9f81f6573dadd5d95a6e4474d9388ab2777e2a.debug]
[...]
This obviously won’t work since gdb will throw away all our hard work because
of stat’s output, and neither will loading the real size of the underlying
file. That only leaves us with hardcoding a file size and hope nothing else
breaks significantly as a result. Let’s try it again:
$ ls -lah /usr/lib/debug/.build-id/e5/9f81f6573dadd5d95a6e4474d9388ab2777e2a.debug
-r--r--r-- 1 root root 954M Dec 31 1969 /usr/lib/debug/.build-id/e5/9f81f6573dadd5d95a6e4474d9388ab2777e2a.debug
$ gdb -q /usr/sbin/netlabelctl
Reading symbols from /usr/sbin/netlabelctl...
Reading symbols from /usr/lib/debug/.build-id/e5/9f81f6573dadd5d95a6e4474d9388ab2777e2a.debug...
(gdb)
Much better. I mean, terrible but better. Better for now, anyway.
Kilroy was here
Do I think this is a particularly good idea? I mean; kinda. I’m probably going
to make some fun 9parigato-based filesystems for use around my LAN, but I
don’t think I’ll be moving to use debugfs until I can figure out how to
ensure the connection is more resilient to changing networks, server restarts
and fixes on i/o performance. I think it was a useful exercise and is a pretty
great hack, but I don’t think this’ll be shipping anywhere anytime soon.
Along with me publishing this post, I’ve pushed up all my repos; so you
should be able to play along at home! There’s a lot more work to be done
on arigato; but it does handshake and successfully export a working
9P2000.u filesystem. Check it out on on my github at
arigato,
debugfs
and also on crates.io
and docs.rs.
At least I can say I was here and I got it working after all these years.
When I first started studying computer science setting up a programming project was easy, write source code files and a Makefile and that was it. IRC was the only IM system and email was the only other communications system that was used much. Writing Makefiles is difficult but products like the Borland Turbo series of IDEs did all that for you so you could just start typing code and press a function key to compile and run (F5 from memory).
Over the years the requirements and expectations of computer use have grown significantly. The typical office worker is now doing many more things with computers than serious programmers used to do. Running an IM system, an online document editing system, and a series of web apps is standard for companies nowadays. Developers have to do all that in addition to tools for version control, continuous integration, bug reporting, and feature tracking. The development process is also more complex with extra steps for reproducible builds, automated tests, and code coverage metrics for the tests. I wonder how many programmers who started in the 90s would have done something else if faced with Github as their introduction.
How much of this is good? Having the ability to send instant messages all around the world is great. Having dozens of different ways of doing so is awful. When a company uses multiple IM systems such as MS-Teams and Slack and forces some of it’s employees to use them both it’s getting ridiculous. Having different friend groups on different IM systems is anti-social networking. In the EU the Digital Markets Act [1] forces some degree of interoperability between different IM systems and as it’s impossible to know who’s actually in the EU that will end up being world-wide.
In corporations document management often involves multiple ways of storing things, you have Google Docs, MS Office online, hosted Wikis like Confluence, and more. Large companies tend to use several such systems which means that people need to learn multiple systems to be able to work and they also need to know which systems are used by the various groups that they communicate with. Microsoft deserves some sort of award for the range of ways they have for managing documents, Sharepoint, OneDrive, Office Online, attachments to Teams rooms, and probably lots more.
During WW2 the predecessor to the CIA produced an excellent manual for simple sabotage [2]. If something like that was written today the section General Interference with Organisations and Production would surely have something about using as many incompatible programs and web sites as possible in the work flow. The proliferation of software required for work is a form of denial of service attack against corporations.
The efficiency of companies doesn’t really bother me. It sucks that companies are creating a demoralising workplace that is unpleasant for workers. But the upside is that the biggest companies are the ones doing the worst things and are also the most afflicted by these problems. It’s almost like the Bureau of Sabotage in some of Frank Herbert’s fiction [3].
The thing that concerns me is the effect of multiple standards on free software development. We have IRC the most traditional IM support system which is getting replaced by Matrix but we also have some projects using Telegram, and Jabber hasn’t gone away. I’m sure there are others too. There are also multiple options for version control (although github seems to dominate the market), forums, bug trackers, etc. Reporting bugs or getting support in free software often requires interacting with several of them. Developing free software usually involves dealing with the bug tracking and documentation systems of the distribution you use as well as the upstream developers of the software. If the problem you have is related to compatibility between two different pieces of free software then you can end up dealing with even more bug tracking systems.
There are real benefits to some of the newer programs to track bugs, write documentation, etc. There is also going to be a cost in changing which gives an incentive for the older projects to keep using what has worked well enough for them in the past,
How can we improve things? Use only the latest tools? Prioritise ease of use? Aim more for the entry level contributors?
It has been a very busy couple of weeks as we worked against some major transitions and a security fix that required a rebuild of the $world. I am happy to report that against all odds we have a beta release! You can read all about it here: https://kubuntu.org/news/kubuntu-24-04-beta-released/ Post beta freeze I have already begun pushing our fixes for known issues today. A big one being our new branding! Very exciting times in the Kubuntu world.
In the snap world I will be using my free time to start knocking out KDE applications ( not covered by the project ). I have also recruited some help, so you should start seeing these pop up in the edge channel very soon!
Now that we are nearing the release of Noble Numbat, my contract is coming to an end with Kubuntu. If you would like to see Plasma 6 in the next release and in a PPA for Noble, please consider donating to extend my contract at https://kubuntu.org/donate !
I tried to upgrade bullseye machien to bookworm, so I got the following error:
File “/usr/lib/python3/dist-packages/django/contrib/auth/mixins.py”, line 5, in from django.contrib.auth.views import redirect_to_login File “/usr/lib/python3/dist-packages/django/contrib/auth/views.py”, line 20, in from django.utils.http import ( ImportError: cannot import name ‘url_has_allowed_host_and_scheme’ from ‘django.utils.http’ (/usr/lib/python3/dist-packages/django/utils/http.py)
During handling of the above exception, another exception occurred:
It is similar to #1000810, but it is already closed.
My solution is:
apt remove mailman3-web
keep db and config files (do not purge)
apt autoremove
remove django related packages
apt install mailman3-web mailman3-full
I tried to send to the report, but it rerutns `550 Unknown or archived bug’ …
Adrian Bunk was responsible for updating gtkwave not only in LTS, but also in unstable, stable, and old-stable as well. This update involved an upload of a new upstream release of gtkwave to each target suite to address 82 separate CVEs. Guilhem Moulin prepared an update of libvirt which was particularly notable, as it fixed multiple vulnerabilities which would lead to denial of service or information disclosure.
In addition to the normal security updates, multiple LTS contributors worked at getting various packages updated in more recent Debian releases, including gross for bullseye/bookworm (by Adrian Bunk), imlib2 for bullseye, jetty9 and tomcat9/10 for bullseye/bookworm (by Markus Koschany), samba for bullseye, py7zr for bullseye (by Santiago Ruano Rincón), cacti for bullseye/bookwork (by Sylvain Beucler), and libmicrohttpd for bullseye (by Thorsten Alteholz). Additionally, Sylvain actively coordinated with cacti upstream concerning an incomplete fix for CVE-2024-29894.
P.S. We’ve completed over a year of writing these blogs. If you have any
suggestions on how to make them better or what you’d like us to cover, or any
other opinions/reviews you might have, et al, please let us know by dropping an
email to us. We’d be
happy to hear your thoughts. :)
SSO Authentication for jitsi.debian.social, by Stefano Rivera
Debian.social’s jitsi instance has been getting
some abuse by (non-Debian) people sharing sexually explicit content on the
service. After playing whack-a-mole with this for a month, and shutting the
instance off for another month, we opened it up again and the abuse immediately
re-started.
Stefano sat down and wrote an
SSO Implementation
that hooks into Jitsi’s existing JWT SSO support. This requires everyone using
jitsi.debian.social to have a Salsa account.
With only a little bit of effort, we could change this in future, to only
require an account to open a room, and allow guests to join the call.
/usr-move, by Helmut Grohne
The biggest task this month was sending mitigation patches for all of the
/usr-move issues arising from package renames due to the 2038 transition. As a
result, we can now say that every affected package in unstable can either be
converted with dh-sequence-movetousr or has an open bug report. The package
set relevant to debootstrap except for the set that has to be uploaded
concurrently has been moved to /usr and is awaiting migration. The move of
coreutils happened to affect piuparts which hard codes the location of
/bin/sync and received multiple updates as a result.
Miscellaneous contributions
Stefano Rivera uploaded a stable release update to python3.11 for bookworm,
fixing a use-after-free crash.
Stefano uploaded a new version of python-html2text, and updated
python3-defaults to build with it.
In support of Python 3.12, Stefano dropped distutils as a Build-Dependency
from a few packages, and uploaded a complex set of patches to python-mitogen.
Stefano landed some merge requests to clean up dead code in dh-python,
removed the flit plugin, and uploaded it.
Stefano uploaded new upstream versions of twisted, hatchling,
python-flexmock, python-authlib, python–mitogen, python-pipx, and xonsh.
Stefano requested removal of a few packages supporting the Opsis HDMI2USB
hardware that DebConf Video team used to use for HDMI capture, as they are
not being maintained upstream. They started to FTBFS, with recent sdcc
changes.
DebConf 24 is getting ready to open registration, Stefano spent some time
fixing bugs in the website, caused by infrastructure updates.
Stefano reviewed all the DebConf 23 travel reimbursements, filing requests
for more information from SPI where our records mismatched.
Roberto C. Sánchez worked on facilitating the transfer of upstream
maintenance responsibility for the dormant Shorewall project to a new team
led by the current maintainer of the Shorewall packages in Debian.
Colin Watson fixed build failures in celery-haystack-ng, db1-compat,
jsonpickle, libsdl-perl, kali, knews, openssh-ssh1,
python-json-log-formatter, python-typing-extensions, trn4, vigor, and
wcwidth. Some of these were related to the 64-bit time_t transition, since
that involved enabling -Werror=implicit-function-declaration.
Colin fixed an
off-by-one error in neovim,
which was already causing a build failure in Ubuntu and would eventually have
caused a build failure in Debian with stricter toolchain settings.
Colin added an sshd@.service template to
openssh to help newer systemd versions make containers and VMs SSH-accessible
over AF_VSOCK sockets.
Following the xz-utils backdoor, Colin
spent some time testing and discussing OpenSSH upstream’s proposed
inline systemd notification patch,
since the current implementation via libsystemd was part of the attack vector
used by that backdoor.
Utkarsh reviewed and sponsored some Go packages for Lena Voytek and Rajudev.
Utkarsh also helped Mitchell Dzurick with the adoption of pyparted package.
Helmut sent 10 patches for cross build failures.
Helmut partially fixed architecture cross bootstrap tooling to deal with
changes in linux-libc-dev and the recent gcc-for-host changes and also
fixed a 64bit-time_t FTBFS in libtextwrap.
Thorsten Alteholz uploaded several packages from debian-printing: cjet,
lprng, rlpr and epson-inkjet-printer-escpr were affected by the newly enabled
compiler switch -Werror=implicit-function-declaration. Besides fixing these
serious bugs, Thorsten also worked on other bugs and could fix one or the
other.
Carles updated simplemonitor and python-ring-doorbell packages with new
upstream versions.
Santiago also reviewed applications for the
improving Salsa CI in Debian
GSoC 2024 project. We received applications from four very talented
candidates. The selection process is currently ongoing. A huge thanks to all
of them!
As part of the DebConf 24 organization, Santiago has taken part in the
Content team discussions.
The diffoscope maintainers are pleased to announce the release of diffoscope
version 264. This version includes the following changes:
[ Chris Lamb ]
* Don't crash on invalid zipfiles, even if we encounter 'badness'
halfway through the file. (Re: #1068705)
[ FC (Fay) Stegerman ]
* Fix a crash when there are (invalid) duplicate entries in .zip files.
(Closes: #1068705)
* Add note when there are duplicate entries in ZIP files.
(Closes: reproducible-builds/diffoscope!140)
[ Vagrant Cascadian ]
* Add an external tool reference for GNU Guix for zipdetails.
I work from home these days, and my nearest office is over 100 miles away, 3 hours door to door if I travel by train (and, to be honest, probably not a lot faster given rush hour traffic if I drive). So I’m reliant on a functional internet connection in order to be able to work. I’m lucky to have access to Openreach FTTP, provided by Aquiss, but I worry about what happens if there’s a cable cut somewhere or some other long lasting problem. Worst case I could tether to my work phone, or try to find some local coworking space to use while things get sorted, but I felt like arranging a backup option was a wise move.
Step 1 turned out to be sorting out recursive DNS. It’s been many moons since I had to deal with running DNS in a production setting, and I’ve mostly done my best to avoid doing it at home too. dnsmasq has done a decent job at providing for my needs over the years, covering DHCP, DNS (+ tftp for my test device network). However I just let it forward to my ISP’s nameservers, which means if that link goes down it’ll no longer be able to resolve anything outside the house.
One option would have been to either point to a different recursive DNS server (Cloudfare’s 1.1.1.1 or Google’s Public DNS being the common choices), but I’ve no desire to share my lookup information with them. As another approach I could have done some sort of failover of resolv.conf when the primary network went down, but then I would have to get into moving files around based on networking status and that felt a bit clunky.
So I decided to finally setup a proper local recursive DNS server, which is something I’ve kinda meant to do for a while but never had sufficient reason to look into. Last time I did this I did it with BIND 9 but there are more options these days, and I decided to go with unbound, which is primarily focused on recursive DNS.
One extra wrinkle, pointed out by Lars, is that having dynamic name information from DHCP hosts is exceptionally convenient. I’ve kept dnsmasq as the local DHCP server, so I wanted to be able to forward local queries there.
I’m doing all of this on my RB5009, running Debian. Installing unbound was a simple matter of apt install unbound. I needed 2 pieces of configuration over the default, one to enable recursive serving for the house networks, and one to enable forwarding of queries for the local domain to dnsmasq. I originally had specified the wildcard address for listening, but this caused problems with the fact my router has many interfaces and would sometimes respond from a different address than the request had come in on.
server:
domain-insecure: "example.org"
do-not-query-localhost: no
forward-zone:
name: "example.org"
forward-addr: 127.0.0.1@5353
I then had to configure dnsmasq to not listen on port 53 (so unbound could), respond to requests on the loopback interface (I have dnsmasq restricted to only explicitly listed interfaces), and to hand out unbound as the appropriate nameserver in DHCP requests - once dnsmasq is not listening on port 53 it no longer does this by default.
With these minor changes in place I now have local recursive DNS being handled by unbound, without losing dynamic local DNS for DHCP hosts. As an added bonus I now get 10/10 on Test IPv6 - previously I was getting dinged on the ability for my DNS server to resolve purely IPv6 reachable addresses.
Welcome to the March 2024 report from the Reproducible Builds project! In our reports, we attempt to outline what we have been up to over the past month, as well as mentioning some of the important things happening more generally in software supply-chain security. As ever, if you are interested in contributing to the project, please visit our Contribute page on our website.
Arch Linux minimal container userland now 100% reproducible
In remarkable news, Reproducible builds developer kpcyrd reported that that the Arch Linux “minimal container userland” is now 100% reproducible after work by developers dvzv and Foxboron on the one remaining package. This represents a “real world”, widely-used Linux distribution being reproducible.
Their post, which kpcyrd suffixed with the question “now what?”, continues on to outline some potential next steps, including validating whether the container image itself could be reproduced bit-for-bit. The post, which was itself a followup for an Arch Linux update earlier in the month, generated a significant number of replies.
Validating Debian’s build infrastructure after the XZ backdoor
From our mailing list this month, Vagrant Cascadian wrote about being asked about trying to perform concrete reproducibility checks for recent Debian security updates, in an attempt to gain some confidence about Debian’s build infrastructure given that they performed builds in environments running the high-profile XZ vulnerability.
Vagrant reports (with some caveats):
So far, I have not found any reproducibility issues; everything I tested I was able to get to build bit-for-bit identical with what is in the
Debian archive.
That is to say, reproducibility testing permitted Vagrant and Debian to claim with some confidence that builds performed when this vulnerable version of XZ was installed were not interfered with.
Documented in more detail on Fedora’s website, the talk touched on topics such as the specifics of implementing reproducible builds in Fedora, the challenges encountered, the current status and what’s coming next. (YouTube video)
“Increasing Trust in the Open Source Supply Chain with Reproducible Builds and Functional Package Management”
Functional package managers (FPMs) and reproducible builds (R-B) are technologies and methodologies that are conceptually very different from the traditional software deployment model, and that have promising properties for software supply chain security. This thesis aims to evaluate the impact of FPMs and R-B on the security of the software supply chain and propose improvements to the FPM model to further improve trust in the open source supply chain. PDF
Julien’s paper poses a number of research questions on how the model of distributions such as GNU Guix and NixOS can “be leveraged to further improve the safety of the software supply chain”, etc.
Software and source code identification with GNU Guix and reproducible builds
In a long line of commendably detailed blog posts, Ludovic Courtès, Maxim Cournoyer, Jan Nieuwenhuizen and Simon Tournier have together published two interesting posts on the GNU Guix blog this month. In early March, Ludovic Courtès, Maxim Cournoyer, Jan Nieuwenhuizen and Simon Tournier wrote about software and source code identification and how that might be performed using Guix, rhetorically posing the questions: “What does it take to ‘identify software’? How can we tell what software is running on a machine to determine, for example, what security vulnerabilities might affect it?”
Later in the month, Ludovic Courtès wrote a solo post describing adventures on the quest for long-term reproducible deployment. Ludovic’s post touches on GNU Guix’s aim to support “time travel”, the ability to reliably (and reproducibly) revert to an earlier point in time, employing the iconic image of Harold Lloyd hanging off the clock in Safety Last! (1925) to poetically illustrate both the slapstick nature of current modern technology and the gymnastics required to navigate hazards of our own making.
Two new Rust-based tools for post-processing determinism
Zbigniew Jędrzejewski-Szmek announced add-determinism, a work-in-progress reimplementation of the Reproducible Builds project’s own strip-nondeterminism tool in the Rust programming language, intended to be used as a post-processor in RPM-based distributions such as Fedora
In Debian this month, since the testing framework no longer varies the build path, James Addison performed a bulk downgrade of the bug severity for issues filed with a level of normal to a new level of wishlist. In addition, 28 reviews of Debian packages were added, 38 were updated and 23 were removed this month adding to ever-growing knowledge about identified issues. As part of this effort, a number of issue types were updated, including Chris Lamb adding a new ocaml_include_directories toolchain issue […] and James Addison adding a new filesystem_order_in_java_jar_manifest_mf_include_resource issue […] and updating the random_uuid_in_notebooks_generated_by_nbsphinx to reference a relevant discussion thread […].
In addition, Roland Clobus posted his 24th status update of reproducible Debian ISO images. Roland highlights that the images for Debian unstable often cannot be generated due to changes in that distribution related to the 64-bit time_t transition.
Lastly, Bernhard M. Wiedemann posted another monthly update for his reproducibility work in openSUSE.
There were made a number of improvements to our website this month, including:
Pol Dellaiera noticed the frequent need to correctly cite the website itself in academic work. To facilitate easier citation across multiple formats, Pol contributed a Citation File Format (CIF) file. As a result, an export in BibTeX format is now available in the Academic Publications section. Pol encourages community contributions to further refine the CITATION.cff file. Pol also added an substantial new section to the “buy in” page documenting the role of Software Bill of Materials (SBOMs) and ephemeral development environments. […][…]
Bernhard M. Wiedemann added a new “commandments” page to the documentation […][…] and fixed some incorrect YAML elsewhere on the site […].
Chris Lamb add three recent academic papers to the publications page of the website. […]
Mattia Rizzolo and Holger Levsen collaborated to add Infomaniak as a sponsor of amd64 virtual machines. […][…][…]
Roland Clobus updated the “stable outputs” page, dropping version numbers from Python documentation pages […] and noting that Python’s set data structure is also affected by the PYTHONHASHSEED functionality. […]
Delta chat clients now reproducible
Delta Chat, an open source messaging application that can work over email, announced this month that the Rust-based core library underlying Delta chat application is now reproducible.
diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made a number of changes such as uploading versions 259, 260 and 261 to Debian and made the following additional changes:
New features:
Add support for the zipdetails tool from the Perl distribution. Thanks to Fay Stegerman and Larry Doolittle et al. for the pointer and thread about this tool. […]
Bug fixes:
Don’t identify Redis database dumps as GNU R database files based simply on their filename. […]
Add a missing call to File.recognizes so we actually perform the filename check for GNU R data files. […]
Don’t crash if we encounter an .rdb file without an equivalent .rdx file. (#1066991)
Correctly check for 7z being available—and not lz4—when testing 7z. […]
Prevent a traceback when comparing a contentful .pyc file with an empty one. […]
Testsuite improvements:
Fix .epub tests after supporting the new zipdetails tool. […]
Don’t use parenthesis within test “skipping…” messages, as PyTest adds its own parenthesis. […]
Factor out Python version checking in test_zip.py. […]
Skip some Zip-related tests under Python 3.10.14, as a potential regression may have been backported to the 3.10.x series. […]
Actually test 7z support in the test_7z set of tests, not the lz4 functionality. (Closes: reproducible-builds/diffoscope#359). […]
Chris Lamb also updated the trydiffoscope command line client, dropping a build-dependency on the deprecated python3-distutils package to fix Debian bug #1065988 […], taking a moment to also refresh the packaging to the latest Debian standards […]. Finally, Vagrant Cascadian submitted an update for diffoscope version 260 in GNU Guix. […]
Upstream patches
This month, we wrote a large number of patches, including:
Similarly, Jean-Pierre De Jesus DIAZ employed reproducible builds techniques in order to test a proposed refactor of the ath9k-htc-firmware package. As the change produced bit-for-bit identical binaries to the previously shipped pre-built binaries:
I don’t have the hardware to test this firmware, but the build produces the same hashes for the firmware so it’s safe to say that the firmware should keep working.
Reproducibility testing framework
The Reproducible Builds project operates a comprehensive testing framework running primarily at tests.reproducible-builds.org in order to check packages and other artifacts for reproducibility.
In March, an enormous number of changes were made by Holger Levsen:
Initial work to clean up a messy NetBSD-related script. […][…]
Roland Clobus:
Show the installer log if the installer fails to build. […]
Avoid the minus character (i.e. -) in a variable in order to allow for tags in openQA. […]
Update the schedule of Debian live image builds. […]
Vagrant Cascadian:
Maintenance on the virt* nodes is completed so bring them back online. […]
Use the fully qualified domain name in configuration. […]
Node maintenance was also performed by Holger Levsen, Mattia Rizzolo […][…] and Vagrant Cascadian […][…][…][…]
If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:
Getting the Belgian eID to work on Linux
systems should be fairly easy, although some people do struggle with it.
For that reason, there is a lot of third-party documentation out there
in the form of blog posts, wiki pages, and other kinds of things.
Unfortunately, some of this documentation is simply wrong. Written by
people who played around with things until it kind of worked, sometimes
you get a situation where something that used to work in the past (but
wasn't really necessary) now stopped working, but it's still added to
a number of locations as though it were the gospel.
And then people follow these instructions and now things don't work
anymore.
OpenSC is an open source smartcard library that has support for a
pretty
large
number of smartcards, amongst which the Belgian eID. It provides a
PKCS#11 module as well as a
number of supporting tools.
For those not in the know, PKCS#11 is a standardized C API for
offloading cryptographic operations. It is an API that can be used when
talking to a hardware cryptographic module, in order to make that module
perform some actions, and it is especially popular in the open source
world, with support in
NSS,
amongst others. This library is written and maintained by mozilla, and
is a low-level cryptographic library that is used by Firefox (on all
platforms it supports) as well as by Google Chrome and other browsers
based on that (but only on Linux, and as I understand it, only for
linking with smartcards; their BoringSSL library is used for other
things).
The official eID software that we ship through
eid.belgium.be,
also known as "BeID", provides a PKCS#11 module for the Belgian eID, as
well as a number of support tools to make interacting with the card
easier, such as the "eID viewer", which provides the ability to read
data from the card, and validate their signatures. While the very first
public version of this eID PKCS#11 module was originally based on
OpenSC, it has since been reimplemented as a PKCS#11 module in its own
right, with no lineage to OpenSC whatsoever anymore.
About five years ago, the Belgian eID card was renewed. At the time, a
new physical appearance was the most obvious difference with the old
card, but there were also some technical, on-chip, differences that are
not so apparent. The most important one here, although it is not the
only one, is the fact that newer eID cards now use a NIST
P-384 elliptic curve-based private
keys, rather than the RSA-based
ones that were used in the past. This change required some changes to
any PKCS#11 module that supports the eID; both the BeID one, as well as
the OpenSC card-belpic driver that is written in support of the Belgian
eID.
Obviously, the required changes were implemented for the BeID module;
however, the OpenSC card-belpic driver was not updated. While I did do
some preliminary work on the required changes, I was unable to get it to
work, and eventually other things took up my time so I never finished
the implementation. If someone would like to finish the work that I
started, the preliminal patch that I
wrote
could be a good start -- but like I said, it doesn't yet work. Also,
you'll probably be interested in the official
documentation
of the eID card.
Unfortunately, in the mean time someone added the Applet 1.8 ATR to the
card-belpic.c file, without also implementing the required changes to
the driver so that the PKCS#11 driver actually supports the eID card.
The result of this is that if you have OpenSC installed in NSS for
either Firefox or any Chromium-based browser, and it gets picked up
before the BeID PKCS#11 module, then NSS will stop looking and pass all
crypto operations to the OpenSC PKCS#11 module rather than to the
official eID PKCS#11 module, and things will not work at all, causing a
lot of confusion.
I have therefore taken the following two steps:
The official eID packages now
conflict
with the OpenSC PKCS#11 module. Specifically only the PKCS#11 module,
not the rest of OpenSC, so you can theoretically still use its tools.
This means that once we release this new version of the eID software,
when you do an upgrade and you have OpenSC installed, it will remove
the PKCS#11 module and anything that depends on it. This is normal
and expected.
I have filed a pull
request against OpenSC
that removes the Applet 1.8 ATR from the driver, so that OpenSC will
stop claiming that it supports the 1.8 applet.
When the pull request is accepted, we will update the official eID
software to make the conflict versioned, so that as soon as it works
again you will again be able to install the OpenSC and BeID packages at
the same time.
In the mean time, if you have the OpenSC PKCS#11 module installed on
your system, and your eID authentication does not work, try removing it.
For a while ceb and I have wanted to contribute directly to green energy provision. This isn’t really possible in the mainstream consumer electricy market.
Mainstream electricity suppliers’ “100% green energy” tariffs are pure greenwashing. In a capitalist boondoogle, they basically “divvy up” the electricity so that customers on the (typically more expensive) “green” tariff “get” the green electricity, and the other customers “get” whatever is left. (Of course the electricity is actually all mixed up by the National Grid.) There are fewer people signed up for these tariffs than there is green power generated, so this basically means signing up for a “green” tariff has no effect whatsoever, other than giving evil people more money.
About a year ago we heard about Ripple. The structure is a little complicated, but the basic upshot is:
Ripple promote and manage renewable energy schemes. The schemes themselves are each an individual company; the company is largely owned by a co-operative. The co-op is owned by consumers of electricity in the UK., To stop the co-operative being an purely financial investment scheme, shares ownership is limited according to your electricity usage. The electricity is be sold on the open market, and the profits are used to offset members’ electricity bills. (One gotcha from all of this is that for this to work your electricity billing provider has to be signed up with Ripple, but ours, Octopus, is.)
It seemed to us that this was a way for us to directly cause (and pay for!) the actual generation of green electricity.
So, we bought shares in one these co-operatives: we are co-owners of the Derril Water Solar Farm. We signed up for the maximum: funding generating capacity corresponding to 120% of our current electricity usage. We paid a little over £5000 for our shares.
The UK has a renewable energy subsidy scheme, which goes by the name of Contracts for Difference. The idea is that a renewable energy generation company bids in advance, saying that they’ll sell their electricity at Y price, for the duration of the contract (15 years in the current round). The lowest bids win. All the electricity from the participating infrastructure is sold on the open market, but if the market price is low the government makes up the difference, and if the price is high, the government takes the winnings.
This is supposedly good for giving a stable investment environment, since the price the developer is going to get now doesn’t depends on the electricity market over the next 15 years. The CfD system is supposed to encourage development, so you can only apply before you’ve commissioned your generation infrastructure.
Ripple recently invited us to agree that the Derril Water co-operative should bid in the current round of CfDs.
If this goes ahead, and we are one of the auction’s winners, the result would be that, instead of selling our electricity at the market price, we’ll sell it at the fixed CfD price.
This would mean that our return on our investment (which show up as savings on our electricity bills) would be decoupled from market electricity prices, and be much more predictable.
They can’t tell us the price they’d want to bid at, and future electricity prices are rather hard to predict, but it’s clear from the accompanying projections that they think we’d be better off on average with a CfD.
The documentation is very full of financial projections and graphs; other factors aren’t really discussed in any detail.
The rules of the co-op didn’t require them to hold a vote, but very sensibly, for such a fundamental change in the model, they decided to treat it roughly the same way as for a rules change: they’re hoping to get 75% Yes votes.
The reason we’re in this co-op at all is because we want to directly fund renewable electricity.
Participating in the CfD auction would involve us competing with capitalist energy companies for government subsidies. Subsidies which are supposed to encourage the provision of green electricity.
It seems to us that participating in this auction would remove most of the difference between what we hoped to do by investing in Derril Water, and just participating in the normal consumer electricity market.
In particular, if we do win in the auction, that’s probably directly removing the funding and investment support model for other, market-investor-funded, projects.
In other words, our buying into Derril Water ceases to be an additional green energy project, changing (in its minor way) the UK’s electricity mix. It becomes a financial transaction much more tenously connected (if connected at all) to helping mitigate the climate emergency.
Now that we are back from our six month period in Argentina, we
decided to adopt a kitten, to bring more diversity into our
lives. Perhaps this little girl will teach us to think outside the
box!
Yesterday we witnessed a solar eclipse — Mexico City was not in the
totality range (we reached ~80%), but it was a great experience to go
with the kids. A couple dozen thousand people gathered for a massive
picnic in las islas, the main area inside our university campus.
Afterwards, we went briefly back home, then crossed the city to fetch
the little kitten. Of course, the kids were unanimous: Her name is
Eclipse.
Those of you who haven’t been in IT for far, far too long might not know that next month will be the 16th(!) anniversary of the disclosure of what was, at the time, a fairly earth-shattering revelation: that for about 18 months, the Debian OpenSSL package was generating entirely predictable private keys.
The recent xz-stential threat (thanks to @nixCraft for making me aware of that one), has got me thinking about my own serendipitous interaction with a major vulnerability.
Given that the statute of limitations has (probably) run out, I thought I’d share it as a tale of how “huh, that’s weird” can be a powerful threat-hunting tool – but only if you’ve got the time to keep pulling at the thread.
Prelude to an Adventure
Our story begins back in March 2008.
I was working at Engine Yard (EY), a now largely-forgotten Rails-focused hosting company, which pioneered several advances in Rails application deployment.
Probably EY’s greatest claim to lasting fame is that they helped launch a little code hosting platform you might have heard of, by providing them free infrastructure when they were little more than a glimmer in the Internet’s eye.
I am, of course, talking about everyone’s favourite Microsoft product: GitHub.
Since GitHub was in the right place, at the right time, with a compelling product offering, they quickly started to gain traction, and grow their userbase.
With growth comes challenges, amongst them the one we’re focusing on today: SSH login times.
Then, as now, GitHub provided SSH access to the git repos they hosted, by SSHing to git@github.com with publickey authentication.
They were using the standard way that everyone manages SSH keys: the ~/.ssh/authorized_keys file, and that became a problem as the number of keys started to grow.
The way that SSH uses this file is that, when a user connects and asks for publickey authentication, SSH opens the ~/.ssh/authorized_keys file and scans all of the keys listed in it, looking for a key which matches the key that the user presented.
This linear search is normally not a huge problem, because nobody in their right mind puts more than a few keys in their ~/.ssh/authorized_keys, right?
Of course, as a popular, rapidly-growing service, GitHub was gaining users at a fair clip, to the point that the one big file that stored all the SSH keys was starting to visibly impact SSH login times.
This problem was also not going to get any better by itself.
Something Had To Be Done.
EY management was keen on making sure GitHub ran well, and so despite it not really being a hosting problem, they were willing to help fix this problem.
For some reason, the late, great, Ezra Zygmuntowitz pointed GitHub in my direction, and let me take the time to really get into the problem with the GitHub team.
After examining a variety of different possible solutions, we came to the conclusion that the least-worst option was to patch OpenSSH to lookup keys in a MySQL database, indexed on the key fingerprint.
We didn’t take this decision on a whim – it wasn’t a case of “yeah, sure, let’s just hack around with OpenSSH, what could possibly go wrong?”.
We knew it was potentially catastrophic if things went sideways, so you can imagine how much worse the other options available were.
Ensuring that this wouldn’t compromise security was a lot of the effort that went into the change.
In the end, though, we rolled it out in early April, and lo! SSH logins were fast, and we were pretty sure we wouldn’t have to worry about this problem for a long time to come.
Normally, you’d think “patching OpenSSH to make mass SSH logins super fast” would be a good story on its own.
But no, this is just the opening scene.
Chekov’s Gun Makes its Appearance
Fast forward a little under a month, to the first few days of May 2008.
I get a message from one of the GitHub team, saying that somehow users were able to access other users’ repos over SSH.
Naturally, as we’d recently rolled out the OpenSSH patch, which touched this very thing, the code I’d written was suspect number one, so I was called in to help.
Eventually, after more than a little debugging, we discovered that, somehow, there were two users with keys that had the same key fingerprint.
This absolutely shouldn’t happen – it’s a bit like winning the lottery twice in a row1 – unless the users had somehow shared their keys with each other, of course.
Still, it was worth investigating, just in case it was a web application bug, so the GitHub team reached out to the users impacted, to try and figure out what was going on.
The users professed no knowledge of each other, neither admitted to publicising their key, and couldn’t offer any explanation as to how the other person could possibly have gotten their key.
Then things went from “weird” to “what the…?”.
Because another pair of users showed up, sharing a key fingerprint – but it was a different shared key fingerprint.
The odds now have gone from “winning the lottery multiple times in a row” to as close to “this literally cannot happen” as makes no difference.
Once we were really, really confident that the OpenSSH patch wasn’t the cause of the problem, my involvement in the problem basically ended.
I wasn’t a GitHub employee, and EY had plenty of other customers who needed my help, so I wasn’t able to stay deeply involved in the on-going investigation of The Mystery of the Duplicate Keys.
However, the GitHub team did keep talking to the users involved, and managed to determine the only apparent common factor was that all the users claimed to be using Debian or Ubuntu systems, which was where their SSH keys would have been generated.
That was as far as the investigation had really gotten, when along came May 13, 2008.
Chekov’s Gun Goes Off
With the publication of DSA-1571-1, everything suddenly became clear.
Through a well-meaning but ultimately disasterous cleanup of OpenSSL’s randomness generation code, the Debian maintainer had inadvertently reduced the number of possible keys that could be generated by a given user from “bazillions” to a little over 32,000.
With so many people signing up to GitHub – some of them no doubt following best practice and freshly generating a separate key – it’s unsurprising that some collisions occurred.
You can imagine the sense of “oooooooh, so that’s what’s going on!” that rippled out once the issue was understood.
I was mostly glad that we had conclusive evidence that my OpenSSH patch wasn’t at fault, little knowing how much more contact I was to have with Debian weak keys in the future, running a huge store of known-compromised keys and using them to find misbehaving Certificate Authorities, amongst other things.
Lessons Learned
While I’ve not found a description of exactly when and how Luciano Bello discovered the vulnerability that became CVE-2008-0166, I presume he first came across it some time before it was disclosed – likely before GitHub tripped over it.
The stable Debian release that included the vulnerable code had been released a year earlier, so there was plenty of time for Luciano to have discovered key collisions and go “hmm, I wonder what’s going on here?”, then keep digging until the solution presented itself.
The thought “hmm, that’s odd”, followed by intense investigation, leading to the discovery of a major flaw is also what ultimately brought down the recent XZ backdoor.
The critical part of that sequence is the ability to do that intense investigation, though.
When I reflect on my brush with the Debian weak keys vulnerability, what sticks out to me is the fact that I didn’t do the deep investigation.
I wonder if Luciano hadn’t found it, how long it might have been before it was found.
The GitHub team would have continued investigating, presumably, and perhaps they (or I) would have eventually dug deep enough to find it.
But we were all super busy – myself, working support tickets at EY, and GitHub feverishly building features and fighting the fires in their rapidly-growing service.
As it was, Luciano was able to take the time to dig in and find out what was happening, but just like the XZ backdoor, I feel like we, as an industry, got a bit lucky that someone with the skills, time, and energy was on hand at the right time to make a huge difference.
It’s a luxury to be able to take the time to really dig into a problem, and it’s a luxury that most of us rarely have.
Perhaps an understated takeaway is that somehow we all need to wrestle back some time to follow our hunches and really dig into the things that make us go “hmm…”.
the odds are actually probably more like winning the lottery about twenty times in a row.
The numbers involved are staggeringly huge, so it’s easiest to just approximate it as “really, really unlikely”. ↩
Python includes some helping support for classes that are designed to just hold some data and not much more: Data Classes.
It uses plain Python type definitions to specify what you can have and some further information for every field.
This will then generate you some useful methods, like __init__ and __repr__, but on request also more.
But given that those type definitions are available to other code, a lot more can be done.
There exists several separate packages to work on data classes.
For example you can have data validation from JSON with dacite.
But Debian likes a pretty strange format usually called Deb822, which is in fact derived from the RFC 822 format of e-mail messages.
Those files includes single messages with a well known format.
So I'd like to introduce some Deb822 format support for Python Data Classes.
For now the code resides in the Debian Cloud tool.
Usage
Setup
It uses the standard data classes support and several helper functions.
Also you need to enable support for postponed evaluation of annotations.
This month I accepted 147 and rejected 12 packages. The overall number of packages that got accepted was 151.
If you file an RM bug, please do check whether there are reverse dependencies as well and file RM bugs for them. It is annoying and time-consuming when I have to do the moreinfo dance.
Debian LTS
This was my hundred-seventeenth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian.
During my allocated time I uploaded:
[DLA 3770-1] libnet-cidr-lite-perl security update for one CVE to fix IP parsing and ACLs based on the result
Unfortunately XZ happened at the end of month and I had to delay/intentionally delayed other uploads: they will appear as DLA-3781-1 and DLA-3784-1 in April
I also continued to work on qtbase-opensource-src and last but not least did a week of FD.
Debian ELTS
This month was the sixty-eighth ELTS month. During my allocated time I uploaded:
[ELA-1062-1]libnet-cidr-lite-perl security update for one CVE to improve parsing of IP addresses in Jessie and Stretch
Due to XZ I also delayed the uploads here. They will appear as ELA-1069-1 and DLA-1070-1 in April
I also continued on an update for qtbase-opensource-src in Stretch (and LTS and other releases as well) and did a week of FD.
Debian Printing
This month I uploaded new upstream or bugfix versions of:
The CNN story says Facebook apologized and said it was a mistake and was fixing it.
Color me skeptical, because today I saw this:
Yes, that’s right: today, April 6, I get a notification that they removed a post from August 12. The notification was dated April 4, but only showed up for me today.
I wonder why my post from August 12 was fine for nearly 8 months, and then all of a sudden, when the same website runs an article critical of Facebook, my 8-month-old post is a problem. Hmm.
Riiiiiight. Cybersecurity.
This isn’t even the first time they’ve done this to me.
That this same pattern has played out a second time — again with something that is a very slight challenege to Facebook — seems to validate my conclusion. Facebook lets all sort of hateful garbage infest their site, but anything about climate change — or their own censorship — gets removed, and this pattern persists for years.
There’s a reason I prefer Mastodon these days. You can find me there as @jgoerzen@floss.social.
So. I’ve written this blog post. And then I’m going to post it to Facebook. Let’s see if they try to censor me for a third time. Bring it, Facebook.
Trying to explain analogue clock. It's hard to
explain. Tried adding some things for affordance, and it is
still not enough. So it's not obvious which arm is the hour
and which arm is the minute.
analog clock
Armadillo is a powerful
and expressive C++ template library for linear algebra and scientific
computing. It aims towards a good balance between speed and ease of use,
has a syntax deliberately close to Matlab, and is useful for algorithm
development directly in C++, or quick conversion of research code into
production environments. RcppArmadillo
integrates this library with the R environment and language–and is
widely used by (currently) 1136 other packages on CRAN, downloaded 33.5 million
times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint
/ vignette) by Conrad and myself has been cited 578 times according
to Google Scholar.
This release brings a new upstream bugfix release Armadillo 12.8.2
prepared by Conrad two days
ago. It took the usual day to noodle over 1100+ reverse dependencies and
ensure two failures were independent of the upgrade (i.e., “no
change to worse” in CRAN
parlance). It took CRAN another
because we hit a random network outage for (spurious) NOTE on a remote
URL, and were then caught in the shrapnel from another large package
ecosystem update spuriously pointing some build failures that were due
to a missing rebuild to us. All good, as human intervention comes to the
rescue.
The set of changes since the last CRAN release follows.
Changes
in RcppArmadillo version 0.12.8.2.0 (2024-04-02)
Upgraded to Armadillo release 12.8.2 (Cortisol Injector)
Workaround for FFTW3 header clash
Workaround in testing framework for issue under macOS
The Debian Project Developers will shortly vote for a new Debian Project Leader
known as the DPL.
The DPL is the official representative of representative of The Debian Project tasked with managing the overall project, its vision, direction, and finances.
The DPL is also responsible for the selection of Delegates, defining areas of
responsibility within the project, the coordination of Developers, and making
decisions required for the project.
Our outgoing and present DPL Jonathan Carter served 4 terms, from 2020
through 2024. Jonathan shared his last Bits from the DPL
post to Debian recently and his hopes for the future of Debian.
Recently, we sat with the two present candidates for the DPL position asking
questions to find out who they really are in a series of interviews about
their platforms, visions for Debian, lives, and even their favorite text
editors. The interviews were conducted by disaster2life (Yashraj Moghe) and
made available from video and audio transcriptions:
Editors' note:
This is our official return to Debian interviews, readers should stay tuned
for more upcoming interviews with Developers and other important figures in
Debian as part of our "Meet your Debian Developer" series. We used the
following tools and services: Turboscribe.ai for the
transcription from the audio and video files, IRC: Oftc.net for
communication, Jitsi meet for interviews, and
Open Broadcaster Software (OBS) for editing and
video. While we encountered many technical difficulties in the return to this
process, we are still able and proud to present the transcripts of the
interviews edited only in a few areas for readability.
Hi Sruthi, so for the first question, who are you and could you tell us a little
bit about yourself?
[Sruthi]:
I usually talk about me whenever I am talking about answering the question who
am I, I usually say like I am a librarian turned free software enthusiast and
a Debian Developer. So I had no technical background and I learned, I was
introduced to free software through my husband and then I learned Debian
packaging, and eventually I became a Debian Developer. So I always give my
example to people who say I am not technically inclined, I don't have technical
background so I can't contribute to free software.
So yeah, that's what I refer to myself.
For the next question, could you tell me what do you do in Debian, and could
you mention your story up until here today?
[Sruthi]:
Okay, so let me start from my initial days in Debian. I started contributing to
Debian, my first contribution was a Tibetan font. We went to a Tibetan place
and they were saying they didn't have a font in Linux.
So that's how I started contributing. Then I moved on to Ruby packages, then I
have some JavaScript and Go packages, all dependencies of GitLab. So I was
involved with maintaining GitLab for some time, now I'm not very active there.
But yeah, so GitLab was the main package I was contributing to since I
contributed since 2016 to maybe like 2020 or something. Later I have come
[over to] packaging. Now I am part of some of the teams, delegated teams, like
community team and outreach team, as well as the Debconf committee. And the
biggest, I think, my activity in Debian, I would say is organizing Debconf
2023. So it was a great experience and yeah, so that's my story in Debian.
So what are three key terms about you and your candidacy?
[Sruthi]:
Okay, let me first think about it. For candidacy, I can start with diversity is
one point I started expressing from the first time I contested for DPL. But to
be honest, that's the main point I want to bring.
[Yashraj]:
So for diversity, if you could break down your thoughts on diversity and make
them, [about] your three points including diversity.
[Sruthi]:
So in addition to, eventually when starting it was just diversity. Now I have
like a bit more ideas, like community, like I want to be a leader for the
Debian community. More than, I don't know, maybe people may not agree, but I
would say I want to be a leader of Debian community rather than a Debian operating system.
I connect to community more and third point I would say.
The term of a DPL lasts for an year. So what do you think during, what would you
try to do during that, that you can't do from your position now?
[Sruthi]:
Okay. So I, like, I am very happy with the structure of Debian and how things
work in Debian. Like you can do almost a lot of things, like almost all things
without being a DPL.
Whatever change you want to bring about or whatever you want to do, you can do
without being a DPL. Anyone, like every DD has the same rights. Only things I
feel [the] DPL has hold on are mainly the budget or the funding part, which like,
that's where they do the decision making part.
And then comes like, and one advantage of DPL driving some idea is that somehow
people tend to listen to that with more, like, tend to give more attention to
what DPL is saying rather than a normal DD. So I wanted to, like, I have
answered some of the questions on how to, how I plan to do the financial
budgeting part, how I want to handle, like, and the other thing is using the
extra attention that I get as a DPL, I would like to obviously start with the
diversity aspect in Debian. And yeah, like, I, what I want to do is not, like,
be a leader and say, like, take Debian to one direction where I want to go, but
I would rather take suggestions and inputs from the whole community and go about
with that.
So yes, that's what I would say.
And taking a less serious question now, what is your preferred text editor?
[Sruthi]:
Vim.
[Yashraj]:
Vim, wholeheartedly team Vim?
[Sruthi]:
Yes.
[Yashraj]:
Great. Well, this was made in Vim, all the text for this.
[Sruthi]:
So, like, since you mentioned extra data, I'll give my example, like, it's just
a fun note, when I started contributing to Debian, as I mentioned, I didn't have
any knowledge about free software, like Debian, and I was not used to even
using Linux. So, and I didn't have experience with these text editors. So, when
I started contributing, I used to do the editing part using gedit.
So, that's how I started. Eventually, I moved to Nano, and once I reached Vim,
I didn't move on.
Team Vim. Next question. What, what do you think is the importance of
the Debian project in the world today? And where would you like to see it in
10 years, like 10 years into the future?
[Sruthi]:
Okay. So, Debian, as we all know, is referred to as the universal operating
system without, like, it is said for a reason. We have hundreds and hundreds of
operating systems, like Linux, distributions based on Debian.
So, I believe Debian, like even now, Debian has good influence on the, at least
on the Linux or Linux ecosystem. So, what we implement in Debian has, like, is
going to affect quite a lot of, like, a very good percentage of people using
Linux. So, yes.
So, I think Debian is one of the leading Linux distributions. And I think in
10 years, we should be able to reach a position, like, where we are not, like,
even now, like, even these many years after having Linux, we face a lot of
problems in newer and newer hardware coming up and installing on them is a big
problem. Like, firmwares and all those things are getting more and more
complicated.
Like, it should be getting simpler, but it's getting more and more complicated.
So, I, one thing I would imagine, like, I don't know if we will ever reach
there, but I would imagine that eventually with the Debian, we should be able
to have some, at least a few of the hardware developers or hardware producers
have Debian pre-installed and those kind of things. Like, not, like, become,
I'm not saying it's all, it's also available right now.
What I'm saying is that it becomes prominent enough to be opted as, like, default
distro.
What part of Debian has made you And what part of the project has kept you going
all through these years?
[Sruthi]:
Okay. So, I started to contribute in 2016, and I was part of the team doing
GitLab packaging, and we did have a lot of training workshops and those kind of
things within India. And I was, like, I had interacted with some of the Indian
DDs, but I never got, like, even through chat or mail.
I didn't have a lot of interaction with the rest of the world, DDs. And the 2019
Debconf changed my whole perspective about Debian. Before that, I wasn't, like,
even, I was interested in free software.
I was doing the technical stuff and all. But after DebConf, my whole idea has
been, like, my focus changed to the community. Debian community is a very
welcoming, very interesting community to be with.
And so, I believe that, like, 2019 DebConf was a for me. And that kept, from
2019, my focus has been to how to support, like, how, I moved to the community
part of Debian from there. Then in 2020 I became part of the community team,
and, like, I started being part of other teams.
So, these, I would say, the Debian community is the one, like, aspect of Debian
that keeps me whole, keeps me held on to the Debian ecosystem as a whole.
Continuing to speak about Debian, what do you think, what is the first thing that
comes to your mind when you think of Debian, like, the word, the community,
what's the first thing?
[Sruthi]:
I think I may sound like a broken record or something.
[Yashraj]:
No, no.
[Sruthi]:
Again, I would say the Debian community, like, it's the people who makes
Debian, that makes Debian special.
Like, apart from that, if I say, I would say I'm very, like, one part of Debian
that makes me very happy is the, how the governing system of Debian works, the
Debian constitution and all those things, like, it's a very unique thing for
Debian. And, and it's like, when people say you can't work without a proper,
like, establishment or even somebody deciding everything for you, it's
difficult. When people say, like, we have been, Debian has been proving
it for quite a long time now, that it's possible.
So, so that's one thing I believe, like, that's one unique point. And I am very
proud about that.
What areas do you think Debian is failing in, how can it (that standing) be improved?
[Sruthi]:
So, I think where Debian is failing now is getting new people into Debian.
Like, I don't remember, like, exactly the answer. But I remember hearing
someone mention, like, the average age of a Debian Developer is, like, above 40
or 45 or something, like, exact age, I don't remember.
But it's like, Debian is getting old. Like, the people in Debian are getting old
and we are not getting enough of new people into Debian. And that's very
important to have people, like, new people coming up.
Otherwise, eventually, like, after a few years, nobody, like, we won't have
enough people to take the project forward. So, yeah, I believe that is where we
need to work on. We are doing some efforts, like, being part of GSOC or
outreachy and having maybe other events, like, local events. Like, we used to
have a lot of Debian packaging workshops in India. And those kind of, I think,
in Brazil and all, they all have, like, local communities are doing. But we are
not very successful in retaining the people who maybe come and try out things.
But we are not very good at retaining the people, like, retaining people who
come. So, we need to work on those things. Right now, I don't have a solid
answer for that.
But one thing, like, I was thinking about is, like, having a Debian specific
outreach project, wherein the focus will be about the Debian, like, starting
will be more on, like, usually what happens in GSOC and outreach is that people
come, have the, do the contributions, and they go back. Like, they don't have
that connection with the Debian, like, Debian community or Debian project. So,
what I envision with these, the Debian outreach, the Debian specific outreach
is that we have some part of the internship, like, even before starting the
internship, we have some sessions and, like, with the people in Debian having,
like, getting them introduced to the Debian philosophy and Debian community and
Debian, how Debian works.
And those things, we focus on that. And then we move on to the technical
internship parts. So, I believe this could do some good in having, like, when
you have people you can connect to, you tend to stay back in a project mode.
When you feel something more than, like, right now, we have so many technical
stuff to do, like, the choice for a college student is endless. So, if they
want, if they stay back for something, like, maybe for Debian, I would say, we
need to have them connected to the Debian project before we go into technical
parts. Like, technical parts, like, there are other things as well, where they
can go and do the technical part, but, like, they can come here, like, yeah.
So, that's what I was saying. Focused outreach projects is one thing. That's
just one.
That's not enough. We need more of, like, more ideas to have more new people
come up. And I'm very happy with, like, the DebConf thing. We tend to get more
and more people from the places where we have a DebConf. Brazil is an example.
After the Debconf, they have quite a good improvement on Debian contributors.
And I think in India also, it did give a good result. Like, we have more people
contributing and staying back and those things. So, yeah.
So, these were the things I would say, like, we can do to improve.
For the final question, what field in free software do you, what
field in free software generally do you think requires the most work to be
put into it? What do you think is Debian's part in that field?
[Sruthi]:
Okay. Like, right now, what comes to my mind is the free software licenses
parts. Like, we have a lot of free software licenses, and there are non-free
software licenses.
But currently, I feel free software is having a big problem in enforcing these
licenses. Like, there are, there may be big corporations or like some people who
take up the whole, the code and may not follow the whole, for example, the GPL
licenses. Like, we don't know how much of those, how much of the free softwares
are used in the bigger things.
Yeah, I agree. There are a lot of corporations who are afraid to touch free
software. But there would be good amount of free software, free work that
converts into property, things violating the free software licenses and those
things.
And we do not have the kind of like, we have SFLC, SFC, etc. But still, we do
not have the ability to go behind and trace and implement the licenses. So,
enforce those licenses and bring people who are violating the licenses forward
and those kind of things is challenging because one thing is it takes time,
like, and most importantly, money is required for the legal stuff.
And not always people who like people who make small software, or maybe big,
but they may not have the kind of time and money to have these things enforced.
So, that's a big challenge free software is facing, especially in our current
scenario. I feel we are having those, like, we need to find ways how we can get
it sorted.
I don't have an answer right now what to do. But this is a challenge I felt
like and Debian's part in that. Yeah, as I said, I don't have a solution for
that.
But the Debian, so DFSG and Debian sticking on to the free software licenses is
a good support, I think.
So, that was the final question, Do you have anything else you want to mention
for anyone watching this?
[Sruthi]:
Not really, like, I am happy, like, I think I was able to answer the questions.
And yeah, I would say who is watching. I won't say like, I'm the best DPL
candidate, you can't have a better one or something.
I stand for a reason. And if you believe in that, or the Debian community and
Debian diversity, and those kinds of things, if you believe it, I hope you
would be interested, like, you would want to vote for me. That's it.
Like, I'm not, I'll make it very clear. I'm not doing a technical leadership
part here. So, those, I can't convince people who want technical leadership to
vote for me.
But I would say people who connect with me, I hope they vote for me.
The Debian Project Developers will shortly vote for a new Debian Project Leader
known as the DPL.
The Project Leader is the official representative of The Debian Project tasked with
managing the overall project, its vision, direction, and finances.
The DPL is also responsible for the selection of Delegates, defining areas of
responsibility within the project, the coordination of Developers, and making
decisions required for the project.
Our outgoing and present DPL Jonathan Carter served 4 terms, from 2020
through 2024. Jonathan shared his last Bits from the DPL
post to Debian recently and his hopes for the future of Debian.
Recently, we sat with the two present candidates for the DPL position asking
questions to find out who they really are in a series of interviews about
their platforms, visions for Debian, lives, and even their favorite text
editors. The interviews were conducted by disaster2life (Yashraj Moghe) and
made available from video and audio transcriptions:
Editors' note:
This is our official return to Debian interviews, readers should stay tuned
for more upcoming interviews with Developers and other important figures in
Debian as part of our "Meet your Debian Developer" series. We used the
following tools and services: Turboscribe.ai for the
transcription from the audio and video files, IRC: Oftc.net for
communication, Jitsi meet for interviews, and
Open Broadcaster Software (OBS) for editing and
video. While we encountered many technical difficulties in the return to this
process, we are still able and proud to present the transcripts of the
interviews edited only in a few areas for readability.
2024 Debian Project Leader Candidate: Andrea Tille
Andreas' Interview
Who are you? Tell us a little about yourself.
[Andreas]:
How am I? Well, I'm, as I wrote in my platform, I'm a proud grandfather doing a
lot of free software stuff, doing a lot of sports, have some goals in mind which
I like to do and hopefully for the best of Debian.
And How are you today?
[Andreas]:
How I'm doing today? Well, actually I have some headaches but it's fine for the
interview.
So, usually I feel very good. Spring was coming here and today it's raining and
I plan to do a bicycle tour tomorrow and hope that I do not get really sick but
yeah, for the interview it's fine.
What do you do in Debian? Could you mention your story here?
[Andreas]:
Yeah, well, I started with Debian kind of an accident because I wanted to have
some package salvaged which is called WordNet. It's a monolingual dictionary and
I did not really plan to do more than maybe 10 packages or so. I had some kind
of training with xTeddy which is totally unimportant, a cute teddy you can put
on your desktop.
So, and then well, more or less I thought how can I make Debian attractive for
my employer which is a medical institute and so on. It could make sense to
package bioinformatics and medicine software and it somehow evolved in a
direction I did neither expect it nor wanted to do, that I'm currently the most
busy uploader in Debian, created several teams around it.
DebianMate is very well known from me. I created the Blends team to create teams
and techniques around what we are doing which was Debian TIS, Debian Edu, Debian
Science and so on and I also created the packaging team for R, for the
statistics package R which is technically based and not topic based. All these
blends are covering a certain topic and R is just needed by lots of these
blends.
So, yeah, and to cope with all this I have written a script which is routing an
update to manage all these uploads more or less automatically. So, I think I had
one day where I uploaded 21 new packages but it's just automatically generated,
right? So, it's on one day more than I ever planned to do.
What is the first thing you think of when you think of Debian?
Editors' note: The question was misunderstood as the “worst thing you think of when you
think of Debian”
[Andreas]:
The worst thing I think about Debian, it's complicated. I think today on Debian
board I was asked about the technical progress I want to make and in my opinion
we need to standardize things inside Debian. For instance, bringing all the
packages to salsa, follow some common standards, some common workflow which is
extremely helpful.
As I said, if I'm that productive with my own packages we can adopt this in
general, at least in most cases I think. I made a lot of good experience by the
support of well-formed teams. Well-formed teams are those teams where people
support each other, help each other.
For instance, how to say, I'm a physicist by profession so I'm not an IT expert.
I can tell apart what works and what not but I'm not an expert in those
packages. I do and the amount of packages is so high that I do not even
understand all the techniques they are covering like Go, Rust and something like
this.
And I also don't speak Java and I had a problem once in the middle of the night
and I've sent the email to the list and was a Java problem and I woke up in the
morning and it was solved. This is what I call a team. I don't call a team some
common repository that is used by random people for different packages also but
it's working together, don't hesitate to solve other people's problems and
permit people to get active.
This is what I call a team and this is also something I observed in, it's hard
to give a percentage, in a lot of other teams but we have other people who do
not even understand the concept of the team. Why is working together make some
advantage and this is also a tough thing. I [would] like to tackle in my term if
I get elected to form solid teams using the common workflow. This is one thing.
The other thing is that we have a lot of good people in our infrastructure like
FTP masters, DSA and so on. I have the feeling they have a lot of work and are
working more or less on their limits, and I like to talk to them [to ask] what
kind of change we could do to move that limits or move their personal health to
the better side.
The DPL term lasts for a year, What would you do during that you couldn't do now?
[Andreas]:
Yeah, well this is basically what I said are my main issues. I need to admit I
have no really clear imagination what kind of tasks will come to me as a DPL
because all these financial issues and law issues possible and issues [that]
people who are not really friendly to Debian might create. I'm afraid these
things might occupy a lot of time and I can't say much about this because I
simply don't know.
What are three key terms about you and your candidacy?
[Andreas]:
As I said, I like to work on standards, I’d like to make Debian try [to get
it right so] that people don't get overworked, this third key point is be
inviting to newcomers, to everybody who wants to come. Yeah, I also mentioned in
my term this diversity issue, geographical and from gender point of view. This
may be the three points I consider most important.
Preferred text editor?
[Andreas]:
Yeah, my preferred one? Ah, well, I have no preferred text editor. I'm using the
Midnight Commander very frequently which has an internal editor which is
convenient for small text. For other things, I usually use VI but I also use
Emacs from time to time. So, no, I have not preferred text editor. Whatever
works nicely for me.
What is the importance of the community in the Debian Project? How would
like to see it evolving over the next few years?
[Andreas]:
Yeah, I think the community is extremely important. So, I was on a lot of
DebConfs. I think it's not really 20 but 17 or 18 DebCons and I really enjoyed
these events every year because I met so many friends and met so many
interesting people that it's really enriching my life and those who I never met
in person but have read interesting things and yeah, Debian community makes
really a part of my life.
And how do you think it should evolve specifically?
[Andreas]:
Yeah, for instance, last year in Kochi, it became even clearer to me that the
geographical diversity is a really strong point. Just discussing with some women
from India who is afraid about not coming next year to Busan because there's a
problem with Shanghai and so on. I'm not really sure how we can solve this but I
think this is a problem at least I wish to tackle and yeah, this is an
interesting point, the geographical diversity and I'm running the so-called
mentoring of the month.
This is a small project to attract newcomers for the Debian Med team which has
the focus on medical packages and I learned that we had always men applying for
this and so I said, okay, I dropped the constraint of medical packages.
Any topic is fine, I teach you packaging but it must be someone who does not
consider himself a man. I got only two applicants, no, actually, I got one
applicant and one response which was kind of strange if I'm hunting for women or
so.
I did not understand but I got one response and interestingly, it was for me one
of the least expected counters. It was from Iran and I met a very nice woman,
very open, very skilled and gifted and did a good job or have even lose contact
today and maybe we need more actively approach groups that are underrepresented.
I don't know if what's a good means which I did but at least I tried and so I
try to think about these kind of things.
What part of Debian has made you smile? What part of the project has kept
you going all through the years?
[Andreas]:
Well, the card game which is called Mao on the DebConf made me smile all the
time. I admit I joined only two or three times even if I really love this kind
of games but I was occupied by other stuff so this made me really smile. I also
think the first online DebConf in 2020 made me smile because we had this kind of
short video sequences and I tried to make a funny video sequence about every
DebConf I attended before. This is really funny moments but yeah, it's not only
smile but yeah.
One thing maybe it's totally unconnected to Debian but I learned personally
something in Debian that we have a do-ocracy and you can do things which you
think that are right if not going in between someone else, right? So respect
everybody else but otherwise you can do so.
And in 2020 I also started to take trees which are growing widely in my garden
and plant them into the woods because in our woods a lot of trees are dying and
so I just do something because I can. I have the resource to do something, take
the small tree and bring it into the woods because it does not harm anybody. I
asked the forester if it is okay, yes, yes, okay. So everybody can do so but I
think the idea to do something like this came also because of the free software
idea. You have the resources, you have the computer, you can do something and
you do something productive, right? And when thinking about this I think it was
also my Debian work.
Meanwhile I have planted more than 3,000 trees so it's not a small number but
yeah, I enjoy this.
What part of Debian would you have some criticisms for?
[Andreas]:
Yeah, it's basically the same as I said before. We need more standards to work
together. I do not want to repeat this but this is what I think, yeah.
What field in Free Software generally do you think requires the most work
to be put into it? What do you think is Debian's part in the field?
[Andreas]:
It's also in general, the thing is the fact that I'm maintaining packages which
are usually as modern software is maintained in Git, which is fine but we have
some software which is at Sourceport, we have software laying around somewhere,
we have software where Debian somehow became Upstream because nobody is caring
anymore and free software is very different in several things, ways and well, I
in principle like freedom of choice which is the basic of all our work.
Sometimes this freedom goes in the way of productivity because everybody is free
to re-implement. You asked me for the most favorite editor. In principle one
really good working editor would be great to have and would work and we have
maybe 500 in Debian or so, I don't know.
I could imagine if people would concentrate and say five instead of 500 editors,
we could get more productive, right? But I know this will not happen, right? But
I think this is one thing which goes in the way of making things smooth and
productive and we could have more manpower to replace one person who's [having]
children, doing some other stuff and can't continue working on something and
maybe this is a problem I will not solve, definitely not, but which I see.
What do you think is Debian's part in the field?
[Andreas]:
Yeah, well, okay, we can bring together different Upstreams, so we are building
some packages and have some general overview about similar things and can say,
oh, you are doing this and some other person is doing more or less the same, do
you want to join each other or so, but this is kind of a channel we have to our
Upstreams which is probably not very successful.
It starts with code copies of some libraries which are changed a little bit,
which is fine license-wise, but not so helpful for different things and so I've
tried to convince those Upstreams to forward their patches to the original one,
but for this and I think we could do some kind of, yeah, [find] someone who
brings Upstream together or to make them stop their forking stuff, but it costs
a lot of energy and we probably don't have this and it's also not realistic that
we can really help with this problem.
Do you have any questions for me?
[Andreas]:
I enjoyed the interview, I enjoyed seeing you again after half a year or so.
Yeah, actually I've seen you in the eating room or cheese and wine party or so,
I do not remember we had to really talk together, but yeah, people around, yeah,
for sure. Yeah.
Here are my notes about copying PGP keys to external hardware devices such as
Yubikeys. Let me begin by saying that the gpg tools are pretty bad at this.
MAKE A COUPLE OF BACKUPS OF ~/.gnupg/ TO DIFFERENT ENCRYPTED USB STICKS
BEFORE YOU START. GPG WILL MESS UP YOUR KEYS. SERIOUSLY.
For example, would you believe me if I said that saving changes results in
the removal of your private key? Well
check this
out.
Now that you have multiple safe, offline backups of your keys, here are my notes.
apt install yubikey-manager scdaemon
Plug the Yubikey in, see if it’s recognized properly:
ykman list
gpg --card-status
Change the default PIN (123456) and Admin PIN (12345678):
gpg --card-edit
gpg/card> admin
gpg/card> passwd
Look at the openpgp information and change the maximum number of retries, if
you like. I have seen this failing a couple of times, unplugging the Yubikey
and putting it back in worked.
ykman openpgp info
ykman openpgp access set-retries 7 7 7
Copy your keys. MAKE A BACKUP OF ~/.gnupg/ BEFORE YOU DO THIS.
gpg --edit-key $KEY_ID
gpg> keytocard # follow the prompts to copy the first key
Now choose the next key and copy that one too. Repeat till all subkeys are
copied.
gpg> key 1
gpg> keytocard
Typing gpg --card-status you should be able to see all your keys on the
Yubikey now.
Using the key on another machine
How do you use your PGP keys on the Yubikey on other systems?
Go to another system, if it does have a ~/.gnupg directory already move it
somewhere else.
The diffoscope maintainers are pleased to announce the release of diffoscope
version 263. This version includes the following changes:
[ Chris Lamb ]
* Add support for the zipdetails(1) tool included in the Perl distribution.
Thanks to Larry Doolittle et al. for the pointer to this tool.
* Don't use parenthesis within test "skipping…" messages; PyTest adds its own
parenthesis, so we were ending up with double nested parens.
* Fix the .epub tests after supporting zipdetails(1).
* Update copyright years and debian/tests/control.
[ FC (Fay) Stegerman ]
* Fix MozillaZipContainer's monkeypatch after Python's zipfile module changed
to detect potentially insecure overlapping entries within .zip files.
(Closes: reproducible-builds/diffoscope#362)
You’ve probably heard of the recent backdoor in xz. There have been a lot of takes on this, most of them boiling down to some version of:
The problem here is with Open Source Software.
I want to say not only is that view so myopic that it pushes towards the incorrect, but also it blinds us to more serious problems.
Now, I don’t pretend that there are no problems in the FLOSS community. There have been various pieces written about what this issue says about the FLOSS community (usually without actionable solutions). I’m not here to say those pieces are wrong. Just that there’s a bigger picture.
So with this xz issue, it may well be a state actor (aka “spy”) that added this malicious code to xz. We also know that proprietary software and systems can be vulnerable. For instance, a Twitter whistleblower revealed that Twitter employed Indian and Chinese spies, some knowingly. A recent report pointed to security lapses at Microsoft, including “preventable” lapses in security. According to the Wikipedia article on the SolarWinds attack, it was facilitated by various kinds of carelessness, including passwords being posted to Github and weak default passwords. They directly distributed malware-infested updates, encouraged customers to disable anti-malware tools when installing SolarWinds products, and so forth.
It would be naive indeed to assume that there aren’t black hat actors among the legions of programmers employed by companies that outsource work to low-cost countries — some of which have challenges with bribery.
So, given all this, we can’t really say the problem is Open Source. Maybe it’s more broad:
The problem here is with software.
Maybe that inches us closer, but is it really accurate? We have all heard of Boeing’s recent issues, which seem to have some element of root causes in corporate carelessness, cost-cutting, and outsourcing. That sounds rather similar to the SolarWinds issue, doesn’t it?
Well then, the problem is capitalism.
Maybe it has a role to play, but isn’t it a little too easy to just say “capitalism” and throw up our hands helplessly, just as some do with FLOSS as at the start of this article? After all, capitalism also brought us plenty of products of very high quality over the years. When we can point to successful, non-careless products — and I own some of them (for instance, my Framework laptop). We clearly haven’t reached the root cause yet.
And besides, what would you replace it with? All the major alternatives that have been tried have even stronger downsides. Maybe you replace it with “better regulated capitalism”, but that’s still capitalism.
Then the problem must be with consumers.
As this argument would go, it’s consumers’ buying patterns that drive problems. Buyers — individual and corporate — seek flashy features and low cost, prizing those over quality and security.
But consumers are also people, and some fraction of them are quite capable of writing fantastic software, and in fact, do so.
So what we need is some way to seize control. Some way to do what is right, despite the pressures of consumers or corporations.
Ah yes, dear reader, you have been slogging through all these paragraphs and now realize I have been leading you to this:
Then the solution is Open Source.
Indeed. Faults and all, FLOSS is the most successful movement I know where people are bringing us back to the commons: working and volunteering for the common good, unleashing a thousand creative variants on a theme, iterating in every direction imaginable. We have FLOSS being vital parts of everything from $30 Raspberry Pis to space missions. It is bringing education and communication to impoverished parts of the world. It lets everyone write and release software. And, unlike the SolarWinds and Twitter issues, it exposes both clever solutions and security flaws to the world.
If an authentication process in Windows got slower, we would all shrug and mutter “Microsoft” under our breath. Because, really, what else can we do? We have no agency with Windows.
If an authentication process in Linux gets slower, anybody that’s interested — anybody at all — can dive in and ask “why” and trace it down to root causes.
Some look at this and say “FLOSS is responsible for this mess.” I look at it and say, “this would be so much worse if it wasn’t FLOSS” — and experience backs me up on this.
FLOSS doesn’t prevent security issues itself.
What it does do is give capabilities to us all. The ability to investigate. Ability to fix. Yes, even the ability to break — and its cousin, the power to learn.
New “netplan status –diff” subcommand, finding differences between configuration and system state
As the maintainer and lead developer for Netplan, I’m proud to announce the general availability of Netplan v1.0 after more than 7 years of development efforts. Over the years, we’ve so far had about 80 individual contributors from around the globe. This includes many contributions from our Netplan core-team at Canonical, but also from other big corporations such as Microsoft or Deutsche Telekom. Those contributions, along with the many we receive from our community of individual contributors, solidify Netplan as a healthy and trusted open source project. In an effort to make Netplan even more dependable, we started shipping upstream patch releases, such as 0.106.1 and 0.107.1, which make it easier to integrate fixes into our users’ custom workflows.
With the release of version 1.0 we primarily focused on stability. However, being a major version upgrade, it allowed us to drop some long-standing legacy code from the libnetplan1 library. Removing this technical debt increases the maintainability of Netplan’s codebase going forward. The upcoming Ubuntu 24.04 LTS and Debian 13 releases will ship Netplan v1.0 to millions of users worldwide.
Highlights of version 1.0
In addition to stability and maintainability improvements, it’s worth looking at some of the new features that were included in the latest release:
Simultaneous WPA2 & WPA3 support.
Introduction of a stable libnetplan1 API.
Mellanox VF-LAG support for high performance SR-IOV networking.
New hairpin and port-mac-learning settings, useful for VXLAN tunnels with FRRouting.
New netplan status –diff subcommand, finding differences between configuration and system state.
Besides those highlights of the v1.0 release, I’d also like to shed some light on new functionality that was integrated within the past two years for those upgrading from the previous Ubuntu 22.04 LTS which used Netplan v0.104:
We added support for the management of new network interface types, such as veth, dummy, VXLAN, VRF or InfiniBand (IPoIB).
Wireless functionality was improved by integrating Netplan with NetworkManager on desktop systems, adding support for WPA3 and adding the notion of a regulatory-domain, to choose proper frequencies for specific regions.
To improve maintainability, we moved to Meson as Netplan’s buildsystem, added upstream CI coverage for multiple Linux distributions and integrations (such as Debian testing, NetworkManager, snapd or cloud-init), checks for ABI compatibility, and automatic memory leak detection.
We increased consistency between the supported backend renderers (systemd-networkd and NetworkManager), by matching physical network interfaces on permanent MAC address, when the match.macaddress setting is being used, and added new hardware offloading functionality for high performance networking, such as Single-Root IO Virtualisation virtual function link-aggregation (SR-IOV VF-LAG).
The much improved Netplan documentation, that is now hosted on “Read the Docs”, and new command line subcommands, such as netplan status, make Netplan a well vested tool for declarative network management and troubleshooting.
Integrations
Those changes pave the way to integrate Netplan in 3rd party projects, such as system installers or cloud deployment methods. By shipping the new python3-netplan Python bindings to libnetplan, it is now easier than ever to access Netplan functionality and network validation from other projects. We are proud that the Debian Cloud Team chose Netplan to be the default network management tool in their official cloud-images for Debian Bookworm and beyond. Ubuntu’s NetworkManager package now uses Netplan as it’s default backend on Ubuntu 23.10 Desktop systems and beyond. Further integrations happened with cloud-init and the Calamares installer.
We are pleased to announce that Proxmox
has committed to sponsor DebConf24 as a
Platinum Sponsor.
Proxmox provides powerful and user-friendly open-source server software.
Enterprises of all sizes and industries use Proxmox solutions to deploy
efficient and simplified IT infrastructures, minimize total cost of ownership,
and avoid vendor lock-in. Proxmox also offers commercial support, training
services, and an extensive partner ecosystem to ensure business continuity
for its customers. Proxmox Server Solutions GmbH was established in 2005 and is
headquartered in Vienna, Austria.
Proxmox builds its product offerings on top of the Debian operating system.
With this commitment as Platinum Sponsor, Proxmox is contributing to make
possible our annual conference, and directly supporting the progress of Debian
and Free Software, helping to strengthen the community that continues to
collaborate on Debian projects throughout the rest of the year.
Thank you very much, Proxmox, for your support of DebConf24!
Become a sponsor too!
DebConf24 will take place from 28th July to 4th August 2024 in Busan,
South Korea, and will be preceded by DebCamp, from 21st to 27th July 2024.
DebConf24 is accepting sponsors! Interested companies and organizations may
contact the DebConf team through sponsors@debconf.org, or
visit the Become a DebConf Sponsor website.
A short status update of what happened on my side last month. I spent
quiet a bit of time reviewing new, code (thanks!) as well as
maintenance to keep things going but we also have some improvements:
Add support for progress indicator and counts to lockscreen launcher entries: Merge request,
Demo using Phosh-EV's charge status as example (Merge request)
Was the ssh backdoor the only goal that "Jia Tan" was pursuing
with their multi-year operation against xz?
I doubt it, and if not, then every fix so far has been incomplete,
because everything is still running code written by that entity.
If we assume that they had a multilayered plan, that their every action was
calculated and malicious, then we have to think about the full threat
surface of using xz. This quickly gets into nightmare scenarios of the
"trusting trust" variety.
What if xz contains a hidden buffer overflow or other vulnerability, that
can be exploited by the xz file it's decompressing? This would let the
attacker target other packages, as needed.
Let's say they want to target gcc. Well, gcc contains a lot of
documentation, which includes png images. So they spend a while getting
accepted as a documentation contributor on that project, and get added to
it a png file that is specially constructed, it has additional binary data
appended that exploits the buffer overflow. And instructs xz to modify the
source code that comes later when decompressing gcc.tar.xz.
More likely, they wouldn't bother with an actual trusting trust attack on
gcc, which would be a lot of work to get right. One problem with the ssh
backdoor is that well, not all servers on the internet run ssh. (Or
systemd.) So webservers seem a likely target of this kind of second stage
attack. Apache's docs include png files, nginx does not, but there's always
scope to add improved documentation to a project.
When would such a vulnerability have been introduced? In February, "Jia
Tan" wrote a new decoder for xz.
This added 1000+ lines of new C code across several commits. So much code
and in just the right place to insert something like this. And why take on
such a significant project just two months before inserting the ssh
backdoor? "Jia Tan" was already fully accepted as maintainer, and doing
lots of other work, it doesn't seem to me that they needed to start this
rewrite as part of their cover.
They were working closely with xz's author Lasse Collin in this, by
indications exchanging patches offlist as they developed it. So Lasse
Collin's commits in this time period are also worth scrutiny, because
they could have been influenced by "Jia Tan". One that
caught my eye comes immediately afterwards:
"prepares the code for alternative C versions and inline assembly"
Multiple versions and assembly mean even more places to hide such a
security hole.
I stress that I have not found such a security hole, I'm only considering
what the worst case possibilities are. I think we need to fully consider
them in order to decide how to fully wrap up this mess.
Whether such stealthy security holes have been introduced into xz by "Jia
Tan" or not, there are definitely indications that the ssh backdoor was not
the end of what they had planned.
For one thing, the "test file" based system they introduced
was extensible.
They could have been planning to add more test files later, that backdoored
xz in further ways.
And then there's the matter of the disabling of the Landlock sandbox. This
was not necessary for the ssh backdoor, because the sandbox is only used by
the xz command, not by liblzma. So why did they potentially tip their
hand by adding that rogue "." that disables the sandbox?
A sandbox would not prevent the kind of attack I discuss above, where xz is
just modifying code that it decompresses. Disabling the sandbox suggests
that they were going to make xz run arbitrary code, that perhaps wrote to
files it shouldn't be touching, to install a backdoor in the system.
Both deb and rpm use xz compression, and with the sandbox disabled,
whether they link with liblzma or run the xz command, a backdoored xz can
write to any file on the system while dpkg or rpm is running and noone is
likely to notice, because that's the kind of thing a package manager does.
My impression is that all of this was well planned and they were in it for
the long haul. They had no reason to stop with backdooring ssh, except for
the risk of additional exposure. But they decided to take that risk, with
the sandbox disabling. So they planned to do more, and every commit
by "Jia Tan", and really every commit that they could have influenced
needs to be distrusted.
I do have a xz-unscathed
fork which I've carefully constructed to avoid all "Jia Tan" involved
commits. It feels good to not need to worry about dpkg and tar.
I only plan to maintain this fork minimally, eg security fixes.
Hopefully Lasse Collin will consider these possibilities and address
them in his response to the attack.
First, thanks to Samuel Henrique for giving notice of recent Firefox
CVEs in Debian
testing/unstable.
At the time I didn't want to upgrade my system (Debian Sid) due to the ongoing
t64 transition transition,
so I decided I could install the Firefox Flatpak app instead, and why not stick
to it long-term?
This blog post details all the steps, if ever others want to go the same road.
Flatpak Installation
Disclaimer: this section is hardly anything more than a copy/paste of the
official documentation, and with time it
will get outdated, so you'd better follow the official doc.
And that's all there is to it! Now come the optional steps.
For GNOME and KDE users, you might want to install a plugin for the software
manager specific to your desktop, so that it can support and manage Flatpak
apps:
And here's an additional check you can do, as it's something that did bite me
in the past: missing xdg-portal-* packages, that are required for Flatpak
applications to communicate with the desktop environment. Just to be sure, you
can check the output of apt search '^xdg-desktop-portal' to see what's
available, and compare with the output of dpkg -l | grep xdg-desktop-portal.
As you can see, if you're a GNOME or KDE user, there's a portal backend for
you, and it should be installed. For reference, this is what I have on my GNOME
desktop at the moment:
This is trivial, but still, there's a question I've always asked myself: should
I install applications system-wide (aka. flatpak --system, the default) or
per-user (aka. flatpak --user)? Turns out, this questions is answered in the
Flatpak documentation:
Flatpak commands are run system-wide by default. If you are installing
applications for day-to-day usage, it is recommended to stick with this
default behavior.
Armed with this new knowledge, let's install the
Firefox app:
$flatpakinstallflathuborg.mozilla.firefox
And that's about it! We can give it a go already:
$flatpakrunorg.mozilla.firefox
Data migration
At this point, running Firefox via Flatpak gives me an "empty" Firefox. That's
not what I want, instead I want my usual Firefox, with a gazillion of tabs
already opened, a few extensions, bookmarks and so on.
As it turns out, Mozilla provides a brief doc for data
migration,
and it's as simple as moving Firefox data directory around!
To clarify, we'll be copying data:
from ~/.mozilla/ -- where the Firefox Debian package stores its data
into ~/.var/app/org.mozilla.firefox/.mozilla/ -- where the Firefox Flatpak
app stores its data
Make sure that all Firefox instances are closed, then proceed:
At this point, flatpak run org.mozilla.firefox takes me to my "usual"
everyday Firefox, with all its tabs opened, pinned, bookmarked, etc.
More integration?
After following all the steps above, I must say that I'm 99% happy. So far,
everything works as before, I didn't hit any issue, and I don't even notice
that Firefox is running via Flatpak, it's completely transparent.
So where's the 1% of unhappiness? The « Run a Command » dialog from GNOME, the
one that shows up via the keyboard shortcut <Alt+F2>. This is how I start my
GUI applications, and I usually run two Firefox instances in parallel (one for
work, one for personal), using the firefox -p <profile> command.
Given that I ran apt purge firefox before (to avoid confusing myself with two
installations of Firefox), now the right (and only) way to start Firefox from a
command-line is to type flatpak run org.mozilla.firefox -p <profile>. Typing
that every time is way too cumbersome, so I need something quicker.
Seems like the most straightforward is to create a wrapper script:
And now I can just hit <Alt+F2> and type firefox -p <profile> to start
Firefox with the profile I want, just as before. Neat!
Looking forward: system updates
I usually update my system manually every now and then, via the well-known pair
of commands:
$sudoaptupdate
$sudoaptfull-upgrade
The downside of introducing Flatpak, ie. introducing another package manager,
is that I'll need to learn new commands to update the software that comes via
this channel.
Fortunately, there's really not much to learn. From
flatpak-update(1):
flatpak update [OPTION...] [REF...]
Updates applications and runtimes. [...] If no REF is given, everything is
updated, as well as appstream info for all remotes.
Could it be that simple? Apparently yes, the Flatpak equivalent of the two apt
commands above is just:
$flatpakupdate
Going forward, my options are:
Teach myself to run flatpak update additionally to apt update, manually,
everytime I update my system.
Go crazy: let something automatically update my Flatpak apps, in my back
and without my consent.
I'm actually tempted to go for option 2 here, and I wonder if GNOME Software
will do that for me, provided that I installed gnome-software-plugin-flatpak,
and that I checked « Software Updates -> Automatic » in the Settings (which I
did).
However, I didn't find any documentation regarding what this setting really
does, so I can't say if it will only download updates, or if it will also
install it. I'd be happy if it automatically installs new version of Flatpak
apps, but at the same time I'd be very unhappy if it automatically upgrades my
Debian system...
So we'll see. Enough for today, hope this blog post was useful!