My employer, the biggest Swiss server hosting company, Infomaniak, has just opened registration for its new IaaS (Infrastructure as a Service) OpenStack-based public cloud. Well, in fact, it s been opened since a week or so. Previously, it was only in beta (during that beta period, we hosted (for free) the whole Debconf 21 infrastructure). Nothing really new in the market, except that it is by far cheaper than most (if not all) of its (OpenStack-based or not) competitors, including AWS, GCE or Azure.
Also, everything is hosted in Switzerland, in our own data centers, where data protection is written in the law (and Infomaniak often advertises about data privacy: this is real here ).
Not only Infomaniak is (by far ) the cheapest offer in the market (including a 300 CHF free tier: enough for our smallest VM for a full year), but we also have very good technical support, and the hardware we used is top notch:
6th Gen NVMe (read intensive) Ceph-based block devices
AMD Epyc CPU (128 threads per server)
2x 25Gbits/s (using BGP-to-the-host networking)
Some of our customers didn t even believe how we could do such pricing. Well, the reason is simple: most of our competitors are simply really overpriced, and are making too much money. Since we re late in the market, and that newer hardware (with many cores on a single server) makes is possible to increase density without too much over-commit, my bosses decided that since we could, we would be the cheapest! Hopefully, this will work as a good business strategy.
All of that public cloud infrastructure has been setup with OpenStack Cluster Installer for which I m the main author, and that is fully in Debian. All of this is running on a plain, unmodified Debian Bullseye (well, with a few OpenStack packages a little bit more up-to-date, but really not much, and all of that is publicly available ).
Last, choosing the cheapest and best offer is also a good action: it promotes OpenStack and cloud computing in Debian, which I believe is the least vendor locked-in IaaS solution.
The goal behind reproducible builds is to ensure that no deliberate flaws have been introduced during compilation processes via promising or mandating that identical results are always generated from a given source. This allowing multiple third-parties to come to an agreement on whether a build was compromised or not by a system of distributed consensus.
In these reports we outline the most important things that have been happening in the world of reproducible builds in the past month:
First mentioned in our March 2021 report, Martin Heinz published two blog posts on sigstore, a project that endeavours to offer software signing as a public good, [the] software-signing equivalent to Let s Encrypt . The two posts, the first entitled Sigstore: A Solution to Software Supply Chain Security outlines more about the project and justifies its existence:
Software signing is not a new problem, so there must be some solution already, right? Yes, but signing software and maintaining keys is very difficult especially for non-security folks and UX of existing tools such as PGP leave much to be desired. That s why we need something like sigstore - an easy to use software/toolset for signing software artifacts.
Some time ago I checked Signal s reproducibility and it failed. I asked others to test in case I did something wrong, but nobody made any reports. Since then I tried to test the Google Play Store version of the apk against one I compiled myself, and that doesn t match either.
BitcoinBinary.org was announced this month, which aims to be a repository of Reproducible Build Proofs for Bitcoin Projects :
Most users are not capable of building from source code themselves, but we can at least get them able enough to check signatures and shasums. When reputable people who can tell everyone they were able to reproduce the project s build, others at least have a secondary source of validation.
Related to this, there was continuing discussion on how to embed/encode the build metadata for the Debian live images which were being worked on by Roland Clobus.
Ariadne Conill published another detailed blog post related to various security initiatives within the Alpine Linux distribution. After summarising some conventional security work being done (eg. with sudo and the release of OpenSSH version 3.0), Ariadne included another section on reproducible builds: The main blocker [was] determining what to do about storing the build metadata so that a build environment can be recreated precisely .
Finally, Bernhard M. Wiedemann posted his monthly reproducible builds status report.
Community news
On our website this month, Bernhard M. Wiedemann fixed some broken links [] and Holger Levsen made a number of changes to the Who is Involved? page [][][]. On our mailing list, Magnus Ihse Bursie started a thread with the subject Reproducible builds on Java, which begins as follows:
I m working for Oracle in the Build Group for OpenJDK which is primary responsible for creating a built artifact of the OpenJDK source code. [ ] For the last few years, we have worked on a low-effort, background-style project to make the build of OpenJDK itself building reproducible. We ve come far, but there are still issues I d like to address. []
diffoscopediffoscope is our in-depth and content-aware diff utility. Not only can it locate and diagnose reproducibility issues, it can provide human-readable diffs from many kinds of binary formats. This month, Chris Lamb prepared and uploaded versions 183, 184 and 185 as well as performed significant triaging of merge requests and other issues in addition to making the following changes:
New features:
Support a newer format version of the R language s .rds files. []
Don t call close_archive when garbage collecting Archive instances, unless open_archive definitely returned successfully. This prevents, for example, an AttributeError where PGPContainer s cleanup routines were rightfully assuming that its temporary directory had actually been created. []
Fix (and test) the comparison of R language s .rdb files after refactoring temporary directory handling. []
Ensure that RPM archives exists in the Debian package description, regardless of whether python3-rpm is installed or not at build time. []
Codebase improvements:
Use our assert_diff routine in tests/comparators/test_rdata.py. []
Move diffoscope.versions to diffoscope.tests.utils.versions. []
Reformat a number of modules with Black. [][]
However, the following changes were also made:
Mattia Rizzolo:
Fix an autopkgtest caused by the androguard module not being in the (expected) python3-androguard Debian package. []
Appease a shellcheck warning in debian/tests/control.sh. []
Ignore a warning from h5py in our tests that doesn t concern us. []
Drop a trailing .1 from the Standards-Version field as it s required. []
Zbigniew J drzejewski-Szmek:
Stop using the deprecated distutils.spawn.find_executable utility. [][][][][]
Adjust an LLVM-related test for LLVM version 13. []
Update invocations of llvm-objdump. []
Adjust a test with a one-byte text file for file version 5.40. []
And, finally, Benjamin Peterson added a --diff-context option to control unified diff context size [] and Jean-Romain Garnier fixed the Macho comparator for architectures other than x86-64 [].
Upstream patches
The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:
#990246 filed against vlc (forwarded upstream [][])
Testing framework
The Reproducible Builds project runs a testing framework at tests.reproducible-builds.org, to check packages and other artifacts for reproducibility. This month, the following changes were made:
Holger Levsen:
Drop my package rebuilder prototype as it s not useful anymore. []
Schedule old packages in Debian bookworm. []
Stop scheduling packages for Debian buster. [][]
Don t include PostgreSQL debug output in package lists. []
Detect Python library mismatches during build in the node health check. []
Update a note on updating the FreeBSD system. []
Mattia Rizzolo:
Silence a warning from Git. []
Update a setting to reflect that Debian bookworm is the new testing. []
Upgrade the PostgreSQL database to version 13. []
Roland Clobus (Debian live image generation):
Workaround non-reproducible config files in the libxml-sax-perl package. []
Use the new DNS for the snapshot service. []
Vagrant Cascadian:
Also note that the armhf architecture also systematically varies by the kernel. []
Contributing
If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:
The fifteenth release of littler as a CRAN package just landed, following in the now fifteen year history (!!) as a package started by Jeff in 2006, and joined by me a few weeks later.
littler is the first command-line interface for R as it predates Rscript. It allows for piping as well for shebang scripting via #!, uses command-line arguments more consistently and still starts faster. It also always loaded the methods package which Rscript only started to do in recent years.
littler lives on Linux and Unix, has its difficulties on macOS due to yet-another-braindeadedness there (who ever thought case-insensitive filesystems as a default were a good idea?) and simply does not exist on Windows (yet the build system could be extended see RInside for an existence proof, and volunteers are welcome!). See the FAQ vignette on how to add it to your PATH.
A few examples are highlighted at the Github repo, as well as in the examples vignette.
This release updates the helper scripts to download nighlies of RStudio Server and Desktop to their new naming scheme, adds a downloader for Quarto, extends the roxy.r wrapper with a new option, and updates the configure setting as requestion by CRAN and more. See the NEWS file entry below for more.
Changes in littler version 0.3.14 (2021-10-05)
Changes in examples
Updated RStudio download helper to changed file names
Added a new option to roxy.r wrapper
Added a downloader for Quarto command-line tool
Changes in package
The configure files were updated to the standard of autoconf version 2.69 following a CRAN request
My CRANberries provides a comparison to the previous release. Full details for the littler release are provided as usual at the ChangeLog page, and now also on the new package docs website. The code is available via the GitHub repo, from tarballs and now of course also from its CRAN page and via install.packages("littler"). Binary packages are available directly in Debian as well as soon via Ubuntu binaries at CRAN thanks to the tireless Michael Rutter.
Comments and suggestions are welcome at the GitHub repo.
If you like this or other open-source work I do, you can now sponsor me at GitHub.
JMP offers VoIP calling via XMPP, but it's also
possibly to use the VoIP using SIP.
The underlying VoIP calling functionality in JMP is provided by
Bandwidth, but their old Asterisk
instructions
didn't quite work for me. Here's how I set it up in my Asterisk server.
Get your SIP credentials
After signing up for JMP and setting it up in your favourite XMPP client,
send the following message to the cheogram.com gateway contact:
reset sip account
In response, you will receive a message containing:
a numerical username
a password (e.g. three lowercase words separated by spaces)
Add SIP account to your Asterisk config
First of all, I added the following near the top of my
/etc/asterisk/sip.conf:
Note that you can have more than one register line in your config if you
use more than one SIP provider, but you must register with the server
whether you want to receive incoming calls or not.
Then I added a new blurb to the bottom of the same file:
I checked that the registration was successful by running asterisk -r and
then typing:
sip set debug on
before reloading the configuration using:
reload
Create Asterisk extensions to send and receive calls
Once I got registration to work, I hooked this up with my other extensions
so that I could send and receive calls using my JMP number.
In /etc/asterisk/extensions.conf, I added the following:
[from-jmp]
include => home
exten => s,1,Goto(1000,1)
where home is the context which includes my local SIP devices and 1000
is the extension I want to ring.
Then I added the following to enable calls to any destination within the
North American Numbering Plan:
Here 5551231234 is my JMP phone number, not my bwsip numerical username.
For reference, here's the rest of my dialplan in /etc/asterisk/extensions.conf:
Firewall
Finally, I opened a few ports in my firewall by putting the following in
/etc/network/iptables.up.rules:
# SIP and RTP on UDP (jmp.cbcbc7.auth.bandwidth.com)
-A INPUT -s 67.231.2.13/32 -p udp --dport 5008 -j ACCEPT
-A INPUT -s 216.82.238.135/32 -p udp --dport 5008 -j ACCEPT
-A INPUT -s 67.231.2.13/32 -p udp --sport 5004:5005 --dport 10001:20000 -j ACCEPT
-A INPUT -s 216.82.238.135/32 -p udp --sport 5004:5005 --dport 10001:20000 -j ACCEPT
Outbound calls not working
While the above setup works for me for inbound calls, it doesn't currently
work for outbound calls.
The hostname currently resolves to one of two IP addresses:
Kali is a Debian based distribution aimed at penetration testing. I haven t felt a need to use it in the past because Debian has packages for all the scanning tools I regularly use, and all the rest are free software that can be obtained separately. But I recently decided to try it.
Here s the URL to get Kali [1]. For a VM you can get VMWare or VirtualBox images, I chose VMWare as it s the most popular image format and also a much smaller download (2.7G vs 4G). For unknown reasons the torrent for it didn t work (might be a problem with my torrent client). The download link for it was extremely slow in Australia, so I downloaded it to a system in Germany and then copied it from there.
I don t want to use either VMWare or VirtualBox because I find KVM/Qemu sufficient to do everything I want and they are in the Main section of Debian, so I needed to convert the image files. Some of the documentation on converting image formats to use with QEMU/KVM says to use a program called kvm-img which doesn t seem to exist, I used qemu-img from the qemu-utils package in Debian/Bullseye. The man page qemu-img(1) doesn t list the types of output format supported by the -O option and the examples returned by a web search show using -O qcow2 . It turns out that the following command will convert the image to raw format which is the format I prefer. I use BTRFS for storing all my VM images and that does all the copy-on-write I need.
After converting it the file was 500M smaller than the VMWare files (10.2 vs 10.7G). Probably the Kali distribution file could be reduced in size by converting it to raw and then back to VMWare format. The Kali VMWare image is compressed with 7zip which has a good compression ratio, I waited almost 90 minutes for zstd to compress it with -19 and the result was 12% larger than the 7zip file.
VMWare apparently likes to use an emulated SCSI controller, I spent some time trying to get that going in KVM. Apparently recent versions of QEMU changed the way this works and therefore older web pages aren t helpful. Also allegedly the SCSI emulation is buggy and unreliable (but I didn t manage to get it going so can t be sure). It turns out that the VM is configured to work with the virtio interface, the initramfs.conf has the configuration option MODULES=most which makes it boot on all common configurations (good work by the initramfs-tools maintainers). The image works well with the Spice display interface, so it doesn t capture my mouse, the window for the VM works the same way as other windows on my desktop and doesn t capture the mouse cursor. I don t know if this level of Spice integration is in Debian now, last time I tested it didn t work that way.
I also downloaded Metasploitable [2] which is a VM image designed to be full of security flaws for testing the tools that are in Kali. Again it worked nicely after converting from VMWare to raw format. One thing to note about Metasploitable is that you must not make it available on the public Internet. My home network has NAT for IPv4 but all systems get public IPv6 addresses. It s usually nice that those things just work on VMs but not for this. So I added an iptables command to block IPv6 to /etc/rc.local.
Conclusion
Installing VMs for both these distributions was quite easy. Most of my time was spent downloading from a slow server, trying to get SCSI emulation working, working out how to convert image files, and testing different compression options. The time spent doing stuff once I knew what to do was very small.
Kali has zsh as the default shell, it s quite nice. I ve been happy with bash for decades, but I might end up trying zsh out on other machines.
The diffoscope maintainers are pleased to announce the release of diffoscope
version 186. This version includes the following changes:
[ Chris Lamb ]
* Don't call close_archive when garbage-collecting Archive instances unless
open_archive returned successfully. This prevents, amongst others, an
AttributeError traceback due to PGPContainer's cleanup routines assuming
that its temporary directory had been created.
(Closes: reproducible-builds/diffoscope#276)
* Ensure that the string "RPM archives" exists in the package description,
regardless of whether python3-rpm is installed or not at build time.
[ Jean-Romain Garnier ]
* Fix the LVM Macho comparator for non-x86-64 architectures.
Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language and is widely used by (currently) 912 other packages on CRAN.
This new release brings us Armadillo 10.7.0 released this morning by Conrad. Leading up to this were three runs of reverse dependencies the first of which uncovered the need for a small PR for subview_cols support which Conrad kindly supplied.
The full set of changes follows. We include the last interim release (sent as usual to the drat repo) as well.
Changes in RcppArmadillo version 0.10.7.0.0 (2021-09-30)
Upgraded to Armadillo release 10.7.0 (Entropy Maximizer)
faster handling of submatrix views accessed by X.cols(first_col,last_col)
faster handling of element-wise min() and max() in compound expressions
expanded solve() with solve_opts::force_approx option to force use of the approximate solver
Changes in RcppArmadillo version 0.10.6.2.0 (2021-08-05)
Upgraded to Armadillo release 10.6.2 (Keep Calm)
fix incorrect use of constexpr for handling fixed-size matrices and vectors
Debian Reunion Hamburg 2021 is almost over...
The Debian Reunion Hamburg 2021 is almost over
now, half the attendees have already left for Regensburg, while five remaining people are still busy here, though
tonight there will be two concerts at the venue, plus some lovely food and more. Together with the day trip tomorrow
(involving lots of water but hopefully not from above...) I don't expect much more work to be done, so that I
feel comfortable publishing the following statistics now, even though I expect some more work will be done while
travelling back or due to renewed energy from the event! So I might update these numbers later
Together we did:
27 uploads plus 117 uploads from Gregor from the Perl term
6 RC bugs closed
2 RC bugs opened
1 presentation given
2 DM upload permission was given
1 DNS entry was setup for beta.tests.reproducible-builds.org, showing preliminary real-world data for Debian and Qubes OS, thanks to Qubes OS developer Fr d ric Pierret
1 dinner cooked
5 people didn't show up, only 2 notified us
2 people showed up without registration
had pretty good times and other quality stuff which is harder to quantify
I think that's a pretty awesome and am very happy we did this event!
Debian Reunion / MiniDebConf Hamburg 2022 - save the date, almost!
Thus I think we should have another Debian event at Fux in 2022, and after checking suitable free dates
with the venue I think what could work out is an event from Monday May 23rd until Sunday May 29th 2022. What do you think?
For now these dates are preliminary. If you know any reasons why these dates could be less than optimal for such
an event, please let me know. Assuming there's no feedback indicating this is a bad idea, the dates shall be
finalized by November 1st 2021. Obviously assuming having physical events is still and again a thing!
One of the assumptions baked deeply into US society (and many others) is
that people are largely defined by the work they do, and that work is the
primary focus of life. Even in Marxist analysis, which is otherwise
critical of how work is economically organized, work itself reigns
supreme. This has been part of the feminist critique of both capitalism
and Marxism, namely that both devalue domestic labor that has
traditionally been unpaid, but even that criticism is normally framed as
expanding the definition of work to include more of human activity. A
few exceptions aside, we shy away from
fundamentally rethinking the centrality of work to human experience.
The Problem with Work begins as a critical analysis of that
centrality of work and a history of some less-well-known movements against
it. But, more valuably for me, it becomes a discussion of the types and
merits of utopian thinking, including why convincing other people is not
the only purpose for making a political demand.
The largest problem with this book will be obvious early on: the writing
style ranges from unnecessarily complex to nearly unreadable. Here's an
excerpt from the first chapter:
The lack of interest in representing the daily grind of work routines
in various forms of popular culture is perhaps understandable, as is
the tendency among cultural critics to focus on the animation and
meaningfulness of commodities rather than the eclipse of laboring
activity that Marx identifies as the source of their fetishization
(Marx 1976, 164-65). The preference for a level of abstraction that
tends not to register either the qualitative dimensions or the
hierarchical relations of work can also account for its relative
neglect in the field of mainstream economics. But the lack of
attention to the lived experiences and political textures of work
within political theory would seem to be another matter. Indeed,
political theorists tend to be more interested in our lives as
citizens and noncitizens, legal subjects and bearers of rights,
consumers and spectators, religious devotees and family members, than
in our daily lives as workers.
This is only a quarter of a paragraph, and the entire book is written like
this.
I don't mind the occasional use of longer words for their precise meanings
("qualitative," "hierarchical") and can tolerate the academic habit of
inserting mostly unnecessary citations. I have less patience with the
meandering and complex sentences, excessive hedge words ("perhaps," "seem
to be," "tend to be"), unnecessarily indirect phrasing ("can also account
for" instead of "explains"), or obscure terms that are unnecessary to the
sentence (what is "animation of commodities"?). And please have mercy and
throw a reader some paragraph breaks.
The writing style means substantial unnecessary effort for the reader,
which is why it took me six months to read this book. It stalled all of
my non-work non-fiction reading and I'm not sure it was worth the effort.
That's unfortunate, because there were several important ideas in here
that were new to me.
The first was the overview of the "wages for housework" movement, which I
had not previously heard of. It started from the common feminist position
that traditional "women's work" is undervalued and advocated taking the
next logical step of giving it equality with paid work by making it
paid work. This was not successful, obviously, although the increasing
prevalence of day care and cleaning services has made it partly true
within certain economic classes in an odd and more capitalist way. While
I, like Weeks, am dubious this was the right remedy, the observation
that household work is essential to support capitalist activity but is
unmeasured by GDP and often uncompensated both economically and socially
has only become more accurate since the 1970s.
Weeks argues that the usefulness of this movement should not be judged by
its lack of success in achieving its demands, which leads to the second
interesting point: the role of utopian demands in reframing and expanding
a discussion. I normally judge a political demand on its effectiveness at
convincing others to grant that demand, by which standard many activist
campaigns (such as wages for housework) are unsuccessful. Weeks points
out that making a utopian demand changes the way the person making the
demand perceives the world, and this can have value even if the demand
will never be granted. For example, to demand wages for housework
requires rethinking how work is defined, what activities are compensated
by the economic system, how such wages would be paid, and the implications
for domestic social structures, among other things. That, in turn, helps
in questioning assumptions and understanding more about how existing
society sustains itself.
Similarly, even if a utopian demand is never granted by society at large,
forcing it to be rebutted can produce the same movement in thinking in
others. In order to rebut a demand, one has to take it seriously and
mount a defense of the premises that would allow one to rebut it. That
can open a path to discussing and questioning those premises, which can
have long-term persuasive power apart from the specific utopian demand.
It's a similar concept as the Overton Window, but with more nuance: the
idea isn't solely to move the perceived range of accepted discussion, but
to force society to examine its assumptions and premises well enough to
defend them, or possibly discover they're harder to defend than one might
have thought.
Weeks applies this principle to universal basic income, as a utopian
demand that questions the premise that work should be central to personal
identity. I kept thinking of the Black Lives Matter movement and the
demand to abolish the police, which (at least in popular discussion) is a
more recent example than this book but follows many of the same
principles. The demand itself is unlikely to be met, but to rebut it
requires defending the existence and nature of the police. That in turn
leads to questions about the effectiveness of policing, such as clearance
rates (which are far lower than one might have assumed). Many more
examples came to mind. I've had that experience of discovering problems
with my assumptions I'd never considered when debating others, but had not
previously linked it with the merits of making demands that may be
politically infeasible.
The book closes with an interesting discussion of the types of utopias,
starting from the closed utopia in the style of Thomas More in which the
author sets up an ideal society. Weeks points out that this sort of
utopia tends to collapse with the first impossibility or inconsistency the
reader notices. The next step is utopias that acknowledge their own
limitations and problems, which are more engaging (she cites Le Guin's
The Dispossessed). More conditional
than that is the utopian manifesto, which only addresses part of society.
The least comprehensive and the most open is the utopian demand, such as
wages for housework or universal basic income, which asks for a specific
piece of utopia while intentionally leaving unspecified the rest of the
society that could achieve it. The demand leaves room to maneuver; one
can discuss possible improvements to society that would approach that
utopian goal without committing to a single approach.
I wish this book were better-written and easier to read, since as it
stands I can't recommend it. There were large sections that I read but
didn't have the mental energy to fully decipher or retain, such as the
extended discussion of Ernst Bloch and Friedrich Nietzsche in the context
of utopias. But that way of thinking about utopian demands and their
merits for both the people making them and for those rebutting them, even
if they're not politically feasible, will stick with me.
Rating: 5 out of 10
Release 0.6.28 of the digest package arrived at CRAN earlier today, and has already been uploaded Debian as well.
digest creates hash digests of arbitrary R objects (using the md5, sha-1, sha-256, sha-512, crc32, xxhash32, xxhash64, murmur32, spookyhash, and blake3 algorithms) permitting easy comparison of R language objects. It is a mature and widely-used as many tasks may involve caching of objects for which it provides convenient general-purpose hash key generation.
This release comes eleven months after the previous releases and rounds out a number of corners. Continuous Integration was updated using r-ci. Several contribututors help with a small fix applied to avoid unaligned reads, a rewording for a help page as well as windows path encoding for in the vectorised use case.
My CRANberries provides the usual summary of changes to the previous version. For questions or comments use the issue tracker off the GitHub repo.
If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.
prrd facilitates the parallel running [of] reverse dependency [checks] when preparing R packages. It is used extensively for Rcpp, RcppArmadillo, RcppEigen, BH, and others.
The key idea of prrd is simple, and described in some more detail on its webpage and its GitHub repo. Reverse dependency checks are an important part of package development that is easily done in a (serial) loop. But these checks are also generally embarassingly parallel as there is no or little interdependency between them (besides maybe shared build depedencies). See the (dated) screenshot (running six parallel workers, arranged in a split byobu session).
This release brings some new features I used of late when testing and re-testing reverse dependencies for Rcpp. Enqueuing jobs can now consider the most recent prior job queue file. This allows us to find new packages that were not part of the previous runs. We added a second toggle to also add those packages who failed in the previous run. Finally, the dequeue interface allows to specify a date (rather than defaulting to the current date, useful for long-running jobs or restarts).
The release is summarised in the NEWS entry:
Changes in prrd version 0.0.5 (2021-09-22)
Some remaing http URLs were changed to https.
The dequeueJobs script has a new argument date to help specify a queue file.
The enqueueJobs can now compute just a delta of (new) packages relative to a given prior queuefile and run.
When running in delta mode, previously failed packages can also be selected.
I upgraded my first servers from buster to bullseye over the weekend and it
went very smoothly, so big thank you to all the debian developers who
contributed your labor to the bullseye release!
This morning, however, I hit a snag when the first windows users tried to login.
It seems like a putty bug (see update below).
First, the user received an error related to algorithm selection. I didn t
record the exact error and simply suggested that the user upgrade.
Once the user was running the latest version of putty (0.76), they received a new error:
Server refused public-key signature despite accepting key!
The server log seems to agree with the client returned message: first the key
was accepted, then it was refused.
We re-generated a new key. We turned off the windows firewall. We deleted all
the putty settings via the windows registry and re-set them from scratch.
Nothing seemed to work. Then, another windows user reported no problem (and
that user was running putty version 0.74). So the first user downgraded to 0.74
and everything worked fine.
Update
Wow, very impressed with the responsiveness of putty devs!
And, who knew that putty is available in debian??
Long story short: putty version 0.76 works on linux and, from what I can tell,
works for everyone except my one user. Maybe it s their provider doing some
filtering? Maybe a nuance to their version of Windows?
Debian Reunion Hamburg 2021, we still have free beds
We still have some free slots and beds available for the "Debian Reunion
Hamburg 2021" taking place in Hamburg at the venue of the 2018 & 2019 MiniDebConfs
from Monday, Sep 27 2021 until Friday Oct 1 2021, with
Sunday, Sep 26 2021 as arrival day.
So again, Debian people will meet in Hamburg.
The exact format is less defined and structured than previous years, probably
we will just be hacking from Monday to Wednesday, have talks on Thursday and
a nice day trip on Friday.
Please read https://wiki.debian.org/DebianEvents/de/2021/DebianReunionHamburg and
if you intend to attend, please register there. If additionally you would
like to stay on site (in a single room or shared with one another person),
please mail me.
I'm looking forward to this event, even though (or maybe also because) it will
be much smaller than last years. I suppose this will lead to more personal
interactions and more intense hacking, though of course it is to be seen
how this works out exactly!
podman build (user namespace) and Rename.
It seems like Debian bullseye, if I run podman, it runs in user namespace mode if you run it inside a regular user.
That's fine, but it uses fuse overlayfs driver.
Now I am yet to pinpoint what is happening, but rename() is
handled in a weird way, I think it's broken. os.Rename() in
golang is copying the file and not deleting the original
file.
Another small release of the tidyCpp package arrived on CRAN overnight. The packages offers a clean C++ layer (as well as one small C++ helper class) on top of the C API for R which aims to make use of this robust (if awkward) C API a little easier and more consistent. See the vignette for motivating examples.
The Protect class now uses the default methods for copy and move constructors and assignment allowing for wide use of the class. The small NumVec class now uses it for its data member.
The NEWS entry (which I failed to update for the releases) follows.
Changes in tidyCpp version 0.0.5 (2021-09-16)
The Protect class uses default copy and move assignments and constructors
Colson Whitehead's latest novel, Harlem Shuffle, was always going to be widely reviewed, if only because his last two books won Pulitzer prizes. Still, after enjoying both The Underground Railroad and The Nickel Boys, I was certainly going to read his next book, regardless of what the critics were saying indeed, it was actually quite agreeable to float above the manufactured energy of the book's launch.
Saying that, I was encouraged to listen to an interview with the author by Ezra Klein. Now I had heard Whitehead speak once before when he accepted the Orwell Prize in 2020, and once again he came across as a pretty down-to-earth guy. Or if I were to emulate the detached and cynical tone Whitehead embodied in The Nickel Boys, after winning so many literary prizes in the past few years, he has clearly rehearsed how to respond to the cliched questions authors must be asked in every interview. With the obligatory throat-clearing of 'so, how did you get into writing?', for instance, Whitehead replies with his part of the catechism that 'It seemed like being a writer could be a cool job. You could work from home and not talk to people.' The response is the right combination of cute and self-effacing... and with its slight tone-deafness towards enforced isolation, it was no doubt honed before Covid-19.
Harlem Shuffle tells three separate stories about Ray Carney, a furniture salesman and 'fence' for stolen goods in New York in the 1960s. Carney doesn't consider himself a genuine criminal though, and there's a certain logic to his relativistic morality. After all, everyone in New York City is on the take in some way, and if some 'lightly used items' in Carney's shop happened to have had 'previous owners', well, that's not quite his problem. 'Nothing solid in the city but the bedrock,' as one character dryly observes. Yet as Ezra pounces on in his NYT interview mentioned abov, the focus on the Harlem underworld means there are very few women in the book, and Whitehead's circular response ah well, it's a book about the criminals at that time! was a little unsatisfying. Not only did it feel uncharacteristically slippery of someone justly lauded for his unflinching power of observation (after all, it was the author who decided what to write about in the first place), it foreclosed on the opportunity to delve into why the heist and caper genres (from The Killing, The Feather Thief, Ocean's 11, etc.) have historically been a 'male' mode of storytelling.
Perhaps knowing this to be the case, the conversation quickly steered towards Ray Carney's wife, Elizabeth, the only woman in the book who could be said possesses some plausible interiority. The following off-hand remark from Whitehead caught my attention:
My wife is convinced that [Elizabeth] knows everything about Carney's criminal life, and is sort of giving him a pass. And I'm not sure if that's true. I have to have to figure out exactly what she knows and when she knows it and how she feels about it.
I was quite taken by this, although not simply due to its effect on the story it self. As in, it immediately conjured up a charming picture of Whitehead's domestic arrangements: not only does Whitehead's wife feel free to disagree with what one of Whitehead's 'own' characters knows or believes, but that Colson has no problem whatsoever sharing that disagreement with the public at large. (It feels somehow natural that Whitehead's wife believes her counterpart knows more than she lets on, whilst Whitehead himself imbues the protagonist's wife with a kind of neo-Victorian innocence.) I'm minded to agree with Whitehead's partner myself, if only due to the passages where Elizabeth is studiously ignoring Carney's otherwise unexplained freak-outs.
But all of these meta-thoughts simply underline just how emancipatory the Death of the Author can be. This product of academic literary criticism (the term was coined by Roland Barthes' 1967 essay of the same name) holds that the original author's intentions, ideas or biographical background carry no especial weight in determining how others should interpret their work. It is usually understood as meaning that a writer's own views are no more valid or 'correct' than the views held by someone else. (As an aside, I've found that most readers who encounter this concept for the first time have been reading books in this way since they were young. But the opposite is invariably true with cinephiles, who often have a bizarre obsession with researching or deciphering the 'true' interpretation of a film.) And with all that in mind, can you think of a more wry example of how freeing (and fun) nature of the Death of the Author than an author's own partner dissenting with their (Pulitzer Prize-winning) husband on the position of a lynchpin character?
The 1964 Harlem riot began after James Powell, a 15-year-old African American, was shot and killed by Thomas Gilligan, an NYPD police officer in front of 10s of witnesses. Gilligan was subsequently cleared by a grand jury.
As it turns out, the reviews for Harlem Shuffle have been almost universally positive, and after reading it in the two days after its release, I would certainly agree it is an above-average book. But it didn't quite take hold of me in the way that The Underground Railroad or The Nickel Boys did, especially the later chapters of The Nickel Boys that were set in contemporary New York and could thus make some (admittedly fairly explicit) connections from the 1960s to the present day that kind of connection is not there in Harlem Shuffle, or at least I did not pick up on it during my reading.
I can see why one might take exception to that, though. For instance, it is certainly true that the week-long Harlem Riot forms a significant part of the plot, and some events in particular are entirely contingent on the ramifications of this momentous event. But it's difficult to argue the riot's impact are truly integral to the story, so not only is this uprising against police brutality almost regarded as a background event, any contemporary allusion to the murder of George Floyd is subsequently watered down. It's nowhere near the historical rubbernecking of Forrest Gump (1994), of course, but that's not a battle you should ever be fighting.
Indeed, whilst a certain smoothness of affect is to be priced into the Whitehead reading experience, my initial overall reaction to Harlem Shuffle was fairly flat, despite all the action and intrigue on the page. The book perhaps belies its origins as a work conceived during quarantine after all, the book is essentially comprised of three loosely connected novellas, almost as if the unreality and mental turbulence of lockdown prevented the author from performing the psychological 'deep work' of producing a novel-length text with his usual depth of craft. A few other elements chimed with this being a 'lockdown novel' as well, particularly the book's preoccupation with the sheer physicality of the city compared to the usual complex interplay between its architecture and its inhabitants. This felt like it had been directly absorbed into the book from the author walking around his deserted city, and thus being able to take in details for the first time:
The doorways were entrances into different cities no, different entrances into one vast, secret city. Ever close, adjacent to all you know, just underneath. If you know where to look.
And I can't fail to mention that you can almost touch Whitehead's sublimated hunger to eat out again as well:
Stickups were chops they cook fast and hot, you re in and out. A stakeout was ribs fire down low, slow, taking your time.
[ ]
Sometimes when Carney jumped into the Hudson when he was a kid, some of that stuff got into his mouth. The Big Apple Diner served it up and called it coffee.
More seriously, however, the relatively thin personalities of minor characters then reminded me of the simulacrum of Zoom-based relationships, and the essentially unsatisfactory endings to the novellas felt reminiscent of lockdown pseudo-events that simply fizzle out without a bang. One of the stories ties up loose ends with: 'These things were usually enough to terminate a mob war, and they appeared to end the hostilities in this case as well.' They did? Well, okay, I guess.
The corner of 125th Street and Morningside Avenue in 2019, the purported location of Carney's fictional furniture store. Signage plays a prominent role in Harlem Shuffle, possibly due to the author's quarantine walks.
Still, it would be unfair to characterise myself as 'disappointed' with the novel, and none of this piece should be taken as really deep criticism. The book certainly was entertaining enough, and pretty funny in places as well:
Carney didn t have an etiquette book in front of him, but he was sure it was bad manners to sit on a man s safe.
[ ]
The manager of the laundromat was a scrawny man in a saggy undershirt painted with sweat stains. Launderer, heal thyself.
Yet I can't shake the feeling that every book you write is a book that you don't, and so we might need to hold out a little longer for Whitehead's 'George Floyd novel'. (Although it is for others to say how much of this sentiment is the expectations of a White Reader for The Black Author to ventriloquise the pain of 'their' community.)
Some room for personal critique is surely permitted. I dearly missed the junk food energy of the dry and acerbic observations that run through Whitehead's previous work. At one point he had a good line on the model tokenisation that lurks behind 'The First Negro to...' labels, but the callbacks to this idea ceased without any payoff. Similar things happened with the not-so-subtle critiques of the American Dream:
Entrepreneur? Pepper said the last part like manure. That s just a
hustler who pays taxes.
[ ]
One thing I ve learned in my job is that life is cheap, and when things start getting expensive, it gets cheaper still.
Ultimately, though, I think I just wanted more. I wanted a deeper exploration of how the real power in New York is not wielded by individual street hoodlums or even the cops but in the form of real estate, essentially serving as a synecdoche for Capital as a whole. (A recent take of this can be felt in Jed Rothstein's 2021 documentary, WeWork: Or the Making and Breaking of a $47 Billion Unicorn and it is perhaps pertinent to remember that the US President at the time this novel was written was affecting to be a real estate tycoon.). Indeed, just like the concluding scenes of J. J. Connolly's Layer Cake, although you can certainly pull off a cool heist against the Man, power ultimately resides in those who control the means of production... and a homespun furniture salesman on the corner of 125 & Morningside just ain't that. There are some nods to kind of analysis in the conclusion of the final story ('Their heist unwound as if it had never happened, and Van Wyck kept throwing up buildings.'), but, again, I would have simply liked more.
And when I attempted then file this book away into the broader media landscape, given the current cultural visibility of 1960s pop culture (e.g. One Night in Miami (2020), Judas and the Black Messiah (2021), Summer of Soul (2021), etc.), Harlem Shuffle also seemed like a missed opportunity to critically analyse our (highly-qualified) longing for the civil rights era. I can certainly understand why we might look fondly on the cultural products from a period when politics was less alienated, when society was less atomised, and when it was still possible to imagine meaningful change, but in this dimension at least, Harlem Shuffle seems to merely contribute to this nostalgic escapism.
tl;dr
dgit clone sourcepackage gets you the source code, as a git tree, in ./sourcepackage. cd into it and dpkg-buildpackage -uc -b.
Do not use: "VCS" links on official Debian web pages like tracker.debian.org; "debcheckout"; searching Debian's gitlab (salsa.debian.org). These are good for Debian experts only.
If you use Debian's "official" source git repo links you can easily build a package without Debian's patches applied.[1]
This can even mean missing security patches. Or maybe it can't even be built in a normal way (or at all).
OMG WTF BBQ, why?
It's complicated. There is History.
Debian's "most-official" centralised source repository is still the Debian Archive,
which is a system based on tarballs and patches. I invented the Debian source package format in 1992/3 and it has been souped up since, but it's still tarballs and patches.
This system is, of course, obsolete, now that we have modern version control systems, especially git.
Maintainers of Debian packages have invented ways of using git anyway, of course.
But this is not standardised.
There is a bewildering array of approaches.
The most common approach is to maintain git tree containing a pile of *.patch files,
which are then often maintained using quilt. Yes, really, many Debian people are still using quilt,
despite having git! There is machinery for converting this git tree containing a series of patches, to an "official" source package. If you don't use that machinery, and just build from git, nothing applies the patches.
[1]
This post was prompted by a conversation with a friend who had wanted to build a Debian package,
and didn't know to use dgit. They had
got the source from salsa via a link on tracker.d.o, and built .debs without Debian's patches.
This not a theoretical unsoundness, but a very real practical risk.
Future is not very bright
In 2013 at the Debconf in Vaumarcus, Joey Hess, myself, and others, came up with a plan to try to improve this which we thought would be deployable. (Previous attempts had failed.)
Crucially, this transition plan does not force change onto any of Debian's many packaging teams,
nor onto people doing cross-package maintenance work.
I worked on this for quite a while, and at a technical level it is a resounding success.
Unfortunately there is a big limitation. At the current stage of the transition, to work at its best, this replacement scheme hopes that maintainers who update a package will use a new upload tool. The new tool fits into their existing Debian git packaging workflow and has some benefits, but it does make things more complicated rather than less (like any transition plan must, during the transitional phase). When maintainers don't use this new tool, the standardised git branch seen by users is a compatibility stub generated from the tarballs-and-patches. So it has the right contents, but useless history.
The next step is to allow a maintainer to update a package without dealing with tarballs-and-patches at all. This would be massively more convenient for the maintainer, so an easy sell. And of course it links the tarballs-and-patches to the git history in a proper machine-readable way.
We held a "git packaging requirements-gathering session" at the Curitiba Debconf in 2019. I think the DPL's intent was to try to get input into the git workflow design problem. The session was a great success: my existing design was able to meet nearly everyone's needs and wants. The room was obviously keen to see progress. The next stage was to deploy tag2upload. I spoke to various key people at the Debconf and afterwards in 2019 and the code has been basically ready since then.
Unfortunately, deployment of tag2upload is mired in politics. It was blocked by a key team because of unfounded security concerns; positive opinions from independent security experts within Debian were disregarded. Of course it is always hard to get a team to agree to something when it's part of a transition plan which treats their systems as an obsolete setup retained for compatibility.
Current status
If you don't know about Debian's git packaging practices (eg, you have no idea what "patches-unapplied packaging branch without .pc directory" means), and don't want want to learn about them, you must use dgit to obtain the source of Debian packages.
There is a lot more information and detailed instructions in dgit-user(7).
Hopefully either the maintainer did the best thing, or, if they didn't, you won't need to inspect the history.
If you are a Debian maintainer, you should use dgit push-source to do your uploads.
This will make sure that users of dgit will see a reasonable git history.
edited 2021-09-15 14:48 Z to fix a typo
This is the fifth and final post in a series about the interface description language Candid.
If you made it this far, you now have a good understanding of what Candid is, what it is for and how it is used. For this final post, I ll put the spotlight on specific aspects of Candid that are maybe surprising, or odd, or quirky. This section will be quite opinionated, and could maybe be called what I d do differently if I d re-do the whole thing .
Note that these quirks are not serious problems, and they don t invalidate the overall design. I am writing this up not to discourage the use of Candid, but merely help interested parties to understand it better.
References in the wire format
When the work on Candid began at DFINITY, the Internet Computer was still far away from being a thing, and many fundamental aspects about it were still in the air. I particular, there was still talk about embracing capabilities as a core feature of the application model, which would be implemented as opaque references on the system level, likely building on WebAssembly s host reference type proposal (which only landed recently), and could be used to model access permissions, custom tokens and many other things.
So Candid is designed with that in mind, and you ll find that its wire format is not just a type table and a value table, but actually
a triple (T, M, R), where T ( type ) and M ( memory ) are sequences of bytes and R ( references ) is a sequence of references.
Also the wire format for values of function service tyeps have an extra byte to distinguish between public references (represented by a principal and possible a method name in the data part), and these opaque references.
Alas, references never made it into the Internet Computer, so all Candid implementations simply ignore that part of the specification. But it s still in the spec, and if it confused you before, now you know why.
Hashed field names
Candid record and variant types look like they have textual field names:
type T = record syndactyle : nat; trustbuster: bool
But that is actually only true superficially. The wire format for Candid only stores hashes of field names. So the above is actually equivalent to
type T = record 4260381820 : nat; 3504418361 : bool
or, for that matter, to
type T = record destroys : bool; rectum : nat
(Yes, I used an english word list to find these hash collisions. There aren t that many actually.)
The benefit of such hashing is that the messages are a bit smaller in most (not all) cases, but it is a big annoyance for dynamic uses of Candid. It s the reason why tools like dfx, if they don t know the Candid interface of a service, will print the result with just the numerical hash, letting you guess which field is which.
It also complicates host languages that derive Candid types from the host language, like Motoko, as some records (e.g. record trustbuster: bool; destroys : int ) with field name hash collisions can not be represented in Candid, and either the host language s type system needs to be Candid aware now (as is the case of Motoko), or serialization/deserialization will fail at runtime, or odd bugs can happen.
(More discussion of this issue).
Tuples
Many languages have a built-in notion of a tuple type (e.g. (Int, Bool)), but Candid does not have such a type. The only first class product type is records.
This means that tuples have to encoded as records somehow. Conveniently(?) record fields are just numbers after all, so the type (Int, Bool) would be mapped to the type
record 0 : int; 1 : bool
So tuples can be expressed. But from my experience implementing the type mappings for Motoko and Haskell this is causing headaches. To get a good experience when importing from Candid, the tools have to try to reconstruct which records may have been tuples originally, and turn them into tuples.
The main argument for the status quo is that Candid types should be canonical, and there should not be more than one product type, and records are fine, and there needs to be no anonymous product type. But that has never quite resonated with me, given the practical reality of tuple types in many host languages.
Argument sequences
Did I say that Candid does not have tuple types? Turns out it does, sort of. There is no first class anonymous product, but since functions take sequences of arguments and results, there is a tuple type right there:
func foo : (bool, int) -> (int, bool)
Again, I found that ergonomic interaction with host languages becomes relatively unwieldy by requiring functions to take and return sequences of values. This is especially true for languages where functions take one argument value or return one result type (the latter being very common). Here, return sequences of length one are turned into that type directly, longer argument sequences turn into the host language s tuple type, and nullary argument sequences turn into the idiomatic unit type. But this means that the types (int, bool) -> () and (record 0: int, 1: bool ) -> () may be mapped to the same host language type, which causes problems when you hope to encode all necessary Candid type information in the host language.
Another oddity with argument and result sequences is that you can give names to the entries, e.g. write
without breaking clients, this does not have the effect you think it has and will likely break.
My suggestion is to never put names on function arguments and result values in Candid interfaces, and for anything that might be extended with new fields or where you want to name the arguments, use a single record type as the only argument:
This allows you to add and remove arguments more easily and reliably.
Type shorthands
The Candid specification defines a system of types, and then adds a number of syntactic short-hands . For example, if you write blob in a Candid type description, it ought to means the same as vec nat8.
My qualm with that is that it doesn t always mean the same. A Candid type description is interpreted by a number of, say, consumers . Two such consumers are part of the Candid specification:
The specification that defines the wire format for that type
The upgrading (subtyping) rules
But there are more! For every host language, there is some mapping from Candid types to host language types, and also generic tools like Candid UI are consumers of the type algebra. If these were to take the Candid specification as gospel, they would be forced to treat blob and vec nat8 the same, but that would be quite unergonomic and might cause performance regressions (most language try to map blob to some compact binary data type, while vec t tends to turn into some form of array structure).
So they need to be pragmatic and treat blob and vec nat8 differently. But then, for all practical purposes, blob is not just a short-hand of vec nat8. They are different types that just happens to have the same wire representations and subtyping relations.
This affects not just blob, but also tuples (record blob; int; bool ) and field names , as discussed above.
The value text format
For a while after defining Candid, the only implementation was in Motoko, and all the plumbing was automatic, so there was never a need for users to to explicitly handle Candid values, as all values were Motoko values. Still, for debugging and testing and such things, we eventually needed a way to print out Candid values, so the text format was defined ( To enable convenient debugging, the following grammar specifies a text format for values ).
But soon the dfx tool learned to talk to canisters, and now users needed to enter Candid value on the command line, possibly even when talking to canisters for which the interface was not known to dfx. And, sure enough, the textual interface defined in the Candid spec was used.
Unfortunately, since it was not designed for that use case, it is rather unwieldy:
It is quite verbose. You have to write record , not just . Vectors are written vec ; instead of some conventional syntax like [ , ]. Variants are written as variant error = " " with braces that don t any value here, and something like #error " " might have worked as well.
With a bit more care, a more concise and ergonomic syntax might have been possible.
It wasn t designed to be sufficient to create a Candid value from it. If you write 5 it s unclear whether that s a nat or an int16 or what (and all of these have different wire representations). Type annotations were later added, but are relatively unwieldy, and don t cover all cases (e.g. a service reference with a recursive type cannot be represented in the textual format at the moment).
Not really the fault of the textual format, but some useful information about the types is not reflected in the type description that s part of the wire format. In particular not the field names, and whether a value was intended to be binary data (blob) or a list of small numbers (vec nat8), so pretty-printing such values requires guesswork. The Haskell library even tries to brute-force the hash to guess the field name, if it is short or in a english word list!
In hindsight I think it was too optimistic to assume that correct static type information is always available, and instead of actively trying to discourage dynamic use, Candid might be better if we had taken these (unavoidable?) use cases into account.
Custom wire format
At the beginning of this post, I have a Candid is list. The list is relatively long, and the actual wire format is just one bullet point. Yes, defining a wire format that works isn t rocket science, and it was easiest to just make one up. But since most of the interesting meat of Candid is in other aspects (subtyping rules, host language integration), I wonder if it would have been better to use an existing, generic wire format, such as CBOR, and build Candid as a layer on top.
This would give us plenty of tools and libraries to begin with. And maybe it would have reduced barrier of entry for developers, which now led to the very strange situation that DFINITY advocates for the universal use of Candid on the Internet Computer, so that all services can smoothly interact, but two of the most important services on the Internet Computer (the registry and the ledger) use Protobuf as their primary interface format, with Candid interfaces missing or an afterthought.
Sideways Interface Evolution
This is not a quirk of Candid itself, but rather an idiom of how you can use Candid that emerged from our solution for record extensions and upgrades.
Consider our example from before, a service with interface
service add_user : (record login : text; name : text ) -> ()
where you want to add an age field, which should be a number.
The official way of doing that is to add that field with an optional type:
service add_user : (record login : text; name : text; age : opt nat ) -> ()
As explained above, this will not break old clients, as the decoder will treat a missing argument as null. So far so good.
But often when adding such a field you don t want to bother new clients with the fact that this age was, at some point in the past, not there yet. And you can do that! The trick is to distinguish between the interface you publish and the interface you implement. You can (for example in your documentation) state that the interface is
service add_user : (record login : text; name : text; age : nat ) -> ()
which is not a subtype of the old type, but it is the interface you want new clients to work with. And then your implementation uses the type with opt nat. Calls from old clients will come through as null, and calls from new clients will come through as opt 42.
We can see this idiom used in the Management Canister of the Internet Computer. The current documented interface only mentions a controllers : vec principal field in the settings, but the implementation still can handle both the old controller : principal and the new controllers field.
It s probably advisable to let your CI system check that new versions of your service continue to implement all published interfaces, including past interfaces. But as long as the actual implementation s interface is a subtype of all interfaces ever published, this works fine.
This pattern is related to when your service implements, say, http_request (so its implemented interface is a subtype of that common interface), but does not include that method in the published documentation (because clients of your service don t need to call it).
Self-describing Services
As you have noticed, Candid was very much designed assuming that all parties always have the service type of services they want to interact with. But the Candid specification does not define how one can obtain the interface of a given service, and there isn t really a an official way to do that on the Internet Computer.
That is unfortunate, because many interesting features depend on that: Such as writing import C "ic:7e6iv-biaaa-aaaaf-aaada-cai" in your Motoko program, and having it s type right there. Or tools like ic.rocks, that allow you to interact with any canister right there.
One reason why we don t really have that feature yet is because of disagreements about how dynamic that feature should be. Should you be able to just ask the canister for its interface (and allow the canister to vary the response, for example if it can change its functionality over time, even without changing the actual wasm code)? Or is the interface a static property of the code, and one should be able to query the system for that data, without the canister s active involvement. Or, completely different, should interfaces be distributed out of band, maybe together with API documentation, or in some canister registry somewhere else?
I always leaned towards the first of these options, but not convincingly enough. The second options requires system assistance, so more components to change, more teams to be involved that maybe intrinsically don t care a lot about this feature. And the third might have emerged as the community matures and builds such infrastructure, but that did not happen yet.
In the end I sneaked in an implementation of the first into Motoko, arguing that even if we don t know yet how this feature will be implemented eventually, we all want the feature to exist somehow, and we really really want to unblock all the interesting applications it enables (e.g. Candid UI). That s why every Motoko canister, and some rust canisters too, implements a method
__get_candid_interface_tmp_hack : () -> (text)
that one can use to get the Candid interface file.
The name was chosen to signal that this may not be the final interface, but like all good provisional solutions, it may last longer than intended. If that s the case, I m not sorry.
This concludes my blog post series about Candid, for now. If you want to know more, feel free to post your question on the DFINTY developer forum, and I ll probably answer.
In the aftermath of my report of Facebook censoring mentions of the open-source social network Mastodon, there was a lot of conversation about whether or not this was deliberate.
That conversation seemed to focus on whether a human speficially added joinmastodon.org to some sort of blacklist. But that s not even relevant.
OF COURSE it was deliberate, because of how Facebook tunes its algorithm.
Facebook s algorithm is tuned for Facebook s profit. That means it s tuned to maximize the time people spend on the site engagement. In other words, it is tuned to keep your attention on Facebook.
Why do you think there is so much junk on Facebook? So much anti-vax, anti-science, conspiracy nonsense from the likes of Breitbart? It s not because their algorithm is incapable of surfacing the good content; we already know it can because they temporarily pivoted it shortly after the last US election. They intentionally undid its efforts to make high-quality news sources more prominent twice.
Facebook has said that certain anti-vax disinformation posts violate its policies. It has an extremely cumbersome way to report them, but it can be done and I have. These reports are met with either silence or a response claiming the content didn t violate their guidelines.
So what algorithm is it that allows Breitbart to not just be seen but to thrive on the platform, lets anti-vax disinformation survive even a human review, while banning mentions of Mastodon?
One that is working exactly as intended.
We may think this algorithm is busted. Clearly, Facebook does not. If their goal is to maximize profit by maximizing engagement, the algorithm is working exactly as designed.
I don t know if joinmastodon.org was specifically blacklisted by a human. Nor is it relevant.
Facebook s choice to tolerate and promote the things that service its greed for engagement and money, even if they are the lowest dregs of the web, is deliberate. It is no accident that Breitbart does better than Mastodon on Facebook. After all, which of these does its algorithm detect keep people engaged on Facebook itself more?
Facebook removes the ban
You can see all the screenshots of the censorship in my original post. Now, Facebook has reversed course:
We also don t know if this reversal was human or algorithmic, but that still is beside the point.
The point is, Facebook intentionally chooses to surface and promote those things that drive engagement, regardless of quality.
Clearly many have wondered if tens of thousands of people have died unnecessary deaths over COVID as a result. One whistleblower says I have blood on my hands and President Biden said they re killing people before walking back his comments slightly . I m not equipped to verify those statements. But what do they think is going to happen if they prioritize engagement over quality? Rainbows and happiness?
My employer, the biggest Swiss server hosting company, Infomaniak, has just opened registration for its new IaaS (Infrastructure as a Service) OpenStack-based public cloud. Well, in fact, it s been opened since a week or so. Previously, it was only in beta (during that beta period, we hosted (for free) the whole Debconf 21 infrastructure). Nothing really new in the market, except that it is by far cheaper than most (if not all) of its (OpenStack-based or not) competitors, including AWS, GCE or Azure.
Also, everything is hosted in Switzerland, in our own data centers, where data protection is written in the law (and Infomaniak often advertises about data privacy: this is real here ).
Not only Infomaniak is (by far ) the cheapest offer in the market (including a 300 CHF free tier: enough for our smallest VM for a full year), but we also have very good technical support, and the hardware we used is top notch:
The fifteenth release of 
The key idea of 
podman build (user namespace) and Rename.
It seems like Debian bullseye, if I run podman, it runs in user namespace mode if you run it inside a regular user.
That's fine, but it uses fuse overlayfs driver.
Now I am yet to pinpoint what is happening, but rename() is
handled in a weird way, I think it's broken. os.Rename() in
golang is copying the file and not deleting the original
file.
Colson Whitehead's latest novel, 
This is the fifth and final post in a 
We also don t know if this reversal was human or algorithmic, but that still is beside the point.
The point is, Facebook intentionally chooses to surface and promote those things that drive engagement, regardless of quality.
Clearly many have wondered if tens of thousands of people have died unnecessary deaths over COVID as a result. One whistleblower says