Note: V3D driver is not present in Raspberry Pi models 0-3.

Raspberry Pi 4 64-bit (arm64)

Diff short summary:

1. instead of getting the .config file from bcm2711_defconfig, get it by running make ARCH=arm64 defconfig
2. compile and install the kernel image and modules as usual, but just copy the dtb file arch/arm64/boot/dts/broadcom/bcm2711-rpi-4-b.dtb to the /boot of your target machine (no /overlays directory to copy)
3. change /boot/config.txt:
   • comment/remove the dt_overlay=vc4-kms-v3d entry
   • add a device_tree=bcm2711-rpi-4-b.dtb entry

Raspberry Pi 4 32-bits (arm)

Diff short summary:

1. get the .config file by running make ARCH=arm multi_v7_defconfig
2. using make ARCH=arm menuconfig or a similar tool, enable CONFIG_ARM_LPAE=y
3. compile and install the kernel image and modules as usual, but just copy the dtb file arch/arm/boot/dts/bcm2711-rpi-4-b.dtb to the /boot of your target machine (no /overlays directory to copy)
4. change /boot/config.txt:
   • comment/remove the dt_overlay=vc4-kms-v3d entry
   • add a device_tree=bcm2711-rpi-4-b.dtb entry

Step-by-step for remote deployment:

Set variables Raspberry Pi 4 64-bit (arm64)

cd <path-to-upstream-linux-directory>
KERNEL= make kernelrelease 
ARCH="arm64"
CROSS_COMPILE="aarch64-linux-gnu-"
DEFCONFIG=defconfig
IMAGE=Image
DTB_PATH="broadcom/bcm2711-rpi-4-b.dtb"
RPI4=<ip>
TMP= mktemp -d 

Raspberry Pi 4 32-bits (arm)

cd <path-to-upstream-linux-directory>
KERNEL= make kernelrelease 
ARCH="arm"
CROSS_COMPILE="arm-linux-gnueabihf-"
DEFCONFIG=multi_v7_defconfig
IMAGE=zImage
DTB_PATH="bcm2711-rpi-4-b.dtb"
RPI4=<ip>
TMP= mktemp -d 

Get default .config file

make ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE $DEFCONFIG

Raspberry Pi 4 32-bit (arm) Additional step for 32-bit system. Enable CONFIG_ARM_LPAE=y using make ARCH=arm menuconfig

Cross-compile the mainline kernel

make ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE $IMAGE modules dtbs

Install modules to send

make ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE INSTALL_MOD_PATH=$TMP modules_install

Copy kernel image, modules and the dtb to your remote system

ssh $RPI4 mkdir -p /tmp/new_modules /tmp/new_kernel
rsync -av $TMP/ $RPI4:/tmp/new_modules/
scp arch/$ARCH/boot/$IMAGE $RPI4:/tmp/new_kernel/$KERNEL
scp arch/$ARCH/boot/dts/$DTB_PATH $RPI4:/tmp/new_kernel
ssh $RPI4 sudo rsync -av /tmp/new_modules/lib/modules/ /lib/modules/
ssh $RPI4 sudo rsync -av /tmp/new_kernel/ /boot/
rm -rf $TMP

Set config.txt of you RPi 4 In your Raspberry Pi 4, open the config file /boot/config.txt

• comment/remove the dt_overlay=vc4-kms-v3d entry
• add a device_tree=bcm2711-rpi-4-b.dtb entry
• add a kernel=<image-name> entry

Why not Kworkflow? You can safely use the steps above, but if you are hacking the kernel and need to repeat this compiling and installing steps repeatedly, why don t try the Kworkflow? Kworkflow is a set of scripts to synthesize all steps to have a custom kernel compiled and installed in local and remote machines and it supports kernel building and deployment to Raspberry Pi machines for Raspbian 32 bit and 64 bit. After learning the kernel compilation and installation step by step, you can simply use kw bd command and have a custom kernel installed in your Raspberry Pi 4.

Note: the userspace 3D acceleration (glx/mesa) is working as expected on arm64, but the driver is not loaded yet for arm. Besides that, a bunch of pte invalid errors may appear when using 3D acceleration, it s a known issue that are still under investigation.

The Road to Gandolfo I think I had read this book almost 10-12 years back and somehow ended up reading it up again. Apparently, he had put this fiction, story, book under some other pen name earlier. It is possible that I might have read it under that name and hence forgotten all about it. This book/story is full of innuendo, irony, sarcasm and basically the thrill of life. There are two main characters in the book, the first is General Mackenzie who has spent almost 3 to 4 decades being a spy/a counterintelligence expert in the Queen s service. And while he outclasses them all even at the ripe age of 50, he is thrown out under the pretext of conduct unbecoming of an officer. The other main character is Sam Devereaux. This gentleman is an army lawyer and is basically counting the days when he completes his tour of duty as a military lawyer and start his corporate civil law with somebody he knows. It is much to his dismay that while under a week is left for his tour of duty to be left over he is summoned to try and extradite General MacKenzie who has been put on house arrest. Apparently, in China there was a sculpture of a great Chinese gentleman in the nude. For reasons unknown or rather not being shared herein, he basically breaks part of the sculpture. This of course, enrages the Chinese and they call it a diplomatic accident and try to put the General into house arrest. Unfortunately for both the General and his captors, he decides to escape/go for it. While he does succeed entering the American embassy, he finds himself to be person non-grata and is thrown back outside where the Chinese recapture him. This is where the Embassy & the Govt. decide it would be better if somehow the General could be removed from China permanently so he doesn t cause any further diplomatic accidents. In order to do that Sam s services are bought. Now in order to understand the General, Sam learns that he has 4 ex-wives. He promptly goes and meet them to understand why the general behaved as he did. He apparently also peed on the American flag. To his surprise, all the four ex-wives are still very much in with the general. During the course of interviewing the ladies he is seduced by them and also gives names to their chests in order to differentiate between each one of them. Later he is seduced by the eldest of the four wives and they spend the evening together. Next day Sam meets and is promptly manhandled by the general and the diplomatic papers are seen by the general. After meeting the general and the Chinese counterpart, they quickly agree to extradite him as they do not know how to keep the general control. During his stay of house arrest, the General reads one of the communist rags as he puts it and gets the idea to kidnap the pope and that forms the basis of the story. Castel Gandolfo seems to be a real place which is in Italy and is apparently is the papal residence where s/he goes to reside every winter. The book is written in 1976 hence in the book, the General decides to form a corporation for which he would raise funds in order to make the kidnapping. The amount in 1976 was 40 million dollars and it was a big sum, to be with times, let s think of say 40 billion dollars so gets the scale of things. Now while a part of me wants to tell the rest of the story, the story isn t really mine to tell. Read The Road to Gandolfo for the rest. While I can t guarantee you much, I can say you might find yourself constantly amused by the antics of both the General, Sam and the General s ex-wives. There are also a few minute characters that you will meet on the way, hope you discover them and enjoy it immensely as I have. One thing I have to say, while I was reading it, I very much got vibes of Not a penny more, not a penny less by Jeffrey Archer. As shared before, lots of twists and turns, enjoy the ride

Webforms Webforms are nothing but a form you fill on the web or www. Webforms are and were a thing from early 90s to today. I was supposed to register for https://www.swavlambancard.gov.in/ almost a month back but procrastinated till few couple of days back and with good reason. I was hoping one of my good friends would help me but they had their own thing. So finally, I tried to fill the form few days back. It took me almost 30 odd attempts to finally fill the form and was given an enrollment number. Why it took me 30 odd attempts and with what should tell you the reason

1. I felt like I was filling the form from 1990 s rather than today because
   
2. The form doesn t know either its state or saves data during a session This lesson has been learned a long time back by almost all service providers except Govt. of India. Both the browsers on a mobile as well as desktop can save data during session. If you don t know what I mean by that go to about:preferences#privacy in Firefox and look at Manage Data. There you will find most sites do put some data along with cookies arguably to help make your web experience better. Chrome or Chromium has the same thing perhaps shared under a different name but its the same thing. But that is not all.
   
3. None of the fields have any verification. The form is of 3 pages. The verification at the end of the document doesn t tell you what is wrong and what needs to be corrected. Really think on this, I am on a 24 LED monitor and I m filling the form and I had to do it at least 20-30 times before it was accepted. And guess what, I have no clue even about why it was selected. The same data, the same everything and after the nth time it accepted. Now if I am facing such a problem when I have some idea how technology works somewhat how are people who are trying to fill this form on 6 mobiles supposed to do? And many of them not at all clued in technology as I am.
   

I could go on outlining many of the issues that I faced but they are all similar in many ways the problems faced while filling the NEW Income Tax forms. Of course the New Income Tax portal is a whole ball-game in itself as it gives new errors every time instead of solving them. Most C.A. s have turned to third-party xml tools that enable you to upload xml compliant data to the New Income tax portal but this is for businesses and those who can afford it. Again, even that is in a sort of messy state but that is a whole another tale altogether. One of the reasons to my mind why the forms are designed the way they are so that people go to specific cybercafes or get individual people to fill and upload it and make more money. I was told to go to a specific cybercafe and meet a certain individual and he asked for INR 500/- to do the work. While I don t have financial problems, I was more worried about my data going into the wrong hands. But I can see a very steady way to make money without doing much hard work.

Hearing Loss info. Now because I had been both to Kamla Nehru Hospital as well as Sasoon and especially the deaf department, I saw many kids with half-formed ears. I had asked the doctors and they had shared this is due to malnutrition. We do know that women during pregnancies need more calories, more everything as they are eating for two bodies, not one. And this is large-scale, apparently more than 5 percent of population have children like this. And this number was of 2014, what is it today nobody knows. I also came to know that at least for some people like me, due to Covid they became deaf. I had asked the doctors if they knew of people who had become deaf due to Covid. They basically replied in the negative as they don t have the resources to monitor the same. The Govt. has an idea of health ID but just like Aadhar has to many serious sinister implications. Somebody had shared with me a long time back that in India systems work inspite of Govt. machinery rather than because of it. Meaning that the Government itself ties itself into several knots and then people have to be creative to try and figure a way out to help people. I found another issue while dealing with them. Apparently, even though I have 60% hearing loss I would be given a certificate of 40% hearing loss and they call it Temporary Progressive Loss. I saw almost all the people who had come, many of them having far severe defencies than me getting the same/similar certificate. All of them got Temporary Progressive. Some of the cases were real puzzling. For e.g. I met another Agarwal who had a severe accident few months ago and there is some kind of paralysis & bone issue. The doctors have given up but even that gentleman was given Temporary Progressive. From what little I could understand, the idea is that over period if there is possibility of things becoming better then it should be given. Another gentleman suffered a case of dwarfism. Even he was given the same certificate. Think there have been orders from above so that people even having difficulties are not helped. Another point if you look in a macro sense, it presents a somewhat rosy picture. If someone were to debunk the Govt. data either from India or abroad then from GOI perspective they have an agenda even though the people who are suffering are our brothers and sisters And all of this is because I can read, write, articulate. Perhaps many of them may not even have a voice or a platform. Even to get this temporary progressive disability certificate there is more than 4 months of running from one place to the other, 4 months of culmination of work. This I can share and tell from my experience, who knows how much else others might have suffered for the same. In my case a review will happen after 5 years, in most other cases they have given only 1 year. Of course, this does justify people s jobs and perhaps partly it may be due to that. Such are times where I really miss that I am unable to hear otherwise could have fleshed out lot more other people s sufferings. And just so people know/understand this is happening in the heart of the city whose population easily exceeds 6 million plus and is supposed to be a progressive city. I do appreciate and understand the difficulties that the doctors are placed under.

Mum s Birthday & Social Engineering. While I don t want to get into details, in the last couple of weeks mum s birthday was there and that had totally escaped me. I have been trying to disassociate myself from her and at times it s hard and then you don t remember and somebody makes you remember. So, on one hand guilty, and the other do not know what to do. If she were alive I would have bought a piece of cake or something. Didn t feel like it, hence donated some money to the local aged home. This way at least I hope they have some semblance of peace. All of them are of her similar age group. The other thing that I began to observe in the earnest, fake identities have become the norm. Many people took elon musk s potrait using their own names in the handles, but even then Elon Free Speech Musk banned them. So much for free speech. Then I saw quite a few handles that have cute women as their profile picture but they are good at social engineering. This has started only a couple of weeks back and have seen quite a few handles leaving Twitter and joining Mastodon. Also, have been hearing that many admins of Mastodon pods are unable to get on top of this. Also, lot of people complaining as it isn t user-friendly UI as twitter is. Do they not realize that Twitter has its own IP and any competing network can t copy or infringe on their product. Otherwise, they will be sued like how Ford was & potentially win. I am not really gonna talk much about it as the blog post has become quite long and that needs its own post to do any sort of justice to it. Till later people

Changes in RcppArmadillo version 0.11.4.2.1 (2022-11-08)

• Upgraded to Armadillo release 11.4.2 (Ship of Theseus)
  • more robust handling of corner cases in multi-threaded contexts
• Internal header organisation with new sub-directories while providing full compatibility via existing paths (#395 #396)

Changes in RcppArmadillo version 0.11.4.1.0 (2022-10-10) (GitHub Only)

• Upgraded to Armadillo release 11.4.1 (Ship of Theseus)
  • fix data race in Cube::slice()
  • workarounds for false warnings emitted by GCC 12 when compiling with FORTIFY_SOURCE enabled (already in RcppArmadillo 0.11.4.0.1 too)

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

• DDPO
• Desde agosto de 2022
• Cidade de resid ncia: Curitiba - PR

• DDPO
• Desde maio de 2021
• Cidade de resid ncia: Curitiba - PR

• DDPO
• Desde maio de 2020
• Cidade de resid ncia: Campinas - SP

• DDPO
• Desde maio de 2020
• Cidade de resid ncia: Campinas - SP

• DDPO
• Desde fevereiro de 2019
• Cidade de resid ncia: Bras lia - DF

• DDPO
• Desde janeiro de 2019
• Cidade de resid ncia: Curitiba - PR

• DDPO
• Desde abril de 2018
• Cidade de resid ncia: Curitiba - PR

• DDPO
• Desde mar o de 2018
• Cidade de resid ncia: Toronto - Canad

• DDPO
• Desde maio de 2017
• Cidade de resid ncia: Curitiba - PR

• DDPO
• Desde dezembro de 2016
• Cidade de resid ncia: Po o Fundo - MG

• Desde agosto de 2016
• Cidade de resid ncia: Novo Hamburgo - RS

• DDPO
• Desde agosto de 2016
• Cidade de resid ncia: Campinas - SP

• DDPO
• Desde junho de 2016
• Cidade de resid ncia: Bras lia - DF

• DDPO
• Desde junho de 2016
• Cidade de resid ncia: Rio de Janeiro RJ

• DDPO
• Desde abril de 2022
• Cidade de resid ncia: Campo Mour o - PR

• DDPO
• Desde janeiro de 2022
• Cidade de resid ncia: Bras lia - DF

• DDPO
• Desde novembro de 2021
• Cidade de resid ncia: S o Vicente - SP

• DDPO
• Desde novembro de 2021
• Cidade de resid ncia: Bras lia - DF

• DDPO
• Desde agosto de 2020
• Cidade de resid ncia: Santa Maria - RS

• DDPO
• Desde dezembro de 2019
• Cidade de resid ncia: Curitiba - PR

• DDPO
• Desde maio de 2016
• Cidade de resid ncia: Belo Horizonte - MG

1. Esta lista ser atualizada quando o time de publicidade do Debian publicar novas listas com DMs e DDs e tiver brasileiros.
2. Para ver a lista completa de Mantenedores(as) e Desenvolvedores(as) Debian, inclusive outros(as) brasileiros(as) antes de julho de 2015 acesse: https://nm.debian.org/public/people

• DDPO
• Desde agosto de 2022
• Cidade de resid ncia: Curitiba - PR

• DDPO
• Desde maio de 2021
• Cidade de resid ncia: Curitiba - PR

• DDPO
• Desde maio de 2020
• Cidade de resid ncia: Campinas - SP

• DDPO
• Desde maio de 2020
• Cidade de resid ncia: Campinas - SP

• DDPO
• Desde fevereiro de 2019
• Cidade de resid ncia: Bras lia - DF

• DDPO
• Desde janeiro de 2019
• Cidade de resid ncia: Curitiba - PR

• DDPO
• Desde abril de 2018
• Cidade de resid ncia: Curitiba - PR

• DDPO
• Desde mar o de 2018
• Cidade de resid ncia: Toronto - Canad

• DDPO
• Desde maio de 2017
• Cidade de resid ncia: Curitiba - PR

• DDPO
• Desde dezembro de 2016
• Cidade de resid ncia: Po o Fundo - MG

• Desde agosto de 2016
• Cidade de resid ncia: Novo Hamburgo - RS

• DDPO
• Desde agosto de 2016
• Cidade de resid ncia: Campinas - SP

• DDPO
• Desde junho de 2016
• Cidade de resid ncia: Bras lia - DF

• DDPO
• Desde junho de 2016
• Cidade de resid ncia: Rio de Janeiro RJ

• DDPO
• Desde abril de 2022
• Cidade de resid ncia: Campo Mour o - PR

• DDPO
• Desde janeiro de 2022
• Cidade de resid ncia: Bras lia - DF

• DDPO
• Desde novembro de 2021
• Cidade de resid ncia: S o Vicente - SP

• DDPO
• Desde novembro de 2021
• Cidade de resid ncia: Bras lia - DF

• DDPO
• Desde agosto de 2020
• Cidade de resid ncia: Santa Maria - RS

• DDPO
• Desde dezembro de 2019
• Cidade de resid ncia: Curitiba - PR

• DDPO
• Desde maio de 2016
• Cidade de resid ncia: Belo Horizonte - MG

1. Esta lista ser atualizada quando o time de publicidade do Debian publicar novas listas com DMs e DDs e tiver brasileiros.
2. Para ver a lista completa de Mantenedores(as) e Desenvolvedores(as) Debian, inclusive outros(as) brasileiros(as) antes de julho de 2015 acesse: https://nm.debian.org/public/people

printf '\a'

Making the bell useful and silent Many terminal emulators have this feature where they can silence the bell somehow. It can be turned into a "visual bell" which basically flashes the screen when a bell arrives. Or that can also be disabled and the bell is just completely ignored. What I did instead is turn the bell into a "urgency hint" (part of the ICCCM standard. In xterm, this is done with this X resource entry (typically in ~/.Xresources):

XTerm*bellIsUrgent:  true
XTerm*visualBell: false

Interestingly, this doesn't clearly say "bell is muted", but it's effectively what it does. Or maybe it works because I have muted "System Sounds" in Pulseaudio. Who knows. I do have this in my startup scripts though:

xset b off

... which, according to the xset(1) manpage, means

If the dash or 'off' are given, the bell will be turned off.

Interestingly, you have the option of setting the bell "volume", "pitch, in hertz, and [...] duration in milliseconds. Note that not all hardware can vary the bell characteristics." In any case, I think that's the magic trick to turn the darn thing off. Now this should send urgency hints to your window manager:

sleep 3 ; printf '\a'

Try it: run the command, switch to another desktop, then wait 3 seconds. You should see the previous desktop show up in red or something. In the i3 window manager I am currently using, this is the default, although I did set the colors (client.urgent and urgent_workspace in bar.colors). Other window managers or desktop environments may require different configurations.

Sending a bell... Now that, on itself, will only be useful when something sets a bell. One place I had found a trick like this, long ago, is this post (dead link, archived) which has various instructions for different tools. I'll recopy some of them here since the original site is dead, but credit goes to the Netbuz blog. (Note that the blog post also features an Awesome WM configuration for urgency hints.)

Mutt

set beep=yes
set beep_new=yes

Irssi

/set beep_when_window_active ON
/set beep_when_away ON
/set beep_msg_level MSGS DCC DCCMSGS HILIGHT

It was also recommending this setting, but it appears to be deprecated and gives a warning in modern irssi versions:

/set bell_beeps ON

GNU Screen This is an important piece of the puzzle, because by default, terminal multiplexers have their own opinion of what to do with the bell as well:

# disabled: we want to propagate bell to clients, which should handle
# it in their own terminal settings. this  vbell off  is also the
# upstream and tmux's default
#
# see also: http://netbuz.org/blog/2011/11/x-bells-and-urgency-hints/
vbell off
# propagate bell from other windows up to the terminal emulator as well
bell_msg 'Bell in window %n^G'

The bell_msg bit is an extra from me: it uses the bell message that pops up when screen detects a bell in another window to resend the bell control character up to the running terminal. This makes it so a bell in any multiplexed window will also propagate to the parent terminal, which is not the default.

Tmux Untested, but that is apparently how you do it:

# listen to alerts from all windows
set -g bell-action any
# notice bell in windows
set -g monitor-bell on
# only propagate bell, don't warn user, as it hangs tmux for a second
set -g visual-bell off
# send bell *and* notify when activity (if monitor-activity)
set -g visual-activity both

Note that this config goes beyond what we have in GNU screen in that inactivity or activity will trigger a bell as well. This might be useful for cases where you don't have the prompt hack (below) but it could also very well be very noisy. It will only generate noise when monitor-activity is enabled though.

bash and shell prompts Now the icing on cake is to actually send the bell when a command completes. This is what I use this for the most, actually. I was previously using undistract-me for this, actually. That was nice: it would send me a nice desktop notification when a command was running more than a certain amount of time, and if the window was unfocused. But configuring this was a nightmare: because it uses a complex PROMPT_COMMAND in bash, it would conflict with my (already existing and way too) complex bash prompt, and lead to odd behaviors. It would also not work for remote commands, of course, as it wouldn't have access to my local D-BUS to send notifications (thankfully!). So instead, what I do now is systematically print a bell whenever a command terminates, in all my shells. I have this in my /root/.bashrc on all my servers, deployed in Puppet:

PROMPT_COMMAND='printf "\a"'

Or you can just put it directly in the shell prompt, with something like:

PS1='\[\a\]'"$PS1"

(I have the equivalent in my own .bashrc, although that thing is much more complex, featuring multi-command pipeline exit status, colors, terminal title setting, and more, which should probably warrant its own blog post.) This sounds a little bonkers and really noisy, but remember that I turned off the audible bell. And urgency hints are going to show up only if the window is unfocused. So it's actually really nice and not really distracting. Or, to reuse the undistract-me concept: it allows me to not lose focus too much when I'm waiting for a long process to complete. That idea actually came from ahf, so kudos to him on that nice hack.

Caveats That is not setup on all the machines I administer for work, that said. I'm afraid that would be too disruptive for people who do not have that configuration. This implies that I don't get notifications for commands that run for a long time on remote servers, most of the time. That said, I could simply run a command with a trailing printf '\a' to get a notification. This might not work in Wayland, your window manager, your desktop environment, your Linux console, or your telegraphy session.

$ kubectl get pods -A
NAMESPACE     NAME                                       READY   STATUS    RESTARTS       AGE
kube-system   calico-kube-controllers-59697b644f-7fsln   1/1     Running   2 (119m ago)   7d
kube-system   calico-node-6ptsh                          1/1     Running   2 (119m ago)   7d
kube-system   calico-node-7x5j8                          1/1     Running   2 (119m ago)   7d
kube-system   calico-node-qlnf6                          1/1     Running   2 (119m ago)   7d
kube-system   coredns-565d847f94-79jlw                   1/1     Running   2 (119m ago)   7d
kube-system   coredns-565d847f94-fqwn4                   1/1     Running   2 (119m ago)   7d
kube-system   etcd-k8s-master                            1/1     Running   2 (119m ago)   7d
kube-system   kube-apiserver-k8s-master                  1/1     Running   2 (119m ago)   7d
kube-system   kube-controller-manager-k8s-master         1/1     Running   2 (119m ago)   7d
kube-system   kube-proxy-4n9b7                           1/1     Running   2 (119m ago)   7d
kube-system   kube-proxy-k4rzv                           1/1     Running   2 (119m ago)   7d
kube-system   kube-proxy-lz2dd                           1/1     Running   2 (119m ago)   7d
kube-system   kube-scheduler-k8s-master                  1/1     Running   2 (119m ago)   7d

$ curl --proto '=https' --tlsv1.2 -sSfL https://run.linkerd.io/install   sh

export PATH=$PATH:~/.linkerd2/bin/

$ linkerd version
Client version: stable-2.12.2
Server version: unavailable

$ linkerd check --pre
Linkerd core checks
===================

kubernetes-api
--------------
  can initialize the client
  can query the Kubernetes API

kubernetes-version
------------------
  is running the minimum Kubernetes API version
  is running the minimum kubectl version

pre-kubernetes-setup
--------------------
  control plane namespace does not already exist
  can create non-namespaced resources
  can create ServiceAccounts
  can create Services
  can create Deployments
  can create CronJobs
  can create ConfigMaps
  can create Secrets
  can read Secrets
  can read extension-apiserver-authentication configmap
  no clock skew detected

linkerd-version
---------------
  can determine the latest version
  cli is up-to-date

Status check results are  

$ linkerd install --crds   kubectl apply -f -
Rendering Linkerd CRDs...
Next, run  linkerd install   kubectl apply -f -  to install the control plane.

customresourcedefinition.apiextensions.k8s.io/authorizationpolicies.policy.linkerd.io created
customresourcedefinition.apiextensions.k8s.io/httproutes.policy.linkerd.io created
customresourcedefinition.apiextensions.k8s.io/meshtlsauthentications.policy.linkerd.io created
customresourcedefinition.apiextensions.k8s.io/networkauthentications.policy.linkerd.io created
customresourcedefinition.apiextensions.k8s.io/serverauthorizations.policy.linkerd.io created
customresourcedefinition.apiextensions.k8s.io/servers.policy.linkerd.io created
customresourcedefinition.apiextensions.k8s.io/serviceprofiles.linkerd.io created

$ linkerd install   kubectl apply -f -       
namespace/linkerd created
clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-identity created
clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-identity created
serviceaccount/linkerd-identity created
clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-destination created
clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-destination created
serviceaccount/linkerd-destination created
secret/linkerd-sp-validator-k8s-tls created
validatingwebhookconfiguration.admissionregistration.k8s.io/linkerd-sp-validator-webhook-config created
secret/linkerd-policy-validator-k8s-tls created
validatingwebhookconfiguration.admissionregistration.k8s.io/linkerd-policy-validator-webhook-config created
clusterrole.rbac.authorization.k8s.io/linkerd-policy created
clusterrolebinding.rbac.authorization.k8s.io/linkerd-destination-policy created
role.rbac.authorization.k8s.io/linkerd-heartbeat created
rolebinding.rbac.authorization.k8s.io/linkerd-heartbeat created
clusterrole.rbac.authorization.k8s.io/linkerd-heartbeat created
clusterrolebinding.rbac.authorization.k8s.io/linkerd-heartbeat created
serviceaccount/linkerd-heartbeat created
clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-proxy-injector created
clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-proxy-injector created
serviceaccount/linkerd-proxy-injector created
secret/linkerd-proxy-injector-k8s-tls created
mutatingwebhookconfiguration.admissionregistration.k8s.io/linkerd-proxy-injector-webhook-config created
configmap/linkerd-config created
secret/linkerd-identity-issuer created
configmap/linkerd-identity-trust-roots created
service/linkerd-identity created
service/linkerd-identity-headless created
deployment.apps/linkerd-identity created
service/linkerd-dst created
service/linkerd-dst-headless created
service/linkerd-sp-validator created
service/linkerd-policy created
service/linkerd-policy-validator created
deployment.apps/linkerd-destination created
cronjob.batch/linkerd-heartbeat created
deployment.apps/linkerd-proxy-injector created
service/linkerd-proxy-injector created
secret/linkerd-config-overrides created

$ kubectl get pods -A
...
linkerd       linkerd-destination-67b9cc8749-xqcbx       4/4     Running   0              69s
linkerd       linkerd-identity-59b46789cc-ntfcx          2/2     Running   0              69s
linkerd       linkerd-proxy-injector-7fc85556bf-vnvw6    1/2     Running   0              69s

$ linkerd check
Linkerd core checks
===================

kubernetes-api
--------------
  can initialize the client
  can query the Kubernetes API

kubernetes-version
------------------
  is running the minimum Kubernetes API version
  is running the minimum kubectl version

linkerd-existence
-----------------
  'linkerd-config' config map exists
  heartbeat ServiceAccount exist
  control plane replica sets are ready
  no unschedulable pods
  control plane pods are ready
  cluster networks contains all pods
  cluster networks contains all services

linkerd-config
--------------
  control plane Namespace exists
  control plane ClusterRoles exist
  control plane ClusterRoleBindings exist
  control plane ServiceAccounts exist
  control plane CustomResourceDefinitions exist
  control plane MutatingWebhookConfigurations exist
  control plane ValidatingWebhookConfigurations exist
  proxy-init container runs as root user if docker container runtime is used

linkerd-identity
----------------
  certificate config is valid
  trust anchors are using supported crypto algorithm
  trust anchors are within their validity period
  trust anchors are valid for at least 60 days
  issuer cert is using supported crypto algorithm
  issuer cert is within its validity period
  issuer cert is valid for at least 60 days
  issuer cert is issued by the trust anchor

linkerd-webhooks-and-apisvc-tls
-------------------------------
  proxy-injector webhook has valid cert
  proxy-injector cert is valid for at least 60 days
  sp-validator webhook has valid cert
  sp-validator cert is valid for at least 60 days
  policy-validator webhook has valid cert
  policy-validator cert is valid for at least 60 days

linkerd-version
---------------
  can determine the latest version
  cli is up-to-date

control-plane-version
---------------------
  can retrieve the control plane version
  control plane is up-to-date
  control plane and cli versions match

linkerd-control-plane-proxy
---------------------------
  control plane proxies are healthy
  control plane proxies are up-to-date
  control plane proxies and cli versions match

Status check results are  

$ linkerd viz install   kubectl apply -f -
namespace/linkerd-viz created
clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-viz-metrics-api created
clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-metrics-api created
serviceaccount/metrics-api created
clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-viz-prometheus created
clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-prometheus created
serviceaccount/prometheus created
clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-viz-tap created
clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-viz-tap-admin created
clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-tap created
clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-tap-auth-delegator created
serviceaccount/tap created
rolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-tap-auth-reader created
secret/tap-k8s-tls created
apiservice.apiregistration.k8s.io/v1alpha1.tap.linkerd.io created
role.rbac.authorization.k8s.io/web created
rolebinding.rbac.authorization.k8s.io/web created
clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-viz-web-check created
clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-web-check created
clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-web-admin created
clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-viz-web-api created
clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-web-api created
serviceaccount/web created
server.policy.linkerd.io/admin created
authorizationpolicy.policy.linkerd.io/admin created
networkauthentication.policy.linkerd.io/kubelet created
server.policy.linkerd.io/proxy-admin created
authorizationpolicy.policy.linkerd.io/proxy-admin created
service/metrics-api created
deployment.apps/metrics-api created
server.policy.linkerd.io/metrics-api created
authorizationpolicy.policy.linkerd.io/metrics-api created
meshtlsauthentication.policy.linkerd.io/metrics-api-web created
configmap/prometheus-config created
service/prometheus created
deployment.apps/prometheus created
service/tap created
deployment.apps/tap created
server.policy.linkerd.io/tap-api created
authorizationpolicy.policy.linkerd.io/tap created
clusterrole.rbac.authorization.k8s.io/linkerd-tap-injector created
clusterrolebinding.rbac.authorization.k8s.io/linkerd-tap-injector created
serviceaccount/tap-injector created
secret/tap-injector-k8s-tls created
mutatingwebhookconfiguration.admissionregistration.k8s.io/linkerd-tap-injector-webhook-config created
service/tap-injector created
deployment.apps/tap-injector created
server.policy.linkerd.io/tap-injector-webhook created
authorizationpolicy.policy.linkerd.io/tap-injector created
networkauthentication.policy.linkerd.io/kube-api-server created
service/web created
deployment.apps/web created
serviceprofile.linkerd.io/metrics-api.linkerd-viz.svc.cluster.local created
serviceprofile.linkerd.io/prometheus.linkerd-viz.svc.cluster.local created

$ kubectl get pods -A
...
linkerd-viz   prometheus-b5865f776-w5ssf                 1/2     Running   0              35s
linkerd-viz   tap-64f5c8597b-rqgbk                       2/2     Running   0              35s
linkerd-viz   tap-injector-7c75cfff4c-wl9mx              2/2     Running   0              34s
linkerd-viz   web-8c444745-jhzr5                         2/2     Running   0              34s

$ linkerd viz dashboard
Linkerd dashboard available at:
http://localhost:50750
Grafana dashboard available at:
http://localhost:50750/grafana
Opening Linkerd dashboard in the default browser
Opening in existing browser session.

$ cat deploy.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 2
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80

$ kubectl apply -f deploy.yaml

$ kubectl get pod nginx-deployment-cd55c47f5-cgxw2 -o yaml
apiVersion: v1
kind: Pod
metadata:
  annotations:
    cni.projectcalico.org/containerID: aee0295dda906f7935ce5c150ae30360005f5330e98c75a550b7cc0d1532f529
    cni.projectcalico.org/podIP: 172.16.36.89/32
    cni.projectcalico.org/podIPs: 172.16.36.89/32
  creationTimestamp: "2022-11-05T19:35:12Z"
  generateName: nginx-deployment-cd55c47f5-
  labels:
    app: nginx
    pod-template-hash: cd55c47f5
  name: nginx-deployment-cd55c47f5-cgxw2
  namespace: default
  ownerReferences:
  - apiVersion: apps/v1
    blockOwnerDeletion: true
    controller: true
    kind: ReplicaSet
    name: nginx-deployment-cd55c47f5
    uid: b604f5c4-f662-4333-aaa0-bd1a2b8b08c6
  resourceVersion: "22979"
  uid: 8fe30214-491b-4753-9fb2-485b6341376c
spec:
  containers:
  - image: nginx:latest
    imagePullPolicy: Always
    name: nginx
    ports:
    - containerPort: 80
      protocol: TCP
    resources:  
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: kube-api-access-2bt6z
      readOnly: true
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  nodeName: k8s-node1
  preemptionPolicy: PreemptLowerPriority
  priority: 0
  restartPolicy: Always
  schedulerName: default-scheduler
  securityContext:  
  serviceAccount: default
  serviceAccountName: default
  terminationGracePeriodSeconds: 30
  tolerations:
  - effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 300
  - effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 300
  volumes:
  - name: kube-api-access-2bt6z
    projected:
      defaultMode: 420
      sources:
      - serviceAccountToken:
          expirationSeconds: 3607
          path: token
      - configMap:
          items:
          - key: ca.crt
            path: ca.crt
          name: kube-root-ca.crt
      - downwardAPI:
          items:
          - fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
            path: namespace
status:
  conditions:
  - lastProbeTime: null
    lastTransitionTime: "2022-11-05T19:35:12Z"
    status: "True"
    type: Initialized
  - lastProbeTime: null
    lastTransitionTime: "2022-11-05T19:35:16Z"
    status: "True"
    type: Ready
  - lastProbeTime: null
    lastTransitionTime: "2022-11-05T19:35:16Z"
    status: "True"
    type: ContainersReady
  - lastProbeTime: null
    lastTransitionTime: "2022-11-05T19:35:13Z"
    status: "True"
    type: PodScheduled
  containerStatuses:
  - containerID: containerd://f088f200315b44cbeed16499aba9b2d1396f9f81645e53b032d4bfa44166128a
    image: docker.io/library/nginx:latest
    imageID: docker.io/library/nginx@sha256:943c25b4b66b332184d5ba6bb18234273551593016c0e0ae906bab111548239f
    lastState:  
    name: nginx
    ready: true
    restartCount: 0
    started: true
    state:
      running:
        startedAt: "2022-11-05T19:35:15Z"
  hostIP: 192.168.2.216
  phase: Running
  podIP: 172.16.36.89
  podIPs:
  - ip: 172.16.36.89
  qosClass: BestEffort
  startTime: "2022-11-05T19:35:12Z"

$ kubectl get deploy nginx-deployment -o yaml   linkerd inject -   kubectl apply -f -

deployment "nginx-deployment" injected

deployment.apps/nginx-deployment configured

$ kubectl get pod nginx-deployment-858bdd545b-55jpf -o yaml
apiVersion: v1
kind: Pod
metadata:
  annotations:
    cni.projectcalico.org/containerID: 1ec3d345f859be8ead0374a7e880bcfdb9ba74a121b220a6fccbd342ac4b7ea8
    cni.projectcalico.org/podIP: 172.16.36.90/32
    cni.projectcalico.org/podIPs: 172.16.36.90/32
    linkerd.io/created-by: linkerd/proxy-injector stable-2.12.2
    linkerd.io/inject: enabled
    linkerd.io/proxy-version: stable-2.12.2
    linkerd.io/trust-root-sha256: 354fe6f49331e8e03d8fb07808e00a3e145d2661181cbfec7777b41051dc8e22
    viz.linkerd.io/tap-enabled: "true"
  creationTimestamp: "2022-11-05T19:44:15Z"
  generateName: nginx-deployment-858bdd545b-
  labels:
    app: nginx
    linkerd.io/control-plane-ns: linkerd
    linkerd.io/proxy-deployment: nginx-deployment
    linkerd.io/workload-ns: default
    pod-template-hash: 858bdd545b
  name: nginx-deployment-858bdd545b-55jpf
  namespace: default
  ownerReferences:
  - apiVersion: apps/v1
    blockOwnerDeletion: true
    controller: true
    kind: ReplicaSet
    name: nginx-deployment-858bdd545b
    uid: 2e618972-aa10-4e35-a7dd-084853673a80
  resourceVersion: "23820"
  uid: 62f1857a-b701-4a19-8996-b5b605ff8488
spec:
  containers:
  - env:
    - name: _pod_name
      valueFrom:
        fieldRef:
          apiVersion: v1
          fieldPath: metadata.name
    - name: _pod_ns
      valueFrom:
        fieldRef:
          apiVersion: v1
          fieldPath: metadata.namespace
    - name: _pod_nodeName
      valueFrom:
        fieldRef:
          apiVersion: v1
          fieldPath: spec.nodeName
    - name: LINKERD2_PROXY_LOG
      value: warn,linkerd=info
    - name: LINKERD2_PROXY_LOG_FORMAT
      value: plain
    - name: LINKERD2_PROXY_DESTINATION_SVC_ADDR
      value: linkerd-dst-headless.linkerd.svc.cluster.local.:8086
    - name: LINKERD2_PROXY_DESTINATION_PROFILE_NETWORKS
      value: 10.0.0.0/8,100.64.0.0/10,172.16.0.0/12,192.168.0.0/16
    - name: LINKERD2_PROXY_POLICY_SVC_ADDR
      value: linkerd-policy.linkerd.svc.cluster.local.:8090
    - name: LINKERD2_PROXY_POLICY_WORKLOAD
      value: $(_pod_ns):$(_pod_name)
    - name: LINKERD2_PROXY_INBOUND_DEFAULT_POLICY
      value: all-unauthenticated
    - name: LINKERD2_PROXY_POLICY_CLUSTER_NETWORKS
      value: 10.0.0.0/8,100.64.0.0/10,172.16.0.0/12,192.168.0.0/16
    - name: LINKERD2_PROXY_INBOUND_CONNECT_TIMEOUT
      value: 100ms
    - name: LINKERD2_PROXY_OUTBOUND_CONNECT_TIMEOUT
      value: 1000ms
    - name: LINKERD2_PROXY_CONTROL_LISTEN_ADDR
      value: 0.0.0.0:4190
    - name: LINKERD2_PROXY_ADMIN_LISTEN_ADDR
      value: 0.0.0.0:4191
    - name: LINKERD2_PROXY_OUTBOUND_LISTEN_ADDR
      value: 127.0.0.1:4140
    - name: LINKERD2_PROXY_INBOUND_LISTEN_ADDR
      value: 0.0.0.0:4143
    - name: LINKERD2_PROXY_INBOUND_IPS
      valueFrom:
        fieldRef:
          apiVersion: v1
          fieldPath: status.podIPs
    - name: LINKERD2_PROXY_INBOUND_PORTS
      value: "80"
    - name: LINKERD2_PROXY_DESTINATION_PROFILE_SUFFIXES
      value: svc.cluster.local.
    - name: LINKERD2_PROXY_INBOUND_ACCEPT_KEEPALIVE
      value: 10000ms
    - name: LINKERD2_PROXY_OUTBOUND_CONNECT_KEEPALIVE
      value: 10000ms
    - name: LINKERD2_PROXY_INBOUND_PORTS_DISABLE_PROTOCOL_DETECTION
      value: 25,587,3306,4444,5432,6379,9300,11211
    - name: LINKERD2_PROXY_DESTINATION_CONTEXT
      value:  
         "ns":"$(_pod_ns)", "nodeName":"$(_pod_nodeName)" 
    - name: _pod_sa
      valueFrom:
        fieldRef:
          apiVersion: v1
          fieldPath: spec.serviceAccountName
    - name: _l5d_ns
      value: linkerd
    - name: _l5d_trustdomain
      value: cluster.local
    - name: LINKERD2_PROXY_IDENTITY_DIR
      value: /var/run/linkerd/identity/end-entity
    - name: LINKERD2_PROXY_IDENTITY_TRUST_ANCHORS
      value:  
        -----BEGIN CERTIFICATE-----
        MIIBiDCCAS6gAwIBAgIBATAKBggqhkjOPQQDAjAcMRowGAYDVQQDExFpZGVudGl0
        eS5saW5rZXJkLjAeFw0yMjExMDUxOTIxMDlaFw0yMzExMDUxOTIxMjlaMBwxGjAY
        BgNVBAMTEWlkZW50aXR5LmxpbmtlcmQuMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcD
        QgAE8AgxbWWa1qgEgN3ykFAOJ3sw9nSugUk1N5Qfvo6jXX/8/TZUW0ddko/N71+H
        EcKc72kK0tlclj8jDi3pzJ4C0KNhMF8wDgYDVR0PAQH/BAQDAgEGMB0GA1UdJQQW
        MBQGCCsGAQUFBwMBBggrBgEFBQcDAjAPBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQW
        BBThSr0yAj5joW7pj/NZPYcfIIepbzAKBggqhkjOPQQDAgNIADBFAiAomg0TVn6N
        UxhOyzZdg848lAvH0Io9Ra/Ef2hxZGN0LgIhAIKjrsgDUqZA8XHiiciYYicxFnKr
        Tw5yj9gBhVAgYCaB
        -----END CERTIFICATE-----
    - name: LINKERD2_PROXY_IDENTITY_TOKEN_FILE
      value: /var/run/secrets/tokens/linkerd-identity-token
    - name: LINKERD2_PROXY_IDENTITY_SVC_ADDR
      value: linkerd-identity-headless.linkerd.svc.cluster.local.:8080
    - name: LINKERD2_PROXY_IDENTITY_LOCAL_NAME
      value: $(_pod_sa).$(_pod_ns).serviceaccount.identity.linkerd.cluster.local
    - name: LINKERD2_PROXY_IDENTITY_SVC_NAME
      value: linkerd-identity.linkerd.serviceaccount.identity.linkerd.cluster.local
    - name: LINKERD2_PROXY_DESTINATION_SVC_NAME
      value: linkerd-destination.linkerd.serviceaccount.identity.linkerd.cluster.local
    - name: LINKERD2_PROXY_POLICY_SVC_NAME
      value: linkerd-destination.linkerd.serviceaccount.identity.linkerd.cluster.local
    - name: LINKERD2_PROXY_TAP_SVC_NAME
      value: tap.linkerd-viz.serviceaccount.identity.linkerd.cluster.local
    image: cr.l5d.io/linkerd/proxy:stable-2.12.2
    imagePullPolicy: IfNotPresent
    lifecycle:
      postStart:
        exec:
          command:
          - /usr/lib/linkerd/linkerd-await
          - --timeout=2m
    livenessProbe:
      failureThreshold: 3
      httpGet:
        path: /live
        port: 4191
        scheme: HTTP
      initialDelaySeconds: 10
      periodSeconds: 10
      successThreshold: 1
      timeoutSeconds: 1
    name: linkerd-proxy
    ports:
    - containerPort: 4143
      name: linkerd-proxy
      protocol: TCP
    - containerPort: 4191
      name: linkerd-admin
      protocol: TCP
    readinessProbe:
      failureThreshold: 3
      httpGet:
        path: /ready
        port: 4191
        scheme: HTTP
      initialDelaySeconds: 2
      periodSeconds: 10
      successThreshold: 1
      timeoutSeconds: 1
    resources:  
    securityContext:
      allowPrivilegeEscalation: false
      readOnlyRootFilesystem: true
      runAsUser: 2102
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: FallbackToLogsOnError
    volumeMounts:
    - mountPath: /var/run/linkerd/identity/end-entity
      name: linkerd-identity-end-entity
    - mountPath: /var/run/secrets/tokens
      name: linkerd-identity-token
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: kube-api-access-9zpnn
      readOnly: true
  - image: nginx:latest
    imagePullPolicy: Always
    name: nginx
    ports:
    - containerPort: 80
      protocol: TCP
    resources:  
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: kube-api-access-9zpnn
      readOnly: true
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  initContainers:
  - args:
    - --incoming-proxy-port
    - "4143"
    - --outgoing-proxy-port
    - "4140"
    - --proxy-uid
    - "2102"
    - --inbound-ports-to-ignore
    - 4190,4191,4567,4568
    - --outbound-ports-to-ignore
    - 4567,4568
    image: cr.l5d.io/linkerd/proxy-init:v2.0.0
    imagePullPolicy: IfNotPresent
    name: linkerd-init
    resources:
      limits:
        cpu: 100m
        memory: 20Mi
      requests:
        cpu: 100m
        memory: 20Mi
    securityContext:
      allowPrivilegeEscalation: false
      capabilities:
        add:
        - NET_ADMIN
        - NET_RAW
      privileged: false
      readOnlyRootFilesystem: true
      runAsNonRoot: true
      runAsUser: 65534
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: FallbackToLogsOnError
    volumeMounts:
    - mountPath: /run
      name: linkerd-proxy-init-xtables-lock
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: kube-api-access-9zpnn
      readOnly: true
  nodeName: k8s-node1
  preemptionPolicy: PreemptLowerPriority
  priority: 0
  restartPolicy: Always
  schedulerName: default-scheduler
  securityContext:  
  serviceAccount: default
  serviceAccountName: default
  terminationGracePeriodSeconds: 30
  tolerations:
  - effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 300
  - effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 300
  volumes:
  - name: kube-api-access-9zpnn
    projected:
      defaultMode: 420
      sources:
      - serviceAccountToken:
          expirationSeconds: 3607
          path: token
      - configMap:
          items:
          - key: ca.crt
            path: ca.crt
          name: kube-root-ca.crt
      - downwardAPI:
          items:
          - fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
            path: namespace
  - emptyDir:  
    name: linkerd-proxy-init-xtables-lock
  - emptyDir:
      medium: Memory
    name: linkerd-identity-end-entity
  - name: linkerd-identity-token
    projected:
      defaultMode: 420
      sources:
      - serviceAccountToken:
          audience: identity.l5d.io
          expirationSeconds: 86400
          path: linkerd-identity-token
status:
  conditions:
  - lastProbeTime: null
    lastTransitionTime: "2022-11-05T19:44:16Z"
    status: "True"
    type: Initialized
  - lastProbeTime: null
    lastTransitionTime: "2022-11-05T19:44:19Z"
    status: "True"
    type: Ready
  - lastProbeTime: null
    lastTransitionTime: "2022-11-05T19:44:19Z"
    status: "True"
    type: ContainersReady
  - lastProbeTime: null
    lastTransitionTime: "2022-11-05T19:44:15Z"
    status: "True"
    type: PodScheduled
  containerStatuses:
  - containerID: containerd://62028867c48aaa726df48249a27c52cd8820cd33e8e5695ad0d322540924754e
    image: cr.l5d.io/linkerd/proxy:stable-2.12.2
    imageID: cr.l5d.io/linkerd/proxy@sha256:787db5055b2a46a3c4318ef3b632461261f81254c8e47bf4b9b8dab2c42575e4
    lastState:  
    name: linkerd-proxy
    ready: true
    restartCount: 0
    started: true
    state:
      running:
        startedAt: "2022-11-05T19:44:16Z"
  - containerID: containerd://8f8ce663c19360a7b6868ace68a4a5119f0b18cd57ffebcc2d19331274038381
    image: docker.io/library/nginx:latest
    imageID: docker.io/library/nginx@sha256:943c25b4b66b332184d5ba6bb18234273551593016c0e0ae906bab111548239f
    lastState:  
    name: nginx
    ready: true
    restartCount: 0
    started: true
    state:
      running:
        startedAt: "2022-11-05T19:44:19Z"
  hostIP: 192.168.2.216
  initContainerStatuses:
  - containerID: containerd://c0417ea9c8418ab296bf86077e81c5d8be06fe9b87390c138d1c5d7b73cc577c
    image: cr.l5d.io/linkerd/proxy-init:v2.0.0
    imageID: cr.l5d.io/linkerd/proxy-init@sha256:7d5e66b9e176b1ebbdd7f40b6385d1885e82c80a06f4c6af868247bb1dffe262
    lastState:  
    name: linkerd-init
    ready: true
    restartCount: 0
    state:
      terminated:
        containerID: containerd://c0417ea9c8418ab296bf86077e81c5d8be06fe9b87390c138d1c5d7b73cc577c
        exitCode: 0
        finishedAt: "2022-11-05T19:44:16Z"
        reason: Completed
        startedAt: "2022-11-05T19:44:15Z"
  phase: Running
  podIP: 172.16.36.90
  podIPs:
  - ip: 172.16.36.90
  qosClass: Burstable
  startTime: "2022-11-05T19:44:15Z"

$ cat deploy.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 2
  template:
    metadata:
      labels:
        app: nginx
      annotations:
        linkerd.io/inject: enabled
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80

$ linkerd viz uninstall   kubectl delete -f -
clusterrole.rbac.authorization.k8s.io "linkerd-linkerd-viz-metrics-api" deleted
clusterrole.rbac.authorization.k8s.io "linkerd-linkerd-viz-prometheus" deleted
clusterrole.rbac.authorization.k8s.io "linkerd-linkerd-viz-tap" deleted
clusterrole.rbac.authorization.k8s.io "linkerd-linkerd-viz-tap-admin" deleted
clusterrole.rbac.authorization.k8s.io "linkerd-linkerd-viz-web-api" deleted
clusterrole.rbac.authorization.k8s.io "linkerd-linkerd-viz-web-check" deleted
clusterrole.rbac.authorization.k8s.io "linkerd-tap-injector" deleted
clusterrolebinding.rbac.authorization.k8s.io "linkerd-linkerd-viz-metrics-api" deleted
clusterrolebinding.rbac.authorization.k8s.io "linkerd-linkerd-viz-prometheus" deleted
clusterrolebinding.rbac.authorization.k8s.io "linkerd-linkerd-viz-tap" deleted
clusterrolebinding.rbac.authorization.k8s.io "linkerd-linkerd-viz-tap-auth-delegator" deleted
clusterrolebinding.rbac.authorization.k8s.io "linkerd-linkerd-viz-web-admin" deleted
clusterrolebinding.rbac.authorization.k8s.io "linkerd-linkerd-viz-web-api" deleted
clusterrolebinding.rbac.authorization.k8s.io "linkerd-linkerd-viz-web-check" deleted
clusterrolebinding.rbac.authorization.k8s.io "linkerd-tap-injector" deleted
role.rbac.authorization.k8s.io "web" deleted
rolebinding.rbac.authorization.k8s.io "linkerd-linkerd-viz-tap-auth-reader" deleted
rolebinding.rbac.authorization.k8s.io "web" deleted
apiservice.apiregistration.k8s.io "v1alpha1.tap.linkerd.io" deleted
mutatingwebhookconfiguration.admissionregistration.k8s.io "linkerd-tap-injector-webhook-config" deleted
namespace "linkerd-viz" deleted
authorizationpolicy.policy.linkerd.io "admin" deleted
authorizationpolicy.policy.linkerd.io "metrics-api" deleted
authorizationpolicy.policy.linkerd.io "proxy-admin" deleted
authorizationpolicy.policy.linkerd.io "tap" deleted
authorizationpolicy.policy.linkerd.io "tap-injector" deleted
server.policy.linkerd.io "admin" deleted
server.policy.linkerd.io "metrics-api" deleted
server.policy.linkerd.io "proxy-admin" deleted
server.policy.linkerd.io "tap-api" deleted
server.policy.linkerd.io "tap-injector-webhook" deleted

$ kubectl get deployments
NAME               READY   UP-TO-DATE   AVAILABLE   AGE
nginx-deployment   2/2     2            2           10m

$ kubectl get deployment nginx-deployment -o yaml   linkerd uninject -   kubectl apply -f -

deployment "nginx-deployment" uninjected

deployment.apps/nginx-deployment configured

$ linkerd uninstall   kubectl delete -f -
clusterrole.rbac.authorization.k8s.io "linkerd-heartbeat" deleted
clusterrole.rbac.authorization.k8s.io "linkerd-linkerd-destination" deleted
clusterrole.rbac.authorization.k8s.io "linkerd-linkerd-identity" deleted
clusterrole.rbac.authorization.k8s.io "linkerd-linkerd-proxy-injector" deleted
clusterrole.rbac.authorization.k8s.io "linkerd-policy" deleted
clusterrolebinding.rbac.authorization.k8s.io "linkerd-destination-policy" deleted
clusterrolebinding.rbac.authorization.k8s.io "linkerd-heartbeat" deleted
clusterrolebinding.rbac.authorization.k8s.io "linkerd-linkerd-destination" deleted
clusterrolebinding.rbac.authorization.k8s.io "linkerd-linkerd-identity" deleted
clusterrolebinding.rbac.authorization.k8s.io "linkerd-linkerd-proxy-injector" deleted
role.rbac.authorization.k8s.io "linkerd-heartbeat" deleted
rolebinding.rbac.authorization.k8s.io "linkerd-heartbeat" deleted
customresourcedefinition.apiextensions.k8s.io "authorizationpolicies.policy.linkerd.io" deleted
customresourcedefinition.apiextensions.k8s.io "httproutes.policy.linkerd.io" deleted
customresourcedefinition.apiextensions.k8s.io "meshtlsauthentications.policy.linkerd.io" deleted
customresourcedefinition.apiextensions.k8s.io "networkauthentications.policy.linkerd.io" deleted
customresourcedefinition.apiextensions.k8s.io "serverauthorizations.policy.linkerd.io" deleted
customresourcedefinition.apiextensions.k8s.io "servers.policy.linkerd.io" deleted
customresourcedefinition.apiextensions.k8s.io "serviceprofiles.linkerd.io" deleted
mutatingwebhookconfiguration.admissionregistration.k8s.io "linkerd-proxy-injector-webhook-config" deleted
validatingwebhookconfiguration.admissionregistration.k8s.io "linkerd-policy-validator-webhook-config" deleted
validatingwebhookconfiguration.admissionregistration.k8s.io "linkerd-sp-validator-webhook-config" deleted
namespace "linkerd" deleted

$ kubectl get pods -A
NAMESPACE     NAME                                       READY   STATUS    RESTARTS        AGE
default       nginx-deployment-cd55c47f5-99xf2           1/1     Running   0               82s
default       nginx-deployment-cd55c47f5-tt58t           1/1     Running   0               86s
kube-system   calico-kube-controllers-59697b644f-7fsln   1/1     Running   2 (3h39m ago)   7d1h
kube-system   calico-node-6ptsh                          1/1     Running   2 (3h39m ago)   7d1h
kube-system   calico-node-7x5j8                          1/1     Running   2 (3h39m ago)   7d1h
kube-system   calico-node-qlnf6                          1/1     Running   2 (3h39m ago)   7d1h
kube-system   coredns-565d847f94-79jlw                   1/1     Running   2 (3h39m ago)   7d2h
kube-system   coredns-565d847f94-fqwn4                   1/1     Running   2 (3h39m ago)   7d2h
kube-system   etcd-k8s-master                            1/1     Running   2 (3h39m ago)   7d2h
kube-system   kube-apiserver-k8s-master                  1/1     Running   2 (3h39m ago)   7d2h
kube-system   kube-controller-manager-k8s-master         1/1     Running   2 (3h39m ago)   7d2h
kube-system   kube-proxy-4n9b7                           1/1     Running   2 (3h39m ago)   7d2h
kube-system   kube-proxy-k4rzv                           1/1     Running   2 (3h39m ago)   7d2h
kube-system   kube-proxy-lz2dd                           1/1     Running   2 (3h39m ago)   7d2h
kube-system   kube-scheduler-k8s-master                  1/1     Running   2 (3h39m ago)   7d2h

• Part one, Culture has been captured (chapters 1 to 11), is a series of case studies that focus on different examples of market failure. The specific sectors analysed are the book market, the news media, the music industry, Hollywood, the mobile apps industry and the online video platforms.
• Part two, Braking anticompetitive flywheels (chapters 12 to 19), looks at different solutions to try to fix these failures.

• Transparency Rights giving creative workers a way to audit the companies that sell their work and make sure they are paid what they are due.
• Collective Action
• Time Limits on Copyright Contracts making sure creators that sell their copyrights to publishers or labels can get them back after a reasonable period of time.
• Radical Interoperability forcing tech giants to make their walled-gardens interoperable.
• Minimum Wages for Creative Work enforcing minimum legal rates for workers in certain domains, like what is already done for screenplays in the US by members of the Writers Guild of America.
• Collective Ownership

• Stick them all in one package:
  • builtins/builtins.go
• Create a package and group them:
  • builtins/maths.go
  • builtins/string.go
  • .. etc

func ABS( args []primitive.Object) (primitive.Object, error)  
.. 
func COS( args []primitive.Object) (primitive.Object, error)  
.. 
func SIN( args []primitive.Object) (primitive.Object, error)  
.. 
func TAN( args []primitive.Object) (primitive.Object, error)  
.. 

• Golang linter that reports failures if functions aren't in alphabetical order
  • https://github.com/skx/alphavet

"And sin, young man, is when you treat people as things. Including yourself. That's what sin is." "It's a lot more complicated than that " "No. It ain't. When people say things are a lot more complicated than that, they means they're getting worried that they won t like the truth. People as things, that's where it starts."

"No, my Uncle Igor thtill workth for him. Been thtruck by lightning three hundred timeth and thtill putth in a full night'th work."

"Ach, hins tak yar scaggie, yer dank yowl callyake!"

Processing an LDPC Message Checking if a message is correct is fairly straightforward with LDPC (as with encoding, I ll note). As a quick refresher given the LDPC H (check) matrix of width N, you can check your message vector (msg) of length N by multipling H and msg, and checking if the output vector is all zero.

 // scheme contains our G (generator) and
 // H (check) matrices.
 scheme :=  G: Matrix ... , H: Matrix ... 
// msg contains our LDPC message (data and
 // check bits).
 msg := Vector ... 
// N is also the length of the encoded
 // msg vector after check bits have been
 // added.
 N := scheme.G.Width
// Now, let's generate our 'check' vector.
 ch := Multiply(scheme.H, msg)

We can now see if the message is correct or not:

 // if the ch vector is all zeros, we know
 // that the message is valid, and we don't
 // need to do anything.
 if ch.IsZero()  
// handle the case where the message
 // is fine as-is.
 return ...
 
// Expensive decode here

This is great for getting a thumbs up / thumbs down on the message being correct, but correcting errors still requires pulling the LDPC matrix values from the g (generator) matrix out, building a bipartite graph, and iteratively reprocessing the bit values, until constraints are satisfied and the message has been corrected. This got me thinking - what is the output vector when it s not all zeros? Since 1 values in the output vector indicates consistency problems in the message bits as they relate to the check bits, I wondered if this could be used to speed up my LDPC decoder. It appears to work, so this post is half an attempt to document this technique before I put it in my hot path, and half a plea for those who do like to talk about FEC to tell me what name this technique actually is.

k-Bit Brute Forcing Given that the output Vector s non-zero bit pattern is set due to the position of errors in the message vector, let s use that fact to build up a table of k-Bit errors that we can index into.

 // for clarity's sake, the Vector
 // type is being used as the lookup
 // key here, even though it may
 // need to be a hash or string in
 // some cases.
 idx := map[Vector]int 
for i := 0; i < N; i++  
// Create a vector of length N
 v := Vector 
v.FlipBit(i)
// Now, let's use the generator matrix to encode
 // the data with checksums, and then use the
 // check matrix on the message to figure out what
 // bit pattern results
 ev := Multiply(scheme.H, Multiply(v, scheme.G))
idx[ev] = i
 

This can be extended to multiple bits (hence: k-Bits), but I ve only done one here for illustration. Now that we have our idx mapping, we can now go back to the hot path on Checking the incoming message data:

 // if the ch vector is all zeros, we know
 // that the message is valid, and we don't
 // need to do anything.
 if ch.IsZero()  
// handle the case where the message
 // is fine as-is.
 return ...
 
errIdx, ok := idx[ch]
if ok  
msg.FlipBit(errIdx)
// Verify the LDPC message using
 // H again here.
 return ...
 
// Expensive decode here

Since map lookups wind up a heck of a lot faster than message-passing bit state, the hope here is this will short-circuit easy to solve errors for k-Bits, for some value of k that the system memory can tolerate.

Does this work? Frankly I have no idea. I ve written a small program and brute forced single-bit errors in all bit positions using random data to start with, and I ve not been able to find any collisions in the 1-bit error set, using the LDPC matrix from 802.3an-2006. Even if I was to find a collision for a higher-order k-Bit value, I m tempted to continue with this approach, and treat each set of bits in the Vector s bin (like a hash-table), checking the LDPC validity after each bit set in the bin. As long as the collision rate is small enough, it should be possible to correct k-Bits of error faster than the more expensive Belief Propagation approach. That being said, I m not entirely convinced collisions will be very common, but it ll take a bit more time working through the math to say that with any confidence. Have you seen this approach called something official in publications? See an obvious flaw in the system? Send me a tip, please!

• Lokalize: Dictionaries are now available via the hunspell content snap! To enable:

• Ktorrent: First time release on both amd64 and arm64! Closes bug 412734
• Kdenlive: Glaximate animations are now available! Closes bug 459338

Kdenlive with Glaximate animations!

• Okular
• Kgeography
• Kpatience
• Palapeli: Now launches! Closes bug 426177
• Krdc: Now on arm64!
• Kdiamond: Now with sound!

Changes in littler version 0.3.17 (2022-10-29)

• Changes in package
  • An internal function prototype was updated for clang-15.
• Changes in examples
  • The install2.r and installBioc. were updated for an update in R-devel (Tatsuya Shima and Dirk in #104).

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

1. a home-made Wireguard VPN, deployed with Puppet
2. a Wireguard VPN overlay, with Tailscale or equivalent
3. IPv6, native or with tunnels

Wireguard Puppet modules As is (unfortunately) typical with Puppet, I found multiple different modules to talk with Wireguard.

module	score	downloads	release	stars	watch	forks	license	docs	contrib	issue	PR	notes
halyard	3.1	1,807	2022-10-14	0	0	0	MIT	no				requires firewall and Configvault_Write modules?
voxpupuli	5.0	4,201	2022-10-01	2	23	7	AGPLv3	good	1/9	1/4	1/61	optionnally configures ferm, uses systemd-networkd, recommends systemd module with manage_systemd to true, purges unknown keys
abaranov	4.7	17,017	2021-08-20	9	3	38	MIT	okay	1/17	4/7	4/28	requires pre-generated private keys
arrnorets	3.1	16,646	2020-12-28	1	2	1	Apache-2	okay	1	0	0	requires pre-generated private keys?

The voxpupuli module seems to be the most promising. The abaranov module is more popular and has more contributors, but it has more open issues and PRs. More critically, the voxpupuli module was written after the abaranov author didn't respond to a PR from the voxpupuli author trying to add more automation (namely private key management). It looks like setting up a wireguard network would be as simple as this on node A:

wireguard::interface   'wg0':
  source_addresses => ['2003:4f8:c17:4cf::1', '149.9.255.4'],
  public_key       => $facts['wireguard_pubkeys']['nodeB'],
  endpoint         => 'nodeB.example.com:53668',
  addresses        => [ 'Address' => '192.168.123.6/30', , 'Address' => 'fe80::beef:1/64' ,],
 

This configuration come from this pull request I sent to the module to document how to use that fact. Note that the addresses used here are examples that shouldn't be reused and do not confirm to RFC5737 ("IPv4 Address Blocks Reserved for Documentation", 192.0.2.0/24 (TEST-NET-1), 198.51.100.0/24 (TEST-NET-2), and 203.0.113.0/24 (TEST-NET-3)) or RFC3849 ("IPv6 Address Prefix Reserved for Documentation", 2001:DB8::/32), but that's another story. (To avoid boostrapping problems, the resubmit-facts configuration could be used so that other nodes facts are more immediately available.) One problem with the above approach is that you explicitly need to take care of routing, network topology, and addressing. This can get complicated quickly, especially if you have lots of devices, behind NAT, in multiple locations (which is basically my life at home, unfortunately). Concretely, basic Wireguard only support one peer behind NAT. There are some workarounds for this, but they generally imply a relay server of some sort, or some custom registry, it's kind of a mess. And this is where overlay networks like Tailscale come in.

Tailscale Tailscale is basically designed to deal with this problem. It's not fully opensource, but pretty close, and they have an interesting philosophy behind that. The client is opensource, and there is an opensource version of the server side, called headscale. They have recently (late 2022) hired the main headscale developer while promising to keep supporting it, which is pretty amazing. Tailscale provides an overlay network based on Wireguard, where each peer basically has a peer-to-peer encrypted connexion, with automatic key rotation. They also ship a multitude of applications and features on top of that like file sharing, keyless SSH access, and so on. The authentication layer is based on an existing SSO provider, you don't just register with Tailscale with new account, you login with Google, Microsoft, or GitHub (which, really, is still Microsoft). The Headscale server ships with many features out of that:

• Full "base" support of Tailscale's features
• Configurable DNS
  • Split DNS
  • MagicDNS (each user gets a name)
• Node registration
  • Single-Sign-On (via Open ID Connect)
  • Pre authenticated key
• Taildrop (File Sharing)
• Access control lists
• Support for multiple IP ranges in the tailnet
• Dual stack (IPv4 and IPv6)
• Routing advertising (including exit nodes)
• Ephemeral nodes
• Embedded DERP server (AKA NAT-to-NAT traversal)

Neither project (client or server) is in Debian (RFP 972439 for the client, none filed yet for the server), which makes deploying this for my use case rather problematic. Their install instructions are basically a curl bash but they also provide packages for various platforms. Their Debian install instructions are surprisingly good, and check most of the third party checklist we're trying to establish. (It's missing a pin.) There's also a Puppet module for tailscale, naturally. What I find a little disturbing with Tailscale is that you not only need to trust Tailscale with authorizing your devices, you also basically delegate that trust also to the SSO provider. So, in my case, GitHub (or anyone who compromises my account there) can penetrate the VPN. A little scary. Tailscale is also kind of an "all or nothing" thing. They have MagicDNS, file transfers, all sorts of things, but those things require you to hook up your resolver with Tailscale. In fact, Tailscale kind of assumes you will use their nameservers, and have suffered great lengths to figure out how to do that. And naturally, here, it doesn't seem to work reliably; my resolv.conf somehow gets replaced and the magic resolution of the ts.net domain fails. (I wonder why we can't opt in to just publicly resolve the ts.net domain. I don't care if someone can enumerate the private IP addreses or machines in use in my VPN, at least I don't care as much as fighting with resolv.conf everywhere.) Because I mostly have access to the routers on the networks I'm on, I don't think I'll be using tailscale in the long term. But it's pretty impressive stuff: in the time it took me to even review the Puppet modules to configure Wireguard (which is what I'll probably end up doing), I was up and running with Tailscale (but with a broken DNS, naturally). (And yes, basic Wireguard won't bring me DNS either, but at least I won't have to trust Tailscale's Debian packages, and Tailscale, and Microsoft, and GitHub with this thing.)

IPv6 IPv6 is actually what is supposed to solve this. Not NAT port forwarding crap, just real IPs everywhere. The problem is: even though IPv6 adoption is still growing, it's kind of reaching a plateau at around 40% world-wide, with Canada lagging behind at 34%. It doesn't help that major ISPs in Canada (e.g. Bell Canada, Videotron) don't care at all about IPv6 (e.g. Videotron in beta since 2011). So we can't rely on those companies to do the right thing here. The typical solution here is often to use a tunnel like HE's tunnelbroker.net. It's kind of tricky to configure, but once it's done, it works. You get end-to-end connectivity as long as everyone on the network is on IPv6. And that's really where the problem lies here; the second one of your nodes can't setup such a tunnel, you're kind of stuck and that tool completely breaks down. IPv6 tunnels also don't give you the kind of security a VPN provides as well, naturally. The other downside of a tunnel is you don't really get peer-to-peer connectivity: you go through the tunnel. So you can expect higher latencies and possibly lower bandwidth as well. Also, HE.net doesn't currently charge for this service (and they've been doing this for a long time), but this could change in the future (just like Tailscale, that said). Concretely, the latency difference is rather minimal, Google:

--- ipv6.l.google.com ping statistics ---
10 packets transmitted, 10 received, 0,00% packet loss, time 136,8ms
RTT[ms]: min = 13, median = 14, p(90) = 14, max = 15
--- google.com ping statistics ---
10 packets transmitted, 10 received, 0,00% packet loss, time 136,0ms
RTT[ms]: min = 13, median = 13, p(90) = 14, max = 14

In the case of GitHub, latency is actually lower, interestingly:

--- ipv6.github.com ping statistics ---
10 packets transmitted, 10 received, 0,00% packet loss, time 134,6ms
RTT[ms]: min = 13, median = 13, p(90) = 14, max = 14
--- github.com ping statistics ---
10 packets transmitted, 10 received, 0,00% packet loss, time 293,1ms
RTT[ms]: min = 29, median = 29, p(90) = 29, max = 30

That is because HE.net peers directly with my ISP and Fastly (which is behind GitHub.com's IPv6, apparently?), so it's only 6 hops away. While over IPv4, the ping goes over New York, before landing AWS's Ashburn, Virginia datacenters, for a whopping 13 hops... I managed setup a HE.net tunnel at home, because I also need IPv6 for other reasons (namely debugging at work). My first attempt at setting this up in the office failed, but now that I found the openwrt.org guide, it worked... for a while, and I was able to produce the above, encouraging, mini benchmarks. Unfortunately, a few minutes later, IPv6 just went down again. And the problem with that is that many programs (and especially OpenSSH) do not respect the Happy Eyeballs protocol (RFC 8305), which means various mysterious "hangs" at random times on random applications. It's kind of a terrible user experience, on top of breaking the one thing it's supposed to do, of course, which is to give me transparent access to all the nodes I maintain. Even worse, it would still be a problem for other remote nodes I might setup where I might not have acess to the router to setup the tunnel. It's also not absolutely clear what happens if you setup the same tunnel in two places... Presumably, something is smart enough to distribute only a part of the /48 block selectively, but I don't really feel like going that far, considering how flaky the setup is already.

Other options If this post sounds a little biased towards IPv6 and Wireguard, it's because it is. I would like everyone to migrate to IPv6 already, and Wireguard seems like a simple and sound system. I'm aware of many other options to make VPNs. So before anyone jumps in and says "but what about...", do know that I have personnally experimented with:

• tinc: nice, automatic meshing, used for the Montreal mesh, serious design flaws in the crypto that make it generally unsafe to use; supposedly, v1.1 (or 2.0?) will fix this, but that's been promised for over a decade by now
• ipsec, specifically strongswan: hard to configure (especially configure correctly!), harder even to debug, otherwise really nice because transparent (e.g. no need for special subnets), used at work, but also considering a replacement there because it's a major barrier to entry to train new staff
• OpenVPN: mostly used as a client for [VPN service][]s like Riseup VPN or Mullvad, mostly relevant for client-server configurations, not really peer-to-peer, shared secrets or TLS, kind of an hassle to maintain, see also SoftEther for an alternative implementation

All of those solutions have significant problems and I do not wish to use any of those for this project. Also note that Tailscale is only one of many projects laid over Wireguard to do that kind of thing, see this LWN review for others (basically NetbBird, Firezone, and Netmaker).

Future work Those are options that came up after writing this post, and might warrant further examination in the future.

• Meshbird, a "distributed private networking" with little information about how it actually works other than "encrypted with strong AES-256"
• Nebula, "A scalable overlay networking tool with a focus on performance, simplicity and security", written by Slack people to replace IPsec, docs, runs as an overlay for Slack's 50k node network, only packaged in Debian experimental, lagging behind upstream (1.4.0, from May 2021 vs upstream's 1.6.1 from September 2022), requires a central CA, Golang, I'm in "wait and see" mode for now
• n2n: "layer two VPN", seems packaged in Debian but inactive
• ouroboros: "peer-to-peer packet network prototype", sounds and seems complicated
• QuickTUN is interesting because it's just a small wrapper around NaCL, and it's in Debian... but maybe too obscure for my own good
• unetd: Wireguard-based full mesh networking from OpenWRT, not in Debian
• vpncloud: "high performance peer-to-peer mesh VPN over UDP supporting strong encryption, NAT traversal and a simple configuration", sounds interesting, not in Debian
• Yggdrasil: actually a pretty good match for my use case, but I didn't think of it when starting the experiments here; packaged in Debian, with the Golang version planned, Puppet module; major caveat: nodes exposed publicly inside the global mesh unless configured otherwise (firewall suggested), requires port forwards, alpha status

Conclusion Right now, I'm going to deploy Wireguard tunnels with Puppet. It seems like kind of a pain in the back, but it's something I will be able to reuse for work, possibly completely replacing strongswan. I have another Puppet module for IPsec which I was planning to publish, but now I'm thinking I should just abort that and replace everything with Wireguard, assuming we still need VPNs at work in the future. (I have a number of reasons to believe we might not need any in the near future anyways...)