Three Ancient Civilizations
I have been reading books for a long time but somehow I don t know how or when I realized that there are three medieval civilizations that time and again seem to fascinate the authors, either American or European. The three civilizations that do get mentioned every now and then are Egyptian, Greece and Roman and I have no clue as to why. Another two civilizations closely follow them, Mesopotian and Sumerian Civilizations. Why most of the authors irrespective of genre are mystified by these 5 civilizations is beyond me but they also conjure lot of imagination. Now if I was on the right side of 40, maybe 22-25 onwards and had the means or the opportunity or both, I would have gone and tried to learn as much as I can about these various civilizations. There is still a lot of enigma attached and it seems that the official explanations of how these various civilizations ended seems too good to be true or whatever. One of the more interesting points has been how Greece mythology were subverted and flipped to make Roman mythology. Apparently, many Greek gods have a Roman aspect and the qualities are opposite to the Greek aspect. I haven t learned the reason why it is so, yet.
Apart from the Iziko Natural Museum in South Africa which did have its share of wonders (the whole South African experience was surreal) nowhere I have witnessed stuff from the above other than in Africa. The whole thing seemed just surreal. I could go on but as I don t know what is real and what is mythology when it comes to various cultures including whatever little I experienced in Africa.
Recent History
Having said the above, I also find that many Indians somehow do not either know or are not interested in understanding recent history. I am talking of the time between 1800s and 1950 s. British apologist Mr. William Dalrymple in his book Anarchy has shared how the East Indian Company looted India and gave less than half back to the British. So where did the rest of the money go ? It basically went to the tax free havens around UK. That tale is very much similar as to how the Axis gold was used to have an entity called Switzerland from scratch. There have been books as well as couple of miniseries or two that document that fact. But for me this is not the real story, this is more of a side dish per-se.
The real story perhaps is how the EU peace project was born. Because of Hitler s rise and the way World War 2 happened, the Allies knew that they couldn t subjugate Germany through another humiliation as they had after World War 1, who knows another Hitler might come. So while they did fine the Germans, they also helped them via the Marshall plan. Germany also realized that it had been warring with other European countries for over a thousand years as well as quite a few other countries. The first sort of treaty immediately in that regard was the Western Union Alliance . While on the face of it, it was a defense treaty between sovereign powers. Interestingly, as can be seen UK at that point in time wanted more countries to be part of the Treaty of Dunkirk. This was quickly followed 2 years later by the Treaty of London which made way for the Council of Europe to be formed. The reason I am sharing is because a lot of Indians whom I meet on SM do not know that UK was instrumental front and center for the formation of European Union (EU). Also they perhaps don t realize that after World War 2, UK was greatly diminished both financially and militarily. That is the reason it gave back its erstwhile colonies including India and other countries. If UK hadn t become a part of EU they would have been called the poor man of Europe as they had been called few hundred years ago.
In fact, after Brexit, UK has been the only nation that has fallen on the dire times as much as it has. They have practically no food, supermarkets running out of veggies and whatnot. electricity sky-high as they don t want to curtail profits of the gas companies which in turn donate money to Tory coffers. Sadly, most of my brethen do not know that hence I have had to share it. The EU became a power in its own right due to Gravity model of trade. The U.S. is attempting the same thing and calls it near off-shoring.
Tesla, Investor Day Presentation, Toyota and Free Software
The above brings it nicely about the Tesla Investor Day that happened couple of weeks back. The biggest news though was broken just 24 hours earlier when the governor of Monterrey] shared how Giga New Mexico would be happening and shared some site photographs. There have been bits of news on that off-and-on since then. Toyota meanwhile has been putting lot of anti-EV posters and whatnot. In fact, India seems to be mirroring Japan in a lot of ways, the only difference is Japan is still superior economically than India. I do sincerely hope that at least Japan can get out of its lost decades. I have asked privately if any of the Japanese translators would be willing to translate the anti-EV poster so we may all know.
Anti-EV poster by Toyota
Urinal outside UK Embassy
About a week back few people chanting Khalistan entered the Indian Embassy and showed the flags. This was in the UK. Now in a retaliatory move against a friendly nation, we want to make a Urinal. after making a security downgrade for the Embassy. The pettiness being shown by Indian Govt. especially when it has very few friends. There are also plans to do the same for the U.S. and other embassies as well. I do not know when we will get common sense.
Holi
Just a few weeks back, Holi happened. It used to be a sweet and innocent festival. But from the last few years, I have been hearing and seeing sexual harassment on the rise. In fact, saw quite a few lewd posts written to women on Holi or verge of Holi and also videos of the same. One of the most shameful incidents occurred with a Japanese tourist. She was not only sexually molested but also terrorized that if she were to report then she could be raped and murdered. She promptly left Delhi. Once it was reported in mass media, Delhi Police tried to show it was doing something. FWIW, Delhi s crime against women stats have been at record high and conviction at record low.
Ecommerce Rules being changed, logistics gonna be tough.
Just today there have been changes in Ecommerce rules (again) and this is gonna be a pain for almost all companies big and small with the exception of Adani and Ambani. Almost all players including the Tatas have called them out. Of course, all such laws have been passed without debate. In such a scenario, small startups like these cannot hope to grow their business.
Inertial Measurement Unit or Full Body Tracking with cheap hardware.
Apparently, a whole host of companies are looking at 3-D tracking using cheap hardware apparently known as IMU sensors. Sooner than later cheap 3-D glasses and IMU sensors should explode the 3-D market worldwide. There is a huge potential and upside to it and will probably overtake smartphones as well. But then the danger will be of our thoughts, ideas, nightmares etc. to be shared without our consent. The more we tap into a virtual world, what stops anybody from tapping into our brain and practically stealing our identity in more than one way. I dunno if there are any Debian people or FSF projects working on the above. Even Laws can only do so much, until and unless there are alternative places and ways it would be difficult to say the least
The importance of clean, fresh indoor air is one of the most tangible takeaways of the Covid-19 pandemic. In addition to being an effective risk mitigation strategy for reducing the spread of respiratory illnesses, clean, fresh air is necessary to enable effective cognitive performance.
Monitoring indoor air quality is relatively easy to do, but traditionally has not been a key focus. I believe air quality monitoring should be accessible for any indoor space, and for highly occupied indoor spaces should be provided on a continuous basis. This post explores the need and an opportunity for a business that can accelerate the adoption of ventilation monitoring through the following topics:
The importance of indoor air quality
Clean, fresh air is fundamental to life and health. That might sound obvious, but unfortunately being obvious is not enough to ensure the air we breathe is in fact always clean and healthy. Repeated studies have revealed that in many cases the air you re breathing at school, in the bus or at work and probably also at home falls well below the ideal of what clean, fresh air should be.
Unclean air has potential long-term health impacts and has also been shown to lower cognitive performance impacting the ability to learn and work as well as increasing the risk of transmission of respiratory illnesses like Covid-19 and the flu.
Ventilation (replacing old stale air with clean fresh air) is the most effective and economical method of improving and maintaining high indoor air quality. Most New Zealand buildings (including schools and houses) are designed to rely on manual ventilation (opening windows), while newer buildings, often including larger or commercial buildings may use mechanical ventilation involving fans and ducts. Mechanical ventilation including filtration may also be required in situations where the outdoor air is not clean and fresh such as in a city or next to a busy intersection.
Observing the invisible
Overall air quality is a complex topic involving many contributing factors, many of which are invisible and not perceptible to us until well after adverse effects or irritation occur. This complexity and lack of visible signal is a large contributing factor to the ignorance and lack of attention towards indoor air quality that is prevalent in most buildings and indoor spaces today. Our attention is biased towards the risks that we can see, and this default bias has not been helped by hesitation and resistance to the idea that aerosol transmission and air quality is an important factor in preventing disease transmission that has only recently started to change. Zeynep Tufekci has a great overview that provides fascinating context for how an overreaction to the early incorrect theories of bad air and miasma causing disease contributed to aerosol transmission and air quality being incorrectly neglected for so long.
Correcting this history of inattention to indoor air quality is going to take time and effort, but one significant step that we can take to help start the journey to ensuring all indoor spaces have clean, healthy air is to make the invisible part visible. The concentration of carbon dioxide (CO2) in a space is an incredibly effective and easy to measure proxy for the ventilation of a space. The atmospheric background level of CO2 is around 420 parts per million (ppm), while our exhaled breath has concentrations as high as 40,000 ppm. Without effective ventilation, one or more people breathing in an enclosed space will rapidly lead to an observable increase in CO2 concentration, which in turn provides a signal that the ventilation is insufficient and needs to be improved.
Monitoring CO2 and improving ventilation is not a panacea for all possible air quality issues, but for the majority of buildings and indoor spaces, using CO2 as a proxy for ventilation and increasing ventilation when CO2 levels rise above recommended levels is a simple, effective and achievable approach that will deliver improvements in cognitive performance and reduction in the risk of disease transmission with few, if any, downsides or risks. See this Public Health Communication Centre briefing for a more detailed explanation.
Adding clean air to our hygiene practices
We have well established expectations of hygiene for the food we eat and the water we drink and these expectations are codified in regulations that ensure those providing these services do so in a way that gives us confidence that we re not going to be at risk of illness. You may recall seeing food grade ratings prominently displayed on the walls of restaurants and cafes that you visit as an example of this.
Why should the air we breathe be treated any differently?
I think there is a strong argument that indoor air quality deserves regulation, both of the absolute quality of the air and ensuring that the practices and achieved air quality are clearly advertised and available. Ventilation monitoring via measurement of CO2 concentration provides an effective and achievable method that can be used to achieve this, and countries like Belgium and Japan are already starting to regulate indoor air quality. In the UK, the independent SAGE group of scientists has published Scores on the Doors , a proposal which demonstrates how CO2 monitoring can be helpful in providing information about the air quality of indoor spaces.
Unfortunately there is no movement in any of these directions in New Zealand yet, and no sign that regulation or even a basic campaign to raise awareness of ventilation and air quality is even being planned. This is disappointing, but even if such work was planned, it would still require appropriate ventilation monitoring products and services to enable it, and while there are some options available, it is not a fully solved space yet.
Existing ventilation monitoring options
Until recently the available offerings for ventilation monitoring have sat at two distinct ends of the price and quality spectrum:
Handheld air quality meters advertised as measuring CO2, but in reality reporting only an approximation. These meters do not contain actual CO2 sensors, and only approximate CO2 levels based on measurements of other components of the air. While cheap (often less than $100), these meters are not useful for providing reliable data that can be systematically used to assess and improve ventilation and should be avoided.
High-end building management systems (BMS), and industrial measurement products targeted at large buildings such as offices or commercial applications such as food production. These systems require specialist installation, often integrated with large whole-building air conditioning systems. These systems, if appropriately configured, can be a great solution for the types of buildings and spaces that can afford them, but by their nature and cost, they do not offer a solution for the majority of smaller buildings and indoor spaces where we tend to spend a lot of our time.
Over the last few years a growing number of companies have developed products that fit in between the unreliable air quality meters and the expensive BMS/industrial measurement products. Promising NZ-based options in this space include Air Suite, Tether and Monkeytronics. These products are wall mountable, resemble a smoke alarm and utilise a WiFi network to report their measurements to a supporting web service. Pricing varies between $200 and $300 ex GST per unit.
Aranet, while not NZ based, provides a handheld monitor the Aranet4 Home, which is well regarded for quality and accuracy. Aranet4 Home devices are the most expensive in this space, retailing at $386 ex GST and offer a clunkier and less convenient set of connectivity options via a Bluetooth connection to an associated phone. To obtain similar reporting functionality to the other products requires upgrading to their Pro model and purchasing a separate base station at a combined cost of $1255 ex GST.
Outside the commercial product offerings are a number of open source DIY options, which can be built by anyone with basic electronics knowledge. AirGradient is a leading example based in Thailand, and within New Zealand Oliver Seiler s CO2 Monitor provides similar functionality. These open source options have a parts cost in the $100-$150 range, depending on volume built and provide high-quality measurements via trusted CO2 sensors while also offering huge flexibility in terms of how they operate, interact with users and potential supporting web services.
An opportunity: Small businesses and organisations
While a growing number of high-quality CO2 monitors has the potential to help drive increased adoption of ventilation monitoring, the plethora of small businesses and organisations that own, operate and manage many of the indoor spaces we visit on a day-to-day basis do not appear to be well served by these existing products.
To deploy ventilation monitoring a small business or organisation needs to first become aware of the need or demand for it, and then have a simple and easy path to acquire and install the monitor and access the data. Little to no marketing or demand generation appears to be targeted towards this market from the existing businesses and tellingly, several of the products are not directly available for sale, requiring interaction with a salesperson to purchase. This indicates a focus on selling to larger customers who have a campus or portfolio of buildings and will purchase in larger quantities than the typical small business or organisation will.
Small businesses and organisations are likely to occupy smaller buildings and spaces where manual ventilation is the prevalent method of improving and maintaining air quality. Maintaining clean, fresh air via manual ventilation requires the occupants of the space to receive an obvious and straightforward signal when action (opening windows, etc) is required. While the products above all tend to provide some form of local feedback and display in the room, the indication provided and notification of when to take action is less obvious and prominent than would be ideal in a situation where manual ventilation is being relied upon.
Informally testing this opportunity with family and friends running small businesses over the last few months has resulted in promising feedback. One particular success story was the discovery of a fresh air duct on the air conditioning unit in a small office that had never been connected to the outside air and was simply recirculating air from the ceiling space back into an office! The resulting stuffiness and poor air quality had been noticed, but without the clear indication from the CO2 monitor that the air conditioning was actually making things worse, rather than better, the underlying issue had not been understood. With the issue fixed and the duct now connected, that business is now enjoying much more productive and healthy working conditions.
Next steps
Many small businesses and organisations are likely to have poor air quality and opportunities for improvement similar to the example above that are waiting to be found and fixed, and the existing products available are neither focused or ideal for the needs of this market.
I have spent some time over the past six months building a basic CO2 monitoring service that I have used to deploy ventilation monitoring to our local school, and a few other local businesses. There are a number of challenges that still need to be addressed in order to scale the business up, but I think there is a reasonable chance that I can build a viable business that offers an attractive and useful solution that would accelerate the deployment of ventilation monitoring for small businesses and organisations.
In an upcoming post, I will explain the foundations of the service that I have built to date, the challenges that need to be overcome and how I plan to evolve the service from the current prototype into a sustainable, bootstrapped business.
I recently mentioned on the internet I did work in this direction and a friend of mine asked me to write a blogpost on this. I didn t blog for a long time (keeping all the goodness for myself hehe), so here we go. To set the scene, let s assume we want to make an exectuable binary for x86_64 Linux that s supposed to be extremely portable. It should work on both Debian and Arch Linux. It should work on systems without glibc like Alpine Linux. It should even work in a FROM scratch Docker container. In a more serious setting you would statically link musl-libc with your Rust program, but today we re in a silly-goofy mood so we re going to try to make this work without a libc. And we re also going to use Rust for this, more specifically the stable release channel of Rust, so this blog post won t use any nightly-only features that might still change/break. If you re using a Rust 1.0 version that was recent at the time of writing or later (>= 1.68.0 according to my computer), you should be able to try this at home just fine .
This tutorial assumes you have no prior programming experience in any programming language, but it s going to involve some x86_64 assembly. If you already know what a syscall is, you ll be just fine. If this is your first exposure to programming you might still be able to follow along, but it might be a wild ride.
If you haven t already, install rustup (possibly also available in your package manager, who knows?)
# when asked, press enter to confirm default settings
curl --proto'=https'--tlsv1.2 -sSf https://sh.rustup.rs sh
This is going to install everything you need to use Rust on Linux (this tutorial assumes you re following along on Linux btw). Usually it s still using a system linker (by calling the cc binary, and errors out if none is present), but instead we re going to use rustup to install an additional target:
rustup target add x86_64-unknown-none
I don t know if/how this is made available by Linux distributions, so I recommend following along with rust installed from rustup.
Anyway, we re creating a new project with cargo, this creates a new directory that we can then change into (you might ve done this before):
cargo new hack-the-planet
cd hack-the-planet
There s going to be a file named Cargo.toml, we don t need to make any changes there, but the one that was auto-generated for me at the time of writing looks like this:
[package]name="hack-the-planet"version="0.1.0"edition="2021"# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html[dependencies]
There s a second file named src/main.rs, it s going to contain some pre-generated hello world, but we re going to delete it and create a new, empty file:
rm src/main.rs
touch src/main.rs
Alrighty, leaving this file empty is not valid but we re going to walk through the individual steps so we re going to try to build with an empty file first. At this point I would like to credit this chapter of a fasterthanli.me series and a blogpost by Philipp Oppermann, this tutorial is merely an 2023 update and makes it work with stable Rust. Let s run the build:
$ cargo build --release --target x86_64-unknown-none
Compiling hack-the-planet v0.1.0 (/hack-the-planet)
error[E0463]: can't find crate for std
= note: the x86_64-unknown-none target may not support the standard library
= note: std is required by hack_the_planet because it does not declare #![no_std]
error[E0601]: main function not found in crate hack_the_planet
= note: consider adding a main function to src/main.rs
Some errors have detailed explanations: E0463, E0601.
For more information about an error, try rustc --explain E0463 .
error: could not compile hack-the-planet due to 2 previous errors
Since this doesn t use a libc (oh right, I forgot to mention this up to this point actually), this also means there s no std standard library. Usually the standard library of Rust still uses the system libc to do syscalls, but since we specify our libc as none this means std won t be available (use std::fs::rename won t work). There are still other functions we can use and import, for example there s core that s effectively a second standard library, but much smaller.
To opt-out of the std standard library, we can put #![no_std] into src/main.rs:
Rust noticed we didn t define a main function and suggest we add one. This isn t what we want though so we ll politely decline and inform Rust we don t have a main and it shouldn t attempt to call it. We re adding #![no_main] to our file and src/main.rs now looks like this:
#![no_std]#![no_main]
Running the build again:
$ cargo build
Compiling hack-the-planet v0.1.0 (/hack-the-planet)
error: #[panic_handler] function required, but not found
error: language item required, but not found: eh_personality
= note: this can occur when a binary crate with #![no_std] is compiled for a target where eh_personality is defined in the standard library
= help: you may be able to compile for a target that doesn't need eh_personality , specify a target with --target or in .cargo/config
error: could not compile hack-the-planet due to 2 previous errors
Rust is asking us for a panic handler, basically I m going to jump to this address if something goes terribly wrong and execute whatever you put there . Eventually we would put some code there to just exit the program, but for now an infinitely loop will do. This is likely going to get stripped away anyway by the compiler if it notices our program has no code-branches leading to a panic and the code is unused. Our src/main.rs now looks like this:
$ objdump -d target/x86_64-unknown-none/release/hack-the-planet
target/x86_64-unknown-none/release/hack-the-planet: file format elf64-x86-64
Ok that looks pretty from scratch to me . The file contains no cpu instructions. Also note how our infinity loop is not present (as predicted).
Making a basic program and executing it
Ok let s try to make a valid program that basically just cleanly exits. First let s try to add some cpu instructions and verify they re indeed getting executed. Lemme introduce, the INT 3 instruction in x86_64 assembly. In binary it s also known as the 0xCC opcode. It crashes our program in a slightly different way, so if the error message changes, we know it worked. The other tutorials use a #[naked] function for the entry point, but since this feature isn t stabilized at the time of writing we re going to use the global_asm! macro. Also don t worry, I m not going to introduce every assembly instruction individually. Our program now looks like this:
The error message of the crash is now slightly different because it s hitting our breakpoint cpu instruction. Funfact btw, if you run this in strace you can see this isn t making any system calls (aka not talking to the kernel at all, it just crashes):
Let s try to make a program that does a clean shutdown. To do this we inform the kernel with a system call that we may like to exit. We can get more info on this with man 2 exit and it defines exit like this:
[[noreturn]] void _exit(int status);
On Linux this syscall is actually called _exit and exit is implemented as a libc function, but we don t care about any of that today, it s going to do the job just fine. Also note how it takes a single argument of type int. In C-speak this means signed 32 bit , i32 in Rust.
Next we need to figure out the syscall number of this syscall. These numbers are cpu architecture specific for some reason (idk, idc). We re looking these numbers up with ripgrep in /usr/include/asm/:
Since we re on x86_64 the correct value is the one in unistd_64.h, 60. Also, on x86_64 the syscall number goes into the rax cpu register, the status argument goes in the rdi register. The return value of the syscall is going to be placed in the rax register after the syscall is done, but for exit the execution is never given back to us. Let s try to write 60 into the rax register and 69 into the rdi register. To copy into registers we re going to use the mov destination, source instruction to copy from source to destination. With these registers setup we can use the syscall cpu instruction to hand execution over to the kernel. Don t worry, there s only one more assembly instruction coming and for everything else we re going to use Rust.
Our code now looks like this:
Writing Rust
Ok but even though cpu instructions can be fun at times, I d rather not deal with them most of the time (this might strike you as odd, considering this blog post). Instead let s try to define a function in Rust and call into that instead. We re going to define this function as unsafe (btw none of this is taking advantage of the safety guarantees by Rust in case it wasn t obvious. This tutorial is mostly going to stick to unsafe Rust, but for bigger projects you can attempt to reduce your usage of unsafe to opt back into normal safe Rust), it also declares the function with #[no_mangle] so the function name is preserved as main and we can call it from our global_asm entry point. Lastely, when our program is started it s going to get the stack address passed in one of the cpu registers, this value is expected to be passed to our function as an argument. Our function declares ! as return type, which means it never returns:
#[no_mangle]unsafefnmain(_stack_top:*constu8)->!// TODO: this is missing
This won t compile yet, we need to add our assembly for the exit syscall back in.
This time we re using the asm! macro, this is a slightly more declarative approach. We want to run the syscall cpu instruction with 60 in the rax register, and this time we want the rdi register to be zero, to indicate a successful exit. We also use options(noreturn) so Rust knows it should assume execution does not resume after this assembly is executed (the Linux kernel guarantees this). We modify our global_asm! entrypoint to call our new main function, and to copy the stack address from rsp into the register for the first argument rdi because it would otherwise get lost forever:
After building and disassembling this the Rust compiler is slowly starting to do work for us:
$ cargo build --release --target x86_64-unknown-none
$ objdump -d target/x86_64-unknown-none/release/hack-the-planet
target/x86_64-unknown-none/release/hack-the-planet: file format elf64-x86-64
Disassembly of section .text:
0000000000001210 <_start>:
1210: 48 89 e7 mov %rsp,%rdi
1213: e8 08 00 00 00 call 1220 <main>
1218: cc int3
1219: cc int3
121a: cc int3
121b: cc int3
121c: cc int3
121d: cc int3
121e: cc int3
121f: cc int3
0000000000001220 <main>:
1220: 50 push %rax
1221: b8 3c 00 00 00 mov $0x3c,%eax
1226: 31 ff xor %edi,%edi
1228: 0f 05 syscall
122a: 0f 0b ud2
The mov and syscall instructions are still the same, but it noticed it can XOR the rdi register with itself to set it to zero. It s using x86 assembly language (the 32 bit variant of x86_64, that also happens to work on x86_64) to do so, that s why the register is refered to as edi in the disassembly. You can also see it s inserting a bunch of 0xCC instructions (for alignment) and Rust puts the opcodes 0x0F 0x0B at the end of the function to force an invalid opcode exception so the program is guaranteed to crash in case the exit syscall doesn t do it.
This code still executes as expected:
Adding functions
Ok we re getting closer but we aren t quite there yet. Let s try to write an exit function for our assembly that we can then call like a normal function. Remember that it takes a signed 32 bit integer that s supposed to go into rdi.
Actually, since this function doesn t take any raw pointers and any i32 is valid for this syscall we re going to remove the unsafe marker of this function. When doing this we still need to use unsafe within the function for our inline assembly.
Running this still works, but interestingly the generated assembly didn t change at all:
$ cargo build --release --target x86_64-unknown-none
$ objdump -d target/x86_64-unknown-none/release/hack-the-planet
target/x86_64-unknown-none/release/hack-the-planet: file format elf64-x86-64
Disassembly of section .text:
0000000000001210 <_start>:
1210: 48 89 e7 mov %rsp,%rdi
1213: e8 08 00 00 00 call 1220 <main>
1218: cc int3
1219: cc int3
121a: cc int3
121b: cc int3
121c: cc int3
121d: cc int3
121e: cc int3
121f: cc int3
0000000000001220 <main>:
1220: 50 push %rax
1221: b8 3c 00 00 00 mov $0x3c,%eax
1226: 31 ff xor %edi,%edi
1228: 0f 05 syscall
122a: 0f 0b ud2
Rust noticed there s no need to make it a separate function at runtime and instead merged the instructions of the exit function directly into our main. It also noticed the 0 argument in exit(0) means rdi is supposed to be zero and uses the XOR optimization mentioned before.
Since main is not calling any unsafe functions anymore we could mark it as safe too, but in the next few functions we re going to deal with file descriptors and raw pointers, so this is likely the only safe function we re going to write in this tutorial so let s just keep the unsafe marker.
Printing text
Ok let s try to do a quick hello world, to do this we re going to call the write syscall. Looking it up with man 2 write:
The write syscall takes 3 arguments and returns a signed size_t. In Rust this is called isize. In C size_t is an unsigned integer type that can hold any value of sizeof(...) for the given platform, ssize_t can only store half of that because it uses one of the bits to indicate an error has occured (the first s means signed, write returns -1 in case of an error).
The arguments for write are:
the file descriptor to write to. stdout is located on file descriptor 1.
a pointer/address to some memory.
the number of bytes that should be written, starting at the given address.
Now that s a lot of stuff at once. Since this syscall is actually going to hand execution back to our program we need to let Rust know which cpu registers the syscall is writing to, so Rust doesn t attempt to use them to store data (that would be silently overwritten by the syscall). inlateout("raw") 1 => r0 means we re writing a value to the register and want the result back in variable r0. in("rdi") fd means we want to write the value of fd into the rdi register. lateout("rcx") _ means the Linux kernel may write to that register (so the previous value may get lost), but we don t want to store the value anywhere (the underscore acts as a dummy variable name).
This doesn t compile just yet though
Rust has inferred the type of r0 is isize since that s what our function returns, but the type of the input value for the register was inferred to be i32. We re going to select a specific number type to fix this.
We need to set the number of bytes we want to write explicitly because there s no concept of null-byte termination in the write system call, it s quite literally write the next X bytes, starting from this address . Our program now looks like this:
This time there are 2 syscalls, first write, then exit. For write it s setting up the 3 arguments in our cpu registers (rdi, rsi, rdx). The lea instruction subtracts 0x102b from the rip register (the instruction pointer) and places the result in the rsi register. This is effectively saying an address relative to wherever this code was loaded into memory . The instruction pointer is going to point directly behind the opcodes of the lea instruction, so 0x1238 - 0x102b = 0x20d. This address is also pointed out in the disassembly as a comment.
We don t see the string in our disassembly but we can convert our 0x20d hex to 525 in decimal and use dd to read 12 bytes from that offset, and sure enough:
$ dd bs=1 skip=525 count=12 if=target/x86_64-unknown-none/release/hack-the-planet
Hello world
12+0 records in
12+0 records out
Execute our binary with strace also shows the new write syscall (and the bytes that are being written mixed up in the output).
$ strace -f ./hack-the-planet
execve("./hack-the-planet", ["./hack-the-planet"], 0x74493abe64a8 /* 39 vars */) = 0
write(1, "Hello world\n", 12Hello world
) = 12
exit(0) = ?
+++ exited with 0 +++
After running strip on it to remove some symbols the binary is so small, if you open it in a text editor it fits on a screenshot:
I ve used hardware-backed OpenPGP keys since 2006 when I imported newly generated rsa1024 subkeys to a FSFE Fellowship card. This worked well for several years, and I recall buying more ZeitControl cards for multi-machine usage and backup purposes. As a side note, I recall being unsatisfied with the weak 1024-bit RSA subkeys at the time my primary key was a somewhat stronger 1280-bit RSA key created back in 2002 but OpenPGP cards at the time didn t support more than 1024 bit RSA, and were (and still often are) also limited to power-of-two RSA key sizes which I dislike.
I had my master key on disk with a strong password for a while, mostly to refresh expiration time of the subkeys and to sign other s OpenPGP keys. At some point I stopped carrying around encrypted copies of my master key. That was my main setup when I migrated to a new stronger RSA 3744 bit key with rsa2048 subkeys on a YubiKey NEO back in 2014. At that point, signing other s OpenPGP keys was a rare enough occurrence that I settled with bringing out my offline machine to perform this operation, transferring the public key to sign on USB sticks. In 2019 I re-evaluated my OpenPGP setup and ended up creating a offline Ed25519 key with subkeys on a FST-01G running Gnuk. My approach for signing other s OpenPGP keys were still to bring out my offline machine and sign things using the master secret using USB sticks for storage and transport. Which meant I almost never did that, because it took too much effort. So my 2019-era Ed25519 key still only has a handful of signatures on it, since I had essentially stopped signing other s keys which is the traditional way of getting signatures in return.
None of this caused any critical problem for me because I continued to use my old 2014-era RSA3744 key in parallel with my new 2019-era Ed25519 key, since too many systems didn t handle Ed25519. However, during 2022 this changed, and the only remaining environment that I still used my RSA3744 key for was in Debian and they require OpenPGP signatures on the new key to allow it to replace an older key. I was in denial about this sub-optimal solution during 2022 and endured its practical consequences, having to use the YubiKey NEO (which I had replaced with a permanently inserted YubiKey Nano at some point) for Debian-related purposes alone.
In December 2022 I bought a new laptop and setup a FST-01SZ with my Ed25519 key, and while I have taken a vacation from Debian, I continue to extend the expiration period on the old RSA3744-key in case I will ever have to use it again, so the overall OpenPGP setup was still sub-optimal. Having two valid OpenPGP keys at the same time causes people to use both for email encryption (leading me to have to use both devices), and the WKD Key Discovery protocol doesn t like two valid keys either. At FOSDEM 23 I ran into Andre Heinecke at GnuPG and I couldn t help complain about how complex and unsatisfying all OpenPGP-related matters were, and he mildly ignored my rant and asked why I didn t put the master key on another smartcard. The comment sunk in when I came home, and recently I connected all the dots and this post is a summary of what I did to move my offline OpenPGP master key to a Nitrokey Start.
First a word about device choice, I still prefer to use hardware devices that are as compatible with free software as possible, but the FST-01G or FST-01SZ are no longer easily available for purchase. I got a comment about Nitrokey start in my last post, and had two of them available to experiment with. There are things to dislike with the Nitrokey Start compared to the YubiKey (e.g., relative insecure chip architecture, the bulkier form factor and lack of FIDO/U2F/OATH support) but as far as I know there is no more widely available owner-controlled device that is manufactured for an intended purpose of implementing an OpenPGP card. Thus it hits the sweet spot for me.
Nitrokey Start
The first step is to run latest firmware on the Nitrokey Start for bug-fixes and important OpenSSH 9.0 compatibility and there are reproducible-built firmware published that you can install using pynitrokey. I run Trisquel 11 aramo on my laptop, which does not include the Python Pip package (likely because it promotes installing non-free software) so that was a slight complication. Building the firmware locally may have worked, and I would like to do that eventually to confirm the published firmware, however to save time I settled with installing the Ubuntu 22.04 packages on my machine:
$ sha256sum python3-pip*
ded6b3867a4a4cbaff0940cab366975d6aeecc76b9f2d2efa3deceb062668b1c python3-pip_22.0.2+dfsg-1ubuntu0.2_all.deb
e1561575130c41dc3309023a345de337e84b4b04c21c74db57f599e267114325 python3-pip-whl_22.0.2+dfsg-1ubuntu0.2_all.deb
$ doas dpkg -i python3-pip*
...
$ doas apt install -f
...
$
Installing pynitrokey downloaded a bunch of dependencies, and it would be nice to audit the license and security vulnerabilities for each of them. (Verbose output below slightly redacted.)
jas@kaka:~$ pip3 install --user pynitrokey
Collecting pynitrokey
Downloading pynitrokey-0.4.34-py3-none-any.whl (572 kB)
Collecting frozendict~=2.3.4
Downloading frozendict-2.3.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (113 kB)
Requirement already satisfied: click<9,>=8.0.0 in /usr/lib/python3/dist-packages (from pynitrokey) (8.0.3)
Collecting ecdsa
Downloading ecdsa-0.18.0-py2.py3-none-any.whl (142 kB)
Collecting python-dateutil~=2.7.0
Downloading python_dateutil-2.7.5-py2.py3-none-any.whl (225 kB)
Collecting fido2<2,>=1.1.0
Downloading fido2-1.1.0-py3-none-any.whl (201 kB)
Collecting tlv8
Downloading tlv8-0.10.0.tar.gz (16 kB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: certifi>=14.5.14 in /usr/lib/python3/dist-packages (from pynitrokey) (2020.6.20)
Requirement already satisfied: pyusb in /usr/lib/python3/dist-packages (from pynitrokey) (1.2.1.post1)
Collecting urllib3~=1.26.7
Downloading urllib3-1.26.15-py2.py3-none-any.whl (140 kB)
Collecting spsdk<1.8.0,>=1.7.0
Downloading spsdk-1.7.1-py3-none-any.whl (684 kB)
Collecting typing_extensions~=4.3.0
Downloading typing_extensions-4.3.0-py3-none-any.whl (25 kB)
Requirement already satisfied: cryptography<37,>=3.4.4 in /usr/lib/python3/dist-packages (from pynitrokey) (3.4.8)
Collecting intelhex
Downloading intelhex-2.3.0-py2.py3-none-any.whl (50 kB)
Collecting nkdfu
Downloading nkdfu-0.2-py3-none-any.whl (16 kB)
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from pynitrokey) (2.25.1)
Collecting tqdm
Downloading tqdm-4.65.0-py3-none-any.whl (77 kB)
Collecting nrfutil<7,>=6.1.4
Downloading nrfutil-6.1.7.tar.gz (845 kB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: cffi in /usr/lib/python3/dist-packages (from pynitrokey) (1.15.0)
Collecting crcmod
Downloading crcmod-1.7.tar.gz (89 kB)
Preparing metadata (setup.py) ... done
Collecting libusb1==1.9.3
Downloading libusb1-1.9.3-py3-none-any.whl (60 kB)
Collecting pc_ble_driver_py>=0.16.4
Downloading pc_ble_driver_py-0.17.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.9 MB)
Collecting piccata
Downloading piccata-2.0.3-py3-none-any.whl (21 kB)
Collecting protobuf<4.0.0,>=3.17.3
Downloading protobuf-3.20.3-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.1 MB)
Collecting pyserial
Downloading pyserial-3.5-py2.py3-none-any.whl (90 kB)
Collecting pyspinel>=1.0.0a3
Downloading pyspinel-1.0.3.tar.gz (58 kB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: pyyaml in /usr/lib/python3/dist-packages (from nrfutil<7,>=6.1.4->pynitrokey) (5.4.1)
Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil~=2.7.0->pynitrokey) (1.16.0)
Collecting pylink-square<0.11.9,>=0.8.2
Downloading pylink_square-0.11.1-py2.py3-none-any.whl (78 kB)
Collecting jinja2<3.1,>=2.11
Downloading Jinja2-3.0.3-py3-none-any.whl (133 kB)
Collecting bincopy<17.11,>=17.10.2
Downloading bincopy-17.10.3-py3-none-any.whl (17 kB)
Collecting fastjsonschema>=2.15.1
Downloading fastjsonschema-2.16.3-py3-none-any.whl (23 kB)
Collecting astunparse<2,>=1.6
Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting oscrypto~=1.2
Downloading oscrypto-1.3.0-py2.py3-none-any.whl (194 kB)
Collecting deepmerge==0.3.0
Downloading deepmerge-0.3.0-py2.py3-none-any.whl (7.6 kB)
Collecting pyocd<=0.31.0,>=0.28.3
Downloading pyocd-0.31.0-py3-none-any.whl (12.5 MB)
Collecting click-option-group<0.6,>=0.3.0
Downloading click_option_group-0.5.5-py3-none-any.whl (12 kB)
Collecting pycryptodome<4,>=3.9.3
Downloading pycryptodome-3.17-cp35-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.1 MB)
Collecting pyocd-pemicro<1.2.0,>=1.1.1
Downloading pyocd_pemicro-1.1.5-py3-none-any.whl (9.0 kB)
Requirement already satisfied: colorama<1,>=0.4.4 in /usr/lib/python3/dist-packages (from spsdk<1.8.0,>=1.7.0->pynitrokey) (0.4.4)
Collecting commentjson<1,>=0.9
Downloading commentjson-0.9.0.tar.gz (8.7 kB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: asn1crypto<2,>=1.2 in /usr/lib/python3/dist-packages (from spsdk<1.8.0,>=1.7.0->pynitrokey) (1.4.0)
Collecting pypemicro<0.2.0,>=0.1.9
Downloading pypemicro-0.1.11-py3-none-any.whl (5.7 MB)
Collecting libusbsio>=2.1.11
Downloading libusbsio-2.1.11-py3-none-any.whl (247 kB)
Collecting sly==0.4
Downloading sly-0.4.tar.gz (60 kB)
Preparing metadata (setup.py) ... done
Collecting ruamel.yaml<0.18.0,>=0.17
Downloading ruamel.yaml-0.17.21-py3-none-any.whl (109 kB)
Collecting cmsis-pack-manager<0.3.0
Downloading cmsis_pack_manager-0.2.10-py2.py3-none-manylinux1_x86_64.whl (25.1 MB)
Collecting click-command-tree==1.1.0
Downloading click_command_tree-1.1.0-py3-none-any.whl (3.6 kB)
Requirement already satisfied: bitstring<3.2,>=3.1 in /usr/lib/python3/dist-packages (from spsdk<1.8.0,>=1.7.0->pynitrokey) (3.1.7)
Collecting hexdump~=3.3
Downloading hexdump-3.3.zip (12 kB)
Preparing metadata (setup.py) ... done
Collecting fire
Downloading fire-0.5.0.tar.gz (88 kB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: wheel<1.0,>=0.23.0 in /usr/lib/python3/dist-packages (from astunparse<2,>=1.6->spsdk<1.8.0,>=1.7.0->pynitrokey) (0.37.1)
Collecting humanfriendly
Downloading humanfriendly-10.0-py2.py3-none-any.whl (86 kB)
Collecting argparse-addons>=0.4.0
Downloading argparse_addons-0.12.0-py3-none-any.whl (3.3 kB)
Collecting pyelftools
Downloading pyelftools-0.29-py2.py3-none-any.whl (174 kB)
Collecting milksnake>=0.1.2
Downloading milksnake-0.1.5-py2.py3-none-any.whl (9.6 kB)
Requirement already satisfied: appdirs>=1.4 in /usr/lib/python3/dist-packages (from cmsis-pack-manager<0.3.0->spsdk<1.8.0,>=1.7.0->pynitrokey) (1.4.4)
Collecting lark-parser<0.8.0,>=0.7.1
Downloading lark-parser-0.7.8.tar.gz (276 kB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: MarkupSafe>=2.0 in /usr/lib/python3/dist-packages (from jinja2<3.1,>=2.11->spsdk<1.8.0,>=1.7.0->pynitrokey) (2.0.1)
Collecting asn1crypto<2,>=1.2
Downloading asn1crypto-1.5.1-py2.py3-none-any.whl (105 kB)
Collecting wrapt
Downloading wrapt-1.15.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (78 kB)
Collecting future
Downloading future-0.18.3.tar.gz (840 kB)
Preparing metadata (setup.py) ... done
Collecting psutil>=5.2.2
Downloading psutil-5.9.4-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (280 kB)
Collecting capstone<5.0,>=4.0
Downloading capstone-4.0.2-py2.py3-none-manylinux1_x86_64.whl (2.1 MB)
Collecting naturalsort<2.0,>=1.5
Downloading naturalsort-1.5.1.tar.gz (7.4 kB)
Preparing metadata (setup.py) ... done
Collecting prettytable<3.0,>=2.0
Downloading prettytable-2.5.0-py3-none-any.whl (24 kB)
Collecting intervaltree<4.0,>=3.0.2
Downloading intervaltree-3.1.0.tar.gz (32 kB)
Preparing metadata (setup.py) ... done
Collecting ruamel.yaml.clib>=0.2.6
Downloading ruamel.yaml.clib-0.2.7-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (485 kB)
Collecting termcolor
Downloading termcolor-2.2.0-py3-none-any.whl (6.6 kB)
Collecting sortedcontainers<3.0,>=2.0
Downloading sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Requirement already satisfied: wcwidth in /usr/lib/python3/dist-packages (from prettytable<3.0,>=2.0->pyocd<=0.31.0,>=0.28.3->spsdk<1.8.0,>=1.7.0->pynitrokey) (0.2.5)
Building wheels for collected packages: nrfutil, crcmod, sly, tlv8, commentjson, hexdump, pyspinel, fire, intervaltree, lark-parser, naturalsort, future
Building wheel for nrfutil (setup.py) ... done
Created wheel for nrfutil: filename=nrfutil-6.1.7-py3-none-any.whl size=898520 sha256=de6f8803f51d6c26d24dc7df6292064a468ff3f389d73370433fde5582b84a10
Stored in directory: /home/jas/.cache/pip/wheels/39/2b/9b/98ab2dd716da746290e6728bdb557b14c1c9a54cb9ed86e13b
Building wheel for crcmod (setup.py) ... done
Created wheel for crcmod: filename=crcmod-1.7-cp310-cp310-linux_x86_64.whl size=31422 sha256=5149ac56fcbfa0606760eef5220fcedc66be560adf68cf38c604af3ad0e4a8b0
Stored in directory: /home/jas/.cache/pip/wheels/85/4c/07/72215c529bd59d67e3dac29711d7aba1b692f543c808ba9e86
Building wheel for sly (setup.py) ... done
Created wheel for sly: filename=sly-0.4-py3-none-any.whl size=27352 sha256=f614e413918de45c73d1e9a8dca61ca07dc760d9740553400efc234c891f7fde
Stored in directory: /home/jas/.cache/pip/wheels/a2/23/4a/6a84282a0d2c29f003012dc565b3126e427972e8b8157ea51f
Building wheel for tlv8 (setup.py) ... done
Created wheel for tlv8: filename=tlv8-0.10.0-py3-none-any.whl size=11266 sha256=3ec8b3c45977a3addbc66b7b99e1d81b146607c3a269502b9b5651900a0e2d08
Stored in directory: /home/jas/.cache/pip/wheels/e9/35/86/66a473cc2abb0c7f21ed39c30a3b2219b16bd2cdb4b33cfc2c
Building wheel for commentjson (setup.py) ... done
Created wheel for commentjson: filename=commentjson-0.9.0-py3-none-any.whl size=12092 sha256=28b6413132d6d7798a18cf8c76885dc69f676ea763ffcb08775a3c2c43444f4a
Stored in directory: /home/jas/.cache/pip/wheels/7d/90/23/6358a234ca5b4ec0866d447079b97fedf9883387d1d7d074e5
Building wheel for hexdump (setup.py) ... done
Created wheel for hexdump: filename=hexdump-3.3-py3-none-any.whl size=8913 sha256=79dfadd42edbc9acaeac1987464f2df4053784fff18b96408c1309b74fd09f50
Stored in directory: /home/jas/.cache/pip/wheels/26/28/f7/f47d7ecd9ae44c4457e72c8bb617ef18ab332ee2b2a1047e87
Building wheel for pyspinel (setup.py) ... done
Created wheel for pyspinel: filename=pyspinel-1.0.3-py3-none-any.whl size=65033 sha256=01dc27f81f28b4830a0cf2336dc737ef309a1287fcf33f57a8a4c5bed3b5f0a6
Stored in directory: /home/jas/.cache/pip/wheels/95/ec/4b/6e3e2ee18e7292d26a65659f75d07411a6e69158bb05507590
Building wheel for fire (setup.py) ... done
Created wheel for fire: filename=fire-0.5.0-py2.py3-none-any.whl size=116951 sha256=3d288585478c91a6914629eb739ea789828eb2d0267febc7c5390cb24ba153e8
Stored in directory: /home/jas/.cache/pip/wheels/90/d4/f7/9404e5db0116bd4d43e5666eaa3e70ab53723e1e3ea40c9a95
Building wheel for intervaltree (setup.py) ... done
Created wheel for intervaltree: filename=intervaltree-3.1.0-py2.py3-none-any.whl size=26119 sha256=5ff1def22ba883af25c90d90ef7c6518496fcd47dd2cbc53a57ec04cd60dc21d
Stored in directory: /home/jas/.cache/pip/wheels/fa/80/8c/43488a924a046b733b64de3fac99252674c892a4c3801c0a61
Building wheel for lark-parser (setup.py) ... done
Created wheel for lark-parser: filename=lark_parser-0.7.8-py2.py3-none-any.whl size=62527 sha256=3d2ec1d0f926fc2688d40777f7ef93c9986f874169132b1af590b6afc038f4be
Stored in directory: /home/jas/.cache/pip/wheels/29/30/94/33e8b58318aa05cb1842b365843036e0280af5983abb966b83
Building wheel for naturalsort (setup.py) ... done
Created wheel for naturalsort: filename=naturalsort-1.5.1-py3-none-any.whl size=7526 sha256=bdecac4a49f2416924548cae6c124c85d5333e9e61c563232678ed182969d453
Stored in directory: /home/jas/.cache/pip/wheels/a6/8e/c9/98cfa614fff2979b457fa2d9ad45ec85fa417e7e3e2e43be51
Building wheel for future (setup.py) ... done
Created wheel for future: filename=future-0.18.3-py3-none-any.whl size=492037 sha256=57a01e68feca2b5563f5f624141267f399082d2f05f55886f71b5d6e6cf2b02c
Stored in directory: /home/jas/.cache/pip/wheels/5e/a9/47/f118e66afd12240e4662752cc22cefae5d97275623aa8ef57d
Successfully built nrfutil crcmod sly tlv8 commentjson hexdump pyspinel fire intervaltree lark-parser naturalsort future
Installing collected packages: tlv8, sortedcontainers, sly, pyserial, pyelftools, piccata, naturalsort, libusb1, lark-parser, intelhex, hexdump, fastjsonschema, crcmod, asn1crypto, wrapt, urllib3, typing_extensions, tqdm, termcolor, ruamel.yaml.clib, python-dateutil, pyspinel, pypemicro, pycryptodome, psutil, protobuf, prettytable, oscrypto, milksnake, libusbsio, jinja2, intervaltree, humanfriendly, future, frozendict, fido2, ecdsa, deepmerge, commentjson, click-option-group, click-command-tree, capstone, astunparse, argparse-addons, ruamel.yaml, pyocd-pemicro, pylink-square, pc_ble_driver_py, fire, cmsis-pack-manager, bincopy, pyocd, nrfutil, nkdfu, spsdk, pynitrokey
WARNING: The script nitropy is installed in '/home/jas/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed argparse-addons-0.12.0 asn1crypto-1.5.1 astunparse-1.6.3 bincopy-17.10.3 capstone-4.0.2 click-command-tree-1.1.0 click-option-group-0.5.5 cmsis-pack-manager-0.2.10 commentjson-0.9.0 crcmod-1.7 deepmerge-0.3.0 ecdsa-0.18.0 fastjsonschema-2.16.3 fido2-1.1.0 fire-0.5.0 frozendict-2.3.5 future-0.18.3 hexdump-3.3 humanfriendly-10.0 intelhex-2.3.0 intervaltree-3.1.0 jinja2-3.0.3 lark-parser-0.7.8 libusb1-1.9.3 libusbsio-2.1.11 milksnake-0.1.5 naturalsort-1.5.1 nkdfu-0.2 nrfutil-6.1.7 oscrypto-1.3.0 pc_ble_driver_py-0.17.0 piccata-2.0.3 prettytable-2.5.0 protobuf-3.20.3 psutil-5.9.4 pycryptodome-3.17 pyelftools-0.29 pylink-square-0.11.1 pynitrokey-0.4.34 pyocd-0.31.0 pyocd-pemicro-1.1.5 pypemicro-0.1.11 pyserial-3.5 pyspinel-1.0.3 python-dateutil-2.7.5 ruamel.yaml-0.17.21 ruamel.yaml.clib-0.2.7 sly-0.4 sortedcontainers-2.4.0 spsdk-1.7.1 termcolor-2.2.0 tlv8-0.10.0 tqdm-4.65.0 typing_extensions-4.3.0 urllib3-1.26.15 wrapt-1.15.0
jas@kaka:~$
Then upgrading the device worked remarkable well, although I wish that the tool would have printed URLs and checksums for the firmware files to allow easy confirmation.
jas@kaka:~$ PATH=$PATH:/home/jas/.local/bin
jas@kaka:~$ nitropy start list
Command line tool to interact with Nitrokey devices 0.4.34
:: 'Nitrokey Start' keys:
FSIJ-1.2.15-5D271572: Nitrokey Nitrokey Start (RTM.12.1-RC2-modified)
jas@kaka:~$ nitropy start update
Command line tool to interact with Nitrokey devices 0.4.34
Nitrokey Start firmware update tool
Platform: Linux-5.15.0-67-generic-x86_64-with-glibc2.35
System: Linux, is_linux: True
Python: 3.10.6
Saving run log to: /tmp/nitropy.log.gc5753a8
Admin PIN:
Firmware data to be used:
- FirmwareType.REGNUAL: 4408, hash: ...b'72a30389' valid (from ...built/RTM.13/regnual.bin)
- FirmwareType.GNUK: 129024, hash: ...b'25a4289b' valid (from ...prebuilt/RTM.13/gnuk.bin)
Currently connected device strings:
Device:
Vendor: Nitrokey
Product: Nitrokey Start
Serial: FSIJ-1.2.15-5D271572
Revision: RTM.12.1-RC2-modified
Config: *:*:8e82
Sys: 3.0
Board: NITROKEY-START-G
initial device strings: [ 'name': '', 'Vendor': 'Nitrokey', 'Product': 'Nitrokey Start', 'Serial': 'FSIJ-1.2.15-5D271572', 'Revision': 'RTM.12.1-RC2-modified', 'Config': '*:*:8e82', 'Sys': '3.0', 'Board': 'NITROKEY-START-G' ]
Please note:
- Latest firmware available is:
RTM.13 (published: 2022-12-08T10:59:11Z)
- provided firmware: None
- all data will be removed from the device!
- do not interrupt update process - the device may not run properly!
- the process should not take more than 1 minute
Do you want to continue? [yes/no]: yes
...
Starting bootloader upload procedure
Device: Nitrokey Start FSIJ-1.2.15-5D271572
Connected to the device
Running update!
Do NOT remove the device from the USB slot, until further notice
Downloading flash upgrade program...
Executing flash upgrade...
Waiting for device to appear:
Wait 20 seconds.....
Downloading the program
Protecting device
Finish flashing
Resetting device
Update procedure finished. Device could be removed from USB slot.
Currently connected device strings (after upgrade):
Device:
Vendor: Nitrokey
Product: Nitrokey Start
Serial: FSIJ-1.2.19-5D271572
Revision: RTM.13
Config: *:*:8e82
Sys: 3.0
Board: NITROKEY-START-G
device can now be safely removed from the USB slot
final device strings: [ 'name': '', 'Vendor': 'Nitrokey', 'Product': 'Nitrokey Start', 'Serial': 'FSIJ-1.2.19-5D271572', 'Revision': 'RTM.13', 'Config': '*:*:8e82', 'Sys': '3.0', 'Board': 'NITROKEY-START-G' ]
finishing session 2023-03-16 21:49:07.371291
Log saved to: /tmp/nitropy.log.gc5753a8
jas@kaka:~$
jas@kaka:~$ nitropy start list
Command line tool to interact with Nitrokey devices 0.4.34
:: 'Nitrokey Start' keys:
FSIJ-1.2.19-5D271572: Nitrokey Nitrokey Start (RTM.13)
jas@kaka:~$
Before importing the master key to this device, it should be configured. Note the commands in the beginning to make sure scdaemon/pcscd is not running because they may have cached state from earlier cards. Change PIN code as you like after this, my experience with Gnuk was that the Admin PIN had to be changed first, then you import the key, and then you change the PIN.
jas@kaka:~$ gpg-connect-agent "SCD KILLSCD" "SCD BYE" /bye
OK
ERR 67125247 Slut p fil <GPG Agent>
jas@kaka:~$ ps auxww grep -e pcsc -e scd
jas 11651 0.0 0.0 3468 1672 pts/0 R+ 21:54 0:00 grep --color=auto -e pcsc -e scd
jas@kaka:~$ gpg --card-edit
Reader ...........: 20A0:4211:FSIJ-1.2.19-5D271572:0
Application ID ...: D276000124010200FFFE5D2715720000
Application type .: OpenPGP
Version ..........: 2.0
Manufacturer .....: unmanaged S/N range
Serial number ....: 5D271572
Name of cardholder: [not set]
Language prefs ...: [not set]
Salutation .......:
URL of public key : [not set]
Login data .......: [not set]
Signature PIN ....: forced
Key attributes ...: rsa2048 rsa2048 rsa2048
Max. PIN lengths .: 127 127 127
PIN retry counter : 3 3 3
Signature counter : 0
KDF setting ......: off
Signature key ....: [none]
Encryption key....: [none]
Authentication key: [none]
General key info..: [none]
gpg/card> admin
Admin commands are allowed
gpg/card> kdf-setup
gpg/card> passwd
gpg: OpenPGP card no. D276000124010200FFFE5D2715720000 detected
1 - change PIN
2 - unblock PIN
3 - change Admin PIN
4 - set the Reset Code
Q - quit
Your selection? 3
PIN changed.
1 - change PIN
2 - unblock PIN
3 - change Admin PIN
4 - set the Reset Code
Q - quit
Your selection? q
gpg/card> name
Cardholder's surname: Josefsson
Cardholder's given name: Simon
gpg/card> lang
Language preferences: sv
gpg/card> sex
Salutation (M = Mr., F = Ms., or space): m
gpg/card> login
Login data (account name): jas
gpg/card> url
URL to retrieve public key: https://josefsson.org/key-20190320.txt
gpg/card> forcesig
gpg/card> key-attr
Changing card key attribute for: Signature key
Please select what kind of key you want:
(1) RSA
(2) ECC
Your selection? 2
Please select which elliptic curve you want:
(1) Curve 25519
(4) NIST P-384
Your selection? 1
The card will now be re-configured to generate a key of type: ed25519
Note: There is no guarantee that the card supports the requested size.
If the key generation does not succeed, please check the
documentation of your card to see what sizes are allowed.
Changing card key attribute for: Encryption key
Please select what kind of key you want:
(1) RSA
(2) ECC
Your selection? 2
Please select which elliptic curve you want:
(1) Curve 25519
(4) NIST P-384
Your selection? 1
The card will now be re-configured to generate a key of type: cv25519
Changing card key attribute for: Authentication key
Please select what kind of key you want:
(1) RSA
(2) ECC
Your selection? 2
Please select which elliptic curve you want:
(1) Curve 25519
(4) NIST P-384
Your selection? 1
The card will now be re-configured to generate a key of type: ed25519
gpg/card>
jas@kaka:~$ gpg --card-edit
Reader ...........: 20A0:4211:FSIJ-1.2.19-5D271572:0
Application ID ...: D276000124010200FFFE5D2715720000
Application type .: OpenPGP
Version ..........: 2.0
Manufacturer .....: unmanaged S/N range
Serial number ....: 5D271572
Name of cardholder: Simon Josefsson
Language prefs ...: sv
Salutation .......: Mr.
URL of public key : https://josefsson.org/key-20190320.txt
Login data .......: jas
Signature PIN ....: not forced
Key attributes ...: ed25519 cv25519 ed25519
Max. PIN lengths .: 127 127 127
PIN retry counter : 3 3 3
Signature counter : 0
KDF setting ......: on
Signature key ....: [none]
Encryption key....: [none]
Authentication key: [none]
General key info..: [none]
jas@kaka:~$
Once setup, bring out your offline machine and boot it and mount your USB stick with the offline key. The paths below will be different, and this is using a somewhat unorthodox approach of working with fresh GnuPG configuration paths that I chose for the USB stick.
jas@kaka:/media/jas/2c699cbd-b77e-4434-a0d6-0c4965864296$ cp -a gnupghome-backup-masterkey gnupghome-import-nitrokey-5D271572
jas@kaka:/media/jas/2c699cbd-b77e-4434-a0d6-0c4965864296$ gpg --homedir $PWD/gnupghome-import-nitrokey-5D271572 --edit-key B1D2BD1375BECB784CF4F8C4D73CF638C53C06BE
gpg (GnuPG) 2.2.27; Copyright (C) 2021 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Secret key is available.
sec ed25519/D73CF638C53C06BE
created: 2019-03-20 expired: 2019-10-22 usage: SC
trust: ultimate validity: expired
[ expired] (1). Simon Josefsson <simon@josefsson.org>
gpg> keytocard
Really move the primary key? (y/N) y
Please select where to store the key:
(1) Signature key
(3) Authentication key
Your selection? 1
sec ed25519/D73CF638C53C06BE
created: 2019-03-20 expired: 2019-10-22 usage: SC
trust: ultimate validity: expired
[ expired] (1). Simon Josefsson <simon@josefsson.org>
gpg>
Save changes? (y/N) y
jas@kaka:/media/jas/2c699cbd-b77e-4434-a0d6-0c4965864296$
At this point it is useful to confirm that the Nitrokey has the master key available and that is possible to sign statements with it, back on your regular machine:
jas@kaka:~$ gpg --card-status
Reader ...........: 20A0:4211:FSIJ-1.2.19-5D271572:0
Application ID ...: D276000124010200FFFE5D2715720000
Application type .: OpenPGP
Version ..........: 2.0
Manufacturer .....: unmanaged S/N range
Serial number ....: 5D271572
Name of cardholder: Simon Josefsson
Language prefs ...: sv
Salutation .......: Mr.
URL of public key : https://josefsson.org/key-20190320.txt
Login data .......: jas
Signature PIN ....: not forced
Key attributes ...: ed25519 cv25519 ed25519
Max. PIN lengths .: 127 127 127
PIN retry counter : 3 3 3
Signature counter : 1
KDF setting ......: on
Signature key ....: B1D2 BD13 75BE CB78 4CF4 F8C4 D73C F638 C53C 06BE
created ....: 2019-03-20 23:37:24
Encryption key....: [none]
Authentication key: [none]
General key info..: pub ed25519/D73CF638C53C06BE 2019-03-20 Simon Josefsson <simon@josefsson.org>
sec> ed25519/D73CF638C53C06BE created: 2019-03-20 expires: 2023-09-19
card-no: FFFE 5D271572
ssb> ed25519/80260EE8A9B92B2B created: 2019-03-20 expires: 2023-09-19
card-no: FFFE 42315277
ssb> ed25519/51722B08FE4745A2 created: 2019-03-20 expires: 2023-09-19
card-no: FFFE 42315277
ssb> cv25519/02923D7EE76EBD60 created: 2019-03-20 expires: 2023-09-19
card-no: FFFE 42315277
jas@kaka:~$ echo foo gpg -a --sign gpg --verify
gpg: Signature made Thu Mar 16 22:11:02 2023 CET
gpg: using EDDSA key B1D2BD1375BECB784CF4F8C4D73CF638C53C06BE
gpg: Good signature from "Simon Josefsson <simon@josefsson.org>" [ultimate]
jas@kaka:~$
Finally to retrieve and sign a key, for example Andre Heinecke s that I could confirm the OpenPGP key identifier from his business card.
jas@kaka:~$ gpg --locate-external-keys aheinecke@gnupg.com
gpg: key 1FDF723CF462B6B1: public key "Andre Heinecke <aheinecke@gnupg.com>" imported
gpg: Total number processed: 1
gpg: imported: 1
gpg: marginals needed: 3 completes needed: 1 trust model: pgp
gpg: depth: 0 valid: 2 signed: 7 trust: 0-, 0q, 0n, 0m, 0f, 2u
gpg: depth: 1 valid: 7 signed: 64 trust: 7-, 0q, 0n, 0m, 0f, 0u
gpg: next trustdb check due at 2023-05-26
pub rsa3072 2015-12-08 [SC] [expires: 2025-12-05]
94A5C9A03C2FE5CA3B095D8E1FDF723CF462B6B1
uid [ unknown] Andre Heinecke <aheinecke@gnupg.com>
sub ed25519 2017-02-13 [S]
sub ed25519 2017-02-13 [A]
sub rsa3072 2015-12-08 [E] [expires: 2025-12-05]
sub rsa3072 2015-12-08 [A] [expires: 2025-12-05]
jas@kaka:~$ gpg --edit-key "94A5C9A03C2FE5CA3B095D8E1FDF723CF462B6B1"
gpg (GnuPG) 2.2.27; Copyright (C) 2021 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
pub rsa3072/1FDF723CF462B6B1
created: 2015-12-08 expires: 2025-12-05 usage: SC
trust: unknown validity: unknown
sub ed25519/2978E9D40CBABA5C
created: 2017-02-13 expires: never usage: S
sub ed25519/DC74D901C8E2DD47
created: 2017-02-13 expires: never usage: A
The following key was revoked on 2017-02-23 by RSA key 1FDF723CF462B6B1 Andre Heinecke <aheinecke@gnupg.com>
sub cv25519/1FFE3151683260AB
created: 2017-02-13 revoked: 2017-02-23 usage: E
sub rsa3072/8CC999BDAA45C71F
created: 2015-12-08 expires: 2025-12-05 usage: E
sub rsa3072/6304A4B539CE444A
created: 2015-12-08 expires: 2025-12-05 usage: A
[ unknown] (1). Andre Heinecke <aheinecke@gnupg.com>
gpg> sign
pub rsa3072/1FDF723CF462B6B1
created: 2015-12-08 expires: 2025-12-05 usage: SC
trust: unknown validity: unknown
Primary key fingerprint: 94A5 C9A0 3C2F E5CA 3B09 5D8E 1FDF 723C F462 B6B1
Andre Heinecke <aheinecke@gnupg.com>
This key is due to expire on 2025-12-05.
Are you sure that you want to sign this key with your
key "Simon Josefsson <simon@josefsson.org>" (D73CF638C53C06BE)
Really sign? (y/N) y
gpg> quit
Save changes? (y/N) y
jas@kaka:~$
This is on my day-to-day machine, using the NitroKey Start with the offline key. No need to boot the old offline machine just to sign keys or extend expiry anymore! At FOSDEM 23 I managed to get at least one DD signature on my new key, and the Debian keyring maintainers accepted my Ed25519 key. Hopefully I can now finally let my 2014-era RSA3744 key expire in 2023-09-19 and not extend it any further. This should finish my transition to a simpler OpenPGP key setup, yay!
A couple of days ago I upgraded my home server from Debian/Bullseye to Debian/Testing (soon to be Bookworm). Since then KDE sessions on that system have had problems of locking the input queue, the mouse can move and mouse-over events work but clicking the mouse or pressing the keyboard does nothing. Various web pages suggested that the xdotool program (in the xdotool package in Debian) can address this. The problem is apparently programs grabbing the input and not letting it go.
The command xdotool key XF86LogGrabInfo causes the xorg server to dump information on it s grabs . After running that command I looked in /var/log/Xorg.0.log and found that active grabs were only held by /usr/bin/kwin_x11 and /usr/bin/kglobalaccel5. So it seems like a KDE issue. Other systems running X11 with Debian/Testing (such as the laptop I m using to write this blog post) don t have the problem, so it could be something related to the KDE configuration of the account used on that system.
The command xdotool key XF86Ungrab is supposed to break out of such a grab, but for me didn t do so.
On the same system running KDE with Wayland works fine in this regard. Does Wayland do things differently and not allow this grabbing to block everything? Does KDE have an X11 specific bug? Is there a race condition that just gets triggered by the speed of Xorg on that system but not by the slightly different timings of Wayland? I might never find out.
I previously wrote about problems with Wayland/KDE on laptops [1]. Fortunately this bug happened to occur on a server so inability to reconfigure monitors isn t necessarily a deal breaker, although being unable to use some of the high-DPI settings for the 4K monitor it has may be an issue. It will be really annoying if some of the laptop configurations I support get this grabbing problem. But since that time I have learned of the kscreen-doctor command which is included in Debian/Testing and can do some of the necessary things, it doesn t have a man page so you have to run kscreen-doctor -h for documentation.
By default, EFI firmwares boot a specific file under /efi/boot/. The name of
such file depends on the architecture: for example, on 64 bit x86 systems it is
bootx64.efi, while on ARM it is bootaa64.efi.
Copy /usr/lib/grub/x86_64-efi/monolithic/grubx64.efi from package
grub-efi-amd64-bin to /efi/boot/bootx64.efi on the boot image, and that
should be enough to start GRUB.
The kernel will start and reach the point of trying to mount the root fs. This
is great but it would now be useful to have some sort of shell access in order
to look around. Let s add an initrd!
There s the option of starting qemu in console, let s try that out. Start qemu
with -nographic, and append console=ttyS0 to the kernel command line
arguments.
We can double-check in the firmware settings if Secure boot is indeed enabled.
At the GRUB prompt, type fwsetup:
grub> fwsetup
Check under "Device Manager" "Secure Boot Configuration" that "Attempt
Secure Boot" is selected, then boot from GRUB as before. If all went well, the
kernel should confirm that we have booted with Secure boot:
Phew!
Today, after four days of combing through code I am unfamiliar with, I
was finally able to change my expression.
I m finally at the part of my PhD work where I am tasked with
implementing the protocol I claim improves from the current
situation. I wrote a script to deploy the infrastructure I need for
the experiment, and was not expecting any issues I am not (yet)
familiar with the Go language (in which the Hockeypuck key
server is developed), but I have managed to
install it several times, and it holds no terrible surprises anymore
for me. Or so I think.
So, how come it s possible the five servers in my laboratory network
don t gossip to each other? The logs don t show anything clear Only
a sucession of this:
hockeypuck[5295]: time="2023-03-24T00:00:28-06:00" level=error msg="recon with :0 failed" error="[ /srv/hockeypuck/packaging/src/gopkg.in/hockeypuck/conflux.v2/recon/gossip.go:109: dial tcp :0: connect: connection refused ]" label="gossip :11370"
hockeypuck[5295]: time="2023-03-24T00:00:28-06:00" level=info msg="waiting 27s for next gossip attempt" label="gossip :11370"
And while tcp :0: connect: connection refused sounds fishy It
took me too long to find the reasons.
But, at least, along the way I decided to find my errors by debugging
the code, rather than by rebuilding the laboratory and random-stabbing
at the configuration.
And yes, finally I came to my senses, and found out my silly
mistake was to have my configuration read:
Because, of course, TOML would find no child declarations for
hockeypuck.conflux.recon.partner.10 (as the following period makes
the rest of the entry an entirely distinct one from what I thought I
specified).
Anyway, this made me at least:
Lose my fear of trying to understand the logic of a Go program (and
of the Go language)
Understand the logging framework for Go
Learning how to use the (great!) format strings infrastructure it
has
Do some minor logic modifications, trying to find the issue
Dig into how TOML parsing is done, idiomatically
Now Why am I posting this? Not only because I feel very happy and
wanted to share my a-ha moment, but also because I m sure this time
that seems that I mindlessly spent poking at Go without knowing the
basics will be somehow rewarded I have to learn bits of the language
anyway, so it s time well spent.
Or so I hope.
(oh, and the funny spectacles? I am not sure, but I believe them to
have been property of my grandfather or great-grandfather when they
came from Europe, in 1947 or in 1928 respectively. One glasspiece is
sadly lost, but other than that, I love them!)
After my last
post,
B lint (who prompted it with his last
post)
suggested I should do a hybrid test of his tests and my extremes. He
suggested I should build the Linux kernel using my Raspberry Pi 4 (8GB
model), but using the Firebuild build
accelerator.
Before going any further: I must make clear that while Firebuild is
freely redistributable, it is not made available under a free
license. It
is free for personal use or commercial trial, but otherwise requires
licensing.
B lint managed to build a Linux kernel in just over 8 seconds. So,
how did my test go? My previous experiment, using -j 4, built Linux
in ~100 minutes; this was about a year ago, and I m now building linux
6.1, so I timed this again. To get a baseline, I built my kernel from
a just-unpacked tree, just as usual:
# cd /usr/src/linux-source-6.1
# make clean
# make defconfig
# time make -j4
(...)
real 117m30.588s
user 392m41.434s
sys 52m2.556s
Of course, having all of the object files built makes the rebuild
process quite faster (this is still done without firebuild). I
understand calling make defconfig without cleaning does not change
much, but I saw it often referenced in firebuild s docs, so I m
leaving it:
# time make -j 4
(...)
real 0m43.822s
user 1m36.577s
sys 0m40.805s
Then, I did a first run using firebuild. Firebuild is a caching
build optimizer, so the first run will naturally be somewhat slower
(but if you often rebuild your kernel, it should be seen as an
investment). Now, in the Raspberry Pi, that uses a slow SD card
interface for its storage It is a heavy investment. The first time
I built with firebuild, it meant almost a 100% build time hit:
# cd /usr/src/linux-source-6.1
# make clean
# make defconfig
# time firebuild make -j 4
(...)
real 212m58.647s
user 391m49.080s
sys 81m10.758s
Not only that; I am using a fairly decent and big 32GB card, but this
is quite a big price to pay in such a limited system!
# du -sh .cache/firebuild/
4.2G .cache/firebuild/
I did a build without cleaning the build directory, using firebuild,
and it does help although not by so much as in higher performance
systems:
# cd /usr/src/linux-source-6.1
# make clean
# make defconfig
# time firebuild make -j 4
(...)
real 68m6.621s
user 98m32.514s
sys 31m41.643s
So, it built in roughly 65% of the time it would take to build
regularly. And what about rebuilding without cleaning?
# make defconfig
# time firebuild make -j 4
(...)
real 1m11.872s
user 2m5.807s
sys 1m46.178s
In this case, using firebuild worked roughly 30% slower than not
using it. I guess the high number of file ops inside
.cache/firebuild are to blame, as in the case of the media I m
using, those are quite expensive; make went its way basically
checking date stamps between *.c and *.o (yes, very roughly), and
while running under firebuild, I suppose each of these meant an extra
lookup inside the cache.
So Experiment requested, experiment performed!
Thief of Time is the 26th Discworld novel and the last Death novel,
although he still appears in subsequent books. It's the third book
starring Susan Sto Helit, so I don't recommend starting here.
Mort is the best starting point for the
Death subseries, and Reaper Man provides
a useful introduction to the villains.
Jeremy Clockson was an orphan raised by the Guild of Clockmakers. He is
very good at making clocks. He's not very good at anything else,
particularly people, but his clocks are the most accurate in Ankh-Morpork.
He is therefore the logical choice to receive a commission by a mysterious
noblewoman who wants him to make the most accurate possible clock: a clock
that can measure the tick of the universe, one that a fairy tale says had
been nearly made before. The commission is followed by a surprise
delivery of an Igor, to help with the clock-making.
People who live in places with lots of fields become farmers. People who
live where there is lots of iron and coal become blacksmiths. And people
who live in the mountains near the Hub, near the gods and full of magic,
become monks. In the highest valley are the History Monks, founded by Wen
the Eternally Surprised. Like most monks, they take apprentices with
certain talents and train them in their discipline. But Lobsang Ludd, an
orphan discovered in the Thieves Guild in Ankh-Morpork, is proving a
challenge. The monks decide to apprentice him to Lu-Tze the sweeper;
perhaps that will solve multiple problems at once.
Since Hogfather, Susan has moved from
being a governess to a schoolteacher. She brings to that job the same
firm patience, total disregard for rules that apply to other people, and
impressive talent for managing children. She is by far the most popular
teacher among the kids, and not only because she transports her class all
over the Disc so that they can see things in person. It is a job that she
likes and understands, and one that she's quite irate to have interrupted
by a summons from her grandfather. But the Auditors are up to something,
and Susan may be able to act in ways that Death cannot.
This was great. Susan has quickly become one of my favorite Discworld
characters, and this time around there is no (or, well, not much)
unbelievable romance or permanently queasy god to distract. The
clock-making portions of the book quickly start to focus on Igor, who is a
delightful perspective through whom to watch events unfold. And the
History Monks! The metaphysics of what they are actually doing (which I
won't spoil, since discovering it slowly is a delight) is perhaps my
favorite bit of Discworld world building to date. I am a sucker for
stories that focus on some process that everyone thinks happens
automatically and investigate the hidden work behind it.
I do want to add a caveat here that the monks are in part a parody of
Himalayan Buddhist monasteries, Lu-Tze is rather obviously a parody of
Laozi and Daoism in general, and Pratchett's parodies of non-western
cultures are rather ham-handed. This is not quite the insulting mess that
the Chinese parody in Interesting Times
was, but it's heavy on the stereotypes. It does not, thankfully,
rely on the stereotypes; the characters are great fun on their own
terms, with the perfect (for me) balance of irreverence and
thoughtfulness. Lu-Tze refusing to be anything other than a sweeper and
being irritatingly casual about all the rules of the order is a classic
bit that Pratchett does very well. But I also have the luxury of ignoring
stereotypes of a culture that isn't mine, and I think Pratchett is on
somewhat thin ice.
As one specific example, having Lu-Tze's treasured sayings be a collection
of banal aphorisms from a random Ankh-Morpork woman is both hilarious and
also arguably rather condescending, and I'm not sure where I landed. It's
a spot-on bit of parody of how a lot of people who get very into "eastern
religions" sound, but it's also equating the Dao De Jing with
advice from the Discworld equivalent of a English housewife. I think the
generous reading is that Lu-Tze made the homilies profound by looking at
them in an entirely different way than the woman saying them, and that's
not completely unlike Daoism and works surprisingly well. But that's
reading somewhat against the grain; Pratchett is clearly making fun of
philosophical koans, and while anything is fair game for some friendly
poking, it still feels a bit weird.
That isn't the part of the History Monks that I loved, though. Their
actual role in the story doesn't come out of the parody. It's something
entirely native to Discworld, and it's an absolute delight. The scene
with Lobsang and the procrastinators is perhaps my favorite Discworld set
piece to date. Everything about the technology of the History Monks, even
the Bond parody, is so good.
I grew up reading the Marvel Comics universe, and Thief of Time
reminds me of a classic John Byrne or Jim Starlin story, where the heroes
are dumped into the middle of vast interdimensional conflicts involving
barely-anthropomorphized cosmic powers and the universe is revealed to
work in ever more intricate ways at vastly expanding scales. The Auditors
are villains in exactly that tradition, and just like the best of those
stories, the fulcrum of the plot is questions about what it means to be
human, what it means to be alive, and the surprising alliances these
non-human powers make with humans or semi-humans. I devoured this kind of
story as a kid, and it turns out I still love it.
The one complaint I have about the plot is that the best part of this book
is the middle, and the end didn't entirely work for me. Ronnie Soak is at
his best as a supporting character about three quarters of the way through
the book, and I found the ending of his subplot much less interesting.
The cosmic confrontation was oddly disappointing, and there's a whole
extended sequence involving chocolate that I think was funnier in
Pratchett's head than it was in mine. The ending isn't bad, but
the middle of this book is my favorite bit of Discworld writing yet, and I
wish the story had carried that momentum through to the end.
I had so much fun with this book. The Discworld novels are clearly
getting better. None of them have yet vaulted into the ranks of my
all-time favorite books there's always some lingering quibble or sagging
bit but it feels like they've gone from reliably good books to more
reliably great books. The acid test is coming, though: the next book is a
Rincewind book, which are usually the weak spots.
Followed by The Last Hero in publication order. There is no direct
thematic sequel.
Rating: 8 out of 10
Github accidentally committed their SSH RSA private key to a repository, and now a bunch of people's infrastructure is broken because it needs to be updated to trust the new key. This is obviously bad, but what's frustrating is that there's no inherent need for it to be - almost all the technological components needed to both reduce the initial risk and to make the transition seamless already exist.
But first, let's talk about what actually happened here. You're probably used to the idea of TLS certificates from using browsers. Every website that supports TLS has an asymmetric pair of keys divided into a public key and a private key. When you contact the website, it gives you a certificate that contains the public key, and your browser then performs a series of cryptographic operations against it to (a) verify that the remote site possesses the private key (which prevents someone just copying the certificate to another system and pretending to be the legitimate site), and (b) generate an ephemeral encryption key that's used to actually encrypt the traffic between your browser and the site. But what stops an attacker from simply giving you a fake certificate that contains their public key? The certificate is itself signed by a certificate authority (CA), and your browser is configured to trust a preconfigured set of CAs. CAs will not give someone a signed certificate unless they prove they have legitimate ownership of the site in question, so (in theory) an attacker will never be able to obtain a fake certificate for a legitimate site.
This infrastructure is used for pretty much every protocol that can use TLS, including things like SMTP and IMAP. But SSH doesn't use TLS, and doesn't participate in any of this infrastructure. Instead, SSH tends to take a "Trust on First Use" (TOFU) model - the first time you ssh into a server, you receive a prompt asking you whether you trust its public key, and then you probably hit the "Yes" button and get on with your life. This works fine up until the point where the key changes, and SSH suddenly starts complaining that there's a mismatch and something awful could be happening (like someone intercepting your traffic and directing it to their own server with their own keys). Users are then supposed to verify whether this change is legitimate, and if so remove the old keys and add the new ones. This is tedious and risks users just saying "Yes" again, and if it happens too often an attacker can simply redirect target users to their own server and through sheer fatigue at dealing with this crap the user will probably trust the malicious server.
Why not certificates? OpenSSH actually does support certificates, but not in the way you might expect. There's a custom format that's significantly less complicated than the X509 certificate format used in TLS. Basically, an SSH certificate just contains a public key, a list of hostnames it's good for, and a signature from a CA. There's no pre-existing set of trusted CAs, so anyone could generate a certificate that claims it's valid for, say, github.com. This isn't really a problem, though, because right now nothing pays attention to SSH host certificates unless there's some manual configuration.
(It's actually possible to glue the general PKI infrastructure into SSH certificates. Please do not do this)
So let's look at what happened in the Github case. The first question is "How could the private key have been somewhere that could be committed to a repository in the first place?". I have no unique insight into what happened at Github, so this is conjecture, but I'm reasonably confident in it. Github deals with a large number of transactions per second. Github.com is not a single computer - it's a large number of machines. All of those need to have access to the same private key, because otherwise git would complain that the private key had changed whenever it connected to a machine with a different private key (the alternative would be to use a different IP address for every frontend server, but that would instead force users to repeatedly accept additional keys every time they connect to a new IP address). Something needs to be responsible for deploying that private key to new systems as they're brought up, which means there's ample opportunity for it to accidentally end up in the wrong place.
Now, best practices suggest that this should be avoided by simply placing the private key in a hardware module that performs the cryptographic operations, ensuring that nobody can ever get at the private key. The problem faced here is that HSMs typically aren't going to be fast enough to handle the number of requests per second that Github deals with. This can be avoided by using something like a Nitro Enclave, but you're still going to need a bunch of these in different geographic locales because otherwise your front ends are still going to be limited by the need to talk to an enclave on the other side of the planet, and now you're still having to deal with distributing the private key to a bunch of systems.
What if we could have the best of both worlds - the performance of private keys that just happily live on the servers, and the security of private keys that live in HSMs? Unsurprisingly, we can! The SSH private key could be deployed to every front end server, but every minute it could call out to an HSM-backed service and request a new SSH host certificate signed by a private key in the HSM. If clients are configured to trust the key that's signing the certificates, then it doesn't matter what the private key on the servers is - the client will see that there's a valid certificate and will trust the key, even if it changes. Restricting the validity of the certificate to a small window of time means that if a key is compromised an attacker can't do much with it - the moment you become aware of that you stop signing new certificates, and once all the existing ones expire the old private key becomes useless. You roll out a new private key with new certificates signed by the same CA and clients just carry on trusting it without any manual involvement.
Why don't we have this already? The main problem is that client tooling just doesn't handle this well. OpenSSH has no way to do TOFU for CAs, just the keys themselves. This means there's no way to do a git clone ssh://git@github.com/whatever and get a prompt asking you to trust Github's CA. Instead, you need to add a @cert-authority github.com (key) line to your known_hosts file by hand, and since approximately nobody's going to do that there's only marginal benefit in going to the effort to implement this infrastructure. The most important thing we can do to improve the security of the SSH ecosystem is to make it easier to use certificates, and that means improving the behaviour of the clients.
It should be noted that certificates aren't the only approach to handling key migration. OpenSSH supports a protocol for key rotation, basically by allowing the server to provide a set of multiple trusted keys that the client can cache, and then invalidating old ones. Unfortunately this still requires that the "new" private keys be deployed in the same way as the old ones, so any screwup that results in one private key being leaked may well also result in the additional keys being leaked. I prefer the certificate approach.
Finally, I've seen a couple of people imply that the blame here should be attached to whoever or whatever caused the private key to be committed to a repository in the first place. This is a terrible take. Humans will make mistakes, and your systems should be resilient against that. There's no individual at fault here - there's a series of design decisions that made it possible for a bad outcome to occur, and in a better universe they wouldn't have been necessary. Let's work on building that better universe.
Update on 2023-03-23: thanks to Daniel Roschka for mentioning the Mandos and TPM approaches, which might be better alternatives, depending on your options and needs. Peter Palfrader furthermore pointed me towards clevis-initramfs and tang.
A customer of mine runs dedicated servers inside a foreign data-center, remote hands only. In such an environment you might need a disk replacement because you need bigger or faster disks, though also a disk might (start to) fail and you need a replacement. One has to be prepared for such a scenario, but fully wiping your used disk then might not always be an option, especially once disks (start to) fail. On the other hand you don t want to end up with (partial) data on your disk handed over to someone unexpected.
By encrypting the data on your disks upfront you can prevent against this scenario. But if you have a fleet of servers you might not want to manually jump on servers during boot and unlock crypto volumes manually. It s especially annoying if it s about the root filesystem where a solution like dropbear-initramfs needs to be used for remote access during initramfs boot stage. So my task for the customer was to adjust encrypted LUKS devices such that no one needs to manually unlock the encrypted device during server boot (with some specific assumptions about possible attack vectors one has to live with, see the disclaimer at the end).
The documentation about this use-case was rather inconsistent, especially because special rules apply for the root filesystem (no key file usage), we see different behavior between what s supported by systemd (hello key file again), initramfs-tools and dracut, not to mention the changes between different distributions. Since tests with this tend to be rather annoying (better make sure to have a Grml live system available :)), I m hereby documenting what worked for us (Debian/bullseye with initramfs-tools and cryptsetup-initramfs).
The system was installed with LVM on-top of an encrypted Software-RAID device, only the /boot partition is unencrypted. But even if you don t use Software-RAID nor LVM the same instructions apply. The system looks like this:
The server will then boot without prompting for a crypto password.
Note that initramfs-tools by default uses an insecure umask of 0022, resulting in the initrd being accessible to everyone. But if you have the dropbear-initramfs package installed, its /usr/share/initramfs-tools/conf-hooks.d/dropbear sets UMASK=0077 , so the resulting /boot/initrd* file should automatically have proper permissions (0600). The cryptsetup hook warns about a permissive umask configuration during update-initramfs runs, but if you want to be sure, explicitly set it via e.g.:
# cat > /etc/initramfs-tools/conf.d/umask << EOF
# restrictive umask to avoid non-root access to initrd:
UMASK=0077
EOF
# update-initramfs -k all -u
Disclaimer: Of course you need to trust users with access to /etc/initramfs-tools/unlock.sh as well as the initramfs/initrd on your system. Furthermore you should wipe the boot partition (to destroy the keyfile information) before handing over such a disk. But that is a risk my customer can live with, YMMV.
Anti-national and Anti-India judges Kiren Rijiju, Law Minister.
For those who can t see the above poster says the following A handful of retired Supreme Court judges who are part of Anti-India and are trying to make Indian judiciary play role of the Opposition party. Law Minister Kiren Rijiju.
Now, just to give bit more of a context, the above has happened as the CJI (Chief Justice has not been listening or toeing their line)
The above is a statement given by CJI DY Chandrachud. He says and I quote Democracy needs truth to survive. Democracy and truth go hand in hand. Speaking truth to power is a right of every citizen in a democracy. It is equally a duty.
Another quote by him. I am personally averse to sealed covers. There has to be transparency in Court. This is about implementing the orders. What can be secrecy here.
Now I need to again give context of the various statements given. The law minister who gave this statement, his name incidentally came first under the radar sometime in 2013-2014 in a list given/shared by Union Home Secretary R K Singh (then in UPA, now in BJP) as an anti-India China sympathizer. Of course today all sorts of thugs use the brand nationalism he is one of them.
Now as far as the CJI is concerned, AFAIK I know he bent over backwards for them but they were not pleased with him. The latest sealed cover statement is because BJP wanted to put a sealed cover in the ongoing Udhav Thackeray vs Eknath Shinde case where BJP or the Eknath Shinde faction tried to give a sealed cover. The problem with sealed cover is for any defence there is nothing to fight against it. It could be a blank paper, it could be whole lot of gossip or innuendo, unless it s out in the open the prosecution in this case i.e. Udhav Thackeray team could not effectively fight as they do not know the contents of the said sealed cover. This goes against all judicial norms. The U.S. tried with sealed cover and ended the practise only after couple of cases. Even in the OROP case (One Rank One Pension) the Govt. tried the same trick.
As I have shared before, this actually comes Senator Joseph Mccarthy who started this whole thing in 1950 s, a bit of background on him. I probably had shared about him before but it bears to know and remember again and again. The same is what Mr. Trump has done time and again. It s a similar script. Bully your opponents and use sealed cover as no questions can be asked about it. The good news though is that the views of CJI have been changing. So, yes they would like to change the CJI, they actually have been trying to have their own man have keys to the judiciary but without success so far, soon though this bastion would also fall, if not today then tomorrow. The EC (Election Commission) has been thoroughly compromised. Would not share more as that EC s bending acts would use a whole blog post or two to share the numerous instances where BJP has been given all rights. In fact, via RTI it came to be known that many of the BJP leaders had given false statements about their education in the EC affidavit. That alone should have been grounds for throwing out the legislators but in their wisdom they see fit to remain blind. The latest case is of Mr. Nishikant Dubey.
Of course, even with the legal documents shared in public domain and EC having powers to verify such documents, they are choosing to remain silent. If one has lied about one thing in an affidavit, what or how many lies he has shared in that same document. And how are you supposed to trust anything that comes out of his mouth. This is the state of not just Mr. Dubey but a whole lot of people who are in BJP today. And EC as an institution seems to be let down. There was a time when it was lead by T.N. Seshan who was courageous, fearless and fair. He later went to become Speaker but even then he was harsh but fair. Perhaps people thought we will always have people like him. Hence instead of strong institutions having strong rules, we based our trust on people and hence we are where we are
There is much to share but some other day, Till later.
Given that B lint just braggedblogged about how
efficiently he can build a Linux
kernel
(less than 8 seconds, wow! Well, yes, until you read it is the
result of aggressive caching and is achieved only for a second run),
and that a question just popped up today on the Debian ARM mailing
list, is an ARM computer a good choice? Which
one? ,
I decided to share my results of an experiment I did several months
ago, to graphically show to my students the effects of parallelism,
the artifacts of hyperthreading, the effects of different architecture
sets, and even illustrate about the actual futility of my experiment
(somewhat referring to John Gustafson s reevaluation of Amdahl s
law, already 30 years
ago One does
not take a fixed-size problem and run it on various numbers of
processors except when doing academic research ; thanks for referring
to my inconsequential reiterative compilations as academic research! )
I don t expect any of the following images to be groundbreaking, but
at least, next time I need to find them it is quite likely I ll be
able to find them and I will be able to more easily refer to them in
online discussions
So What did I do? I compiled Linux repeatedly, on several of the
machines I had available, varying the -j flag (how many cores to use
simultaneously), starting with single-core, and pushing up until just
a bit over the physical number of cores the CPU has.
Sadly, I lost several of my output images, but the three following are
enough to tell interesting bits of the story:
A nice little server my Institute acquired in early 2021: Xeon
Silver 4208, with 8 physical cores (plus hyperthreading)
My laptop, an 8-ARM-core Lenovo Yoga C630. Do note it s a
big.LITTLE system, where 4 cores are smaller and 4 are bigger.
A Raspberry Pi 4 (8GB version)
Of course, I have to add that this is not a scientific comparison; the
server and my laptop have much better I/O than the Raspberry s puny
micro-SD card (and compiling hundreds of thousands of files is quite
an IO-stressed job, even though the full task does exhibit the very
low compared single-threaded performance of the Raspberry even
compared with the Yoga).
No optimizations were done (they would be harmful to the effects I
wanted to show!), the compile was made straight from the upstream
sources.
Utkarsh Gupta
did 24.25h (out of 49.25h assigned), thus carrying over 8.0h to the next month.
Evolution of the situation
In February, we have released 44 DLAs, which resolved 156 CVEs.
We are glad to welcome some new contributors who will hopefully help us fix CVEs in the supported release even faster.
However, we also experienced some setbacks as a few sponsors have stopped (or decreased) their support. If your company ever hesitated to sponsor Debian LTS, now might be a good time to join to ensure that we can continue this important work without having to scale down on the number of packages that we are able to support.
Thanks to our sponsors
Sponsors that joined recently are in bold.
If you're familiar with Elie Mystal's previous work (writer for The
Nation, previously editor for Above the Law, Twitter gadfly, and
occasional talking head on news commentary programs), you'll have a good
idea what to expect from this book: pointed liberal commentary, frequently
developing into rants once he works up a head of steam. The subtitle of
A Black Guy's Guide to the Constitution tells you that the topic is
US constitutional law, which is very on brand. You're going to get
succinct and uncompromising opinions at the intersection of law and
politics. If you agree with them, you'll probably find them funny; if you
disagree with them, you'll probably find them infuriating.
In other words, Elie Mystal is the sort of writer one reads less for "huh,
I disagreed with you but that's a good argument" and more for "yeah, you
tell 'em, Elie!" I will be very surprised if this book changes anyone's
mind about a significant political debate. I'm not sure if people who
disagree are even in the intended audience.
I'm leery of this sort of book. Usually its function is to feed
confirmation bias with some witty rejoinders and put-downs that only sound
persuasive to people who already agree with them. If I want that, I can
just read Twitter (and you will be unsurprised to know that Mystal has
nearly 500,000 Twitter followers). This style can also be boring at book
length if the author is repeating variations on a theme.
There is indeed a lot of that here, particularly in the first part of this
book. If you don't generally agree with Mystal already, save yourself the
annoyance and avoid this like the plague. It's just going to make you
mad, and I don't think you're going to get anything useful out of it. But
as I got deeper into this book, I think Mystal has another, more
interesting purpose that's aimed at people who do largely agree. He's
trying to undermine a very common US attitude (even on the left) about the
US constitution.
I don't know if most people from the US (particularly if they're white and
male) realize quite how insufferably smug we tend to be about the US
constitution. When you grow up here, the paeans to the constitution and
the Founding Fathers (always capitalized like deities) are so ubiquitous
and unremarked that it's difficult not to absorb them at a subconscious
level. There is a national mythology about the greatness of our charter
of government that crosses most political divides. In its modern form,
this comes with some acknowledgment that some of its original provisions
(the notorious three-fifths of a person clause, for instance) were bad,
but we subsequently fixed them and everything is good now. Nearly
everyone gets taught this in school, and it's almost never challenged.
Even the edifices of the US left, such as the ACLU and the NAACP, tend to
wrap themselves in the constitution.
It's an enlightening experience to watch someone from the US corner a
European with a discussion of the US constitution and watch the European
plan escape routes while their soul attempts to leave their body. And I
think it's telling that having that experience, as rare as it might be
given how oblivious we can be, is still more common than a white person
having a frank conversation with a black person in the US about the merits
of the constitution as written. For various reasons, mostly because this
is not very safe for the black person, this rarely happens.
This book is primarily Mystal giving his opinion on various current
controversies in constitutional law, but the underlying refrain is that
the constitution is a trash document written by awful people that sets up
a bad political system. That system has been aggressively defended by
reactionary Supreme Courts, which along with the designed difficulty of
the amendment process has prevented fixing many obviously broken parts.
This in turn has led to numerous informal workarounds and elaborate
"interpretations" to attempt to make the system vaguely functional.
In other words, Mystal is trying to tell the US reader to stop being so
precious about this specific document, and is using its truly egregious
treatment of black people as the main fulcrum for his argument. Along the
way, he gives an abbreviated tour of the highlights of constitutional law,
but if you're at all interested in politics you've probably heard most of
that before. The main point, I think, is to dig up any reverence left
over from a US education, haul it out into the light of day, and compare
it to the obvious failures of the constitution as a body of law and the
moral failings of its authors. Mystal then asks exactly why we should
care about original intent or be so reluctant to change the resulting
system of government.
(Did I mention you should not bother with this book if you don't agree
with Mystal politically? Seriously, don't do that to yourself.)
Readers of my reviews will know that I'm fairly far to the left
politically, particularly by US standards, and yet I found it fascinating
how much lingering reverence Mystal managed to dig out of me while reading
this book. I found myself getting defensive in places, which is absurd
because I didn't write this document. But I grew up surrounded by
nigh-universal social signaling that the US constitution was the greatest
political document ever, and in a religious tradition that often argued
that it was divinely inspired. If one is exposed to enough of this, it
becomes part of your background understanding of the world. Sometimes it
takes someone being deliberately provocative to haul it back up to the
surface where it can be examined.
This book is not solely a psychological intervention in national
mythology. Mystal gets into detailed legal arguments as well. I thought
the most interesting was the argument that the bizarre and unconvincing
"penumbras" and "emanations" reasoning in
Griswold
v. Connecticut (which later served as the basis of
Roe v. Wade) was
in part because the
Lochner
era Supreme Court had, in the course of trying to strike down all worker
protection laws, abused the concept of substantive due process so badly
that Douglas was unwilling to use it in the majority opinion and instead
made up entirely new law. Mystal argues that the Supreme Court should
have instead tackled the true meaning of substantive due process head-on
and decided Griswold on 14th Amendment equal protection and
substantive due process grounds. This is probably a well-known argument
in legal circles, but I'd not run into it before (and Mystal makes it far
more interesting and entertaining than my summary).
Mystal also joins the tradition of thinking of the Reconstruction
Amendments (the 13th, 14th, and 15th amendments passed after the Civil
War) as a second revolution and an attempt to write a substantially new
constitution on different legal principles, an attempt that subsequently
failed in the face of concerted and deadly reactionary backlash. I first
encountered this perspective via Jamelle Bouie, and it added a lot to my
understanding of Reconstruction to see it as a political fight about the
foundational principles of US government in addition to a fight over
continuing racism in the US south. Maybe I was unusually ignorant of it
(I know I need to read W.E.B. DuBois), but I think this line of reasoning
doesn't get enough attention in popular media. Mystal provides a good
introduction.
But, that being said, Allow Me to Retort is more of a vibes book
than an argument. As in his other writing, Mystal focuses on what he sees
as the core of a controversy and doesn't sweat the details too much. I
felt like he was less trying to convince me and more trying to model a
different way of thinking and talking about constitutional law that isn't
deferential to ideas that are not worthy of deference. He presents his
own legal analysis and possible solutions to current US political
challenges, but I don't think the specific policy proposals are the strong
part of this book. The point, instead, is to embrace a vigorous politics
based on a modern understanding of equality, democracy, and human rights,
without a lingering reverence for people who mostly didn't believe in any
of those things. The role of the constitution in that politics is a flawed
tool rather than a sacred text.
I think this book is best thought of as an internal argument in the US
left. That argument is entirely within the frame of the US legal
tradition, so if you're not in the US, it will be of academic interest at
best (and probably not even that). If you're on the US right, Mystal
offers lots of provocative pull quotes to enjoy getting outraged over, but
he provides that service on Twitter for free.
But if you are on the US left, I think Allow Me to Retort is worth
more consideration than I'd originally given it. There's something here
about how we engage with our legal history, and while Mystal's approach is
messy, maybe that's the only way you can get at something that's more
emotion than logic. In some places it degenerates into a Twitter rant,
but Mystal is usually entertaining even when he's ranting. I'm not sorry
I read it.
Rating: 7 out of 10
After my last post a few things have happened. First, I want to thank all of you for your support, monetary and moral. The open source community is amazing and I love being a part of it. We are surviving thanks to all of you. Despite my failed interview, a new door has opened up and I am happy to announce that Canonical is funding me to work part time for a period of 3 months on KDE Snaps! While not the full time role I was hoping for, I ll take it, and who knows, maybe they will see just how awesome I am!
I started back to work on these last Tuesday. So far I have made good progress and even have a working Core22 snap!
Work done on upstream snapcraft ( tool used to make snaps ):
New content pack version, fixed an issue with snapcraft remote-build choking on part with /sdk, fixed regex as our naming scheme has changed:
https://github.com/snapcore/snapcraft/pull/4069
Ran into a new bug with snapcraft expand-extensions and so I must now enter all of this information manually into snapcraft.yaml until fixed, bug report here:
https://bugs.launchpad.net/snapcraft/+bug/2011719
And without further ado, our first core22 snap is Dragonplayer Version 22.12.3 available in the snap store. Many more coming soon!
KDE Dragon media player
With a new month upon us, I must ask for help again, I will not see any money for this work until next month. Please consider a dontation. Thank you from the bottom of our hearts.
https://gofund.me/c9cc02ed
Some notes on using debvm, an amazing piece of software I ve started using
only recently.
Create a new virtual machine:
$ debvm-create
You now have a virtual machine with Debian Sid of your host native
architecture (probably amd64). The image file is called rootfs.ext4. You ve
got 1G of disk space in the VM.
You can now just run the VM! You will be automatically logged is as root.
$ debvm-run
Experiment in the VM, run all the sort of tests you have in mind. For example,
one thing I commonly do is verifying that things work in a clean environment,
as opposed to "on my machine".
If anything goes wrong, or if you just want to repeat the test: shutdown the
guest, remove rootfs.ext4, and start again with debvm-create.
Now, especially if you intend creating and recreating VMs multiple times, it
helps to use a local apt mirror as a cache to avoid fetching all the required
packages from the internet over and over again. Install apt-cacher-ng on
your host machine and point debvm-create at it:
$ debvm-create -- http://10.0.3.1:3142/debian
The additional options after -- are passed to mmdebstrap. In this case
we re specifying http://10.0.3.1:3142/debian as the URL of our local apt
mirror. It is going to be used both for image creation and as the only entry in
/etc/apt/sources.list on the guest. This is the reason for using 10.0.3.1,
the IP address of the lxcbr0 interface used by qemu, instead of 127.0.0.1: to
make sure that the guest VM has access to it too.
For a slightly more advanced example, we now want to:
run a arm64 VM
have more disk space availably, say 2G
install additional packages (curl and locales)
allow SSH as root with the given public keys, in this example the
authorized_keys installed on the host
start the VM in the background with no console output
Core Python Packages, by Stefano Rivera
Just before the freeze, pip added support for PEP-668.
This is a scheme devised by Debian with other distributions and the Python
Packaging Authority, to allow distributors to mark Python installations as
being managed by a distribution package manager. When this EXTERNALLY-MANAGED
flag is present, installers like pip will refuse to install packages outside
a virtual environment. This protects users from breaking unrelated software
on their systems, when installing packages with pip or similar tools. Stefano
quickly got this version of pip into the archive, marked Debian s Python
interpreters as EXTERNALLY-MANAGED, and worked with the upstream to add a
mechanism to allow users to override the restriction. Debian bookworm will
likely be the first distro release to implement this change.
The transition from Python 3.10 to 3.11 was one of the last to complete before
the bookworm freeze (as 3.11 only released at the end of October 2022). Stefano
helped port some Python packages to 3.11, in January, and also kicked off the
final transition to remove Python 3.10 support.
Stefano did a big round of bug triage in the cPython interpreter (and related)
packages, applying some provided patches, and fixing some long-standing minor
bugs in the packaging.
To allow Debian packages to more accurately reflect upstream-specified
dependencies that only apply under specific Python interpreter versions, in the
future, Stefano
added more metadata
to the python3 binary package.
Python s unittest runner would successfully exit with 0 passed tests, if it
couldn t find any tests. This means that configuration / layout changes can
cause test failures to go unnoticed, because the tests aren t being run any
more in Debian packages. Stefano
proposed a change to Python
3.12 to change this behavior and treat 0 tests as a kind of failure.
debvm, by Helmut Grohne
With support from Johannes Schauer Marin Rodrigues, and Jochen Sprickerhof,
Helmut Grohne wrote debvm, a tool for
quickly creating and running Debian virtual machine images for various
architectures and Debian and Ubuntu releases. This is meant for development
and testing purposes and has already identified a number of bugs in e.g.
fakechroot (#1029490), Linux
(#1029270), and runit
(#1028181).
Rails 6 and Redmine 5 available in bullseye-backports, by Utkarsh Gupta
Bullseye users can now upgrade to the latest 6.1 branch of Rails, v6.1.7, and
the latest Redmine version, v5.0.4. The Ruby team received numerous requests
to backport the latest version of Rails and Redmine, especially since there was
no redmine shipped in the bullseye release itself. So this is big news for all
users as we ve not only successfully backported both the packages, but also
fixed all the CVEs and RC bugs in the process!
This work was sponsored by Entrouvert.
Patches metadata in the Package Tracker, by Rapha l Hertzog
Building on the great Ultimate Debian Database work of Lucas Nussbaum and on
his suggestion, Rapha l enhanced the
Debian Package Tracker to display action items
when the patches metadata indicate that some patches were not forwarded
upstream, or when the metadata were invalid. One can now also browse the
patches metadata from the Links panel on the right.
Fixed kernel bug that broke debian-installer on computers with Mediatek wifi devices, by Helmut Grohne
As part of our regular work on Kali Linux for OffSec,
they funded Helmut s work to fix the MT7921e driver. When being loaded without
firmware available, it would not register itself, but upon module release it
would unregister itself causing a kernel oops.
This was commonly observed in Kali Linux when reloading the module to add
firmware. Helmut Grohne identified the cause and
sent a patch, a
different variant of which is now
heading into Linux
and available from Kali Linux.
Printing in Debian, by Thorsten Alteholz
There are about 40 packages in Debian that take care of sending output to
printers, scan documents, or even send documents to fax machines. In the light
of the upcoming/already ongoing freeze, these packages had to be updated to
the latest version and bugs had to be fixed. Basically this applies to large
packages like cups, cups-filters, hplip but also the smaller ones that
shouldn t be neglected. All in all Thorsten uploaded 13 packages with new
upstream versions or improved packaging and could resolve 14 bugs. Further
triaging led to 35 bugs that could be closed, either because they were already
fixed and not closed in an earlier upload or they could not be reproduced with
current software versions.
There is also work to do to prepare for the future. Historically, printing on
Linux required finding a PPD file for your printer and finding some software
that is able to render your documents with this PPD. These days, driverless
printing is becoming more common and the use of PPD files has decreased.
In the upcoming version 3.0 of cups, PPD files are no longer supported and so
called printer applications need to be used. In order not to lose the ability
to print documents, this big transition needs to be carefully planned. This
started in the beginning of 2023 and will hopefully be finished with the
release of Debian Trixie. More information can be found in
this Debian Printing Wiki article.
In preparation for this transition Thorsten created three new packages.
Yade update, by Anton Gladky
Last month, Anton updated the yade package to the newest 2023.02a version,
which includes new features.
Yade is a software package for discrete element method (DEM) simulations, which
are widely used in scientific and engineering fields for the simulation of
granular systems. Yade is an open-source project that is being used worldwide
for different tasks, such as geomechanics, civil engineering, mining, and
materials science.
The Yade package in Debian supports different precision levels for its
simulations. This means that researchers and engineers can select the needed
precision level without recompiling the package, saving time and effort.
Miscellaneous contributions
Helmut Grohne continues to improve cross building (mostly Qt) and
architecture bootstrap (mostly loong64 and musl).
After reading B lint s blog post about Firebuild (a compile cache) [1] I decided to give it a go. It s non-free, the project web site [2] says that it s free for non-commercial use or commercial trials.
My first attempt at building a Debian package failed due to man-recode using a seccomp() sandbox, I filed Debian bug #1032619 [3] about this (thanks for the quick response B lint). The solution for me was to edit /etc/firebuild.conf and add man-recode to the dont_intercept list. The new version that s just been uploaded to Debian fixes it by disabling seccomp() and will presumably allow slightly better performance.
Here are the results of building the refpolicy package with Firebuild, a regular build, the first build with Firebuild (30% slower) and a rebuild with Firebuild that reduced the time by almost 42%.
real 1m32.026s
user 4m20.200s
sys 2m33.324s
real 2m4.111s
user 6m31.769s
sys 3m53.681s
real 0m53.632s
user 1m41.334s
sys 3m36.227s
Next I did a test of building a Linux 6.1.10 kernel with make bzImage -j18 , here are the results from a normal build, first build with firebuild, and second build. The real time is worse with firebuild for this on my machine. I think that the relative speeds of my CPU (reasonably fast 18 core) and storage (two of the slower NVMe devices in a BTRFS RAID-1) is the cause of the first build being relatively so much slower for make bzImage than for building the refpolicy, as the kernel build process involves a lot more data. For the final build I moved ~/.cache/firebuild to a tmpfs (I have 128G of RAM and not much running on my machine at the time of the tests), even then building with firebuild was slightly slower in real time but took significantly less CPU time (user+real being 20mins instead of 36m). I also ran several tests with the kernel source tree on a tmpfs but for unknown reasons those tests each took about 6 minutes. Does firebuild or the Linux kernel build process dislike tmpfs for some reason?
real 2m43.020s
user 31m30.551s
sys 5m15.279s
real 8m49.675s
user 64m11.258s
sys 19m39.016s
real 3m6.858s
user 7m47.556s
sys 9m22.513s
real 2m51.910s
user 10m53.870s
sys 9m21.307s
One thing I noticed from the kernel build tests is that the total CPU time taken by the firebuild process (as reported by ps) was more than 2/3 of the run time and top usually reported it as taking around 75% of a CPU core. It seems to me that the firebuild process itself is a bottleneck on build speed. Building refpolicy without firebuild has an average of 4.5 cores in use while building the kernel haas 13.5. Unless they make a multi-threaded version of firebuild it seems that it won t give the performance one would hope for from a CPU with 18+ cores. I presume that if I had been running with hyper-threading enabled then firebuild would have been even worse for kernel builds as it would sometimes get on the second thread of a core. It looks like firebuild would perform better on AMD CPUs as they tend to have fewer CPU cores with greater average performance per core so a single CPU core for firebuild will be less limited. I presume that the firebuild developers will make it perform better with large numbers of cores in future, the latest Intel laptop CPUs have 16+ cores and servers with 2*40core CPUs are common.
The performance improvement for refpolicy is significant as a portion of build time, but insignificant in terms of real time. A full build of refpolicy doesn t take enough time to get a Coke and reducing it doesn t offer a huge benefit, if Firebuild was available in past years when refpolicy took 20 minutes to build (when DDR2 was the best RAM available) then it would be a different story.
There is some potential to optimise the build of refpolicy for the non-firebuild case. Getting it to average more than 4.5 cores in use when there s 18 available should be possible, there are a number of shell for loops in the main Makefile and maybe some of them can be replaced by make constructs to allow running in parallel. If it used 7 cores on average then it would be faster in a regular build than it currently is with firebuild and a hot cache. Any advice from make experts would be appreciated.
The importance of clean, fresh indoor air is one of the most tangible takeaways of the Covid-19 pandemic. In addition to being an effective risk mitigation strategy for reducing the spread of respiratory illnesses, clean, fresh air is necessary to enable effective cognitive performance.
Monitoring indoor air quality is relatively easy to do, but traditionally has not been a key focus. I believe air quality monitoring should be accessible for any indoor space, and for highly occupied indoor spaces should be provided on a continuous basis. This post explores the need and an opportunity for a business that can accelerate the adoption of ventilation monitoring through the following topics:
Nitrokey Start
Phew!
Today, after four days of combing through code I am unfamiliar with, I
was finally able to change my expression.
I m finally at the part of my PhD work where I am tasked with
implementing the protocol I claim improves from the current
situation. I wrote a script to deploy the infrastructure I need for
the experiment, and was not expecting any issues I am not (yet)
familiar with the Go language (in which 
Github 
Update on 2023-03-23: thanks to Daniel Roschka for mentioning the 
Anti-national and Anti-India judges Kiren Rijiju, Law Minister.
So What did I do? I compiled Linux repeatedly, on several of the
machines I had available, varying the 
Like each month, have a look at the work funded by 
Snowy Sunrise Witch Wells, Az
I started back to work on these last Tuesday. So far I have made good progress and even have a working Core22 snap!
Work done on upstream snapcraft ( tool used to make snaps ):
New content pack version, fixed an issue with snapcraft remote-build choking on part with /sdk, fixed regex as our naming scheme has changed:
KDE Dragon media player