Apparently I got an honour from OpenUK. There are a bunch of people I know on that list. Chris Lamb and Mark Brown are familiar names from Debian. Colin King and Jonathan Riddell are people I know from past work in Ubuntu. I ve admired David MacIver s work on Hypothesis and Richard Hughes work on firmware updates from afar. And there are a bunch of other excellent projects represented there: OpenStreetMap, Textualize, and my alma mater of Cambridge to name but a few. My friend Stuart Langridge wrote about being on a similar list a few years ago, and I can t do much better than to echo it: in particular he wrote about the way the open source development community is often at best unwelcoming to people who don t look like Stuart and I do. I can t tell a whole lot about demographic distribution just by looking at a list of names, but while these honours still seem to be skewed somewhat male, I m fairly sure they re doing a lot better in terms of gender balance than my home project of Debian is, for one. I hope this is a sign of improvement for the future, and I ll do what I can to pay it forward.

Krebs on Security has an informative article about web notifications and how they are being used for spamming and promoting malware [1]. He also includes links for how to permanently disable them. If nothing else clicking no on each new site that wants to send notifications is annoying. Michael Stapelberg wrote an insightful posts about inefficiencies in the Debian development processes [2]. While I agree with most of his assessment of Debian issues I am not going to decrease my involvement in Debian. Of the issues he mentions the 2 that seem to have the best effort to reward ratio are improvements to mailing list archives (to ideally make it practical to post to lists without subscribing and read responses in the archives) and the issues of forgetting all the complexities of the development process which can be alleviated by better Wiki pages. In my Debian work I ve contributed more to the Wiki in recent times but not nearly as much as I should. Jacobin has an insightful article Ending Poverty in the United States Would Actually Be Pretty Easy [3]. Mark Brown wrote an interesting blog post about the Rust programming language [4]. He links to a couple of longer blog posts about it. Rust has some great features and I ve been meaning to learn it. Scientific America has an informative article about research on the spread of fake news and memes [5]. Something to consider when using social media. Bruce Schneier wrote an insightful blog post on whether there should be limits on persuasive technology [6]. Jonathan Dowland wrote an interesting blog post about git rebasing and lab books [7]. I think it s an interesting thought experiment to compare the process of developing code worthy of being committed to a master branch of a VCS to the process of developing a Ph.D thesis. CBS has a disturbing article about the effect of Covid19 on people s lungs [8]. Apparently it usually does more lung damage than long-term smoking and even 70%+ of people who don t have symptoms of the disease get significant lung damage. People who live in heavily affected countries like the US now have to worry that they might have had the disease and got lung damage without knowing it. Russ Allbery wrote an interesting review of the book Because Internet about modern linguistics [9]. The topic is interesting and I might read that book at some future time (I have many good books I want to read). Jonathan Carter wrote an interesting blog post about CentOS Streams and why using a totally free OS like Debian is going to be a better option for most users [10]. Linus has slammed Intel for using ECC support as a way of segmenting the market between server and desktop to maximise profits [11]. It would be nice if a company made a line of Ryzen systems with ECC RAM support, but most manufacturers seem to be in on the market segmentation scam. Russ Allbery wrote an interesting review of the book Can t Even about millenials as the burnout generation and the blame that the corporate culture deserves for this [12].

• [1] https://tinyurl.com/y37jh9t4
• [2] https://tinyurl.com/y45o88ma
• [3] https://www.jacobinmag.com/2020/12/broke-in-america-ending-us-poverty
• [4] https://tinyurl.com/y98llves
• [5] https://tinyurl.com/yy67g9nq
• [6] https://tinyurl.com/y99wjkvx
• [7] https://jmtd.net/log/git_rebasing_and_lab_books/
• [8] https://tinyurl.com/yy6l22x4
• [9] https://tinyurl.com/yxtwtbhe
• [10] https://jonathancarter.org/2020/12/10/centos-stream-or-debian/
• [11] https://tinyurl.com/y3kolzzs
• [12] https://tinyurl.com/y4v9xglj

Earlier this month Daniel, Lars and myself got together to discuss Bryan Cantrill s article Rust after the honeymoon. This is an overview of what keeps him enjoying working with Rust after having used it for an extended period of time for low level systems work at Oxide, we were particularly interested to read a perspective from someone who was both very experienced in general and had been working with the language for a while. While I have no experience with Rust both Lars and Daniel have been using it for a while and greatly enjoy it. One of the first areas we discussed was data bearing enums these have been very important to Bryan. In keeping with a pattern we all noted these take a construct that s relatively commonly implemented by hand in C (or skipped as too much effort, as Lars found) and provides direct support in the language for it. For both Daniel and Lars this has been key to their enjoyment of Rust, it makes things that are good practice or common idioms in C and C++ into first class language features which makes them more robust and allows them to fade into the background in a way they can t when done by hand. Daniel was also surprised by some omissions, some small such as the ? operator but others much more substantial the standout one being editions. These aim to address the problems seen with version transitions in other languages like Python, allowing individual parts of a Rust program to adopt potentially incompatible language features while remaining interoperability with older editions of the language rather than requiring the entire program to be upgraded en masse. This helps Rust move forwards with less need to maintain strict source level compatibility, allowing much more rapid evolution and helping deal with any issues that are found. Lars expressed the results of this very clearly, saying that while lots of languages offer a 20%/80% solution which does very well in specific problem domains but has issues for some applications Rust is much more able to move towards a much more general applicability by addressing problems and omissions as they are understood. This distracted us a bit from the actual content of the article and we had an interesting discussion of the issues with handling OS differences in filenames portably. Rather than mapping filenames onto a standard type within the language and then have to map back out into whatever representation the system actually uses Rust has an explicit type for filenames which must be explicitly converted on those occasions when it s required, meaning that a lot of file handling never needs to worry about anything except the OS native format and doesn t run into surprises. This is in keeping with Rust s general approach to interfacing with things that can t be represented in its abstractions, rather than hide things it keeps track of where things that might break the assumptions it makes are and requires the programmer to acknowledge and handle them explicitly. Both Lars and Daniel said that this made them feel a lot more confident in the code that they were writing and that they had a good handle on where complexity might lie, Lars noted that Rust is the first languages he s felt comfortable writing multi threaded code in. We all agreed that the effect here was more about having idioms which tend to be robust and both encourage writing things well and gives readers tools to help know where particular attention is required no tooling can avoid problems entirely. This was definitely an interesting discussion for me with my limited familiarity with Rust, hopefully Daniel and Lars also got a lot out of it!

This month for our book club Daniel, Lars, Vince and I read Hardcoded secrets, unverified tokens, and other common JWT mistakes which wasn t quite what we d thought when it was picked. We had been expecting an analysis of JSON web tokens themselves as several us had been working in the area and had noticed various talk about problems with the standard but instead the article is more a discussion of the use of semgrep to find and fix common issues, using issues with JWT as examples. We therefore started off with a bit of a discussion of JWT, concluding that the underlying specification was basically fine given the problem to be solved but that as with any security related technology there were plenty of potential pitfalls in implementation and that sadly many of the libraries implementing the specification make it far too easy to make mistakes such as those covered by the article through their interface design and defaults. For example interfaces that allow interchangable use of public keys and shared keys are error prone, as is is making it easy to access unauthenticated data from tokens without clearly flagging that it is unauthenticated. We agreed that the wide range of JWT implementations available and successfully interoperating with each other is a sign that JWT is getting something right in providing a specification that is clear and implementable. Moving on to semgrep we were all very enthusiastic about the technology, language independent semantic matching with a good set of rules for a range of languages available. Those of us who work on the Linux kernel were familiar with semantic matching and patching as implemented by Coccinelle which has been used quite successfully for years to both avoiding bad patterns in code and making tree wide changes, as demonstrated by the article it is a powerful technique. We were impressed by the multi-language support and approachability of semgrep, with tools like their web editor seeming particularly helpful for people getting started with the tool, especially in conjunction with the wide range of examples available. This was a good discussion (including the tangential discussions of quality problems we had all faced dealing with software over the years, depressing though those can be) and semgrep was a great tool to learn about, I know I m going to be using it for some of my projects.

Previously: v5.5. Linux v5.6 was released back in March. Here s my quick summary of various features that caught my attention: WireGuard
The widely used WireGuard VPN has been out-of-tree for a very long time. After 3 1/2 years since its initial upstream RFC, Ard Biesheuvel and Jason Donenfeld finished the work getting all the crypto prerequisites sorted out for the v5.5 kernel. For this release, Jason has gotten WireGuard itself landed. It was a twisty road, and I m grateful to everyone involved for sticking it out and navigating the compromises and alternative solutions. openat2() syscall and RESOLVE_* flags
Aleksa Sarai has added a number of important path resolution scoping options to the kernel s open() handling, covering things like not walking above a specific point in a path hierarchy (RESOLVE_BENEATH), disabling the resolution of various magic links (RESOLVE_NO_MAGICLINKS) in procfs (e.g. /proc/$pid/exe) and other pseudo-filesystems, and treating a given lookup as happening relative to a different root directory (as if it were in a chroot, RESOLVE_IN_ROOT). As part of this, it became clear that there wasn t a way to correctly extend the existing openat() syscall, so he added openat2() (which is a good example of the efforts being made to codify Extensible Syscall arguments). The RESOLVE_* set of flags also cover prior behaviors like RESOLVE_NO_XDEV and RESOLVE_NO_SYMLINKS. pidfd_getfd() syscall
In the continuing growth of the much-needed pidfd APIs, Sargun Dhillon has added the pidfd_getfd() syscall which is a way to gain access to file descriptors of a process in a race-less way (or when /proc is not mounted). Before, it wasn t always possible make sure that opening file descriptors via /proc/$pid/fd/$N was actually going to be associated with the correct PID. Much more detail about this has been written up at LWN. openat() via io_uring
With my attack surface reduction hat on, I remain personally suspicious of the io_uring() family of APIs, but I can t deny their utility for certain kinds of workloads. Being able to pipeline reads and writes without the overhead of actually making syscalls is pretty great for performance. Jens Axboe has added the IORING_OP_OPENAT command so that existing io_urings can open files to be added on the fly to the mapping of available read/write targets of a given io_uring. While LSMs are still happily able to intercept these actions, I remain wary of the growing syscall multiplexer that io_uring is becoming. I am, of course, glad to see that it has a comprehensive (if out of tree ) test suite as part of liburing. removal of blocking random pool
After making algorithmic changes to obviate separate entropy pools for random numbers, Andy Lutomirski removed the blocking random pool. This simplifies the kernel pRNG code significantly without compromising the userspace interfaces designed to fetch cryptographically secure random numbers. To quote Andy, This series should not break any existing programs. /dev/urandom is unchanged. /dev/random will still block just after booting, but it will block less than it used to. See LWN for more details on the history and discussion of the series. arm64 support for on-chip RNG
Mark Brown added support for the future ARMv8.5 s RNG (SYS_RNDR_EL0), which is, from the kernel s perspective, similar to x86 s RDRAND instruction. This will provide a bootloader-independent way to add entropy to the kernel s pRNG for early boot randomness (e.g. stack canary values, memory ASLR offsets, etc). Until folks are running on ARMv8.5 systems, they can continue to depend on the bootloader for randomness (via the UEFI RNG interface) on arm64. arm64 E0PD
Mark Brown added support for the future ARMv8.5 s E0PD feature (TCR_E0PD1), which causes all memory accesses from userspace into kernel space to fault in constant time. This is an attempt to remove any possible timing side-channel signals when probing kernel memory layout from userspace, as an alternative way to protect against Meltdown-style attacks. The expectation is that E0PD would be used instead of the more expensive Kernel Page Table Isolation (KPTI) features on arm64. powerpc32 VMAP_STACK
Christophe Leroy added VMAP_STACK support to powerpc32, joining x86, arm64, and s390. This helps protect against the various classes of attacks that depend on exhausting the kernel stack in order to collide with neighboring kernel stacks. (Another common target, the sensitive thread_info, had already been moved away from the bottom of the stack by Christophe Leroy in Linux v5.1.) generic Page Table dumping
Related to RISCV s work to add page table dumping (via /sys/fs/debug/kernel_page_tables), Steven Price extracted the existing implementations from multiple architectures and created a common page table dumping framework (and then refactored all the other architectures to use it). I m delighted to have this because I still remember when not having a working page table dumper for ARM delayed me for a while when trying to implement upstream kernel memory protections there. Anything that makes it easier for architectures to get their kernel memory protection working correctly makes me happy. That s in for now; let me know if there s anything you think I missed. Next up: Linux v5.7.

2020, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 License.

Recently I was part of a call with Daniel and Lars to discuss Zettelkasten, a system for building up a cross-referenced archive of notes to help with research and study that has been getting a lot of discussion recently, the key thing being the building of links between ideas. Tomas Vik provided an overview of the process that we all found very helpful, and the information vs knowledge picture in Eugene Yan s blog on the topic (by @gapingvoid) really helped us crystalize the goals. It s not at all new and as Lars noted has a lot of similarities with a wikis in terms of what it produces but it couples this with an emphasis on the process and constant generation of new entries which Daniel found similar to some of the Getting Things Done recommendations. We all liked the emphasis on constant practice and how that can help build skills around effective note taking, clear writing and building links between ideas. Both Daniel and Lars already have note taking practicies that they find useful, combinations of journalling and building up collections of notes of learnings over time, and felt that there could be value in integrating aspects of Zettelkasten into these practices so we talked quite a bit about how that could be done. There was a consensus that journalling is useful so the main idea we had was to keep maintaining the journal, using that as an inbox and setting aside time to write entries into a Zettelkasten. This is also a useful way to approach recording things when away from a computer, taking notes and then writing them up later. Daniel suggested that one way to migrate existing notes might be to simply start anew, moving things over from old notes as required and then after a suitably long period (for example a year) review anything that was left and migrate anything that was left. We were all concerned about the idea of using any of the non-free solutions for something that is intended to be used long term, especially where the database isn t in an easily understood format. Fortunately there are free software tools like Zettlr which seem to address these concerns well. This was a really useful discussion, it really helps to bounce ideas off each other and this was certainly an interesting topic to learn about with some good ideas which will hopefully be helpful to us.

A short while ago Daniel, Lars and I met to discuss Russ Cox s excellent essay Our Software Dependency Problem. This essay looks at software reuse in general, especially in the context of modern distribution methods like PyPI and NPM which make the whole process much more frictionless than traditional distribution methods used with languages like C. Possibly our biggest conclusion was that the essay is so eminently sensible that we mostly just talked about how much we agreed with it and how comprehensive it was, we particularly admired the clarity with which it explores how to evaluate the quality of free software projects. Next time we ll have to pick something more controversial to discuss!

I recently got a Bronica SQ-Ai medium format film camera which came with the Motor Drive SQ-i. Since I couldn t find any documentation at all about it on the internet and had to figure it out for myself I figured I d put what I figured out here. Hopefully this will help the next person trying to figure one out, or at least by virtue of being wrong on the internet I ll be able to get someone who knows what they re doing to tell me how the thing really works. The motor drive attaches to the camera using the tripod socket, a replacement tripod socket is provided on the base of plate. There s also a metal plate with the bottom of the hand grip attached to it held on to the base plate with a thumb screw. When this is released it gives access to the screw holding in the battery compartment which (very conveniently) takes 6 AA batteries. This also provides power to the camera body when attached. On the back of the base of the camera there s a button with a red LED next to it which illuminates slightly when the button is pressed (it s visible in low light only). I m not 100% sure what this is for, I d have guessed a battery check if the light were easier to see. On the top of the camera there is a hot shoe (with a plastic blanking plate, a nice touch), a mode selector and two buttons. The larger button on the front replicates the shutter release button on the body (which continues to function as well) while the smaller button to the rear of the camera controls the motor depending on the current state of the camera it cocks the shutter, winds the film and resets the mirror when it is locked up. The mode dial offers three modes: off, S and C. S and C appear to correspond to the S and C modes of the main camera, single and continuous mirror lockup shots. Overall with this grip fitted and a prism attached the camera operates very similarly to a 35mm SLR in terms of film winding and so on. It is of course heavier (the whole setup weighs in at 2.5kg) but balanced very well and the grip is very comfortable to use.

It s really common for pitches to managements within companies about Linux kernel upstreaming to focus on cost savings to vendors from getting their code into the kernel, especially in the embedded space. These benefits are definitely real, especially for vendors trying to address the general market or extend the lifetime of their devices, but they are only part of the story. The other big thing that happens as a result of engaging upstream is that this is a big part of how other upstream developers become aware of what sorts of hardware and use cases there are out there. From this point of view it s often the things that are most difficult to get upstream that are the most valuable to talk to upstream about, but of course it s not quite that simple as a track record of engagement on the simpler drivers and the knowledge and relationships that are built up in that process make having discussions about harder issues a lot easier. There are engineering and cost benefits that come directly from having code upstream but it s not just that, the more straightforward upstreaming is also an investment in making it easier to work with the community solve the more difficult problems. Fundamentally Linux is made by and for the people and companies who show up and participate in the upstream community. The more ways people and companies do that the better Linux is likely to meet their needs.

This weekend Toby Churchill kindly hosted a hacking weekend for OpenTAC myself, Michael Grzeschik, Steve McIntyre and Andy Simpkins got together to bring up the remaining bits of the hardware on the current board revision and get some of the low level tooling like production flashing for the FTDI serial ports on the board up and running. It was a very productive weekend, we verified that everything was working with only few small mods needed for the board . Personally the main thing I worked on was getting most of an initial driver for the EMC1701 written. That was the one component without Linux support and allowed us to verify that the power switching and measurement for the systems under test was working well. There s still at least one more board revision and quite a bit of software work to do (I m hoping to get the EMC1701 upstream for v4.8) but it was great to finally see all the physical components of the system working well and see it managing a system under test, this board revision should support all the software development that s going to be needed for the final board. Thanks to all who attended, Pengutronix for sponsoring Michael s attendance and Toby Churchill for hosting!

I attended this year's Linux Kernel Summit in Santa Fe, NM, USA and made notes on some of the sessions that were relevant to Debian. LWN also reported many of the discussions. This is the second and last part of my notes; part 1 is here. Kernel Hardening Kees Cook presented the ongoing work on upstream kernel hardening, also known as the Kernel Self-Protection Project or KSPP. GCC plugins The kernel build system can now build and use GCC plugins to implement some protections. This requires gcc 4.5 and the plugin headers installed. It has been tested on x86, arm, and arm64. It is disabled by CONFIG_COMPILE_TEST because CI systems using allmodconfig/allyesconfig probably don't have those installed, but this ought to be changed at some point. There was a question as to how plugin headers should be installed for cross-compilers or custom compilers, but I didn't hear a clear answer to this. Kees has been prodding distribution gcc maintainers to package them. Mark Brown mentioned the Linaro toolchain being widely used; Kees has not talked to its maintainers yet. Probabilistic protections These protections are based on hidden state that an attacker will need to discover in order to make an effective attack; they reduce the probability of success but don't prevent it entirely. Kernel address space layout randomisation (KASLR) has now been implemented on x86, arm64, and mips for the kernel image. (Debian enables this.) However there are still lots of information leaks that defeat this. This could theoretically be improved by relocating different sections or smaller parts of the kernel independently, but this requires re-linking at boot. Aside from software information leaks, the branch target predictor on (common implementations of) x86 provides a side channel to find addresses of branches in the kernel. Page and heap allocation, etc., is still quite predictable. struct randomisation (RANDSTRUCT plugin from grsecurity) reorders members in (a) structures containing only function pointers (b) explicitly marked structures. This makes it very hard to attack custom kernels where the kernel image is not readable. But even for distribution kernels, it increases the maintenance burden for attackers. Deterministic protections These protections block a class of attacks completely. Read-only protection of kernel memory is either mandatory or enabled by default on x86, arm, and arm64. (Debian enables this.) Protections against execution of user memory in kernel mode are now implemented in hardware on x86 (SMEP, in Intel processors from Skylake onward) and on arm64 (PXN, from ARMv8.1). But Skylake is not available for servers and ARMv8.1 is not yet implemented at all! s390 always had this protection. It may be possible to 'emulate' this using other hardware protections. arm (v7) and arm64 now have this, but x86 doesn't. Linus doesn't like the overhead of previously proposed implementations for x86. It is possible to do this using PCID (in Intel processors from Sandy Bridge onward), which has already been done in PaX - and this should be fast enough. Virtually mapped stacks protect against stack overflow attacks. They were implemented as an option for x86 only in 4.9. (Debian enables this.) Copies to or from user memory sometimes use a user-controlled size that is not properly bounded. Hardened usercopy, implemented as an option in 4.8 for many architectures, protects against this. (Debian enables this.) Memory wiping (zero on free) protects against some information leaks and use-after-free bugs. It was already implemented as debug feature with non-zero poison value, but at some performance cost. Zeroing can be cheaper since it allows allocator to skip zeroing on reallocation. That was implemented as an option in 4.6. (Debian does not currently enable this but we might do if the performance cost is low enough.) Constification (with the CONSTIFY gcc plugin) reduces the amount of static data that can be written to. As with RANDSTRUCT, this is applied to function pointer tables and explicitly marked structures. Instances of some types need to be modified very occasionally. In PaX/Grsecurity this is done with pax_ open,close _kernel() which globally disable write protection temporarily. It would be preferable to override write protection in a more directed way, so that the permission to write doesn't leak into any other code that interrupts this process. The feature is not in mainline yet. Atomic wrap detction protects against reference-counting bugs which can result in a use-after-free. Overflow and underflow are trapped and result in an 'oops'. There is no measurable performance impact. It would be applied to all operations on the atomic_t type, but there needs to be an opt-out for atomics that are not ref-counters - probably by adding an atomic_wrap_t type for them. This has been implemented for x86, arm, and arm64 but is not in mainline yet. Kernel Freezer Hell For the second year running, Jiri Kosina raised the problem of 'freezing' kthreads (kernel-mode threads) in preparation for system suspend (suspend to RAM, or hibernation). What are the semantics? What invariants should be met when a kthread gets frozen? They are not defined anywhere. Most freezable threads don't actually need to be quiesced. Also many non-freezable threads are pointlessly calling try_to_freeze() (probably due to copying code without understanding it)). At a system level, what we actually need is I/O and filesystem consistency. This should be achieved by:

• Telling mounted filesystems to freeze. They can quiesce any kthreads they created.
• Device drivers quiescing any kthreads they created, from their PM suspend implementation.

The system suspend code should not need to directly freeze threads. Kernel Documentation Jon Corbet and Mauro Carvalho presented the recent work on kernel documentation. The kernel's documentation system was a house of cards involving DocBook and a lot of custom scripting. Both the DocBook templates and plain text files are gradually being converted to reStructuredText format, processed by Sphinx. However, manual page generation is currently 'broken' for documents processed by Sphinx. There are about 150 files at the top level of the documentation tree, that are being gradually moved into subdirectories. The most popular files, that are likely to be referenced in external documentation, have been replaced by placeholders. Sphinx is highly extensible and this has been used to integrate kernel-doc. It would be possible to add extensions that parse and include the MAINTAINERS file and Documentation/ABI/ files, which have their own formats, but the documentation maintainers would prefer not to add extensions that can't be pushed to Sphinx upstream. There is lots of obsolete documentation, and patches to remove those would be welcome. Linus objected to PDF files recently added under the Documentation/media directory - they are not the source format so should not be there! They should be generated from the corresponding SVG or image files at build time. Issues around Tracepoints Steve Rostedt and Shuah Khan led a discussion about tracepoints. Currently each maintainer decides which tracepoints to create. The cost of each added tracepoint is minimal, but the cost of very many tracepoints is more substantial. So there is such a thing as too many tracepoints, and we need a policy to decide when they are justified. They advised not to create tracepoints just in case, since kprobes can be used for tracing (almost) anywhere dynamically. There was some support for requiring documentation of each new tracepoint. That may dissuade introduction of obscure tracepoints, but also creates a higher expectation of stability. Tools such as bcc and IOVisor are now being created that depend on specific tracepoints or even function names (through kprobes). Should we care about breaking them? Linus said that we should strive to be polite to developers and users relying on tracepoints, but if it's too painful to maintain a tracepoint then we should go ahead and change it. Where the end users of the tool are themselves developers it's more reasonable to expect them to upgrade the tool and we should care less about changing it. In some cases tracepoints could provide dummy data for compatibility (as is done in some places in procfs).

Last weekend saw a small group get togeather in Cambridge to hack on the OpenTAC. OpenTAC is an OpenHardware OpenSoftware test platform, designed specificly to aid automated testing and continious intergration. Aimed at small / mobile / embedded targets OpenTAC v1 provides all of the support infrastructure to drive up to 8 DUTs (Device Under Test) to your test or CI system.
Each of the 8 EUT ports provides:

• A serial port (either RS232 levels on an DB9 socket, or 3V3 TTL on a molex kk plug)
• USB Power (up-to 2A with a software defined fuse, and alarm limits)
• USB data interconnect
• Ethernet

All ports on the EUT interface are relay issolated, this means that cables to your EUT can be unplugged under software control (we are aware of several SoC development boards that latch up if there is a serial port connected before power is applied). Additionly there are 8 GPIO lines that can be used as switch controls to any EUT (perhaps to put a specific EUT into a programming mode, reboot it or even start it) Anyway, back to the hacking weekend. .. Joining Steve McIntyre and myself were Mark Brown, and Michael Grzeschik (sorry Michael, I couldn t find a homepage). Mark traveled down from Scotland whilst Michael flew in from Germany for the weekend. Gents we greatly apprecate you taking the time and expence to join us this weekend. I should also thank my employer Toby Churchill Ltd. for allowing us to use the office to host the event. A lot of work got done, and I beleive we have now fully tested and debugged the hardware. We have also made great progress with the device tree and dvice drivers for the platform. Mark got the EUT power system working as proof of concept, and has taken an OpenTAC board back with him to turn this into suitable drivers and hopfully push them up stream. Meanwhile Michael spent his time working on the system portion of the device tree; OpenTAC s internal power sequancing, thermal managment subsystem, and USB hub control. Steve got to grips with the USB serial converters (including how to read and program their internal non-volatile settings). Finally I was able to explain hardware sequancing to everyone, and to modify boards to overcome some of my design mistakes (the biggest was by far the missing sence resistors for the EUT power managment)

The biggest change we ve seen in the Linux kernel for ARM over the past few years has been the transition to providing descriptions of the hardware in systems via device tree. This splits out the description of the devices in the system that can t be automatically enumerated from the kernel into a separate binary instead of being part of the kernel binary. Currently for most systems that are actively used upstream the device tree source code is kept in the kernel but the goal is to allow people to use device trees that are distributed separately to the kernel, especially device trees that are shipped as part of the board firmware. This is something that other platforms have done for a long time, PowerPC Macs and Sun SPARC systems use device tree as the mechanism for describing the hardware to the operating system. One consequence of this desire to allow the kernel and device tree to be shipped separately is that the device tree becomes an ABI. This is a really big change for people working in the embedded and consumer electronics areas where ARM has been most widely deployed, it means that any descriptions of the hardware need to be something that can stand the test of time. Anything we release is something we have to expect to carry code for indefinitely. When everything was done as part of the kernel binary we could easily do something that doesn t quite represent the hardware with the intention of replacing it later, now it is much harder to do that. An example of this is the SAW in Qualcomm SoCs. This is a block in the SoC which provides control of some regulators used for the CPU cores in the PMIC in very low power states and also allows the CPU to control the regulator with fast memory mapped registers rather than the slower buses used to control the PMIC. Unfortunately it doesn t fully replace direct access to the PMIC, it supports a subset of the control we need for the PMIC but not all of it. We could represent the SAW as an independent regulator but from a system integration point of view it is functioning as an extra control interface for the external PMIC and if we want to use the extra functionality that is only available via direct access to the PMIC we need to consider that and represent the SAW as an extension of it. If we don t need that extra PMIC functionality at the current time this means we need to do some extra work to make sure we describe the PMIC before we can use the SAW even if we have no intention to use anything other than the SAW. Now, few if any people are actually using the device tree as an ABI at present so those working on enabling platforms often forget about the requirement and find it an obstacle to getting things done they have pressure to get things done, they don t have quite the same pressure to make sure that attention is paid to device tree compatibility so it can easily get forgotten. Over time this may change, especially if people start to take advantage of the device tree as an ABI that become more and more important, but for now if we want to enable that in the future it s something we have to actively think about and work on, accepting that this means we won t always be able to do the most expedient thing.

Just over a year ago I implemented an optimization to the SPI core code in Linux that avoids some needless context switches to a worker thread in the main data path that most clients use. This was really nice, it was simple to do but saved a bunch of work for most drivers using SPI and made things noticeably faster. The code got merged in v4.0 and that was that, I kept on kicking a few more ideas for optimizations in this area around but that was that until the past month. What happened then was that for whatever reason people started picking up v4.0 and using it in production more. On some systems people started seeing problems when there was heavy SPI flash usage, often during things like distribution installation. In some cases the lockup detector fired, but the most entertaining error was that on Marvell Orion systems (which are single core) when the flash was being heavily used the SATA controller started having trouble handling interrupts. These problems all bisected down to the key commit in that series, 0461a4149836c79 (spi: Pump transfers inside calling context for spi_sync()). The problem is that there are a number of widely deployed SPI controllers out there that don t support DMA and instead require the CPU to explicitly read and write everything sent to and from registers in the controller. To make matters worse these accesses to the controller will usually take many CPU cycles to complete, each one stalling the CPU while they happen. This is fine for short transfers or if the CPU has nothing else to do but on a busy multitasking system it s an issue. Before the optimization the switches between the worker thread interacting with the hardware and the thread initiating the SPI operations provided breaks in this activity which allowed other things to switch in. Unfortunately when we optimize those away then if there s a lot of work for the controller being done from one thread then that thread can run for a long time without pause. The fix for affected drivers if there are no less CPU intensive ways of driving the hardware is to add some explicit sleeps into the driver itself, either at the end of the transfer_one() or perhaps in an unprepare_message() function. In a way I was quite pleased to see this, it was a clear demonstration that the optimization was having the intended effect though obviously users of affected systems will not find that so comforting. It s not the first time that making things faster or fixing a bug has revealed an underlying problem, I m sure it won t be the last.

One of the difficulties of being a kernel maintainer for a busy subsystem is that you will often end up getting a lot of mail that requires reading and handling which in turn requires sending a lot of mail out in reply. Some of that requires thought and careful consideration but a lot of it is quite routine and (perhaps surprisingly) there is often more challenge in doing a good job of handling these routine messages. For a long time I used to hand write every reply I sent but the problem with doing that is that sending the same message a lot of times tends to result in the messages getting more and more brief as the message becomes routine and practised. Your words become more optimised and if you ve stopped thinking about the message before you ve finished typing it then there s a desire to finish the typing and get on to the next thing. This is I think a lot of the reputation that kernel maintainers have for being terse and unhelpful comes from messages that are very practised for someone sending them all the time aren t always going to be obvious or helpful for someone who s not so intimately familiar with what s going on. The good part of it is that everyone is getting a personalised response and it s easy to insert a comment about that specific situation when you re already replying but it s not clear that the tradeoff is a good one. What I ve started doing instead for most things is keeping a set of pre-written paragraphs for common cases that I can just insert into a mail and edit as needed. Hopefully it s working well for people, it means the replies are that bit more verbose than they might otherwise be (mainly adding an explanation of why a given thing is being asked for) but can easily be adapted as needed. The one exception is the Applied, thanks mails I used to send when I apply a patch (literally just saying that). Those are now automatically generated by the script I use to sync my local git repository with kernel.org and very much more verbose:

From: Mark Brown <broonie@kernel.org>
To: $ CCS 
Cc: $ LIST 
Subject: $ SUBJECT 
In-Reply-To: $ MSGID 
The patch
   $ TITLE 
has been applied to the $ REPO  tree at
   $ URL  $ BRANCH 
All being well this means that it will be integrated into the linux-next
tree (usually sometime in the next 24 hours) and sent to Linus during
the next merge window (or sooner if it is a bug fix), however if
problems are discovered then the patch may be dropped or reverted.
You may get further e-mails resulting from automated or manual testing
and review of the tree, please engage with people reporting problems and
send followup patches addressing any issues that are reported if needed.

(unfortunately this bit seems to be something that it s worth pointing out)

If any updates are required or you are submitting further changes they
should be sent as incremental updates against current git, existing
patches will not be replaced.
Please add any relevant lists and maintainers to the CCs when replying
to this mail.
Thanks,
Mark

(the script does try to CC relevant lists). As well as giving people more information this also means that the mails only get sent out when things actually get published to my public repositories which avoids some confusion that used to happen sometimes with people getting my replies before I d pushed, especially when I d been working with poor connectivity as often happens when travelling. On the down side it s very much an obvious form letter which some people don t like and which can make people glaze over. My hope with this is to make things easier on average for patch submitters and easier for me, feedback on the scripted e-mails appears to be good thus far and the goal with the pasted in content is that it should be less obvious that it s happening so I d expect less feedback there.

Linux v4.4 will include a cool new feature contributed by Sascha Hauer of Pengutronix which propagates voltages set on a regulator to the regulators that supply it (taking into account the minimum headroom that the child regulator needs). The original reason for implementing it was to allow us to set voltages through simple unregulated power switches but the cool bit is that we can also use this to save power in some systems. There are two standard types of voltage regulator, DCDCs which are very efficient but produce noisy output and LDOs which are much less efficient but a lot cheaper and simpler and produce much cleaner output. What a lot of systems do to avoid a lot of the inefficiency of LDOs is to use a DCDC to reduce the voltage from the main system power supply (eg, the battery) to something close to the minimum power supply for the LDOs in the system This means that most of the voltage reduction (which is what generates inefficiency) comes from the DCDC rather than the LDO but you still get the clean power supply from the LDO and can have several different output voltages from a single expensive DCDC. By managing the voltage we set on the DCDC at runtime depending on the LDO configurations we can maximise the power savings from this setup, putting as much of the work onto the DCDC as we can at any given moment. This has been at the back of my mind for a long time, I m really pleased to see it implemented. It s a pretty small change code wise and probably not worth implementing for any one system but when we do it in the core like this hopefully many systems will be able to use it and the effects will add up.

Matthew Garrett s recent very good response to Eric Raymond s recent post opposing inclusiveness efforts in free software reminded me of something I ve been noticing more and more often: a very substantial proportion of the female developers I encounter working on the kernel are from non-European cultures where I (and I expect most people from western cultures) lack familiarity with the gender associations of all but the most common and familiar names. This could be happening for a lot of reasons it could be better entry paths to kernel development in those cultures (though my experience visiting companies in the relevant countries makes me question that), it could be that the sample sizes are so regrettably small that this really is just anecdote but I worry that some of what s going on is that the cultural differences are happening to mask and address some of the unconscious barriers that get thrown up.

Since I just had cause to do this and it was harder than it needed to be due to bitrot in the public documentation I could find I thought I d write up how to get a modern bootloader onto older Atmel boards. These instructions are written for the AT91SAM9G20-EK though they should also apply to other Atmel boards of a similar generation. These instructions are for booting from NAND since it s the default thing for the board, for this J34 should be fitted to enable the chip select and J33 disconnected to disable the dataflash. If there is something broken programmed into flash then booting while holding down BP4 should cause the second stage bootloader to trash itself and ensure the ROM bootloader puts itself into recovery mode, or just removing both J33 and J34 during power on will also ensure no second stage bootloader is found. There is a ROM bootloader but it just loads a small region from the boot media and jumps into it which isn t enough for u-boot so there is a second stage bootloader called AT91Bootstrap. Download sources for current versions from github. If it (or a more sensibly written equivalent) is not yet merged upstream you ll need to apply this patch to get it to build with a modern compiler, or you could use an old toolchain (which you ll need in the next step anyway):

diff --git a/board/at91sam9g20ek/board.mk b/board/at91sam9g20ek/board.mk
index 45f59b1822a6..b8251ca2fbad 100644
--- a/board/at91sam9g20ek/board.mk
+++ b/board/at91sam9g20ek/board.mk
@@ -1,7 +1,7 @@
 CPPFLAGS += \
        -DCONFIG_AT91SAM9G20EK \
-       -mcpu=arm926ej-s
+       -mcpu=arm926ej-s -mfloat-abi=soft
 
 ASFLAGS += \
        -DCONFIG_AT91SAM9G20EK \
-       -mcpu=arm926ej-s
+       -mcpu=arm926ej-s -mfloat-abi=soft

Once that s done you can build with:

make at91sam9g20eknf_uboot_defconfig
make CROSS_COMPILE=arm-linux-gnueabihf-

producing binaries/at91sam9g20ek-nandflashboot-uboot-$ VERSION .bin. This configuration will look for u-boot at 0x40000 in the flash so we need a u-boot binary. Unfortunately modern compilers seem to produce binaries that fail with no output. This is normally a sign that they need the ABI specifying more clearly as above but I got fed up trying to spot what was missing so I used an old CodeSourcery 2013.05 release instead, hopefully future versions of u-boot will be able to build for this target with older toolchains. Grab a recent release (I used 2015.01) and build with:

cd $ UBOOT 
make at91sam9g20ek_nandflash_defconfig
make CROSS_COMPILE=arm-linux-gnueabihf-

to get u-boot.bin. These can then be flashed using the Atmel flashing tool SAM-BA. Start it and connect to the target (there is a Linux version, though it appears to rely on old versions of TCL/TK so if you get trouble starting it the easiest thing is to use the sacrificial Windows laptop you ve obtained in order to run the entertaining flashing tools companies sometimes provide without risking a real system, or in my case your shiny new laptop that you ve not yet installed Linux on). Start it then:

1. Connect SAM-BA to the device following the dialog on start.
2. Make sure you ve selected NandFlash in the memory type tabs in the center of the window.
3. Run the Enable NandFlash script.
4. Run the Erase All script.
5. Run the Send Boot File script and provide the at91bootstrap binary.
6. Set Send File Name to be the u-boot binary you built earlier and Address to be 0x40000.
7. Click Send File
8. Press the reset button

which should result in at91bootstrap output followed by u-boot output on the serial console. A similar process works for the AT91SAM9263, there the jumper you need is J19 (sadly u-boot does not flash pictures of cute animals or forested shorelines on the screen as the default Basic LCD Project 1.4 firmware does, I m not sure this full operating system thing is really delivering improved functionality).

Recently I was in Seoul in the middle of three weeks of travel and my laptop died on me. Since I had some work that needed doing fairly urgently I took myself over to Yongsan Electronics Market and got myself a cheap replacement to tide myself over. What I ended up with was an Acer Aspire E11. There s a bunch of different models all with very similar plastics, I got one which has a N2940 SoC, 2G of RAM (upgraded to 4G in store), a 500G hard disk and no fans for just over 200000 Korean Won, or about $200. As you d expect at that price it s got shortcomings but overall I ve been extremely happy with it, it s worth looking at if you need something cheap. The keyboard in particular is probably the nicest I ve used 0n a laptop in a long time with a good, definite but not excessive click feel as you press. Battery life is about 5 hours as advertised which is not wonderful but basically fine for me most of the time, and while not exactly Retina it s clear with good viewing angles and generally pleasant to look at. Everything is plastic but feels very solid and robust, better than a lot of more expensive devices I ve used, and there s not much bezel around the screen which means it s the first laptop I ve had which has been comfortable to use in a standard economy seat on a plane. The biggest drawback is performance it s a little slow opening applications sometimes and kernel builds crawl with an x86 allmodconfig taking about one and three quarter hours. For e-mail and web browsing there s no problem at all, I did have to move from offlineimap to mbsync to get my mail to sync in a reasonable time but that s more to do with the performance of offlineimap than that of the system. Overall in use it feels like the Dell I was using from about 2008-2011 or so, comfortable in use outside of builds, and I do appreciate having a system with no fans. There were a couple of small tricks getting Debian installed this is the first system I ve seen with secure boot enabled by default which took me a few moments to work out (but is really good to see). Once that was disabled the install was smooth other than being bitten by Debian bug#778810 which meant I needed a manual fixup to actually get it to boot from the disk. It s also got a Broadcom WiFi module which means it doesn t work at all with mainline but it looked like that was on a standard mini PCI Express module so easily replaceable (I happened to have a USB dongle handy so haven t bothered) and the wired ethernet just worked. Like I say I ve been very happy with it, there s a bunch of other models with different specs for everything except the case (some touchscreen, some with small 32G eMMC drives) as well. Were it not for my need to do kernel builds I d probably be keeping it as my primary laptop.

A few months ago I decided it would be good to re-rip my CD collection, retaining a lossless digital copy, and set about planning the project. I then realised I hadn't the time to take the project on for the time being and parked it, but not before figuring a few bits and pieces out. Starting at the beginning, with ripping the CDs. The most widely used CD ripping software on Linux systems is still cdparanoia, which is pretty good, but it's still possible to get bad CD rips, and I've had several in a very small sample size. On Windows systems, the recommended ripper is Exact Audio Copy, or EAC for short. EAC calculates checksums of ripped CDs or tracks and compares them against an online database of rips called AccurateRip. It also calibrates your CD drive against the same database and uses the calculated offset during the rip. I wasn't aware of any AccurateRip-supporting rippers until recently when Mark Brown introduced me to morituri. I've done some tentative experiments and it appears to be produce identical rips to EAC for some sample CDs (with different CD reading hardware too). Fundamentally, AccurateRip is a proprietary database, and so I think the longer term goal in the F/OSS community should be to create an alternative, open database of rip checksums and drive offsets. The audio community has already been burned by the CDDB database going proprietary, but at least we now have the far superior MusicBrainz.