I sent this email to debian-private a few days ago, on the 10th anniversary of my Debian account creation:

Date: Fri, 14 Aug 2015 19:37:20 +0200
From: David Moreno 
To: debian-private@lists.debian.org
Subject: Retiring from Debian
User-Agent: Mutt/1.5.23 (2014-03-12)
[-- PGP output follows (current time: Sun 23 Aug 2015 06:18:36 PM CEST) --]
gpg: Signature made Fri 14 Aug 2015 07:37:20 PM CEST using RSA key ID 4DADEC2F
gpg: Good signature from "David Moreno "
gpg:                 aka "David Moreno "
gpg:                 aka "David Moreno (1984-08-08) "
[-- End of PGP output --]
[-- The following data is signed --]
Hi,
Ten years ago today (2005-08-14) my account was created:
https://nm.debian.org/public/person/damog
Today, I don't feel like Debian represents me and neither do I represent the
project anymore.
I had tried over the last couple of years to retake my involvement but lack of
motivation and time always got on the way, so the right thing to do for me is
to officially retire and gtfo.
I certainly learned a bunch from dozens of Debian people over these many years,
and I'm nothing but grateful with all of them; I will for sure carry the project
close to my heart   as I carry it with the Debian swirl I still have tattooed
on my back ;)
http://damog.net/blog/2005/06/29/debian-tattoo/
I have three packages left that have not been updated in forever and you can
consider orphaned now: gcolor2, libperl6-say-perl and libxml-treepp-perl.
With all best wishes,
David Moreno.
http://damog.net/
[-- End of signed data --]

I received a couple of questions about my decision here. I basically don t feel like Debian represents my interests and neither do I represent the project this doesn t mean I don t believe in free software, to the contrary. I think some of the best software advancements we ve made as society are thanks to it. I don t necessarily believe on how the project has evolved itself, whether that has been the right way, to regain relevancy and dominance, and if it s remained primarily a way to feed dogmatism versus pragmatism. This is the perfect example of a tragic consequence. I was very happy to learn that the current Debian Conference being held in Germany got the highest attendance ever, hopefully that can be utilized in a significant and useful way. Regardless, my contributions to Debian were never noteworthy so it s also not that big of a deal. I just need to close cycles myself and move forward, and the ten year anniversary looked like a significant mark for that. Poke me in case you wanna discuss some more. I ll always be happy to. Specially over beer :) Peace.

There has been a great deal of comment among my friends recently about a particularly British cookery program called "The Great British Bake Off". There has been some controversy as the program is moving from the BBC to a commercial broadcaster.

Part of this discussion comes from all the presenters, excepting Paul Hollywood, declining to sign with the new broadcaster and partly because of speculation the BBC might continue with a similar format show with a new name.

Rob Kendrick provided the start to this conversation by passing on a satirical link suggesting Samuel L Jackson might host "cakes on a plane"

This caused a large number of suggestions for alternate names which I will be reporting but Rob Kendrick, Vivek Das Mohapatra, Colin Watson, Jonathan McDowell, Oki Kuma, Dan Alderman, Dagfinn Ilmari Manns ke, Lesley Mitchell and Daniel Silverstone are the ones to blame.

• Strictly come baking
• Stars and their pies
• Baking with the stars
• Bake/Off.
• Blind Cake
• Cake or no cake?
• The cake is a lie
• Bake That.
• Bake Me On
• Bake On Me
• Bakin' Stevens.
• The Winner Bakes It All
• Bakerloo
• Bake Five
• Every breath you bake
• Every bread you bake
• Unbake my heart
• Knead and let prove
• Bake me up before you go-go
• I want to bake free
• Another bake bites the dust
• Cinnamon whorl is not enough
• The pie who loved me
• The yeast you can do.
• Total collapse of the tart
• Bake and deliver
• You Gotta Bake
• Bake's Seven
• Natural Born Bakers
• Bake It Or Leaven It
• Driving the last pikelet
• Pie crust on the dancefloor
• Tomorrow never pies
• Murder on the pie crust
• The pie who came in from the cold.
• You only bake twice (Every body has to make one sweet and one savoury dish).

So that is our list, anyone else got better ideas?

Categories:

• Debian-Blog
• Computing
• Tools

Keywords:

• GNOME Shell Extensions
• Chromium
• Chrome
• Browser

Like:

Previously: v4.5. The v4.6 Linux kernel release included a bunch of stuff, with much more of it under the KSPP umbrella. seccomp support for parisc Helge Deller added seccomp support for parisc, which including plumbing support for PTRACE_GETREGSET to get the self-tests working. x86 32-bit mmap ASLR vs unlimited stack fixed Hector Marco-Gisbert removed a long-standing limitation to mmap ASLR on 32-bit x86, where setting an unlimited stack (e.g. ulimit -s unlimited ) would turn off mmap ASLR (which provided a way to bypass ASLR when executing setuid processes). Given that ASLR entropy can now be controlled directly (see the v4.5 post), and that the cases where this created an actual problem are very rare, means that if a system sees collisions between unlimited stack and mmap ASLR, they can just adjust the 32-bit ASLR entropy instead. x86 execute-only memory Dave Hansen added Protection Key support for future x86 CPUs and, as part of this, implemented support for execute only memory in user-space. On pkeys-supporting CPUs, using mmap(..., PROT_EXEC) (i.e. without PROT_READ) will mean that the memory can be executed but cannot be read (or written). This provides some mitigation against automated ROP gadget finding where an executable is read out of memory to find places that can be used to build a malicious execution path. Using this will require changing some linker behavior (to avoid putting data in executable areas), but seems to otherwise Just Work. I m looking forward to either emulated QEmu support or access to one of these fancy CPUs. CONFIG_DEBUG_RODATA enabled by default on arm and arm64, and mandatory on x86 Ard Biesheuvel (arm64) and I (arm) made the poorly-named CONFIG_DEBUG_RODATA enabled by default. This feature controls whether the kernel enforces proper memory protections on its own memory regions (code memory is executable and read-only, read-only data is actually read-only and non-executable, and writable data is non-executable). This protection is a fundamental security primitive for kernel self-protection, so making it on-by-default is required to start any kind of attack surface reduction within the kernel. On x86 CONFIG_DEBUG_RODATA was already enabled by default, but, at Ingo Molnar s suggestion, I made it mandatory: CONFIG_DEBUG_RODATA cannot be turned off on x86. I expect we ll get there with arm and arm64 too, but the protection is still somewhat new on these architectures, so it s reasonable to continue to leave an out for developers that find themselves tripping over it. arm64 KASLR text base offset Ard Biesheuvel reworked a ton of arm64 infrastructure to support kernel relocation and, building on that, Kernel Address Space Layout Randomization of the kernel text base offset (and module base offset). As with x86 text base KASLR, this is a probabilistic defense that raises the bar for kernel attacks where finding the KASLR offset must be added to the chain of exploits used for a successful attack. One big difference from x86 is that the entropy for the KASLR must come either from Device Tree (in the /chosen/kaslr-seed property) or from UEFI (via EFI_RNG_PROTOCOL), so if you re building arm64 devices, make sure you have a strong source of early-boot entropy that you can expose through your boot-firmware or boot-loader. zero-poison after free Laura Abbott reworked a bunch of the kernel memory management debugging code to add zeroing of freed memory, similar to PaX/Grsecurity s PAX_MEMORY_SANITIZE feature. This feature means that memory is cleared at free, wiping any sensitive data so it doesn t have an opportunity to leak in various ways (e.g. accidentally uninitialized structures or padding), and that certain types of use-after-free flaws cannot be exploited since the memory has been wiped. To take things even a step further, the poisoning can be verified at allocation time to make sure that nothing wrote to it between free and allocation (called sanity checking ), which can catch another small subset of flaws. To understand the pieces of this, it s worth describing that the kernel s higher level allocator, the page allocator (e.g. __get_free_pages()) is used by the finer-grained slab allocator (e.g. kmem_cache_alloc(), kmalloc()). Poisoning is handled separately in both allocators. The zero-poisoning happens at the page allocator level. Since the slab allocators tend to do their own allocation/freeing, their poisoning happens separately (since on slab free nothing has been freed up to the page allocator). Only limited performance tuning has been done, so the penalty is rather high at the moment, at about 9% when doing a kernel build workload. Future work will include some exclusion of frequently-freed caches (similar to PAX_MEMORY_SANITIZE), and making the options entirely CONFIG controlled (right now both CONFIGs are needed to build in the code, and a kernel command line is needed to activate it). Performing the sanity checking (mentioned above) adds another roughly 3% penalty. In the general case (and once the performance of the poisoning is improved), the security value of the sanity checking isn t worth the performance trade-off. Tests for the features can be found in lkdtm as READ_AFTER_FREE and READ_BUDDY_AFTER_FREE. If you re feeling especially paranoid and have enabled sanity-checking, WRITE_AFTER_FREE and WRITE_BUDDY_AFTER_FREE can test these as well. To perform zero-poisoning of page allocations and (currently non-zero) poisoning of slab allocations, build with:

CONFIG_DEBUG_PAGEALLOC=n
CONFIG_PAGE_POISONING=y
CONFIG_PAGE_POISONING_NO_SANITY=y
CONFIG_PAGE_POISONING_ZERO=y
CONFIG_SLUB_DEBUG=y

and enable the page allocator poisoning and slab allocator poisoning at boot with this on the kernel command line:

page_poison=on slub_debug=P

To add sanity-checking, change PAGE_POISONING_NO_SANITY=n, and add F to slub_debug as slub_debug=PF . read-only after init I added the infrastructure to support making certain kernel memory read-only after kernel initialization (inspired by a small part of PaX/Grsecurity s KERNEXEC functionality). The goal is to continue to reduce the attack surface within the kernel by making even more of the memory, especially function pointer tables, read-only (which depends on CONFIG_DEBUG_RODATA above). Function pointer tables (and similar structures) are frequently targeted by attackers when redirecting execution. While many are already declared const in the kernel source code, making them read-only (and therefore unavailable to attackers) for their entire lifetime, there is a class of variables that get initialized during kernel (and module) start-up (i.e. written to during functions that are marked __init ) and then never (intentionally) written to again. Some examples are things like the VDSO, vector tables, arch-specific callbacks, etc. As it turns out, most architectures with kernel memory protection already delay making their data read-only until after __init (see mark_rodata_ro()), so it s trivial to declare a new data section ( .data..ro_after_init ) and add it to the existing read-only data section ( .rodata ). Kernel structures can be annotated with the new section (via the __ro_after_init macro), and they ll become read-only once boot has finished. The next step for attack surface reduction infrastructure will be to create a kernel memory region that is passively read-only, but can be made temporarily writable (by a single un-preemptable CPU), for storing sensitive structures that are written to only very rarely. Once this is done, much more of the kernel s attack surface can be made read-only for the majority of its lifetime. As people identify places where __ro_after_init can be used, we can grow the protection. A good place to start is to look through the PaX/Grsecurity patch to find uses of __read_only on variables that are only written to during __init functions. The rest are places that will need the temporarily-writable infrastructure (PaX/Grsecurity uses pax_open_kernel()/pax_close_kernel() for these). That s it for v4.6, next up will be v4.7!

2016, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.

Review: Sailing to Sarantium, by Guy Gavriel Kay

Series:	Sarantine Mosaic #1
Publisher:	HarperPrism
Copyright:	1998
Printing:	January 2000
ISBN:	0-06-105990-0
Format:	Mass market
Pages:	533

Crispin is a well-respected mosaicist, a Rhodian, heir to a long tradition of artistic accomplishment. Rhodias is largely in ruins, and the kings that fight over its land are far removed from the emperors of Rhodias at its height, but there's still demand for skilled craftsmanship. Crispin has a partner, assistants, and the opportunity to do at least adequate work. It's not a bad life, or at least wasn't before plague took his wife and daughters while he watched, helpless. Now, he's closed off, angry, and consumed by fatalism and futility. His partner, Martinian, could never say why, when the imperious and obnoxious courier came from the far-off and half-legendary of Sarantium to summon him to assist with the rebuilding of the Sanctuary of Jad, he pretended not to be himself and pointed the courier at Crispin instead. Nor was Crispin entirely sure why he played along, although the courier was so arrogant and officious that he invited such treatment. He had no intention of going in Martinian's place and name until his friends intervened, united in their conviction that he needed something to focus on and a reason to live. But nothing about Sarantium is simple. Even before Crispin leaves, the local queen, precariously balancing between warring factions, reveals her own reason to send Crispin to the capital of the far-off empire: using him as a desperate messenger to propose an alliance through marriage. And when he finally arrives, after unexpected perils on the road, he tumbles immediately into the deep complexities of the Sarantine court and its remarkable rulers. Most of Guy Gavriel Kay's work is historical fiction with the serial numbers partly filed off and some magic added. The degree to which the serial numbers are removed varies by work; here, the history is obvious. Sarantium is Byzantium, fallen Rhodias is Rome, and the emperor Valerius and his former dancer empress Alixana are clearly Justinian and Theodora. Knowing that will provide a lot of context, but can also be distracting, since the temptation to scramble for Wikipedia and line things up with real history is strong. I read this book the first time without knowing the history and this second time knowing it. I think I enjoyed it a bit more on its own terms, instead of as a reflection and reinterpretation. Either way, though, it's one of Kay's best novels, which is saying quite a bit. Sailing to Sarantium has two very different parts: Crispin's journey to Sarantium, and then what happens when he arrives. The first is a fairly linear travel narrative mixed with old religion, magic, and an experience in the woods that's a little disconnected from the rest of the story. The second is the fast-entangling politics and factionalism of the city itself, which wastes no time pulling Crispin into meetings with the most influential people at court. In my memory, I liked the Sarantium narrative the best, and found the travel story less significant. Re-reading, I'm not sure I agree with my earlier self. Kay does write some of the best conversations in fiction; if you want to see careful verbal maneuvering or gut reactions with wide-ranging consequences, I can't think of any author who shows both the words and the nuances as well as Kay. He makes the court maneuvering feel truly epic. Watching Crispin's blunt competence cut through the Sarantine court, and seeing him stay focused on his life work despite all those distractions, is a truly rewarding experience. But that second part of the book also has some structural problems, and a few characterization problems. The structural problem is that Kay wants to tell the same story from multiple angles, viewpoints, and timelines to show the reader every implication of the hornet's nest that Crispin kicks over, since quite a lot happens in the course of one day and night. This mostly works, but it is so laden with flashbacks, foreshadowing, and rewinds that I started losing track of the time relationship between the scenes. Kay is a very skilled author, so he avoids confusing the reader entirely, but I think he still tried too complex of a temporal structure here. The characterization problem is that I previously wasn't paying as close of attention to Kay's portrayal of women and of male sexual reactions to women. It didn't bother me before; this time, it started getting on my nerves. Yes, this is historical fiction (although limitations on roles for women in history is rather overstated in fiction), and to give Kay credit it does feature a very strong female character in Alixana. But the male protector and female seducer dynamic is very strong, and all the women in this book seem to get shoehorned into it. And perhaps I'm missing some weakness that other men have, but I have never had the sort of overwhelming, thought-destroying reaction to the presence of a seductive woman as Kay's men seem to routinely have in this book. To give Crispin credit, he maneuvers his way through those conversations without doing anything stupid. (Kay's characters very rarely do stupid things, which is refreshing.) But the effort required started undermining my suspension of disbelief. Even making allowances for a culture that might consider a woman with her hair down the equivalent of an entirely nude woman, I simply don't believe that a man who had been married with two daughters, had a long career, and had the life experience Crispin had would have that much difficulty with an aggressively seductive woman that he already doesn't trust. As a result, as much as I love Kay's political conversational fencing, I enjoyed the travel portion of the book more on this re-read. Crispin is a type of character Kay writes extremely well: an honest, honorable man trying to navigate unfamiliar waters without a lot of context, but with a firm sense of internal morality, a lot of natural intelligence, and a true passion for something. At one point in this book, another character decides to walk away from his current job and follow Crispin wherever he goes. Kay makes you believe that and want to do it yourself. It's a very rewarding experience as a reader to watch a character you can trust find careful, courageous, and tricky solutions to complicated problems. But I think the best part of this book is Kay's ability to put Crispin's art, the creation of mosaics, on the page in a way that lets the reader viscerally appreciate it. Crispin is passionate about mosaics, about how to make them and how to think about them, and the passion is contagious and forms the heart and soul of this novel. Kay manages to make mosaics overshadow some of the most dramatic politics in history, which says quite a lot. There is almost nothing I love more than reading about talented, passionate people doing things they are extremely good at. I know nothing about mosaic that I haven't read in this book and its sequel, so I have no basis from which to judge the accuracy of Kay's portrayal, but he made me feel like I could appreciate some of the nuance, skill, and design constraints of the art form. It's a wonderful reading experience. Sailing to Sarantium is best thought of half of a long book that was divided for publication reasons, so don't expect much of an ending here. It can only be read in combination with Lord of Emperors, which you'll want to have on hand when you finish this book so that you can continue the story. Followed by Lord of Emperors. Rating: 9 out of 10

Here is my monthly update covering what I have been doing in the free software world (previous month):

• Fixed a number of instances in the Django web development framework where methods had mutable defaults arguments such as lists or dictionaries. (#7253)
• Made a large number of improvements to travis.debian.net, my hosted service for projects that host their Debian packaging on GitHub to use the Travis CI continuous integration platform to test builds on every update:
  • Worked with Evgeni Golov to run autopkgtest and autodep8 tests after builds. (#23, #24 & #25)
  • Ensured that all remote branches are created locally so that packages that specify options such as pristine_tar=True in debian/gbp.conf work as expected. (commit)
  • Fixed an issue where checking out other branches did not reliably not work due to Travis cloning the repository with --depth=50. (#21)
  • Updated the documentation after prompting from Valerie Young. (db7238 & 470d24)
• Improved my Chrome extension for the FastMail web interface to collapse whitespace when appending to the subject. (#9)
• Submitted a pull request for the Handbrake video transcoder to make the build reproducible. (#320)
• Updated the configuration for my OpenWRT-based router (a device I use as a range extender and to save tedious reconfiguration of all my wifi devices in new locales) to enter a "fallback" mode if it cannot connect to the client network. This works around a longstanding "wontfix" upstream issue. (diff)
• Updated django-agpl, my reusable Django application to assist in releasing projects under the GNU Affero General Public License:
  • Merged support from defivelo to support Django 1.8. (#1)
  • Improved the code quality & testability throughout the package. (diff)
• Filed in issue in Mailvelope (a browser extension for OpenPGP encryption) to support composing/viewing messages in a fixed-width font. (#422)
• Blogged about how to write your first Lintian check as well as on the progress bar I added to Diffoscope.

---

Previously: v4.4. Some things I found interesting in the Linux kernel v4.5: CONFIG_IO_STRICT_DEVMEM The CONFIG_STRICT_DEVMEM setting that has existed for a long time already protects system RAM from being accessible through the /dev/mem device node to root in user-space. Dan Williams added CONFIG_IO_STRICT_DEVMEM to extend this so that if a kernel driver has reserved a device memory region for use, it will become unavailable to /dev/mem also. The reservation in the kernel was to keep other kernel things from using the memory, so this is just common sense to make sure user-space can t stomp on it either. Everyone should have this enabled. (And if you have a system where you discover you need IO memory access from userspace, you can boot with iomem=relaxed to disable this at runtime.) If you re looking to create a very bright line between user-space having access to device memory, it s worth noting that if a device driver is a module, a malicious root user can just unload the module (freeing the kernel memory reservation), fiddle with the device memory, and then reload the driver module. So either just leave out /dev/mem entirely (not currently possible with upstream), build a monolithic kernel (no modules), or otherwise block (un)loading of modules (/proc/sys/kernel/modules_disabled). ptrace fsuid checking Jann Horn fixed some corner-cases in how ptrace access checks were handled on special files in /proc. For example, prior to this fix, if a setuid process temporarily dropped privileges to perform actions as a regular user, the ptrace checks would not notice the reduced privilege, possibly allowing a regular user to trick a privileged process into disclosing things out of /proc (ASLR offsets, restricted directories, etc) that they normally would be restricted from seeing. ASLR entropy sysctl Daniel Cashman standardized the way architectures declare their maximum user-space ASLR entropy (CONFIG_ARCH_MMAP_RND_BITS_MAX) and then created a sysctl (/proc/sys/vm/mmap_rnd_bits) so that system owners could crank up entropy. For example, the default entropy on 32-bit ARM was 8 bits, but the maximum could be as much as 16. If your 64-bit kernel is built with CONFIG_COMPAT, there s a compat version of the sysctl as well, for controlling the ASLR entropy of 32-bit processes: /proc/sys/vm/mmap_rnd_compat_bits. Here s how to crank your entropy to the max, without regard to what architecture you re on:

for i in "" "compat_"; do f=/proc/sys/vm/mmap_rnd_$ i bits; n=$(cat $f); while echo $n > $f ; do n=$(( n + 1 )); done; done

strict sysctl writes Two years ago I added a sysctl for treating sysctl writes more like regular files (i.e. what s written first is what appears at the start), rather than like a ring-buffer (what s written last is what appears first). At the time it wasn t clear what might break if this was enabled, so a WARN was added to the kernel. Since only one such string showed up in searches over the last two years, the strict writing mode was made the default. The setting remains available as /proc/sys/kernel/sysctl_writes_strict. seccomp UM support Micka l Sala n added seccomp support (and selftests) for user-mode Linux. Moar architectures! seccomp NNP vs TSYNC fix Jann Horn noticed and fixed a problem where if a seccomp filter was already in place on a process (after being installed by a privileged process like systemd, a container launcher, etc) then the setting of the no new privs flag could be bypassed when adding filters with the SECCOMP_FILTER_FLAG_TSYNC flag set. Bypassing NNP meant it might be possible to trick a buggy setuid program into doing things as root after a seccomp filter forced a privilege drop to fail (generally referred to as the sendmail setuid flaw ). With NNP set, a setuid program can t be run in the first place. That s it! Next I ll cover v4.6 Edit: Added notes about iomem=

2016, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.

Diffoscope is a diff utility which recursively unpacks archives, ISOs, etc., transforming a wide variety of files into human-readable forms before comparison instead of simply showing the raw difference in hexadecimal. I recently added a progress bar when diffoscope is run on a terminal: Note that as diffoscope can, at any point, encounter an archive or format that requires unpacking, the progress will always be approximate and may even appear to go "backwards". The implementation, available in version 61, is simple (see #1, #2, #3 & #4) but takes into account of a number of subtleties by using context managers to correctly track the state throughout.

I was assigned 14.75 hours of work by Freexian's Debian LTS initiative and carried over 0.7 from last month. I worked a total of 14 hours, carrying over 1.45 hours. I finished preparing and finally uploaded an update for linux (3.2.81-2). This took longer than expected due to the difficulty of reproducing CVE-2016-5696 and verifying the backported fix. I also released an upstream stable update (3.2.82) which will go into the next update in wheezy LTS. I discussed a few other security updates and issues on the debian-lts mailing list.

When I gave my State of the Kernel Self-Protection Project presentation at the 2016 Linux Security Summit, I included some slides covering some quick bullet points on things I found of interest in recent Linux kernel releases. Since there wasn t a lot of time to talk about them all, I figured I d make some short blog posts here about the stuff I was paying attention to, along with links to more information. This certainly isn t everything security-related or generally of interest, but they re the things I thought needed to be pointed out. If there s something security-related you think I should cover from v4.3, please mention it in the comments. I m sure I haven t caught everything. :) A note on timing and context: the momentum for starting the Kernel Self Protection Project got rolling well before it was officially announced on November 5th last year. To that end, I included stuff from v4.3 (which was developed in the months leading up to November) under the umbrella of the project, since the goals of KSPP aren t unique to the project nor must the goals be met by people that are explicitly participating in it. Additionally, not everything I think worth mentioning here technically falls under the kernel self-protection ideal anyway some things are just really interesting userspace-facing features. So, to that end, here are things I found interesting in v4.3: CONFIG_CPU_SW_DOMAIN_PAN Russell King implemented this feature for ARM which provides emulated segregation of user-space memory when running in kernel mode, by using the ARM Domain access control feature. This is similar to a combination of Privileged eXecute Never (PXN, in later ARMv7 CPUs) and Privileged Access Never (PAN, coming in future ARMv8.1 CPUs): the kernel cannot execute user-space memory, and cannot read/write user-space memory unless it was explicitly prepared to do so. This stops a huge set of common kernel exploitation methods, where either a malicious executable payload has been built in user-space memory and the kernel was redirected to run it, or where malicious data structures have been built in user-space memory and the kernel was tricked into dereferencing the memory, ultimately leading to a redirection of execution flow. This raises the bar for attackers since they can no longer trivially build code or structures in user-space where they control the memory layout, locations, etc. Instead, an attacker must find areas in kernel memory that are writable (and in the case of code, executable), where they can discover the location as well. For an attacker, there are vastly fewer places where this is possible in kernel memory as opposed to user-space memory. And as we continue to reduce the attack surface of the kernel, these opportunities will continue to shrink. While hardware support for this kind of segregation exists in s390 (natively separate memory spaces), ARM (PXN and PAN as mentioned above), and very recent x86 (SMEP since Ivy-Bridge, SMAP since Skylake), ARM is the first upstream architecture to provide this emulation for existing hardware. Everyone running ARMv7 CPUs with this kernel feature enabled suddenly gains the protection. Similar emulation protections (PAX_MEMORY_UDEREF) have been available in PaX/Grsecurity for a while, and I m delighted to see a form of this land in upstream finally. To test this kernel protection, the ACCESS_USERSPACE and EXEC_USERSPACE triggers for lkdtm have existed since Linux v3.13, when they were introduced in anticipation of the x86 SMEP and SMAP features. Ambient Capabilities Andy Lutomirski (with Christoph Lameter and Serge Hallyn) implemented a way for processes to pass capabilities across exec() in a sensible manner. Until Ambient Capabilities, any capabilities available to a process would only be passed to a child process if the new executable was correctly marked with filesystem capability bits. This turns out to be a real headache for anyone trying to build an even marginally complex least privilege execution environment. The case that Chrome OS ran into was having a network service daemon responsible for calling out to helper tools that would perform various networking operations. Keeping the daemon not running as root and retaining the needed capabilities in children required conflicting or crazy filesystem capabilities organized across all the binaries in the expected tree of privileged processes. (For example you may need to set filesystem capabilities on bash!) By being able to explicitly pass capabilities at runtime (instead of based on filesystem markings), this becomes much easier. For more details, the commit message is well-written, almost twice as long as than the code changes, and contains a test case. If that isn t enough, there is a self-test available in tools/testing/selftests/capabilities/ too. PowerPC and Tile support for seccomp filter Michael Ellerman added support for seccomp to PowerPC, and Chris Metcalf added support to Tile. As the seccomp maintainer, I get excited when an architecture adds support, so here we are with two. Also included were updates to the seccomp self-tests (in tools/testing/selftests/seccomp), to help make sure everything continues working correctly. That s it for v4.3. If I missed stuff you found interesting, please let me know! I m going to try to get more per-version posts out in time to catch up to v4.8, which appears to be tentatively scheduled for release this coming weekend.

2016, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.

Here is what happened in the Reproducible Builds effort between Sunday September 18 and Saturday September 24 2016: Outreachy We intend to participate in Outreachy Round 13 and look forward for new enthusiastic applications to contribute to reproducible builds. We're offering four different areas to work on:

• Improve test and debugging tools.
• Improving reproducibility of Debian packages.
• Improving Debian infrastructure.
• Help collaboration across distributions.

Reproducible Builds World summit #2 We are planning e a similar event to our Athens 2015 summit and expect to reveal more information soon. If you haven't been contacted yet but would like to attend, please contact holger. Toolchain development and fixes Mattia uploaded dpkg/1.18.10.0~reproducible1 to our experimental repository. and covered the details for the upload in a mailing list post. The most important change is the incorporation of improvements made by Guillem Jover (dpkg maintainer) to the .buildinfo generator. This is also in the hope that it will speed up the merge in the upstream. One of the other relevant changes from before is that .buildinfo files generated from binary-only builds will no longer include the hash of the .dsc file in Checksums-Sha256 as documented in the specification. Even if it was considered important to include a checksum of the source package in .buildinfo, storing it that way breaks other assumptions (eg. that Checksums-Sha256 contains only files part of that are part of a single upload, wheras the .dsc might not be part of that upload), thus we look forward for another solution to store the source checksum in .buildinfo. Bugs filed

• #838713 filed against python-xlib by Chris Lamb.
• #838754 filed against golang-google-grpc by Chris Lamb.
• #838188 filed against ocaml by Johannes Schauer.
• #838785 filed against funnelweb by Reiner Herrmann.

Reviews of unreproducible packages 250 package reviews have been added, 4 have been updated and 4 have been removed in this week, adding to our knowledge about identified issues. 4 issue types have been added:

• captures_users_gecos issue
• timestamps_in_org_mode_html_output toolchain issue.
• varnish_vmodtool_random_file_id
• gpg_keyring_magic_bytes_differ

3 issue types have been updated:

• timestamps_in_org_mode_html_output more generally.
• timestamps_in_documentation_generated_by_texi2html
• randomness_in_ocaml_preprocessed_files

Weekly QA work FTBFS bugs have been reported by:

• Chris Lamb (11)
• Santiago Vila (2)

Documentation updates h01ger created a new Jenkins job so that every commit pushed to the master branch for the website will update reproducible-builds.org. diffoscope development

• Mattia Rizzolo:
  • Skip rlib tests if the "nm" tool is missing
• Ximin Luo:
  • Give better advice about what envvar to set to make the console work
  • tests/basic-command-line: check exit code and use a more complex example
  • Add a script to check sizes of dependencies
  • Don't use unicode quotes to avoid breakage under LC_ALL=C

strip-nondeterminism development

• Chris Lamb:
  • .perltidyrc: Add from lintian.
  • .perltidyrc: We use tabs, not spaces.
  • Run perltidy

reprotest development

• Ximin Luo uploaded reprotest 0.3 and 0.3.1 to unstable with these changes:
  • Make some variations more reliable, so tests don't fail
  • Add a safety device to guard against typos
  • Address lintian warnings
  • Remove any existing artifact, in case the build script doesn't overwrite it
  • Fix the logic of some tests, and don't vary fileordering on Debian buildds
  • Use the magic of VIRTUALENV_DOWNLOAD=no, seen in tox's own autopkgtest tests
  • Flush so subprocess output is guaranteed to appear later
  • Don't error if the build command generates stderr
  • Default tests to run on "null" only since it takes effort to set up the others
  • hey dawg i herd u liek tests so i put some tests in ur tests so u can test while u test
  • Make no_clear_on_error optional; we don't want to pass it in everywhere e.g. tests
  • Output a nice big obvious summary at the end when successful
  • Don't repeat documentation in two different places, move it all to --help
  • More reliable build artifact pattern matching, and update docs
  • Use a list comprehension for slightly more idiomatic python than map/lambda
  • Make multi-component artifact patterns like '.deb .changes' work correctly
  • Add a --no-clean-on-error option so you can analyse what went wrong
  • More help text for virtual_server_args
  • Support shell patterns as the build_artifact
  • Use sys.exit inside main(), not check() as it's more idiomatic
  • Pass kind='build' to check_exec so it doesn't time out after 100s
  • adt_testbed: add stdout/stderr to the "auxverb failed" error message, if available
  • If no virtual_server is given then use "null"

tests.reproducible-builds.org

• The full rebuild of all packages in unstable (for all tested archs) with the new build path variation has been completed. This has had the result that we are down to ~75% reproducible packages in unstable now. In comparison, for testing (where we don't vary the build path) we are still at ~90%. IRC notifications for unstable have been enabled again. (Holger)
• Make the notes job robust about bad data (see #833695 and #833738). (Holger)
• Setup profitbricks-build7 running stretch as testing reproducible builds of F-Droid need to use a newer version of vagrant in order to support running vagrant VMs with kvm on kvm. (Holger)
• The misbehaving 'opi2a' armhf node has been replaced with a Jetson-TK1 board kindly donated by NVidia. This machine is using an NVIDIA tegra-k1 (cortex-a15) quad-core board. (vagrant and Holger)

Misc. This week's edition was written by Chris Lamb, Holger Levsen and Mattia Rizzolo and reviewed by a bunch of Reproducible Builds folks on IRC.

Sometimes it really helps to have a different view on a problem and after my recent writings on my Public Suffix List (PSL) library I was fortunate to receive a suggestion from my friend Enrico Zini.

I had asked for suggestions on reducing the size of the library further and Enrico simply suggested Huffman coding. This was a technique I had learned about long ago in connection with data compression and the intervening years had made all the details fuzzy which explains why it had not immediately sprung to mind.

Huffman coding named for David A. Huffman is an algorithm that enables a representation of data which is very efficient. In a normal array of characters every character takes the same eight bits to represent which is the best we can do when any of the 256 values possible is equally likely. If your data is not evenly distributed this is not the case for example if the data was english text then the value is fifteen times more likely to be that for e than k.

So if we have some data with a non uniform distribution of probabilities we need a way the data be encoded with fewer bits for the common values and more bits for the rarer values. To be efficient we would need some way of having variable length representations without storing the length separately. The term for this data representation is a prefix code and there are several ways to generate them.

Such is the influence of Huffman on the area of prefix codes they are often called Huffman codes even if they were not created using his algorithm. One can dream of becoming immortalised like this, to join the ranks of those whose names are given to units or whole ideas in a field must be immensely rewarding, however given Huffman invented his algorithm and proved it to be optimal to answer a question on a term paper in his early twenties I fear I may already be a bit too late.

The algorithm itself is relatively straightforward. First a frequency analysis is performed, a fancy way of saying count how many of each character is in the input data. Next a binary tree is created by using a priority queue initialised with the nodes sorted by frequency.

The two least frequent items count is summed together and a node placed in the tree with the two original entries as child nodes. This step is repeated until a single node exists with a count value equal to the length of the input.

To encode data once simply walks the tree outputting a 0 for a left node or 1 for right node until reaching the original value. This generates a mapping of values to bit output, the input is then simply converted value by value to the bit output. To decode the data the data is used bit by bit to walk the tree to arrive at values.

If we perform this algorithm on the example string table *!asiabvcomcoopitamazonawsarsaves-the-whalescomputebasilicata we can reduce the 488 bits (61 * 8 bit characters) to 282 bits or 40% reduction. Obviously in a real application the huffman tree would need to be stored which would probably exceed this saving but for larger data sets it is probable this technique would yield excellent results on this kind of data.

Once I proved this to myself I implemented the encoder within the existing conversion program. Although my perl encoder is not very efficient it can process the entire PSL string table (around six thousand labels using 40KB or so) in less than a second, so unless the table grows massively an inelegant approach will suffice.

The resulting bits were packed into 32bit values to improve decode performance (most systems prefer to deal with larger memory fetches less frequently) and resulted in 18KB of output or 47% of the original size. This is a great improvement in size and means the statically linked test program is now 59KB and is actually smaller than the gzipped source data.

$ ls -alh test_nspsl
-rwxr-xr-x 1 vince vince 59K Sep 25 23:58 test_nspsl
$ ls -al public_suffix_list.dat.gz 
-rw-r--r-- 1 vince vince 62K Sep  1 08:52 public_suffix_list.dat.gz

To be clear the statically linked program can determine if a domain is in the PSL with no additional heap allocations and includes the entire PSL ordered tree, the domain label string table and the huffman decode table to read it.

An unexpected side effect is that because the decode loop is small it sits in the processor cache. This appears to cause the string comparison function huffcasecmp() (which is not locale dependant because we know the data will be limited ASCII) performance to be close to using strcasecmp() indeed on ARM32 systems there is a very modest improvement in performance.

I think this is as much work as I am willing to put into this library but I am pleased to have achieved a result which is on par with the best of breed (libpsl still has a data representation 20KB smaller than libnspsl but requires additional libraries for additional functionality) and I got to (re)learn an important algorithm too.

Abstract I introduce TrieHash an algorithm for constructing perfect hash functions from tries. The generated hash functions are pure C code, minimal, order-preserving and outperform existing alternatives. Together with the generated header files,they can also be used as a generic string to enumeration mapper (enums are created by the tool). Introduction APT (and dpkg) spend a lot of time in parsing various files, especially Packages files. APT currently uses a function called AlphaHash which hashes the last 8 bytes of a word in a case-insensitive manner to hash fields in those files (dpkg just compares strings in an array of structs). There is one obvious drawback to using a normal hash function: When we want to access the data in the hash table, we have to hash the key again, causing us to hash every accessed key at least twice. It turned out that this affects something like 5 to 10% of the cache generation performance. Enter perfect hash functions: A perfect hash function matches a set of words to constant values without collisions. You can thus just use the index to index into your hash table directly, and do not have to hash again (if you generate the function at compile time and store key constants) or handle collision resolution. As #debian-apt people know, I happened to play a bit around with tries this week before guillem suggested perfect hashing. Let me tell you one thing: My trie implementation was very naive, that did not really improve things a lot Enter TrieHash Now, how is this related to hashing? The answer is simple: I wrote a perfect hash function generator that is based on tries. You give it a list of words, it puts them in a trie, and generates C code out of it, using recursive switch statements (see code generation below). The function achieves competitive performance with other hash functions, it even usually outperforms them. Given a dictionary, it generates an enumeration (a C enum or C++ enum class) of all words in the dictionary, with the values corresponding to the order in the dictionary (the order-preserving property), and a function mapping strings to members of that enumeration. By default, the first word is considered to be 0 and each word increases a counter by one (that is, it generates a minimal hash function). You can tweak that however:

= 0
WordLabel ~ Word
OtherWord = 9

will return 0 for an unknown value, map Word to the enum member WordLabel and map OtherWord to 9. That is, the input list functions like the body of a C enumeration. If no label is specified for a word, it will be generated from the word. For more details see the documentation C code generation

switch(string[0]   32)  
case 't':
    switch(string[1]   32)  
    case 'a':
        switch(string[2]   32)  
        case 'g':
            return Tag;
         
     
 
return Unknown;

Yes, really recursive switches they directly represent the trie. Now, we did not really do a straightforward translation, there are some optimisations to make the whole thing faster and easier to look at: First of all, the 32 you see is used to make the check case insensitive in case all cases of the switch body are alphabetical characters. If there are non-alphabetical characters, it will generate two cases per character, one upper case and one lowercase (with one break in it). I did not know that lowercase and uppercase characters differed by only one bit before, thanks to the clang compiler for pointing that out in its generated assembler code! Secondly, we only insert breaks only between cases. Initially, each case ended with a return Unknown, but guillem (the dpkg developer) suggested it might be faster to let them fallthrough where possible. Turns out it was not faster on a good compiler, but it s still more readable anywhere. Finally, we build one trie per word length, and switch by the word length first. Like the 32 trick, his gives a huge improvement in performance. Digging into the assembler code The whole code translates to roughly 4 instructions per byte:

1. A memory load,
2. an or with 32
3. a comparison, and
4. a conditional jump.

(On x86, the case sensitive version actually only has a cmp-with-memory and a conditional jump). Due to https://gcc.gnu.org/bugzilla/show_bug.cgi?id=77729 this may be one instruction more: On some architectures an unneeded zero-extend-byte instruction is inserted this causes a 20% performance loss. Performance evaluation I run the hash against all 82 words understood by APT in Packages and Sources files, 1,000,000 times for each word, and summed up the average run-time:

host	arch	Trie	TrieCase	GPerfCase	GPerf	DJB
plummer	ppc64el	540	601	1914	2000	1345
eller	mipsel	4728	5255	12018	7837	4087
asachi	arm64	1000	1603	4333	2401	1625
asachi	armhf	1230	1350	5593	5002	1784
barriere	amd64	689	950	3218	1982	1776
x230	amd64	465	504	1200	837	693

Suffice to say, GPerf does not really come close. All hosts except the x230 are Debian porterboxes. The x230 is my laptop with a a Core i5-3320M, barriere has an Opteron 23xx. I included the DJB hash function for another reference. Source code The generator is written in Perl, licensed under the MIT license and available from https://github.com/julian-klode/triehash I initially prototyped it in Python, but guillem complained that this would add new build dependencies to dpkg, so I rewrote it in Perl. Benchmark is available from https://github.com/julian-klode/hashbench Usage See the script for POD documentation.
Filed under: General

When Samfundet wanted to make their own Boiler Room spinoff (called Fyrrom more or less a direct translation), it was a great opportunity to try out the new multitrack code in Nageru. After all, what can go wrong with a pretty much untested and unfinished git branch, right? So we cobbled together a bunch of random equipment from here and there: Hooked it up to Nageru: and together with some great work from the people actually pulling together the event, this was the result. Lots of fun. And yes, some bugs were discovered of course, field testing without followup patches is meaningless (that would either mean you're not actually taking your test experience into account, or that your testing gave no actionable feedback and thus was useless), so they will be fixed in due time for the 1.4.0 release. Edit: Fixed a screenshot link.

Around a week ago, I started to play with programmatically controlling Azure. I needed to create and destroy a bunch of VMs over and over again, and this seemed like something I would want to automate once instead of doing manually and repeatedly. I started to look into the azure-sdk-for-python and mentioned that I wanted to look into this in #debian-python. ardumont from Software Heritage noticed me, and was planning to package azure-storage-python. We joined forces and started a packaging team for Azure-related software. I spoke with the upstream developer of the azure-sdk-for-python and he pointed me towards azure-cli. It looked to me that this fit my use case better than the SDK alone, as it had the high level commands I was looking for. Between me and ardumont, in the space of just under a week, we have now packaged: python-msrest (#838121), python-msrestazure (#838122), python-azure (#838101), python-azure-storage (#838135), python-adal (#838716), python-applicationinsights (#838717) and finally azure-cli (#838708). Some of these packages are still in the NEW queue at the time I m writing this, but I don t foresee any issues with these packages entering unstable. azure-cli, as we have packaged, is the new Python-based CLI for Azure. The Microsoft developers gave it the tagline of our next generation multi-platform command line experience for Azure . In the short time I ve been using it I ve been very impressed with it. In order to set it up initially, you have to configure a couple of of defaults using az configure. After that, you need to az login which again is an entirely painless process as long as you have a web browser handy in order to perform the login. After those two steps, you re only two commands away from deploying a Debian virtual machine:

az resource group create -n testgroup -l "West US"
az vm create -n testvm -g testgroup --image credativ:Debian:8:latest --authentication-type ssh

This will create a resource group, and then create a VM within that resource group with a user automatically created with your current username and with your SSH public key (~/.ssh/id_rsa.pub) automatically installed. Once it returns you the IP address, you can SSH in straight away. Looking forward to some next steps for Debian on Azure, I d like to get images built for Azure using vmdebootstrap and I ll be exploring this in the lead up to, and at, the upcoming vmdebootstrap sprint in Cambridge, UK later in the year (still being organised).

Categories:

• Debian-Blog
• Tools

Keywords:

• Laptop Mode Tools
• power saving

Like:

This blog has finally moved away from blogger to jekyll, also changing the hosting and the domain. No new content will be published here.

New coordinates:

• name: ral-arturo.org
• hosting: github pages
• technology: jekyllrb

After a month of development, keysafe 0.20160922 is released, and ready for beta testing. And it needs servers. With this release, the whole process of backing up and restoring a gpg secret key to keysafe servers is implemented. Keysafe is started at desktop login, and will notice when a gpg secret key has been created, and prompt to see if it should back it up. At this point, I recommend only using keysafe for lower-value secret keys, for several reasons:

• There could be some bug that prevents keysafe from restoring a backup.
• Keysafe's design has not been completely reviewed for security.
• None of the keysafe servers available so far or planned to be deployed soon meet all of the security requirements for a recommended keysafe server. While server security is only the initial line of defense, it's still important.

Currently the only keysafe server is one that I'm running myself. Two more keysafe servers are needed for keysafe to really be usable, and I can't run those. If you're interested in running a keysafe server, read the keysafe server requirements and get in touch.

Finally, I decided it was time to switch from blogger to jekyllrb hosted at github pages. My old blog at http://ral-arturo.blogspot.com.es will still be online as an archive, since I don t plan to migrate the content from there to here.

Next week our friends at Igalia will be hosting this year s Web Engines Hackfest. Collabora will be there! We are gold sponsors, and have three developers attending. It will also be an opportunity to celebrate Igalia s 15th birthday \o/. Looking forward to meet you there! =) Carlos Garcia has recently released WebKitGTK+ 2.14, the latest stable release. This is a great release that brings a lot of improvements and works much better on Wayland, which is becoming mature enough to be used by default. In particular, it fixes the clipboard, which was one of the main missing features, thanks to Carlos Garnacho! We have also been able to contribute a bit to this release =) One of the biggest changes this cycle is the threaded compositor, which was implemented by Igalia s Gwang Yoon Hwang. This work improves performance by not stalling other web engine features while compositing. Earlier this year we contributed fixes to make the threaded compositor work with the web inspector and fixed elements, helping with the goal of enabling it by default for this release. Wayland was also lacking an accelerated compositing implementation. There was a patch to add a nested Wayland compositor to the UIProcess, with the WebProcesses connecting to it as Wayland clients to share the final rendering so that it can be shown to screen. It was not ready though and there were questions as to whether that was the way to go and alternative proposals were floating around on how to best implement it. At last year s hackfest we had discussions about what the best path for that would be where collaborans Emanuele Aina and Daniel Stone (proxied by Emanuele) contributed quite a bit on figuring out how to implement it in a way that was both efficient and platform agnostic. We later picked up the old patchset, rebased on the then-current master and made it run efficiently as proof of concept for the Apertis project on an i.MX6 board. This was done using the fancy GL support that landed in GTK+ in the meantime, with some API additions and shortcuts to sidestep performance issues. The work was sponsored by Robert Bosch Car Multimedia. Igalia managed to improve and land a very well designed patch that implements the nested compositor, though it was still not as efficient as it could be, as it was using glReadPixels to get the final rendering of the page to the GTK+ widget through cairo. I have improved that code by ensuring we do not waste memory when using HiDPI. As part of our proof of concept investigation, we got this WebGL car visualizer running quite well on our sabrelite imx6 boards. Some of it went into the upstream patches or proposals mentioned below, but we have a bunch of potential improvements still in store that we hope to turn into upstreamable patches and advance during next week s hackfest. One of the improvements that already landed was an alternate code path that leverages GTK+ s recent GL super powers to render using gdk_cairo_draw_from_gl(), avoiding the expensive copying of pixels from the GPU to the CPU and making it go faster. That improvement exposed a weird bug in GTK+ that causes a black patch to appear when shrinking the window, which I have a tentative fix for. We originally proposed to add a new gdk_cairo_draw_from_egl() to use an EGLImage instead of a GL texture or renderbuffer. On our proof of concept we noticed it is even more efficient than the texturing currently used by GTK+, and could give us even better performance for WebKitGTK+. Emanuele Bassi thinks it might be better to add EGLImage as another code branch inside from_gl() though, so we will look into that. Another very interesting igalian addition to this release is support for the MemoryPressureHandler even on systems with no cgroups set up. The memory pressure handler is a WebKit feature which flushes caches and frees resources that are not being used when the operating system notifies it memory is scarce. We worked with the Raspberry Pi Foundation to add support for that feature to the Raspberry Pi browser and contributed it upstream back in 2014, when Collabora was trying to squeeze as much as possible from the hardware. We had to add a cgroups setup to wrap Epiphany in, back then, so that it would actually benefit from the feature. With this improvement, it will benefit even without the custom cgroups setups as well, by having the UIProcess monitor memory usage and notify each WebProcess when memory is tight. Some of these improvements were achieved by developers getting together at the Web Engines Hackfest last year and laying out the ground work or ideas that ended up in the code base. I look forward to another great few days of hackfest next week! See you there o/