Search Results: "ron"

5 October 2016

Kees Cook: security things in Linux v4.8

Previously: v4.7. Here are a bunch of security things I m excited about in Linux v4.8: SLUB freelist ASLR Thomas Garnier continued his freelist randomization work by adding SLUB support. x86_64 KASLR text base offset physical/virtual decoupling On x86_64, to implement the KASLR text base offset, the physical memory location of the kernel was randomized, which resulted in the virtual address being offset as well. Due to how the kernel s -2GB addressing works (gcc s -mcmodel=kernel ), it wasn t possible to randomize the physical location beyond the 2GB limit, leaving any additional physical memory unused as a randomization target. In order to decouple the physical and virtual location of the kernel (to make physical address exposures less valuable to attackers), the physical location of the kernel needed to be randomized separately from the virtual location. This required a lot of work for handling very large addresses spanning terabytes of address space. Yinghai Lu, Baoquan He, and I landed a series of patches that ultimately did this (and in the process fixed some other bugs too). This expands the physical offset entropy to roughly $physical_memory_size_of_system / 2MB bits. x86_64 KASLR memory base offset Thomas Garnier rolled out KASLR to the kernel s various statically located memory ranges, randomizing their locations with CONFIG_RANDOMIZE_MEMORY. One of the more notable things randomized is the physical memory mapping, which is a known target for attacks. Also randomized is the vmalloc area, which makes attacks against targets vmalloced during boot (which tend to always end up in the same location on a given system) are now harder to locate. (The vmemmap region randomization accidentally missed the v4.8 window and will appear in v4.9.) x86_64 KASLR with hibernation Rafael Wysocki (with Thomas Garnier, Borislav Petkov, Yinghai Lu, Logan Gunthorpe, and myself) worked on a number of fixes to hibernation code that, even without KASLR, were coincidentally exposed by the earlier W^X fix. With that original problem fixed, then memory KASLR exposed more problems. I m very grateful everyone was able to help out fixing these, especially Rafael and Thomas. It s a hard place to debug. The bottom line, now, is that hibernation and KASLR are no longer mutually exclusive. gcc plugin infrastructure Emese Revfy ported the PaX/Grsecurity gcc plugin infrastructure to upstream. If you want to perform compiler-based magic on kernel builds, now it s much easier with CONFIG_GCC_PLUGINS! The plugins live in scripts/gcc-plugins/. Current plugins are a short example called Cyclic Complexity which just emits the complexity of functions as they re compiled, and Sanitizer Coverage which provides the same functionality as gcc s recent -fsanitize-coverage=trace-pc but back through gcc 4.5. Another notable detail about this work is that it was the first Linux kernel security work funded by Linux Foundation s Core Infrastructure Initiative. I m looking forward to more plugins! If you re on Debian or Ubuntu, the required gcc plugin headers are available via the gcc-$N-plugin-dev package (and similarly for all cross-compiler packages). hardened usercopy Along with work from Rik van Riel, Laura Abbott, Casey Schaufler, and many other folks doing testing on the KSPP mailing list, I ported part of PAX_USERCOPY (the basic runtime bounds checking) to upstream as CONFIG_HARDENED_USERCOPY. One of the interface boundaries between the kernel and user-space are the copy_to_user()/copy_from_user() family of functions. Frequently, the size of a copy is known at compile-time ( built-in constant ), so there s not much benefit in checking those sizes (hardened usercopy avoids these cases). In the case of dynamic sizes, hardened usercopy checks for 3 areas of memory: slab allocations, stack allocations, and kernel text. Direct kernel text copying is simply disallowed. Stack copying is allowed as long as it is entirely contained by the current stack memory range (and on x86, only if it does not include the saved stack frame and instruction pointers). For slab allocations (e.g. those allocated through kmem_cache_alloc() and the kmalloc()-family of functions), the copy size is compared against the size of the object being copied. For example, if copy_from_user() is writing to a structure that was allocated as size 64, but the copy gets tricked into trying to write 65 bytes, hardened usercopy will catch it and kill the process. For testing hardened usercopy, lkdtm gained several new tests: USERCOPY_HEAP_SIZE_TO, USERCOPY_HEAP_SIZE_FROM, USERCOPY_STACK_FRAME_TO,
USERCOPY_STACK_FRAME_FROM, USERCOPY_STACK_BEYOND, and USERCOPY_KERNEL. Additionally, USERCOPY_HEAP_FLAG_TO and USERCOPY_HEAP_FLAG_FROM were added to test what will be coming next for hardened usercopy: flagging slab memory as safe for copy to/from user-space , effectively whitelisting certainly slab caches, as done by PAX_USERCOPY. This further reduces the scope of what s allowed to be copied to/from, since most kernel memory is not intended to ever be exposed to user-space. Adding this logic will require some reorganization of usercopy code to add some new APIs, as PAX_USERCOPY s approach to handling special-cases is to add bounce-copies (copy from slab to stack, then copy to userspace) as needed, which is unlikely to be acceptable upstream. seccomp reordered after ptrace By its original design, seccomp filtering happened before ptrace so that seccomp-based ptracers (i.e. SECCOMP_RET_TRACE) could explicitly bypass seccomp filtering and force a desired syscall. Nothing actually used this feature, and as it turns out, it s not compatible with process launchers that install seccomp filters (e.g. systemd, lxc) since as long as the ptrace and fork syscalls are allowed (and fork is needed for any sensible container environment), a process could spawn a tracer to help bypass a filter by injecting syscalls. After Andy Lutomirski convinced me that ordering ptrace first does not change the attack surface of a running process (unless all syscalls are blacklisted, the entire ptrace attack surface will always be exposed), I rearranged things. Now there is no (expected) way to bypass seccomp filters, and containers with seccomp filters can allow ptrace again. That s it for v4.8! The merge window is open for v4.9

2016, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.

3 October 2016

Markus Koschany: My Free Software Activities in September 2016

Welcome to gambaru.de. Here is my monthly report that covers what I have been doing for Debian. If you re interested in Android, Java, Games and LTS topics, this might be interesting for you. Debian Android

I sponsored a new upstream release of android-platform-tools-base prepared by Kai-Chung Yan and Chirayu Desai.

Debian Games

I packaged a new upstream release of hyperrogue, a rogue-like game settled in a non-euclidian world, fixing one RC bug (#811991). I uploaded two more revisions later that addressed build failures on arm64 and hppa.
I fixed more RC bugs (build failures with GCC-6) in torus-trooper (#835712) and fife (#811858).
I packaged new upstream releases of pygame-sdl2, renpy, freeorion, netrek-client-cow, redeclipse, redeclipse-data, hitori, atomix, adonthell and adonthell-data.
I updated gtkballs and fixed a documentation bug (#820588) but also a /usr/share/locale issue that prevented the actual use of the translations.
I raised the severity of #797998 to grave in unknown-horizons because the game cannot be started currently. In order to fix this issue I packaged a new build-dependency, fifechan, which is currently awaiting approval by the FTP team. As soon as fifechan got accepted I will upload new upstream releases of fife and unknown-horizons.
I released debian-games 1.5, a Debian blend and collection of games metapackages.
Hardening-wrapper has been deprecated for some time and this issue became release critical now. I updated cookietool, alex4 and netrek-client-cow to use dpkg-buildflags instead.
Together with Russel Coker I packaged a new upstream release of warzone2100. This package would benefit from a new regular uploader. If you are interested in it, please get involved. (Same story for hyperrogue, redeclipse, renpy and unknown-horizons and many other games.)
I started a new Bullet transition (#839243). The package is currently waiting in the NEW queue and I hope to complete this work in October.
I triaged #838199 and reassigned the issue to fonts-roboto. Initially I prepared an NMU but eventually the maintainer uploaded a new revision himself. It is now possible to install the hinted and unhinted versions of fonts-roboto together which also resolved former installation problems with kodi and freeorion.

Debian Java

I packaged new upstream releases of undertow, activemq and jackrabbit.
I fixed RC bugs in libphonenumber (#836768), wagon2 (#837022) and activemq (#839244).
I updated syncany in experimental and simplified the packaging a little. Unfortunately upstream has been on hiatus for the past year and we haven t seen new releases in the meantime. Nevertheless give it a try, even though it is still alpha software, it s an useful cloud-storage and synchronization tool.
I sponsored a new upstream release of freeplane for Felix Natter.
I prepared and uploaded security updates for jackrabbit and zookeeper in Jessie.

Debian LTS This was my eight month as a paid contributor and I have been paid to work 12,25 hours on Debian LTS, a project started by Rapha l Hertzog. In that time I did the following:

From 12. September until 19. September I was in charge of our LTS frontdesk. I triaged bugs in tiff3, mysql-5.5, curl, dropbear, mantis, icu, dwarfutils, jackrabbit, zendframework, zookeeper and graphicsmagick. For the latter I skimmed through all commits since the last version to identify the patches that fix the recent issues in graphicsmagick. I also answered questions on the mailing list and contacted Diego Biurrun again about his progress with libav. It is now anticipated that Hugo Lefeuvre and Diego will issue a new libav security release this month.
I reviewed and tested a patch by Rapha l Hertzog for roundcube.
DLA-629-1. Issued a security update for jackrabbit fixing 1 CVE.
DLA-630-1. Issued a security update for zookeeper fixing 1 CVE.
DLA-633-1. Issued a security update for wordpress fixing 7 CVE. This one also required backports of certain functions from newer releases and a database upgrade that required careful testing.
I also issued DLA-622-1 and DLA-623-1, two security issues that I already mentioned last month. It was discovered that Debian s versions of Tomcat were vulnerable to a root privilege escalation issue. However it was also necessary that another exploit, for instance in a web application, could be used to gain write access as the tomcat user. Former security issues were already fixed and new ones are not known. Nevertheless since a zero-day exploit could not be ruled out, the issue was embargoed for a month to give other distributions time to fix this issue as well. You can read more about this topic at legalhackers.com.

Non-maintainer uploads

I fixed various RC bugs in several games that are not maintained by the Games Team. The following games will be available in Stretch again soon: solarwolf, enigma, open-invaders, crrcsim, noiz2sa, csmash, csmash-demosong and glob2.

Misc

I packaged a new upstream release of MediathekView.
I uploaded a new revision of xarchiver and applied a patch from Helmut Grohne that made it possible to cross-build the package.

Matthew Garrett: The importance of paying attention in building community trust

Trust is important in any kind of interpersonal relationship. It's inevitable that there will be cases where something you do will irritate or upset others, even if only to a small degree. Handling small cases well helps build trust that you will do the right thing in more significant cases, whereas ignoring things that seem fairly insignificant (or saying that you'll do something about them and then failing to do so) suggests that you'll also fail when there's a major problem. Getting the small details right is a major part of creating the impression that you'll deal with significant challenges in a responsible and considerate way.

This isn't limited to individual relationships. Something that distinguishes good customer service from bad customer service is getting the details right. There are many industries where significant failures happen infrequently, but minor ones happen a lot. Would you prefer to give your business to a company that handles those small details well (even if they're not overly annoying) or one that just tells you to deal with them?

And the same is true of software communities. A strong and considerate response to minor bug reports makes it more likely that users will be patient with you when dealing with significant ones. Handling small patch contributions quickly makes it more likely that a submitter will be willing to do the work of making more significant contributions. These things are well understood, and most successful projects have actively worked to reduce barriers to entry and to be responsive to user requests in order to encourage participation and foster a feeling that they care.

But what's often ignored is that this applies to other aspects of communities as well. Failing to use inclusive language may not seem like a big thing in itself, but it leaves people with the feeling that you're less likely to do anything about more egregious exclusionary behaviour. Allowing a baseline level of sexist humour gives the impression that you won't act if there are blatant displays of misogyny. The more examples of these "insignificant" issues people see, the more likely they are to choose to spend their time somewhere else, somewhere they can have faith that major issues will be handled appropriately.

There's a more insidious aspect to this. Sometimes we can believe that we are handling minor issues appropriately, that we're acting in a way that handles people's concerns, while actually failing to do so. If someone raises a concern about an aspect of the community, it's important to discuss solutions with them. Putting effort into "solving" a problem without ensuring that the solution has the desired outcome is not only a waste of time, it alienates those affected even more - they're now not only left with the feeling that they can't trust you to respond appropriately, but that you will actively ignore their feelings in the process.

It's not always possible to satisfy everybody's concerns. Sometimes you'll be left in situations where you have conflicting requests. In that case the best thing you can do is to explain the conflict and why you've made the choice you have, and demonstrate that you took this issue seriously rather than ignoring it. Depending on the issue, you may still alienate some number of participants, but it'll be fewer than if you just pretend that it's not actually a problem.

One warning, though: while building trust in this way enhances people's willingness to join your community, it also builds expectations. If a significant issue does arise, and if you fail to handle it well, you'll burn a lot of that trust in the process. The fact that you've built that trust in the first place may be what saves your community from disintegrating completely, but people will feel even more betrayed if you don't actively work to rebuild it. And if there's a pattern of mishandling major problems, no amount of getting the details right will matter.

Communities that ignore these issues are, long term, likely to end up weaker than communities that pay attention to them. Making sure you get this right in the first place, and setting expectations that you will pay attention to your contributors, is a vital part of building a meaningful relationship between your community and its members.

comments

Shirish Agarwal: Using JOSM and gpx tracks

This would be a longish post. I had bought a Samsung Galaxy J-5/500 just a few days before Debconf16 which I had written about a bit earlier as well. As can be seen in the specs there isn t much to explore other than A-GPS. There were a couple of temperature apps. which I wanted to explore before buying the smartphone but as there were budget constraints and there weren t any good budget smartphones with environmental sensors built-in had to let go of those features. I was looking for a free app. which would have OSM support and came across osmand . I was looking for an app. which would have support for OSM and support for the gpx format. I was planning to use osmand in South Africa but due to the over-whelming nature of meeting people, seeing places and just being didn t actually get the time and place to try it. Came back home and a month and a half passed. In-between I had done some simple small tracks but nothing major. This week-end I got the opportunity as I got some free data balance from my service provider (a princely 50 MB) as well an opportunity to go about 40 odd kms. from the city. I had read about osmand and was looking if the off-line method worked or not from the webpage Works online (fast) or offline (no roaming charges when you are abroad) So armed with a full battery I started the journey which took about an hour and half even though technically it was a holiday. On the way back, got a different route and recorded that as well. The app. worked flawlessly. I was able to get the speed of the vehicle and everything. The only thing I haven t understood till date is how to select waypoints but other than that I got the whole route on my mobile. What was cool to see that on most roads, at least through the app. there were speed limits. Now I don t know from where the data was getting there. Most city roads had 65 and some roads had 55 at the maximum. I am assuming that it was at km/hr as also shared later. Just for fun I also looked at the gpx file after copying it from mobile to hdd (an extract) https://paste.debian.net/853231/ While it s not a complete extract, What was interesting for me to note here is the time was in UTC . What was also interesting is that in the gpx tracks I also saw some entries about speed as can be seen in the paste above. Although it doesn t say whether it was in km/hr or mph, I believe it probably is km/hr. as that is the unit I defined in the app. Anyways, the next step was trying to see which tool was good enough to show me the tracks with tiles underneath and labels of places, paths etc. I tried three tools 1. jmapviewer this didn t work at all.
2. gnome-maps this worked remarkably well but has numerous gtk3.0 warnings

 [shirish@debian] - [~/osmand] - [10149]

 [$] gnome-maps 2016-10-01_08-11_Sat.gpx


(gnome-maps:21017): Gtk-WARNING **: Theme parsing error: gtk.css:63:28: The :prelight pseudo-class is deprecated. Use :hover instead.
(gnome-maps:21017): Gtk-WARNING **: Theme parsing error: gtk.css:73:35: The :prelight pseudo-class is deprecated. Use :hover instead.
(gnome-maps:21017): Gtk-WARNING **: Theme parsing error: application.css:14:30: The style property GtkButton:image-spacing is deprecated and shouldn't be used anymore. It will be removed in a future version
(gnome-maps:21017): Gtk-WARNING **: Theme parsing error: application.css:15:31: The style property GtkWidget:interior-focus is deprecated and shouldn't be used anymore. It will be removed in a future version
(gnome-maps:21017): Gdk-WARNING **: /build/gtk+3.0-Tod2iD/gtk+3.0-3.22.0/./gdk/x11/gdkwindow-x11.c:5554 drawable is not a native X11 window
(gnome-maps:21017): Gdk-WARNING **: /build/gtk+3.0-Tod2iD/gtk+3.0-3.22.0/./gdk/x11/gdkwindow-x11.c:5554 drawable is not a native X11 window
(gnome-maps:21017): Gdk-WARNING **: /build/gtk+3.0-Tod2iD/gtk+3.0-3.22.0/./gdk/x11/gdkwindow-x11.c:5554 drawable is not a native X11 window
(gnome-maps:21017): Gdk-WARNING **: /build/gtk+3.0-Tod2iD/gtk+3.0-3.22.0/./gdk/x11/gdkwindow-x11.c:5554 drawable is not a native X11 window
(gnome-maps:21017): Gdk-WARNING **: /build/gtk+3.0-Tod2iD/gtk+3.0-3.22.0/./gdk/x11/gdkwindow-x11.c:5554 drawable is not a native X11 window
(gnome-maps:21017): Gdk-WARNING **: /build/gtk+3.0-Tod2iD/gtk+3.0-3.22.0/./gdk/x11/gdkwindow-x11.c:5554 drawable is not a native X11 window
(gnome-maps:21017): Gtk-WARNING **: GtkClutterOffscreen 0x4c4f3f0 is drawn without a current allocation. This should not happen.

(gnome-maps:21017): Gtk-WARNING **: GtkImage 0x4ed4140 is drawn without a current allocation. This should not happen. Now I m not sure whether all of those are gtk3+ issues or me running them under Debian MATE. I know that there are issues with mate and gtk3+ as had been told/shared a few times in p.d.o. Anyways, one of the issues I encountered is that gnome-maps doesn t work in offline-mode, saw https://bugzilla.gnome.org/show_bug.cgi?id=708799 . Also saw ~/.cache/champlain/osm-mapquest and the listing underneath is gibberish in the sense you don t know what it meant to do



 [shirish@debian] - [~/.cache/champlain/osm-mapquest] - [10163]

 [$] ll -h

drwx------  6 shirish shirish 4.0K Jun 11  2015 10

drwx------ 26 shirish shirish 4.0K Oct 24  2014 11

drwx------ 10 shirish shirish 4.0K Jun 11  2015 12

drwx------ 11 shirish shirish 4.0K Jun 11  2015 13

drwx------ 12 shirish shirish 4.0K Jun 11  2015 14

drwx------ 12 shirish shirish 4.0K Jun 11  2015 15

drwx------ 27 shirish shirish 4.0K Oct 24  2014 16

drwx------ 25 shirish shirish 4.0K Oct 24  2014 17

drwx------  4 shirish shirish 4.0K Mar  4  2014 3

drwx------  5 shirish shirish 4.0K Mar  4  2014 8

drwx------  9 shirish shirish 4.0K Mar 29  2014 9

What was/is interesting to see things like this maybe imagery offset

As I was in a moving vehicle, it isn t easy to know if the imagery is at fault or was it app. , sensor of my mobile ? Did see http://learnosm.org/en/josm/correcting-imagery-offset/ but as can be seen that requires more effort from my side. The last tool proved to be the most problematic 3. JOSM Getting the tracks into josm which was easily done. While firing up josm came across https://josm.openstreetmap.de/ticket/13735 which I subsequently filed. One of the other first things which has been a major irritant for a long time is JOSM is, for a lack of better term, ugly. See the interface, especially the one having preferences, all cluttered look and specifically see the plugins corner/tab josm-preferences-plugin tab

The part about it being ugly, I dunno but have seen most java apps are a bit ugly. It is a bit generalist I know but that has been my experience with whatever little java apps. I have used. I don t know what the reasons for that are, maybe because java is known/rumoured to use lot of memory which seems true in my case as well OR it doesn t have toolkits like gtk3+ or qt quick, although have to say that the looks have improved from before when I used it last some years ago



 [shirish@debian] - [~] - [10340]

 [$] ps -eo size,pid,user,command   awk '  hr=$1/1024 ; printf("%13.6f Mb ",hr)     for ( x=4 ; x<=NF ; x++ )   printf("%s ",$x)   print ""  '   grep josm

     0.324219 Mb /bin/sh /usr/bin/josm

   419.468750 Mb /usr/lib/jvm/java-8-openjdk-amd64/bin/java -Djosm.restart=true -Djava.net.useSystemProxies=true -jar /usr/share/josm/josm.jar

This is when I m just opening josm and have not added any tracks or done any work. Now I wanted to explore the routing in good amount of detail on josm. This was easily said than done. When trying to get imagery I got the Download area too large issue/defect . Multiple tries didn t get me anywhere. Then hunting on the web came across the continous-download plugin which is part of the plugin infrastructure . This I found to be a very good tool. It downloads the tiles and puts them in ~/.josm/cache/tiles



 [shirish@debian] - [~/.josm/cache/tiles] - [10147]

 [$] ll -h

total 28M

-rw-r--r-- 1 shirish shirish  28M Oct  2 02:13 TMS_BLOCK_v2.data

-rw-r--r-- 1 shirish shirish 290K Oct  3 12:59 TMS_BLOCK_v2.key

-rw-r--r-- 1 shirish shirish    4 Oct  3 12:59 WMS_BLOCK_v2.key

-rw-r--r-- 1 shirish shirish    4 Oct  3 12:59 WMTS_BLOCK_v2.key

While unfortunately I cannot see this/make sense of it I m guessing it is some sort of database with key and data files. What did become apparent is that the OSM needs lots more love if it is to become something which can be used everyday. At the end I had to change the open-source gpx track file to a Google map kml file to be able to make sense as there are whole areas which need to be named, numbered etc. One of the newbie mistakes that I did was trying to use the slippy map in josm (using Openstreetmap/Mapnik at the back-end) to move/pan using the left-hand mouse button. It took me quite sometime to figure out that it is with right-hand mouse button that you can make the slippy map pan. This is different from almost all maps, gnome-maps uses the traditional left-hand button, Google maps also uses the same. I have filed it in upstream as https://josm.openstreetmap.de/ticket/13751 . So at least, in these rounds it is gnome-maps which has kind of won even though it doesn t do any of the things that josm claims to do. I am sure there might be some interesting tricks and tips that people might have to share about mapping

Filed under: Miscellenous Tagged: #Debconf16, #GNOME-MAPS, #JOSM, OSM

Kees Cook: security things in Linux v4.7

Previously: v4.6. Onward to security things I found interesting in Linux v4.7: KASLR text base offset for MIPS Matt Redfearn added text base address KASLR to MIPS, similar to what s available on x86 and arm64. As done with x86, MIPS attempts to gather entropy from various build-time, run-time, and CPU locations in an effort to find reasonable sources during early-boot. MIPS doesn t yet have anything as strong as x86 s RDRAND (though most have an instruction counter like x86 s RDTSC), but it does have the benefit of being able to use Device Tree (i.e. the /chosen/kaslr-seed property) like arm64 does. By my understanding, even without Device Tree, MIPS KASLR entropy should be as strong as pre-RDRAND x86 entropy, which is more than sufficient for what is, similar to x86, not a huge KASLR range anyway: default 8 bits (a span of 16MB with 64KB alignment), though CONFIG_RANDOMIZE_BASE_MAX_OFFSET can be tuned to the device s memory, giving a maximum of 11 bits on 32-bit, and 15 bits on EVA or 64-bit. SLAB freelist ASLR Thomas Garnier added CONFIG_SLAB_FREELIST_RANDOM to make slab allocation layouts less deterministic with a per-boot randomized freelist order. This raises the bar for successful kernel slab attacks. Attackers will need to either find additional bugs to help leak slab layout information or will need to perform more complex grooming during an attack. Thomas wrote a post describing the feature in more detail here: Randomizing the Linux kernel heap freelists. (SLAB is done in v4.7, and SLUB in v4.8.) eBPF JIT constant blinding Daniel Borkmann implemented constant blinding in the eBPF JIT subsystem. With strong kernel memory protections (CONFIG_DEBUG_RODATA) in place, and with the segregation of user-space memory execution from kernel (i.e SMEP, PXN, CONFIG_CPU_SW_DOMAIN_PAN), having a place where user-space can inject content into an executable area of kernel memory becomes very high-value to an attacker. The eBPF JIT was exactly such a thing: the use of BPF constants could result in the JIT producing instruction flows that could include attacker-controlled instructions (e.g. by directing execution into the middle of an instruction with a constant that would be interpreted as a native instruction). The eBPF JIT already uses a number of other defensive tricks (e.g. random starting position), but this added randomized blinding to any BPF constants, which makes building a malicious execution path in the eBPF JIT memory much more difficult (and helps block attempts at JIT spraying to bypass other protections). Elena Reshetova updated a 2012 proof-of-concept attack to succeed against modern kernels to help provide a working example of what needed fixing in the JIT. This serves as a thorough regression test for the protection. The cBPF JITs that exist in ARM, MIPS, PowerPC, and Sparc still need to be updated to eBPF, but when they do, they ll gain all these protections immediatley. Bottom line is that if you enable the (disabled-by-default) bpf_jit_enable sysctl, be sure to set the bpf_jit_harden sysctl to 2 (to perform blinding even for root). fix brk ASLR weakness on arm64 compat There have been a few ASLR fixes recently (e.g. ET_DYN, x86 32-bit unlimited stack), and while reviewing some suggested fixes to arm64 brk ASLR code from Jon Medhurst, I noticed that arm64 s brk ASLR entropy was slightly too low (less than 1 bit) for 64-bit and noticeably lower (by 2 bits) for 32-bit compat processes when compared to native 32-bit arm. I simplified the code by using literals for the entropy. Maybe we can add a sysctl some day to control brk ASLR entropy like was done for mmap ASLR entropy. LoadPin LSM LSM stacking is well-defined since v4.2, so I finally upstreamed a small LSM that implements a protection I wrote for Chrome OS several years back. On systems with a static root of trust that extends to the filesystem level (e.g. Chrome OS s coreboot+depthcharge boot firmware chaining to dm-verity, or a system booting from read-only media), it s redundant to sign kernel modules (you ve already got the modules on read-only media: they can t change). The kernel just needs to know they re all coming from the correct location. (And this solves loading known-good firmware too, since there is no convention for signed firmware in the kernel yet.) LoadPin requires that all modules, firmware, etc come from the same mount (and assumes that the first loaded file defines which mount is correct , hence load pinning ). That s it for v4.7. Prepare yourself for v4.8 next!

2016, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.

2 October 2016

Thorsten Alteholz: My Debian Activities in September 2016

FTP assistant This month I was rather busy with other stuff and only marked 191 packages for accept and rejected 21 packages. I also sent 6 emails to maintainers asking questions. Debian LTS This was my twenty-seventh month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian. This month my all in all workload has been 12.25h. During that time I did an upload of php5 fixing 17 CVEs and two additional bugs, I uploaded mactelnet and fixed one CVE. I also prepared a package for testing of zendframework, which will fix one CVE. Unfortunately my bind9 upload needed to be postponed as Florian Weimer found an incomplete patch of a previous CVE. I am trying to fix that as well. I also had some progress with the asterisk CVEs and of course the next round of php5 patches is waiting This month I also had a few days of frontdesk work at the beginning of the month and a few days at the end. Other stuff For the Alljoyn framework I uploaded alljoyn-services-1604 and as I forgot a Conflict:, I had to take care of RC-bugs: #836717, #836718 and #836719. Thanks a lot to Ralf Treinen for his automatic installation tests. As mentioned earlier, openzwave is on its way to the Debian archive. While it is still in non-free, the author of a used library gave his permission to relicense this code, so the way to main is paved now.

Russ Allbery: Review: Winds of Change

Review: Winds of Change, by Mercedes Lackey

Series:	Mage Winds #2
Publisher:	DAW
Copyright:	August 1993
ISBN:	0-88677-563-9
Format:	Mass market
Pages:	475

Winds of Change is a direct sequel to Winds of Fate. This is a more closely connected trilogy than the previous Valdemar books. It's not the sort of thing you want to read out of order. The events of Winds of Fate predictably left the multiple protagonists united and with some breathing space, but none of their problems are resolved. The Heartstone is still a mess; in fact, it may be getting worse. Elspeth needs to learn how to wield the magical power she apparently has. And there are a lot of interpersonal tensions, lingering hurt feelings, and (in the case of Elspeth and Darkwind) a truly prodigious quantity of whining that has to be worked through before the protagonists can feel safe and happy. Winds of Change is the training montage book, and wow did my memory paper over a lot of flaws in this series. This is 475 pages of not much happening, occasionally in very irritating ways. Yes, we do finally meander to a stronger conclusion than the last book, and there is much resolving of old hurts and awkward interactions, as well as a bit of discovery of true love (this is Lackey, after all). But far, far too much of the book is Elspeth and Darkwind sniping at each other, being immature, not communicating, and otherwise being obnoxious while all the people around them try to gently help. Lackey's main characterization flaw for me is that she tends to default into generating characters who badly need to be smacked upside the head, and then does so in ways and for things at odd angles to the reasons why I think they should be smacked. It can make for frustrating reading. The introduction of Firesong as a character about halfway through this book does not help. Firesong is a flamboyant, amazingly egotistical, and stunningly arrogant show-off who also happens to be a magical super-genius and hence has "earned" his arrogance. This is an intentional character design, not my idiosyncratic reaction to the character, since every other character in the book finds him insufferable as well at first. But he's also a deeply insightful healing Adept by, honestly, authorial fiat, so by the end of the novel he's helped patch up everyone's problems and the other characters have accepted his presentation as a character quirk. Sigh. So, okay, one doesn't read popcorn fantasy for its deep characterization or realistic grasp of human subtlety. But this is just way more than I can swallow. Lackey's concept of a healing Adept (which I like a great deal as a bit of world-building) necessarily involves both deep knowledge and deep empathy and connection with other people. Firesong is so utterly full of himself that there's simply no way that he could have the empathy required to do what he is shown to do here. (Lackey does try to explain this away in the book, but the explanation didn't work for me.) Every time he successfully intervenes in other people's emotional lives, he does so with a sudden personality change, some stunning insight that he previously showed no evidence of ability to understand, and somehow only enough arrogance in his presentation to prickle but not to close people's mind to whatever he's trying to say. That's not how this works. That's not how any of this works. Lackey always treats psychology as a bit of a blunt instrument, and one either learns to tolerate that or gives up on her series, but Firesong is flatly the most unbelievable emotional mentor figure in any of her books I've read. (One of the more satisfying, if slight, bits of this series comes up in the next book, where Firesong runs into someone else who can do the same thing but has actually earned the empathy the hard way, and is a bit taken aback by it.) My other complaint with this book is that Lackey adds more chapters from the viewpoint of the big bad of the series. These are deeply unpleasant, since he's a deeply unpleasant person, and seem largely unnecessary. It's vaguely interesting to follow the magical maneuverings from both sides, but there are more of these scenes than strictly necessary for that purpose, and the sheer unmitigated evil of Lackey's evil characters is a bit hard to take. Also, he somehow has vast resources of staff and assistants, and much suspension of disbelief is required to believe that anyone would continue working for this person. It's one thing to imagine people being drawn to a charismatic Hitler type; it's quite another when the boss is a brooding, imperious asshole who roams the hallways and tortures random people to death whenever he's bored. Fear and magic only go so far in maintaining a large following when you do that, and he generates dead bodies at a remarkable rate. The best characters in this series continue to be Nyara, Need, and the gryphons. I'd rather read a book just about them. Need does use a bit too much of Lackey's tough love technique (another recurring theme of this larger series), but from Need that's wholly believable; her gruff and dubious empathy is in line with her character and history and fits a talking sword extremely well. But they, despite having a bit of their own training montage, are a side story here. The climax of the story is moderately satisfying, but the book takes far too long to get to it. I remember liking this series when I first read it, and I still like some aspects of Lackey's world-building and a few of the characters, but it's much weaker than I had remembered. I can't really recommend it. Followed by Winds of Fury. Rating: 5 out of 10

1 October 2016

Kees Cook: security things in Linux v4.6

Previously: v4.5. The v4.6 Linux kernel release included a bunch of stuff, with much more of it under the KSPP umbrella. seccomp support for parisc Helge Deller added seccomp support for parisc, which including plumbing support for PTRACE_GETREGSET to get the self-tests working. x86 32-bit mmap ASLR vs unlimited stack fixed Hector Marco-Gisbert removed a long-standing limitation to mmap ASLR on 32-bit x86, where setting an unlimited stack (e.g. ulimit -s unlimited ) would turn off mmap ASLR (which provided a way to bypass ASLR when executing setuid processes). Given that ASLR entropy can now be controlled directly (see the v4.5 post), and that the cases where this created an actual problem are very rare, means that if a system sees collisions between unlimited stack and mmap ASLR, they can just adjust the 32-bit ASLR entropy instead. x86 execute-only memory Dave Hansen added Protection Key support for future x86 CPUs and, as part of this, implemented support for execute only memory in user-space. On pkeys-supporting CPUs, using mmap(..., PROT_EXEC) (i.e. without PROT_READ) will mean that the memory can be executed but cannot be read (or written). This provides some mitigation against automated ROP gadget finding where an executable is read out of memory to find places that can be used to build a malicious execution path. Using this will require changing some linker behavior (to avoid putting data in executable areas), but seems to otherwise Just Work. I m looking forward to either emulated QEmu support or access to one of these fancy CPUs. CONFIG_DEBUG_RODATA enabled by default on arm and arm64, and mandatory on x86 Ard Biesheuvel (arm64) and I (arm) made the poorly-named CONFIG_DEBUG_RODATA enabled by default. This feature controls whether the kernel enforces proper memory protections on its own memory regions (code memory is executable and read-only, read-only data is actually read-only and non-executable, and writable data is non-executable). This protection is a fundamental security primitive for kernel self-protection, so making it on-by-default is required to start any kind of attack surface reduction within the kernel. On x86 CONFIG_DEBUG_RODATA was already enabled by default, but, at Ingo Molnar s suggestion, I made it mandatory: CONFIG_DEBUG_RODATA cannot be turned off on x86. I expect we ll get there with arm and arm64 too, but the protection is still somewhat new on these architectures, so it s reasonable to continue to leave an out for developers that find themselves tripping over it. arm64 KASLR text base offset Ard Biesheuvel reworked a ton of arm64 infrastructure to support kernel relocation and, building on that, Kernel Address Space Layout Randomization of the kernel text base offset (and module base offset). As with x86 text base KASLR, this is a probabilistic defense that raises the bar for kernel attacks where finding the KASLR offset must be added to the chain of exploits used for a successful attack. One big difference from x86 is that the entropy for the KASLR must come either from Device Tree (in the /chosen/kaslr-seed property) or from UEFI (via EFI_RNG_PROTOCOL), so if you re building arm64 devices, make sure you have a strong source of early-boot entropy that you can expose through your boot-firmware or boot-loader. zero-poison after free Laura Abbott reworked a bunch of the kernel memory management debugging code to add zeroing of freed memory, similar to PaX/Grsecurity s PAX_MEMORY_SANITIZE feature. This feature means that memory is cleared at free, wiping any sensitive data so it doesn t have an opportunity to leak in various ways (e.g. accidentally uninitialized structures or padding), and that certain types of use-after-free flaws cannot be exploited since the memory has been wiped. To take things even a step further, the poisoning can be verified at allocation time to make sure that nothing wrote to it between free and allocation (called sanity checking ), which can catch another small subset of flaws. To understand the pieces of this, it s worth describing that the kernel s higher level allocator, the page allocator (e.g. __get_free_pages()) is used by the finer-grained slab allocator (e.g. kmem_cache_alloc(), kmalloc()). Poisoning is handled separately in both allocators. The zero-poisoning happens at the page allocator level. Since the slab allocators tend to do their own allocation/freeing, their poisoning happens separately (since on slab free nothing has been freed up to the page allocator). Only limited performance tuning has been done, so the penalty is rather high at the moment, at about 9% when doing a kernel build workload. Future work will include some exclusion of frequently-freed caches (similar to PAX_MEMORY_SANITIZE), and making the options entirely CONFIG controlled (right now both CONFIGs are needed to build in the code, and a kernel command line is needed to activate it). Performing the sanity checking (mentioned above) adds another roughly 3% penalty. In the general case (and once the performance of the poisoning is improved), the security value of the sanity checking isn t worth the performance trade-off. Tests for the features can be found in lkdtm as READ_AFTER_FREE and READ_BUDDY_AFTER_FREE. If you re feeling especially paranoid and have enabled sanity-checking, WRITE_AFTER_FREE and WRITE_BUDDY_AFTER_FREE can test these as well. To perform zero-poisoning of page allocations and (currently non-zero) poisoning of slab allocations, build with:

CONFIG_DEBUG_PAGEALLOC=n
CONFIG_PAGE_POISONING=y
CONFIG_PAGE_POISONING_NO_SANITY=y
CONFIG_PAGE_POISONING_ZERO=y
CONFIG_SLUB_DEBUG=y

and enable the page allocator poisoning and slab allocator poisoning at boot with this on the kernel command line:

page_poison=on slub_debug=P

To add sanity-checking, change PAGE_POISONING_NO_SANITY=n, and add F to slub_debug as slub_debug=PF . read-only after init I added the infrastructure to support making certain kernel memory read-only after kernel initialization (inspired by a small part of PaX/Grsecurity s KERNEXEC functionality). The goal is to continue to reduce the attack surface within the kernel by making even more of the memory, especially function pointer tables, read-only (which depends on CONFIG_DEBUG_RODATA above). Function pointer tables (and similar structures) are frequently targeted by attackers when redirecting execution. While many are already declared const in the kernel source code, making them read-only (and therefore unavailable to attackers) for their entire lifetime, there is a class of variables that get initialized during kernel (and module) start-up (i.e. written to during functions that are marked __init ) and then never (intentionally) written to again. Some examples are things like the VDSO, vector tables, arch-specific callbacks, etc. As it turns out, most architectures with kernel memory protection already delay making their data read-only until after __init (see mark_rodata_ro()), so it s trivial to declare a new data section ( .data..ro_after_init ) and add it to the existing read-only data section ( .rodata ). Kernel structures can be annotated with the new section (via the __ro_after_init macro), and they ll become read-only once boot has finished. The next step for attack surface reduction infrastructure will be to create a kernel memory region that is passively read-only, but can be made temporarily writable (by a single un-preemptable CPU), for storing sensitive structures that are written to only very rarely. Once this is done, much more of the kernel s attack surface can be made read-only for the majority of its lifetime. As people identify places where __ro_after_init can be used, we can grow the protection. A good place to start is to look through the PaX/Grsecurity patch to find uses of __read_only on variables that are only written to during __init functions. The rest are places that will need the temporarily-writable infrastructure (PaX/Grsecurity uses pax_open_kernel()/pax_close_kernel() for these). That s it for v4.6, next up will be v4.7!

2016, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.

Russ Allbery: Review: Sailing to Sarantium

Review: Sailing to Sarantium, by Guy Gavriel Kay

Series:	Sarantine Mosaic #1
Publisher:	HarperPrism
Copyright:	1998
Printing:	January 2000
ISBN:	0-06-105990-0
Format:	Mass market
Pages:	533

Crispin is a well-respected mosaicist, a Rhodian, heir to a long tradition of artistic accomplishment. Rhodias is largely in ruins, and the kings that fight over its land are far removed from the emperors of Rhodias at its height, but there's still demand for skilled craftsmanship. Crispin has a partner, assistants, and the opportunity to do at least adequate work. It's not a bad life, or at least wasn't before plague took his wife and daughters while he watched, helpless. Now, he's closed off, angry, and consumed by fatalism and futility. His partner, Martinian, could never say why, when the imperious and obnoxious courier came from the far-off and half-legendary of Sarantium to summon him to assist with the rebuilding of the Sanctuary of Jad, he pretended not to be himself and pointed the courier at Crispin instead. Nor was Crispin entirely sure why he played along, although the courier was so arrogant and officious that he invited such treatment. He had no intention of going in Martinian's place and name until his friends intervened, united in their conviction that he needed something to focus on and a reason to live. But nothing about Sarantium is simple. Even before Crispin leaves, the local queen, precariously balancing between warring factions, reveals her own reason to send Crispin to the capital of the far-off empire: using him as a desperate messenger to propose an alliance through marriage. And when he finally arrives, after unexpected perils on the road, he tumbles immediately into the deep complexities of the Sarantine court and its remarkable rulers. Most of Guy Gavriel Kay's work is historical fiction with the serial numbers partly filed off and some magic added. The degree to which the serial numbers are removed varies by work; here, the history is obvious. Sarantium is Byzantium, fallen Rhodias is Rome, and the emperor Valerius and his former dancer empress Alixana are clearly Justinian and Theodora. Knowing that will provide a lot of context, but can also be distracting, since the temptation to scramble for Wikipedia and line things up with real history is strong. I read this book the first time without knowing the history and this second time knowing it. I think I enjoyed it a bit more on its own terms, instead of as a reflection and reinterpretation. Either way, though, it's one of Kay's best novels, which is saying quite a bit. Sailing to Sarantium has two very different parts: Crispin's journey to Sarantium, and then what happens when he arrives. The first is a fairly linear travel narrative mixed with old religion, magic, and an experience in the woods that's a little disconnected from the rest of the story. The second is the fast-entangling politics and factionalism of the city itself, which wastes no time pulling Crispin into meetings with the most influential people at court. In my memory, I liked the Sarantium narrative the best, and found the travel story less significant. Re-reading, I'm not sure I agree with my earlier self. Kay does write some of the best conversations in fiction; if you want to see careful verbal maneuvering or gut reactions with wide-ranging consequences, I can't think of any author who shows both the words and the nuances as well as Kay. He makes the court maneuvering feel truly epic. Watching Crispin's blunt competence cut through the Sarantine court, and seeing him stay focused on his life work despite all those distractions, is a truly rewarding experience. But that second part of the book also has some structural problems, and a few characterization problems. The structural problem is that Kay wants to tell the same story from multiple angles, viewpoints, and timelines to show the reader every implication of the hornet's nest that Crispin kicks over, since quite a lot happens in the course of one day and night. This mostly works, but it is so laden with flashbacks, foreshadowing, and rewinds that I started losing track of the time relationship between the scenes. Kay is a very skilled author, so he avoids confusing the reader entirely, but I think he still tried too complex of a temporal structure here. The characterization problem is that I previously wasn't paying as close of attention to Kay's portrayal of women and of male sexual reactions to women. It didn't bother me before; this time, it started getting on my nerves. Yes, this is historical fiction (although limitations on roles for women in history is rather overstated in fiction), and to give Kay credit it does feature a very strong female character in Alixana. But the male protector and female seducer dynamic is very strong, and all the women in this book seem to get shoehorned into it. And perhaps I'm missing some weakness that other men have, but I have never had the sort of overwhelming, thought-destroying reaction to the presence of a seductive woman as Kay's men seem to routinely have in this book. To give Crispin credit, he maneuvers his way through those conversations without doing anything stupid. (Kay's characters very rarely do stupid things, which is refreshing.) But the effort required started undermining my suspension of disbelief. Even making allowances for a culture that might consider a woman with her hair down the equivalent of an entirely nude woman, I simply don't believe that a man who had been married with two daughters, had a long career, and had the life experience Crispin had would have that much difficulty with an aggressively seductive woman that he already doesn't trust. As a result, as much as I love Kay's political conversational fencing, I enjoyed the travel portion of the book more on this re-read. Crispin is a type of character Kay writes extremely well: an honest, honorable man trying to navigate unfamiliar waters without a lot of context, but with a firm sense of internal morality, a lot of natural intelligence, and a true passion for something. At one point in this book, another character decides to walk away from his current job and follow Crispin wherever he goes. Kay makes you believe that and want to do it yourself. It's a very rewarding experience as a reader to watch a character you can trust find careful, courageous, and tricky solutions to complicated problems. But I think the best part of this book is Kay's ability to put Crispin's art, the creation of mosaics, on the page in a way that lets the reader viscerally appreciate it. Crispin is passionate about mosaics, about how to make them and how to think about them, and the passion is contagious and forms the heart and soul of this novel. Kay manages to make mosaics overshadow some of the most dramatic politics in history, which says quite a lot. There is almost nothing I love more than reading about talented, passionate people doing things they are extremely good at. I know nothing about mosaic that I haven't read in this book and its sequel, so I have no basis from which to judge the accuracy of Kay's portrayal, but he made me feel like I could appreciate some of the nuance, skill, and design constraints of the art form. It's a wonderful reading experience. Sailing to Sarantium is best thought of half of a long book that was divided for publication reasons, so don't expect much of an ending here. It can only be read in combination with Lord of Emperors, which you'll want to have on hand when you finish this book so that you can continue the story. Followed by Lord of Emperors. Rating: 9 out of 10

30 September 2016

Chris Lamb: Free software activities in September 2016

Here is my monthly update covering what I have been doing in the free software world (previous month):

Fixed a number of instances in the Django web development framework where methods had mutable defaults arguments such as lists or dictionaries. (#7253)
Made a large number of improvements to travis.debian.net, my hosted service for projects that host their Debian packaging on GitHub to use the Travis CI continuous integration platform to test builds on every update:
- Worked with Evgeni Golov to run autopkgtest and autodep8 tests after builds. (#23, #24 & #25)
- Ensured that all remote branches are created locally so that packages that specify options such as pristine_tar=True in debian/gbp.conf work as expected. (commit)
- Fixed an issue where checking out other branches did not reliably not work due to Travis cloning the repository with --depth=50. (#21)
- Updated the documentation after prompting from Valerie Young. (db7238 & 470d24)
Improved my Chrome extension for the FastMail web interface to collapse whitespace when appending to the subject. (#9)
Submitted a pull request for the Handbrake video transcoder to make the build reproducible. (#320)
Updated the configuration for my OpenWRT-based router (a device I use as a range extender and to save tedious reconfiguration of all my wifi devices in new locales) to enter a "fallback" mode if it cannot connect to the client network. This works around a longstanding "wontfix" upstream issue. (diff)
Updated django-agpl, my reusable Django application to assist in releasing projects under the GNU Affero General Public License:
- Merged support from defivelo to support Django 1.8. (#1)
- Improved the code quality & testability throughout the package. (diff)
Filed in issue in Mailvelope (a browser extension for OpenPGP encryption) to support composing/viewing messages in a fixed-width font. (#422)
Blogged about how to write your first Lintian check as well as on the progress bar I added to Diffoscope.

Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most Linux distributions provide binary (or "compiled") packages to end users. The motivation behind the Reproducible Builds effort is to allow verification that no flaws have been introduced either maliciously and accidentally during this compilation process by promising identical binary packages are always generated from a given source. My work in the Reproducible Builds project was also covered in our weekly reports #71, #72, #71 & #74. I made the following improvements to our tools:

diffoscope

diffoscope is our "diff on steroids" that will not only recursively unpack archives but will transform binary formats into human-readable forms in order to compare them.

Added a global Progress object to track the status of the comparison process allowing for graphical and machine-readable status indicators. I also blogged about this feature in more detail.
Moved the global Config object to a more Pythonic "singleton" pattern and ensured that constraints are checked on every change.

disorderfs

disorderfs is our FUSE filesystem that deliberately introduces nondeterminism into the results of system calls such as readdir(3).

Display the "disordered" behaviour we intend to show on startup. (#837689)
Support relative paths in command-line parameters (previously only absolute paths were permitted).

strip-nondeterminism

strip-nondeterminism is our tool to remove specific information from a completed build.

Fix an issue where temporary files were being left on the filesystem and add a test to avoid similar issues in future. (#836670)
Print an error if the file to normalise does not exist. (#800159)
Testsuite improvements:
- Set the timezone in tests to avoid a FTBFS and add a File::StripNondeterminism::init method to the API to to set tzset everywhere. (#837382)
- "Smoke test" the strip-nondeterminism(1) and dh_strip_nondeterminism(1) scripts to prevent syntax regressions.
- Add a testcase for .jar file ordering and normalisation.
- Check the stripping process before comparing file attributes to make it less confusing on failure.
- Move to a lookup table for descriptions of stat(1) indices and use that for nicer failure messages.
- Don't uselessly test whether the inode number has changed.
Run perlcritic across the codebase and adopt some of its prescriptions including explicitly using oct(..) for integers with leading zeroes, avoiding mixing high and low-precedence booleans, ensuring subroutines end with a return statement, etc.

I also submitted 4 patches to fix specific reproducibility issues in golang-google-grpc, nostalgy, python-xlib & torque.

Debian

https://lamby-www.s3.amazonaws.com/yadt/blog.Image/image/original/28.jpeg

I attended the Debian Bug Squashing Party (BSP) in Salzburg, Austria where 70+ "release-critical" bugs were fixed. Sincere thanks to Bernd Zeimetz (bzed) for organising and Conova for sponsoring/hosting. The event was covered by the Salzburg Cityguide.
I attended NixOS user group in London, England.
Niels Thykier granted me commit access to the Lintian Git repository and I added myself to the debian/copyright there.
Filed an ITP for the roughtime secure time synchronisation client and server. This is blocked on packaging the Bazel build system. (#838416)

Patches contributed

Lintian:
- Tag debian/*.maintscript files that include "maint-script-parameters". (#838195.)
- Tag initscripts that use /lib/lsb/init-functions without the corresponding dependency on lsb-base. (#838997)
- Catch false positives in systemd-service-file-missing-install-key check. (#837057)
rtl-sdr: Correct invocation of rm_conffile
dh-haskell: Experimental push on scalar is now forbidden
fortunes-es: Correct reference to $msg_installing_link
webkit2pdf: FTBFS with dash as /bin/sh
tj3: Cannot remove 'data/tjp.vim': No such file or directory
greylistd: Fails to install in testing
witty: Drop the build dependency on hardening-wrapper
ruby-em-hiredis: Gem::LoadError: Could not find hiredis
highlight.js: File iftex.sty' not found
transmission-remote-gtk: Option release requires an argument

Debian LTS

This month I have been paid to work 12.75 hours on Debian Long Term Support (LTS). In that time I did the following:

"Frontdesk" duties, triaging CVEs, etc.
Issued DLA 608-1 for mailman fixing a CSRF vulnerability.
Issued DLA 611-1 for jsch correcting a path traversal vulnerability.
Issued DLA 620-1 for libphp-adodb patching a SQL injection vulnerability.
Issued DLA 631-1 for unadf correcting a buffer underflow issue.
Issued DLA 634-1 for dropbear fixing a buffer overflow when parsing ASN.1 keys.
Issued DLA 635-1 for dwarfutils working around an out-of-bounds read issue.
Issued DLA 638-1 for the SELinux policycoreutils, patching a sandbox escape issue.
Enhanced Brian May's find-work --unassigned switch to take an optional "except this user" argument.
Marked matrixssl and inspircd as being unsupported in the current LTS version.

Uploads

python-django 1:1.10.1-1 New upstream release and ensure that django-admin startproject foo creates files with the correct shebang under Python 3.
gunicorn:
- 19.6.0-5 Don't call chown(2) if it would be a no-op to avoid failure under snap.
- 19.6.0-6 Remove now-obsolete conffiles and logrotate scripts; they should have been removed in 19.6.0-3.
redis:
- 3.2.3-2 Call ulimit -n 65536 by default from SysVinit scripts to normalise the behaviour with systemd. I also bumped the Debian package epoch as the "2:" prefix made it look like we are shipping version 2.x. I additionaly backported this upload to Debian Jessie.
- 3.2.4-1 New upstream release, add missing -ldl for dladdr(3) & add missing dependency on lsb-base.
python-redis (2.10.5-2) Bump python-hiredis to Suggests to sync with Ubuntu and move to a machine-readable debian/copyright. I also backported this upload to Debian Jessie.
adminer (4.2.5-3) Move mysql-server dependencies to default-mysql-server. I also backported this upload to Debian Jessie.
gpsmanshp (1.2.3-5) on behalf of the QA team:
- Move to "minimal" debhelper style, making the build reproducible. (#777446 & #792991)
- Reorder linker command options to build with --as-needed (#729726) and add hardening flags.
- Move to machine-readable copyright file, add missing #DEBHELPER# tokens to postinst and prerm scripts, tidy descriptions & other debian/control fields and other smaller changes.

I sponsored the upload of 5 packages from other developers:

stressapptest (1.0.6-3) Make the build reproducible. (#831587)
zeroinstall-injector (2.12-1) Fix FTBFS due to failing tests. (#837226)
libisofs, libburn & libisoburn (1.4.6-1) New upstream releases.

I also NMU'd:

cfingerd (1.4.3-3.2) Fix a FTBFS due to lack of "." in Perl's @INC. (#837267)
dot-forward (0.71-2.1) Make the build reproducible. (#776760)
8 packages to remove dependencies on hardening-wrapper and hardening-includes: switchsh, pwauth, grap, libmsn, libevocosm, passwordmaker-cli, srtp & netatalk.
12 packages that were using now-unsupported debhelper levels: freetable, alsamixergui, linklint, openjade, python-cddb, dh-kpatches, dsh, jfsuils, lpr, myspell & libxml-dumper-perl.

RC bugs I filed 37 FTBFS bugs against csoundqt, cups-filters, dymo-cups-drivers, easytag, erlang-p1-oauth2, erlang-p1-sqlite3, erlang-p1-xmlrpc, erlang-redis-client, fso-datad, gnome-python-desktop, gnote, gstreamermm-1.0, gtkglextmm, gupnp-dlna, haskell-hmatrix-gsl, jdeb, kryo-serializers, libcmrt, libfso-glib, libmonitoring-livestatus-perl, librasterlite2, network-manager, print-manager, psychtoolbox-3, python-3to2, python-tidylib, recutils, slang2, snd, sugar, tj3, transmission-remote-gtk, vino, webkit2pdf, xml-core, xml-core & xml-core. I additionally filed 2 "important" bugs for packages that access the internet during build against gnupg2 & libgdata.

FTP Team

As a Debian FTP assistant I ACCEPTed 147 packages: alljoyn-services-1604, android-platform-external-doclava, android-platform-system-tools-aidl, aufs, bcolz, binwalk, bmusb, bruteforce-salted-openssl, cappuccino, captagent, chrome-gnome-shell, ciphersaber, cmark, colorfultabs, cppformat, dnsrecon, dogtag-pki, dxtool, e2guardian, flask-compress, fonts-mononoki, fwknop-gui, gajim-httpupload, glbinding, glewmx, gnome-2048, golang-github-googleapis-proto-client-go, google-android-installers, gsl, haskell-hmatrix-gsl, haskell-relational-query, haskell-relational-schemas, haskell-secret-sharing, hindsight, i8c, ip4r, java-string-similarity, khal, khronos-opencl-headers, liblivemedia, libshell-config-generate-perl, libshell-guess-perl, libstaroffice, libxml2, libzonemaster-perl, linux, linux-grsec-base, linux-signed, lua-sandbox, lua-torch-trepl, mbrola-br2, mbrola-br4, mbrola-de1, mbrola-de2, mbrola-de3, mbrola-ir1, mbrola-lt1, mbrola-lt2, mbrola-mx1, mimeo, mimerender, mongo-tools, mozilla-gnome-keyring, munin, node-grunt-cli, node-js-yaml, nova, open-build-service, openzwave, orafce, osmalchemy, pgespresso, pgextwlist, pgfincore, pgmemcache, pgpool2, pgsql-asn1oid, postbooks-schema, postgis, postgresql-debversion, postgresql-multicorn, postgresql-mysql-fdw, postgresql-unit, powerline-taskwarrior, prefix, pycares, pydl, pynliner, pytango, pytest-cookies, python-adal, python-applicationinsights, python-async-timeout, python-azure, python-azure-storage, python-blosc, python-can, python-canmatrix, python-chartkick, python-confluent-kafka, python-jellyfish, python-k8sclient, python-msrestazure, python-nss, python-pytest-benchmark, python-tenacity, python-tmdbsimple, python-typing, python-unidiff, python-xstatic-angular-schema-form, python-xstatic-tv4, quilt, r-bioc-phyloseq, r-cran-filehash, r-cran-png, r-cran-testit, r-cran-tikzdevice, rainbow-mode, repmgr, restart-emacs, restbed, ruby-azure-sdk, ruby-babel-source, ruby-babel-transpiler, ruby-diaspora-prosody-config, ruby-haikunator, ruby-license-finder, ruby-ms-rest, ruby-ms-rest-azure, ruby-rails-assets-autosize, ruby-rails-assets-blueimp-gallery, ruby-rails-assets-bootstrap, ruby-rails-assets-bootstrap-markdown, ruby-rails-assets-emojione, ruby-sprockets-es6, ruby-timeliness, rustc, skytools3, slony1-2, snmp-mibs-downloader, syslog-ng, test-kitchen, uctodata, usbguard, vagrant-azure, vagrant-mutate & vim.

26 September 2016

Kees Cook: security things in Linux v4.3

When I gave my State of the Kernel Self-Protection Project presentation at the 2016 Linux Security Summit, I included some slides covering some quick bullet points on things I found of interest in recent Linux kernel releases. Since there wasn t a lot of time to talk about them all, I figured I d make some short blog posts here about the stuff I was paying attention to, along with links to more information. This certainly isn t everything security-related or generally of interest, but they re the things I thought needed to be pointed out. If there s something security-related you think I should cover from v4.3, please mention it in the comments. I m sure I haven t caught everything. :) A note on timing and context: the momentum for starting the Kernel Self Protection Project got rolling well before it was officially announced on November 5th last year. To that end, I included stuff from v4.3 (which was developed in the months leading up to November) under the umbrella of the project, since the goals of KSPP aren t unique to the project nor must the goals be met by people that are explicitly participating in it. Additionally, not everything I think worth mentioning here technically falls under the kernel self-protection ideal anyway some things are just really interesting userspace-facing features. So, to that end, here are things I found interesting in v4.3: CONFIG_CPU_SW_DOMAIN_PAN Russell King implemented this feature for ARM which provides emulated segregation of user-space memory when running in kernel mode, by using the ARM Domain access control feature. This is similar to a combination of Privileged eXecute Never (PXN, in later ARMv7 CPUs) and Privileged Access Never (PAN, coming in future ARMv8.1 CPUs): the kernel cannot execute user-space memory, and cannot read/write user-space memory unless it was explicitly prepared to do so. This stops a huge set of common kernel exploitation methods, where either a malicious executable payload has been built in user-space memory and the kernel was redirected to run it, or where malicious data structures have been built in user-space memory and the kernel was tricked into dereferencing the memory, ultimately leading to a redirection of execution flow. This raises the bar for attackers since they can no longer trivially build code or structures in user-space where they control the memory layout, locations, etc. Instead, an attacker must find areas in kernel memory that are writable (and in the case of code, executable), where they can discover the location as well. For an attacker, there are vastly fewer places where this is possible in kernel memory as opposed to user-space memory. And as we continue to reduce the attack surface of the kernel, these opportunities will continue to shrink. While hardware support for this kind of segregation exists in s390 (natively separate memory spaces), ARM (PXN and PAN as mentioned above), and very recent x86 (SMEP since Ivy-Bridge, SMAP since Skylake), ARM is the first upstream architecture to provide this emulation for existing hardware. Everyone running ARMv7 CPUs with this kernel feature enabled suddenly gains the protection. Similar emulation protections (PAX_MEMORY_UDEREF) have been available in PaX/Grsecurity for a while, and I m delighted to see a form of this land in upstream finally. To test this kernel protection, the ACCESS_USERSPACE and EXEC_USERSPACE triggers for lkdtm have existed since Linux v3.13, when they were introduced in anticipation of the x86 SMEP and SMAP features. Ambient Capabilities Andy Lutomirski (with Christoph Lameter and Serge Hallyn) implemented a way for processes to pass capabilities across exec() in a sensible manner. Until Ambient Capabilities, any capabilities available to a process would only be passed to a child process if the new executable was correctly marked with filesystem capability bits. This turns out to be a real headache for anyone trying to build an even marginally complex least privilege execution environment. The case that Chrome OS ran into was having a network service daemon responsible for calling out to helper tools that would perform various networking operations. Keeping the daemon not running as root and retaining the needed capabilities in children required conflicting or crazy filesystem capabilities organized across all the binaries in the expected tree of privileged processes. (For example you may need to set filesystem capabilities on bash!) By being able to explicitly pass capabilities at runtime (instead of based on filesystem markings), this becomes much easier. For more details, the commit message is well-written, almost twice as long as than the code changes, and contains a test case. If that isn t enough, there is a self-test available in tools/testing/selftests/capabilities/ too. PowerPC and Tile support for seccomp filter Michael Ellerman added support for seccomp to PowerPC, and Chris Metcalf added support to Tile. As the seccomp maintainer, I get excited when an architecture adds support, so here we are with two. Also included were updates to the seccomp self-tests (in tools/testing/selftests/seccomp), to help make sure everything continues working correctly. That s it for v4.3. If I missed stuff you found interesting, please let me know! I m going to try to get more per-version posts out in time to catch up to v4.8, which appears to be tentatively scheduled for release this coming weekend.

2016, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.

Reproducible builds folks: Reproducible Builds: week 74 in Stretch cycle

Here is what happened in the Reproducible Builds effort between Sunday September 18 and Saturday September 24 2016: Outreachy We intend to participate in Outreachy Round 13 and look forward for new enthusiastic applications to contribute to reproducible builds. We're offering four different areas to work on:

Improve test and debugging tools.
Improving reproducibility of Debian packages.
Improving Debian infrastructure.
Help collaboration across distributions.

Reproducible Builds World summit #2 We are planning e a similar event to our Athens 2015 summit and expect to reveal more information soon. If you haven't been contacted yet but would like to attend, please contact holger. Toolchain development and fixes Mattia uploaded dpkg/1.18.10.0~reproducible1 to our experimental repository. and covered the details for the upload in a mailing list post. The most important change is the incorporation of improvements made by Guillem Jover (dpkg maintainer) to the .buildinfo generator. This is also in the hope that it will speed up the merge in the upstream. One of the other relevant changes from before is that .buildinfo files generated from binary-only builds will no longer include the hash of the .dsc file in Checksums-Sha256 as documented in the specification. Even if it was considered important to include a checksum of the source package in .buildinfo, storing it that way breaks other assumptions (eg. that Checksums-Sha256 contains only files part of that are part of a single upload, wheras the .dsc might not be part of that upload), thus we look forward for another solution to store the source checksum in .buildinfo. Bugs filed

#838713 filed against python-xlib by Chris Lamb.
#838754 filed against golang-google-grpc by Chris Lamb.
#838188 filed against ocaml by Johannes Schauer.
#838785 filed against funnelweb by Reiner Herrmann.

Reviews of unreproducible packages 250 package reviews have been added, 4 have been updated and 4 have been removed in this week, adding to our knowledge about identified issues. 4 issue types have been added:

3 issue types have been updated:

Weekly QA work FTBFS bugs have been reported by:

Chris Lamb (11)
Santiago Vila (2)

Documentation updates h01ger created a new Jenkins job so that every commit pushed to the master branch for the website will update reproducible-builds.org. diffoscope development

strip-nondeterminism development

Chris Lamb:

reprotest development

Ximin Luo uploaded reprotest 0.3 and 0.3.1 to unstable with these changes:

tests.reproducible-builds.org

The full rebuild of all packages in unstable (for all tested archs) with the new build path variation has been completed. This has had the result that we are down to ~75% reproducible packages in unstable now. In comparison, for testing (where we don't vary the build path) we are still at ~90%. IRC notifications for unstable have been enabled again. (Holger)
Make the notes job robust about bad data (see #833695 and #833738). (Holger)
Setup profitbricks-build7 running stretch as testing reproducible builds of F-Droid need to use a newer version of vagrant in order to support running vagrant VMs with kvm on kvm. (Holger)
The misbehaving 'opi2a' armhf node has been replaced with a Jetson-TK1 board kindly donated by NVidia. This machine is using an NVIDIA tegra-k1 (cortex-a15) quad-core board. (vagrant and Holger)

Misc. This week's edition was written by Chris Lamb, Holger Levsen and Mattia Rizzolo and reviewed by a bunch of Reproducible Builds folks on IRC.

25 September 2016

Julian Andres Klode: Introducing TrieHash, a order-preserving minimal perfect hash function generator for C(++)

Abstract I introduce TrieHash an algorithm for constructing perfect hash functions from tries. The generated hash functions are pure C code, minimal, order-preserving and outperform existing alternatives. Together with the generated header files,they can also be used as a generic string to enumeration mapper (enums are created by the tool). Introduction APT (and dpkg) spend a lot of time in parsing various files, especially Packages files. APT currently uses a function called AlphaHash which hashes the last 8 bytes of a word in a case-insensitive manner to hash fields in those files (dpkg just compares strings in an array of structs). There is one obvious drawback to using a normal hash function: When we want to access the data in the hash table, we have to hash the key again, causing us to hash every accessed key at least twice. It turned out that this affects something like 5 to 10% of the cache generation performance. Enter perfect hash functions: A perfect hash function matches a set of words to constant values without collisions. You can thus just use the index to index into your hash table directly, and do not have to hash again (if you generate the function at compile time and store key constants) or handle collision resolution. As #debian-apt people know, I happened to play a bit around with tries this week before guillem suggested perfect hashing. Let me tell you one thing: My trie implementation was very naive, that did not really improve things a lot Enter TrieHash Now, how is this related to hashing? The answer is simple: I wrote a perfect hash function generator that is based on tries. You give it a list of words, it puts them in a trie, and generates C code out of it, using recursive switch statements (see code generation below). The function achieves competitive performance with other hash functions, it even usually outperforms them. Given a dictionary, it generates an enumeration (a C enum or C++ enum class) of all words in the dictionary, with the values corresponding to the order in the dictionary (the order-preserving property), and a function mapping strings to members of that enumeration. By default, the first word is considered to be 0 and each word increases a counter by one (that is, it generates a minimal hash function). You can tweak that however:

= 0
WordLabel ~ Word
OtherWord = 9

will return 0 for an unknown value, map Word to the enum member WordLabel and map OtherWord to 9. That is, the input list functions like the body of a C enumeration. If no label is specified for a word, it will be generated from the word. For more details see the documentation C code generation

switch(string[0]   32)  
case 't':
    switch(string[1]   32)  
    case 'a':
        switch(string[2]   32)  
        case 'g':
            return Tag;
         
     
 
return Unknown;

Yes, really recursive switches they directly represent the trie. Now, we did not really do a straightforward translation, there are some optimisations to make the whole thing faster and easier to look at: First of all, the 32 you see is used to make the check case insensitive in case all cases of the switch body are alphabetical characters. If there are non-alphabetical characters, it will generate two cases per character, one upper case and one lowercase (with one break in it). I did not know that lowercase and uppercase characters differed by only one bit before, thanks to the clang compiler for pointing that out in its generated assembler code! Secondly, we only insert breaks only between cases. Initially, each case ended with a return Unknown, but guillem (the dpkg developer) suggested it might be faster to let them fallthrough where possible. Turns out it was not faster on a good compiler, but it s still more readable anywhere. Finally, we build one trie per word length, and switch by the word length first. Like the 32 trick, his gives a huge improvement in performance. Digging into the assembler code The whole code translates to roughly 4 instructions per byte:

A memory load,
an or with 32
a comparison, and
a conditional jump.

(On x86, the case sensitive version actually only has a cmp-with-memory and a conditional jump). Due to https://gcc.gnu.org/bugzilla/show_bug.cgi?id=77729 this may be one instruction more: On some architectures an unneeded zero-extend-byte instruction is inserted this causes a 20% performance loss. Performance evaluation I run the hash against all 82 words understood by APT in Packages and Sources files, 1,000,000 times for each word, and summed up the average run-time:

host	arch	Trie	TrieCase	GPerfCase	GPerf	DJB
plummer	ppc64el	540	601	1914	2000	1345
eller	mipsel	4728	5255	12018	7837	4087
asachi	arm64	1000	1603	4333	2401	1625
asachi	armhf	1230	1350	5593	5002	1784
barriere	amd64	689	950	3218	1982	1776
x230	amd64	465	504	1200	837	693

Suffice to say, GPerf does not really come close. All hosts except the x230 are Debian porterboxes. The x230 is my laptop with a a Core i5-3320M, barriere has an Opteron 23xx. I included the DJB hash function for another reference. Source code The generator is written in Perl, licensed under the MIT license and available from https://github.com/julian-klode/triehash I initially prototyped it in Python, but guillem complained that this would add new build dependencies to dpkg, so I rewrote it in Perl. Benchmark is available from https://github.com/julian-klode/hashbench Usage See the script for POD documentation.
Filed under: General

Steinar H. Gunderson: Nageru @ Fyrrom

When Samfundet wanted to make their own Boiler Room spinoff (called Fyrrom more or less a direct translation), it was a great opportunity to try out the new multitrack code in Nageru. After all, what can go wrong with a pretty much untested and unfinished git branch, right? So we cobbled together a bunch of random equipment from here and there: Video equipment

Hooked it up to Nageru:

and together with some great work from the people actually pulling together the event, this was the result. Lots of fun. And yes, some bugs were discovered of course, field testing without followup patches is meaningless (that would either mean you're not actually taking your test experience into account, or that your testing gave no actionable feedback and thus was useless), so they will be fixed in due time for the 1.4.0 release. Edit: Fixed a screenshot link.

22 September 2016

Gustavo Noronha Silva: WebKitGTK+ 2.14 and the Web Engines Hackfest

Next week our friends at Igalia will be hosting this year s Web Engines Hackfest. Collabora will be there! We are gold sponsors, and have three developers attending. It will also be an opportunity to celebrate Igalia s 15th birthday \o/. Looking forward to meet you there! =) Carlos Garcia has recently released WebKitGTK+ 2.14, the latest stable release. This is a great release that brings a lot of improvements and works much better on Wayland, which is becoming mature enough to be used by default. In particular, it fixes the clipboard, which was one of the main missing features, thanks to Carlos Garnacho! We have also been able to contribute a bit to this release =) One of the biggest changes this cycle is the threaded compositor, which was implemented by Igalia s Gwang Yoon Hwang. This work improves performance by not stalling other web engine features while compositing. Earlier this year we contributed fixes to make the threaded compositor work with the web inspector and fixed elements, helping with the goal of enabling it by default for this release. Wayland was also lacking an accelerated compositing implementation. There was a patch to add a nested Wayland compositor to the UIProcess, with the WebProcesses connecting to it as Wayland clients to share the final rendering so that it can be shown to screen. It was not ready though and there were questions as to whether that was the way to go and alternative proposals were floating around on how to best implement it. At last year s hackfest we had discussions about what the best path for that would be where collaborans Emanuele Aina and Daniel Stone (proxied by Emanuele) contributed quite a bit on figuring out how to implement it in a way that was both efficient and platform agnostic. We later picked up the old patchset, rebased on the then-current master and made it run efficiently as proof of concept for the Apertis project on an i.MX6 board. This was done using the fancy GL support that landed in GTK+ in the meantime, with some API additions and shortcuts to sidestep performance issues. The work was sponsored by Robert Bosch Car Multimedia. Igalia managed to improve and land a very well designed patch that implements the nested compositor, though it was still not as efficient as it could be, as it was using glReadPixels to get the final rendering of the page to the GTK+ widget through cairo. I have improved that code by ensuring we do not waste memory when using HiDPI. As part of our proof of concept investigation, we got this WebGL car visualizer running quite well on our sabrelite imx6 boards. Some of it went into the upstream patches or proposals mentioned below, but we have a bunch of potential improvements still in store that we hope to turn into upstreamable patches and advance during next week s hackfest. One of the improvements that already landed was an alternate code path that leverages GTK+ s recent GL super powers to render using gdk_cairo_draw_from_gl(), avoiding the expensive copying of pixels from the GPU to the CPU and making it go faster. That improvement exposed a weird bug in GTK+ that causes a black patch to appear when shrinking the window, which I have a tentative fix for. We originally proposed to add a new gdk_cairo_draw_from_egl() to use an EGLImage instead of a GL texture or renderbuffer. On our proof of concept we noticed it is even more efficient than the texturing currently used by GTK+, and could give us even better performance for WebKitGTK+. Emanuele Bassi thinks it might be better to add EGLImage as another code branch inside from_gl() though, so we will look into that. Another very interesting igalian addition to this release is support for the MemoryPressureHandler even on systems with no cgroups set up. The memory pressure handler is a WebKit feature which flushes caches and frees resources that are not being used when the operating system notifies it memory is scarce. We worked with the Raspberry Pi Foundation to add support for that feature to the Raspberry Pi browser and contributed it upstream back in 2014, when Collabora was trying to squeeze as much as possible from the hardware. We had to add a cgroups setup to wrap Epiphany in, back then, so that it would actually benefit from the feature. With this improvement, it will benefit even without the custom cgroups setups as well, by having the UIProcess monitor memory usage and notify each WebProcess when memory is tight. Some of these improvements were achieved by developers getting together at the Web Engines Hackfest last year and laying out the ground work or ideas that ended up in the code base. I look forward to another great few days of hackfest next week! See you there o/

20 September 2016

Gunnar Wolf: Proposing a GR to repeal the 2005 vote for declassification of the debian-private mailing list

For the non-Debian people among my readers: The following post presents bits of the decision-taking process in the Debian project. You might find it interesting, or terribly dull and boring :-) Proceed at your own risk. My reason for posting this entry is to get more people to read the accompanying options for my proposed General Resolution (GR), and have as full a ballot as possible. Almost three weeks ago, I sent a mail to the debian-vote mailing list. I'm quoting it here in full:

Some weeks ago, Nicolas Dandrimont proposed a GR for declassifying
debian-private[1]. In the course of the following discussion, he
accepted[2] Don Armstrong's amendment[3], which intended to clarify the
meaning and implementation regarding the work of our delegates and the
powers of the DPL, and recognizing the historical value that could lie
within said list.
[1] https://www.debian.org/vote/2016/vote_002
[2] https://lists.debian.org/debian-vote/2016/07/msg00108.html
[3] https://lists.debian.org/debian-vote/2016/07/msg00078.html
In the process of the discussion, several people objected to the
amended wording, particularly to the fact that "sufficient time and
opportunity" might not be sufficiently bound and defined.
I am, as some of its initial seconders, a strong believer in Nicolas'
original proposal; repealing a GR that was never implemented in the
slightest way basically means the Debian project should stop lying,
both to itself and to the whole free software community within which
it exists, about something that would be nice but is effectively not
implementable.
While Don's proposal is a good contribution, given that in the
aforementioned GR "Further Discussion" won 134 votes against 118, I
hereby propose the following General Resolution:
=== BEGIN GR TEXT ===
Title: Acknowledge that the debian-private list will remain private.
1. The 2005 General Resolution titled "Declassification of debian-private
   list archives" is repealed.
2. In keeping with paragraph 3 of the Debian Social Contract, Debian
   Developers are strongly encouraged to use the debian-private mailing
   list only for discussions that should not be disclosed.
=== END GR TEXT ===
Thanks for your consideration,
--
Gunnar Wolf
(with thanks to Nicolas for writing the entirety of the GR text ;-) )

Yesterday, I spoke with the Debian project secretary, who confirmed my proposal has reached enough Seconds (that is, we have reached five people wanting the vote to happen), so I could now formally do a call for votes. Thing is, there are two other proposals I feel are interesting, and should be part of the same ballot, and both address part of the reasons why the GR initially proposed by Nicolas didn't succeed:

Ian Jackson's Acknowledge difficulty of declassifying debian-private makes explicit the role of the listmasters and allows for a formal declassification process to take place, as long as the privacy guarantees we had after the 2005 GR are not diminished.
Iain Lane's reply to Ian is not yet formally proposed, but makes it spelt out that no declassification should ever occur unless all of the involved authors have explicitly consented

So, once more (and finally!), why am I posting this?

To invite Iain to formally propose his text as an option to mine
To invite more DDs to second the available options
To publicize the ongoing discussion

~~I plan to do the formal call for votes by Friday 23.~~
[update] Kurt informed me that the discussion period started yesterday, when I received the 5th second. The minimum discussion period is two weeks, so I will be doing a call for votes at or after 2016-10-03.

Reproducible builds folks: Reproducible Builds: week 73 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday September 11 and Saturday September 17 2016: Toolchain developments Ximin Luo started a new series of tools called (for now) debrepatch, to make it easier to automate checks that our old patches to Debian packages still apply to newer versions of those packages, and still make these reproducible. Ximin Luo updated one of our few remaining patches for dpkg in #787980 to make it cleaner and more minimal. The following tools were fixed to produce reproducible output:

naturaldocs/1.51-2 by Petter Reinholdtsen, original patch by Chris Lamb.

Packages reviewed and fixed, and bugs filed The following updated packages have become reproducible - in our current test setup - after being fixed:

elog/3.1.2-1-1 by Roger Kalt, original patch by Reiner Herrmann.
eyed3/0.6.18-3 by Petter Reinholdtsen, original patch by Chris Lamb.
frog/0.13.5-1 by Maarten van Gompel, original patch by Chris Lamb.
gtranslator/2.91.7-3 by Andreas Henriksson, original patch by Reiner Herrmann.
sozi/12.05-1.1 by Daniel Kahn Gillmor, original patch by Chris Lamb.

The following updated packages appear to be reproducible now, for reasons we were not able to figure out. (Relevant changelogs did not mention reproducible builds.)

evince/3.21.92-1 by Michael Biebl.
gnome-control-center/1:3.21.92-2 by Rapha l Hertzog.
libipathverbs/1.3-2 by Ana Beatriz Guerrero Lopez.
pagekite/0.5.8e-2 by Petter Reinholdtsen.

The following 3 packages were not changed, but have become reproducible due to changes in their build-dependencies: jaxrs-api python-lua zope-mysqlda. Some uploads have addressed some reproducibility issues, but not all of them:

eurephia/1.1.0-6 by Alberto Gonzalez Iniesta, original patch by Chris Lamb.
fdroidserver/0.7.0-1 by Hans-Christoph Steiner, original patch by Chris Lamb.
mini-buildd/1.0.18 by Stephan S rken.
nbc/1.2.1.r4+dfsg-3 by Petter Reinholdtsen, original patch by Chris Lamb.
ncurses/6.0+20160910-1 by Sven Joachim, #818067 by Niels Thykier.
python-kinterbasdb/3.3.0-4 by Santiago Vila, original patch by Chris Lamb.
snapper/0.3.3-1 Hideki Yamane, original patch by Sascha Steinbiss.

Patches submitted that have not made their way to the archive yet:

#838188 filed against ocaml by Johannes Schauer.

Reviews of unreproducible packages 462 package reviews have been added, 524 have been updated and 166 have been removed in this week, adding to our knowledge about identified issues. 25 issue types have been updated:

Added a new annotation for issues called "fix-deterministic" to help us update package reviews more easily. This indicates whether we expect that an issue would always happen on Jenkins; i.e. if there is a successful build, then we know the issue is fixed for that package and can update our notes.
Added random_order_in_sisu_javax_inject_named and too_much_input_for_diff.
Removed timestamps_in_manpages_generated_by_ronn.
Updated timestamps_in_allegro_dat_files. Additionally, 21 issues were marked with "fix-deterministic".

Weekly QA work FTBFS bugs have been reported by:

Chris Lamb (10)
Filip Pytloun (1)
Santiago Vila (1)

diffoscope development A new version of diffoscope 60 was uploaded to unstable by Mattia Rizzolo. It included contributions from:

Mattia Rizzolo:
- Various packaging and testing improvements.
HW42:
- minor wording fixes
Reiner Herrmann:
- minor wording fixes

It also included from changes previous weeks; see either the changes or commits linked above, or previous blog posts 72 71 70. strip-nondeterminism development New versions of strip-nondeterminism 0.027-1 and 0.028-1 were uploaded to unstable by Chris Lamb. It included contributions from:

Chris Lamb:
- Testing improvements, including better handling of timezones.

disorderfs development A new version of disorderfs 0.5.1 was uploaded to unstable by Chris Lamb. It included contributions from:

Andrew Ayer and Chris Lamb:
- Support relative paths for ROOTDIR; it no longer needs to be an absolute path.
Chris Lamb:
- Print the behaviour (shuffle/reverse/sort) on startup to stdout.

It also included from changes previous weeks; see either the changes or commits linked above, or previous blog posts 70. Misc. This week's edition was written by Ximin Luo and reviewed by a bunch of Reproducible Builds folks on IRC.

18 September 2016

Norbert Preining: Fixing packages for broken Gtk3

As mentioned on sunweaver s blog Debian s GTK-3+ v3.21 breaks Debian MATE 1.14, Gtk3 is breaking apps all around. But not only Mate, probably many other apps are broken, too, in particular Nemo (the file manager of Cinnamon desktop) has redraw issues (bug 836908), and regular crashes (bug 835043). gtk-breakage

I have prepared packages for mate-terminal and nemo built from the most recent git sources. The new mate-terminal now does not crash anymore on profile changes (bug 835188), and the nemo redraw issues are gone. Unfortunately, the other crashes of nemo are still there. The apt-gettable repository with sources and amd64 binaries are here:

deb http://www.preining.info/debian/ gtk3fixes main
deb-src http://www.preining.info/debian/ gtk3fixes main

and are signed with my usual GPG key. Last but not least, I quote from sunweaver s blog: Questions

Isn t GTK-3+ a shared library? This one was rhetorical Yes, it is.
One that breaks other application with every point release? Well, unfortunately, as experience over the past years has shown: Yes, this has happened several times, so far and it happened again.
Why is it that GTK-3+ uploads appear in Debian without going through a proper transition? This question is not rhetorical. If someone has an answer, please enlighten me.

(end of quote) <rant>
My personal answer to this is: Gtk is strongly related to Gnome, Gnome is strongly related to SystemD, all this is pushed onto Debian users in the usual way of we don t care for breaking non-XXX apps (for XXX in Gnome, SystemD). It is very sad to see this recklessness taking more and more space all over Debian.
</rant> I finish with another quote from sunweaver s blog:

already scared of the 3.22 GTK+ release, luckily the last development release of the GTK+ 3-series

17 September 2016

Jonas Meurer: hello world

Hello world! Finally, this is my first post to my own blog. For years, I refrained from having an own blog, even though I think about it since 10 years at least. Today I decided to give it a try, let's see where it takes me. I'll write about Free Software topics from time to time, probably with a strong emphasis on Debian GNU/Linux. Welcome to everybody who steps by. Cheers,
mejo

Norbert Preining: Android 7.0 Nougat Root PokemonGo

Since my switch to Android my Nexus 6p is rooted and I have happily fixed the Android (<7) font errors with Japanese fonts in English environment (see this post). The recently released Android 7 Nougat finally fixes this problem, so it was high time to update. In addition, a recent update to Pokemon Go excluded rooted devices, so I was searching for a solution that allows me to: update to Nougat, keep root, and run PokemonGo (as well as some bank security apps etc).

After some playing around here are the steps I took: Installation of necessary components Warning: The following is for Nexus6p device, you need different image files and TWRP recovery for other devices. Flash Nougat firmware images Get it from the Google Android Nexus images web site, unpack the zip and the included zip one gets a lot of img files.

unzip angler-nrd90u-factory-7c9b6a2b.zip
cd angler-nrd90u/
unzip image-angler-nrd90u.zip

As I don t want my user partition to get flashed, I did not use the included flash script, but did it manually:

fastboot flash bootloader bootloader-angler-angler-03.58.img
fastboot reboot-bootloader
sleep 5
fastboot flash radio radio-angler-angler-03.72.img
fastboot reboot-bootloader
sleep 5
fastboot erase system
fastboot flash system system.img
fastboot erase boot
fastboot flash boot boot.img
fastboot erase cache
fastboot flash cache cache.img
fastboot erase vendor
fastboot flash vendor vendor.img
fastboot erase recovery
fastboot flash recovery recovery.img
fastboot reboot

After that boot into the normal system and let it do all the necessary upgrades. Once this is done, let us prepare for systemless root and possible hiding of it. Get the necessary file Get Magisk, SuperSU-magisk, as well as the Magisk-Manager.apk from this forum thread (direct links as of 2016/9: Magisk-v6.zip, SuperSU-v2.76-magisk.zip, Magisk-Manager.apk). Transfer these two files to your device I am using an external USB stick that can be plugged into the device, or copy it via your computer or via a cloud service. Also we need to get a custom recovery image, I am using TWRP. I used the version 3.0.2-0 of TWRP I had already available, but that version didn t manage to decrypt the file system and hangs. One needs to get at least version 3.0.2-2 from the TWRP web site. Install latest TWRP recorvery Reboot into boot-loader, then use fastboot to flash twrp:

fastboot erase recovery
fastboot flash recovery twrp-3.0.2-2-angler.img
fastboot reboot-bootloader

After that select Recovery with the up-down buttons and start twrp. You will be asked you pin if you have one set. Install Magisk-v6.zip Select Install in TWRP, select the Magisk-v6.zip file, and see you device being prepared for systemless root. Install SuperSU, Magisk version Again, boot into TWRP and use the install tool to install SuperSU-v2.76-magisk.zip. After reboot you should have a SuperSU binary running. Install the Magisk Manager From your device browse to the .apk and install it. How to run safety net programs Those programs that check for safety functions (Pokemon Go, Android Pay, several bank apps) need root disabled. Open the Magisk Manager and switch the root switch to the left (off). After this starting the program should bring you past the special check.

Next.

Previous.