My monthly report covers a large part of what I have been doing in the free software world. I write it for my donors (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. Debian LTS In the 11 hours of (paid) work I had to do, I managed to release DLA-716-1 aka tiff 4.0.2-6+deb7u8 fixing CVE-2016-9273, CVE-2016-9297 and CVE-2016-9532. It looks like this package is currently getting new CVE every month. Then I spent quite some time to review all the entries in dla-needed.txt. I wanted to get rid of some misleading/no longer applicable comments and at the same time help Olaf who was doing LTS frontdesk work for the first time. I ended up tagging quite a few issues as no-dsa (meaning that we will do nothing for them as they are not serious enough) such as those affecting dwarfutils, dokuwiki, irssi. I dropped libass since the open CVE is disputed and was triaged as unimportant. While doing this, I fixed a bug in the bin/review-update-needed script that we use to identify entries that have not made any progress lately. Then I claimed libgc and and released DLA-721-1 aka libgc 1:7.1-9.1+deb7u1 fixing CVE-2016-9427. The patch was large and had to be manually backported as it was not applying cleanly. The last thing I did was to test a new imagemagick and review the update prepared by Roberto. pkg-security work The pkg-security team is continuing its good work: I sponsored patator to get rid of a useless dependency on pycryptopp which was going to be removed from testing due to #841581. After looking at that bug, it turns out the bug was fixed in libcrypto++ 5.6.4-3 and I thus closed it. I sponsored many uploads: polenum, acccheck, sucrack (minor updates), bbqsql (new package imported from Kali). A bit later I fixed some issues in the bbsql package that had been rejected from NEW. I managed a few RC bugs related to the openssl 1.1 transition: I adopted sslsniff in the team and fixed #828557 by build-depending on libssl1.0-dev after having opened the proper upstream ticket. I did the same for ncrack and #844303 (upstream ticket here). Someone else took care of samdump2 but I still adopted the package in the pkg-security team as it is a security relevant package. I also made an NMU for axel and #829452 (it s not pkg-security related but we still use it in Kali). Misc Debian work Django. I participated in the discussion about a change letting Django count the number of developers that use it. Such a change has privacy implications and the discussion sparked quite some interest both in Debian mailing lists and up to LWN. On a more technical level, I uploaded version 1.8.16-1~bpo8+1 to jessie-backports (security release) and I fixed RC bug #844139 by backporting two upstream commits. This led to the 1.10.3-2 upload. I ensured that this was fixed in the 1.10.x upstream branch too. dpkg and merged /usr. While reading debian-devel, I discovered dpkg bug #843073 that was threatening the merged-/usr feature. Since the bug was in code that I wrote a few years ago, and since Guillem was not interested in fixing it, I spent an hour to craft a relatively clean patch that Guillem could apply. Unfortunately, Guillem did not yet manage to pull out a new dpkg release with the patches applied. Hopefully it won t be too long until this happens. Debian Live. I closed #844332 which was a request to remove live-build from Debian. While it was marked as orphaned, I was always keeping an eye on it and have been pushing small fixes to git. This time I decided to officially adopt the package within the debian-live team and work a bit more on it. I reviewed all pending patches in the BTS and pushed many changes to git. I still have some pending changes to finish to prettify the Grub menu but I plan to upload a new version really soon now. Misc bugs filed. I filed two upstream tickets on uwsgi to help fix currently open RC bugs on the package. I filed #844583 on sbuild to support arbitrary version suffix for binary rebuild (binNMU). And I filed #845741 on xserver-xorg-video-qxl to get it fixed for the xorg 1.19 transition. Zim. While trying to fix #834405 and update the required dependencies, I discovered that I had to update pygtkspellcheck first. Unfortunately, its package maintainer was MIA (missing in action) so I adopted it first as part of the python-modules team. Distro Tracker. I fixed a small bug that resulted in an ugly traceback when we got queries with a non-ASCII HTTP_REFERER. Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Drought here since August. The small cistern ran dry a month ago, which has never happened before. The large cistern was down to some 900 gallons. I don't use anywhere near the national average of 400 gallons per day. More like 10 gallons. So could have managed for a few more months. Still, this was worrying, especially as the area moved from severe to extreme drought according to the US Drought Monitor. Two days of solid rain fixed it, yay! The small cistern has already refilled, and the large will probably be full by tomorrow. The winds preceeding that same rain storm fanned the flames that destroyed Gatlinburg. Earlier, fire got within 10 miles of here, although that may have been some kind of controlled burn. Climate change is leading to longer duration weather events in this area. What tended to be a couple of dry weeks in the fall, has become multiple months of drought and weeks of fire. What might have been a few days of winter weather and a few inches of snow before the front moved through has turned into multiple weeks of arctic air, with multiple 1 ft snowfalls. What might have been a few scorching summer days has become a week of 100-110 degree temperatures. I've seen all that over the past several years. After this, I'm adding "additional, larger cistern" to my todo list. Also "larger fire break around house".

Here is my monthly update covering what I have been doing in the free software world (previous month):

• Started work on a Python API to the UK Postbox mail scanning and forwarding service. (repo)
• Lots of improvements to buildinfo.debian.net, my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them, including making GPG signatures mandatory (#7), updating jenkins.debian.net to sign them and moving to SSL.
• Improved the Django client to the KeyError error tracking software, enlarging the test coverage and additionally adding support for grouping errors using a context manager.
• Made a number of improvements to travis.debian.net, my hosted service for projects that host their Debian packaging on GitHub to use the Travis CI continuous integration platform to test builds on every code change:
  • Install build-dependencies with debugging output. Thanks to @waja. (#31)
  • Install Lintian by default. Thanks to @freeekanayaka. (#33).
  • Call mktemp with --dry-run to avoid having to delete it later. (commit)
• Submitted a pull request to Wheel (a utility to package Python libraries) to make the output of METADATA files reproducible. (#73)
• Submitted some miscellaneous documentation updates to the Tails operating system. (patches)

---

---

Do you love porting? For ideas on how to make GHC suck less on your favorite architecture, see this not-at-all ugly table.

Since 2008 I use two monitors in my desktop. Yesterday I bought a new graphics interface and a third monitor. Some time I was looking for a low cost graphics interface. Ok, I am using GeForce GT 740 which has three output ports: VGA, DVI and HDMI. In Brazil this interface card can be found around R$ 400 (US$ 117, but my card was US$ 87 in Brazilian Black Friday). In Amazon.com, it is between US$ 51 and US$ 109. The chosen manufacturer was Zotac, but all GT 740 and 750 will work fine (I tested the GT 750 too). The GeForce GT 740 was imediatelly recognised by Debian Jessie with kernel Linux 4.7.0 from Backports (it is my default, so I didn't test with original 3.16 kernel). The driver used was the default X.Org Nouveau. I use KDE and the management was easy. I hope this post can help people interested in use 3 monitors. Enjoy!

This is now a little overdue, but here it is. On the 10th and 11th of November, the second vmdebootstrap sprint took place. Lars Wirzenius (liw), Ana Custura (ana_c) and myself were present. liw focussed on the core of vmdebootstrap, where he sketched out what the future of vmdebootstrap may look like. He documented this in a mailing list post and also presented (video). Ana and myself worked on live-wrapper, which uses vmdebootstrap internally for the squashfs generation. I worked on improving logging, using a better method for getting paths within the image, enabling generation of Packages and Release files for the image archive and also made the images installable (live-wrapper 0.5 onwards will include an installer by default). Ana worked on the inclusion of HDT and memtest86+ in the live images and enabled both ISOLINUX (for BIOS boot) and GRUB (for EFI boot) to boot the text-mode and graphical installers. live-wrapper 0.5 was released on the 16th November with these fixes included. You can find live-wrapper documentation at https://live-wrapper.readthedocs.io/en/latest/. (The documentation still needs some work, some options may be incorrectly described). Thanks to the sponsors that made this work possible. You re awesome. (:

Categories:

• Debian-Blog
• Computing
• Tools

Keywords:

• SAN
• Storage
• iscsi
• multipath
• scsi

Like:

I am a incorrigibly in picking non-mainstream, open smartphones, and then struggling hard. Back then in 2008, I tried to use the OpenMoko FreeRunner, but eventually gave up because of hardware glitches and reverted to my good old Siemens S35. It was not that I would not be willing to put up with inconveniences, but as soon as it makes live more difficult for the people I communicate with, it becomes hard to sustain. Two years ago I tried again, and got myself a Jolla phone, running Sailfish OS. Things are much nicer now: The hardware is mature, battery live is good, and the Android compatibility layer enables me to run many important apps that are hard to replace, especially the Deutsche Bahn Navigator and various messengers, namely Telegram, Facebook Messenger, Threema and GroupMe. Some apps that require Google Play Services, which provides a bunch of common tasks and usually comes with the Google Play store would not run on my phone, as Google Play is not supported on Sailfish OS. So far, the most annoying ones of that sort were Uber and Lyft, making me pay for expensive taxis when others would ride cheaper, but I can live with that. I tried to install Google Play Services from shady sources, but it would regularly crash.

Signal on Jolla Now in Philadelphia, people urged me to use the Signal messenger, and I was convinced by its support for good end-to-end crypto, while still supporting offline messages and allowing me to switch from my phone to my desktop and back during a conversation. The official Signal app uses Google Cloud Messaging (GCM, part of Google Play Services) to get push updates about new posts, and while I do not oppose this use of Google services (it really is just a ping without any metadata), this is a problem on Sailfish OS. Luckily, the Signal client is open source, and someone created a LibreSignal edition that replaced the use of GCM with websockets, and indeed, this worked on my phone, and I could communicate. Things were not ideal, though: I would often have to restart the app to get newly received messages; messages that I send via Signal Desktop would often not show up on the phone and, most severe, basically after every three messages, sending more messages from Desktop would stop working for my correspondents, which freaked them out. (Strangely it continued working from their phone app, so we coped for a while.) So again, my choice of non-standard devices causes inconveniences to others. This, and the fact that the original authors of Signal and the maintainers of LibreSignal got into a fight that ended LibreSignal discontinued, meant that I have to change something about this situation. I was almost ready to give in and get myself a Samsung S7 or something boring of the sort, but then I decided to tackle this issue once more, following some of the more obscure instructions out there, trying to get vanilla Signal working on my phone. About a day later, I got it, and this is how I did it.

microG So I need Google Play Services somehow, but installing the real thing did not seem to be very promising (I tried, and regularly got pop-ups telling me that Play Services has crashed.) But I found some references to a project called microG , which is an independent re-implementation of (some of) of the play services, in particular including GCM. Installing microG itself was easy, as you can add their repository to F-Droid. I installed the core services, the services framework and the fake store apps. If this had been all that was to do, things would be easy!

Play Store detection work arounds But Signal would still complain about the lack of Google Play Services. It asks Android if an app with a certain name is installed, and would refuse to work if this app does not exist. For some reason, the microG apps cannot just have the names of the real Google apps. There seem to be two ways of working around this: Patching Signal, or enabling Signature Spoofing. The initially most promising instructions (which are in a README in a tarball on a fishy file hoster linked from an answer on the Jolla support forum ) suggested patching Signal, and actually came both with a version of an app called Lucky Patcher as well as a patched Android package, but both about two years old. I tried a recent version of the Lucky Patcher, but it failed to patch the current version of Signal.

Signature Spoofing So on to Signature Spoofing. This is a feature of some non-standard Android builds that allow apps (such as microG) to fake the existence of other apps (the Play Store), and is recommended by the microG project. Sailfish OS s Android compatibility layer Alien Dalvik does not support it out of the box, but there is a tool tingle that adds this feature to existing Android systems. One just has to get the /system/framework/framework.jar file, put it into the input folder of this project, run python main.py, select 2, and copy the framework.jar from output/ back. Great.

Deodexing Alien Dalvik Only that it only works on deodexed files. I did not know anything about odexed Android Java classes (and did not really want to know), but there was not way around. Following this explanation I gathered that one finds files foo.odex in the Android system folder, runs some tool on them to create a classes.dex file, and adds that to the corresponding foo.jar or foo.apk file, copies this back to the phone and deletes the foo.odex file. The annoying this is that one does not only have to do it for framework.jar in order to please tingle, because if one does it to one odex file, one has to do to all! It seems that for people using Windows, the Universal Deodexer V5 seems to be a convenient tool, but I had to go more manually. So I first fetched smali , compiled it using ./gradlew build. Then I fetched the folders /opt/alien/system/framework and /opt/alien/system/app from the phone (e.g. using scp). Keep a backup of these in case something breaks. Then I ran these commands (disclaimer: I fetched these from my bash history and slightly cleaned them up. This is not a fire-and-forget script! Use it when you know what it and you are doing):

cd framework
for file in *.odex
do
  java -jar ~/build/smali/baksmali/build/libs/baksmali.jar deodex $file -o out
  java -jar ~/build/smali/smali/build/libs/smali.jar a out -o classes.dex
  zip -u $(basename $file .odex).jar classes.dex
  rm -rf out classes.dex $file
done
cd ..
cd app
for file in *.odex
do
  java -jar ~/build/smali/baksmali/build/libs/baksmali.jar deodex -d ../framework $file -o out
  java -jar ~/build/smali/smali/build/libs/smali.jar a out -o classes.dex
  zip -u $(basename $file .odex).apk classes.dex
  rm -rf out classes.dex $file
done
cd ..

The resulting framework.jar can now be patched with tingle:

mv framework/framework.jar ~/build/tingle/input
cd ~/build/tingle
./main.py
# select 2
cd -
mv ~/build/tingle/output/framework.jar framework/framework.jar

Now I copy these framework and app folders back on my phone, and restart Dalvik:

devel-su systemctl restart aliendalvik.service

It might start a bit slower than usually, but eventually, all the Android apps should work as before. The final bit that was missing in my case was that I had to reinstall Signal: If it is installed before microG is installed, it does not get permission to use GCM, and when it tries (while registering: After generating the keys) it just crashes. I copied /data/data/org.thoughtcrime.secretsms/ before removing Signal and moved it back after (with cp -a to preserve permissions) so that I could keep my history. And now, it seems, vanilla Signal is working just fine on my Jolla phone!

What s missing Am I completely happy with Signal? No! An important feature that it is lacking is a way to get out all data (message history including media files) in a file format that can be read without Signal; e.g. YAML files or clean HTML code. I do want to be able to re-read some of the more interesting conversations when I am 74 or 75, and I doubt that there will be a Signal App, or even Android, then. I hope that this becomes available in time, maybe in the Desktop version. I would also hope that pidgin gets support to the Signal protocol, so that I conveniently use one program for all my messaging needs on the desktop. Finally it would be nice if my Signal identity was less tied to one phone number. I have a German and a US phone number, and would want to be reachable under both on all my clients. (If you want to contact me on Signal, use my US phone number.)

Alternatives Could I have avoided this hassle by simply convincing people to use something other than Signal? Tricky, at the moment. Telegram (which works super reliable for me, and has a pidgin plugin) has dubious crypto and does not support crypto while using multiple clients. Threema has no desktop client that I know of. OTR on top of Jabber does not support offline messages. So nothing great seems to exist right now. In the long run, the best bet seems to be OMEMO (which is, in essence, the Signal protocol) on top of Jabber. It is currently supported by one Android Jabber client (Conversations) and one Desktop application (gajim, via a plugin). I should keep an eye on pidgin support for OMEMO and other development around this.

I don't often write on this blog, and when I do, it's either tech related, or light life stuff. Over the next few weeks, it's going to get a lot more political. If you currently follow this blog for its technical content, you may be tempted to tune out. I would encourage you to stay and listen. I'm passionate about the technology that I work on; but the greatest problems facing our world today are not ones that will be solved with software. American democracy is in bad shape, and it's because of what we're doing to it. This is not a problem of the Right or of the Left; it is not a problem that began with the election of Donald Trump, and it's not a problem that will go away at the end of his term. It is partly a structural problem with the way our elections work, but more than that it's a problem of how we're splitting into separate tribes, isolating ourselves from those who don't agree with us. As Russ Allbery wrote the morning after the election, everything about how we organize ourselves online today - and how we let Facebook and Twitter organize us - leads us to surround ourselves with people who already think the same way we do. That leaves all of us with huge blind spots for other people in our country, and it stifles the free exchange of ideas that is so essential for a healthy democracy. We need leaders who will work to make America a better and more just place for all our neighbors, not just a two-party system that plays tug-of-war using two different sets of voters that feel shut out. And the way we organize ourselves today (online and off) does not let us recognize those leaders. There's a lot of talk now about Facebook changing how it decides what to show people; and maybe they can manage to help everyone's online experience be a little less of a bubble. But part of the change needs to come from us. We need to be willing to engage, civilly, with people whose perspective is different from ours, and make the effort to understand where the other is coming from. So for the next few weeks, I'm going to talk. And I'm going to listen. I have no unique qualifications to speak about the country's issues. But I do have a perspective of my own, which might be different enough from yours to be useful. I was born and raised in Iowa, and graduated from college there. This election cycle, I learned that Iowa holds the distinction of being the state with the lowest percentage of college-educated whites. I'm part of that statistic, because a few years after graduating I moved to Portland, Oregon - a place that's notoriously so far to the left of what we think of as the middle, that it actually has anarchists who would shamefully use a peaceful protest as cover to commit property crime. So I know a few things about the people in each state, that I think the other should hear. I'm also that rarest of creatures, a Portlander who goes to church (Catholic). But I still choose as my neighbors the weird, wonderful, and welcoming community that we have here, whatever Glenn Beck might think. I have a son, and I worry about what kind of world he'll grow up to live in. I work in software, which means I'm doing a lot better than a lot of people in the country right now; it also means that from where I sit, I see trends already in progress that will have an effect on the working class and the middle class that makes NAFTA look like a gnat's fart in comparison. And so I worry for what kind of world we will all live in, if we don't make some changes fast. Let's have a conversation. No comments enabled on this blog, but you can find me on G+ or on Facebook.

TL;DR; If you run a FLOSS development project and you notice D0n1elT appearing on your IRC channel, please give him a warm welcome. D0n1elT is a young man highy talented in various FLOSS related topics already. He probably needs some guidance at the beginning and I hope he won't be too shy to ask for it. But you can be sure: your channel has been joined by someone you should consider as a future resource. The Long Story During the last two weeks I had the great pleasure of supervising a fine young man (very young, still, indeed) in all sorts of IT topics. This young man turned out to be so skilled and interested in various FLOSS related areas, I really want to introduce him to all of you. The young man's real name is Daniel Teichmann. On IRC he may appear under his nick: D0n1elT. His GnuPG Fingerprint is: 6C6E 7F8F F7E8 B22E FC76 E9F7 8A79 028F DA56 7C6C. Daniel goes to a local school here in Nothern Germany, near where I live. He attends the 9th grade at his school, and as common for students of his age and grade, practical training was scheduled for the last two weeks. Daniel had originally applied for practical training at some other business near his place of living (which is quite far off from the school, actually). However, that company cancelled his training position two work days before the training was supposed to start. Daniel's teacher rang me up and asked for help. He advertised Daniel as someone who is far advanced in IT topics compared to his co-students. "He even writes his own programs (in Java and C++)." Spontaneously, Andreas Buchholz (CEO of LOGO EDV-Systeme GmbH) and I decided to accept Daniel as a trainee. Without having met him, with no application interview beforehand. The deal was: Daniel comes to Andreas business location in Kiel (40-50km away from Daniel's place of living) and I (working as freelancer for LOGO on a regular basis) do the supervising part. On day one and two, as a warm-up, Daniel installed a Debian Edu Main Server, worked himself through GOsa, LDAP, SSH, GnuPG, Jabber and IRC and configured two routers. All topics were new to him and I could hardly think of new tasks to give to him. As means of communication we set up a Jabber account, then an IRC account (as backup). However, it turned out that Daniel really got a hang of IRC over the next couple of days, so we used that as primary communication channel. Daniel had already programmed various projects in Java (whereas I have never touched Java, so far :-( ). He has written plugins for Minecraft servers. He knows well how to implement object oriented coding models. His coding style looks very good and clean (esp. for someone who has never head a nitpicking code reviewer). He started coding at the age of 9. Instead of diving into Java (where I would not have been of much help, anyway) I decided to provide him with some really basic and Unix-like knowledge: Bash scripting. I wanted to see how he handles another "language" and how he applies his Java knowledge to a lower level, syntactically weaker language. Guess what, he managed that assignment very well. Working on Impressive Display At Daniel's school we run substitute teacher info screens based on a fancy Bash script, named impressive-display, and the impressive PDF viewer. The Impressive Display tool is available in Debian testing/unstable under the same name. So over the next couple of days we worked on Impressive Display. Daniel contributed so many new code passages, conceptual ideas and security concerns, that I decided to make him co-copyright holder. Every change contributed by him received intensive testing before committing to Git. While working on Impressive Display, collaborating with Daniel via Git was a mere pleasure. In his spare time Daniel likes watching Github tutorials. Quite extraordinary. The result is a new major release of Impressive Display: Version 0.3.1 (bumped up from 0.2.3). We added the feature of handling info screen farms based on PXE boot images. It is now possible to configure as many different info screens as needed within the same PXE bootable chroot. Furthermore, Impressive Display now has a PDF presentation (written in LaTeX Beamer) that documents how to setup your own info screens. The PDF presentation is the default PDF that comes up when you start Impressive Display directly after installation. Investigating other Realms We also took a deeper look at remote desktop stuff, one of my most favourite topics. By that impulse Daniel set up his first Vserver machine at some hosting provider. He figured out how to run X2Go Server on that machine with an XFCE desktop. Next step was to run the irssi instance from his notebook inside a screen session on the Vserver. Some days later, Daniel PM'ed me: "I have an IRC bouncer now...". Quintessence It was a great pleasure meeting this young, highly curious and already highly skilled young man over the past two weeks. Daniel, it was an asset to me working with you. You are such a fast learner when it comes to getting accustomed to new working environments, it is amazing. I cannot deny having observed the tendency of preferring rather geeky tools. I was highly delighted, that What's-That and Facebook are nothing that rocks you so much. Unfortunately, all of the above makes you quite unique and non-mainstream among people of your age. My wish for you (and the FLOSS world) is that you start getting in touch with other (FLOSS) developers, maybe of your age, maybe older, and that you (if this is what you want) become an asset to the world of Free Software. The Free Software world can be a world where technical, political and spiritual work become one with friendship among people. Take care and farewell! I am sure, we will meet again. light+love Mike Gabriel (aka sunweaver on IRC and debian.org)

Like I wrote before, we at Collabora have been working on improving WebKitGTK+ performance for customer projects, such as Apertis. We took the opportunity brought by recent improvements to WebKitGTK+ and GTK+ itself to make the final leg of drawing contents to screen as efficient as possible. And then we went on investigating why so much CPU was still being used in some of our test cases. The first weird thing we noticed is performance was actually degraded on Wayland compared to running under X11. After some investigation we found a lot of time was being spent inside GTK+, painting the window s background. Here s the thing: the problem only showed under Wayland because in that case GTK+ is responsible for painting the window decorations, whereas in the X11 case the window manager does it. That means all of that expensive blurring and rendering of shadows fell on GTK+ s lap. During the web engines hackfest, a couple of months ago, I delved deeper into the problem and noticed, with Carlos Garcia s help, that it was even worse when HiDPI displays were thrown into the mix. The scaling made things unbearably slower. You might also be wondering why would painting of window decorations be such a problem, anyway? They should only be repainted when a window changes size or state anyway, which should be pretty rare, right? Right, that is one of the reasons why we had to make it fast, though: the resizing experience was pretty terrible. But we ll get back to that later. So I dug into that, made a few tries at understanding the issue and came up with a patch showing how applying the blur was being way too expensive. After a bit of discussion with our own Pekka Paalanen and Benjamin Otte we found the root cause: a fast path was not being hit by pixman due to the difference in scale factors on the shadow mask and the target surface. We made the shadow mask scale the same as the surface s and voil , sane performance. I keep talking about this being a performance problem, but how bad was it? In the following video you can see how huge the impact in performance of this problem was on my very recent laptop with a HiDPI display. The video starts with an Epiphany window running with a patched GTK+ showing a nice demo the WebKit folks cooked for CSS animations and 3D transforms. After a few seconds I quickly alt-tab to the version running with unpatched GTK+ I made the window the exact size and position of the other one, so that it is under the same conditions and the difference can be seen more easily. It is massive. Yes, all of that slow down was caused by repainting window shadows! OK, so that solved the problem for HiDPI displays, made resizing saner, great! But why is GTK+ repainting the window even if only the contents are changing, anyway? Well, that turned out to be an off-by-one bug in the code that checks whether the invalidated area includes part of the window decorations. If the area being changed spanned the whole window width, say, it would always cause the shadows to be repainted. By fixing that, we now avoid all of the shadow drawing code when we are running full-window animations such as the CSS poster circle or gtk3-demo s pixbufs demo. As you can see in the video below, the gtk3-demo running with the patched GTK+ (the one on the right) is using a lot less CPU and has smoother animation than the one running with the unpatched GTK+ (left). Pretty much all of the overhead caused by window decorations is gone in the patched version. It is still using quite a bit of CPU to animate those pixbufs, though, so some work still remains. Also, the overhead added to integrate cairo and GL rendering in GTK+ is pretty significant in the WebKitGTK+ CSS animation case. Hopefully that ll get much better from GTK+ 4 onwards.

Last week we had an interesting Debian meeting in Seville, Spain. This has been the third time (in recent years) the local community meets around Debian. We met at about 20:00 at Rompemoldes, a crafts creation space. There we had a very nice dinner while talking about Debian and FLOSS. The dinner was sponsored by the Plan4D assosiation. The event was joined by almost 20 people which different relations to Debian:

• Debian users
• DDs
• Debian contributors
• General FLOSS interested people

I would like to thank all the attendants and Pablo Neira from Plan4D for the organization. I had to leave the event after 3.5 hours of great talking and networking, but the rest of the people stayed there. The climate was really good :-) Looking forward to another meeting in upcomings times! Header picture by Ana Rey.

Even though the World Chess Championship takes up a lot of time these days, I've still found some time for Nageru, my live video mixer. But this time it doesn't come in form of code; rather, I've spent my time writing documentation. I spent some time fretting over what technical solution I wanted. I explicitly wanted end-user documentation, not developer documentation I rarely find HTML-rendered versions of every member function in a class the best way to understand a program anyway. Actually, on the contrary: Having all sorts of syntax interwoven in class comments tends to be more distracting than anything else. Eventually I settled on Sphinx, not because I found it fantastic (in particular, ReST is a pain with its bizarre variable punctuation-based syntax), but because I'm convinced it has all the momentum right now. Just like git did back in the day, the fact that the Linux kernel has chosen it means it will inevitably grow a quite large ecosystem, and I won't be ending up having to maintain it anytime soon. I tried finding a balance between spending time on installation/setup (only really useful for first-time users, and even then, only a subset of them), concept documentation (how to deal with live video in general, and how Nageru fits into a larger ecosystem of software and equipment) and more concrete documentation of all the various features and quirks of Nageru itself. Hopefully, most people will find at least something that's not already obvious to them, without drowning in detail. You can read the documentation at https://nageru.sesse.net/doc/, or if you want to send patches, the right place to patch is the git repository.

Updates to Altos Lisp I wrote a few days ago about a tiny lisp interpreter I wrote for AltOS Really, it's almost "done" now, I just wanted to make a few improvements Incremental Collection I was on a walk on Wednesday when I figured out that I didn't need to do a full collection every time; a partial collection that only scanned the upper portion of memory would often find plenty of free space to keep working for a while. To recap, the heap is in two pieces; the ROM piece and the RAM piece. The ROM piece is generated during the build process and never changes afterwards (hence the name), so the only piece which is collected is the RAM piece. Collection works like:

chunk_low = heap base
new_top = heap base
For all of the heap
    Find the first 64 live objects above chunk_low
    Compact them all to new_top
    Rewrite references in the whole heap for them
    Set new_top above the new locations
    Set chunk_low above the old locations
top = new_top

The trick is to realize that there's really no need to start at the bottom of the heap; you can start anywhere you like and compact stuff, possibly leaving holes below that location in the heap. As the heap tends to have long-lived objects slowly sift down to the beginning, it's useful to compact objects higher than that, skipping the compaction process for the more stable area in memory. Each time the whole heap is scanned, the top location is recorded. After that, incremental collects happen starting at that location, and when that doesn't produce enough free space, a full collect is done. The collector now runs a bunch faster on average now. Binary Searches I stuck some linear searches in a few places in the code, the first was in the collector when looking to see where an object had moved to. As there are 64 entries, the search is reduced from 32 to 6 compares on average. The second place was in the frame objects, which hold the list of atom/value bindings for each lexical scope (including the global scope). These aren't terribly large, but a binary search is still a fine plan. I wanted to write down here the basic pattern I'm using for binary searches these days, which avoids some of the boundary conditions I've managed to generate in the past:

int find (needle)  
    int l = 0;
    int r = count - 1;
    while (l <= r)  
        int m = (l + r) >> 1;
        if (haystack[m] < needle)
            l = m + 1;
        else
            r = m - 1;
     
    return l;
 

With this version, the caller can then check to see if there's an exact match, and if not, then the returned value is the location in the array where the value should be inserted. If the needle is known to not be in the haystack, and if the haystack is large enough to accept the new value:

void insert(needle)  
    int l = find(needle);
    memmove(&haystack[l+1],
        &haystack[l],
        (num - l) * sizeof (haystack[0]));
    haystack[l] = needle;
 

Similarly, if the caller just wants to know if the value is in the array:

bool exists(needle)  
    int l = find(needle);
    return (l < count && haystack[l] == needle);
 

Call with Current Continuation Because the execution stack is managed on the heap, it's completely trivial to provide the scheme-like call with current continuation, which constructs an object which can be 'called' to transfer control to a saved location:

> (+ "hello " (call/cc (lambda (return) (setq boo return) (return "foo "))) "world")
"hello foo world"
> (boo "bar ")
"hello bar world"
> (boo "yikes ")
"hello yikes world"

One thing I'd done previously is dump the entire state of the interpreter on any error, and that included a full stack trace. I adopted that code for printing of these continuation objects:

boo
[
    expr:   (call/cc (lambda (return) (set (quote boo) return) (return "foo ")))
    state:  val
    values: (call/cc
             [recurse...]
             )
    sexprs: ()
    frame:   
]
[
    expr:   (+ "hello " (call/cc (lambda (return) (set (quote boo) return) (return "foo "))) "world")
    state:  formal
    values: (+
             "hello "
             )
    sexprs: ((call/cc (lambda (return) (set (quote boo) return) (return "foo ")))
             "world"
             )
    frame:   
]

The top stack frame is about to return from the call/cc spot with a value; supply a value to 'boo' and that's where you start. The next frame is in the middle of computing formals for the + s-expression. It's found the + function, and the "hello " string and has yet to get the value from call/cc or the value of the "world" string. Once the call/cc "returns", that value will get moved to the values list and the sexpr list will move forward one spot to compute the "world" value. Implementing this whole mechanism took only a few dozen lines of code as the existing stack contexts were already a continuation in effect. The hardest piece was figuring out that I needed to copy the entire stack each time the continuation was created or executed as it is effectively destroyed in the process of evaluation. I haven't implemented dynamic-wind yet; when I did that for nickle, it was a bit of a pain threading execution through the unwind paths. Re-using Frames I decided to try and re-use frames (those objects which hold atom/value bindings for each lexical scope). It wasn't that hard; the only trick was to mark frames which have been referenced from elsewhere as not-for-reuse and then avoid sticking those in the re-use queue. This reduced allocations even further so that for simple looping or tail-calling code, the allocator may never end up being called. How Big Is It? I've managed to squeeze the interpreter and all of the rest of the AltOS system into 25kB of Cortex-M0 code. That leaves space for the 4kB boot loader and 3kB of flash to save/restore the 3kB heap across resets. Adding builtins to control timers and GPIOs would make this a reasonable software load for an Arduino; offering a rather different programming model for those with a taste for adventure. Modern ARM-based Arduino boards have plenty of flash and ram for this. It might be interesting to get this running on the Arduino Zero; there's no real reason to replace the OS either; porting the lisp interpreter into the bare Arduino environment wouldn't take long.

Yesterday the eighth update in the 0.12.* series of Rcpp made it to the CRAN network for GNU R where the Windows binary has by now been generated too; the Debian package is on its way as well. This 0.12.8 release follows the 0.12.0 release from late July, the 0.12.1 release in September, the 0.12.2 release in November, the 0.12.3 release in January, the 0.12.4 release in March, the 0.12.5 release in May, the 0.12.6 release in July, and the 0.12.7 release in September --- making it the twelveth release at the steady bi-montly release frequency. While we are keeping with the pattern, we have managed to include quite a lot of nice stuff in this release. None of it is a major feauture, though, and so we have not increased the middle number. Among the changes this release are (once again) much improved exception handling (thanks chiefly to Jim Hester), better large vector support (by Qiang), a number of Sugar extensions (mostly Nathan, Qiang and Dan) and beginnings of new DateVector and DatetimeVectors classes, and other changes detailed below. We plan to properly phase in the new date(time) classes. For now, you have to use a #define such as this one in Rcpp.h which remains commented-out for now. We plan to switch this on as the new default no earlier than twelve months from now. Rcpp has become the most popular way of enhancing GNU R with C or C++ code. As of today, 843 packages on CRAN depend on Rcpp for making analytical code go faster and further. That is up by eightyfour packages, or a full ten percent, just since the last release in early September! Again, we are lucky to have such a large group of contributors. Among them, we have invited Nathan Russell to the Rcpp Core team given his consistently excellent pull requests (as well as many outstanding Stackoverflow answers for Rcpp). More details on changes are below.

Changes in Rcpp version 0.12.8 (2016-11-16)

• Changes in Rcpp API:
  • String and vector elements now use extended R_xlen_t indices (Qiang in PR #560)
  • Hashing functions now return unsigned int (Qiang in PR #561)
  • Added static methods eye(), ones(), and zeros() for select matrix types (Nathan Russell in PR #569)
  • The exception call stack is again correctly reported; print methods and tests added too (Jim Hester in PR #582 fixing #579)
  • Variatic macros no longer use a GNU extensions (Nathan in PR #575)
  • Hash index functions were standardized on returning unsigned integers (Also PR #575)
• Changes in Rcpp Sugar:
  • Added new Sugar functions rowSums(), colSums(), rowMeans(), colMeans() (PR #551 by Nathan Russell fixing #549)
  • Range Sugar now used R_xlen_t type for start/end (PR #568 by Qiang Kou)
  • Defining RCPP_NO_SUGAR no longer breaks the build. (PR #585 by Daniel C. Dillon)
• Changes in Rcpp unit tests
  • A test for expression vectors was corrected.
  • The constructor test for datetime vectors reflects the new classes which treats Inf correctly (and still as a non-finite value)
• Changes in Rcpp Attributes
  • An 'empty' return was corrected (PR #589 fixing issue #588, and with thanks to Duncan Murdoch for the heads-up)
• Updated Date and Datetime vector classes:
  • The DateVector and DatetimeVector classes were renamed with a prefix old; they are currently typedef'ed to the existing name (#557)
  • New variants newDateVector and newDatetimeVector were added based on NumericVector (also #557, #577, #581, #587)
  • By defining RCPP_NEW_DATE_DATETIME_VECTORS the new classes can activated. We intend to make the new classes the default no sooner than twelve months from this release.
  • The capabilities() function can also be used for presence of this feature

Thanks to CRANberries, you can also look at a diff to the previous release. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads page, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

What happened in the Reproducible Builds effort between Sunday November 6 and Saturday November 12 2016: Media coverage Matthew Garrett blogged about Tor, TPMs and service integrity attestation and how reproducible builds are the base for systems integrity. The Linux Foundation announced renewed funding for us as part of the Core Infrastructure Initiative. Thank you! Outreachy updates Maria Glukhova has been accepted into the Outreachy winter internship and will work with us the Debian reproducible builds team. To quote her words

siamezzze: I've been accepted to #outreachy winter internship - going to
work with Debian reproducible builds team. So excited about that! <3
Debian

Toolchain development and fixes dpkg:

• Thanks to a series of dpkg uploads by Guillem Jover, all our toolchain changes are now finally available in sid!
• This means your packages should now be reproducible without having to use our custom APT repository.
• Ximin Luo opened #843925 to remind the fact that dpkg-buildpackage should sign buildinfo files.
• We hope to have detailed post about the new dpkg and the new .buildinfo files for debian-devel-announce soon!

debrebuild:

• srebuild / debrebuild work was resumed by Johannes Schauer and others in #774415.

Bugs filed Chris Lamb:

• #844102 filed against lava-dispatcher
• #844103 filed against lava-server
• #844111 filed against python-defaults
• #843698 filed against tunnelx
• #843967 filed against asyncpg
• #843865 filed against keystone
• #844101 filed against suil

Daniel Shahaf:

• #844232 filed against daisy-player
• #844236 filed against libhtml-lint-perl
• #844228 filed against ebook-speaker

Niko Tyni:

• #843432 filed against libwww-curl-perl

Reiner Herrman:

• #844095 filed against fdm
• #844096 filed against gringo
• #843469 filed against dpkg

Reviews of unreproducible packages 136 package reviews have been added, 5 have been updated and 7 have been removed in this week, adding to our knowledge about identified issues. 3 issue types have been updated:

• Added: random_order_in_dh_pythonX_substvars, valac_permutes_get_type_calls
• Updated: timestamps_in_static_libraries

Weekly QA work During of reproducibility testing, some FTBFS bugs have been detected and reported by:

• Chris Lamb (29)
• Niko Tyni (1)

diffoscope development A new version of diffoscope 62~bpo8+1 was uploaded to jessie-backports by Mattia Rizzolo. Meanwhile in git, Ximin Luo greatly improved speed by fixing a O(n²) lookup which was causing diffs of large packages such as GCC and glibc to take many more hours than was necessary. When this commit is released, we should hopefully see full diffs for such packages again. Currently we have 197 source packages which - when built - diffoscope fails to analyse. buildinfo.debian.net development

• Submissions with duplicate Installed-Build-Depends entries are rejected now that a bug in dpkg causing them has been fixed. Thanks to Guillem Jover.
• Add a new page for every (source, version) combination, for example diffoscope 62.
• DigitalOcean have generously offered to sponsor the hardware buildinfo.debian.net is running on.

tests.reproducible-builds.org Debian:

• For privacy reasons, the new dpkg-genbuildinfo includes Build-Path only if it is under /build. HW42 updated our jobs so this is the case for our builds too, so you can see the build path in the .buildinfo files.
• HW42 also updated our jobs to vary the basename of the source extraction directory. This detects packages that incorrectly assume a $pkg-$version directory naming scheme (which is what dpkg-source -x gives but is not mandated by Debian nor always-true) or that they're being built from a SCM.
• The new dpkg-genbuildinfo also records a sanitised Environment. This is different in our builds, so HW42, Reiner and Holger updated our jobs to hide these differences from diffoscope output.
• Package-set improvements:
  • Holger refactored the create_meta_pkg_sets job so that it's easier to add new package sets.
  • This job is now also using dose-extra from jessie-backports so that it can deal with versioned provides.
  • Added 4 new package sets: debian-edu, debian-edu_build-depends, maint_pkg-grass-devel, maint_debian-accessibility, maint_pkg-openstack.
  • Switched to using the new URL for tails manifests to generate the tails package set.
  • Renamed maint_lua to maint_debian-lua
• Valerie Young contributed four patches for our long-planned transition from SQLite to PostgreSQL.
• In anticipation of the freeze, already-tested packages from unstable and testing on amd64 are now scheduled with equal priority.

reproducible-builds.org website F-Droid was finally added to our list of partner projects. (This was an oversight and they had already been working with us for some time.) Misc. This week's edition was written by Ximin Luo and Holger Levsen and reviewed by a bunch of Reproducible Builds folks on IRC.

At the beginning of this year, Irene Soria invited me to start a series of talks on the topic of hacker ethics, security and surveillance. I presented a talk titled Cryptography and identity: Not everything is anonymity. The talk itself is recorded and available in archive.org (sidenote: I find it amazing that Universidad del Claustro de Sor Juana uses archive.org as their main multimedia publishing platform!) But as part of this excercise, Irene invited me to write a chapter for a book covering the series. And, yes, she delivered! So, finally, we will have the book presentation: I know, not everybody following my posts (that means... Only those at or near Mexico City) will be able to join. But the good news: The book, as soon as it is presented, will be published under a CC BY-SA license. Of course, I will notify when it is ready.

The past few weeks have been a whirlwind of work with my application process for Outreachy taking my full attention. When I got to know about Outreachy, I was intrigued as well as dubious. I had many unanswered questions in my mind. Honestly, I had that fear of failure which prevented me from submitting my partially filled application form. I kept contemplating if the application was good enough , if my answers were perfect and had the right balance of du-uh and oh-ah!
In moments of doubt, it is important to surround yourself with people who believe in you more than you believe in yourself. I ve been fortunate enough to have two amazing people, my sister Anjali, who is an engineer at Intel and my friend, Pranav Jain who completed his GSoC 16 with Debian. They believed in me when I sat staring at my application and encouraged me to click that final button. When I initially applied for Outreachy, I was given a task for building Lumicall and subsequent task was to examine a BASH script which solves the DNS-01 Challenge.
I deployed the DNS-01 challenge in Java and tested my solution against a server.
Within a limited time frame, I figured things out and wrote my solution in Java and then eagerly waited for the results to come out. Going through a full cycle of : I was elated with joy when I got to know I ve been selected for Outreachy to work with Debian. I was excited about open source & found the idea of working on the project open source fun because of the numerous possibilities of contributing towards a voice video and chat communication software.
My project mentor, Daniel Pocock, played a pivotal role in the time after I had submitted my application. Like a true mentor, he replied to my queries promptly and guided me towards finding the solutions to problems on my own. He exemplified how to feel comfortable with developing on open source. I felt inspired and encouraged to move along in my work. Beyond him, The MiniDebConf was when I was finally introduced to the Debian community. It was an overwhelming experience and I felt proud to have come so far.. It was pretty cool to see JitsiMeet being used for this video call. I was also introduced to two of my mentors , Juliana Louback & Bruno Magalh es . I am very excited to learn from them.

I am glad I applied for Outreachy which helped me identify my strengths and I am totally excited to be working with Debian on the project and learn as much as I can throughout the period. I am not a blog person, this is my first blog ever! I would love to share my experience with you all in the hopes of inspiring someone else who is afraid of clicking that final button!

This morning I gave a guest lecture to students at the EPSRC Centre for Doctoral Training in Cloud Computing for Big Data. The subject was a gentle introduction to Docker. This was the first guest lecture I've given for a couple of years so I thought I was a little rusty but I had a good time giving it and hopefully it went across OK. slides.pdf including speaker notes in boxes at the bottom of each slide; handouts.pdf (3 slides to a page with space for notes): demo steps.txt (the steps I followed for some of the demos). I mentioned a couple of things worth linking to here

• I referenced the architecture diagrams at https://www.docker.com/what-docker
• the Docker Hub, https://hub.docker.com/
• most of my demos used the wildfly image https://hub.docker.com/r/jboss/wildfly/
• github issue describing the Steam "delete all your files" bug
• Research paper A Framework for Scientific Workflow Reproducibility in the Cloud (PDF available)
• ikiwiki
• my work-in-progress Ikiwiki-in-a-box Docker image

There was some discussion about alternatives to Docker, things which were briefly mentioned include

• CoreOS, a stripped-down OS for containers
• the Rkt container manager (alternative for Docker daemon)
• the App Container Specification