The 12 member nations of the Trans-Pacific Partnership pact have agreed to prohibit demands that companies reveal software source code*, a step that appears aimed at curbing efforts by China to gain access to this sensitive information, The Yomiuri Shimbun has learned. Source code is the confidential information for software and is a blueprint embedded in many commonly used products, such as vehicles, mobile phones and home appliances. Source code is usually tightly guarded because it conatins commands that make using the software easier. China requires foreign companies operating there to hand over source code, a move that has sparked sharp criticism from many countries. Observers believe the decision by TPP participants to ban demands to reveal this code is intended to restrain China's move. The TPP's electronic commerce chapter in principle prohibits the 12 member nations from demanding access to source code for mass-produced software. According to the Economy, Trade and Industry Ministry, the Japan-Mongolia economic partnership agreement signed in February contains a stipulation banning demands for such information, but there are very few other examples around the world. Japan, the United States and other nations that are home to many information technology companies want to make the stipulation effectively a global standard, and they are considering whether to incorporate such a condition in economic partnership agreements that will be inked in the future. Source code is an important corporate secret for development firms. *Source code A software program written in a language a computer understands. As well as containing expert details about the unique functions of the product, the software is essential for fixing glitches and making improvements. Hackers have attempted to access source code because gaining this information would make it easier to create viruses that could exploit any software defects. In the software business, it is rarely made public, as it is considered a corporate secret.

Review: The Oathbound, by Mercedes Lackey

Series:	Vows and Honor #1
Publisher:	DAW
Copyright:	July 1988
ISBN:	0-88677-414-4
Format:	Mass market
Pages:	302

This book warrants a bit of explanation. Before Arrows of the Queen, before Valdemar (at least in terms of publication dates), came Tarma and Kethry short stories. I don't know if they were always intended to be set in the same world as Valdemar; if not, they were quickly included. But they came from another part of the world and a slightly different sub-genre. While the first two Valdemar trilogies were largely coming-of-age fantasy, Tarma and Kethry are itinerant sword-and-sorcery adventures featuring two women with a soul bond: the conventionally attractive, aristocratic mage Kethry, and the celibate, goddess-sworn swordswoman Tarma. Their first story was published, appropriately, in Marion Zimmer Bradley's Swords and Sorceress III. This is the first book about Tarma and Kethry. It's a fix-up novel: shorter stories, bridged and re-edited, and glued together with some additional material. And it does not contain the first Tarma and Kethry story. As mentioned in my earlier Valdemar reviews, this is a re-read, but it's been something like twenty years since I previously read the whole Valdemar corpus (as it was at the time; I'll probably re-read everything I have on hand, but it's grown considerably, and I may not chase down the rest of it). One of the things I'd forgotten is how oddly, from a novel reader's perspective, the Tarma and Kethry stories were collected. Knowing what I know now about publishing, I assume Swords and Sorceress III was still in print at the time The Oathbound was published, or the rights weren't available for some other reason, so their first story had to be omitted. Whatever the reason, The Oathbound starts with a jarring gap that's no less irritating in this re-read than it was originally. Also as is becoming typical for this series, I remembered a lot more world-building and character development than is actually present in at least this first book. In this case, I strongly suspect most of that characterization is in Oathbreakers, which I remember as being more of a coherent single story and less of a fix-up of puzzle and adventure stories with scant time for character growth. I'll be able to test my memory shortly. What we do get is Kethry's reconciliation of her past, a brief look at the Shin'a'in and the depth of Tarma and Kethry's mutual oath (unfortunately told more than shown), the introduction of Warrl (again, a relationship that will grow a great deal more depth later), and then some typical sword and sorcery episodes: a locked room mystery, a caravan guard adventure about which I'll have more to say later, and two rather unpleasant encounters with a demon. The material is bridged enough that it has a vague novel-like shape, but the bones of the underlying short stories are pretty obvious. One can tell this isn't really a novel even without the tell of a narrative recap in later chapters of events that you'd just read earlier in the same book. What we also get is rather a lot of rape, and one episode of seriously unpleasant "justice." A drawback of early Lackey is that her villains are pure evil. My not entirely trustworthy memory tells me that this moderates over time, but early stories tend to feature villains completely devoid of redeeming qualities. In this book alone one gets to choose between the rapist pedophile, the rapist lord, the rapist bandit, and the rapist demon who had been doing extensive research in Jack Chalker novels. You'll notice a theme. Most of the rape happens off camera, but I was still thoroughly sick of it by the end of the book. This was already a cliched motivation tactic when these stories were written. Worse, as with the end of Arrow's Flight, the protagonists don't seem to be above a bit of "turnabout is fair play." When you're dealing with rape as a primary plot motivation, that goes about as badly as you might expect. The final episode here involves a confrontation that Tarma and Kethry brought entirely on themselves through some rather despicable actions, and from which they should have taken a lesson about why civilized societies have criminal justice systems. Unfortunately, despite an ethical priest who is mostly played for mild amusement, no one in the book seems to have drawn that rather obvious conclusion. This, too, I recall as getting better as the series goes along and Lackey matures as a writer, but that only helps marginally with the early books. Some time after the publication of The Oathbound and Oathbreakers, something (presumably the rights situation) changed. Oathblood was published in 1998 and includes not only the first Tarma and Kethry story but also several of the short stories that make up this book, in (I assume) something closer to their original form. That makes The Oathbound somewhat pointless and entirely skippable. I re-read it first because that's how I first approached the series many years ago, and (to be honest) because I'd forgotten how much was reprinted in Oathblood. I'd advise a new reader to skip it entirely, start with the short stories in Oathblood, and then read Oathbreakers before reading the final novella. You'd miss the demon stories, but that's probably for the best. I'm complaining a lot about this book, but that's partly from familiarity. If you can stomach the rape and one stunningly unethical protagonist decision, the stories that make it up are solid and enjoyable, and the dynamic between Tarma and Kethry is always a lot of fun (and gets even better when Warrl is added to the mix). I think my favorite was the locked room mystery. It's significantly spoiled by knowing the ending, and it has little deeper significance, but it's a classic sort unembellished, unapologetic sword-and-sorcery tale that's hard to come by in books. But since it too is reprinted (in a better form) in Oathblood, there's no point in reading it here. Followed by Oathbreakers. Rating: 6 out of 10

What happened in the reproducible builds effort this week: Toolchain fixes

• Scott Kitterman uploaded python3-defaults/3.4.3-7 which changes py3versions to list versions in a consistent order. Issue reported by Santiago Vila with a tentative patch by Chris Lamb. Sadly, it appears the problem is not entirely solved.
• Martin Pitt uploaded init-system-helpers/1.24 which makes the order of unit files in maintainer scripts stable. Original patch by Reiner Herrmann.
• Niko Tyni uploaded libextutils-xsbuilder-perl/0.28-3 which makes the generated XS code reproducible.

Niko Tyni wrote a new patch adding support for SOURCE_DATE_EPOCH in Pod::Man. This would complement or replace the previously implemented POD_MAN_DATE environment variable in a more generic way. Niko Tyni proposed a fix to prevent mtime variation in directories due to debhelper usage of cp --parents -p. Packages fixed The following 119 packages became reproducible due to changes in their build dependencies: aac-tactics, aafigure, apgdiff, bin-prot, boxbackup, calendar, camlmix, cconv, cdist, cl-asdf, cli-common, cluster-glue, cppo, cvs, esdl, ess, faucc, fauhdlc, fbcat, flex-old, freetennis, ftgl, gap, ghc, git-cola, globus-authz-callout-error, globus-authz, globus-callout, globus-common, globus-ftp-client, globus-ftp-control, globus-gass-cache, globus-gass-copy, globus-gass-transfer, globus-gram-client, globus-gram-job-manager-callout-error, globus-gram-protocol, globus-gridmap-callout-error, globus-gsi-callback, globus-gsi-cert-utils, globus-gsi-credential, globus-gsi-openssl-error, globus-gsi-proxy-core, globus-gsi-proxy-ssl, globus-gsi-sysconfig, globus-gss-assist, globus-gssapi-error, globus-gssapi-gsi, globus-net-manager, globus-openssl-module, globus-rsl, globus-scheduler-event-generator, globus-xio-gridftp-driver, globus-xio-gsi-driver, globus-xio, gnome-control-center, grml2usb, grub, guilt, hgview, htmlcxx, hwloc, imms, kde-l10n, keystone, kimwitu++, kimwitu-doc, kmod, krb5, laby, ledger, libcrypto++, libopendbx, libsyncml, libwps, lprng-doc, madwimax, maria, mediawiki-math, menhir, misery, monotone-viz, morse, mpfr4, obus, ocaml-csv, ocaml-reins, ocamldsort, ocp-indent, openscenegraph, opensp, optcomp, opus, otags, pa-bench, pa-ounit, pa-test, parmap, pcaputils, perl-cross-debian, prooftree, pyfits, pywavelets, pywbem, rpy, signify, siscone, swtchart, tipa, typerep, tyxml, unison2.32.52, unison2.40.102, unison, uuidm, variantslib, zipios++, zlibc, zope-maildrophost. The following packages became reproducible after getting fixed:

• autoconf2.13/2.13-67 uploaded by Ben Pfaff, original patch by Santiago Vila.
• ding/1.8-3 by Roland Rosenfeld.
• dropbear/2015.68-1 by Guilhem Moulin. First set of patches (#777324, #793006) by Chris Lamb and akira.
• nvram-wakeup/1.1-3 by Tobias Grimm.
• original-awk/2012-12-20-5 by Santiago Vila.
• resiprocate/1:1.10.0-1 uploaded by Daniel Pocock, patch by Reiner Herrmann merged upstream.
• sbuild/0.66.0-5 by Johannes Schauer, reported by Jakub Wilk.
• scribus/1.4.5+dfsg1-4 by Mattia Rizzolo.
• sgmltools-lite/3.0.3.0.cvs.20010909-19 by Santiago Vila.
• unicode-data/8.0-2 uploaded by Alastair McKinstry, original patch by Esa Peuha.
• xmoto/0.5.11+dfsg-3 by Stephen Kitt.

Packages which could not be tested:

• mp4h/1.3.1-11 by Axel Beckert (shared memory issue on reproducible.debian.net).
• nvidia-graphics-drivers-legacy-304xx/304.128-5 by Andreas Beckmann (non-free).

Some uploads fixed some reproducibility issues but not all of them:

• libvirt/1.2.20-1 by Guido G nther.
• libgpars-groovy-java/1.2.1-4 by Emmanuel Bourg.

Patches submitted which have not made their way to the archive yet:

• #801520 on libapache2-mod-perl2 by Niko Tyni: sort the list of files used to build the documentation.
• #801523 on perl by Niko Tyni: sort the list of input files in PPPort_xs.PL. Forwarded upstream.
• #801864 on ncurses by Esa Peuha: use C locale when sorting the list of keys.
• #802042 on libchado-perl by Niko Tyni: sort keys in installed configuration file.

Lunar reported that test strings depend on default character encoding of the build system in ongl. reproducible.debian.net The 189 packages composing the Arch Linux core repository are now being tested. No packages are currently reproducible, but most of the time the difference is limited to metadata. This has already gained some interest in the Arch Linux community. An explicit log message is now visible when a build has been killed due to the 12 hours timeout. (h01ger) Remote build setup has been made more robust and self maintenance has been further improved. (h01ger) The minimum age for rescheduling of already tested amd64 packages has been lowered from 14 to 7 days, thanks to the increase of hardware resources sponsored by ProfitBricks last week. (h01ger) diffoscope development diffoscope version 37 has been released on October 15th. It adds support for two new file formats (CBFS images and Debian .dsc files). After proposing the required changes to TLSH, fuzzy hashes are now computed incrementally. This will avoid reading entire files in memory which caused problems for large packages. New tests have been added for the command-line interface. More character encoding issues have been fixed. Malformed md5sums will now be compared as binary files instead of making diffoscope crash amongst several other minor fixes. Version 38 was released two days later to fix the versioned dependency on python3-tlsh. strip-nondeterminism development strip-nondeterminism version 0.013-1 has been uploaded to the archive. It fixes an issue with nonconformant PNG files with trailing garbage reported by Roland Rosenfeld. disorderfs development disorderfs version 0.4.1-1 is a stop-gap release that will disable lock propagation, unless --share-locks=yes is specified, as it still is affected by unidentified issues. Documentation update Lunar has been busy creating a proper website for reproducible-builds.org that would be a common location for news, documentation, and tools for all free software projects working on reproducible builds. It's not yet ready to be published, but it's surely getting there. Package reviews 103 reviews have been removed, 394 added and 29 updated this week. 72 FTBFS issues were reported by Chris West and Niko Tyni. New issues: random_order_in_static_libraries, random_order_in_md5sums.

So, since several years I've had a second gpg key, 4096R/6AABE354. Several of you have already signed it, and I've been using it in Debian for some time already, but I've not announced it more widely yet, and I occasionally still get mail encrypted to the old key (which remains valid and usable, but it's 1024R). Of course, I've also made a formal transition statement:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
OpenPGP Key Transition Statement for Simon Richter
I have created a new OpenPGP key and will be transitioning away from
my old key.  The old key has not been compromised and will continue to
be valid for some time, but I prefer all future correspondence to be
encrypted to the new key, and will be making signatures with the new
key going forward.
I would like this new key to be re-integrated into the web of trust.
This message is signed by both keys to certify the transition.  My new
and old keys are signed by each other.  If you have signed my old key,
I would appreciate signatures on my new key as well, provided that
your signing policy permits that without re-authenticating me.
The old key, which I am transitioning away from, is:
pub   1024D/5706A4B4 2002-02-26
      Key fingerprint = 040E B5F7 84F1 4FBC CEAD  ADC6 18A0 CC8D 5706 A4B4
The new key, to which I am transitioning, is:
pub   4096R/6AABE354 2009-11-19
      Key fingerprint = 9C43 2534 95E4 DCA8 3794  5F5B EBF6 7A84 6AAB E354
The entire key may be downloaded from: http://www.simonrichter.eu/simon.asc
To fetch the full new key from a public key server using GnuPG, run:
  gpg --keyserver keys.gnupg.net --recv-key 6AABE354
If you already know my old key, you can now verify that the new key is
signed by the old one:
  gpg --check-sigs 6AABE354
If you are satisfied that you've got the right key, and the User IDs
match what you expect, I would appreciate it if you would sign my key:
  gpg --sign-key 6AABE354
You can upload your signatures to a public keyserver directly:
  gpg --keyserver keys.gnupg.net --send-key 6AABE354
Or email sr@simonrichter.eu (possibly encrypted) the output from:
  gpg --armor --export 6AABE354
If you'd like any further verification or have any questions about the
transition please contact me directly.
To verify the integrity of this statement:
  wget -q -O- http://www.simonrichter.eu/key-transition-2015-03-09.txt   gpg --verify
   Simon
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAEBAgAGBQJU/bbEAAoJEH69OHuwmQgRWOIH/AogHxhhVO5Tp5FFGpBFwljf
NzKTPBExMhZ/8trAzYybOWFv3Bx4AGdWkYfDnxP6oxQJOXVq4KL6ZcPPuIZuZ6fZ
bu0XHdPMU89u0TymR/WJENRCOcydRBe/lZs+zdJbKQtEZ+on4uNXxHpUiZPi1xxM
ggSdVBKn2PlCBcYih40S9Oo/rM9uBmYcFavX7JMouBSzgX78cVoIcY6zPRmHoq4k
TkGKfvHeSu+wbzWRmDwu/PFHRA4TKNvR6oeO+Et1yk454zjrHMXineBILRvvMCIA
t54pV6n+XzOUmtXcKnkIGltK+ZaJSV6am0swtx84RaevVXrknIQE8NvlA4MNgguI
nAQBAQIABgUCVP23tAAKCRDSx966V9+/u3j4BACVHifAcO86jAc5dn+4OkFZFhV1
l3MKIolL+E7Q7Ox+vJunGJJuGnOnazUK+64yDGZ2JxNJ4QNWD1FOs/Ng2gm82Vin
ArBtyp1ZGWUa+349X+1qarUQF9qAaUXDZjFp5Hzh/o6KC4t3eECxcb41og3LUTQD
VuG2KWNXYBe5P5ak9Q==
=o61r
-----END PGP SIGNATURE-----

I've been doing a little bit of dynamically typed programming in Haskell, to improve Propellor's Info type. The result is kind of interesting in a scary way. Info started out as a big record type, containing all the different sorts of metadata that Propellor needed to keep track of. Host IP addresses, DNS entries, ssh public keys, docker image configuration parameters... This got quite out of hand. Info needed to have its hands in everything, even types that should have been private to their module. To fix that, recent versions of Propellor let a single Info contain many different types of values. Look at it one way and it contains DNS entries; look at it another way and it contains ssh public keys, etc. As an migr from lands where you can never know what type of value is in a $foo until you look, this was a scary prospect at first, but I found it's possible to have the benefits of dynamic types and the safety of static types too. The key to doing it is Data.Dynamic. Thanks to Joachim Breitner for suggesting I could use it here. What I arrived at is this type (slightly simplified): So Info is a monoid, and it holds of a bunch of dynamic values, which could each be of any type at all. Eep! So far, this is utterly scary to me. To tame it, the Info constructor is not exported, and so the only way to create an Info is to start with mempty and use this function: The important part of that is that only allows adding values that are in the IsInfo type class. That prevents the foot shooting associated with dynamic types, by only allowing use of types that make sense as Info. Otherwise arbitrary Strings etc could be passed to addInfo by accident, and all get concated together, and that would be a total dynamic programming mess. Anything you can add into an Info, you can get back out: Only monoids can be stored in Info, so if you ask for a type that an Info doesn't contain, you'll get back mempty. Crucially, IsInfo is an open type class. Any module in Propellor can make a new data type and make it an instance of IsInfo, and then that new data type can be stored in the Info of a Property, and any Host that uses the Property will have that added to its Info, available for later introspection.

---

For example, this weekend I'm extending Propellor to have controllers: Hosts that are responsible for running Propellor on some other hosts. Useful if you want to run propellor once and have it update the configuration of an entire network of hosts. There can be whole chains of controllers controlling other controllers etc. The problem is, what if host foo has the property controllerFor bar and host bar has the property controllerFor foo? I want to avoid a loop of foo running Propellor on bar, running Propellor on foo, ... To detect such loops, each Host's Info should contain a list of the Hosts it's controlling. Which is not hard to accomplish: This is all internal to the module that needs it; the rest of propellor doesn't need to know that the Info is using used for this. And yet, the necessary information about Hosts is gathered as propellor runs.

---

So, that's a useful technique. I do wonder if I could somehow make addInfo combine together values in the list that have the same type; as it is the list can get long. And, to show Info, the best I could do was this: The resulting long list of the types of vales stored in a host's info is not a useful as it could be. Of course, getInfo can be used to get any particular type of value:

*Main> hostInfo kite
Info [InfoVal System,PrivInfo,PrivInfo,Controlling,DnsInfo,DnsInfo,DnsInfo,AliasesInfo, ...
*Main> getInfo (hostInfo kite) :: AliasesInfo
AliasesInfo (fromList ["downloads.kitenet.net","git.joeyh.name","imap.kitenet.net","nntp.olduse.net" ...

And finally, I keep trying to think of a better name than "Info".

I have worked with ARM based systems for longer than I care to admit to myself. From the Acorn Archimedes 305 in 1987 through to modern 64bit systems I have seen many many changes in the ARM community. One big change has been the rise of the inexpensive single board computer (SBC).

Arguably the Raspberry Pi (RPi) was responsible for starting this trend. Before RPi there were small development boards available, I was even involved in producing some of them, none of these really became a big thing and were principally vehicles for silicon vendors to showcase their SoC in an accessible way. When I say accessible I mean for silicon vendors who were previously used to charging many thousands of pounds for their development boards now only charging hundreds.

In my opinion the RPi was a complete disruption to the SBC market. In early 2012 a complete ARM computer system could now be purchased for $35 ( 25) which was substantially cheaper than the best contemporary competitor the Beaglebone $89 ( 60)

To be clear, the reason the RPi succeeded (millions sold, household name) was not on price alone but also the large amount of good supporting software and how easy it was made for teachers and makers to use.

There were several areas that the RPi managed to change perceived issues into opportunities for using what was already available or third parties to provide. There were also several issues raised about the original RPi:

• no case
• no integrated storage
• not having an x86 processor
• being a slow processor
• limited peripheral support
• no real time clock
• not supplying keyboards, mice
• not providing displays.

Some of these have been addressed since release as the foundation now sells cases, hardware revisions with much more powerful processors, pre-configured storage, cameras and displays. The important thing to note here though is the RPi has made the bare SBC a much more widely accepted product where all the non critical parts are considered "extras" and a lot is forgiven because of the price.

With that acceptance there have been many, many new SBC coming to market with better peripherals and increasingly competitive pricing. These are technically not clones as none of them use the Broadcom processor of the RPi but they often share many features and possibly even a compatible expansion header.

The Banana Pi was one of these copycats which I acquired for a similar price as an RPi in 2014. The main processor of this system was a 1Ghz dual core Allwinner A20 processor (a considerable advance on the 700MHz single core of the original RPi) coupled to a gigabyte of memory. Additionally the board benefited from having SATA and gigabit Ethernet MAC which made for a much more versatile system. Various third parties filled in the missing peripherals including my own attempt at a case.

I acquired a cubietruck for the NetSurf project to use as a build node in their CI system this is again based on the A20 but with more memory and somewhat better peripheral support but at a substantial cost over the Banana Pi.

The most recent addition to this form factor is the introduction by Xunlong Software of the Orange Pi PC. This little board is the same footprint as the RPi2 B+ design but with differing connector placement. The processor is a quad core 1.6GHz Allwinner H3 with a gigabyte of memory and has has Ethernet and USB but no SATA.

The big news about this board though is the price, at $15 ( 10) it resets the price expectations just as the RPi did before it. I was initially sceptical of the quality of the product (or if it would arrive at all) but I have acquired five of these boards and every one of them came well packaged, boxed and in a static bag, just like the RPi does, and they all worked.

I created a case design based on my RPi slimline case so they would be protected when piled up with all the other boards. The use of a DC barrel jack instead of micro USB for power is better in that the connector is more robust and intended for higher current draws but does mean additional leads are needed. There are, however, two flies in the ointment, neither are showstoppers but make the board a little more difficult to use.

One is a simple necessity of a substantial heatsink on the H3 processor. Initially I used a small 20 x 20mm copper heatsink (around 800mm square surface area) but this was insufficient under full load. I did not want to have to use a fan so I milled some slots into copper round bar, then cut off sections and faced them on a lathe. The completed design had more than 2200 square mm surface area and cost around $2.5 ( 1.5) in material (and a couple of hours in the workshop but that was fun and I made something)

The second issue and arguably much more serious is that of software. Let me be honest, it is dreadful, I mean very bad indeed. The images provided from the Orange Pi website are some of the worst examples of "do something quick" I have experienced.

Fortunately a user on the forums named loboris decided to create scripts that generate a Debian (and Ubuntu and Fedora) distribution images that can be installed from SD card. He relies on a somewhat patched 3.4 kernel full of Allwinner vendor changes and the inevitable binary blobs for the Marli GPU but the result does work.

I have had a few units acting as distcc compiler slaves for two weeks now at 100% CPU loading and they are still running. The processor does not get overly warm with the heatsink installed and Debian behaves just fine. The main caveat being that it is definitely not going to work if you try and update the kernel through packaging.

The state of software support and Xunlong Software relying on a forum user to complete their product does tarnish an otherwise impressive and possibly market changing SBC.

Perhaps I expect too much for fifteen bucks? I guess when the cost of the case, heatsink, cables and memory storage card is similar to the rest of the computer there is simply no margin left for anything else.

In conclusion hopefully this brief overview has provided some insight into what is available in this market and that the Raspberry Pi is indeed not the only option.

I finish with an image is of my ARM build farm consisting of every SBC I mention here (including a couple of RPi)

Note: 2015/10/16: I need to add some links but I won t delay this more, posting now, will edit later. Summer ended long time ago, but believe me, I m still catching up with all the things that I began in June/July, all the things I left in August when I went holidays, and more things that appeared in August and September. This is a long overdue post, I hope you bear with me for waiting so long, and writing (now) so long too! June In June, I was 100% sure that I would not attend DebConf15 (well, I was 98% sure until then), and when the new Outreach Sponsorship grants were announced, I decided to write some mails to several Debian contributors, so they consider applying for the grant and attend DebConf (and maybe trigger some i18n/l10n meeting ). They kindly declined, and I understood their reasons, but also wondered what would have happened if the proposal would have come from somebody more official instead of a random contributor that they don t know. I also hoped that lots of other Debianites also write to newbies or not-yet-DD-contributors or non-packaging contributors to invite them to DebConf, and I hoped that they had better luck than me in convincing them :) July In July I usually work hard preparing the computer labs for next academic year at my workplace in the University, but I also have more free time in the long afternoons and evenings, since I don t sleep much, and there is not much to do outside with the summer hot. So I used that month to go on contributing to DebConf publicity and think a bit more about Debian and the other free software communities. I didn t put much time in advancing my selfhosting (no SSL yet in *.larjona.net! booooo!) but I decided to deep my toe in Sandstorm.io, and try to selfhost an instance ( http://lacaja.larjona.net ) and try Etherpad inside Sandstorm (since I failed in deploying Etherpad by myself in my jessie+nginx+postgres box). Sandstorm worked, and Etherpad was packaged in Sandstorm so it worked too; and I have my free-software-base pads now for writing and share. So I joined #sandstorm IRC channel since then, and there I learnt that Asheesh Laroia (who works in Sandstorm.io and is also a Debian Developer and was going to give a talk about Sandstorm.io in DebConf15) was offering mentorship for people wanting to learn Sandstorm packaging, and his proposal was to begin packaging Framadate. I also failed in selfhosting Dudle (prepared for Apache + FastCGI, couldn t make it work in my Nginx), so Asheesh s proposal looked suitable for me. We talked and decided to invest the rest of July and first days of August in learning to package Framadate. I learned a lot, but couldn t finish the task. I encountered many issues (setting my dev environment, and later trying to package), and we solved some of them but my time ran out. I posted my work in the list, and I hope that my feedback on the documentation and the issues I encountered helped Asheesh and the Sandstorm community. Framadate is packaged in Sandstorm.io now, Drew Fisher packaged it, not sure if my stuff was useful or not (it s been useful for me, for learning, at least). I ll talk more about Sandstorm.io in a future blog post updating on my selfhosting adventures. What I liked most was the kind of proposal of mentoring that Asheesh made. It was very detailed in every aspect: the task, the things you need to accomplish it, details about his availability for mentorship I try to be welcoming in the teams in which I participate, but the fact is that I fail in actually mentor, maybe because of not making specific proposals to people (until now, I was like Hi, newcomer! Go read this, this and this, and try for yourself any task you feel you like it, and come back if you have issues , la Debian ). This, plus the thoughts about my mails in June for diversity outreach in DebConf, made me feel the need of having a team where people willing to welcome newcomers share tricks and procedures, write together more specific proposals, and follow up the newcomers experiences in a regular way. I talked with Enrico Zini and we wrote down some notes for a Welcome Team in Debian; he said he would spread the word during DebCamp/DebConf and we would see what people thinks about it. August August came, and the day before going on holidays I was really tired: too much luggage to prepare, too many hours in front of the computer, and the usual stress of traveling; and I took the bad decision of signing some GPG keys of several Debianites that I met in July. I say bad decision because the lack of sleep showed its black magic and I accidentally deleted my secring.gpg file. I knew I had a backup but I didn t have too much time to invest and I didn t want to mess it with the backup too, and my laptop was going to stay at home, powered off, during the whole month, so I just went on holidays and left the GPG issue for later. The day after, meanwhile I was waiting in the airport for my boarding time, I received a mail accepting me as Debian Developer. Wow!! Really, I was not expecting that the process was already finished, I had interchanged several mails with my Application Manager (who happens to be the current DPL!) and I thought that his summer could be quite packed of Debian/DebConf work and my process could wait a bit. So it was a very happy news and very motivating after one month (July) full of free software work. On the other side, I was a bit scared: what type of Debian Developer are you, larjona, not capable to sign some GPG keys without breaking your setup?! but I answered myself well, I m the type of Debian Developer that has backups :) and then, with that mixed feelings of excitement and impostor syndrome, I took my plane and went on holidays, not expecting to touch any computer until the end of the month. August is probably the month in the year when I have more free time (holidays), but less time to dedicate to free software. I devote most of the month to visit family and stay with them, with no internet connection available or no free time to look at the mailbox or social networks or IRC But DebCamp and DebConf15 were happening during my holidays. And this DebConf15 was the first one in which I participated in the organization, and the first one in which I felt more than being a consumer of Debian videos . I could not follow the streamings, my only internet-capable device was my Android 2.x phone, but when I had wifi I fetched the mail, and during the nights, while everybody else was sleeping and I was laying on the terrace, below the sky full of stars, I could read batches of hundred of mails from debconf-discuss mailing list. And I could get some feeling from DebConf life, because I learned about the ad-hoc BoFs and discussions, the morning bike rides and swimming proposals, and the dancing classes, the i18m/l10n meeting, and many other things. I could answer some mail from time to time, and I also knew that a fellow Debianite from Madrid was going to bring me some stickers, maybe a t-shirt, and shake hands in my name to some persons. September and October September was about finishing reading all the mails and try to answer the pending ones, and preparing my computer to use my new Debian identity (and stop using larjona-guest). I still have some things to do, pending technical work, and some mails that I should have answered and I ve forgotten, for sure (if you sent me a mail that needs answer or would be fine that I answer (even if it was months ago!), please resend or ping me). I recovered my secring.gpg but and just now I added larjona@debian.org to the ID in my GPG key, but didn t signed the pending keys again (sorry dkg and holger! will catch up there soon). My subkeys expired and I m trying to find out how to proceed (they are in my FSFE SmartCard) :/ About the Debian teams, I ve resumed my work in publicity team (this year I ll try to be more involved, in Debian Project News in particular), partially in the website team, and recently I ve finished catching up with the Spanish translation of the website. I ve also joined the DebConf team again (for DebConf16, no matter I probably won t attend) and documented the Publicity task for DebConf, and I try to engage the mailing list and the IRC meetings. I finally could have time to watch some DebConf15 videos and Andreas Tille s talk ( Creating a more inviting environment for newcomers New experiences from MoM, SoB, Teammetrics ) helped me to step ahead in welcoming people with more useful stuff than Hi, newcomer! Go read this (general URLs), try for yourself whatever you like . I have made specific proposals for two people. In mid September I accepted an interview about Debian for a podcast with quite a lot audience (in Spanish), in which I explained the idea of the Welcome Team and offered myself as first-contact. Since then, two more people have contacted me and I have offered specific tasks I think are suitable for them. I also try to be more available in the IRC and offer some time spans for new contributors to DebConf to explain the git setup, the wiki, and all this stuff that looks more complicated than what it is. And I think that s all. My Debianite friend kindly brought me some stickers and a DebConf t-shirt, plus the organization t-shirt that the team gave me as present for my contributions in DebConf15. Neil McGovern kindly sent me a certificate of my new Debian Developer status (thanks!!), and it s posted in my wall at work. Here you are a photo! (Note: my wall is full of stickers and pieces of papers with things I need, things I like and things I use to explain my work (sometimes sarcastically/ironically ). Maybe some day I ll make a blog post about that!) I feel very proud and happy. Still, a lot of things to learn and work to do, but my intentions are: to keep on progressing (sometimes fast, sometimes slowly), never give up, and enjoy the multiple flowers I find in my way :) Thanks everybody! October and future Some other ideas/plans for the future (the ones I didn t say yet):

• Try to catch up with Debian Contributors development/news, adopt some data source (www.debian.org?), try to create some new data source (about translators, for example).
• Clean spam in the archive lists (now that I am DD I may help in not only Mark as spam , but actually remove the spam from the archive)
• Help in the review/updates of the Debian website (ongoing: /users; future: children-distros/derivatives, /partners).
• Try to get more involved in the Debian i18n team, at least in the part of internationalizing the Debian infrastructure (make translatable via PO files/templates many texts that used repetitively in the Debian website, for example) and in welcoming new members in translation teams or helping the weak ones (although I don t know many languages, the workflow and tools are common, so I think I could help new translators at least to get hands on the matter).
• Maybe some short screencasts/videos about the Debian infrastructure, for newcomers? (mailing lists, IRC, wiki, Alioth )

Comments? If you want to comment you can use this pump.io thread.
Filed under: My experiences and opinion, News Tagged: Communities, Contributing to libre software, Debian, Developer motivations, encryption, English, Free Software, gpg, libre software

Two years ago, I discussed various ways of constructing a list in a Monad (or, more specifically, in IO) in Haskell, and compared their merits in terms of elegance, stack usage, number of traversals of the list and run-time efficiency. Recently, two blog posts discussed the issue further and proposed new, more daring alternatives. Neil Mitchell breaks through the abstraction provided by IO, duplicates the world and traverses the list twice, and obtains a speed-up for long lists. Twarn van Laarhoven went even further and wrote custom C-- code to destructively update the tail-pointer of the list cell to be able to create the list completely evaluated on the first start. This basically answers my question from two years ago:

I m still wondering: What would be required from Haskell, GHC or the monads in question to have a fully satisfactory solution here, i.e. one that is as fast as the naive recursion, as stack efficient as the difference list solution, and allocates no thunks, only list cells?

He also has a variant with a slightly nicer interface around holes , i.e. explicit objects on the heap that can later be replaced by indirections. Obviously, both approaches are very unsafe. I took this as an opportunity to redo my benchmark measurements, and include their variants (named escapeIO, hackIO and holeIO). The following table omits the variants with quadratic performance, as I ran it on longer lists now:

Variant	10^0	10^1	10^2	10^3	10^4	10^5	10^6
accumReverse	37ns	153ns	1134ns	12 s	208 s	8540 s	97ms
recursion	29ns	139ns	680ns	6790ns	160 s	6441 s	76ms
replicateM	26ns	126ns	677ns	6785ns	168 s	6314 s	78ms
accumDList	35ns	165ns	995ns	10 s	190 s	9706 s	100ms
streams	27ns	136ns	691ns	6788ns	173 s	5771 s	75ms
unsafeInterleave	60ns	329ns	2804ns	28 s	373 s	5605 s	57ms
listFix	51ns	412ns	4109ns	56 s	2761 s	42ms	445ms
escapeIO	41ns	187ns	1808ns	16 s	234 s	4409 s	45ms
hackIO	30ns	152ns	1199ns	11 s	140 s	3701 s	42ms
holeIO	40ns	222ns	1725ns	17 s	218 s	4446 s	53ms

The following graph shows that around 10000, the naive approaches become much slower and the fancy hacks pay of, with Twarn s tail-pointer-updating code performing the best: I would really like to see a package that provides a API like Twarn s holes, either in this raw unsafe variant (but with the garbage collector related code checked), or with a safe API using type hackery similar to the ST monad that ensures that after normal code gets its hands on a term possibly involving holes, the holes may no longer be modified. I have put the code and results on GitHub.

Now that the tsunami that followed my recent blog post has passed, I can read through all the nice comments and blogs of other people. I didn t expect that just writing up some links to posts on LKML, plus adding my personal opinion on what has happened and about my interpretation concerning Debian could gather so much attention. Anyway, let us look at the blog posts written in the aftermath. In particular do I want to answer to two blogs: Lars Wirzenius blog and Mart n Ferrari s blog, both of which are excellent examples of insults. I also want to mention that although the previous two are bad examples, there are fortunately also those who are able to resist primitive insults, in particular Gunnar Wolf (direct link to the blog does not work), and Miriam Ruiz blog, which contradict my blog and propose a different view, without pulling the knife. Lars Wirzenius Let us start with Lars (ah sorry, I have to call him Wirzenius here, strange custom has suddenly popped up in Debian!):

I think Preining mis-characterises what happens on the Linux kernel mailing list, and in Debian, in free software development in general, and what Sarah Sharp has said and done. I continue to think he is wrong about everything on this topic.

I am very surprised that Lars^WWirzenius is in possession of a crystal ball that allows him to see and evaluate my attitude towards all these items, considering that I have presented links to posts on the LKML, and yes, my interpretation of the matter. Mind that the whole text amounts to about 130 lines on my screen, while my personal opinion was stated in two paragraphs of total 9 lines plus some interspersed comments. And in these maybe 20 lines I didn t make general statements about open source, kernel development etc etc. Lars, I know you have closed your blog for comments so I couldn t ask you but please, can you send me one of your crystal balls if you have more of them? At least he has managed to keep a bit of proper writing and disagreed with my statement on Debian (whether there is fun or not in Debian). He is of course free to do that, but please, don t rob me the right to state that I think Debian has changed. All this dispute centers around people not being capable to distinguish two things: One, being against the Code of Conduct due to the inclusion of administrative actions without clear definitions, and Two, being pro offensive behavior and and insults. Now, dear Lars^WWirzenius, please listen: I never advocated abusive behavior or insults, nor do I defend it. (Did you hear that!) I simply opposed the Code of Conduct as ruling instrument. And what kind of emails I got due to my opposition was far outside the Code of Conduct you are so strongly defending. So please, stay at the facts, and stop insulting me. Thanks. Mart n Ferrari Concerning Mart n I don t have much to say but please, stop spreading lies. You stated:

Once again, he s complaining about how the fun from Debian has been lost because making sexist jokes, or treating other people like shit is not allowed any more.

Could you please come up with a reference to this? Or are you just interpreting? I am very disappointed about this level of discussion between Debian Developers. You not even cared to answer my comment on your blog. Should I say something clear here you should be happy that this has not been written on a Debian mailing list, otherwise I hope the Code of Conduct hammer would hit you.

---

So yes, it seems that at least these two blogs underline exactly what my opinion is: Communication culture in Debian has changed to be more company-like. Probably due to the ever increasing amount of developers that are paid for their work on Debian (Ubuntu), the pressure to follow US company codes has taken a firm grip, so firm that even stating my diverging opinion is already enough to get branded. Good Future!

What happened in the reproducible builds effort this week: Toolchain fixes

• Niels Thykier uploaded debhelper/9.20151004 which exports SOURCE_DATE_EPOCH when running dh_auto_* (patch by Dhole). dh now also calls dh_strip_nondeterminism during build (patch by Andrew Ayer). The only remaining change regarding reproducibility in the custom debhelper is related to mtimes in data.tar which might be fixed directly in dpkg instead.
• Niels Thykier made another upload of debhelper/9.20151005 the next day to sort Build IDs added by dh_strip in control files.

Scott Kitterman fixed an issue with non-deterministic Depends generated by dh-python identified by Santiago Vila and Chris Lamb. Lunar updated the patch against dpkg which makes the order of files in control.tar.gz deterministic using the new --sort=name option available in GNU Tar 1.28. josch released sbuild version 0.66.0-1 with several fixes and improvements. The most notable one for reproducible builds is the new --build-path option and $build_path configuration variable added by akira which allows to explicitly chose a given build path. Reiner Herrmann wrote a new patch for dh-systemd to sort the list of unit files in the generated maintainer scripts. Packages fixed The following packages became reproducible due to changes in their build dependencies: aoeui, apron, camlmix, cudf, findlib, glpk-java, hawtjni, haxe, java-atk-wrapper, llvm-py, misery, mtasc, ocamldsort, optcomp, spamoracle. The following packages became reproducible after getting fixed:

• classified-ads/0.08-1 uploaded by Antti J rvinen, fix merged upstream, original patch by Reiner Herrmann.
• criu/1.7-3 uploaded by Salvatore Bonaccorso, original patch by Chris Lamb.
• doomsday/1.15.4-1 by Michael Gilbert.
• ejabberd/15.07-1~exp1 by Philipp Huebner.
• libxbean-java/4.4-1 by Emmanuel Bourg.
• markdown/1.0.1-8 uploaded by Matt Kraai, original patches by Chris Lamb (#776925) and akira (#793701).
• nedit/1:5.6a-3 by Paul Gevers.
• nvidia-settings/340.93-1 uploaded by Andreas Beckmann, original patch by Lunar.
• ocaml/4.02.3-2 by St phane Glondu.
• polygen/1.0.6.ds2-14 uploaded by Mehdi Dogguy, original patch by Chris Lamb.
• stsci.distutils/0.3.7-4 by Aurelien Jarno.
• vdr/2.2.0-4 by Tobias Grimm.
• vdr-plugin-xineliboutput/1.1.0+cvs20150907-3 by Tobias Grimm.
• w3c-markup-validator/1.3+dfsg-3 by Santiago Vila.
• xcolorsel/1.1a-19 by Santiago Vila.
• xmoto/0.5.11+dfsg-3 by Stephen Kitt.

Some uploads fixed some reproducibility issues but not all of them:

• ognl/2.7.3-6 by Emmanuel Bourg.
• enblend-enfuse/4.1.4+dfsg-2 by Andreas Metzler.

Untested

• nvidia-xconfig/340.93-1 by Andreas Beckmann.
• nvidia-settings-legacy-304xx/304.128-1 by Andreas Beckmann.

Patches submitted which have not made their way to the archive yet:

• #801212 on unicode-data by Esa Peuha: set TZ=UTC when calling unzip.

reproducible.debian.net ProfitBricks once again increased their support for reproducible builds in Debian and in other free software projects by adding 58 new cores and 138 GiB of RAM to the already existing setup. Two new amd64 build nodes and 16 new amd64 build jobs have been added which doubles the build capacity per day and allows us to spot many kind of problems earlier. The size of the tmpfs where builds are performed has also been increased from 70 to 200 GiB on all amd64 build nodes. Huge thanks! When examining a package, a link now points to a table listing all previous recorded tests for the same package. (Mattia) The menu on the package pages has also been improved. (h01ger) Packages in the depwait state are now rescheduled automatically after five days. (h01ger) Links to documentation and other projects being tested have been made more visible on the landing page. (h01ger) To reduce noise on the team IRC channel five different types of notifications have been turned into mail notifications. The remaining ones have been shortened and the status changes have been limited to unstable and experimental. (h01ger) Maintainer notifications about status changes in a package will only be sent out once per day, and not on each status change. (h01ger) diffoscope development Some more experiments of concurrent processing have been made. None were good and reliable enough to be shared, though. Package reviews 48 reviews have been removed, 189 added and 23 updated this week. 9 FTBFS bugs were reported by Chris Lamb. Misc. h01ger met with Levente Polyak to discuss testing Arch Linux on Debian continuous test system with an easily extensible framework. The idea is to also allow testing of other distributions, and provide a nice package based view like the one for Debian.

We got pretty good feedback on Gnocchi so far, even if we only had little. Recently, in order to have a better feeling of where we were at, we wanted to know how fast (or slow) Gnocchi was. The early benchmarks that some of the Mirantis engineers ran last year showed pretty good signs. But a year later, it was time to get real numbers and have a good understanding of Gnocchi capacity. Benchmark tools The first thing I realized when starting that process, is that we were lacking of tools to run benchmarks. Therefore I started to write some benchmark tools in python-gnocchiclient, which provides a command line tool to interrogate Gnocchi. I added a few basic commands to measure metric performance, such as:
The command line tool supports the --verbose switch to have detailed progress report on the benchmark progression. So far it supports metric operations only, but that's the most interesting part of Gnocchi. Spinning up some hardware I got a couple of bare metal servers to test Gnocchi on. I dedicated the first one to Gnocchi, and used the second one as the benchmark client, plugged on the same network. Each server is made of 2 Intel Xeon E5-2609 v3 (12 cores in total) and 32 GB of RAM. That provides a lot of CPU to handle requests in parallel. Then I simply performed a basic RHEL 7 installation and ran devstack to spin up an installation of Gnocchi based on the master branch, disabling all of the others OpenStack components. I then tweaked the Apache httpd configuration to use the worker MPM and increased the maximum number of clients that can sent request simultaneously. I configured Gnocchi to use the PostsgreSQL indexer, as it's the recommended one, and the file storage driver, based on Carbonara (Gnocchi own storage engine). That means files were stored locally rather than in Ceph or Swift. Using the file driver is less scalable (you have to run on only one node or uses a technology like NFS to share the files), but it was good enough for this benchmark and to have some numbers and profiling the beast. The OpenStack Keystone authentication middleware was not enabled in this setup, as it would add some delay validating the authentication token. Metric CRUD operations Metric creation is pretty fast. I managed to attain 1500 metric/s created pretty easily. Deletion is now asynchronous, which means it's faster than in Gnocchi 1.2, but it's still slower than creation: 300 metric/s can be deleted. That does not sound like a huge issue since metric deletion is actually barely used in production. Retrieving metric information is also pretty fast and goes up to 800 metric/s. It'd be easy to achieve very higher throughput for this one, as it'd be easy to cache, but we didn't feel the need to implement it so far. Another important thing is that all of these numbers are constant and barely depends on the number of the metric already managed by Gnocchi.

Operation	Details	Rate
Create metric	Created 100k metrics in 77 seconds	1300 metric/s
Delete metric	Deleted 100k metrics in 190 seconds	524 metric/s
Show metric	Show a metric 100k times in 149 seconds	670 metric/s

Sending and getting measures Pushing measures into metrics is one of the hottest topic. Starting with Gnocchi 1.1, the measures pushed are treated asynchronously, which makes it much faster to push new measures. Getting new numbers on that feature was pretty interesting. The number of metric per second you can push depends on the batch size, meaning the number of actual measurements you send per call. The naive approach is to push 1 measure per call, and in that case, Gnocchi is able to handle around 600 measures/s. With a batch containing 100 measures, the number of calls per second goes down to 450, but since you push 100 measures each time, that means 45k measures per second pushed into Gnocchi! I've pushed the test further, inspired by the recent blog post of InfluxDB claiming to achieve 300k points per second with their new engine. I ran the same benchmark on the hardware I had, which is roughly two times smaller than the one they used. I achieved to push Gnocchi to a little more than 120k measurement per second. If I had same hardware as they used, I could interpolate the results to achieve almost 250k measures/s pushed. Obviously, you can't strictly compare Gnocchi and InfluxDB since they are not doing exactly the same thing, but it still looks way better than what I expected. Using smaller batch sizes of 1k or 2k improve the throughput further to around 125k measures/s.

Operation	Details	Rate
Push metric 5k	Push 5M measures with batch of 5k measures in 40 seconds	122k measures/s
Push metric 4k	Push 5M measures with batch of 4k measures in 40 seconds	125k measures/s
Push metric 3k	Push 5M measures with batch of 3k measures in 40 seconds	123k measures/s
Push metric 2k	Push 5M measures with batch of 2k measures in 41 seconds	121k measures/s
Push metric 1k	Push 5M measures with batch of 1k measures in 44 seconds	113k measures/s
Push metric 500	Push 5M measures with batch of 500 measures in 51 seconds	98k measures/s
Push metric 100	Push 5M measures with batch of 100 measures in 112 seconds	45k measures/s
Push metric 10	Push 5M measures with batch of 10 measures in 852 seconds	6k measures/s
Push metric 1	Push 500k measures with batch of 1 measure in 800 seconds	624 measures/s
Get measures	Push 43k measures of 1 metric	260k measures/s

What about getting measures? Well, it's actually pretty fast too. Retrieving a metric with 1 month of data with 1 minute interval (that's 43k points) takes less than 2 second. Though it's actually slower than what I expected. The reason seems to be that the JSON is 2 MB big and encoding it takes a lot of time for Python. I'll investigate that. Another point I discovered, is that by default Gnocchi returns all the datapoints for each granularities available for the asked period, which might double the size of the returned data for nothing if you don't need it. It'll be easy to add an option to the API to only retrieve what you need though! Once benchmarked, that meant I was able to retrieve 6 metric/s per second, which translates to around 260k measures/s. Metricd speed New measures that are pushed into Gnocchi are processed asynchronously by the gnocchi-metricd daemon. When doing the benchmarks above, I ran into a very interesting issue: sending 10k measures on a metric would make gnocchi-metricd uses up to 2 GB RAM and 120 % CPU for more than 10 minutes. After further investigation, I found that the naive approach we used to resample datapoints in Carbonara using Pandas was causing that. I reported a bug on Pandas and the upstream author was kind enough to provide a nice workaround, that I sent as a pull request to Pandas documentation. I wrote a fix for Gnocchi based on that, and started using it. Computing the standard aggregation methods set (std, count, 95pct, min, max, sum, median, mean) for 10k batches of 1 measure (worst case scenario) for one metric with 10k measures now takes only 20 seconds and uses 100 MB of RAM 45 faster. That means that in normal operations, where only a few new measures are processed, the operation of updating a metric only takes a few milliseconds. Awesome! Comparison with Ceilometer For comparison sake, I've quickly run some read operations benchmark in Ceilometer. I've fed it with one month of samples for 100 instances polled every minute. That represents roughly 4.3M samples injected, and that took a while almost 1 hour whereas it would have taken less than a minute in Gnocchi. Then I tried to retrieve some statistics in the same way that we provide them in Gnocchi, which mean aggregating them over a period of 60 seconds over a month.

Operation	Details	Rate
Read metric SQL	Read measures for 1 metric	2min 58s
Read metric MongoDB	Read measures for 1 metric	28s
Read metric Gnocchi	Read measures for 1 metric	2s

Obviously, Ceilometer is very slow. It has to look into 4M of samples to compute and return the result, which takes a lot of time. Whereas Gnocchi just has to fetch a file and pass it over. That also means that the more samples you have (so the more time you collect data and the more resources you have), slower Ceilometer will become. This is not a problem with Gnocchi, as I emphasized when I started designing it. Most Gnocchi operations are O(log R) where R is the number of metrics or resources, whereas most Ceilometer operations are O(log S) where S is the number of samples (measures). Since is R millions of time smaller than S, Gnocchi gets to be much faster. And what's even more interesting, is that Gnocchi is entirely scalable horizontally. Adding more Gnocchi servers (for the API and its background processing worker metricd) will multiply Gnocchi performances by the number of servers added. Improvements There are several things to improve in Gnocchi, such as splitting Carbonara archives to make them more efficient, especially from drivers such as Ceph and Swift. It's already on my plate, and I'm looking forwarding to working on that! And if you have any questions, feel free to shoot them in the comment section.

I m currently on the way home, returning from a four-day workshop for high school students; ages 13 to 20, grade 9 to 13. My goal was to teach them something about proofs and logic, in a ambitiously formal and abstract way. For this workshop, I created a computer program called The Incredible Proof Machine that allows the kids to explore creating a proof, by placing blocks on a canvas and drawing connections between them (see my previous blog post for an introduction). Subtracting time spent on on breaks, organizational stuff, the time the student needed to prepare a final presentation, I think we were working for a total of 14 hours, during which we covered propositional logic. I generally let the students explore and try to solve the tasks of one session mostly on their own, followed by a common discussion of what they have just done, what it means why it makes sense etc. The sessions were: assumptions and conclusions in general, with conjunction; implication; disjunction; falsum and tertium non datur. We also briefly discussed paper proofs with the student: how they look, and how they relate to the Proof Machine proofs. We had some lecture notes that we handed out pice-wise after each session. The sections were mildly¹ password-protected to avoid that the quicker students would work ahead, thus keeping the group together. One or two of the 13 students were doing very well and eventually, I gave them the passwords to the predicate logic section and let them work them on them on their own. The quickest managed to solve almost all of these as well, but (as far I as I can tell) without a deeper understanding of the quantifiers, and more a mechanical intuition. As expected, the students were able to solve most of the exercises, even when a proper understanding of the logical semantics was not yet fully developed. This was by design: I believe that this way it was more motivating and encouraging, as they could make it work , compared to a more traditional approach of first throwing a lot of theory at them and then expecting them to apply it. This was confirmed by their feedback after the workshop. I was happy with my implementation. The students immediately could work with it with very few hick-ups, and only one minor bug occurred², which I could fix on the spot. Having everything run on in the browser was a big plus, given that we had no stable internet connection for everyone: Once the Incredible Proof Machine was loaded, the student could continue to work offline. Personally, I find that the UI is occasionally too sluggish, especially on the weaker laptops, but it seems that the students did not seem to mind. Some students tried to connect outputs with outputs or inputs with inputs and the visualization did make it clearly visible that such a link is not actually connected to the block. The predicate logic part is a bit less convincing, with e.g. scoping of local constants not easily understood. I would say that this part would work better if some explanation is given before the students start working on the more involved rules. Our room was equipped with a smartboard, and I was delighted when I found out, mostly by accident, that I could actually use my finger to drag the blocks of the proof and to draw connections. This not only gave me a Minority Report -like feeling, but also meant that it was much easier for the students to follow my explanations when they could just watch my hand, instead of trying to locate the mouse pointer on a regular projector. I m generally doubtful whether such fancy technological toys are useful in the class room, but in this case at least I liked it. The ability to scribble anywhere on the screen was occasionally a plus as well. All in all I believe the Proof Machine was a useful tool, and I am sure that without it, it would have been tricky to have students voluntarily spend 14 hours on such a relatively dry and abstract topic. Given the amount of work that went into the development, I hope that this will not be the last occasion where it is put to good use. So if you have to teach formal logic and natural deduction-style proofs, you are welcome to use the Incredible Proof Machine to get your students excited. It is purely static, i.e. needs no special server-side infrastructure, and you can define your own logic (i.e. proof rules), sessions and tasks. Also, there are a few interesting way in which the Proof Machine could be extended. In particular, I d like it to be able to generate a normal , natural-language proof from a given proof even if it will sound a bit mechanical and then use hover-highlight effects to relate the formulas and sentences in the text proof to the connections and blocks in the graphical proof. Contributions are welcome!

I disagree with pretty much everything Norbert Preining says in his blog post about Sarah Sharp in response to Sharp's blog post Closing a door about leaving Linux kernel development. I think Preining mis-characterises what happens on the Linux kernel mailing list, and in Debian, in free software development in general, and what Sarah Sharp has said and done. I do not wish to participate in yet another discussion on the Debian Code of Conduct, respect, appropriate behaviour, or whether these things result in worse software. I won't spend any further of my time on debating, point-by-point, the arguments and straw-men Preining makes. Those discussions have happened often enough, without any effect on Preining and those who think like him. Likewise, despite the many times he makes the arguments, I continue to think he is wrong about everything on this topic. I do feel it is important to make it clear to the people reading Planet Debian, where both Preining's and my blogs are published, that his opinions are not mainstream in the Debian project, and that despite what he says, Debian development continues to be fun.

I got home from a law lecture[1] today and took apart my TV. As you do. I d a couple of motivations for this. The TV is an LG 55LM8600-UC. That -UC is important - it means it s the US model. I shipped it back to the UK along with the rest of my belongings at the start of the year. It s a lovely TV, and I m very happy with the picture quality. Up until now I ve been running it through a 240v->120v transformer, because the back clearly states 120v 50/60Hz. Having had success converting my Roomba PSU to UK voltage I was wondering if it might be easy enough to do the same for the TV. So the first reason for taking it apart was to look at the PSU board and work out if I could replace it with a UK version, or fix up a handful of components myself. Turns out its rated for 100-240v 50/60Hz so it looks like it should be fine to plug into UK mains directly. And, having done so, it all seems fine. However I don t take any responsibility if you try this yourself and it all goes up in smoke. The other reason was to have a general poke at trying to get access to the onboard CPU - the firmware updates are signed and encrypted so I m a bit stuck working from just those. My TV is from the US; it therefore has an ATSC tuner, and is no good for picking up the DVB-T/T2 broadcast in the UK. It also has a range of smart apps built in. Of course are also tailored to the US, so the Amazon Instant Video app connects to amazon.com and is no use with my Amazon UK account. Netflix is smarter about this and detects where you re located rather than requiring logging in to the appropriate geographic site. However the Netflix app is old and doesn t support user profiles. Which causes a problem when I spend an afternoon watching Unbreakable Kimmy Schmidt and it screws up the recommendations for my girlfriend. In an ideal world I d easily be able to update the apps on my TV to recent versions appropriate for my geographic location. Instead, the current state of home entertainment software platforms is at least 10 years behind mobile phones. Some devices allow you to get hold of SDKs even as an end user, but you re stuck with a multiple ecosystems even between different models from the same manufacturer. LG has no interest in updating my 2012-era TV. They d rather I buy a nice new 2015 TV. Which means at present my TV is mostly a really nice monitor for my XBMC/Kodi box. I can get away with using my Blu-ray player for Netflix, as it does user profile, but it s also a US model and again Amazon Instant Video doesn t work with a UK account. I should point out /that/ box is from Panasonic, just in case anyone thinks I m needlessly picking on LG. My amp is an Onkyo and it would be nice if it could stream music from Amazon/Google as well (it supports Spotify and a couple of others already), but again no joy there. I understand there are DRM issues around video and audio content, but these problems seem to have largely been solved in the Android world while still providing a fairly standard development environment and allowing end users to install their own software. It would seem to make more sense if there was a standard TV / Blu-ray platform that the likes of Netflix and Amazon could write apps once for, reducing the effort required on both sides to get the latest and greatest services on end user equipment. I guess this is some of what Google was trying to achieve with Google TV, but it s probably less interesting to them than mobile devices as I certainly don t tend to do things like search on my TV; I just want to view content and would be mighty pissed off if it started advertising things at me unexpectedly. While I m not planning to buy a new TV any time soon, I could do with a UK spec 3D-capable Blu-ray player. Is there anything out there with some commitment to open apps / proper updates? I haven t found anything, but I d love to be proved wrong. [1] It was on the duty of care and negligence, if you care.

Today, Norbert Preining wrote a blog post syndicated in planet talking about Sarah Sharp quitting Linux development, and making a parallel with the Debian community. Once again, he's complaining about how the fun from Debian has been lost because making sexist jokes, or treating other people like shit is not allowed any more. He seems to think the LKML is the ideal environment and that Debian should be more like it. He also celebrates that Ms Sharp12 left Linux, because evidently the freedom to be a jerk to each other is more important than the contributions that she -or any other person that finds that atmosphere toxic, like mjg- has produced or could have produced in the future.

---

I would like to tell readers of Planet Debian that this view is not the norm in Debian any more. We are working hard to make the project more inclusive, fun, and welcoming to everybody. Sadly, some people are still pining for the good ol' days, and won't give up so easily on their privilege. 6 comments

Much has been written around the internet about this geeky kernel maintainer Sarah Sharp who left kernel development. I have now spent two hours reading through lkml posts, and want to summarize a few mails from the long thread, since most of the usual news sites just rewrap the original blog of hers without adding any background. The whole thread evolved out call for stable kernel review by Greg Kroah-Hartman where he complained about too many patches that are not actually in rc1 before going into stable: from where it developed into a typical Linus rant on people flagging crap for stable, followed by some jokes: and Ingo Molnar giving advice to Greg KH: with Greg KH taking a funny position in answering: Up to now a pretty decent and normal thread with some jokes and poking, nobody minded, and reading through it I had a good time. The thread continues with a discussion on requirements what to submit to stable, and some side threads on particular commits. And then, out of the blue, Social Justice Warrior (SJW) Sarah Sharp pops in with a very important contribution: Onto which Linus answers in a great way: On goes Sarah, gearing up in her SJW mode and starting to rant: Linus himself made clear what he thinks of her: It is easy to verify what Linus said, by reading the above two links and the answers of the maintainers, both agreed that it was their failure and were sorry. (Mauro s answer, Rafael s answer) It is just the geeky SJW that was not even attacked (who would dare to attack a woman nowadays?). The overall reaction to her by the maintainers can be exemplified by Thomas Gleixner s post: What follows is a nearly endless discussion with Sarah meandering around, changing permanently her opinion what is acceptable. Linus tried to explain to her in simple words, without success, she continues to rant around. Here arguments are so weak I had nothing but good laugh: Fortunately, she was immediately corrected and Ingo Molnar wrote an excellent refutation (starting another funny thread) of all her emails, statements, accusations (all of the email is a good read): (He retracted that last statement, though I don t see a reason for it) Last but not least, let us return to her blog post, where she states herself that: and she made lots of use of it, I counted at least 10 instances. She seems to remove or fart fart fart any comment that is not in line with her opinion. Further evidence is provided by this post on lkml. Everyone is free to have his own opinion (sorry, his/her), and I am free to form my own opinion on Sarah Sharp by just simply reading the facts. I am more than happy that one more SJW has left Linux development, as the proliferation of cleaning of speech from any personality has taken too far a grip. Coming to my home-base in Debian, unfortunately there is no one in the position and the state of mind of Linus, so we are suffering the same stupidities imposed by social justice worriers and some brainless feminists (no, don t get me wrong, these are two independent attributes. I do NOT state that feminism is brainless) that Linus and the maintainer crew was able to fend of this time. I finish with my favorite post from that thread, by Steven Rosted (from whom I also stole the above image!): Post Scriptum (after a bike ride) The last point by Linus is what I criticize most on Debian nowdays, it has become a sterilized over-governed entity, where most fun is gone. Making fun is practically forbidden, since there is the slight chance that some minority somewhere on this planet might feel hurt, and by this we are breaking the CoC. Emotions are restricted to the Happy happy how nice we are and how good we are level of US and also Japanese self-reenforcement society. Post Post Scriptum I just read Sarah Sharp s post on What makes a good community? , and without giving a full account or review, I am just pi**ed by the usage of the word microaggressions I can only recommend everyone to read this article and this article to get a feeling how bs the idea of microaggressions has taken over academia and obviously not only academia. Post³ Scriptum I am happy to see Lars Wirzenius, Gunnar Wolf, and Mart n Ferrari opposing my view. I agree with them that my comments concerning Debian are not mainstream in Debian something that is not very surprising, though, and I think it is great that they have fun in Debian, like many other contributors. Post⁴ Scriptum Although nobody will read this, here is a great response from a female developer: Nothing to add here!

Debian LTS August was the fifth month I contributed to Debian LTS under the Freexian umbrella. In total I spent eight hours working on:

• glibc: I fixed four security issues resulting in DLA-316-1 mostly by backporting fixes from current Git. This involved some issues that didn't have a CVE assigned yet. Salvatore gave a nice summary on how to handle these.
• nss: The backport of the fixes for CVE-2015-2721 and CVE-2015-2730 resulted in DLA-315-1.

Besides that I did CVE triaging of 9 CVEs to check if and how they affect oldoldstable security as part of my LTS front desk work. Other Debian work I finally sent out the summary of the 8th Debian Groupware Meeting we had in the Linuxhotel earlier this year and gave a short talk about Debian at the Zarafa Tour in the Netherlands.

I've been allowing users to upload SSH public-keys, and displaying them online in a form. Displaying an SSH public key is a pain, because they're typically long. That means you need to wrap them, or truncate them, or you introduce a horizontal scroll-bar. So rather than displaying them I figured I'd generate a fingerprint when the key was uploaded and show that instead - This is exactly how github shows your ssh-keys. Annoyingly there is only one reasonable way to get a fingerprint from a key:

• Write it to a temporary file.
• Run "ssh-keygen -lf temporary/file/name".

You can sometimes calculate the key via more direct, but less obvious methods:

awk ' print $2 ' ~/.ssh/id_rsa.pub   base64 -d   md5sum

But that won't work for all key-types. It is interesting to look at the various key-types which are available these days:

mkdir ~/ssh/
cd ~/ssh/
for i in dsa ecdsa ed25519 rsa rsa1 ; do
  ssh-keygen -P "" -t $i -f $ i -key
done

I've never seen an ed25519 key in the wild. It looks like this:

$ cat ~/ssh/ed25519-key.pub
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMcT04t6UpewqQHWI4gfyBpP/ueSjbcGEze22vdlq0mW skx@shelob

Similarly curve-based keys are short too, but not as short:

ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLTJ5+  \
 rWoq5cNcjXdhzRiEK3Yq6tFSYr4DBsqkRI0ZqJdb+7RxbhJYUOq5jsBlHUzktYhOahEDlc9Lezz3ZUqXg= skx@shelob

Remember what I said about wrapping? Ha! Anyway for the moment I've hacked up a simple perl module SSH::Key::Fingerprint which will accept a public key and return the fingerprint, as well as validating the key is well-formed and of a known-type. I might make it public in the future, but I think the name is all wrong. The only code I could easily find to do a similar job is this node.js package, but it doesn't work on all key-types. Shame. And that concludes this weeks super-happy fun-time TODO-list item.

FTP assistant Another month passed and another statistic arrives: This month I marked 341 packages for accept and rejected only 48 of them. Almost like last month I had to send 14 emails to maintainers. Squeeze LTS This was my fifteenth month that I did some work for the Squeeze LTS initiative, started by Raphael Hertzog at Freexian. This month I only got a workload of 14.5h. I finally uploaded a new version of php5. Unfortunately in one library a parameter to a function call introduced new values. As a result all running processes that used the old version of that library produced an error message until they got restarted. As complaints showed up on all channels, I rechecked my patches again and again but could not find an error. I wonder whether this happened once before. At least the php package does not have a mechanism to restart something
Altogether I uploaded those DLAs:

• [DLA 307-1] php5 security update
• [DLA 309-1] openldap security update
• [DLA 311-1] rpcbind security update
• [DLA 312-1] libtorrent-rasterbar security update

I also started to work on an upload of freeimage and the next upload of php5. This month I also had another term of doing frontdesk work. So I answered questions on the IRC channel and looked for CVEs that are important for Squeeze LTS or could be ignored. Other stuff Some time ago someone mentioned pump.io and that it would be nice to have it in Debian. I found a Wiki page listing dependencies, with lots of stuff already done and just a few holes. It didn t look like much work todo until I realized that this page showed only the surface and the shoals are hidden below. Anyway, I started to work on it and up to now

• node-boolbase
• node-domelementtype
• node-eventsource
• node-querystringify
• node-rai
• node-requires-port
• node-url-parse
• node-wrappy
• node-xoauth2

are uploaded and

• node-schlock
• node-array-parallel
• node-css-what
• node-bufferjs
• node-exit

are still in NEW. Luckily most of them could be handled by npm2deb, so it was mainly routine piece of work. So, expect more to come I also polished some smaller packages and could even close some bugs:

• dict-elements
• rplay -> #741567 #597152
• setserial -> #786976 #761951 #761951
• siggen -> #772364
• texify