I don't know about you, but I frequently get mails announcing that I was picked as the lucky winner of a lottery, compensation program or simply as "business associate". Obvious Spam of course, that never happens in reality. Just like my personal "favorite" at the moment: Mails notifying me of inheritance from a previously (more or less) unknown relative. Its just that this is what has happend basically a few weeks ago in reality to me (over the phone through) and I am still dealing with the bureaucracy required of teaching the legal system that I have absolutely no intention of becoming the legal successor of someone I haven't seen in 20 years, regardless of how close the family relation is on paper but that might be the topic of another day. On the 1st March a mail titled "Google Open Source Peer Bonus Program" looked at first as if it would fall into this lottery spam class. It didn't exactly help that the mail was multipart HTML and text, but the text really only the text, not mentioning the embedded links used in the HTML part. It even included a prominent and obvious red flag: "Please fill out the form". 20% Bayes score didn't come from nothing. Still, for better or worse the words "Open Source" made it unlikely to be spam similar to how the word PGP indicates authenticity. So it happened, another spam message became true for me. I wonder which one will be next You have probably figured out by now that I didn't know that program before. Kinda embarrassing for a previous Google Summer of Code student (GSoC is run by the same office), but the idea behind it is simple: Google employees can nominate contributors to open source stuff for a small monetary "thank you!" gift card. Earlier this week winners for this round were announced 52 contributors including yours truly. You might be surprised, but the rational given behind my name is APT (I got a private mail with the full rational from my "patron", just in case you wonder if at least I would know more). It is funny how a guy who was taken aback by the prospect of needing a package manager like YaST to use Linux contributed just months later the first patch to apt and has roughly 8 years later amassed more than 2400 commits. It's birthday season in my family with e.g. mine just a few days ago, so its seems natural that apt has its own birthday today just as if it would be part of my family: 19th years this little bundle of bugs joy is now! In more sober moments I wonder sometimes how apt and I would have turned out if we hadn't meet. Would apt have met someone else? Would I? Given that I am still the newest team member and only recently joined Debian as DD at all APT has some strange ways of showing that it loves you: It e.g. helps users compose mails which end in a dilemma to give a recent example. Perhaps you need to be a special kind of crazy¹ to consider this good, but as I see it apt has a big enough userbase that regardless of what your patch is doing, someone will like it. That drastically increases the chances that someone will also like it enough to say so in public offsetting complains from all those who don't like the (effects of the) patch which are omnipresent. And twice in a blue moon some of those will even step forward and thank you explicitly. Not that it would be necessary, but it is nice anyhow. So, thanks for the love supercow, Google & apt users! Or in other words: APT might very well be one of the most friendly (package manager related) project to contribute to as the language-specific managers have smaller userbases and hence a smaller chance of having someone liking your work (in public) so contribute a patch or two and be loved, too! Disclaimer: I get no bonus for posting this nor are any other strings attached. Birthdays are just a good time to reflect. In terms of what I do with my new found riches (in case I really receive them I haven't yet so that could still be an elaborate scam ): APT is a very humble program, but even it is thinking about moving away from a dev-box with less than 4 GB of RAM and no SSD, so it is happily accepting the gift and expects me to upgrade sooner now. What kind of precedence this sets for the two decades milestone next year? If APT isn't obsolete by then We will see.

This all started when I figured out that git-cinnabar was using crazy amounts of memory when cloning mozilla-central. That pointed to memory allocation patterns that triggered a suboptimal behavior in the glibc memory allocator, and, while overall, git-cinnabar wasn t really abusing memory all things considered, it happened to be realloc()ating way too much. It also turned out that recent changes on the master branch had made most uses of fast-import synchronous, making the whole process significantly slower. This is where we started from on 0.4.0: And on the master branch as of be75326: An interesting thing to note here is that the glibc allocator runaway memory use was, this time, more pronounced on 0.4.0 than on master. It was the opposite originally, but as I mentioned in the past ASLR is making it not happen exactly the same way each time. While I m here, one thing I failed to mention in the previous posts is that all these measurements were done by cloning a local mercurial clone of mozilla-central, served from localhost via HTTP to eliminate the download time from hg.mozilla.org. And while mozilla-central itself has received new changesets since the first post, the local clone has not been updated, such that all subsequent clone tests I did were cloning the exact same repository under the exact same circumstances. After last blog post, I focused on the low hanging fruits identified so far:

• Moving the mercurial to git SHA1 mapping to the helper process (Finding a git bug in the process).
• Tracking mercurial manifest heads in the helper process.
• Removing most of the synchronous calls to the helper happening during a clone.

And this is how things now look on the master branch as of 35c18e7: So where does that put us?

• The overall clone is now about 11 minutes faster than 0.4.0 (and about 50 minutes faster than master as of be75326!)
• Non-shared memory use of the git-remote-hg process stays well under 2GB during the whole clone, with no spike at the end.
• git-cinnabar-helper now uses more memory, but the sum of both processes is less than what it used to be, even when compensating for the glibc memory allocator issue. One thing to note is that while the git-cinnabar-helper memory use goes above 2GB at the end of the clone, a very large part is due to the pack window size being 1GB on 64-bits (vs. 32MB on 32-bits). Memory usage should stay well under the 2GB address space limit on a 32-bits system.
• CPU usage is well above 100% for most of the clone.

On a more granular level:

• The Import manifests phase is now 13 minutes faster than it was in 0.4.0.
• The Read and import files phase is still almost 4 minutes slower than in 0.4.0.
• The Import changesets phase is still almost 2 minutes slower than in 0.4.0.
• But the Finalization phase is now 3 minutes faster than in 0.4.0.

What this means is that there s still room for improvement. But at this point, I d rather focus on other things. Logging all the memory allocations with the python allocator disabled still resulted in a 6.5GB compressed log file, containing 2.6 billion calls to malloc, calloc, free and realloc (down from 2.7 billions in be75326). The number of allocator calls done by the git-remote-hg process is down to 2.25 billions (from 2.34 billion in be75326). Surprisingly, while more things were moved to the helper, it still made less allocations than in be75326: 345 millions, down from 363 millions. Presumably, this is because the number of commands processed by the fast-import code was reduced. Let s now take a look at the various metrics we analyzed previously (the horizontal axis represents the number of allocator calls that happened before the measurement): A few observations to make here:

• The allocated memory (requested bytes) is well below what it was, and the spike at the end is entirely gone. It also more closely follows the amount of raw data we re holding on to (which makes sense since most of the bookkeeping was moved to the helper)
• The number of live allocations (allocated memory pointers that haven t been free()d yet) has gone significantly down as well.
• The cumulated[*] bytes are now in a much more reasonable range, with the lower bound close to the total amount of data we re dealing with during the clone, and the upper bound slightly over twice that amount (the upper bound for the be75326 is not shown here, but it was around 45TB; less than 7TB is a big improvement).
• There are less allocator calls during the first phases and the Importing changesets phase, but more during the Reading and importing files and Importing manifests phases.

[*] The upper bound is the sum of all sizes ever given to malloc, calloc, realloc etc. and the lower bound is the same, but removing the size of allocations passed as input to realloc (in practical words, this pretends reallocs never happened and that the final size for a given reallocated pointer is the one that counts) So presumably, some of the changes led to more short-lived allocations. Considering python uses its own allocator for sizes smaller than 512 bytes, it s probably not so much of a problem. But let s look at the distribution of buffer sizes (including all sizes given to realloc). (Bucket size is 16 bytes) What is not completely obvious from the logarithmic scale is that, in fact, 98.4% of the allocations are less than 512 bytes with the current master (35c18e7), and they were 95.5% with be75326. Interestingly, though, in absolute numbers, there are less allocations smaller than 512 bytes in current master than in be75326 (1,194,268,071 vs 1,214,784,494). This suggests the extra allocations that happen during some phases are larger than that. There are clearly less allocations across the board (apart from very few exceptions), and close to an order of magnitude less allocations larger than 1MiB. In fact, widening the bucket size to 32KiB shows an order of magnitude difference (or close) for most buckets: An interesting thing to note is how some sizes are largely overrepresented in the data with buckets of 16 bytes, like 768, 1104, 2048, 4128, with other smaller bumps for e.g. 2144, 2464, 2832, 3232, 3696, 4208, 4786, 5424, 6144, 6992, 7920 While some of those are powers of 2, most aren t, and some of them may actually represent objects sized with a power of 2, but that have an extra PyObject overhead. While looking at allocation stats, I got to wonder what the lifetimes of those allocations looked like. So I scanned the allocator logs and measured the distance between when an allocation is made and when it is freed, ignoring reallocs. To give a few examples of what I mean, the following allocation for p gets a lifetime of 0:

void *p = malloc(42);
free(p);

The following a lifetime of 1:

void *p = malloc(42);
void *other = malloc(42);
free(p);

And the following a lifetime of 1 as well:

void *p = malloc(42);
p = realloc(p, 84);
free(p);

(that is, it is not counted as two malloc/free pairs) The further away the free is from the corresponding malloc, the larger the lifetime. And the largest the lifetime can ever be is the total number of allocator function calls minus two, in the hypothetical case the very first allocation is freed as the very last (minus two because we defined the lifetime as the distance). What comes out of this data:

• As expected, there are more short-lived allocations in 35c18e7.
• Around 90% of allocations have a lifetime spanning 10% of the process life or less. This is a rather surprisingly large amount of allocations with a very large lifetime.
• Around 80% of allocations have a lifetime spanning 0.01% of the process life or less.
• The median lifetime is around 0.0000002% (2*10^-7%) of the process life, which, in absolute terms is around 500 allocator function calls between a malloc and a free.
• If we consider every imported changeset, manifest and file to require a similar number of allocations, and considering there are about 2.7M of them in total, each spans about 3.7*10^-7%. About 53% of all allocations on be75326 and 57% on 35c18e7 have a lifetime below that. Whenever I get to look more closely to memory usage again, I ll probably look at the data separately for each individual phase.
• One surprising fact, that doesn t appear on the graph because of the logarithmic scale not showing 0 on the horizontal axis, is that 9.3% on be75326 and 7.3% on 35c18e7 of all allocations have a lifetime of 0. That is, whatever the code using them is doing, it s not allocating or freeing anything else, and not reallocating them either.

All in all, what the data shows is that we re definitely in a better place now than we used to be a few days ago, and that there is still work to do on the memory front, but:

• As mentioned in a previous post, there are bigger wins to be had from not keeping manifests data around in memory at all, and by importing it directly instead.
• In time, a lot of the import code is meant to move to the helper, where the constraints are completely different, and it might not be worth spending time now on reducing the memory usage of python code that might go away soon(ish). The situation was bad and necessitated action rather quickly, but we re now in a place where it s not as bad anymore.

So at this point, I won t look any deeper into the memory usage of the git-remote-hg python process, and will instead focus on the planned metadata storage changes. They will allow to share the metadata more easily (allowing faster and more straightforward gecko-dev graft), and will allow to import manifests earlier, which, as mentioned already, will help reduce memory use, but, more importantly, will allow to do more actual work while downloading the data. On slow networks, this is crucial to make clones and pulls faster.

Review: Two Serpents Rise, by Max Gladstone

Series:	Craft #2
Publisher:	Tor
Copyright:	October 2013
ISBN:	1-4668-0204-9
Format:	Mobi
Pages:	350

This is the second book in the Craft Sequence, coming after Three Parts Dead, but it's not a sequel. The only thing shared between the books is the same universe and magical system. Events in Two Serpents Rise were sufficiently distant from the events of the first book that it wasn't obvious (nor did it matter) where it fit chronologically. Caleb is a gambler and an investigator for Red King Consolidated, the vast firm that controls the water supply, and everything else, in the desert city of Dresediel Lex. He has a fairly steady and comfortable job in a city that's not comfortable for many, one of sharp divisions between rich and poor and which is constantly one water disturbance away from riot. His corporate work life frustrates his notorious father, a legendary priest of the old gods who were defeated by the Red King and who continues to fight an ongoing terrorist resistance to the new corporate order. But Caleb has as little as possible to do with that. Two Serpents Rise opens with an infiltration of the Bright Mirror Reservoir, one of the key components of Dresediel Lex's water supply. It's been infested with Tzimet: demon-like creatures that, were they to get into the city's water supply, would flow from faucets and feed on humans. Red King Incorporated discovered this one and sealed the reservoir before the worst could happen, but it's an unsettling attack. And while Caleb is attempting to determine what happened, he has an unexpected encounter with a cliff runner: a daredevil parkour enthusiast with an unexpected amulet of old Craft that would keep her invisible from most without the magical legacy Caleb is blessed (or cursed) with. He doesn't think her presence is related to the attack, but he can't be sure, particularly with the muddling fact that he finds her personally fascinating. Like Three Parts Dead, you could call Two Serpents Rise an urban fantasy in that it's a fantasy that largely takes place in cities and is concerned with such things as infrastructure, politics, and the machinery of civilization. However, unlike Three Parts Dead, it takes itself much more seriously and has less of the banter and delightful absurdity of the previous book. The identification of magic with contracts and legalities is less amusingly creative here and more darkly sinister. Partly this is because the past of Dresediel Lex is full of bloodthirsty gods and human sacrifice, and while Red King Consolidated has put an end to that practice, it lurks beneath the surface and is constantly brought to mind by some grisly artifacts. I seem to always struggle with fantasy novels based loosely on central American mythology. An emphasis on sacrifice and terror always seems to emerge from that background, and it verges too close to horror for me. It also seems prone to clashes of divine power and whim instead of thoughtful human analysis. That's certainly the case here: instead of Tara's creative sleuthing and analysis, Caleb's story is more about uncertainty, obsession, gambling, and shattering revelations. Magical rituals are described more in terms of their emotional impact than their world-building magical theory. I think this is mostly a matter of taste, and it's possible others would like Two Serpents Rise better than the previous book, but it wasn't as much my thing. The characters are a mixed bag. Caleb was a bit too passive to me, blown about by his father and his employer and slow to make concrete decisions. Mal was the highlight of the book for me, but I felt at odds with the author over that, which made the end of the book somewhat frustrating. Caleb has some interesting friends, but this is one of those books where I would have preferred one of the supporting cast to be the protagonist. That said, it's not a bad book. There are some very impressive set pieces, the supporting cast is quite good, and I am wholeheartedly in favor of fantasy novels that are built around the difficulties of water supply to a large, arid city. This sort of thing has far more to do with human life than the never-ending magical wars over world domination that most fantasy novels focus on, and it's not at all boring when told properly. Gladstone is a good writer, and despite the focus of this book not being as much my cup of tea, I'll keep reading this series. Followed by Full Fathom Five. Rating: 7 out of 10

Changes

• gitk: colour of remote refs
• myrepos: accumulated patches
• check-all-the-things: fix the yamllint cmd, add TODO items for haxelint and Dockerfile checkers, add checks for dodgy stuff, dead code, GIFs, Ruby (rubocop, roodi), Mono CIL binaries, FITS images and TeX
• gnome-shell-extension-show-ip: came up with the idea to fix the main blocker to deforking it, cleanups, show the primary IP address and show IP addresses in the menu
• Spelling:
  • codespell: broser, optionnal, optionnaly, reuest, ressource (hi samba folks), swtich, assumme(s), checg(ed), workes, accidentially, uninitalized, permision(s), whe, gruop(s), determing, imilar, tigthly, standar, iff, nott, technlogy, playble, redeable, diagnotic, langauage, mor, soundard, firmwware, containt(s), mazilla (hi malware authors), mozila, cilyndre(s), sepatae
  • lintian: broser, optionnaly, kenrel, swtich, assumme(s), checg(ed), workes, whe, gruops, imilar, tigthly, standar, iff, nott, technlogy, playble, debia, redeable, diagnotic, mor, soundard, firmwware, containt, mazilla, mozila, cilyndre(s), sepatae
• Wikipedia: Message-ID
• Debian wiki: AndroidTools, AppArmor/Contribute/Upstream, BSP/2017/05/ch/Zurich, CrossCompiling, DateTime, DebianEdu/Status/Stretch, DebianEvents/br/2017/MiniDebconfCuritiba/Atividades/CFP, DebianKernel/UserspaceTools, DebianMultimedia/FFmpeg (Unstripped), DebianRepository (Unofficial, UseThirdParty), DebianScience/UnofficialRepository, DebianUnofficial, dedup.debian.net, Derivatives/Census (QA (1, 2), Qubes, SparkyLinux), Exploits, fr/Simple-CDD, Games/Unsuitable, Hardening/RepoAndImages, Ideas/Ports, iMacG5, IRC, it/UltimateDebianDatabase, Merchandise, PaulWise/InterestingSoftware (1, 2), piuparts/FAQ, pkg-escience, qa.debian.org, QuickInstall, RepositoryLocal, Services/PublicUddMirror, Simple-CDD/Howto, SuitesAndReposExtension, SummerOfCode2006, SummerOfCode2014/Projects/DebianMetricsPortal, SummerOfCode2017/Projects/QA_BiologyApps, sv/IRC, SystemBuildTools, UltimateDebianDatabase, Vangelis Mouhtsis
• Debian website: Remove old unused and not working website hosting sponsor functionality (1 2 3 4 5 6 7), use correct SSL CA cert store, fix and https links to debian.ch and fix HTML entity escaping in the mirrors list (1 2)
• Debian ports: help rescue the #debian-ports IRC channel, remove dead mirror, update mirrors page, remove d-p.o reference, update bits.d.o and update packages.d.o
• Debian packages site: remove some git cruft
• Debian security tracker: remove NFU from webkit issue and add source link to DLAs
• Planet Debian derivatives: Add Qubes
• Debian package uploads: needrestart-session 0.3-4.1 and valgrind 1:3.10.0-4~bpo7+1

Issues

• Crashes in evolution, hexchat, gpaste, pidgin and eog
• Perl @INC issue in libthrift-perl
• Regression in samba-common-bin
• Debian migration unblocks of needrestart-session (1, 2)
• Debian Policy violation in selenium-firefoxdriver
• Important issue in needrestart-session
• Minor issues in gnupg and giflib-tools
• Features in file, debdelta, diffoscope, nss-passwords and linux

Review

• Spam: reported 5 Debian bug reports and 289 Debian mailing list posts
• Debian wiki: RecentChanges for the month
• Debian screenshots:
  • approved spectrwm (2), ranger, i3, kupfer, zathura (3), gimagereader, yubioath-desktop (3), fluxbox, libreoffice-writer, gtklp, terminator, wireshark-qt, wireshark-gtk, synaptic, empathy, gnome-control-center, shutter (2), gnome-online-accounts, gnome-documents, gnome-contacts, network-manager-openvpn-gnome, gnome-shell-extension-system-monitor, neofetch, octave-sparsersb, audacious, smplayer, bluefish, gpodder, delaboratory, geeqie, fracplanet and braillegraph
  • rejected hotssh (artefacts in screenshot), libsixel-bin (copied from elsewhere), spectrwm (not a screenshot) and geeqie (worse version of accepted image)
  • approved deletion (despite bogus reason) of snmp (screenshots site not a support forum, was a diagram not a screenshot)
  • rejected deletion of kupfer (we do not remove old versions), vlc (external UI in the screenshot is minimal), vokoscreen/fonts-ecolier-lignes-court (garbage in reason field), aptitude (discussing non-English screenshots with Christoph Haas), hydra-gtk (greetings are not valid reasons) and oxygen-icon-theme (the screenshot is not a reCAPTCHA)

Administration

• Debian systems: apply a patch to userdir-ldap, ask a local admin to reset a dead powerpc buildd, remove dead SH4 porterboxen from LDAP, fix perms on www.d.o OC static mirror, report false positives in an an automated abuse report, redirect 1 student to FAQs/support/DebianEdu, redirect 1 event organiser to partners/trademark/merchandise/DPL, redirect 1 guest account seeker to NM, redirect 1 @debian.org desirer to NM, redirect 1 email bounce to a changes@db.d.o user, redirect 2 people to the listmasters, redirect 1 person to Debian Pure Blends, redirect 1 user to a service admin and redirect 2 users to support
• Debian packages site: deploy my ports/cruft changes
• Debian wiki: poke at HP page history and advise a contributor, whitelist 13 email address, whitelist 1 domain, check out history of a banned IP, direct 1 hoster to DebConf17 sponsors team, direct 1 user to OpenStack packaging, direct 1 user to InstallingDebianOn and h-node.org, direct 2 users to different ways to help Debian and direct 1 emeritus DD on repository wiki page reorganisation
• Debian QA: fix an issue with the PTS news, remove some debugging cruft I left behind, fix the usertags on a QA bug and deploy some code fixes
• Debian mentors: security upgrades and service restarts
• Openmoko: security upgrades and reboots

Communication

• Invite GALPon MiniNo, Scibian, Roche Linux and ubilinux to the Debian derivatives census
• Welcome Qubes to the Debian derivatives census
• Start discussion about how many people contact the wrong part of Debian for their query

Sponsors The valgrind backport, samba and libthrift-perl bug reports were sponsored by my employer. All other work was done on a volunteer basis.

People that know me know that I do whatever I can in order to avoid watching videos online if there's any other way to get to the content. It may be that I'm too old-fashioned, or that I have low attention and prefer to use a media where I can quickly scroll up and down a paragraph, or that I feel the time between bits of content is just a useless transition or whatever... But I bit. And I loved it. A couple of days ago, OS News featured a post titled From the AT&T Archives: The UNIX Operating System. It links to a couple of videos in AT&T's Youtube channel. I watched
AT&T Archives: The UNIX Operating System, an amazing historic evidence: A 27 minute long documentary produced in 1981 covering... What is Unix. Why Unix is so unique, useful and friendly. What's the big deal about it? That this document shows first-hand that we are not repeating myths we came up with along the way: The same principles of process composition, of simplicity and robustness, but spoken directly by many core actors of the era Brian Kernighan (who drove a great deal of the technical explanation), Alfred Aho, Dennis Ritchie, Ken Thompson... And several more I didn't actually catch the names of. Of course, the video includes casual shots of life at AT&T, including lots of terminals (even some of which are quite similar to the first ones I used here in Mexico, of course), then-amazing color animation videos showing the state of the art of computer video 35 years ago... A delightful way to lose half an hour of productivity. And a bit of material that will surely find its way into my classes for some future semester :) [ps] Yes, I don't watch videos in Youtube. I don't want to enable its dirty Javascript. So, of course, I use the great Youtube-dl tool. I cannot share the video file itself here due to Youtube's service terms, but Youtube-dl is legal and free.

My Raspberry Pi hifi component setup lacked a way to cleanly shutdown the system without ssh. I wished the Raspberry Pi had a button so I can tell it to shutdown. I added one to the GPIO connector. It turns out many wishes can come true when one has a GPIO board. This is the /usr/local/bin/stereo-gpio script that reacts to the button press and triggers a shutdown:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

#!/usr/bin/python3 # http://razzpisampler.oreilly.com/ch07.html # http://web.archive.org/web/20160305001215/http://razzpisampler.oreilly.com/ch07.html from RPi import GPIO import time import subprocess def on_button(pin): print("Button pressed", pin) GPIO.setmode(GPIO.BCM) GPIO.setup(18, GPIO.IN, pull_up_down=GPIO.PUD_UP) while True: pin = GPIO.wait_for_edge(18, GPIO.FALLING) if pin == 18: subprocess.check_call(["/bin/systemctl", "halt"])

This is the /etc/systemd/system/stereo-gpio.service systemd unit file that runs the script as a daemon: Then systemctl start stereo-gpio to start the script, and systemctl enable stereo-gpio to start the script at boot.

As I have previously blogged, I recently managed to resurrect my old Drupal site that ran in the Amazon AWS cloud, and get it working again on a new host. I have just written up a summary of how I battled through the process, which can be found here. Unfortunately, I took a long time to write it up. So it is not as detailed as I originally intended. But if like me you run a Drupal site, or you did and it is also broken, then feel free to follow the link for a read. It may at least give some ideas to follow up. I made heavy use of DrupalVM. If you are just starting out with a Drupal website, and you have more than FTP access to your hosting, I recommend using DrupalVM (which is built with Vagrant & Ansible) for local development and testing.

We needed a router and wifi access point in the office, and simultaneously both I and my co-worker Ivan needed such a thing at our respective homes. After some discussion, and after reading articles in Ars Technica about building PCs to act as routers, we decided to do just that.

• The PC solution seem to offer better performance, but this is actually not a major reason for us.
• We want to have systems we understand and can hack. A standard x86 PC running Debian sounds ideal to use.
• Why not a cheap commercial router? They tend to be opaque and mysterious, and can't be managed with standard tooling such as Ansible. They may or may not have good security support. Also, they may or may not have sufficient functionality to be nice things, such as DNS for local machines, or the full power if iptables for firewalling.
• Why not OpenWRT? Some models of commercial routers are supported by OpenWRT. Finding good hardware that is also supported by OpenWRT is a task in itself, and not the kind of task especially I like to do. Even if one goes this route, the environment isn't quite a standard Linux system, because of various hardware limitations. (OpenWRT is a worthy project, just not our preference.)

We got some hardware:

Component	Model	Cost
Barebone	Qotom Q190G4, VGA, 2x USB 2.0, 134x126x36mm, fanless	130
CPU	Intel J1900, 2-2.4GHz quad-core	-
NIC	Intel WG82583, 4x 10/100/1000	-
Memory	Crucial CT102464BF160B, 8GB DDR3L-1600 SODIMM 1.35V CL11	40
SSD	Kingston SSDNow mS200, 60GB mSATA	42
WLAN	AzureWave AW-NU706H, Ralink RT3070L, 300M 802.11b/g/n, half mPCIe	17
mPCIe adapter	Half to full mPCIe adapter	3
Antennas	2x 2.4/5GHz 6dBi, RP-SMA, U.FL Cables	7

These were bought at various online shops, including AliExpress and verkkokauppa.com. After assembling the hardware, we installed Debian on them:

• Connect the PC to a monitor (VGA) and keyboard (USB), as well as power.
• I built a "factory image" to be put on the SSD, and a USB stick installer image, which includes the factory one. Write the installer image on a USB stick, boot off that, then copy the factory image to the SSD and reboot off the SSD.
• The router now runs a very bare-bones, stripped-down Debian system, which runs a DHCP server on eth3 (marked LAN4 on the box). You can log as root on the console (no password), or via ssh, but for ssh you need to replace the /home/ansible/.ssh/authorized_keys file with one that contains only your public ssh key.
• Connect a laptop to the Ethernet port marked LAN4, and get an IP address with DHCP.
• Log in with ssh to ansible@10.0.0.4, and verify that sudo id works without password. Except you can't do this, unless you put in your ssh key in the authorized keys file above.
• Git clone the ansible playbooks, adjust their parameters in minipc-router.yml as wanted, and run the playbook. Then reboot the router again.
• You should now have wifi, routing (with NAT), and be generally speaking able to do networking.

There's a lot of limitations and problems:

• There's no web UI for managing anything. If you're not comfortable doing sysadmin via ssh (with or without ansible), this isn't for you.
• No IPv6. We didn't want to enable it yet, until we understand it better. You can, if you want to.
• No real firewalling, but adjust roles/router/files/ferm.conf as you wish.
• The router factory image is 4 GB in size, and our SSD is 60 GB. That's a lot of wasted space.
• The router factory image embeds our public keys in the ansible user's authorized keys file for ssh. This is because we built this for ourselves first. If there's interest by others in using the images, we'll solve this.
• Probably a lot of stupid things. Feel free to tell us what it is (bugs@liw.fi would be a good address for that).

If you'd like to use the images and Ansible playbooks, please do. We'd be happy to get feedback, bug reports, and patches. Send them to me (liw@liw.fi) or my ticketing system (bugs@liw.fi).

DRM display resource leasing (kernel side) So, you've got a fine head-mounted display and want to explore the delights of virtual reality. Right now, on Linux, that means getting the window system to cooperate because the window system is the DRM master and holds sole access to all display resources. So, you plug in your device, play with RandR to get it displaying bits from the window system and then carefully configure your VR application to use the whole monitor area and hope that the desktop will actually grant you the boon of page flipping so that you will get reasonable performance and maybe not even experience tearing. Results so far have been mixed, and depend on a lot of pieces working in ways that aren't exactly how they were designed to work. We could just hack up the window system(s) and try to let applications reserve the HMD monitors and somehow removing them from the normal display area so that other applications don't randomly pop up in the middle of the screen. That would probably work, and would take advantage of much of the existing window system infrastructure for setting video modes and performing page flips. However, we've got a pretty spiffy standard API in the kernel for both of those, and getting the window system entirely out of the way seems like something worth trying. I spent a few hours in Hobart chatting with Dave Airlie during LCA and discussed how this might actually work. Goals

1. Use KMS interfaces directly from the VR application to drive presentation to the HMD.
2. Make sure the window system clients never see the HMD as a connected monitor.
3. Maybe let logind (or other service) manage the KMS resources and hand them out to the window system and VR applications.

Limitations

1. Don't make KMS resources appear and disappear. It turns out applications get confused when the set of available CRTCs, connectors and encoders changes at runtime.

An Outline for Multiple DRM masters By the end of our meeting in Hobart, Dave had sketched out a fairly simple set of ideas with me. We'd add support in the kernel to create additional DRM masters. Then, we'd make it possible to 'hide' enough state about the various DRM resources so that each DRM master would automagically use disjoint subsets of resources. In particular, we would.

1. Pretend that connectors were always disconnected
2. Mask off crtc and encoder bits so that some of them just didn't seem very useful.
3. Block access to resources controlled by other DRM masters, just in case someone tried to do the wrong thing.

Refinement with Eric over Swedish Pancakes A couple of weeks ago, Eric Anholt and I had breakfast at the original pancake house and chatted a bit about this stuff. He suggested that the right interface for controlling these new DRM masters was through the existing DRM master interface, and that we could add new ioctls that the current DRM master could invoke to create and manage them. Leasing as a Model I spent some time just thinking about how this might work and came up with a pretty simple metaphor for these new DRM masters. The original DRM master on each VT "owns" the output resources and has final say over their use. However, a DRM master can create another DRM master and "lease" resources it has control over to the new DRM master. Once leased, resources cannot be controlled by the owner unless the owner cancels the lease, or the new DRM master is closed. Here's some terminology:

DRM Master

Any DRM file which can perform mode setting.

Owner

The original DRM Master, created by opening /dev/dri/card*

Lessor

A DRM master which has leased out resources to one or more other DRM masters.

Lessee

A DRM master which controls resources leased from another DRM master. Each Lessee leases resources from a single Lessor.

Lessee ID

An integer which uniquely identifies a lessee within the tree of DRM masters descending from a single Owner.

Lease

The contract between the Lessor and Lessee which identifies which resources which may be controlled by the Lessee. All of the resources must be owned by or leased to the Lessor.

With Eric's input, the interface to create a lease was pretty simple to write down:

int drmModeCreateLease(int fd,
               const uint32_t *objects,
               int num_objects,
               int flags,
               uint32_t *lessee_id);

Given an FD to a DRM master, and a list of objects to lease, a new DRM master FD is returned that holds a lease to those objects. 'flags' can be any combination of O_CLOEXEC and O_NONBLOCK for the newly minted file descriptor. Of course, the owner might want to take some resources back, or even grant new resources to the lessee. So, I added an interface that rewrites the terms of the lease with a new set of objects:

int drmModeChangeLease(int fd,
               uint32_t lessee_id,
               const uint32_t *objects,
               int num_objects);

Note that nothing here makes any promises about the state of the objects across changes in the lease status; the lessor and lessee are expected to perform whatever modesetting is required for the objects to be useful to them. Window System Integration There are two ways to integrate DRM leases into the window system environment:

1. Have logind "lease" most resources to the window system. When a HMD is connected, it would lease out suitable resources to the VR environment.
2. Have the window system "own" all of the resources and then add window system interfaces to create new DRM masters leased from its DRM master.

I'll probably go ahead and do 2. in X and see what that looks like. One trick with any of this will be to hide HMDs from any RandR clients listening in on the window system. You probably don't want the window system to tell the desktop that a new monitor has been connected, have it start reconfiguring things, and then have your VR application create a new DRM master, making the HMD appear to have disconnected to the window system and have that go reconfigure things all over again. I'm not sure how this might work, but perhaps having the VR application register something like a passive grab on hot plug events might make sense? Essentially, you want it to hear about monitor connect events, go look to see if the new monitor is one it wants, and if not, release that to other X clients for their use. This can be done in stages, with the ability to create a new DRM master over X done first, and then cleaning up the hotplug stuff later on. Current Status I hacked up the kernel to support the drmModeCreateLease API, and then hacked up kmscube to run two threads with different sets of KMS resources. That ran for nearly a minute before crashing and requiring a reboot. I think there may be some locking issues with page flips from two threads to the same device. I think I also made the wrong decision about how to handle lessors closing down. I tried to let the lessors get deleted and then 'orphan' the lessees. I've rewritten that so that lessees hold a reference on their lessor, keeping the lessor in place until the lessee shuts down. I've also written the kernel parts of the drmModeChangeLease support. Questions

• What should happen when a Lessor is closed? Should all access to controlled resources be revoked from all descendant Lessees? Proposed answer -- lessees hold a reference to their lessor so that the entire tree remains in place. A Lessor can clean up before exiting by revoking lessee access if it chooses.
• How about when a Lessee is closed? Should the Lessor be notified in some way?
• CRTCs and Encoders have properties. Should these properties be automatically included in the lease? Proposed answer -- no, userspace is responsible for constructing the entire lease.

About a year ago I bought a new workstation computer for myself at home. It s a Tuxedo XUX_Cube which is advertised as gaming PC. But I ordered a slightly atypical non-gamer configuration:

• As much RAM as possible (64 GB)
• Intel i7 CPU, but the low power variant
• Only with the onboard Intel graphics card. (No need for NVidia binary crap drivers.)
• 2 Samsung 128 GB SSD for OS and $HOME plus 2 3 TB WD Red disks for media storage; both pairs set up as RAID 1
• Bitfenix Prodigy-M case in Orange. (Not available in Tuxedo Computer s online shop, but they nevertheless ordered it for me. :-)

Of course the box runs Debian. To be more precise, it runs Debian Sid with sysvinit-core as init system and i3 as window manager. As I usually have no monitoring clients on my laptops and private workstations, I rather often felt the urge to do a cat /proc/mdstat on that box. So at some point I wanted something like smart-notifier, but for Linux Software (MD) RAIDs. And since I found nothing, I did what Open Source guys usually do in such cases: I wrote it myself of course in Perl and called it systray-mdstat. First I wondered about which build system would be most suitable for that task, but in the end I once again went with Dist::Zilla for the upstream build system and hence dh-dist-zilla for the Debian packaging. Ideas for the actual implementation were taken from Wouter s fdpowermon for the systray icon framework in Perl and Myon s mdstat Xymon plugin for an already proven logic to parse /proc/mdstat. (Both, Wouter and Myon have stated in a GnuPG-signed e-mail that I copied less code than would validate their copyrights, so I was able to license it under a single license, namely GNU GPL version 3.) As of now, systray-mdstat is also available as package in Debian Unstable. It won t make it to Stretch as its first line of code has been written after the soft-freeze for Stretch was already in place.

Work has been hellishly busy lately, so that's pretty much all I've been doing. The major project I'm working on should be basically done in the next couple of weeks, though (fingers crossed), so maybe I'll be able to surface a bit more after that. In the meantime, I'm still acquiring books I don't have time to read, since that's my life. In this case, two great Humble Book Bundles were too good of a bargain to pass up. There are a bunch of books in here that I already own in paperback (and hence showed up in previous haul posts), but I'm running low on shelf room, so some of those paper copies may go to the used bookstore to make more space. Kelley Armstrong Lost Souls (sff)
Clive Barker Tortured Souls (horror)
Jim Butcher Working for Bigfoot (sff collection)
Octavia E. Butler Parable of the Sower (sff)
Octavia E. Butler Parable of the Talents (sff)
Octavia E. Butler Unexpected Stories (sff collection)
Octavia E. Butler Wild Seed (sff)
Jacqueline Carey One Hundred Ablutions (sff)
Richard Chizmar A Long December (sff collection)
Jo Clayton Skeen's Leap (sff)
Kate Elliot Jaran (sff)
Harlan Ellison Can & Can'tankerous (sff collection)
Diana Pharoh Francis Path of Fate (sff)
Mira Grant Final Girls (sff)
Elizabeth Hand Black Light (sff)
Elizabeth Hand Saffron & Brimstone (sff collection)
Elizabeth Hand Wylding Hall (sff)
Kevin Hearne The Purloined Poodle (sff)
Nalo Hopkinson Skin Folk (sff)
Katherine Kurtz Camber of Culdi (sff)
Katherine Kurtz Lammas Night (sff)
Joe R. Lansdale Fender Lizards (mainstream)
Robert McCammon The Border (sff)
Robin McKinley Beauty (sff)
Robin McKinley The Hero and the Crown (sff)
Robin McKinley Sunshine (sff)
Tim Powers Down and Out in Purgatory (sff)
Cherie Priest Jacaranda (sff)
Alastair Reynolds Deep Navigation (sff collection)
Pamela Sargent The Shore of Women (sff)
John Scalzi Miniatures (sff collection)
Lewis Shiner Glimpses (sff)
Angie Thomas The Hate U Give (mainstream)
Catherynne M. Valente The Bread We Eat in Dreams (sff collection)
Connie Willis The Winds of Marble Arch (sff collection)
M.K. Wren Sword of the Lamb (sff)
M.K. Wren Shadow of the Swan (sff)
M.K. Wren House of the Wolf (sff)
Jane Yolen Sister Light, Sister Dark (sff)

I got an amazing chance to speak at FOSSASIA 2017 held at Singapore on Seasons of Debian Summer of Code and Winter of Outreachy . I gave a combined talk with my co-speaker Pranav Jain, who contributed to Debian through GSoC. We talked about two major open source initiatives Outreachy and Google Summer of Code and the work we did on a common project Lumicall under Debian. The excitement started even before the first day! On 16th March, there was a speakers meetup at Microsoft office in Singapore. There, I got the chance to connect with other speakers and learn about their work The meetup was concluded by Microsoft Office tour! As a student it was very exciting to see first hand the office of a company that I had only dreamt of being at. On 17th March, i.e the first day of the three days long conference, I met Hong Phuc Dang, Founder of FOSSASIA. She is very kind and just talking to her just made me cheerful!
Meeting so many great developers from different organizations was exciting. On 18th March, was the day of our talk! I was a bit nervous to speak in front of amazing developers but, that s how you grow Our talk was preceded by a lovely introduction by Mario Behling. I talked about how Outreachy Programme has made a significant impact in increasing the participation of women in Open Source, with one such woman being me I also talked about Android Programming concepts which I used in while adding new features into Lumicall. Pranav talked about Debian Organization and how to get started with GSoC by sharing his journey! After our talk, students approached us asking questions about how to participate in Outreachy and GsOC. I felt that a lot more students were receptive to knowing about this new opportunity. Our own talk was part of the mini DebConf track. Under this track, there were two other amazing sessions namely, Debian The best Linux distribution and Open Build Service in Debian . The variety of experiences I gained from FOSSASIA was very diverse. I learned how to speak at a huge platform, learned from other interesting talks, share ideas with smart developers and saw an exciting venue and wonderful city! I would not be able to experience this without the continuous support of Debian and Outreachy !

Up to now, Linux packages on mono-project.com have come in two flavours RPM built for CentOS 7 (and RHEL 7), and .deb built for Debian 7. Universal packages that work on the named distributions, and anything newer. Except that s not entirely true. Firstly, there have been compatibility repositories users need to add, to deal with ABI changes in libtiff, libjpeg, and Apache, since Debian 7. Then there s the packages for ARM64 and PPC64el neither of those architectures is available in Debian 7, so they re published in the 7 repo but actually built on 8. A large reason for this is difficulty in our package publishing pipeline apt only allows one version-architecture mix in the repository at once, so I can t have, say, 4.8.0.520-0xamarin1 built on AMD64 on both Debian 7 and Ubuntu 16.04. We ve been working hard on a new package build/publish pipeline, which can properly support multiple distributions, based on Jenkins Pipeline. This new packaging system also resolves longstanding issues such as can t really build anything except Mono and Architecture: All packages still get built on Jo s laptop, with no public build logs So, here s the old build matrix:

Distribution	Architectures
Debian 7	ARM hard float, ARM soft float, ARM64 (actually Debian 8), AMD64, i386, PPC64el (actually Debian 8)
CentOS 7	AMD64

And here s the new one:

Distribution	Architectures
Debian 7	ARM hard float (v7), ARM soft float, AMD64, i386
Debian 8	ARM hard float (v7), ARM soft float, ARM64, AMD64, i386, PPC64el
Raspbian 8	ARM hard float (v6)
Ubuntu 14.04	ARM hard float (v7), ARM64, AMD64, i386, PPC64el
Ubuntu 16.04	ARM hard float (v7), ARM64, AMD64, i386, PPC64el
CentOS 6	AMD64, i386
CentOS 7	AMD64

The compatibility repositories will no longer be needed on recent Ubuntu or Debian just use the right repository for your system. If your distribution isn t listed sorry, but we need to draw a line somewhere on support, and the distributions listed here are based on heavy analysis of our web server logs and bug requests. You ll want to change your package manager repositories to reflect your system more accurately, once Mono vNext is published. We re debating some kind of automated handling of this, but I m loathe to touch users sources.list without their knowledge. CentOS builds are going to be late I ve been doing all my prototyping against the Debian builds, as I have better command of the tooling. Hopefully no worse than a week or two. edit I guess Ubuntu 12.04 is coming back too, despite being EOL, for TravisCI support.

At the GTK hackfest in London (which accidentally became mostly a Flatpak hackfest) I've mainly been looking into how to make D-Bus work better for app container technologies like Flatpak and Snap. The initial motivating use cases are:

• Portals: Portal authors need to be able to identify whether the container is being contacted by an uncontained process (running with the user's full privileges), or whether it is being contacted by a contained process (in a container created by Flatpak or Snap).
• dconf: Currently, a contained app either has full read/write access to dconf, or no access. It should have read/write access to its own subtree of dconf configuration space, and no access to the rest.

At the moment, Flatpak runs a D-Bus proxy for each app instance that has access to D-Bus, connects to the appropriate bus on the app's behalf, and passes messages through. That proxy is in a container similar to the actual app instance, but not actually the same container; it is trusted to not pass messages through that it shouldn't pass through. The app-identification mechanism works in practice, but is Flatpak-specific, and has a known race condition due to process ID reuse and limitations in the metadata that the Linux kernel maintains for AF_UNIX sockets. In practice the use of X11 rather than Wayland in current systems is a much larger loophole in the container than this race condition, but we want to do better in future. Meanwhile, Snap does its sandboxing with AppArmor, on kernels where it is enabled both at compile-time (Ubuntu, openSUSE, Debian, Debian derivatives like Tails) and at runtime (Ubuntu, openSUSE and Tails, but not Debian by default). Ubuntu's kernel has extra AppArmor features that haven't yet gone upstream, some of which provide reliable app identification via LSM labels, which dbus-daemon can learn by querying its AF_UNIX socket. However, other kernels like the ones in openSUSE and Debian don't have those. The access-control (AppArmor mediation) is implemented in upstream dbus-daemon, but again doesn't work portably, and is not sufficiently fine-grained or flexible to do some of the things we'll likely want to do, particularly in dconf. After a lot of discussion with dconf maintainer Allison Lortie and Flatpak maintainer Alexander Larsson, I think I have a plan for fixing this. This is all subject to change: see fd.o #100344 for the latest ideas. Identity model Each user (uid) has some uncontained processes, plus 0 or more containers. The uncontained processes include dbus-daemon itself, desktop environment components such as gnome-session and gnome-shell, the container managers like Flatpak and Snap, and so on. They have the user's full privileges, and in particular they are allowed to do privileged things on the user's session bus (like running dbus-monitor), and act with the user's full privileges on the system bus. In generic information security jargon, they are the trusted computing base; in AppArmor jargon, they are unconfined. The containers are Flatpak apps, or Snap apps, or other app-container technologies like Firejail and AppImage (if they adopt this mechanism, which I hope they will), or even a mixture (different app-container technologies can coexist on a single system). They are containers (or container instances) and not "apps", because in principle, you could install com.example.MyApp 1.0, run it, and while it's still running, upgrade to com.example.MyApp 2.0 and run that; you'd have two containers for the same app, perhaps with different permissions. Each container has an container type, which is a reversed DNS name like org.flatpak or io.snapcraft representing the container technology, and an app identifier, an arbitrary non-empty string whose meaning is defined by the container technology. For Flatpak, that string would be another reversed DNS name like com.example.MyGreatApp; for Snap, as far as I can tell it would look like example-my-great-app. The container technology can also put arbitrary metadata on the D-Bus representation of a container, again defined and namespaced by the container technology. For instance, Flatpak would use some serialization of the same fields that go in the Flatpak metadata file at the moment. Finally, the container has an opaque container identifier identifying a particular container instance. For example, launching com.example.MyApp twice (maybe different versions or with different command-line options to flatpak run) might result in two containers with different privileges, so they need to have different container identifiers. Contained server sockets App-container managers like Flatpak and Snap would create an AF_UNIX socket inside the container, bind() it to an address that will be made available to the contained processes, and listen(), but not accept() any new connections. Instead, they would fd-pass the new socket to the dbus-daemon by calling a new method, and the dbus-daemon would proceed to accept() connections after the app-container manager has signalled that it has called both bind() and listen(). (See fd.o #100344 for full details.) Processes inside the container must not be allowed to contact the AF_UNIX socket used by the wider, uncontained system - if they could, the dbus-daemon wouldn't be able to distinguish between them and uncontained processes and we'd be back where we started. Instead, they should have the new socket bind-mounted into their container's XDG_RUNTIME_DIR and connect to that, or have the new socket set as their DBUS_SESSION_BUS_ADDRESS and be prevented from connecting to the uncontained socket in some other way. Those familiar with the kdbus proposals a while ago might recognise this as being quite similar to kdbus' concept of endpoints, and I'm considering reusing that name. Along with the socket, the container manager would pass in the container's identity and metadata, and the method would return a unique, opaque identifier for this particular container instance. The basic fields (container technology, technology-specific app ID, container ID) should probably be added to the result of GetConnectionCredentials(), and there should be a new API call to get all of those plus the arbitrary technology-specific metadata. When a process from a container connects to the contained server socket, every message that it sends should also have the container instance ID in a new header field. This is OK even though dbus-daemon does not (in general) forbid sender-specified future header fields, because any dbus-daemon that supported this new feature would guarantee to set that header field correctly, the existing Flatpak D-Bus proxy already filters out unknown header fields, and adding this header field is only ever a reduction in privilege. The reasoning for using the sender's container instance ID (as opposed to the sender's unique name) is for services like dconf to be able to treat multiple unique bus names as belonging to the same equivalence class of contained processes: instead of having to look up the container metadata once per unique name, dconf can look it up once per container instance the first time it sees a new identifier in a header field. For the second and subsequent unique names in the container, dconf can know that the container metadata and permissions are identical to the one it already saw. Access control In principle, we could have the new identification feature without adding any new access control, by keeping Flatpak's proxies. However, in the short term that would mean we'd be adding new API to set up a socket for a container without any access control, and having to keep the proxies anyway, which doesn't seem great; in the longer term, I think we'd find ourselves adding a second new API to set up a socket for a container with new access control. So we might as well bite the bullet and go for the version with access control immediately. In principle, we could also avoid the need for new access control by ensuring that each service that will serve contained clients does its own. However, that makes it really hard to send broadcasts and not have them unintentionally leak information to contained clients - we would need to do something more like kdbus' approach to multicast, where services know who has subscribed to their multicast signals, and that is just not how dbus-daemon works at the moment. If we're going to have access control for broadcasts, it might as well also cover unicast. The plan is that messages from containers to the outside world will be mediated by a new access control mechanism, in parallel with dbus-daemon's current support for firewall-style rules in the XML bus configuration, AppArmor mediation, and SELinux mediation. A message would only be allowed through if the XML configuration, the new container access control mechanism, and the LSM (if any) all agree it should be allowed. By default, processes in a container can send broadcast signals, and send method calls and unicast signals to other processes in the same container. They can also receive method calls from outside the container (so that interfaces like org.freedesktop.Application can work), and send exactly one reply to each of those method calls. They cannot own bus names, communicate with other containers, or send file descriptors (which reduces the scope for denial of service). Obviously, that's not going to be enough for a lot of contained apps, so we need a way to add more access. I'm intending this to be purely additive (start by denying everything except what is always allowed, then add new rules), not a mixture of adding and removing access like the current XML policy language. There are two ways we've identified for rules to be added:

• The container manager can pass a list of rules into the dbus-daemon at the time it attaches the contained server socket, and they'll be allowed. The obvious example is that an org.freedesktop.Application needs to be allowed to own its own bus name. Flatpak apps' implicit permission to talk to portals, and Flatpak metadata like org.gnome.SessionManager=talk, could also be added this way.
• System or session services that are specifically designed to be used by untrusted clients, like the version of dconf that Allison is working on, could opt-in to having contained apps allowed to talk to them (effectively making them a generalization of Flatpak portals). The simplest such request, for something like a portal, is "allow connections from any container to contact this service"; but for dconf, we want to go a bit finer-grained, with all containers allowed to contact a single well-known rendezvous object path, and each container allowed to contact an additional object path subtree that is allocated by dconf on-demand for that app.

Initially, many contained apps would work in the first way (and in particular sockets=session-bus would add a rule that allows almost everything), while over time we'll probably want to head towards recommending more use of the second. Related topics Access control on the system bus We talked about the possibility of using a very similar ruleset to control access to the system bus, as an alternative to the XML rules found in /etc/dbus-1/system.d and /usr/share/dbus-1/system.d. We didn't really come to a conclusion here. Allison had the useful insight that the XML rules are acting like a firewall: they're something that is placed in front of potentially-broken services, and not part of the services themselves (which, as with firewalls like ufw, makes it seem rather odd when the services themselves install rules). D-Bus system services already have total control over what requests they will accept from D-Bus peers, and if they rely on the XML rules to mediate that access, they're essentially rejecting that responsibility and hoping the dbus-daemon will protect them. The D-Bus maintainers would much prefer it if system services took responsibility for their own access control (with or without using polkit), because fundamentally the system service is always going to understand its domain and its intended security model better than the dbus-daemon can. Analogously, when a network service listens on all addresses and accepts requests from elsewhere on the LAN, we sometimes work around that by protecting it with a firewall, but the optimal resolution is to get that network service fixed to do proper authentication and access control instead. For system services, we continue to recommend essentially this "firewall" configuration, filling in the $ variables as appropriate:

<busconfig>
    <policy user="$ the daemon uid under which the service runs ">
        <allow own="$ the service's bus name "/>
    </policy>
    <policy context="default">
        <allow send_destination="$ the service's bus name "/>
    </policy>
</busconfig>

We discussed the possibility of moving towards a model where the daemon uid to be allowed is written in the .service file, together with an opt-in to "modern D-Bus access control" that makes the "firewall" unnecessary; after some flag day when all significant system services follow that pattern, dbus-daemon would even have the option of no longer applying the "firewall" (moving to an allow-by-default model) and just refusing to activate system services that have not opted in to being safe to use without it. However, the "firewall" also protects system bus clients, and services like Avahi that are not bus-activatable, against unintended access, which is harder to solve via that approach; so this is going to take more thought. For system services' clients that follow the "agent" pattern (BlueZ, polkit, NetworkManager, Geoclue), the correct "firewall" configuration is more complicated. At some point I'll try to write up a best-practice for these. New header fields for the system bus At the moment, it's harder than it needs to be to provide non-trivial access control on the system bus, because on receiving a method call, a service has to remember what was in the method call, then call GetConnectionCredentials() to find out who sent it, then only process the actual request when it has the information necessary to do access control. Allison and I had hoped to resolve this by adding new D-Bus message header fields with the user ID, the LSM label, and other interesting facts for access control. These could be "opt-in" to avoid increasing message sizes for no reason: in particular, it is not typically useful for session services to receive the user ID, because only one user ID is allowed to connect to the session bus anyway. Unfortunately, the dbus-daemon currently lets unknown fields through without modification. With hindsight this seems an unwise design choice, because header fields are a finite resource (there are 255 possible header fields) and are defined by the D-Bus Specification. The only field that can currently be trusted is the sender's unique name, because the dbus-daemon sets that field, overwriting the value in the original message (if any). To make it safe to rely on the new fields, we would have to make the dbus-daemon filter out all unknown header fields, and introduce a mechanism for the service to check (during connection to the bus) whether the dbus-daemon is sufficiently new that it does so. If connected to an older dbus-daemon, the service would not be able to rely on the new fields being true, so it would have to ignore the new fields and treat them as unset. The specification is sufficiently vague that making new dbus-daemons filter out unknown header fields is a valid change (it just says that "Header fields with an unknown or unexpected field code must be ignored", without specifying who must ignore them, so having the dbus-daemon delete those fields seems spec-compliant). This all seemed fine when we discussed it in person; but GDBus already has accessors for arbitrary header fields by numeric ID, and I'm concerned that this might mean it's too easy for a system service to be accidentally insecure: It would be natural (but wrong!) for an implementor to assume that if g_message_get_header (message, G_DBUS_MESSAGE_HEADER_FIELD_SENDER_UID) returned non-NULL, then that was guaranteed to be the correct, valid sender uid. As a result, fd.o #100317 might have to be abandoned. I think more thought is needed on that one. Unrelated topics As happens at any good meeting, we took the opportunity of high-bandwidth discussion to cover many useful things and several useless ones. Other discussions that I got into during the hackfest included, in no particular order:

• .desktop file categories and how to adapt them for AppStream, perhaps involving using the .desktop vocabulary but relaxing some of the hierarchy restrictions so they behave more like "tags"
• how to build a recommended/reference "app store" around Flatpak, aiming to host upstream-supported builds of major projects like LibreOffice
• how Endless do their content-presenting and content-consuming apps in GTK, with a lot of "tile"-based UIs with automatic resizing and reflowing (similar to responsive design), and the applicability of similar widgets to GNOME and upstream GTK
• whether and how to switch GNOME developer documentation to Hotdoc
• whether pies, fish and chips or scotch eggs were the most British lunch available from Borough Market
• the distinction between stout, mild and porter

More notes are available from the GNOME wiki. Acknowledgements The GTK hackfest was organised by GNOME and hosted by Red Hat and Endless. My attendance was sponsored by Collabora. Thanks to all the sponsors and organisers, and the developers and organisations who attended.

Apart from the memory considerations, one thing that the data presented in the When the memory allocator works against you post that I haven t touched in the followup posts is that there is a large difference in the time it takes to clone mozilla-central with git-cinnabar 0.4.0 vs. the master branch. One thing that was mentioned in the first followup is that reducing the amount of realloc and substring copies made the cloning more than 15 minutes faster on master. But the same code exists in 0.4.0, so this isn t part of the difference. So what s going on? Looking at the CPU usage during the clone is enlightening. On 0.4.0: On master: (Note: the data gathering is flawed in some ways, which explains why the git-remote-hg process goes above 100%, which is not possible for this python process. The data is however good enough for the high level analysis that follows, so I didn t bother to get something more acurate) On 0.4.0, the git-cinnabar-helper process was saturating one CPU core during the File import phase, and the git-remote-hg process was saturating one CPU core during the Manifest import phase. Overall, the sum of both processes usually used more than one and a half core. On master, however, the total of both processes barely uses more than one CPU core. What happened? This and that happened. Essentially, before those changes, git-remote-hg would send instructions to git-fast-import (technically, git-cinnabar-helper, but in this case it s only used as a wrapper for git-fast-import), and use marks to track the git objects that git-fast-import created. After those changes, git-remote-hg asks git-fast-import the git object SHA1 of objects it just asked to be created. In other words, those changes replaced something asynchronous with something synchronous: while it used to be possible for git-remote-hg to work on the next file/manifest/changeset while git-fast-import was working on the previous one, it now waits. The changes helped simplify the python code, but made the overall clone process much slower. If I m not mistaken, the only real use for that information is for the mapping of mercurial to git SHA1s, which is actually rarely used during the clone, except at the end, when storing it. So what I m planning to do is to move that mapping to the git-cinnabar-helper process, which, incidentally, will kill not 2, but 3 birds with 1 stone:

• It will restore the asynchronicity, obviously (at least, that s the expected main outcome).
• Storing the mapping in the git-cinnabar-helper process is very likely to take less memory than what it currently takes in the git-remote-hg process. Even if it doesn t (which I doubt), that should still help stay under the 2GB limit of 32-bit processes.
• The whole thing that spikes memory usage during the finalization phase, as seen in previous post, will just go away, because the git-cinnabar-helper process will just have prepared the git notes-like tree on its own.

So expect git-cinnabar 0.5 to get moar faster, and to use moar less memory.

In previous post, I was looking at the allocations git-cinnabar makes. While I had the data, I figured I d also look how the memory use correlates with expectations based on repository data, to put things in perspective. As a reminder, this is what the allocations look like (horizontal axis being the number of allocator function calls): There are 7 different phases happening during a git clone using git-cinnabar, most of which can easily be identified on the graph above:

• Negotiation. During this phase, git-cinnabar talks to the mercurial server to determine what needs to be pulled. Once that is done, a getbundle request is emitted, which response is read in the next three phases. This phase is essentially invisible on the graph.
• Reading changeset data. The first thing that a mercurial server sends in the response for a getbundle request is changesets. They are sent in the RevChunk format. Translated to git, they become commit objects. But to create commit objects, we need the entire corresponding trees and files (blobs), which we don t have yet. So we keep this data in memory. In the git clone analyzed here, there are 345643 changesets loaded in memory. Their raw size in RawChunk format is 237MB. I think by the end of this phase, we made 20 million allocator calls, have about 300MB of live data in about 840k allocations. (No certainty because I don t actually have definite data that would allow to correlate between the phases and allocator calls, and the memory usage change between this phase and next is not as clear-cut as with other phases). This puts us at less than 3 live allocations per changeset, with only about 60MB overhead over the raw data.
• Reading manifest data. In the stream we receive, manifests follow changesets. Each changeset points to one manifest ; several changesets can point to the same manifest. Manifests describe the content of the entire source code tree in a similar manner as git trees, except they are flat (there s one manifest for the entire tree, where git trees would reference other git trees for sub directories). And like git trees, they only map file paths to file SHA1s. The way they are currently stored by git-cinnabar (which is planned to change) requires knowing the corresponding git SHA1s for those files, and we haven t got those yet, so again, we keep everything in memory. In the git clone analyzed here, there are 345398 manifests loaded in memory. Their raw size in RawChunk format is 1.18GB. By the end of this phase, we made 23 million more allocator calls, and have about 1.52GB of live data in about 1.86M allocations. We re still at less than 3 live allocations for each object (changeset or manifest) we re keeping in memory, and barely over 100MB of overhead over the raw data, which, on average puts the overhead at 150 bytes per object. The three phases so far are relatively fast and account for a small part of the overall process, so they don t appear clear-cut to each other, and don t take much space on the graph.
• Reading and Importing files. After the manifests, we finally get files data, grouped by path, such that we get all the file revisions of e.g. .cargo/.gitignore, followed by all the file revisions of .cargo/config.in, .clang-format, and so on. The data here doesn t depend on anything else, so we can finally directly import the data. This means that for each revision, we actually expand the RawChunk into the full file data (RawChunks contain patches against a previous revision), and don t keep the RawChunk around. We also don t keep the full data after it was sent to the git-cinnabar-helper process (as far as cloning is concerned, it s essentially a wrapper for git-fast-import), except for the previous revision of the file, which is likely the patch base for the next revision. We however keep in memory one or two things for each file revision: a mapping of its mercurial SHA1 and the corresponding git SHA1 of the imported data, and, when there is one, the file metadata (containing information about file copy/renames) that lives as a header in the file data in mercurial, but can t be stored in the corresponding git blobs, otherwise we d have irrelevant data in checkouts. On the graph, this is where there is a steady and rather long increase of both live allocations and memory usage, in stairs for the latter. In the git clone analyzed here, there are 2.02M file revisions, 78k of which have copy/move metadata for a cumulated size of 8.5MB of metadata. The raw size of the file revisions in RawChunk format is 3.85GB. The expanded data size is 67GB. By the end of this phase, we made 622 million more allocator calls, and peaked at about 2.05GB of live data in about 6.9M allocations. Compared to the beginning of this phase, that added about 530MB in 5 million allocations. File metadata is stored in memory as python dicts, with 2 entries each, instead of raw form for convenience and future-proofing, so that would be at least 3 allocations each: one for each value, one for the dict, and maybe one for the dict storage ; their keys are all the same and are probably interned by python, so wouldn t count. As mentioned above, we store a mapping of mercurial to git SHA1s, so for each file that makes 2 allocations, 4.04M total. Plus the 230k or 310k from metadata. Let s say 4.45M total. We re short 550k allocations, but considering the numbers involved, it would take less than one allocation per file on average to go over this count. As for memory size, per this answer on stackoverflow, python strings have an overhead of 37 bytes, so each SHA1 (kept in hex form) will take 77 bytes (Note, that s partly why I didn t particularly care about storing them as binary form, that would only save 25%, not 50%). That s 311MB just for the SHA1s, to which the size of the mapping dict needs to be added. If it were a plain array of pointers to keys and values, it would take 2 * 8 bytes per file, or about 32MB. But that would be a hash table with no room for more items (By the way, I suspect the stairs that can be seen on the requested and in-use bytes is the hash table being realloc()ed). Plus at least 290 bytes per dict for each of the 78k metadata, which is an additional 22M. All in all, 530MB doesn t seem too much of a stretch.
• Importing manifests. At this point, we re done receiving data from the server, so we begin by dropping objects related to the bundle we got from the server. On the graph, I assume this is the big dip that can be observed after the initial increase in memory use, bringing us down to 5.6 million allocations and 1.92GB. Now begins the most time consuming process, as far as mozilla-central is concerned: transforming the manifests into git trees, while also storing enough data to be able to reconstruct manifests later (which is required to be able to pull from the mercurial server after the clone). So for each manifest, we expand the RawChunk into the full manifest data, and generate new git trees from that. The latter is mostly performed by the git-cinnabar-helper process. Once we re done pushing data about a manifest to that process, we drop the corresponding data, except when we know it will be required later as the delta base for a subsequent RevChunk (which can happen in bundle2). As with file revisions, for each manifest, we keep track of the mapping of SHA1s between mercurial and git. We also keep a DAG of the manifests history (contrary to git trees, mercurial manifests track their ancestry ; files do too, but git-cinnabar doesn t actually keep track of that separately ; it just relies on the manifests data to infer file ancestry). On the graph, this is where the number of live allocations increases while both requested and in-use bytes decrease, noisily. By the end of this phase, we made about 1 billion more allocator calls. Requested allocations went down to 1.02GB, for close to 7 million live allocations. Compared to the end of the dip at the beginning of this phase, that added 1.4 million allocations, and released 900MB. By now, we expect everything from the Reading manifests phase to have been released, which means we allocated around 620MB (1.52GB 900MB), for a total of 3.26M additional allocations (1.4M + 1.86M). We have a dict for the SHA1s mapping (345k * 77 * 2 for strings, plus the hash table with 345k items, so at least 60MB), and the DAG, which, now that I m looking at memory usage, I figure has the one of the possibly worst structure, using 2 sets for each node (at least 232 bytes per set, that s at least 160MB, plus 2 hash tables with 345k items). I think 250MB for those data structures would be largely underestimated. It s not hard to imagine them taking 620MB, because really, that DAG implementation is awful. The number of allocations expected from them would be around 1.4M (4 * 345k), but I might be missing something. That s way less than the actual number, so it would be interesting to take a closer look, but not before doing something about the DAG itself. Fun fact: the amount of data we re dealing with in this phase (the expanded size of all the manifests) is close to 2.9TB (yes, terabytes). With about 4700 seconds spent on this phase on a real clone (less with the release branch), we re still handling more than 615MB per second.
• Importing changesets. This is where we finally create the git commits corresponding to the mercurial changesets. For each changeset, we expand its RawChunk, find the git tree we created in the previous phase that corresponds to the associated manifest, and create a git commit for that tree, with the right date, author, and commit message. For data that appears in the mercurial changeset that can t be stored or doesn t make sense to store in the git commit (e.g. the manifest SHA1, the list of changed files[*], or some extra metadata like the source of rebases), we keep some metadata we ll store in git notes later on. [*] Fun fact: the list of changed files stored in mercurial changesets does not necessarily match the list of files in a git diff between the corresponding git commit and its parents, for essentially two reasons:
  • Old buggy versions of mercurial have generated erroneous lists that are now there forever (they are part of what makes the changeset SHA1).
  • Mercurial may create new revisions for files even when the file content is not modified, most notably during merges (but that also happened on non-merges due to, presumably, bugs).
  so we keep it verbatim. On the graph, this is where both requested and in-use bytes are only slightly increasing. By the end of this phase, we made about half a billion more allocator calls. Requested allocations went up to 1.06GB, for close to 7.7 million live allocations. Compared to the end of the previous phase, that added 700k allocations, and 400MB. By now, we expect everything from the Reading changesets phase to have been released (at least the raw data we kept there), which means we may have allocated at most around 700MB (400MB + 300MB), for a total of 1.5M additional allocations (700k + 840k). All these are extra data we keep for the next and final phase. It s hard to evaluate the exact size we d expect here in memory, but if we divide by the number of changesets (345k), that s less than 5 allocations per changeset and less than 2KB per changeset, which is low enough not to raise eyebrows, at least for now.
• Finalizing the clone. The final phase is where we actually go ahead storing the mappings between mercurial and git SHA1s (all 2.7M of them), the git notes where we store the data necessary to recreate mercurial changesets from git commits, and a cache for mercurial tags. On the graph, this is where the requested and in-use bytes, as well as the number of live allocations peak like crazy (up to 21M allocations for 2.27GB requested). This is very much unwanted, but easily explained with the current state of the code. The way the mappings between mercurial and git SHA1s are stored is via a tree similar to how git notes are stored. So for each mercurial SHA1, we have a file that points to the corresponding git SHA1 through git links for commits or directly for blobs (look at the output of git ls-tree -r refs/cinnabar/metadata^3 if you re curious about the details). If I remember correctly, it s faster if the tree is created with an ordered list of paths, so the code created a list of paths, and then sorted it to send commands to create the tree. The former creates a new str of length 42 and a tuple of 3 elements for each and every one of the 2.7M mappings. With the 37 bytes overhead by str instance and the 56 + 3 * 8 bytes per tuple, we have at least 429MB wasted. Creating the tree itself keeps the corresponding fast-import commands in a buffer, where each command is going to be a tuple of 2 elements: a pointer to a method, and a str of length between 90 and 93. That s at least another 440MB wasted. I already fixed the first half, but the second half still needs addressing.

Overall, except for the stupid spike during the final phase, the manifest DAG and the glibc allocator runaway memory use described in previous posts, there is nothing terribly bad with the git-cinnabar memory usage, all things considered. Mozilla-central is just big. The spike is already half addressed, and work is under way for the glibc allocator runaway memory use. The manifest DAG, interestingly, is actually mostly useless. It s only used to track the heads of the DAG, and it s very much possible to track heads of a DAG without actually storing the entire DAG. In fact, that s what git-cinnabar already does for changeset heads so we would only need to do the same for manifest heads. One could argue that the 1.4GB of raw RevChunk data we re keeping in memory for later user could be kept on disk instead. I haven t done this so far because I didn t want to have to handle temporary files (and answer questions like where to put them? , what if there isn t enough disk space there? , what if disk access is slow? , etc.). But the majority of this data is from manifests. I m already planning changes in how git-cinnabar stores manifests data that will actually allow to import them directly, instead of keeping them in memory until files are imported. This would instantly remove 1.18GB of memory usage. The downside, however, is that this would be more CPU intensive: Importing changesets will require creating the corresponding git trees, and getting the stored manifest data. I think it s worth, though. Finally, one thing that isn t obvious here, but that was found while analyzing why RSS would be going up despite memory usage going down, is that git-cinnabar is doing way too many reallocations and substring allocations. So let s look at two metrics that hopefully will highlight the problem:

• The cumulated amount of requested memory. That is, the sum of all sizes ever given to malloc, realloc, calloc, etc.
• The compensated cumulated amount of requested memory (naming is hard). That is, the sum of all sizes ever given to malloc, calloc, etc. except realloc. For realloc, we only count the delta in size between what the size was before and after the realloc.

Assuming all the requested memory is filled at some point, the former gives us an upper bound to the amount of memory that is ever filled or copied (the amount that would be filled if no realloc was ever in-place), while the the latter gives us a lower bound (the amount that would be filled or copied if all reallocs were in-place). Ideally, we d want the upper and lower bounds to be close to each other (indicating few realloc calls), and the total amount at the end of the process to be as close as possible to the amount of data we re handling (which we ve seen is around 3TB). and this is clearly bad. Like, really bad. But we already knew that from the previous post, although it s nice to put numbers on it. The lower bound is about twice the amount of data we re handling, and the upper bound is more than 10 times that amount. Clearly, we can do better. We ll see how things evolve after the necessary code changes happen. Stay tuned.

Apologies the first blog post got sent out by mistake. Weather comparisons between the two countries Last year, I had come to know that this year s debconf is happening in Canada, a cold country. Hence, few weeks/month back, I started trying to find information online when I stumbled across few discussion boards where people were discussing about Innerwear and Outerwear and I couldn t understand what that was all about. Then somehow stumbled across this Video, which is of a game called the Long Dark and just seeing couple of episodes it became pretty clear to me why the people there were obsessing with getting the right clothes and everything about it. Couple of Debconf people were talking about the weather in Montreal, and surprise, surprise it was snowing there, in fact supposed to be near the storm of the century. Was amazed to see that they have a website to track how much snow has been lifted. If we compare that to Pune, India weather-wise we are polar opposites. There used to be a time, when I was very young, maybe 5 yrs. old that once the weather went above 30 degree celsius, rains would fall, but now its gonna touch 40 degrees soon. And April and May, the two hottest months are yet to come. China Gate Before I venture further, I was gifted the book China Gate written by an author named William Arnold. When I read the cover and the back cover, it seemed the story was set between China and Taiwan, later when I started reading it, it shares history of Taiwan going back 200 or so odd years. This became relevant as next year s Debconf, Debconf 2018 will be in Taiwan, yes in Asia very much near to India. I am ashamed to say that except for the Tiananmen Square Massacre and the Chinese High-Speed Rail there wasn t much that I knew. According to the book, and I m paraphrasing here the gist I got was that for a long time, the Americans promised Taiwan it will be an Independent country forever, but due to budgetary and other political constraints, the United States took the same stand as China from 1979. Interestingly, now it seems Mr. Trump wants to again recognize Taiwan as a separate entity from China itself but as is with Mr. Trump you can t be sure of why he does, what he does. Is it just a manoeuvrer designed to out-smart the chinese and have a trade war or something else, only time will tell. One thing which hasn t been shared in the book but came to know via web is that Taiwan calls itself Republic of China . If Taiwan wants to be independent then why the name Republic of China ? Doesn t that strengthen China s claim that Taiwan is an integral part of China. I don t understand it. The book does seduce you into thinking that the events are happening in real-time, as in happening now. That s enough OT for now. Population Density As well in the game and whatever I could find on the web, Canada seems to be on the lower side as far as population is concerned. IIRC, few years back, Canadians invited Indian farmers and gave them large land-holdings for over 100 years on some small pittance. While the link I have shared is from 2006, I read it online and in newspapers even as late as in 2013/2014. The point being there seems to be lot of open spaces in Canada, whereas in India we fight for even one inch literally, due to overpopulation. This sharing reminded me of Mark of Gideon . While I was young, I didn t understand the political meaning of it and still struggle to understand about whom the show was talking about. Was it India, Africa or some other continent they were talking about ? This also becomes obvious when you figure out the surface area of the two countries. When I had started to learn about Canada, I had no idea, nor a clue that Canada is three times the size of India. And this is when I know India is a large country. but seeing that Canada is thrice larger just boggled my mind. As a typical urbanite, would probably become mad if in a rural area in Canada. Montreal, however seems to be something like Gwalior or Bangalore before IT stormed in, seems to be a place where people can work, play and have quite a few gardens as well. Rail This is one thing that is similar in both the great countries. India has Indian Railways and while the Canadians have their own mountain railway called viarail. India chugs on its 68k kilometre network, Canada is at fourth position with 52k network. With thrice the land size, it should have been somewhere where Russia is or even better than them. It would be interesting if a Canadian/s comment about their railway network and why it is so bad in terms of reach. As far as food is concerned, somebody shared this Also, have no idea if Canadian trains are as entertaining as Indian ones, in terms of diverse group of people as well as variety of food to eat as also shared a bit in the video. I am not aware whether Via Rail is the only network operator and there are other network operators unlike Indian Railways which has monopoly on most of the operations. Business houses, Political Families This is again something that is similar in both the countries, it seems (from afar) that its only few business houses and more importantly political families which have governed for years. From what little I could understand, both India and Canada have first past the post system which as shared by its critics is unfair to new and small parties. It would be interesting to see if Canada does a re-think. For India, it would need a massive public education outreach policy and implementation. We just had elections in 5 states of India with U.P. (with respect to area-size and population density) and from the last several years, the EVM s (Electronic Voting Machines) tries to make sure that nobody could know which area which party got the most votes. This is to make sure the winning party is not able to take revenge on people or areas which did not vote for them. Instead you have general region counting of votes with probably even the Election Commission not knowing which EVM went to what area and what results are there in sort of double-blind methodology. As far as Business houses are concerned, I am guessing it s the same world-over, only certain people hold the wealth while majority of us are in hard-working, non-wealthy status. Northern Lights, Aurora Borealis Apart from all the social activities that Montreal is famous for, somebody told/shared with me that it is possible to see the Northern Lights, Aurora Borealis can be seen in Canada. I dunno how true or not it is, while probably in Montreal it isn t possible to see due to light pollution, but maybe around 40-50 kms. from the city ? Can people see it from Canada ? IF yes, how far would you have to go ? Are there any companies or people who take people to see the Northern Lights. While I still have to apply for bursary, and if that gets ok, then try getting the visa, but if that goes through, apart from debconf and social activities happening in and around Montreal, Museums, Music etc. , this would be something I would like to experience if it s possible. While I certainly would have to be prepared for the cold that would be, if it s possible, no offence to debconf or anybody else but it probably would be the highlight of the entire trip if its possible. This should be called/labelled as the greatest show on earth TM.
Filed under: Miscellenous Tagged: # Population Density, #Area size, #Aurora Borealis, #Canada, #Trains, DebConf, India, politics

Weather comparisons between the two countries Few days/weeks back, I had come to know that Canada, where this year s debconf is happening is cold country. I started trying to find information online when I stumbled across few boards where people were discussing about innerwear and outerwear and I couldn t understand what that was all about. Then somehow stumbled across this game, it s called the Long Dark and just seeing couple of episodes it became pretty clear to me why the people there were obsessing with getting the right clothes and everything about it. Couple of Debconf people were talking about weather in Montreal, and surprise, surprise it was snowing there, in fact supposed to be near the storm of the century. Was amazed to see that they have a website to track how much snow has been lifted. If we compare that to Pune, India weather-wise we are polar opposites. There used to be a time, when I was very young, maybe 5 yrs. old that once the weather went above 30 degree celcius, rains would fall, but now its gonna touch 40 degrees soon. And April and May, the two hottest months are yet to come. China Gate Before I venture further, I was gifted the book China Gate written by an author named William Arnold. When I read the cover and the backcover, it seemed the story was set between China and Taiwan, later when I started reading it, it shares history of Taiwan going back 200 or so odd years. This became relevant as next year s Debconf, Debconf 2018 will be in Taiwan, yes in Asia very much near to India. I am ashamed to say that except for the Tiananmen Square Massacre and the Chinese High-Speed Rail there wasn t much that I knew. According to the book, and I m paraphrasing here the gist I got was that for a long time, the Americans promised Taiwan it will be an Independent country forever, but due to budgetary and other political constraints, the United States took the same stand as China from 1979 and now it seems Mr. Trump wants to again recognize Taiwan as a separate entity from China itself. One thing which hasn t been shared in the book but came to know via web is that Taiwan calls itself Republic of China . If Taiwan wants to be independent then why the name Republic of China ? Doesn t that strengthen China s claim that Taiwan is an integral part of China. I don t understand it. The book does seduce you into thinking that the events are happening in real-time, as in happening now. That s enough OT for now. Population Density As well in the game and whatever I could find on the web, Canada seems to be on the lower side as far as population is concerned. IIRC, few years back, Canadians invited Indian farmers and gave them large land-holdings for over 100 years on some small pittance. While the link I have shared is from 2006, I read it online and in newspapers even as late as in 2013/2014. The point being there seems to be lot of open spaces in Canada, whereas in India we fight for even one inch literally, due to overpopulation. This sharing reminded me of Mark of Gideon . While I was young, I didn t understand the political meaning of it and still struggle to understand about whom the show was talking about. Was it India, Africa or some other continent they were talking about ? This also becomes obvious when you figure out the surface area of the two countries. When I had started to learn about Canada, I had no idea, nor a clue that Canada is three times the size of India. And this is when I know India is a large country. but seeing that Canada is thrice larger just boggled my mind. As a typical urbanite, would probably become mad if in a rural area in Canada. Montreal, however seems to be something like Gwalior or Bangalore before IT stormed in, seems to be a place where people can work, play and have quite a few gardens as well. Rail This is one thing that is similar in both the great countries. India has Indian Railways and while the Canadians have their own mountain railway called viarail. India chugs on its 68k kilometre network, Canada is at fourth position with 52k network. With thrice the land size, it should have been somewhere where Russia is or even better than them. It would be interesting if Canadians comment about their railway network and why it is so bad in terms of reach. Business houses, Political Families This is again something that is similar in both the countries, it seems (from afar) that its only few business houses and more importantly political families which have governed for years.
Filed under: Miscellenous

DebConf17 will take place in Montreal, Canada in August 2017. We are working hard to provide fuel for hearts and minds, to make this conference once again a fertile soil for the Debian Project flourishing. Please join us and support this landmark in the Free Software calendar. Eighteen companies have already committed to sponsor DebConf17! With a warm welcome, we'd like to introduce them to you. Our first Platinum sponsor is Savoir-faire Linux, a Montreal-based Free/Open-Source Software company which offers Linux and Free Software integration solutions and actively contributes to many free software projects. "We believe that it's an essential piece [Debian], in a social and political way, to the freedom of users using modern technological systems", said Cyrille B raud, president of Savoir-faire Linux. Our first Gold sponsor is Valve, a company developing games, social entertainment platform, and game engine technologies. And our second Gold sponsor is Collabora, which offers a comprehensive range of services to help its clients to navigate the ever-evolving world of Open Source. As Silver sponsors we have credativ (a service-oriented company focusing on open-source software and also a Debian development partner), Mojatatu Networks (a Canadian company developing Software Defined Networking (SDN) solutions), the Bern University of Applied Sciences (with over 6,600 students enrolled, located in the Swiss capital), Microsoft (an American multinational technology company), Evolix (an IT managed services and support company located in Montreal), Ubuntu (the OS supported by Canonical) and Roche (a major international pharmaceutical provider and research company dedicated to personalized healthcare). ISG.EE, IBM, Bluemosh, Univention and Skroutz are our Bronze sponsors so far. And finally, The Linux foundation, R seau Koumbit and adte.ca are our supporter sponsors. Become a sponsor too! Would you like to become a sponsor? Do you know of or work in a company or organization that may consider sponsorship? Please have a look at our sponsorship brochure (or a summarized flyer), in which we outline all the details and describe the sponsor benefits. For further details, feel free to contact us through sponsors@debconf.org, and visit the DebConf17 website at https://debconf17.debconf.org.