Search Results: "reg"

4 January 2017

Raphaël Hertzog: My Free Software Activities in December 2016

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donors (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. Debian LTS I was allocated 10 hours to work on security updates for Debian 7 Wheezy. During this time I did the following: Misc packaging With the strong freeze approaching, I had some customer requests to push packages into Debian and/or to fix packages that were in danger of being removed from stretch. While trying to bring back uwsgi into testing I filed #847095 (libmongoclient-dev: Should not conflict with transitional mongodb-dev) and #847207 (uwsgi: FTBFS on multiple architectures with undefined references to uwsgi_* symbols) and interacted on some of the RC bugs that were keeping the package out of testing. I also worked on a few new packages (lua-trink-cjson, lua-inotify, lua-sandbox-extensions) that enhance hindsight in some use cases and sponsored a rozofs update in experimental to fix a file conflict with inn2 (#846571). Misc Debian work Debian Live. I released two live-build updates. The second update added more options to customize the grub configuration (we use it in Kali to override the theme and add more menu entries) both for EFI boot and normal boot. Misc bugreports. #846569 on libsnmp-dev to accomodate the libssl transition (I noticed the package was not maintained, I asked for new maintainers on debian-devel). #847168 on devscripts for debuild that started failing when lintian was failing (unexpected regression). #847318 on lintian to not emit spurious errors for kali packages (which was annoying with the debuild regression above). #847436 for an upgrade problem I got with tryton-server. #847223 on firefoxdriver as it was still depending on iceweasel instead of firefox. Sponsorship. I sponsored a new version of asciidoc (#831965) and of ssldump 0.9b3-6 (for libssl transition). I also uploaded a new version of mutter to fix #846898 (it was ready in SVN already). Distro Tracker Not much happening, I fixed #814315 by switching a few remaining URLs to https. I merged patches from efkin to fix the functional test suite (#814315), that was a really useful contribution! The same contributer started to tackle another ticket (#824912) about adding an API to retrieve action items. This is a larger project and needs some thoughts. I still have to respond to him on his latest patches (after two rounds already). Misc stuff I updated the letsencrypt-sh salt formula for version 0.3.0 and added the possibility to customize the hook script to reload the webserver. The @planetdebian twitter account is no longer working since twitterfeed.com closed doors and the replacement (dlvr.it) is unhappy about the RSS feed of planet.debian.org. I filed bug #848123 against planet-venus since it does not preserve the isPermalink attribute in the guid tag Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Dirk Eddelbuettel: digest 0.6.11

A new minor release with version number 0.6.11 of the digest package is now on CRAN and in Debian. It is mostly a maintenance release. Sometime last spring we were asked to consider changing the license GPL-2 to GPL (>= 2). Having gotten agreement of all copyright holders, this could finally happen. But it so happens that the last yay just after the last release, so it took another cycle. In other changes, I also made the makeRaw function fully generic and added documentation. The pull request by Jim Hester to add covr support was finally folded in, leading as always to some gaming and improvement of the coverage metrics. A polite request by Radford Neal to support a nosharing option in base::serialize was also honoured; this should help for use with his interesting pqR variant of R. CRANberries provides the usual summary of changes to the previous version. For questions or comments use the issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

3 January 2017

Reproducible builds folks: Reproducible Builds: week 88 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday December 25 and Saturday December 31 2016: Media coverage Reproducible bugs filed Chris West: Chris Lamb: Rob Browning: Reviews of unreproducible packages 7 package reviews have been added, 12 have been updated and 14 have been removed in this week, adding to our knowledge about identified issues. 2 issue types have been updated: Weekly QA work During our reproducibility testing, the following FTBFS bugs have been detected and reported by: diffoscope development strip-nondeterminism development try.diffoscope.org development tests.reproducible-builds.org Misc. This week's edition was written by Chris Lamb, Holger Levsen and was reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Russ Allbery: Review: Castle of Wizardry

Review: Castle of Wizardry, by David Eddings
Series: The Belgariad #4
Publisher: Del Rey
Copyright: May 1984
Printing: September 1991
ISBN: 0-345-33570-8
Format: Mass market
Pages: 373
Castle of Wizardry is the fourth book of the Belgariad and very much the middle of the story. Despite coming after an intermediate climax, this isn't the sort of series you can start in the middle. The problem with intermediate climaxes in a long series is that the next book can be a bit of a let-down as the characters do the necessary regrouping and reorienting and determine next steps. I think that hurts Castle of Wizardry quite a lot. The best bits are at the beginning, as the party escapes the consequences of Magician's Gambit, collects one more party member, shows us a lot more of Errand (who is always delightful), and confounds Relg's life and world view considerably. (Although there's a good bit of authorial fiat in the last.) This builds into a major story event, which would normally help avoid the let-down after the climax, but it's the major story event that is so frequently and obviously foreshadowed that you'd have to be as dumb as, well, Garion to not know what's coming. That gives a certain "yes, yes, we know already" tone to proceedings that robs it of its ability to rebuild tension. That said, the appeal of this series continues to be in the small details. While the first major event of this book goes pretty much as expected (including Ce'Nedra's reaction, which is just as irritating as you might be expecting), my favorite part was the endless, bubbly enthusiasm of the incredibly powerful artifact that features heavily. Usually epic fantasy will treat such world-breaking objects with seriousness and awe, as treasures to be admired and sacred (or terrifying) great works. See, for instance, the ur-example of Tolkien's rings, both the One Ring and the elven rings of power. Eddings manages a mix of awe and bemusement that doesn't undermine their power but that adds a delightful human element. This series pulls off treating a powerful magical artifact like an over-enthusiastic puppy without making it feel any less dangerous. It's a very neat, and I think underappreciated, trick to pull off. Another part of this book I liked, if a more stock one, is Garion's reactions after the big story event. This isn't the first book to portray basic decency and thoughtfulness as a major feature in people from humble backgrounds elevated to great power, but I always enjoy seeing that. Garion stops whining (mostly) and starts acting like a decent, level-headed person who doesn't assume he has the right to arrange other people's lives, and is rewarded for it. Real life is often not that fair or ethical, but that's why one reads wish-fulfillment fantasy like this: for a world in which being a good person is rewarded. However, Eddings does have some structural issues here. The narrative arc of this book, as a stand-alone entity, is odd. Its most dramatic event is in the middle, and then has a long traveling section that's, by comparison, much less exciting. The events of that section feel more like random encounters than a coherent part of the story, and are preceded by the most utterly ridiculous temper tantrums. I think the tantrums were meant to be pure humor, but my reaction was primarily eye-rolling. I have a hard time reconciling a screaming fit and breaking furniture with the long life experience and thoughtful planning of the character in question. And then there is the Ce'Nedra section that closes this book, and Ce'Nedra in this book more generally. To be fair, Castle of Wizardry is clearly intended to be Ce'Nedra's moment to grow as a person and stop being a childish brat. This does happen somewhat, and there are moments in the last section of the book where she does admirable things. But I couldn't quite believe in the mechanism, and it doesn't help that it's one of the most ham-handed bits of pre-ordained success in a book that has a tendency towards them. That undermines the real attempts Eddings makes to ground that success in Ce'Nedra's actual skills. Also undermining this is that those skills are manipulating people shamelessly, which Eddings seems to think is charming and attractive and I... don't. But the real problem is that I flatly disbelieve in Ce'Nedra as a character, or, given the apparent existence of such a creature, the level of tolerance that other characters show her. If I'd been Polgara, within fifteen minutes of meeting her I would have been seriously debating whether the destruction of the world might be a small price to pay for the satisfaction of dumping her down the nearest well. And not only is she awful by herself but Garion also descends to the same level whenever he's around her, until both of them are behaving like blithering idiots. I suspect part of my issue is that, to the extent that she is realistic at all, Ce'Nedra is the sort of intensely high-drama person who I have some amount of life experience with, and that life experience says "do not let this person anywhere near your life." Red flags all over everything. Garion needs to nope the hell out, because this will not end well. (Except, of course, it will, because it's that sort of series and the power of the author is strong.) I want female characters with real agency in my fantasy, and I want a female protagonist who is doing things of equal importance as the male protagonist (not that Eddings attempts to go that far). But Ce'Nedra reads like a fictional character written by someone who had never met a woman, but has extensively studied female supporting characters in books about junior-high social cliques and then tried to reconcile that research with the stereotype of women as manipulative seductresses. Yes, this series is full of stereotypes and characters painted in broad strokes, but Ce'Nedra is several tiers below every other supporting character in the book in both believability and in my desire to read about her. It's not that Eddings doesn't know how to write women at all. Polgara still falls into a few stereotyped categories, but she's sensible, opinionated, and has clear agency throughout the story. Taiba is delightful, if minor here. Poledra is absolutely wonderful whenever she appears. Some of the queens are obviously practical and sensible. And this book features a surprisingly good resolution to the subplot around Barak's wife, although the mechanism is a bit eye-rollingly cliched. Ce'Nedra's character is unusual for the series and almost certainly a deliberate authorial choice, and this book is supposed to be her coming of age. But I am baffled by that choice, and there's very little about it that I enjoyed reading. One more minor complaint: Silk gets a "tragic secret" in this book, and I really wish he hadn't. More time with Silk is always a feature, and I still love the character, but his oddities were already adequately explained by both his innate character and his way of dealing with a particularly awkward court situation. (One that ties into Eddings's habit of using some bad relationship stereotypes, but that's a rant for another day.) I think this additional tragic secret was gratuitous and really unnecessary, not to mention weirdly implausible and oddly cruel towards the other character involved. I was hoping that Magician's Gambit had turned a corner for the series, but Castle of Wizardry, despite having some neat moments, has some serious flaws. One more book to go, in which we learn that some of the eastern races have redeeming qualities! Followed by Enchanter's End Game. Rating: 6 out of 10

Maria Glukhova: Getting to know diffoscope better

I apologize to all potential readers of this blog for not writing a comprehensive Introduction post with details of the project I am taking part in during my internship, as well as some story about how I ended up there. Let me just say that I was a Debian user for years when I discovered it is taking part in Outreachy as one of organisations. Their Reproducible Builds effort has a noble goal and a bunch of great people behind it - I had no chances not to get excited by it. Looking for a place where my skills could be of any use, I discovered diffoscope - the tool for in-depth comparassion of files, archives etc. My mentor, Mattia Rizzolo, supported my decision to work on it, so now I am concentrating my efforts on improving diffoscope. As my first steps, I am doing small (but hopefully still somewhat important) job of fixing existing bugs. It helps me to better understand how diffoscope works, as well as introduces me to the workflow of opensource development. During December, I have done several small contributions, mostly fixing bugs.

Test data and jessie-backports First of them could be somewhat called cleaning up after my own mistake, although that mistake wasn t trivial. During the application period, I have fixed a bug with diffoscope failing while comparing symlinks to directory. That was a small change, but I included some tests for that case anyway. And that actually caused problems. With these tests, I included test data: two folders with symlinks. All was good in unstable version of Debian, but in jessie-backports, that commit caused build to fail. After some digging, I discovered the problem was caused by build process including copying that data. That was done using shutils Python module, and older version of that module, included in jessie, could not handle copying symlinks to directory properly. Thanks to my mentor for giving me a hint on how to resolve this: using temporary folders and creating these symlinks at runtime. That way, we ensured tests run without problems during build process on jessie. What have I learned: A great deal, actually. I spent too much time on that one, but I learned how to build packages, what happens during dpkg-buildpackage run and what debhelper tools are for. I also learned a bit about what chroot is and how to use it for testing.

ICC profile files and file type recognizing regexp Another one was also about failing tests and, therefore, failing build. Failing tests were all due to ICC files were not recognized by diffoscope. Turned out libmagic got an update which changed the description of ICC profile files. Diffoscope was relying on regexp applied to file type description to recognize the file, so I changed regexp to reflect the changes in libmagic. What have I learned: How diffoscope recognizes file types. Got me thinking: maybe there is a better way? That regexp-based approach is doomed to cause problems with every file type description change. I have this question still lingering in my mind - maybe I will come up with an idea later.

Order-like difference in text files Next, I decided to do something a bit bigger and fullfilled a feature request. That request was for detecting order-like difference in text files (when files has the same lines, but in different order). I did it by collecting added and removed lines in diff output in lists, sorting and then comparing them. Sadly, I forgot about one particular case - when one of the files is missing the newline at the end of file. I was kindly reminded of that quite soon in comments on the bug-tracker (thanks danielsh!) and have already fixed that. I also recieved feedback on how better implement it deeper in the diffoscope - not using the results of diff, but rather comparing sum of hashes of the lines directly in the difference module. I am yet to try that. What have I learned: That a call to diff is actually the slowest part of the diffoscope run when done on two big text files. Could it help somehow in speeding it up? I don t know yet. I also learned to comment on bugs in Debian bugtracker and was surprised by how much feedback I got. Thanks to my mentor for pushing me to do that - I definetely need to overcome my fear of communications to be more effective!

Random FTBFS There was also a very nasty bug that caused diffoscope to fail to be built from source randomly, failing with non-informative Fatal Python error: deallocated None. It already seemed strange when it was first reported; It got only more strange when suddenly that bug ceased to be reproducible. We hoped that would mean that bug was caused by some external tool, and was fixed there. Turns out it was not that easy. I tested this on two separate computers and on virtual machine; I used different versions of diffoscope. Well. Seems like that bug is still somehow tied to diffoscope version and not some external tool version - I still can do git checkout 64 and be able to reproduce the bug (still randomly, though). Although I spent quite a lot of time on that one, the only result was the information about connection between bug apperances and diffoscope version. I still wasn t able to get to the root of the problem - hopefully, someone else will be able to, given the information I found. What have I learned: git-bisect! Thanks to my friend for pointing me to it, that tool came handy in that situation. Also, got some experience in catching nasty bugs like that (pity that no experience in squashing them). I had some extra time commitements in December, one of them (Reproducible Builds Summit II) connected to my internship and one (my exam session in university) not. In January, I should be able to allocate more time to that work - I hope it will help me achieve more significant results. Many thanks to Mattia Rizzolo, Chris Lamb, Holger Levsen and all other folks of Reproducible Builds project - I cannot stress enough how important your support is to me. Wish you all a great 2017!

2 January 2017

Markus Koschany: My Free Software Activities in December 2016

Welcome to gambaru.de. Here is my monthly report that covers what I have been doing for Debian. If you re interested in Android, Java, Games and LTS topics, this might be interesting for you. Debian Android Debian Games Debian Java Debian LTS This was my tenth month as a paid contributor and I have been paid to work 13,5 hours on Debian LTS, a project started by Rapha l Hertzog. In that time I did the following: Non-maintainer uploads

1 January 2017

Russ Allbery: 2016 Book Reading in Review

So, I did not accomplish my reading goal for 2016 (reading and reviewing more books in 2016 than I did in 2015). Many things contributed to that, but the root cause was that I didn't make enough time for reading. Much of the time that could have gone to reading went to playing Hearthstone (a good thing) and obsessing over the 2016 US election (mostly a waste of time and particularly energy, although I'm not sure I could have stopped). That said, I did get quite a lot of reading done at the end of the year, and I'm hoping to keep up that momentum for next year. In 2016, I did a lot of re-reading and comfort reading. I'm probably going to continue with some of the re-reading in 2017, since I'm enjoying it, but my reading goal for the year is to get back to reading award nominees and previous award winners. There's so much great new stuff being published that I want to discover. I'm not going to set an explicit goal around number of books, but I am going to make an effort to carve out more time in my schedule for reading books (and less for reading on-line news). This was another year with two 10 out of 10 books. One of them was a re-read: Lord of Emperors, the second book of Guy Gavriel Kay's Sarantine Mosaic. (I also re-read the first book this year, Sailing to Sarantium, and gave it a 9.) I like nearly all of Kay's historical fantasies, but this duology is one of my personal favorites. The second 10 out of 10 book was a complete surprise: A Man Called Ove by Fredrik Backman (translated by Henning Koch). My mother found this book and suggested it to me, and I loved every moment of it. I will definitely be reading more of Backman's work. There were two more fiction standouts this year: Digger by Ursula Vernon, and The Philosopher Kings by Jo Walton. The first is a graphic novel about a wombat who is trying to make her way home from an unexpected detour into a mess of magic and gods. The second is the middle book in a trilogy about an attempt to construct Plato's Just City and all of the philosophical and social problems that ensue (with some bonus science fiction and fantasy elements). Both of them are excellent. Walton is consistently one of my favorite authors, and Ursula Vernon was my great discovery of a new author to read this year. (Not that I've followed through on that much, the year in reading being what it was, but I will be doing so.) My favorite non-fiction book of the year continues my interest in time management in general and Mark Forster's approaches in particular. Secrets of Productive People was the last book I reviewed this year (just a coincidence, not any intentional attempt to set things up for next year) and the best version of his overall approach to date. If you've not read any of Forster based on my previous recommendations, this is a good place to start. Also worth mentions were Jeffrey Toobin's The Run of His Life, on the O.J. Simpson case, and Andrew Groen's The Empires of EVE, on the history of player empires in the EVE Online MMORPG. I Kickstarted the latter and didn't regret it. The full analysis includes some additional personal reading statistics, probably only of interest to me.

Joey Hess: p2p dreams

In one of the good parts of the very mixed bag that is "Lo and Behold: Reveries of the Connected World", Werner Herzog asks his interviewees what the Internet might dream of, if it could dream. The best answer he gets is along the lines of: The Internet of before dreamed a dream of the World Wide Web. It dreamed some nodes were servers, and some were clients. And that dream became current reality, because that's the essence of the Internet. Three years ago, it seemed like perhaps another dream was developing post-Snowden, of dissolving the distinction between clients and servers, connecting peer-to-peer using addresses that are also cryptographic public keys, so authentication and encryption and authorization are built in. Telehash is one hopeful attempt at this, others include snow, cjdns, i2p, etc. So far, none of them seem to have developed into a widely used network, although any of them still might get there. There are a lot of technical challenges due to the current Internet dream/nightmare, where the peers on the edges have multiple barriers to connecting to other peers. But, one project has developed something similar to the new dream, almost as a side effect of its main goals: Tor's onion services. I'd wanted to use such a thing in git-annex, for peer-to-peer sharing and syncing of git-annex repositories. On November 13th, I started building it, using Tor, and I'm releasing it concurrently with this blog post.
git-annex's Tor support replaces its old hack of tunneling git protocol over XMPP. That hack was unreliable (it needed a TCP on top of XMPP layer) but worse, the XMPP server could see all the data being transferred. And, there are fewer large XMPP servers these days, so fewer users could use it at all. If you were using XMPP with git-annex, you'll need to switch to either Tor or a server accessed via ssh.
Now git-annex can serve a repository as a Tor onion service, and that can then be accessed as a git remote, using an url like tor-annex::tungqmfb62z3qirc.onion:42913. All the regular git, and git-annex commands, can be used with such a remote. Tor has a lot of goals for protecting anonymity and privacy. But the important things for this project are just that it has end-to-end encryption, with addresses that are public keys, and allows P2P connections. Building an anonymous file exchange on top of Tor is not my goal -- if you want that, you probably don't want to be exchanging git histories that record every edit to the file and expose your real name by default. Building this was not without its difficulties. Tor onion services were originally intended to run hidden websites, not to connect peers to peers, and this kind of shows.. Tor does not cater to end users setting up lots of Onion services. Either root has to edit the torrc file, or the Tor control port can be used to ask it to set one up. But, the control port is not enabled by default, so you still need to su to root to enable it. Also, it's difficult to find a place to put the hidden service's unix socket file that's writable by a non-root user. So I had to code around this, with a git annex enable-tor that su's to root and sets it all up for you.
One interesting detail about the implementation of the P2P protocol in git-annex is that it uses two Free monads to build up actions. There's a Net monad which can be used to send and receive protocol messages, and a Local monad which allows only the necessary modifications to files on disk. Interpreters for Free monad actions can chose exactly which actions to allow for security reasons. For example, before a peer has authenticated, the P2P protocol is being run by an interpreter that refuses to run any Local actions whatsoever. Other interpreters for the Net monad could be used to support other network transports than Tor.
When two peers are connected over Tor, one knows it's talking to the owner of a particular onion address, but the other peer knows nothing about who's talking to it, by design. This makes authentication harder than it would be in a P2P system with a design like Telehash. So git-annex does its own authentication on top of Tor. With authentication, users would need to exchange absurdly long addresses (over 150 characters) to connect their repositories. One very convenient thing about using XMPP was that a user would have connections to their friend's accounts, so it was easy to share with them. Exchanging long addresses is too hard. This is where Magic Wormhole saved the day. It's a very elegant way to get any two peers in touch with each other, and the users only have to exchange a short code phrase, like "2-mango-delight", which can only be used once. Magic Wormhole makes some security tradeoffs for this simplicity. It's got vulnerabilities to DOS attacks, and its MITM resistance could be improved. But I'm lucky it came along just in time. So, it takes only installing Tor and Magic Wormhole, running two git-annex commands, and exchanging short code phrases with a friend, perhaps over the phone or in an encrypted email, to get your git-annex repositories connected and syncing over Tor. See the documentation for details. Also, the git-annex webapp allows setting the same thing up point-and-click style. The Tor project blog has throughout December been featuring all kinds of projects that are using Tor. Consider this a late bonus addition to that. ;) I hope that Tor onion services will continue to develop to make them easier to use for peer-to-peer systems. We can still dream a better Internet.
This work was made possible by all my supporters on Patreon.

Russ Allbery: Review: Secrets of Productive People

Review: Secrets of Productive People, by Mark Forster
Publisher: Teach Yourself
Copyright: 2015
ISBN: 1-4736-0885-6
Format: Kindle
Pages: 289
Regular readers of my reviews will know that Mark Forster is my favorite writer on time management and productivity. That's mostly because of his flexible toolkit approach that talks about theory and overall goals and then describes multiple ways to get there, rather than presenting a single system that will solve all your problems. There are a lot of writers who explain productivity tips and tricks or describe systems that work for them. There are fewer who can explain why those tricks work (and why they sometimes don't work), and even fewer who can put them into a meaningful analytical framework for thinking about productivity. Forster has several books, but they're a mixed bag. His clearest and most coherent book prior to this one was Do It Tomorrow. Secrets of Productive People is organized differently, chopped up in to small bite-sized chunks with synopses to an extent that it felt a bit choppy to me, but apart from that it's the closest I've seen to an updating of Do It Tomorrow. His other books can be slight (The Pathway to Awesomeness) or downright weird (How to Make Your Dreams Come True). I still have a soft spot for Do It Tomorrow, and I like how it was organized a bit better than this one, but I think Secrets of Productive People has become my new recommendation for where to start with Forster. Secrets of Productive People is divided into five sections: The basics of productivity, the productive attitude, productive projects, aids to productivity, and productivity in action. Each part is divided into several small chapters, which open with a generous helping of quotes about some productivity topic (and I'm going to go back and save some of those), present some easily-digestible related set of thoughts (usually with an exercise), and conclude with a summary. Each chapter is about the length of a long blog post. I think this structure interferes with developing an idea at greater length, but it does make for good reading material in an environment where you're regularly interrupted or only have five or ten minutes. I think I've mentioned in every review that Forster won me over by being willing to talk about the problems with attempting to do too much, not just presenting a system to allow one to accomplish more. Some other books, such as David Allen's famous Getting Things Done, seem to assume you already know what you need to get done, or will easily be able to figure that out when you think for a while, and just need a system to manage all the things you've decided to do. Forster takes the opposite approach, and this book is the clearest yet on this point: most productivity problems are not from being insufficiently efficient, but from doing the wrong things and too many things. You don't need more time; everyone gets the same amount of time. You need to do the right things with the time you have, and that usually means doing fewer things. Readers of Forster's previous books will recognize many of the themes here. Some of the techniques from Do It Tomorrow and some of the exercises from Get Everything Done show up again here. But Forster has streamlined and focused the advice, discarded some things, made his task management recommendations less elaborate and more focused, and spends much more time hammering home the point that the only prioritization that really matters is whether you commit to doing something or don't. The specific task management system he recommends here is one of the variations he's been talking about in his blog and is much simpler than the Do It Tomorrow system: pick five tasks, work on them until you've finished three of them, and then refill to five tasks. I've been using it since reading this book, replacing one of Forster's more elaborate systems from his blog, and it's surprisingly effective. He breaks down in some detail why this works and how to extract additional useful feedback information from it, and now I want to do some of the additional exercises he describes. (That said, I'm still dubious about his advice to not keep any larger to-do list and only rely on your analysis at the time of refilling the list to decide what to do. I use this system with a supplemental, longer list of ideas for future tasks, and that works better for me, although I do have to fight forming a sense of obligation about the things on the longer list. David Allen still has a point that if you don't write down a complete inventory of the things you're worrying about, your brain will try to obsess over them to keep from forgetting them.) The productive projects section told me some things I needed to hear about time commitments. Projects take regular, focused attention, and starting numerous things without giving them that attention is much worse than doing far fewer things but doing them regularly and reliably. I think Forster's coaching on focus and persistence is very valuable; the trick, of course, is building up your willpower for it and learning to say no. This is in-line with recent psychological analysis of multi-tasking and its various negative effects. The working to-do list capped at five things provides enough variety to mentally shift gears if one runs out of steam on some specific project while maintaining enough focus to not leave things behind half-done. The productivity aids section provides a more random collection of tips and tricks, many of which I've seen before in his previous books. I've mostly not tried these, so can't say much about how effective they are, but Forster's ideas almost always sound interesting and plausible. I thought the weakest part of the book was the last section, on applied productivity. Here, Forster takes various life areas (exercise, parenting, finances, writing, etc.) and talks about how the principles of this book can be applied to them. Each area could be a book in itself, and the short essay format of these chapters doesn't do justice to large topics. The result is a rather repetitive section that just stresses analysis, metrics, and repeated, focused attention all valid points, but ones you can pick up from the previous chapters. I don't think these case studies added much value. Do It Tomorrow is still my favorite Forster book to read, but I think Secrets of Productive People is now the best and most polished overview of his time management and productivity approach. If you're interested in this topic and not already sick of Forster from my previous recommendations, this would be a great place to start. I got a lot out of it even with all the time management reading I've previously done, and will probably re-read sections of it and try more of the exercises. Rating: 9 out of 10

Sam Hartman: 2016

I was in such a different place at the beginning of 2016: I was poised to continue to work to help the world find love. Professionally, I was ready to make a much needed transition and find new projects to work on. The year 2016 sucked. It feels like the year was filled with many different versions of the universe saying "Not interested in what you have to offer." At the beginning of the year, I had the energy to try and reach across large disagreements and help find common ground even when compromise was not possible. Now, my blog lies fallow because I cannot find the strength to be vulnerable enough to write what I would choose to say. Certainly a lot of the global changes of the last year have felt like a strong rejection of the world I'd like to see. However, many of the rejections have been personal. Beyond that, most of the people who stood as pillars of support in my life, together helping me find the strength to be vulnerable, are no longer available. When the universe sends such strong messages, it's a good idea to ask whether you are on the right path. I certainly have discovered training I need and things I need to improve in order to avoid making costly mistakes that hurt others. However, among the rejections were clear demonstrations of the value of reaching out with love and compassion. Besides, this is what I'm called to do. It's what I want to do. I certainly will not force it on anyone. But it looks like the next few years may be a hard struggle to find pockets of people interested in that work, finding people who will choose love even in the current world, along with some difficult training to learn from challenges of the last year. Amongst all this, my life if filled with love. There are new connections even as old connections are strained. There is always the hope of finding new ways to connect when the old ones are no longer right. I will rebuild and regain safety. I have the tools to do that. The process is just long and complicated.

31 December 2016

Chris Lamb: Free software activities in December 2016

Here is my monthly update covering what I have been doing in the free software world (previous month):
Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users. The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced either maliciously or accidentally during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. This month:
I also made the following changes to our tooling:
diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.

  • Optimisations:
    • Avoid unnecessary string manipulation writing --text output (~20x speedup).
    • Avoid n iterations over archive files (~8x speedup).
    • Don't analyse .deb s twice when comparing .changes files (2x speedup).
    • Avoid shelling out to colordiff by implementing color support directly.
    • Memoize calls to distutils.spawn.find_executable to avoid excessive stat(1) syscalls.
  • Progress bar:
    • Show current file / ELF section under analysis etc. in progress bar.
    • Move the --status-fd output to use JSON and to include the current filename.
  • Code tidying:
    • Split out the try.diffoscope.org client so that it can be released separately on PyPI.
    • Completely rework the diffoscope and diffoscope.comparators modules, grouping similar utilities into their own modules, etc.
  • Miscellaneous:
    • Update dex_expected_diffs test to ensure compatibility with enjarify 1.0.3.
    • Ensure that running from Git will always use that checkout's Python modules.
    • Add a simple profiling framework.

strip-nondeterminism

strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.

  • Makefile.PL: Change NAME argument to a Perl package name.
  • Ensure our binaries are available in autopkgtest tests.

try.diffoscope.org

trydiffoscope is a web-based version of the diffoscope in-depth and content-aware diff utility. Continued thanks to Bytemark for sponsoring the hardware.

  • Show progress bar and position in queue, etc. (#25 & #26)
  • Promote command-line client with PyPI instructions.
  • Increase comparison time limit to 90 seconds.

buildinfo.debian.net

buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.

  • Added support for version 0.2 .buildinfo files. (#15)

Debian
Debian LTS

This month I have been paid to work 13 hours on Debian Long Term Support (LTS). In that time I did the following:
  • "Frontdesk" duties, triaging CVEs, etc.
  • Issued DLA 733-1 for openafs, fixing an information leak vulnerability. Due to incomplete initialization or clearing of reused memory, directory objects could contain 'dead' directory entry information.
  • Issued DLA 734-1 for mapserver closing an information leakage vulnerability.
  • Issued DLA 737-1 for roundcube preventing arbitrary remote code execution by sending a specially crafted email.
  • Issued DLA 738-1 for spip patching a cross-site scripting (XSS) vulnerability.
  • Issued DLA 740-1 for libgsf fixing a null pointer deference exploit via a crafted .tar file.

Debian Uploads
  • redis:
    • 3.2.5-5 Add RunTimeDirectory=redis to systemd .service files.
    • 3.2.5-6 Add missing Depends on lsb-base for /lib/lsb/init-functions usage in redis-sentinel's initscript.
    • 3.2.6-1 New upstream release.
    • 4.0-1 & 4.0-rc2-1 New upstream experimental releases.
  • aptfs: 0.9-1 & 0.10-1 New upstream releases.


Debian FTP Team

As a Debian FTP assistant I ACCEPTed 107 packages: android-platform-libcore, compiz, debian-edu, dehydrated, dh-cargo, gnome-shell-extension-pixelsaver, golang-1.8, golang-github-btcsuite-btcd-btcec, golang-github-elithrar-simple-scrypt, golang-github-pelletier-go-toml, golang-github-restic-chunker, golang-github-weaveworks-mesh, golang-google-genproto, igmpproxy, jimfs, kpmcore, libbio-coordinate-perl, libdata-treedumper-oo-perl, libdate-holidays-de-perl, libpgobject-type-bytestring-perl, libspecio-library-path-tiny-perl, libterm-table-perl, libtext-hogan-perl, lighttpd, linux, linux-signed, llmnrd, lua-geoip, lua-sandbox-extensions, lua-systemd, node-cli-cursor, node-command-join, node-death, node-detect-indent, node-domhandler, node-duplexify, node-end-of-stream, node-first-chunk-stream, node-from2, node-glob-stream, node-has-binary, node-inquirer, node-interpret, node-is-negated-glob, node-is-unc-path, node-lazy-debug-legacy, node-lazystream, node-load-grunt-tasks, node-merge-stream, node-object-assign-sorted, node-orchestrator, node-pkg-up, node-resolve-from, node-resolve-pkg, node-rx, node-sorted-object, node-stream-shift, node-streamtest, node-string.prototype.codepointat, node-strip-bom-stream, node-through2-filter, node-to-absolute-glob, node-unc-path-regex, node-vinyl, openzwave, openzwave-controlpanel, pcb-rnd, pd-upp, pg-partman, postgresql-common, pybigwig, python-acora, python-cartopy, python-codegen, python-efilter, python-flask-sockets, python-intervaltree, python-jsbeautifier, python-portpicker, python-pretty-yaml, python-protobix, python-sigmavirus24-urltemplate, python-sqlsoup, python-tinycss, python-watson-developer-cloud, python-zc.customdoctests, python-zeep, r-cran-dbitest, r-cran-dynlm, r-cran-mcmcpack, r-cran-memoise, r-cran-modelmetrics, r-cran-plogr, r-cran-prettyunits, r-cran-progress, r-cran-withr, ruby-clean-test, ruby-gli, ruby-json-pure, ruby-parallel, rustc, sagemath, sbuild, scram, sidedoor, toolz & yabasic. I additionally filed 4 RC bugs against packages that had incomplete debian/copyright files against jimfs, compiz, python-efilter & ruby-json-pure.

30 December 2016

Antoine Beaupr : My free software activities, November and December 2016

Debian Long Term Support (LTS) Those were 8th and 9th months working on Debian LTS started by Raphael Hertzog at Freexian. I had trouble resuming work in November as I had taken a long break during the month and started looking at issues only during the last week of November.

Imagemagick, again I have, again, spent a significant amount of time fighting the ImageMagick (IM) codebase. About 15 more vulnerabilities were found since the last upload, which resulted in DLA-756-1. In the advisory, I unfortunately forgot to mention CVE-2016-8677 and CVE-2016-9559, something that was noticed by my colleague Roberto after the upload... More details about the upload are available in the announcement. When you consider that I worked on IM back in october, which lead to an upload near the end of November covering around 80 more vulnerabilities, it doesn't look good for the project at all. Of the 15 vulnerabilities I worked on, only 6 had CVEs assigned and I had to request CVEs for the other 9 vulnerabilities plus 11 more that were still unassigned. This lead to the assignment of 25 distinct CVE identifiers as a lot of issues were found to be distinct enough to warrant their own CVEs. One could also question how many of those issues affect the fork, Graphicsmagick. A lot of the vulnerabilities were found through fuzzing searches that may not have been tested on Graphicsmagick. It seems clear to me that a public corpus of test data should be available to test regressions and cross-project vulnerabilities. It's already hard enough to track issues withing IM itself, I can't imagine what it would be for the fork to keep track of those issues, especially since upstream doesn't systematically request CVEs for issues that they find, a questionable practice considering the number of issues we all need to keep track of.

Nagios I have also worked on the Nagios package and produced DLA 751-1 which fixed two fairly major issues (CVE-2016-9565 and CVE-2016-9566) that could allow remote root access under certain conditions. Fortunately, the restricted permissions setup by default in the Debian package made both exploits limited to information disclosure and privilege escalation if the debug log is enabled. This says a lot about how proper Debian packaging can help in limiting the attack surface of certain vulnerabilities. It was also "interesting" to have to re-learn dpatch to add patches to the package: I regret not converting it to quilt, as the operation is simple and quilt is so much easier to use. People new to Debian packaging may be curious to learn about the staggering number of patching systems historically used in Debian. On that topic, I started a conversation about how much we want to reuse existing frameworks when we work on those odd packages, and the feedback was interesting. Basically, the answer is "it depends"...

NSS I had already worked on the package in November and continued the work in December. Most of the work was done by Raphael, which fixed a lot of issues with the test suite. I tried to wrap this up by fixing CVE-2016-9074, the build on armel and the test suite. Unfortunately, I had to stop again because I ran out of hours and the fips test suite was still failing, but fortunately Raphael was able to complete the work with DLA-759-1. As things stand now, the package is in better shape than in other suites as tests (Debian bug #806639) and autopkgtest (Debian bug #806207) are still not shipped in the sid or stable releases.

Other work For the second time, I forgot to formally assign myself a package before working on it, which meant that I wasted part of my hours working on the monit package. Those hours, of course, were not counted in my regular hours. I still spent some time reviewing mejo's patch to ensure it was done properly and it turned out we both made similar patches working independently, always a good sign. As I reported in my preliminary November report, I have also triaged issues in libxml2, ntp, openssl and tiff. Finally, I should mention my short review of the phpMyAdmin upload, among the many posts i sent to the LTS mailing list.

Other free software work One reason why I had so much trouble getting paid work done in November is that I was busy with unpaid work...

manpages.debian.org A major time hole for me was trying to tackle the manpages.debian.org service, which had been offline since August. After a thorough evaluation of the available codebases, I figured the problem space wasn't so hard and it was worth trying to do an implementation from scratch. The result is a tool called debmans. It took, obviously, way longer than I expected, as I experimented with Python libraries I had been keeping an eye on for a while. For the commanline interface, I used the click library, which is really a breeze to use, but a bit heavy for smaller scripts. For a web search service prototype, I looked at flask, which was also very interesting, as it is light and simple enough to use that I could get started quickly. It also, surprisingly, fares pretty well in the global TechEmpower benchmarking tests. Those interested in those tools may want to look at the source code, in particular the main command (using an interesting pattern itself, __main__.py) and the search prototype. Debmans is the first project for which I have tried the CII Best Practices Badge program, an interesting questionnaire to review best practices in software engineering. It is an excellent checklist I recommend every project manager and programmer to get familiar with. I still need to complete my work on Debmans: as I write this, I couldn't get access to the new server the DSA team setup for this purpose. It was a bit of a frustrating experience to wait for all the bits to get into place while I had a product ready to test. In the end, the existing manpages.d.o maintainer decided to deploy the existing codebase on the new server while the necessary dependencies are installed and accesses are granted. There's obviously still a bunch of work to be done for this to be running in production so I have postponed all this work to January. My hope is that this tool can be reused by other distributions, but after talking with Ubuntu folks, I am not holding my breath: it seems everyone has something that is "good enough" and that they don't want to break it...

Monkeysign I spent a good chunk of time giving a kick in the Monkeysign project, with the 2.2.2 release, which features contributions from two other developers, which may be a record for a single release. I am especially happy to have adopted a new code of conduct - it has been an interesting process to adapt the code of conduct for such a relatively small project. Monkeysign is becoming a bit of a template on how to do things properly for my Python projects: documentation on readthedocs.org including a code of conduct, support and contribution information, and so on. Even though the code now looks a bit old to me and I am embarrassed to read certain parts, I still think it is a solid project that is useful for a lot of people. I would love to have more time to spend on it.

LWN publishing As you may have noticed if you follow this blog, I have started publishing articles for the LWN magazine, filed here under the lwn tag. It is a way for me to actually get paid for some of my blogging work that used to be done for free. Reports like this one, for example, take up a significant amount of my time and are done without being paid. Converting parts of this work into paid work is part of my recent effort to reduce the amount of time I spend on the computer. An funny note: I always found the layout of the site to be a bit odd, until I looked at my articles posted there in a different web browser, which didn't have my normal ad blocker configuration. It turns out LWN uses ads, and Google ones too, which surprised me. I definitely didn't want to publish my work under banner ads, and will never do so on this blog. But it seems fair that, since I get paid for this work, there is some sort of revenue stream associated with it. If you prefer to see my work without ads, you can wait for it to be published here or become a subscriber which allows you to get rid of the ads on the site. My experience with LWN is great: they're great folks, and very supportive. It's my first experience with a real editor and it really pushed me in improving my writing to make better articles that I normally would here. Thanks to the LWN folks for their support! Expect more of those quality articles in 2017.

Debian packaging I have added a few packages to the Debian archive:
  • magic-wormhole: easy file-transfer tool, co-maintained with Jamie Rollins
  • slop: screenshot tool
  • xininfo: utility used by teiler
  • teiler (currently in NEW): GUI for screenshot and screencast tools
I have also updated sopel and atheme-services.

Other work Against my better judgment, I worked again on the borg project. This time I tried to improve the documentation, after a friend asked me for help on "how to make a quick backup". I realized I didn't have any good primer to send regular, non-sysadmin users to and figured that, instead of writing a new one, I could improve the upstream documentation instead. I generated a surprising 18 commits of documentation during that time, mainly to fix display issues and streamline the documentation. My final attempt at refactoring the docs eventually failed, unfortunately, again reminding me of the difficulty I have in collaborating on that project. I am not sure I succeeded in making the project more attractive to non-technical users, but maybe that's okay too: borg is a fairly advanced project and not currently aimed at such a public. This is yet another project I am thinking of creating: a metabackup program like backupninja that would implement the vision created by liw in his A vision for backups in Debian post, which was discarded by the Borg project. Github also tells me that I have opened 19 issues in 14 different repositories in November. I would like to particularly bring your attention to the linkchecker project which seems to be dead upstream and for which I am looking for collaborators in order to create a healthy fork. Finally, I started on reviving the stressant project and changing all my passwords, stay tuned for more!

Shirish Agarwal: Mausaji, Samosaji

mausaji

Mausaji, Never born Never died, Always in the heart.

Dear Friends, I have shared a few times that I had a privileged childhood. I never had led a hand-to-mouth existence but more than that I was privileged to have made the acquaintance of Jaipur wale Mausaji while I was very young. I have been called miserly by my cousin sisters whenever they write letters to me and I don t answer simply because whatever I feel for them, words feel inadequate and meaningless. The same thing applies in this as well. I am sharing few bits here as there are too many memories of a golden past which will not let me go till I have shared a few of them. First let me start by sharing the relation I had with him. By relation he was my mother s-sister s husband. In English, he would probably be termed as Maternal Uncle although he was much more than that. My one of the first remembrances of him was during Madhu Didi s Shaadi (marriage). Madhu Didi is uncle s daughter and I would have been a impish 4-5 year old at the time. This was the first time I was gonna be part of The Great North Indian Wedding and I didn t know what was in store for me as I had grown in Pune. I remember finishing my semester tests and mummy taking me to Pune Station. I was just excited that I would be travelling somewhere and had no clue what would be happening. We landed in Agra, took another train and landed in Jaipur in the middle of the night at their home at Sangram Colony. While I had known few of the cousins, I was stumped to see so many cousins jumping out of everywhere. The look on my face was one of stupefaction and surprise . The only thing which would closely resemble that would be Bilbo s 111st Birthday party in Lord of the Rings (Part 1). In fact, by a curious quirk/twist of fate, I came to know of a Naniji or somebody like that who by relation was far elder to me, while she was either my age or below my age. As was customary, had to bow down sheepishly. As a somewhat alone boy, to be thrown in this rambunctious bunch and be the babe in the woods, I was quickly chopped and eaten up but had no complaints. I would get into trouble onto a drop of a hat. While Mausiji would threaten me, Mausaji would almost always defend me. While Mausiji could see through me, the twinkle in Mausaji s eyes used to tell me that while he knew what I was upto, for reasons unknown, he would always defend me. Mausaji s Sangram Colony s house became my cricket ground, football ground and all and any ground to play and be. Mausaji and his brothers used to live near each other and the lane they had, had hardly any vehicles on it, so all the cousins could play all they want with me being the longest, perhaps unconsciously trying either to make for lost time or knowing/unknowing this was too good to last. Today s Pokemon generation might not be able to get it but that s alright. They also had a beautiful garden where Mausiji used to grow vegetables. While playing, we sometimes used to hurt her veggies (unconsciously) or just have shower with the garden hose. Mausaji used to enjoy seeing our antics. One of the relatives even had a dog who used to join in the fun sometimes. When mummy and Mausiji expressed concern about the dog biting, Mausaji would gently brush it aside. One of the other things in Didi s marriage is we got a whole lot of sweets. While Mausiji tried to keep us in check with sweets, both Manish Bhaiya and Mausaji used to secrete sweets from time to time. When I was hungry and used to steal food (can t wait till the appointed time) either Bhaiya or Mausaji would help me with the condition I would have to take the blame if and when we got caught as we invariably did. Mausaji s house had a basement where all the secreted sweets and food used to get in. Both me and Manish Bhaiya would be there and we would have a riot in ourselves. We would enjoy the adrenaline when we were stealing the food. As I was pretty young, I was crazy about the Tom and Jerry cartoons that used to come on Television that time. I and Bhaiya used to act like Jerry and/or his cronies while Mausiji would invariably be the Tom with Mausaji all-knowing about it but acting as a mere bystander. I remember him egging me for many of the antics I would do and get in trouble in but as shared would also be defended by him. The basement was also when I was becoming a teenager where Manish Bhaiya showed me his collection and we had a heart-to-heart about birds and bees. While whatever little I had known till that time was from school-friends and my peers at school and I didn t know what was right or wrong. Bhaiya clarified lot of things, concepts which I was either clueless or confused about. When I look back now, it is possible that Mausaji might have instructed Bhaiya to be my tutor as I used to be somewhat angry and lash out by the confusing times. As we used to go there for part of holidays, I remember doing all sorts of antics to make sure I would get an extra day, an extra hour to be there. I never used to understand why we had to go to meet the other relatives when all the fun I could have was right there only, couldn t Mummy know/see that I used to enjoy the most here. Mausaji was a clothier as we understand the term today and a gentleman to the core. He was the co-owner of Rajputana Cloth Store in Jaipur. Many VIP s as well as regular people used to visit him for getting clothes designed and stitched under his watchful eyes. I never saw him raise his voice against any of the personnel working under him and used to be a thorough gentleman to one and all. Later, as I grew up I came to know and see that people would phone up and just ask him to do the needful. He would get the right cloth, stitch it right and people used to trust him for that. He was such an expert on cloth and type of clothes, that by mere touch he could talk/share about what sort of cloth it is. One of his passions was driving and from the money he had saved, he had got an Ambassador Car. Every day or every other day or whenever he felt like it, he used to take either the gang or me with mummy or me with anybody else. Each ride used to be an adventure in itself, with a start beginning and an end. I always used to watch out for the car-rides as I knew we would get sweets or something as well as he would regale us with stories about a place here and there. There was a childlike curiosity and interest in him which was infectious to one and all. The only weakness that he had was he liked to drink wine once in a while. When I was a kid, I was never able to give him company, only few years back, for the first time I was able to share wine with him which was also a memory I treasure. Those who know him closely knew the many up and downs that he went through, but as a gentleman he never let on the hurts he had or didn t curse his fate or anything else that we do when things go bad from our perspective. While there is much to write about him, it will not accomplish anything that is not known about him. I ll add with the private joke that was between him and me. When I was little, I used to call him Mausaji, Samosaji for a) I liked Samosa and b) Samosa has a bit thick skin outside and underneath it s all gravy. In reality though, he was butter all the way. I miss you Mausaji and wish I could turn the clock back and come with Mummy to visit both Mausiji and you. I hope your new journey takes you to even further heights than this life. Savouring the memories mummy and I, hope we meet you again in some new Avataar
Filed under: Miscellenous Tagged: #antics, #growing up, #holidays, #Manish Bhiaya, #Mausaji, #Sangram Colony

29 December 2016

Sven Hoexter: Out of the comfort zone: OpenSuSE support for an ordinary user - f*ck my morals

A friend of mine choose for $reasons to install the latest OpenSuSE 42.2 release as his new laptop operating system. It's been a while that I had contact with the SuSE Linux distribution. Must be around 12 years or so. The unsual part here is that I've to support a somewhat eccentric, but mostly ordinary user of computers. And to my surprise it's still hard to just plug in your existing stuff and expect it work. I've done so many dirty things to this installation in the last three days, my system egineering heart is bleeding. printing with a Canon Pixma iP100 printer This is a small portable Canon printer, about four years old. It provides a decent quality and its main strength is that it's small and really portable. Sadly the gutenprint driver just pushes through a blank page. No ink wasted on it at all. So the only reasonable other choice was a four year old binary rpm package provided by Canon. It has a file dependency on "libtiff.so.3" which is no longer available in recent GNU/Linux distributions. So I cheated and
- unpacked the tarball
- installed the rpm from the "packages" folder
zypper install cnijfilter-common-3.70-1.x86_64.rpm cnijfilter-ip100series-3.70-1.x86_64.rpm
... and choose to ignore the missing file dependency on libtiff.so.5.
ln -s /usr/lib64/libtiff.so /usr/lib64/libtiff.so.5
- re-ran the ./install.sh which registered the printer with cups and does whatever else
  magic is included in 1906 lines of shell.
To my surprise this driver still works and provides the expected quality. Though it's just a question of time until this setup will break. Be it an incompatible ABI change in libtiff or another lib in use by those Canon provided tools. QGIS and gdal with ECW support While the printer stuff is a rather common use case, having a map viewer for map files in the ECW format is the eccentric part. I found some hints on stackoverflow and subsequently https://trac.osgeo.org/gdal/wiki/ECW that a non-free library is required and a specific build of gdal. Then QGIS should be able to work with ECW files. Lucky us there is at least a OpenSuSE repository for gdal and QGIS. So I did the following:
zypper addrepo http://download.opensuse.org/repositories/Application:/Geo/openSUSE_Leap_42.2/Application:Geo.repo
zypper install qgis
Then I had to download the non-free ECW SDK from http://download.hexagongeospatial.com/downloads/erdas-ecw-jp2-sdk-v5.3-%28linux%29 - you'll and up with a '.bin' installer file. The installation process left me with "ERDAS-ECW_JPEG_2000_SDK-5.3.0" folder in my $HOME. I moved that one to /opt. Next step is adding the library to the ldconfig search path.
echo "/opt/ERDAS-ECW_JPEG_2000_SDK-5.3.0/Desktop_Read-Only/lib/x64/release/" > /etc/ld.so.conf.d/ecw.conf; ldconfig
Now it was "just" about rebuild gdal with ECW support. So I downloaded the required source packages with "zypper source-install gdal", edited the spec somewhere in "/usr/src/" to make the following modifications
--with-ecw=/opt/ERDAS-ECW_JPEG_2000_SDK-5.3.0/Desktop_Read-Only
added to the "./configure" invocation. And somewhere at the top we had to relax the requirement that all installed files have to be referenced inside the package.
%define _unpackaged_files_terminate_build 0
As a last step I had to "rpmbuild -ba" the package and force the installation via zypper once more, because this time we have a file depedency on the libecw stuff and it's obviously not listed in the rpm database. Last but not least I tried to put the gdal build on hold with
zypper addlock gdal libgdal20
to ensure it's not removed on the next update. Other non-free tools Beside of those two issues I had to install a range of other non-free tools, but currently they work without further issues or modifications. One is Teamviewer (i686 multiarch rpm) and the other one is XnViewMP. XnView is also able to show ECW files, but only the smaller ones. It crashes on bigger ones but that's also the case on Windows. Then there is also (required by some Italian map related websites) the ugly Adobe Flash Plugin for Firefox, but that one is sadly still a widespread issue. We also tried to try out the nvidia graphic drivers but at the moment we could only get the build in Intel card to work. Usually the preferred solution from my point of view but sometimes we see rendering glitches and I'm not sure if it's the driver or something else. my personal take away I hate to admit it but it's nothing extraordinary that was requested here. But still it took me the better part of two evenings to figure everything out. And even now it's not properly integrated and doomed to fail any day due to various updates and changes in the surounding ecosystem. I've full sympathy for every average user that would give up after two hours of research and try&error on this journey. For the printer drivers I'm happy to blame Canon. The printer situation as a whole improved from my point of view during the last decade, but it's still a pain in the ass with the very short shelf life you usually see with consumer models. For the ECW case one could discuss if it would be legally possible and helpful to do ugly dlopen() stuff to dynamcially load the shared libs. But then again someone has to make his hands dirty during the build and discussions about the legal use of header files will be the next chapter (hello Oracle). It's just ugly. Actually I know too little about the world of image formats to judge if someone has a good reason to keep this format commercial or not. From my personal point of view it's not useful and maybe even morally wrong. Technically one could argue if it would make sense to keep a local copy of the gdal build in "/opt" and start QGIS with a modified library path to prefer the private gdal build. Not sure if that is any better. On the other hand there are evolving mechanism like flatpack that would ease the handling of such situations. Buth then again we would be catering non-free software. It feels a lot like giving up. While my private working environment is except for firmware blobs free, I now created for someone a real "FrankenSuSE" to satisfy his everyday needs. On the one hand we now have another mostly satisfied user of a mostly free operating system. On the other hand that was only possible by adding a vast amount of non-free software. For sure we did not win the war, I'm not even sure if we've won a single battle here. It's just frustrating to see what is required to get someone up and running. With my personal attitude towards open source software it even feels wrong to invest so much time into fiddling with non-free components. What is still missing We currently lack an image viewer that allows us to print only a selection of an image, which is useful to print parts of a map. That usually works with XnView on Windows but does not work with the Linux version at the moment. I also tried gwenview and geeqie and had the same issue. Not sure if it's maybe a bug in XnView or one of the Qt parts (gwenview is also Qt based). I did not research that yet. Update: I spent quite some time looking into open bug reports for geeqie and gwenview. Seems the feature to print only a section of an image is something new. I've created #374299 (gwenview) and #457 (geeqie). For XnView I expect it's a difference between XnViewMP (the portable version) and the Windows only XnView Classic. Needs to be clarified and it might be worth to try XnView Classic with wine. Maybe printing with wine via cups works, I found at least some results for it on the internet.

Reproducible builds folks: Reproducible Builds: week 87 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday December 18 and Saturday December 24 2016: Media coverage 100% Of The 289 Coreboot Images Are Now Built Reproducibly by Phoronix, with more details in German by Pro-Linux.de. We have further reports on our Reproducible Builds World summit #2 in Berlin from Rok Garbas of NixOS as well as Clemens Lang of MacPorts Debian infrastructure work Dak now archives buildinfo files thanks to a patch from Chris Lamb. We also have mostly finalised a design of how they will be distributed by the Debian FTP mirror network which we will start implementing soon. This is great for the future of Debianb but unfortunately this also means that we won't have .buildinfo files for Stretch as Debian will not rebuild its source packages and because these binary packages currently in the archive were mostly built with dpkg > 1.18.11. reprepro/5.0.0-1 has added support for dealing with .buildinfo files that are included in .changes files. (Closes: #843402) Reproducible work in other projects The Chromium project is now working on making their build process (mostly) deterministic. Their motivation is to save both "[money] (less hardware is required) and developer time (reduced latency by having less work to do on the TS and CI)". Unreproducible bugs filed Reviews of unreproducible packages 39 package reviews have been added, 75 have been updated and 44 have been removed in this week, adding to our knowledge about identified issues. 2 issue types have been updated: Weekly QA work During our reproducibility testing, some FTBFS bugs have been detected and reported by: diffoscope development diffoscope 66 was uploaded to unstable by Chris Lamb. It included contributions from: strip-nondeterminism development strip-nondeterminism 0.029-1 was uploaded to unstable by Chris Lamb. It included no new content from this week, but rather included contributions from previous weeks. reproducible-website development The website is now also accessible via the https://www.reproducible-builds.org URL. tests.reproducible-builds.org Misc. This week's edition was written by Ximin Luo, Holger Levsen & Chris Lamb and reviewed by a bunch of Reproducible Builds folks on IRC and the mailing lists.

Russ Allbery: Review: Pawn of Prophecy

Review: Pawn of Prophecy, by David Eddings
Series: The Belgariad #1
Publisher: Del Rey
Copyright: April 1982
Printing: August 1990
ISBN: 0-345-33551-1
Format: Mass market
Pages: 258
I hit a period in November when I felt like reading something entirely undemanding and fun, something that was predictable, comfortable, and guaranteed to be a happy ending. There are times in life for books like that. I've been vaguely thinking of re-reading the Belgariad for a while, which I'd not read since I was in college, and that seemed to fit the bill. Garion is a farm-boy in Sendaria, a quiet and industrious kingdom full of farmers. His only family is his Aunt Pol, who ran the kitchen at Faldor's farm. His life has all the hallmarks of being entirely unremarkable, despite occasional visits by a traveling vagabond, Wolf, who clearly knows his Aunt Pol. But then something extremely important is stolen elsewhere, Wolf and Pol have to leave to pursue it for reasons that are quite murky to Garion, and Pol is unwilling to leave Garion behind. This begins Garion's great adventure, in which he becomes far more important and powerful than he would have ever imagined. That's the promise, at least, and the stock plot of every series like this. And let me be clear up front: the Belgariad is very stock. That's part of what I was in the mood for, but don't come to this series looking for interesting differentiation. It's clearly in the center of the secondary world chosen one fantasy genre, with a great evil, forces of good arrayed against it, and someone who unknowingly has the power to stop evil but has to learn how to use it. If you're in the mood for this, every note will be comfortably familiar; if not, it might be boringly predictable. Unfortunately, this particular book does not cast the Belgariad in a good light. Garion's introduction to this story is intensely irritating, almost entirely because no one will tell him what's going on. Pol is excessively protective (smotheringly so), but shuts down any questions he has and doesn't explain why she's freaking out. (This doesn't become entirely clear until towards the end of the second book.) What little scraps Garion picks up, he has to get by eavesdropping. Even by the end of this book, when Garion has met royalty and started helping with larger political issues (albeit mostly by accident), everyone is keeping him in the dark and he doesn't believe the hints he's gotten. I can't remember how I reacted to this the first time I read this series, but the second time through, it was extremely frustrating. The frustration is stronger because any regular reader of this genre knows exactly what Garion is, although not precisely how the story will play out, and can make fairly good guesses about Pol and Wolf from early in the story. Everyone playing coy doesn't even add legitimate suspense or set up a big reveal. It just made me irritated at Garion's blindness (it's mildly understandable given how foreign the idea is to his life, but he seems remarkably dense even with that), and even more irritated at Wolf and, in particular, Aunt Pol for keeping him so thoroughly in the dark. Garion is very attached to his aunt; on this re-reading, I wasn't entirely sure why. She occasionally shows love and support, and she's certainly wise, but for most of the book she treats him with brusque condescension and tries to keep him as ignorant as possible. Garion's weird entanglement with a black rider that he'd seen for most of his life doesn't make this any better. It's creepy and disturbing, it's not entirely resolved in this book, and Garion is clearly in danger, which wasn't the vibe that I was looking for when starting this re-read. And it's hard to escape the feeling that Garion might have managed to tell someone if all the adult figures in his life weren't so secretive, dismissive, and excessively controlling. So, all in all, an inauspicious beginning. It was good enough to keep me reading when I first read the series, and it was good enough on this re-read, but just barely. The saving grace is Silk and Barak, when the story finally meets them. Those who have read this series will know that they're just the beginning of the party-building process that will take up much of this series (so many Lord of the Rings resonances), but the one thing I will give Eddings is that he writes entertaining supporting characters. Both of them are a bit stereotyped, but they add so much to this story. Silk in particular is a delight and remains a delight throughout the series, adding a much-needed level of snark and sneaky enjoyment that lightens so many of the serious episodes of this story. The Belgariad in general is highly derivative map exploration fantasy, and you shouldn't expect anything deeper. It is, however, enjoyable map exploration fantasy with some entertaining supporting cast members. Sadly, only the glimmers of that are visible from this book. Pawn of Prophecy is mostly an irritating tour of bad parenting techniques, refusal to talk to teenagers, sneaking around to get information that should have been legitimately shared, and taking forever to get around to the mythology that we all know is coming. I therefore can't recommend this book, although one does have to read it in order to read the rest of the series, which as I recall gets considerably better. Followed by Queen of Sorcery. Rating: 5 out of 10

27 December 2016

Joey Hess: multi-terminal applications

While learning about and configuring weechat this evening, I noticed a lot of complexity and unsatisfying tradeoffs related to its UI, its mouse support, and its built-in window system. Got to wondering what I'd do differently, if I wrote my own IRC client, to avoid those problems. The first thing I realized is, it is not a good idea to think about writing your own IRC client. Danger will robinson.. So, let's generalize. This blog post is not about designing an IRC client, but about exploring simpler ways that something like an IRC client might handle its UI, and perhaps designing something general-purpose that could be used by someone else to build an IRC client, or be mashed up with an existing IRC client. What any modern IRC client needs to do is display various channels to the user. Maybe more than one channel should be visible at a time in some kind of window, but often the user will have lots of available channel and only want to see a few of them at a time. So there needs to be an interface for picking which channel(s) to display, and if multiple windows are shown, for arranging the windows. Often that interface also indicates when there is activity on a channel. The most recent messages from the channel are displayed. There should be a way to scroll back to see messages that have already scrolled by. There needs to be an interface for sending a message to a channel. Finally, a list of users in the channel is often desired. Modern IRC clients implement their own UI for channel display, windowing, channel selection, activity notification, scrollback, message entry, and user list. Even the IRC clients that run in a terminal include all of that. But how much of that do they need to implement, really? Suppose the user has a tabbed window manager, that can display virtual terminals. The terminals can set their title, and can indicate when there is activity in the terminal. Then an IRC client could just open a bunch of terminals, one per channel. Let the window manager handle channel selection, windowing (naturally), and activity notification. For scrollback, the IRC client can use the terminal's own scrollback buffer, so the terminal's regular scrollback interface can be used. This is slightly tricky; can't use the terminal's alternate display, and have to handle the UI for the message entry line at the bottom. That's all the UI an IRC client needs (except for the user list), and most of that is already implemented in the window manager and virtual terminal. So that's an elegant way to make an IRC client without building much new UI at all. But, unfortunately, most of us don't use tabbed window managers (or tabbed terminals). Such an IRC client, in a non-tabbed window manager, would be a many-windowed mess. Even in a tabbed window manager, it might be annoying to have so many windows for one program. So we need fewer windows. Let's have one channel list window, and one channel display window. There could also be a user list window. And there could be a way to open additional, dedicated display windows for channels, but that's optional. All of these windows can be seperate virtual terminals. A potential problem: When changing the displayed channel, it needs to output a significant number of messages for that channel, so that the scrollback buffer gets populated. With a large number of lines, that can be too slow to feel snappy. In some tests, scrolling 10 thousand lines was noticiably slow, but scrolling 1 thousand lines happens fast enough not to be noticiable. (Terminals should really be faster at scrolling than this, but they're still writing scrollback to unlinked temp files.. sigh!) An IRC client that uses multiple cooperating virtual terminals, needs a way to start up a new virtual terminal displaying the current channel. It could run something like this:
x-terminal-emulator -e the-irc-client --display-current-channel
That would communicate with the main process via a unix socket to find out what to display. Or, more generally:
x-terminal-emulator -e connect-pty /dev/pts/9
connect-pty would simply connect a pty device to the terminal, relaying IO between them. The calling program would allocate the pty and do IO to it. This may be too general to be optimal though. For one thing, I think that most curses libraries have a singleton terminal baked into them, so it might be hard to have a single process control cursors on multiple pty's. And, it might be innefficient to feed that 1 thousand lines of scrollback through the pty and copy it to the terminal. Less general than that, but still general enough to not involve writing an IRC client, would be a library that handled the irc-client-like channel display, with messages scrolling up the terminal (and into the scrollback buffer), a input area at the bottom, and perhaps a few other fixed regions for status bars and etc. Oh, I already implemented that! In concurrent-output, over a year ago: a tiling region manager for the console I wonder what other terminal applications could be simplified/improved by using multiple terminals? One that comes to mind is mutt, which has a folder list, a folder view, and an email view, that all are shoehorned, with some complexity, into a single terminal.

Gunnar Wolf: Giving up on the Drupal 8 debianization

I am sad (but feel my duty) to inform the world that we will not be providing a Drupal 8 package in Debian. I filed an Intent To Package bug a very long time ago, intending to ship it with Jessie; Drupal 8 was so deep a change that it took their community overly long to achieve and stabilize. Still, Drupal 8 was released over a year ago today. I started working on debianizing the package shortly afterwards. There is also some online evidence As my call for help sent through this same blog. I have been too busy this last year. I let the packaging process lay dormant for too long, without even touching it for even half a year. Then, around September, I started working with the very nice guys of Indava, David and Enrique, and did very good advances. They clearly understood Debian's needs when it comes to full source inclusion (as D8 ships many minified Javascript libraries), attribution (as additionally to all those, many third-party PHP projects are bundled in the infamous vendor/ directory), and system-wide dependency management (as Drupal builds on some frameworks and libraries already available within Debian, chiefly Symfony, Doctrine, Twig... Even more, most of them appeared to work at the version levels we will be shipping, so all was dandy and for some weeks, I was quite optimistic on finishing the package on time and with the needed quality and testing. Yay! But... Reality bites. When I started testing my precious package... It broke in horrible ways. Uncomprehensible PHP errors (and I have to add here, I am a PHP newbie and am reluctant to learn better a language that strikes me as so inconsistent, so ugly), which we spent some time tackling... Of course, configuration changes are more than expected... But, just as we Debianers learnt some important lessons after the way-too-long Sarge freeze (ten years ago, many among you won't remember those frustrating days), Drupal learnt as well. They changed their release strategy Instead of describing it, those interested can read it at its source. What it meant for me, sadly, is that this process does not align with the Debian maintenance model. This means: The Drupal API stays mostly-stable between 8.0.x, 8.1.x, 8.2.x, etc. However, Drupal will incorporate new versions of their bundled libraries. I understood the new versions would be incorporated at minor-level branches, but if I read correctly some of my errors, some dependencies change even at patch-level updates. And... Well, if you update a PHP library, and the invoking PHP code (that is, Drupal) relies in this new version... Sadly, it makes it unmaintainable for Debian. So, long story short: I have decided to drop Drupal8 support in Debian. Of course, if somebody wants to pick up the pieces, the Git repository is still there (although I do plan on erasing it in a couple of weeks, as it means useless waste of project resources otherwise), and you could probably even target unstable+backports in a weird way (as it's software that, given our constraints, shouldn't enter testing, at least during a freeze). So... Sigh, a tear is dropped for every lost hour of work, and my depeest regrets to David and Enrique who put their work as well to make D8 happen in Debian. I will soon be closing the ITP and... Forgetting about the whole issue?

26 December 2016

Lucas Nussbaum: The Linux 2.5, Ruby 1.9 and Python 3 release management anti-pattern

There s a pattern that comes up from time to time in the release management of free software projects. To allow for big, disruptive changes, a new development branch is created. Most of the developers focus moves to the development branch. However at the same time, the users focus stays on the stable branch. As a result: This situation can grow up to a quasi-deadlock, with people questioning whether it was a good idea to do such a massive fork in the first place, and if it is a good idea to even spend time switching to the new branch. To make things more unclear, the development branch is often declared stable by its developers, before most of the libraries or applications have been ported to it. This has happened at least three times. First, in the Linux 2.4 / 2.5 era. Wikipedia describes the situation like this:

Before the 2.6 series, there was a stable branch (2.4) where only relatively minor and safe changes were merged, and an unstable branch (2.5), where bigger changes and cleanups were allowed. Both of these branches had been maintained by the same set of people, led by Torvalds. This meant that users would always have a well-tested 2.4 version with the latest security and bug fixes to use, though they would have to wait for the features which went into the 2.5 branch. The downside of this was that the stable kernel ended up so far behind that it no longer supported recent hardware and lacked needed features. In the late 2.5 kernel series, some maintainers elected to try backporting of their changes to the stable kernel series, which resulted in bugs being introduced into the 2.4 kernel series. The 2.5 branch was then eventually declared stable and renamed to 2.6. But instead of opening an unstable 2.7 branch, the kernel developers decided to continue putting major changes into the 2.6 branch, which would then be released at a pace faster than 2.4.x but slower than 2.5.x. This had the desirable effect of making new features more quickly available and getting more testing of the new code, which was added in smaller batches and easier to test. Then, in the Ruby community. In 2007, Ruby 1.8.6 was the stable version of Ruby. Ruby 1.9.0 was released on 2007-12-26, without being declared stable, as a snapshot from Ruby s trunk branch, and most of the development s attention moved to 1.9.x. On 2009-01-31, Ruby 1.9.1 was the first release of the 1.9 branch to be declared stable. But at the same time, the disruptive changes introduced in Ruby 1.9 made users stay with Ruby 1.8, as many libraries (gems) remained incompatible with Ruby 1.9.x. Debian provided packages for both branches of Ruby in Squeeze (2011) but only changed the default to 1.9 in 2012 (in a stable release with Wheezy 2013). Finally, in the Python community. Similarly to what happened with Ruby 1.9, Python 3.0 was released in December 2008. Releases from the 3.x branch have been shipped in Debian Squeeze (3.1), Wheezy (3.2), Jessie (3.4). But the python command still points to 2.7 (I don t think that there are plans to make it point to 3.x, making python 3.x essentially a different language), and there are talks about really getting rid of Python 2.7 in Buster (Stretch+1, Jessie+2). In retrospect, and looking at what those projects have been doing in recent years, it is probably a better idea to break early, break often, and fix a constant stream of breakages, on a regular basis, even if that means temporarily exposing breakage to users, and spending more time seeking strategies to limit the damage caused by introducing breakage. What also changed since the time those branches were introduced is the increased popularity of automated testing and continuous integration, which makes it easier to measure breakage caused by disruptive changes. Distributions are in a good position to help here, by being able to provide early feedback to upstream projects about potentially disruptive changes. And distributions also have good motivations to help here, because it is usually not a great solution to ship two incompatible branches of the same project. (I wonder if there are other occurrences of the same pattern?) Update: There s a discussion about this post on HN

25 December 2016

Gregor Herrmann: RC bugs 2016/46-51

it's amazing how many new bugs appear; luckily in the Debian Perl Group we're not too bad at fixing them as well. here's the list of my contributions over the last weeks:

Next.

Previous.