Search Results: "zed"

18 July 2014

Mario Lang: Croudsourced accessibility: Self-made digital menus

Something straight out from the real world: Menu cards in restaurants are not nice to deal with if you are blind. It is an old problem we grow used to ignoring over time, but still something that can be quite nagging. There are a lot of psychological issues involved in this one. Of course, you can ask for the menu to be read out to you by the staff. While they usually do their best, you end up missing out on some things most of the time. First of all, depending on the current workload in the restaurant, the staff will usually try to cut some time and not read everything to you. What they usually do is to try to understand what type of meal you are interested in, and just read the choices from that category to you. While this can be considered a service in some situations (human preprocessing), there are situations were you will definitely miss a highlight on the menu that you would have liked to choose if you knew that it was there. And even if the staff decides to read the complete menu to you (which is rare), you are confronted with the 7-things-in-my-head-at-once problem. It is usually rather hard to decide amongst a list of more then 7 items, because our short-term memory is sort of limited. What the sighted restaurant goers do, is to skip back and forth between the available options, until they hit a decisive moment. True, that can take a while, but it is definitely a lot easier if you can perform "random access reads" to the list of choices yourself. However, if someone presents a substantial number of choices to you in a row, as sequential speech, you loose the random access ability. You either remember every choice from the beginning and do your choosing mentaully (if you do have extraordinary mental abilities), or you end up asking the staff to read previous items aloud again. This can work, but usually it doesn't. At some point, you do not want to bother the staff anymore, and you even start to feel stupid for asking again and again, while this is something totally normal to every sighted person, just that "they" do their "random access browsing" on their own, so "they" have no need to feel bad about how long it takes them to decide, minus the typical social pressure that arises after a a certain time for everyone, at least if you are dining in a group. In very rare cases, you happen to meet staff that is truly "awake", doing their best to not let you feel that they might be pressed on time, and really taking as much time as necessary to help you make the perfect decision. This is rare, but if it happens, it is almost a magical moment. One of these moments, where there are no "artificial" barriers between humans doing communcation. Anyway, I am drifting away. The perfect solution to this problem is to provide random access browsing of a restaurant menu with the help of digital devices. Trying to make braille menus available in all restaurants is a goal which is not realistically reachable. Menus go out of date, and need changing. And getting a physical braille copy updated and reprinted is considerably more involved as with digital media. Restaurant owners will also likely not see the benefit to rpvide a braille card for a very small circle of customers. With a digital online menu, that is a completely different story. These days, almost every blind person in at least my social circles owns an iOS (or similar) device. These devices have speech synthesis and web browsers. Of course, some restaurants especially in urban areas do already have a menu online. I have found them manually with google and friends sometimes in the past, which has already given me the ability to actually sit back, and really comfortably choose amongst the available offerings myself, without having to bother a human, and without having to feel bad about (ab)using their time. However, the case where a restaurant really has their menu online is rather rare still in the area where I am. And, it can be tedious to google for a restaurant website. Sometimes, the website itself is just marginally accessible, which makes it even more frustrating to get a relaxed dinner-experience. I have discovered a location-based solution for the restaurant-menu problem recently. Foursquare offers the ability to provide a direct link to the menu in a restaurant-entry. I figured, since all you need to do is write a single webpage where the (common) menu items are listed per restaurant, that I could begin to create restaurant menus for my favourite locations, on my own. Well, not quite, but almost. I will sometimes need help from others to get the menu digitized, but that's just a one-time piece of work I hopefully can outsource :-). Once the actual content is in my INBUX, I create a nice HTML page listing the menu in a rather speech-based browser friendly way. I have begun to do this today, with the menu of a restaurant just about 500 meters away from my apartment. Unterm goldenen Dachl now has a menu online, and the foursquare change request to publish the corresponsing URL is already pending. I don't fully understand how the Foursquare change review process works yet, but I hope the URL should be published in the upcoming days/weeks. I am using Foursquare because it is the backend of a rather popular mobile navigation App for blind people, called Blindsquare. Blindsquare lets you comfortably use Open Street Map and Foursquare data to get an overview of your surroundingds. If a food place has a menu entry listed in Foursquare, Blindsquare conveniently shows it to you and opens a browser if you tap it. So there is no need to actually search for the restaurant, you can just use the location based search of Blindsquare to discover the restaurant entry and its menu link directly from within Blindsquare. Actually, you could even find a restaurant by accident, and with a little luck, find the menu for it by location, without even knowing how the restaurant is called. Isn't that neat? Yeah, that's how it is supposed to work, that's as much independence as you can get. And, it is, as the title suggests, croudsourced accessibility. Becuase while it is nice if a restaurant owner cares to publish their menu themselves, if they haven't, you can do it yourself. Either as a user of assistive technologies, to scratch your own itch. Or as a friend of a person with a need for assistive technologies. Next time you go to lunch with your blind friend, consider making available the menu to them digitally in advance, instead of reading it. Other people will likely thank you for that, and you have actually achieved something today. And if you happne to put a menu online, make sure to submit a change request to Foursquare. Many blind people are using blindsquare these days, which makes it super-easy for them to discover the menu.

13 July 2014

Russ Allbery: Review: Neptune's Brood

Review: Neptune's Brood, by Charles Stross
Series: Freyaverse #2
Publisher: Ace
Copyright: July 2013
ISBN: 1-101-62453-1
Format: Kindle
Pages: 325
Neptune's Brood is set in the same universe as Saturn's Children, but I wouldn't call it a sequel. It takes place considerably later, after substantial expansion of the robot civilization to the stars, and features entirely different characters (or, if there was overlap, I didn't notice). It also represents a significant shift in tone: while Saturn's Children is clearly a Heinlein pastiche and parody, Neptune's Brood takes its space opera more seriously. There is some situational humor assault auditors, for example but this book is played mostly straight, and I detected little or no Heinlein. This is Stross fleshing out his own space opera concept. This being Stross, that concept is not exactly conventional. This is a space opera about economics. Specifically, it's a space opera about interstellar economics, a debt pyramid, and a very interesting remapping of the continual growth requirements of capitalism to the outward expansion of colonization. The first-person protagonist comes from a "family" (as in Saturn's Children, the concept exists but involves rather more aggressive control of the instantiated "children") of bankers, but she is a forensic accountant and historian who specializes in analysis of financial scams. As you might expect, this is a significant clue about the plot. Neptune's Brood opens with Krina in search of her sister. She is supposed to be an itinerant scholar, moving throughout colonized space to spend some time with various scattered sisters, spreading knowledge and expanding her own. But it's clear from the start of the book that something else is going on, even before an assassin with Krina's face appears on her trail. Unfortunately, it takes roughly a third of the book to learn just what is happening beneath the surface, and most of that time is spent in a pointless interlude on a flying cathedral run by religious fanatics. The religion is a callback to Saturn's Children: robots who are trying to spread original humanity (the Fragiles) to the stars. This mostly doesn't go well, and is going particularly poorly for the ship that Krina works for passage on. But this is a brief gag that I thought went on much too long. The plot happens to Krina for this first section of the book rather than the other way around, little of lasting significance other than some character introductions occurs, and the Church itself, while playing a minor role in the later plot, is not worth the amount of attention that it gets. The best parts of the early book are the interludes in which Krina explains major world concepts to the reader. These are absolutely blatant infodumping, and I'm not sure how Stross gets away with them, but somehow he does, at least for me. They remind me of some of his blog posts, except tighter and fit into an interesting larger structure. Thankfully, once Krina finally arrives on Shin-Tethys, the plot improves considerably. There was a specific moment for me when the book became interesting: when Krina finds her sib's quarters in Shin-Tethys and analyzes what she finds there. It's the first significant thing in the book that she does rather than have done to her or thrust upon her, and she's a much better character when she's making decisions. This is also about the point where Stross starts fully explaining slow money, which is key to both the economics and the plot, and the plot starts to unwind its various mysteries and identify the motives of the players. Even then, Krina suffers from a lack of agency. Only at rare intervals does she get a chance to affect the story. Most of what she did of relevance to this book she did in the past, and while those descriptions of the backstory are interesting, they don't entirely make up for a passive protagonist. Thankfully, the other characters are varied and interesting enough, and the political machinations and cascading revelations captivating enough, that the last part of the book was very satisfying even with Krina on for the ride. This is a Stross novel, so it's full of two-dollar technical words mixed with technobabble. However, it shares with Saturn's Children the recasting of robots as the norm and fleshy humans as the exception, which means much of the technobabble is a straight substitution for our normal babble about meaty bodies and often works as an alienation technique. That makes it a bit more tolerable for me, although I still wished Stross would turn down the manic vocabulary in places. This bothers some people more than others; if you had no trouble with Accelerando, Neptune's Brood will pose no problems. I don't think the first section of this book was successful, but I liked the rest well enough to recommend it. If you like your space opera with a heavy dose of economics, a realistic attitude towards deep space exploration without faster-than-light technology, and a realistic perspective on the hostility of alien planets to Earth life, Neptune's Brood is a good choice. And any book that quotes David Graeber's Debt, and whose author has clearly paid attention to its contents, wins bonus points from me. Rating: 8 out of 10

8 July 2014

Steve Kemp: What do you do when your free service is too popular?

Once upon a time I setup a centralized service for spam-testing blog/forum-comments in real time, that service is BlogSpam.net. This was created because the Debian Administration site was getting hammered with bogus comments, as was my personal blog. Today the unfortunate thing happened, the virtual machine this service was running on ran out of RAM and died - The redis-store that holds all the state has now exceeded the paltry 512Mb allocated to the guest, so OOM killed it. So I'm at an impasse - I either recode it to use MySQL instead of Redis, or something similar to allow the backing store to exceed the RAM-size, or I shut the thing down. There seems to be virtually no liklihood somebody would sponsor a host to run the service, because people just don't pay for this kind of service. I've temporarily given the guest 1Gb of RAM, but that comes at a cost. I've had to shut down my "builder" host - which is used to build Debian packages via pbuilder. Offering an API, for free, which has become increasingly popular and yet equally gets almost zero feedback or "thanks" is a bit of a double-edged sword. Because it has so many users it provides a better service - but equally costs more to run in terms of time, effort, and attention. (And I just realized over the weekend that my Flattr account is full of money (~50 euro) that I can't withdraw - since I deleted my paypal account last year. Ooops.) Meh. Happy news? I'm avoiding the issue of free service indefinitely with the git-based DNS product which was covering costs and now is .. doing better. (20% off your first months bill with coupon "20PERCENT".)

27 June 2014

Russ Allbery: Review: Due Diligence

Review: Due Diligence, by David Roodman
Publisher: Center for Global Development
Copyright: 2012
ISBN: 1-933286-48-2
Format: Trade paperback
Pages: 335
Due Diligence: An Impertinent Inquiry into Microfinance is a detailed analysis of the effects, philosophy, approaches, and merits of microfinance as an institution and as, increasingly, a charitable focus of the developed world. The author, David Roodman, was a senior fellow at the Center for Global Development at the time of writing this book, and is an expert in microfinance and economic development. This book starts as an overview and a history of the spectrum of microfinance institutions (something I found very valuable, since I knew very little about microfinance prior to Kiva's popularity), and then attempts a critical, data-driven analysis of whether, to what extent, and by which measures microfinance is successful. The most common aspect of microfinance, made popular and prominent by Kiva among others, is microcredit: small loans to people in developing countries. The underlying theory of microcredit is that lack of financial services handicaps prospective entrepreneurs and business owners in the third world, preventing them from growing businesses, funding expansion, and moving out of poverty. By extending relatively small amounts of credit to these people, or so the theory goes, they can better smooth out the uncertainties of day-to-day life and get the necessary capital to expand small local business. Microcredit is not the only type of microfinance, though, and Roodman's careful discussion of that point was one of the most informative parts of this book for me. It can also come in the form of microsavings tiny bank accounts or small-scale savings plans or microinsurance. Roodman makes a very interesting argument that these three forms of microfinance form two opposing continuums. Microcredit is the easiest to provide and microinsurance is the hardest, for many reasons he discusses including some that were surprising to me, but microinsurance is arguably the most helpful and beneficial to the individual poor and microcredit is the least. The opening history and survey does an excellent job showing the range of techniques and approaches used in microfinance, and showing the challenges of providing it. I had some vague understanding of the logistical challenges in serving areas far away from traditional infrastructure. I had much less understanding of the fascinating array of techniques in use, both inside and outside what's become the microfinance industry. Some of these techniques are brilliant examples of human economic creativity and skill at balancing incentives with risk protection. There are a much wider variety of approaches and instruments available than the individual or group loans that one typically sees on a site like Kiva, all with different trade-offs in group size, flexibility, rigidity, risk, and complexity. This book is well worth reading just to get a feel for the huge array of financial arrangements possible outside of the traditional banking structures that most readers of this review are used to. Another question Roodman answers thoroughly is why microfinance is so focused on microcredit. My vague unease with that was strengthened by reading David Graeber's Debt, which among many other things makes clear some of the social perils and power imbalances in lending. Due Diligence both crystallized that discomfort and explained why alternatives are difficult. Both microsavings and microinsurance necessarily involve considerably more regulation (which has both advantages and complexity), since they involve taking and safeguarding people's money and promising service in the future. Lenders require less trust; lenders instead extend trust and assume that risk themselves, and are in a position to do so. Microinsurance is also harder to explain and harder to target, although it is potentially the most helpful financial instrument when well-targeted. And microcredit can absorb huge influxes of capital from the outside, lending it out in a region, while both microsavings and microinsurance are necessarily built on the money already in the local economy and are inherently more resistant to external involvement. (I'm of the opinion this is a significant feature, but it makes it far harder for them to catch on as a charitable trend.) The heart of the book is a hard look at whether microfinance, and particularly microcredit, works. Roodman attempts this analysis for three different definitions of working: lifting people out of poverty (the most commonly claimed success), giving people more freedom and control over their lives even if their poverty level does not change in absolute terms, and developing the local economy. Sadly for charities like Kiva, microcredit does surprisingly poorly on the first two metrics. There is still little clear data, and the degree to which data from one region is applicable to a different region is highly debatable, but what evidence we have does not support the theory that microcredit lifts people out of poverty. In fact, the more intense the microcredit, the more likely it appears to be that credit pathologies common in the developed world, such as taking out loans from one organization to pay off loans from another, spirals of debt that is unlikely to ever be repaid, and aggressive debt collection methods, turn up in microcredit as well. The vision of the entrepreneur who succeeds through new-found access to capital is largely a myth. Indeed, the current packaging and presentation of loans to distant charities in developed countries means that even the supposed purpose of the loans is dubious. Loans are almost always presented as being for some sort of business activity, since that's what external charities want to fund, but money is fungible and the funds are more often used for all the things people normally need to use funds for: weddings, funerals, food for the family in hard times, paying off other loans to relatives, and other daily expenses. On the empowerment front, microcredit fairs little better, and here I particularly appreciated Roodman's clear-eyed and honest portrayal of common microcredit practices and their benefits for the creditors. For example, microcredit makes much of loaning to women, and presents this as a story of female empowerment. Roodman argues that there is some reality behind that, but it's also clear that microcredit organizations loan to women because women are more likely to repay not due to any inherent gender trait, but because women are easier to intimidate and shame into repaying. Similarly, the much-vaunted group loans, usually presented as a way for the community to support each other and teach each other financial skills, are most common in the poorest regions because they represent an outsourcing of costs. The main risk to the lender is that poor people have no such thing as credit histories, and detailed investigations of someone's trustworthiness is expensive. Groups are jointly liable and therefore will police themselves and check each other's reliability at no cost to the lender. Groups will also use intimidation tactics and even violence against members who don't or can't repay their share, without the lender having to get involved or have any responsibility for those tactics. Roodman takes a philosophical tone on this topic, noting several times that services for the very poor are generally of low quality and finance is no exception. I'm less inclined to be philosophical; after reading this book, I find it very hard to support group loans. This leaves economic development as the most compelling argument left for microcredit. This is weak tea for people like myself who wanted to believe in the transformative power of microcredit, but Roodman does a solid job explaining why it's still important. There is a great deal to be said for the perspective that only industrialization actually lifts people out of poverty on any substantial scale, and therefore the most useful type of aid is aid that somehow contributes towards industrialization. Building a native financial industry could arguably be such a step. And there seems to be some real merit and lasting effect to having a richer and more capable set of local financial institutions. However, here, it's not at all clear that external money helps. In fact, there's quite a bit of evidence that external money hurts, since it pushes microfinance institutions to find ways of using that money and pleasing external donors instead of building their infrastructure and capabilities as financial institutions. The earlier institutions, such as the Grameen Bank, seem to have done a better job at being local institutions woven into the local economy than have the later organizations that are largely supported by western donors. Roodman makes a good case for the usefulness of microfinance as a part of economic development, but it comes with a strong argument against too much external funding, and particularly against the practice of developed world donors funding specific loans. I came away from this book feeling like I had a much better understanding of the core problems and techniques of financial services for the poor, and with a strong desire to see more microsavings than microcredit. The two are closer in nature than I had expected, and Roodman's analysis of the similarities was fascinating, but microsavings seems less vulnerable to some of the credit pathologies. There is also some statistical evidence that microsavings does provide some modest reductions in poverty, while the evidence for microcredit is much less compelling. Sadly, from the perspective of a resident of the developed world such as myself who would very much like to help, neither microsavings nor effective microcredit seems likely to benefit from my contributions, and particularly not in the form of a site like Kiva. The end of this book is a long appendix that reprints multiple blog posts on related topics, including additional discussion of the problems with the Kiva model. Effectively, Kiva is a way for people like myself to subsidize microcredit organizations. It's not at all clear that subsidies are needed, and there's some reason to believe that they encourage negative and destructive behavior. The way they are marketed involves a certain amount of lying, or at least distortion, on the part of multiple actors in the microcredit picture. And they're a distraction from microsavings, and from microcredit based on loaning out savings from within the same local economy, both of which seem to have considerably more positive properties. Due Diligence is a book with a somewhat narrow target audience. It will appeal primarily to people with an interest in economics, particularly development economics, or who are evaluating microfinance charities or analyzing their support of groups like Kiva. But within that context, this is an excellent book, and I recommend it to your attention. Roodman is clear, thorough, and fair within my ability to measure that, and while no one book can be the final word on a topic, he adds considerable data and thoughtful analysis. If you are a supporter of Kiva, like I was, I fear you'll find this book rather discouraging. But it's better to have the information than not, and I found it a good reminder of the inherent drawbacks of loans and debt. There is, sadly, nothing magical about microcredit that removes those problems. Rating: 9 out of 10

25 June 2014

Keith Packard: Altos1.4.1

AltOS 1.4.1 Fix ups for 1.4 Bdale and I are pleased to announce the release of AltOS version 1.4.1. AltOS is the core of the software for all of the Altus Metrum products. It consists of firmware for our cc1111, STM32L151, LPC11U14 and ATtiny85 based electronics and Java-based ground station software. This is a minor release of AltOS, incorporating a small handful of build and install issues. No new features have been added, and the only firmware change was to make sure that updated TeleMetrum v2.0 firmware is included in this release. AltOS TeleMetrum v2.0 firmware included AltOS version 1.4 shipped without updated firmware for TeleMetrum v2.0. There are a couple of useful new features and bug fixes in that version, so if you have a TeleMetrum v2.0 board with older firmware, you should download this release and update it. AltosUI and TeleGPS Signed Windows Drivers, faster maps downloading We finally figured out how to get our Windows drivers signed making it easier for Windows 7 and 8 users to install our software and use our devices. Also for Windows users, we ve fixed the Java version detection so that if you have Java 8 already installed, AltOS and TeleGPS won t try to download Java 7 and install that. We also fixed the Java download path so that if you have no Java installed, we ll download a working version of Java 6 instead of using an invalid Java 7 download URL. Finally, for everyone, we fixed maps downloading to use the authorized Google API key method for getting map tiles. This makes map downloading faster and more reliable. Thanks for flying with Altus Metrum!

23 June 2014

Francesca Ciceri: Adventures in Mozillaland #2

Time for an update from my internship at Mozilla, as part of the OPW. The last weeks have been a bit rough: I have my usual migraines to thank for that. It's not easy to work with them: you are either stoned by the meds, or cannot look at a monitor. And while you're trying to sleep your headaches away, the world keeps rolling. Silly world.
As Liz, my mentor, suggested I tried to stick with the little things and to do a bit of something everyday.
"Waiting for the miracle", you know.
So, here's what I've been up to: Triaging I probably failed my personal 5-bugs-a-day policy, in the last two weeks, but beside that I'm pretty satisfied of my progress: All in all, I realized that I really love triaging/verifying bugs: being not a developer nor a simple user, but a bit of both, gives me the right mindset - I think - to mediate between these two worlds. Community On the community front, I finally managed to meet - virtually - the Mozilla Italian community. The guys are great and beside running the forum to give user support, they do a whole lot of activities related to localization. They are also trying to encourage participation of Italian users and enthusiasts to Mozilla events: don't miss, for instance, the upcoming Marketplace Day when a couple of us will be available to help other Italian users with the day's activities. Read Daniele's post on the forum for more information in Italian. Documentation On the documentation front, I finally managed to get out the Bugdays FAQ, draft a guide on how to run different versions of Firefox and multiple profiles for triaging purposes on Linux and Windows - still have to finish this one, though -, and participate to a very interesting discussion on the current state of QMO - the entry point for QA contributors in Mozilla.
The site, in my opinion, needs some love and I'd very much like to help in that sense.
Check out the discussion, and give us some feedback about the website on the dev-quality mailing list! Lessons learned

22 June 2014

Asheesh Laroia: Interactive semi-automated package review (by abusing Travis-CI)

I just did some Debian package review in a somewhat unusual way, and I wanted to share that. I'm hoping other Debian developers (and other free software contributors) that need to review others' contributions can learn something from this, and that I can use this blog post as a way to find out if other people are doing something similar. It was pretty exciting! At the end of it, I joined #debian-mentors to talk about how my cool process. Someone summarized it very accurately:
<sney> it almost sounds like you're working to replace yourself with automation

Context about alpine in Debian (Skip to "Package review, with automation" if you're familiar with Debian.) I'm the maintainer of alpine in Debian. There are quite a few problems with the alpine package in Debian right now, the biggest of which are:
  • We're one version behind -- 2.11 is the latest available, but 2.10 is the newest that we have in Debian.
  • The packaging uses a decreasingly-popular packaging helper, cdbs, about which I happen to know less than the dh-style helper (aka dh7).
  • There are lots of bugs filed, and I don't respond in a timely fashion.
This doesn't change my deep love for alpine -- I've had that for about half my life now, and so far, I don't see it going away. A month or so ago, I got a friendly private message from Unit193, saying he had converted the package to the dh style, and also packaged the newer version. They wanted to know if they should clean this up into something high-enough quality to land in Debian. (In Debian, we have a common situation where enthusiastic users update or create a new package, and aren't yet Debian Developers, so they don't have permission to upload that directly to the "Debian archive", which is the Debian equivalent of git master. Package "sponsorship" is how we handle that -- a Debian Developer reviews the package, makes sure it is of high quality, and uploads it to the Debian archive along with the Debian Developer's OpenPGP signature, so the archive processing tools know to trust it.) On Friday evening, I had a spare moment, so I sent a private message to Unit193 apologizing for not getting back to them in a reasonable amount of time. Having another person help maintain is a pretty exciting prospect, and I wanted to treat that enthusiasm with the respect it deserves, or at least apologize when I haven't. I was surprised to see a reply within a few minutes. At that point, I thought: I wasn't planning on doing any package review this weekend, but if they're online and I'm online... might as well!

Package review, with automation Unit193 and I popped into ##alpine on irc.freenode.net, and I started reading through their packaging changes, asking questions. As I asked questions, I wondered -- how will I know if they are going to fix the issues I'm raising? Luckily, Unit193 wanted to use git to track the packaging, and we settled on using git-buildpackage, a tool that was fairly new to both of us. I thought, I might as well have some executable documentation so I don't forget how to use it. ("Executable documentation" is Asheesh-speak for a shell script.) One thing I knew was that I'd have to test the package in a pbuilder, or other pristine build environment. But all I had on me at the moment was my work laptop, which didn't have one set up. Then I had a bright idea: I could use Travis-CI, a public continuous integration service, to check Unit193's packaging. If I had any concerns, I could add them to the shell script and then point at the build log and say, "This needs to be fixed." Then there would be great clarity about the problems. Some wonderful things about Travis-CI:
  • They give you root access on an Ubuntu Precise (10.04) virtual machine.
  • Their build hosts are well-connected to the Internet, which means fast downloads in e.g. pbuilder.
  • They will let you run a build for up to 50 minutes, for free.
  • Build just means "command" or "set of commands," so you can just write a shell script and they will run it.
  • Travis-CI will watch a github.com repository, if you like. This means you can 'git commit --allow-empty' then 'git push' and ask it to re-run your script.
Since Unit193's packaging was in git (but not on github), I created a git repo containing the same contents, where I would experiment with fixes for packaging problems I found. It'd be up to Unit193 to fix the problems in the Alioth packaging. This way, I would be providing advice, and Unit193 would have an opportunity to ask questions, so it would be more like mentorship and less like me fixing things. We did a few rounds of feedback this way, and got the packaging to higher and higher quality. Every time Unit193 made a fix and pushed it, I would re-run the auto-build, and see if the problems I spotted had gone away. While the auto-build runs, I can focus on conversing with my mentee about problems or just generally chatting. Chatting is valuable community-building! It's extremely nice that I can do that while waiting on the build, knowing that I don't have to read it carefully -- I can just wait a few minutes, then see if it's done, and see if it's red or green. Having the mentee around while I'm reviewing it means that I can use the time I'm waiting on builds as fun free software social time. (Contrast this with asynchronous review, where, all alone, I would wait for a build to finish, then write up an email at the end of it all.) This kind of mentorship + chatting was spread out over Friday night, Saturday night, and Sunday morning. By the end of it, we had a superb package that I'm excited to sign and push into Debian when I'm next near my OpenPGP key.

Implementation details You can see the final shell script here: And you can see the various builds here: The shell script:
  • Alternates between the Alioth packaging vs. my fork of it. (This way, I can test packaging changes/suggestions.)
  • Disables ccache in pbuilder, due to a permissions problem with ccache/pbuilder/travis-ci, and I didn't need ccache anyway.
  • Handles 'git dch' slightly wrong. I need to figure that out.
  • Optionally passes --git-ignore-new to git-buildpackage, which was required initially, but should not be required by the time the package is ready. (This is an example of a thing I might forget to remark upon to my mentee.)
  • Plays games with git branches so that git-buildpackage on Travis-CI can find the pristine-tar branch.
  • Tries to use cdn.debian.net as its mirror, but on Saturday ran into problems with whicever mirror that is, so it falls back to mirror.mit.edu in case that fails.
  • Contains a GPG homedir, and imports the Debian archive key, so that it can get past Ubuntu-Debian pbuilder trust issues.
I also had a local shell script that would run, effectively:
  • git commit --allow-empty -m 'Trigger build'
  • git push
This was needed since I was basically using Travis-CI as remote shell service -- moreover, the scripts Travis-CI runs are in a different repo (travis-debcheck) than the software I'm actually testing (collab-maint/alpine.git). Unit193 and I had a technical disagreement at one point, and I realized that rather than discuss it, I could just ask Travis-CI to test which one of us was right. At one point in the revisions, the binary package build failed to build on Precise Pangolin (the Ubuntu release that the Travis-CI worker is running), and Unit193 said that it was probably due to a problem with building on Ubuntu. So I added a configuration option to build just the source package in Ubuntu, keeping the binary package test-build within the Debian sid pbuilder, although I believed that there was actually a problem with the packaging. This way, I could modify the script so that I could demonstrate the problem could be reproduced in a sid pbuilder. Of course, by the time I got that far, Unit193 had figured out that it was indeed a packaging bug. I also created an option to SKIP_PBUILDER; initially, I wanted to get quick automated feedback on the quality of the source package without waiting for pbuilder to create the chroot and for the test build to happen. You might notice that the script is not very secure -- Niels Thykier already did! That's fine by me; it's only Travis-CI's machines that could be worsened by that insecurity, and really, they already gave me a root shell with no password. (This might sound dismissive of Travis-CI -- I don't mean it to be! I just mean that their security model already presumably involves throwing away the environment in which my code is executing, and I enjoy taking advantage of that.) Since the Travis virtual machine is Ubuntu, and we want to run the latest version of lintian (a Debian packaging "lint" checker), we run lintian within the Debian sid pbuilder. To do that, I use the glorious "B90lintian" sample pbuilder hook script, which comes bundled with pbuilder in /usr/share/doc/pbuilder/. The full build, which includes creating a sid pbuilder from scratch, takes merely 7-10 minutes. I personally find this kind of shockingly speedy -- in 2005, when I first got involved, doing a pbuilder build seemed like it would take forever. Now, a random free shell service on the Internet will create a pbuilder, and do a test build within it, in about 5 minutes.

Package review, without automation I've done package review for other mentees in the past. I tend to do it in a very bursty fashion -- one weekend day or one weeknight I decide sure, it's a good day to read Debian packages and provide feedback. Usually we do it asynchronously on the following protocol:
  1. I dig up an email from someone who needed review.
  2. I read through the packaging files, doing a variety of checks as they occur to me.
  3. If I find problems, I write an email about them to the mentee. If not, success! I sign and upload the package.
There are some problems with the above:
  • The burstiness means that if someone fixes the issues, I might not have time to re-review for another month or longer.
  • The absence of an exhaustive list of things to look for means that I could fail to provide that feedback in the first round of review, leading to a longer wait time.
  • The person receiving the email might not understand my comments, which could interact really badly with the burstiness.
I did this for Simon Fondrie-Teitler's python-pypump package recently. We followed the above protocol. I wrote a long email to Simon, where I remarked on various good and bad points of the packaging. It was part commentary, part terminal transcript -- I use the terminal transcripts to explain what I mean. This is part of the email I sent:
I got an error in the man page generation phase -- because at 
build-time, I don't have requests-oauthlib:
make[2]: Leaving directory  /tmp/python-pypump-0.5-1+dfsg/docs'
help2man --no-info \
	-n 'sets up an environment and oauth tokens and allows for interactive testing' \
        --version-string=0.5.1 /tmp/python-pypump-0.5-1+dfsg/pypump-shell > /tmp/python-pypump-0.5-1+dfsg/debian/pypump-shell.1
help2man: can't get  --help' info from /tmp/python-pypump-0.5-1+dfsg/pypump-shell
Try  --no-discard-stderr' if option outputs to stderr
make[1]: *** [override_dh_auto_build] Error 1
This seems to be because:
   python-pypump-0.5-1+dfsg  ./pypump-shell 
Traceback (most recent call last):
  File "./pypump-shell", line 26, in <module>
    from pypump import PyPump, Client
  File "/tmp/python-pypump-0.5-1+dfsg/pypump/__init__.py", line 19, in <module>
    from pypump.pypump import PyPump, WebPump
  File "/tmp/python-pypump-0.5-1+dfsg/pypump/pypump.py", line 28, in <module>
    from six.moves.urllib import parse
ImportError: No module named urllib
$ ./pypump-shell 
Traceback (most recent call last):
  File "./pypump-shell", line 26, in <module>
    from pypump import PyPump, Client
  File "/tmp/python-pypump-0.5-1+dfsg/pypump/__init__.py", line 19, in <module>
    from pypump.pypump import PyPump, WebPump
  File "/tmp/python-pypump-0.5-1+dfsg/pypump/pypump.py", line 29, in <module>
    from requests_oauthlib import OAuth1
ImportError: No module named requests_oauthlib
The deeper problem was a missing build-dependency, and I explained that in my email. But the meta problem is that Simon didn't try building this in a pbuilder, or otherwise clean build environment. Simon fixed these problems, and submitted a fresh package to Paul Tagliamonte and myself. It happened to have some typos in the names of the new build dependencies. Paul reviewed the fixed package, noticed the typos, fixed them, and uploaded it. Simon had forgotten to do a test build the second time, too, which is an understandable human failure. There was a two-day delay between Simon's fixed resubmission, and Paul signing+uploading the fixed result. In a pedagogical sense, there's something disappointing about that exchange for me: Paul fixed an error Simon introduced, so we're not teaching Simon to take total responsibility for his packages in Debian, nor to understand the Debian system as well as he could. (Luckily, I think Simon already understands the importance of taking responsibility! In this case, it's just a hypothetical in this case.)

For the future The next time I review a package, I'm going to try to do something similar to my Travis-CI hack. It would be nice to have the do.sh script be a little more abstract; I imagine that as I try to use it for a different package, I'll discover the right abstractions. I'd love it if Travis-CI did not require the git repositories to be on GitHub. I'd also like if the .travis.yml file could be in a different path. If so, we could create debian/travis-configuration (or something) and keep the packaging files nice and separate from the upstream source. I'd also love to hear about other people's feedback. Are you doing something similar? Do you want to be? Would you have done some of this differently? Leave a comment here, or ping me (paulproteus) on #debian-mentors on irc.debian.org (aka irc.oftc.net). I'll leave you with some conversation from #debian-mentors:
<paulproteus> The automation here, I think, is really interesting.
<paulproteus> What I really want is for mentees to show up to me and say "Here is my package + build log with pbuilder, can you sponsor it please?"
<Unit193> Oooooh!
-*- Unit193 gets ideas.
<paulproteus> Although the irony is that I actually like the community-building and relationship-building nature of having these things be conversations.
<bremner> how will this brave new world cope with licensing issues?
<paulproteus> bremner: It's not a replacement for actual review, just a tool-assist.
<paulproteus> bremner: You might be relieved to know that much of Unit193's and my back and forth related to get-orig-source and licensing. (-:
<bremner> I didn't doubt you ;).
<paulproteus> If necessary I can just be a highly productive reviewer, but I would prefer to figure out some way that I can get other non-paulproteus people to get a similar assist.
<paulproteus> I think the current blocker is "omg travis why are you bound to githubbbbbbbb" which is a reasonable concern.

20 June 2014

Ian Donnelly: Week 4: New Release Work

Hi All, Team Elektra recently released version 0.8.6 to download, more information on that here: http://sourceforge.net/p/registry/mailman/message/32418614/ However, since I am working with Debian, and the Debian package isn t ready yet I have spent a large chunk of the week working on that. Namely I fixed a few minor lintian errors and worked on packaging the lua and python bindings for Elektra. These two packages, libelektra-python4 and libelektra-lua4 uses a tool called SWIG. SWIG allows C and C++ programs to connect to programs written in higher-level languages such as Python. So these two packages use the power of SWIG to bring the features of the Elektra API (which is written in C) to Python and Lua developers. I personally think it is a really neat feature and allows Elektra to be used more widely than ever before. As our project develops Elektra will support more languages and we are even moving to allowing Developers to write Elektra plugins using other languages. Additionally, I worked on improving the MergeTools library for Elektra. This library just contains the necessary functions to allow developers to complete a three-way merge of KeySets. This library also serves as the backbone of the kdb merge command. First, the KeySetMerge command now returns an empty KeySet if its unsuccessful. I added operator overloads for == and != to the KeySet.hpp class so developers can tell if the returned keyset is empty. I created a new function that takes in 3 KeySets, their parent keys (for the purposes of merging), and a merge_root key which will be where the merged keys are stored under. The prototype for this function is KeySet(KeySet base, Key base_root, KeySet ours, Key our_root, KeySet theirs, Key their_root, Key merge_root). A function still exists that just takes in 3 KeySets and a key, merge_root. That function attempts to find each root key and then calls the previously mentioned function. Why? This week I finalized the kdb merge command (not including options). This command takes in 4 root keys, one for base, one for ours, one for theirs, and one for where the merged keys should be stored. The merge command now uses the new function to be able to handle more complex KeySets and so far it works great! In this case we don t need to make any assumptions about the KeySet since the user needs to pass in the root keys (this is how the kdb commands typically work) so we make good use of this information. Now the kdb merge command only appends the merge KeySet to the database upon successful merge, otherwise it tells you what went wrong. Also, looking into the main feature of my GSoC project, automatic conffile merging, it seems that it will require patching dpkg. Originally I though package maintainers would have to edit their packages to not define their conffiles as such in order to use the automatic merge. However this is obviously a messy workaround. Patching dpkg however would allow very clean integration of this feature. If this project is successful and the patch makes its way upstream, all Debian packages that utilize conffiles (most of them) will basically allow automatic merging practically overnight. dpkg would attempt a merge on the conffiles during an upgrade and if unsuccessful would just prompt the user like it does now. In most situations however, because of the features of Elektra, conffiles should merge fine. This would make Debian much more user-friendly and could save users a lot of time manually merging conffiles. Oh, and look for a little tutorial and explanation of the kdb mount command in the next few days. I will post it here. Until Then,
Ian S. Donnelly

17 June 2014

Tiago Bortoletto Vaz: Little garden project

After three years living in Montr al we're still amazed by the fact that here we have er... seasons!! I believe that things have a special beauty when they don't last the time we want them to last, and seasons are all about it! (some people will exclude winter here) Anyways, taking pictures may give them more lifetime without compromising too much their beauty. And sharing them here and there is somewhat sharing our happiness, which might be a good balance in the end :)
/images/01.JPG

At the time we moved the garden was just a single white thing...

/images/02.JPG

then we realized that it provided some chairs, a table and a compost container, yay!

/images/03.JPG

But the grey ground was something to be fixed...

/images/04.JPG

so we hacked it and got some land :)

/images/05.JPG

Few months later and it's finally becoming chard, beets, carrots, spinach, berries, cheries, okra, tomatoes, basil, potatoes, sage, agastache and a saskatoon berry tree!

Steve Kemp: DNS is now resolved

I used to work for Bytemark, being a sysadmin and sometimes handling support requests from end-users, along with their clients. One thing that never got old was marking DNS-related tickets as "resolved", or managing to slip that word into replies. Similarly being married to a Finnish woman you'd be amazed how often Finnish and Finished become interchangable. Anyway that's enough pun-discussion. Over the past few days I've, obviously, been playing with DNS. There are two public results:
DHCP.io
This is my simple Dynamic-DNS host, which has now picked up a few users. I posted a token on previous entry, and I've had fun seeing how people keep changing the IP address of the host skx.dhcp.io.. I should revoke the token and actually claim the name - but to be honest it is more fun seeing it update. What is most interesting is that I can see it being used for real - I see from the access logs some people have actually scheduled curl to run on an hourly basis. Neat.
DNS-API.org
This is a simple lookup utility, allowing queries to be made, such as: Of the two sites this is perhaps the most useful, but again I expect it isn't unique.
That about wraps things up for the moment. It may well be the case that in the future there is some Git + DNS + Amazon integration for DNS-hosting, but I'm going to leave it alone for the moment. Despite writing about DNS several times in the past the only reason this flurry of activity arose is that I'm hacking some Amazon & CPanel integration at the moment - and I wanted to experiment with Amazon's API some more. So, we'll mark this activity as resolved, and I shall go make some coffee now this entry is Finnish. ObRandomUpdate: At least there was a productive side-effect here - I created/uploaded to CPAN CGI::Application::Plugin::Throttle.

16 June 2014

Simon Kainz: on using crippled installers

Maybe someone else is using iDataPlex HW, so this might come in helpful... TL, DR: Dear Hardware Vendors! Please don't force people to use specific Linux distributions when a simple unzip command would do the same. At work we run (amongst others) an HPC system consisting of 120 IBM System X iDataPlex dx360 M4 Nodes. Recently, after some MCE logs caused by DIMM errors, the IBM guy refused to send me replacement DIMMs if if didn't update IMM firware and tortue test the machine some days more. Well, ok... IBM uses the so called UpdateXpress tool to update IMM and UEFI firmware. The tool is available for rhel4 to rhel7 (both 32 and 64 bit), SLES10 & SLES11 and Microsoft Windows. I tried all of the available UpdateXpress System Pack Installers, with basically the same result everytime:
WARNING! This package doesn't appear to match your system.
         The following information was determined for your system:
           distribution = Unknown
           release = -1
           processor architecture = Unknown
I tried to set up a fake RHEL environment (customized /usr/local/uname, custom /et/RHel-release, etc...) but after some more tries, the installer (yes, it's still the installer I am trying to run) fails with some strange SEGFAULTS. After some closer inspection, (well, the RHEL4 uxspi binary is about 109MB big!) i tried to unzip it, and voila:
~/firmware$ unzip ibm_utl_uxspi_9.60_rhel4_32-64.bin
Archive:  ibm_utl_uxspi_9.60_rhel4_32-64.bin
   creating: rhel6_64/image/
  inflating: rhel6_64/image/libpegasus.nsp.so  
  inflating: rhel6_64/image/libUxliteImmUsbLan.so  
  inflating: rhel6_64/image/eccConnect.properties  
  inflating: rhel6_64/image/libpegslp_client.so  
   creating: rhel6_64/image/esxi_assoc_templates/
  inflating: rhel6_64/image/esxi_assoc_templates/elxFC.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/elxCNA.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/ibmc.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/FPGA-S.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/degraded.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/brocadeFC.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/qlgcFC.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/broadcom.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/LSI.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/brocadeCNA.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/uefi.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/diags.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/qlgcCNA.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/VMwareESXi.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/ibmc2.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/FPGA.uxt  
  inflating: rhel6_64/image/libipmi_client.so  
  inflating: rhel6_64/image/UXLite_UPDATEID.dat  
  inflating: rhel6_64/image/libacpi.so  
  inflating: rhel6_64/image/libibmsp6_openipmi.so  
  ...
I managed to find a iflash64 binary, and after so trial and error, came up with the following workflow:
./iflash64 --host your_host_imm_IP --user your_username --password your_password --package imm2_1aoo56d-3.73.upd --reboot
IBM Command Line IMM Flash Update Utility v2.02.11
Licensed Materials - Property of IBM
(C) Copyright IBM Corp. 2009 - 2014  All Rights Reserved.
Connected to IMM at IP address x.x.x.x.
Update package firmware type: IMM2
Update package build level:   xxxxxx
Target's current build level: xxxxxx
Would you like to continue with the update? y/n: y
Node 0: The IMM is preparing to receive the update.
Node 0: Transferring image: 98%
Node 0: Transfer complete.
Node 0: Validating image.
Node 0: Updating firmware:  0%
Node 0: Updating firmware:  100%
Node 0: Update complete.
Performing activation of the firmware:
Connected to IMM at IP address 10.8.2.4 successfully.
..........
Waiting up to 360 seconds for the activation to complete:
..
..
And so on... Conclusio: Dear Hardware Vendors! Please don't force people to use specific Linux distributions when a simple unzip command would do the same.

9 June 2014

Konstantinos Margaritis: SIMD book, second update!

From the Indiegogo page: https://www.indiegogo.com/projects/simd-engines-comparative-reference-guide/x/4966960#activity
  • Added titlepage (simple, but will do the job)
  • Reorganized ALL instructions to include both unsigned/signed in the same entity
  • Added Saturated Addition, Subtraction and Saturated Subtraction
  • Added ARMv8 NEON instructions taken from ARM infocenter draft
  • Fixed some instructions (added 64-bit arithmetic for NEON)
  • Added some special addition/subtraction, like add/sub with carry(vmx/vsx), addsub(SSE3/AVX)
  • Added some in-vector sum additions, sum reductions but no descriptions yet
  • Added diagram for 8-bit addition/subtraction (still need lots more).
  • Removed VMX128, couldn't find enough information, an email to some IBM toolchain developers was left unanswered, so I guess noone really will really care if that engine is left out, if enough people insist on it, please also be kind enough to provide some documentation on it.

7 June 2014

Tim Retout: Day of Action

Today I attended the Don't Spy On Us campaign's Day Of Action at Shoreditch Town Hall in London. I'm not sure how much actual action there was, but the talking was interesting. Retrospective Day of Action drinking game: drink every time you hear the phrase "If you have nothing to hide, you have nothing to fear." The spooks have a really good marketing department. I don't write a lot on the internet any more - something I regret, actually. It can't even be because there are alternative places I am writing, because over the last couple of years I have been closing most of my social media accounts. I just share much less of myself online. On the internet, nothing is ephemeral. Bruce Schneier says so. Choose your words carefully. The thing about blogging is that it's so public. It's often tied to the writer's real-life identity. One of the valuable things about social media services is that they supposedly let you restrict who can read your words - the trade-off being that you must also grant access to advertisers, spies, cybercriminals... Most memorable moments of the day: In general, there were lots of famous people walking around as if they were normal. I was in the same room as Cory Doctorow, Jimmy Wales and Bruce Schneier at the same time. Ahem, action (mostly for UK citizens):

5 June 2014

Johannes Schauer: botch updates

My last update about ongoing development of botch, the bootstrap/build ordering tool chain, was four months ago and about several incremental updates. This post will be of similar nature. The most interesting news is probably the additional data that bootstrap.debian.net now provides. This is listed in the next section. All subsequent sections then list the changes under the hood that made the additions to bootstrap.debian.net possible.

bootstrap.debian.net The bootstrap.debian.net service used to have botch as a git submodule but now runs botch from its Debian package. This at least proves that the botch Debian package is mature enough to do useful stuff with it. In addition to the bootstrapping results by architecture, bootstrap.debian.net now also hosts the following additional services: Further improvements concern how dependency cycles are now presented in the html overviews. While before, vertices in a cycle where separated by commas as if they were simple package lists, vertices are now connected by unicode arrows. Dashed arrows indicate build dependencies while solid arrows indicate builds-from relationships. For what it's worth, installation set vertices now contain their installation set in their title attribute.

Debian package Botch has long depended on features of an unreleased version of dose3 which in turn depended on an unrelease version of libcudf. Both projects have recently made new releases so that I was now able to drop the dose3 git submodule and rely on the host system's dose3 version instead. This also made it possible to create a Debian package of botch which currently sits at Debian mentors. Writing the package also finally made me create a usable install target in the Makefile as well as adding stubs for the manpages of the 44 applications that botch currently ships. The actual content of these manpages still has to be written. The only documentation botch currently ships in the botch-doc package is an offline version of the wiki on gitorious. The new page ExamplesGraphs even includes pictures.

Cross By default, botch analyzes the native bootstrapping phase. That is, assume that the initial set of Essential:yes and build-essential packages magically exists and find out how to bootstrap the rest from there through native compilation. But part of the bootstrapping problem is also to create the set of Essential:yes and build-essential packages from nothing via cross compilation. Botch is unable to analyze the cross phase because too many packages cannot satisfy their crossbuild dependencies due to multiarch conflicts. This problem is only about the dependency metadata and not about whether a given source package actually crosscompiles fine in practice. Helmut Grohne has done great work with rebootstrap which is regularly run by jenkins.debian.net. He convinced me that we need an overview of what packages are blocking the analysis of the cross case and that it was useful to have a crossbuild order even if that was a fake order just to have a rough overview of the current situation in Debian Sid. I wrote a couple of scripts which would run dose-builddebcheck on a repository, analyze which packages fail to satisfy their crossbuild dependencies and why, fix those cases by adjusting package metadata accordingly and repeat until all relevant source packages satisfy their crossbuild dependencies. The result of this can then be used to identify the packages that need to be modified as well as to generate a crossbuild order. The fixes to the metadata are done in an automatic fashion and do not necessarily reflect the real fix that would solve the problem. Nevertheless, I ended up agreeing that it is better to have a slightly wrong overview than no overview at all.

Minimizing the dependency graph size Installation sets in the dependency graph are calculated independent from each other. If two binary packages provide A, then dependencies on A in different installation sets might choose different binary packages as providers of A. The same holds for disjunctive dependencies. If a package depends on A C and another package depends on C A then there is no coordination to choose C so to minimize the overall amount of vertices in the graph. I implemented two methods to minimize the impact of cases where the dependency solver has multiple options to satisfy a dependency through Provides and dependency disjunctions. The first method is inspired by Helmut Grohne. An algorithm goes through all disjunctive binary dependencies and removes all virtual packages, leaving only real packages. Of the remaining real packages, the first one is selected. For build dependencies, the algorithm drops all but the first package in every disjunction. This is also what sbuild does. Unfortunately this solution produces an unsatisfiable dependency situation in most cases. This is because oftentimes it is necessary to select the virtual disjunctive dependency because of a conflict relationship introduced by another package. The second method involves aspcud, a cudf solver which can optimize a solution by a criteria. This solution is based on an idea by Pietro Abate who implemented the basis for this idea back in 2012. In contrast to a usual cudf problem, binary packages now also depend on the source packages they build from. If we now ask aspcud to find an installation set for one of the base source packages (I chose src:build-essential) then it will return an installation set that includes source packages. As an optimization criteria the number of source packages in the installation set is minimized. This solution would be flawless if there were no conflicts between binary packages. Due to conflicts not all binary packages that must be coinstallable for this strategy to work can be coinstalled. The quick and dirty solution is to remove all conflicts before passing the cudf universe to aspcud. But this also means that the solution does sometimes not work in practice.

Test cases Botch now finally has a test target in its Makefile. The test target tests two code paths of the native.sh script and the cross.sh script. Running these two scripts covers testing most parts of botch. Given that I did lots of refactoring in the past weeks, the test cases greatly helped to assure that I didnt break anything in the process. I also added autopkgtests to the Debian packaging which test the same things as the test target but naturally run the installed version of botch instead. The autopkgtests were a great help in weeding out some lasts bugs which made botch depend on being executed from its source directory.

Python 3 Reading the suggestions in the Debian python policy I evaluated the possibility to use Python 3 for the Python scripts in botch. While I was at it I added transparent decompression for gzip, bz2 and xz based on the file magic, replaced python-apt with python-debian because of bug#748922 and added argparse argument parsing to all scripts. Unfortunately I had to find out that Python 3 support does not yet seem to be possible for botch for the following reasons:
  • no soap module for Python 3 in Debian (needed for bts access)
  • hash randomization is turned on by default in Python 3 and therefore the graph output of networkx is not deterministic anymore (bug#749710)
Thus I settled for changing the code such that it would be compatible with Python 2 as well as with Python 3. Because of the changed string handling and sys.stdout properties in Python 3 this proved to be tricky. On the other hand this showed me bugs in my code where I was wrongly relying on deterministic dictionary key traversal.

2 June 2014

Rapha&#235;l Hertzog: My Free Software Activities since January 2014

If you follow my blog closely, you noticed that I skipped all my usual monthly summaries in 2014. It s not that I stopped doing free software work, instead I was just too busy to be able to report about what I did. As an excuse, let me tell you that we just moved into a new house which was in construction since may last year. The lack of visible activity on my blog resulted in a steady decrease of the amount of donations received (January: 70.72 , February: 71.75 , March: 51.25 , April: 39.9 , May: 40.33 ). Special thanks to all the people who kept supporting my work even though I stopped reporting about it. So let s fix this. This report will be a bit less detailed since it covers the whole period since the start of the year. Debian France Preparations related to general assemblies. The year started with lots of work related to Debian France. First I took care of setting up limesurvey with Alexandre Delano to handle the vote to pick our new logo:
The new logo of Debian France I also helped Sylvestre Ledru to finalize and close the accounting books for 2013 in preparation for the general assembly that was due later in the month. I wrote the moral report of the president to be presented to the assembly. And last step, I collected vote mandates to ensure that we were going to meet the quorum for the extraordinary assembly that was planned just after the usual yearly assembly. The assemblies took place during a two days mini-debconf in Paris (January 17-18) where I was obviously present even though I gave no talk besides announcing the logo contest winner and thanking people for their participation.
Assembl e g n rale 2014 de Debian France

The Debian France members during the general assembly

It s worth noting that the extraordinary assembly was meant primarily to enshrine in our bylaws the possibility to act as a trusted organization for Debian. This status should be officialized by the Debian project leader (Lucas Nussbaum) in the upcoming weeks since we answered satisfactorily to all questions. Our paypal donation form and the accounting tools behind it are ready. Galette packaging and members map. I managed to hand over the package maintenance of galette to Fran ois-R gis Vuillemin. I sponsored all his uploads and we packaged a new plugin that allows to create a map with all the members who accept to share their location. The idea was to let people meet each other when they don t live far away with the long term goal to have Debian France organized activities not only in Paris but everywhere in France. New contributor game. Last but not least, I organized a game to encourage people to do their first contribution to Debian by offering them a copy of my book if they managed to complete a small Debian project. We got many interesting projects but the result so far seem to be very mixed. Many people did not complete their project (yet) that said for the few that did substantial work, it was rather good and they seem to be interested to continue to contribute. Debian France booth at Solutions Linux in Paris. Like each year, I spent two days in Paris to help man the Debian France booth at Solutions Linux. We had lots of goodies on sale and we made more than 2000 EUR in earnings during the two days. I also used this opportunity to try to convince companies to support the new Debian LTS effort.
Debian France booth at Solutions Linux

Tanguy Ortolo and Fernando Lagrange behind the Debian France booth

The Debian Administrator s Handbook In the last days of 2013, we released the wheezy update of the book. Then I quickly organized everything needed so that the various translation teams can now focus their efforts on the latest release of the book. Later (in February) I announced the availability of the French and Spanish translations. Debian Squeeze LTS When the security team called for help to try to put in place long term support for Squeeze, I replied positively because I m convinced that it s very important if Debian wants to stay an acceptable choice in big deployments and because I knew that some of my customers would be interested Thus I followed all the discussions (on a semi-private list first and then on debian-lts@lists.debian.org) and contributed my own experience. I have also taken up the responsibility to coordinate with the Debian contributors who can be hired to work on Squeeze LTS so that we have a clear common offer for all the companies who have offered financial support towards Squeeze LTS. Expect further news on this front in the upcoming days/weeks. Tryton I have been a long time user of SQL-Ledger to manage the accounting of my company Freexian. But while the license is free software, the project is not. It s the work of a single developer who doesn t really accept help. I have thus been considering to move to something else for a long time but never did anything. This year, after some rough evaluation, I decided to switch to Tryton for my company. It s probably not a wise choice from a business perspective because that migration took me many hours of unpaid labor but from a free software perspective it s definitely better than everything else I saw. I contributed a lot of bug reports and a few patches already (#3596, #3631, #3633, #3665, #3667, #3694, #3695, #3696, #3697) mainly about problems found in the French chart of accounts but also about missing features for my use case. I also accepted to sponsor Matthias Berhle, who is maintaining the official Debian packages of Tryton. He s already a Debian maintainer so it s mainly a matter of reviewing new source packages and granting him the required rights. Misc Debian work Thanks See you next month for a new summary of my activities.

2 comments Liked this article? Click here. My blog is Flattr-enabled.

Russ Allbery: Review: Debt

Review: Debt, by David Graeber
Publisher: Melville House
Copyright: 2011, 2012
Printing: October 2012
ISBN: 1-61219-129-0
Format: Trade paperback
Pages: 453
You probably remember this story from introductory economics. Originally, people exchanged things through barter. I needed grain but had a cow. You had a farm but needed milk. I gave you milk for your grain. But this was tedious and awkward, and required finding the right combination of people who needed something the other person had on hand. Then, people invented money: small objects with an agreed-upon value that you could keep and use later when you needed something else. In other words, both a medium of exchange and a store of value. And that improved matters for a long time, although war-torn societies or societies that collapsed (the supposed "Dark Ages" often raises its head) would sometimes "revert to barter" for some period of time until things were stable enough for currency to reappear. Following that story was usually some story about the emergence of money based on credit: goldsmith shops that started a side business in storing other people's coins and giving them receipts, leading to the receipts trading like the coins since the shops were trustworthy, and from there into the Bank of England and eventually the fractional reserve banking system, fiat money, and all the modern machinery of finance. All of this was dated from the early Enlightenment and the development of modern finance in Europe. It's a very neat story, and it makes a great deal of intuitive sense. That barter system does feel like what you'd have to do without money, but one can immediately see how obnoxious and limiting it would be. These stories play into our intuitive sense of history: people started with emergent properties of the physical world (different people have different goods and different skills and want to exchange them) and then develop layers of abstraction on top of them, eventually leading to the sophistication of modern societies. However, David Graeber is not an economist. He's an anthropologist: the profession devoted to understanding how people actually do things rather than how we've reconstructed our history based on our modern perspective. And, as he points out, engagingly and comprehensively, in chapter two of Debt, there is no evidence that this story about economic history has anything whatsoever to do with reality. And quite a bit of evidence that it does not. As best as we've been able to determine, not only is there no society in the world that deals with routine, day-to-day needs like grain and milk through barter, there never has been such a society in human history. Instead, history and anthropology shows that credit is, in a sense, older than currency: the earliest recorded economic transactions were built on a rich system of credit, but in the form of purchases from shopkeepers on credit, or credit clearinghouses through the local government or temple. Those credit records were often denominated in some standardized commodity so many cattle or bushels of grain that could create the impression that the economy worked on barter. But the anthropological evidence indicates that this was more an accounting technique than a practical currency. People rarely brought the named commodity to the temple to pay off some debt. Rather, there was an agreed conversion from various other goods to that standardized commodity, and its primary purpose was consistent bookkeeping. Specie minted coins appear to come later, and wax and wane throughout history depending on local circumstance. For example, Europe did not "revert to barter" after the collapse of the Roman Empire, but the inhabitants did stop using specie and returned to a system of credit and record-keeping... records that continued to be denominated in Roman currency, even though few people still used the actual coins. That's one fascinating observation with which Graeber begins this book. The other is the question of why we have such a strong social and moral belief that people must pay their debts. This moral belief is ever-present in discussions about the 2008 financial collapse, and more broadly in discussions about the modern economy, but it's not as obvious of a belief as it might appear on the surface. After all, the banking and investment system is founded on the principle that not everyone will repay their debts, and therefore lenders receive a risk premium based on the likelihood that the debt won't be repaid. But, despite building the possibility of non-repayment into the system, debt forgiveness or intentional default is almost unthinkable and considered a huge moral problem. To this, Graeber brings the perspective of historical anthropology: human societies have struggled with the problems of debt and repayment from the beginning of recorded history, and have attempted a wide variety of solutions to those problems, including massive debt forgiveness. The jubilee described in the Bible was not novel; rather, it reflected a common practice to keep abuses of debt under control in ancient Sumer. The subtitle of Debt is The First 5,000 Years, and this book is a historical survey. But Graeber puts off the history to first lay an intellectual groundwork for our understanding of debt, and I found that preliminary discussion extremely valuable. Most memorable was the way Graeber divides human economic relationships into communism (in the old sense, not the political sense), exchange, and hierarchy. Capitalism has consumed our economic analysis to such a degree that exchange is the only economic basis that gets much discussion, but the other two are both obvious and pervasive once Graeber points them out. Human civilization could not exist without all three. And Graeber also points out one aspect of exchange-based economics that had not previously occurred to me: it's the economic relationship that one creates with strangers. Debt has the unique characteristic that it can be discharged, at which point the relationship ceases. This has far more complex and far-reaching moral and social implications than one might initially realize, and Graeber did a wonderful job opening my eyes to some of the subtleties. Debt is clearly a scholarly work, but Graeber's writing is clear and engaging. I found most of this book to be surprisingly easy reading. The hardest going was Graeber's discussion of societies that use a form of currency to arrange relationships between people (marriages, births, and deaths, primarily), but not day-to-day economic transactions. I suspect this area is closer to Graeber's areas of personal research and field work, which resulted in more technical detail. I'm still not sure I completely grasp the principles that Graeber was trying to communicate. But I was struck by the observation of alienation's role in turning a human being into a commodity, and how that links with debt's role as the economic transaction one has with strangers. Graeber covers slavery only glancingly, but makes some memorable points about the use of violence to rip someone out of their social context, and how that is necessary in human cultures before humans can be reduced to a commodity. There's a lot here, and I've only scratched the surface. I haven't mentioned, for example, the fascinatingly elegant theory that coining money and then requiring taxes be paid in the same money is a simple and highly effective way of funding armies, an explanation for specie that is largely unproven but that I find more compelling than the ones I've previously heard. Approaching debt from an anthropological instead of economic perspective is surprisingly enlightening. Debt is primarily a historical and cultural discussion rather than a set of proposed solutions, but Graeber does effectively show that debt as a moral obligation is not an unquestionable moral stance, but rather has a long history as one side of a two-sided political debate. I also came away from this book more conscious of the social implications and costs of debt-structured interactions, and wanting to push more of the language of debt out of my day-to-day dealings. Graeber is well known as one of the supporters of Occupy Wall Street, and Debt, while a well-defended academic work, certainly does advocate a position. But the academic analysis is more prominent than the advocacy, and I found his positions well-defended and well-argued. I do need to give the caveat that I don't have the anthropological background to distinguish the statements from Graeber that are well-established common knowledge in anthropology from the ones that are more controversial, and I would be a bit leery of taking this book as the final word on the topic. But it fully deserves its popularity and reputation as a thought-provoking and valuable contribution to the conversation. It's another book that I want to re-read someday to digest further, and the sort of book whose observations keep occurring to me in subsequent discussions or news stories. If you're at all interested in the way in which we construct the morality around economics and debt, I think this is a book that you should read. It's thoughtful, challenging, and surprising, and it passes my acid test for books of this sort with flying colors: after reading it, you realize that many things are more complicated, more historical, and less novel than you had originally thought. Highly recommended. Rating: 9 out of 10

1 June 2014

Paul Tagliamonte: Back from TCamp 2014

Great time, super well organized by this year s TCamp staff. Really outstanding. Lots of really amazing discussion, and I feel a lot of effort is finally jelling around Open Civic Data, which is an absolute thrill for me. Can t wait to see what the next few months bring!

31 May 2014

Dirk Eddelbuettel: RcppGSL 0.2.1 and 0.2.2

A few days ago, version 0.2.1 of RcppGSL---our interface package between R and the GNU GSL using our Rcpp package for seamless R and C++ integration---appeared on CRAN making it the first release in some time. And it turned that this version tickled an obscure and long-dormant bug under clang which was found on OS X Mavericks---which Kevin Ushey consequently squashed. So this is now reflected in version 0.2.2 which just arrived on CRAN. Besides the bugfix, a few things were modernized to reflect capabilities of Rcpp 0.11.0 and later. The releases also contain a few changes that had accumulated since the previous release in 2012 such as an additional example using B-splines, and use of updated vignette build options provided by R. The NEWS file entries follows below:
Changes in version 0.2.2 (2014-05-31)
  • A subtle bug (tickled only by clang on some OS versions) in vector and matrix view initialization was corrected by Kevin Ushey
Changes in version 0.2.1 (2014-05-26)
  • Added new example based on B-splines example in GSL manual illustrating simple GSL use via Rcpp attributes
  • Vignette compilation has been reverted to using highlight since version 0.4.2 or greater can be used as a vignette engine (with R 3.0.* or later).
  • Vignette compilation is now being done by R CMD build as R 3.0.0 supports different vignette engines, so the vignette build process has been simplified. A convenience helper script has also been added for command-line builds.
  • Unit tests now use sourceCpp() instead of cxxfunction() from the inline package
  • The DESCRIPTION file now uses Suggests: Rcpp (instead of Depends: Rcpp) to permit building of the vignette
  • The package now takes advantage of the simplified build process available with Rcpp (>= 0.11.0)
  • Similar updates to the build process were made for the example package included with RcppGSL
And courtesy of CRANberries, a summaries of changes to release 0.2.0 and this week's release 0.2.1 are available. More information is on the RcppGSL page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

30 May 2014

Julien Danjou: OpenStack Design Summit Juno, from a Ceilometer point of view

Last week was the OpenStack Design Summit in Atlanta, GA where we, developers, discussed and designed the new OpenStack release (Juno) coming up. I've been there mainly to discuss Ceilometer upcoming developments. The summit has been great. It was my third OpenStack design summit, and the first one not being a PTL, meaning it was a largely more relaxed summit for me! On Monday, we started by a 2.5 hours meeting with Ceilometer core developers and contributors about the Gnocchi experimental project that I've started a few weeks ago. It was a great and productive afternoon, and allowed me to introduce and cover this topic extensively, something that would not have been possible in the allocated session we had later in the week. Ceilometer had his design sessions running mainly during Wednesday. We noted a lot of things and commented during the sessions in our Etherpads instances. Here is a short summary of the sessions I've attended. Scaling the central agent I was in charge of the first session, and introduced the work that was done so far in the scaling of the central agent. Six months ago, during the Havana summit, I proposed to scale the central agent by distributing the tasks among several node, using a library to handle the group membership aspect of it. That led to the creation of the tooz library that we worked on at eNovance during the last 6 months. Now that we have this foundation available, Cyril Roelandt started to replace the Ceilometer alarming job repartition code by Taskflow and Tooz. Starting with the central agent is simpler and will be a first proof of concept to be used by the central agent then. We plan to get this merged for Juno. For the central agent, the same work needs to be done, but since it's a bit more complicated, it will be done after the alarming evaluators are converted. Test strategy The next session discussed the test strategy and how we could improve Ceilometer unit and functional testing. There is a lot in this area to be done, and this is going to be one of the main focus of the team in the upcoming weeks. Having Tempest tests run was a goal for Havana, and even if we made a lot of progress, we're still no there yet. Complex queries and per-user/project data collection This session, led by Ildik V ncsa, was about adding finer-grained configuration into the pipeline configuration to allow per-user and per-project data retrieval. This was not really controversial, though how to implement this exactly is still to be discussed, but the idea was well received. The other part of the session was about adding more in the complex queries feature provided by the v2 API. Rethinking Ceilometer as a Time-Series-as-a-Service This was my main session, the reason we met on Monday for a few hours, and one of the most promising session I hope of the week. It appears that the way Ceilometer designed its API and storage backends a long time ago is now a problem to scale the data storage. Also, the events API we introduced in the last release partially overlaps some of the functionality provided by the samples API that causes us scaling troubles. Therefore, I've started to rethink the Ceilometer API by building it as a time series read/write service, letting the audit part of our previous sample API to the event subsystem. After a few researches and experiments, I've designed a new project called Gnocchi, which provides exactly that functionality in a hopefully scalable way. Gnocchi is split in two parts: a time series API and its driver, and a resource indexing API with its own driver. Having two distinct driver sets allows it to use different technologies to store each data type in the best storage engine possible. The canonical driver for time series handling is based on Pandas and Swift. The canonical resource indexer driver is based on SQLAlchemy. The idea and project was well received and looked pretty exciting to most people. Our hope is to design a version 3 of the Ceilometer API around Gnocchi at some point during the Juno cycle, and have it ready as some sort of preview for the final release. Revisiting the Ceilometer data model This session led by Alexei Kornienko, kind of echoed the previous session, as it clearly also tried to address the Ceilometer scalability issue, but in a different way. Anyway, the SQL driver limitations have been discussed and Mehdi Abaakouk implemented some of the suggestions during the week, so we should very soon see more performances in Ceilometer with the current default storage driver. Ceilometer devops session We organized this session to get feedbacks from the devops community about deploying Ceilometer. It was very interesting, and the list of things we could improve is long, and I think will help us to drive our future efforts. SNMP inspectors This session, led by Lianhao Lu, discussed various details of the future of SNMP support in Ceilometer. Alarm and logs improvements This mixed session, led by Nejc Saje and Gordon Chung, was about possible improvements on the alarm evaluation system provided by Ceilometer, and making logging in Ceilometer more effective. Both half-sessions were interesting and led to several ideas on how to improve both systems. Conclusion Considering the current QA problems with Ceilometer, Eoghan Glynn, the new Project Technical Leader for Ceilometer, clearly indicated that this will be the main focus of the release cycle. Personally, I will be focused on working on Gnocchi, and will likely be joined by others in the next weeks. Our idea is to develop a complete solution with a high velocity in the next weeks, and then works on its integration with Ceilometer itself.

Jaldhar Vyas: OOH OOH Microagression!

Debian recently instituted a code of conduct which makes the astute observation that "a community in which people feel threatened is not a healthy community". So imagine my shock when I read Russell Cokers blog via Debian Planet, where commenting on a link from the intellectual powerhouse known as salon.com he said "The next logical step is to treat conservatism as a mental health problem." Being a conservative myself I immediately lost all my self-esteem and Dovecots SSL cert handling stopped working. What if I attend debconf and Coker uses his white privilege to sic Mental Health Services on me? I'm scared, so very scared and I don't know if I can continue not working on Debian Minix any longer. I demand that the Debian project, nay the world, should be OUTRAGED I'm talking spelling naughty words in hexadecimal levels of outrage and force this maniac to shut up before any more innocent flowers are brutalized by his "opinions."

Next.

Previous.