Search Results: "cas"

6 November 2012

Andrew Pollock: [life] On leasing a Leaf

Circumstances have required us to become a two-car family, and I've been very keen on the Nissan Leaf for some time. It's an excellent little car. A friend had leased one, and had positive things to say about the whole leasing process, so on Saturday we leased one ourselves. Now that we've been through the process, I'm not sure why you'd ever buy a car instead of lease. We paid $4,628 as a "down payment" (of which we'll get a $2,500 rebate) and it's going to cost I think $257 a month. Maintenance for the 3 years is included in those monthly payments as is any damage to the vehicle at the end of the lease. After the 3 years you hand the car back or pay out the residual. We plan on handing the car back. We get 12,000 miles a year, after which it's 15 cents a mile. There's no way in the world we'll do 12,000 miles a year in it. The last three cars I've owned I've just paid for in cash, because I generally don't like debt and like to minimise my monthly outlays, but in the case of cars, I also don't like forking out a large wad of cash for something that instantly loses value the moment it drives off the lot. I also think that given the pace of innovation in the electric vehicle space, the new hotness in three years time will be even more awesome than the Leaf is today. It gets delivered tomorrow.

Raphaël Hertzog: My Free Software Activities in October 2012

This is my monthly summary of my free software related activities. If you re among the people who made a donation to support my work (120.46 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Dpkg At the start of the month, I reconfigured dpkg s git repository to use KGB instead of the discontinued CIA to send out commit notices to IRC (on #debian-dpkg on OFTC, aka irc.debian.org). I didn t do anything else that affects dpkg and I must say that Guillem does not make it easy for others to get involved. He keeps all his work hidden in his private for 1.17.x branch and refuses to open an official jessie branch as can be seen from the lack of answer to this mail. On the bright side, he deals with almost all incoming bugs even before I have a chance to take care of them. But it s a pity that I can never review any of his fixes because they are usually pushed shortly before an upload. Misc packaging I helped to get #689336 fixed so that the initrd properly setups the keymap before asking for a passphrase for an encrypted partition. Related to this I filed #689722 so that cryptsetup gains a dependency ensuring that the required tools for keymap setup are available. I packaged a new upstream version of zim (0.57) and also a security update for python-django that affected both Squeeze and Wheezy. I uploaded an NMU of revelation (0.4.13-1.2) so that it doesn t get dropped from Wheezy (it was on the release team list of leaf packages that would be removed if unfixed) since my wife is using it to store her passwords. I sponsored a new upstream version of ledgersmb. Debian France We managed to elect new officers for Debian France. I m taking over the role of president, Sylveste Ledru is the new treasurer and Julien Danjou is the new secretary. Thank you very much to the former officers: Carl Chenet, Aur lien Jarno and Julien Cristau. We re in the process of managing this transition which will be completed during the next mini-Debconf in Paris so that we can exchange some papers and the like. In the first tasks that I have set myself, there s recruiting two new members for the boards of directors since we re only 7 and there are 9 seats. I made a call for volunteers and we have two volunteers. If you want to get involved and help Debian France, please candidate by answering that message as soon as possible. The Debian Handbook I merged the translations contributed on debian.weblate.org (which led me to file this wishlist bug on Weblate itself) and I fixed a number of small issues that had been reported. I made an upload to Debian to incorporate all those fixes But this is still the book covering Squeeze so I started to plan the work to update it for Wheezy and with Roland we have decided who is going to take care of updating each chapter. Librement Progress is annoyingly slow on this project. Handling money for others is highly regulated, at least in the EU apparently. I only wanted an escrow account to secure the money of users of the service but opening this account requires either to be certified as a payment institution by the Autorit de contr le prudentiel or to get an exemption from the same authority (covering only some special cases) or to sign a partnership with an established payment institution. Being certified is out of scope for now since it requires a minimum of 125000 EUR in capital (which I don t have). My bank can t sign the kind of partnership that I would need. So I have to investigate whether I can make it fit in the limited cases of exemption or I need to find another payment institution that is willing to work with me. Gittip uses Balanced a payment service specialized in market places but unfortunately it s US-only if you want to withdraw money from the system. I would love a similar service in Europe If I can t position Librement as a market place for the free software world (and save each contributor the hassle to open a merchant account), then I shall fallback to the solution where Librement only provides the infrastructure but no account, and developers who want to collect donations will have to use either Paypal or any other supported merchant account to collect funds. That s why my latest spec updates concerning the donation service and the payment service mentions Paypal and the possibility of choosing your payment service for your donation form. Thanks See you next month for a new summary of my activities.

5 comments Liked this article? Click here. My blog is Flattr-enabled.

MJ Ray: Why co-ops and social enterprises should avoid publishing Word files

I sometimes ask other co-ops and social enterprises to publish things as web pages, PDFs, ODFs or basically any standard format instead of Microsoft Word Doc files. Doc files have the practical problem that they look different even in different versions of Microsoft Word, but also, Microsoft is not a co-operative (a private-sector firm oft accused of bring a monopolist, in fact!), whereas PDF, ODF and so on are more co-operatively developed, so I feel that we should support the better alternatives. The usual first move is to PDFs. I don t like them as much as web pages (it s a file to download and open in an app or plugin, rather than something I can just read without interrupting my flow), but the process is basically the same. Just upload a PDF where you would have uploaded a Doc. Occasionally, I get a reply saying that PDF is no better than Word because Adobe are also a private-sector supplier and therefore not much different from Microsoft (not as big, though?), while doc files are an ISO standard too. I m careful to suggest using ISO PDFs rather than Adobe PDFs. Although PDF started with Adobe, it was given to an independent ISO process (currently co-chaired by someone from Microsoft, which amuses me) and so software to read it has been developed by a wide range of people, including our co-op in a small part. You can find some non-Adobe software for reading PDFs at http://pdfreaders.org there is also alternative PDF creation software, such as that built-in to libreoffice, but I don t know of a good similar listing for them. Microsoft have only given part of Doc (called Docx) to the ISO process and there is a limited range of other readers for it, which suffer the same different on every version problem as Word as far as I ve seen. There s also the added complication that Microsoft Office reportedly won t comply with the Strict standard before Office 2013 More widely, Microsoft are so un-cooperative that they don t seem to want to share a marketplace with anyone else and have been hauled in front of regulators numerous times for monopoly offences. They seem to be on their way there again. So Adobe is not great (does its best to continue marketing and extending PDF as if it had sole control), but Microsoft is much worse. ISO ODF and web pages are other possibilities. The ODF standards are mainly developed by the democratic non-profit Organization for the Advancement of Structured Information Standards, while web standards are mainly developed by an unincorporated trade consortium, the W3C. While they are not themselves co-operatives, co-operatives can be members of both of those. So, whatever the problem, please prefer to publish web pages, PDF or ODF files, rather than Microsoft Word Doc files.

Russ Allbery: Review: Cerebus

Review: Cerebus, by Dave Sim
Series: Cerebus #1
Publisher: Aardvark-Vanaheim
Copyright: August 1987
Printing: July 2003
ISBN: 0-919359-08-6
Format: Graphic novel
Pages: 546
Cerebus is something of a legend in comics. Begun in December of 1977 by Dave Sim, it was one of the first entirely independent, self-published comics in a field dominated by the large work-for-hire companies like Marvel and DC. It ran for 300 issues and nearly 27 years and became one of the most influential independent comic books of all time, in part due to Sim's outspoken views in favor of creator rights and his regular use of the editorial pages in Cerebus issues to air those views. This collection (the first "phonebook") collects issues 1 through 25, with one of the amazing wrap-around covers that makes all of the phonebooks so beautiful (possibly partly by later Cerebus collaborator Gerhard, although if so it's uncredited so far as I can tell). Cerebus reliably has some of the best black-and-white art you will ever see in comics. There is some debate over where to start with Cerebus, and a faction that, for good reasons, argues for starting with the second phonebook (High Society). While these first twenty-five issues do introduce the reader to a bunch of important characters (Elrod, Lord Julius, Jaka, Artemis Roach, and Suenteus Po, for example), all those characters are later reintroduced and nothing that happens here is hugely vital for the overall story. It's also quite rough, starting as Conan parody with almost no depth. The first half or so of this collection features lots of short stories with little or no broader significance, and the early ones are about little other than Cerebus's skills and fighting abilities. That said, when reading the series, I like to start at the beginning. It is nice to follow the characters from their moment of first introduction, and it's delightful to watch Sim's ability grow (surprisingly quickly) through the first few issues. Cerebus #1 is bad: crude, simplistic artwork, almost nothing in the way of a story, and lots of purple narration. But flipping forward even to Cerebus #6 (the first appearance of Jaka), one sees a remarkable difference. By Cerebus #7, Cerebus looks like himself, the plot is getting more complex, and Sim is clearly hitting his stride. And, by the end of this collection, the art has moved from crude past competent and into truly beautiful in places. It's one of the few black-and-white comics where I never miss color. The detailed line work is more enjoyable than I think any coloring could be. The strength of Cerebus as an ongoing character slowly emerges from behind the parody. What I like the most about Cerebus is that he's neither a predestined victor (apart from the early issues that follow the Conan model most closely) nor a pure loner who stands apart from the world. He gets embroiled in political affairs, but almost always for his own reasons (primarily wealth). He has his own moral code, but it's fluid and situational; it's the realistic muddle of impulse and vague principle that most of us fall back on in our everyday life, which is remarkably unlike the typical moral code in comics (or even fiction in general). And while he is in one sense better and more powerful than anyone else in the story, that doesn't mean Cerebus gets what he wants. Most stories here end up going rather poorly for him, forcing daring escapes or frustrating cutting of losses. Sim quickly finds a voice for Cerebus that's irascible, wise, practical, and a bit world-weary, as well as remarkably unflappable. He's one of the best protagonists in comics, and that's already clear by the end of this collection. Parody is the focus of these first issues, which is a mixed bag. The early issues are fairly weak sword-and-sorcery parody (particularly Red Sonja, primarily a vehicle for some tired sexist jokes) and worth reading only for the development in Sim's art style and the growth of Cerebus as a unique voice. Sim gets away from straight parody for the middle of the collection, but then makes an unfortunate return for the final few issues, featuring parodies of Man-Thing and X-Men that I thought were more forced than funny. You have to have some tolerance for this, and (similar to early Pratchett) a lot of it isn't as funny as the author seems to think it is. That said, three of Sim's most brilliant ongoing characters are parodies, just ones that are mixed and inserted into the "wrong" genres in ways that bring them alive. Elrod of Melvinbone, a parody of Moorcock's Elric of Melnibone who speaks exactly like Foghorn Leghorn, should not work and yet does. He's the source of the funniest moments in this collection. His persistant treatment of Cerebus as a kid in a bunny suit shouldn't be as funny as it is, but it reliably makes me laugh each time I re-read this collection. Lord Julius is a straight insertion of Groucho Marx who really comes into his own in the next collection, High Society, but some of the hilarious High Society moments are foreshadowed here. And Artemis Roach, who starts as a parody of Batman and will later parody a huge variety of comic book characters, provides several delightful moments with Cerebus as straight man. I'm not much of a fan of parody, but I still think Cerebus is genuinely funny. High Society is definitely better, but I think one would miss some great bits by skipping over the first collection. Much of what makes it work is the character of Cerebus, who is in turn a wonderful straight man for Sim's wilder characters and an endless source of sharp one-liners. It's easy to really care about and root for Cerebus, even when he's being manipulative and amoral, because he's so straightforward and forthright about it. The world Sim puts him into is full of chaos, ridiculousness, and unfairness, and Cerebus is the sort of character to put his head down, make a few sarcastic comments, and then get on with it. It's fun to watch. One final note: I've always thought the "phonebook" collections were one of Sim's best ideas. Unlike nearly all comic book collections, a Cerebus phonebook provides enough material to be satisfying and has always felt like a good value for the money. I wish more comic book publishers would learn from Sim's example and produce larger collections that aren't hardcover deluxe editions (although Sim has an admitted advantage from not having to reproduce color). Followed by High Society. Rating: 7 out of 10

5 November 2012

Russ Allbery: WebAuth 4.3.3

So, funny story. And by funny, I mean "doh." We had a production outage of our WebLogin servers a bit before I went on vacation because the hardware load balancer died, and then when it came back the load was so heavy that the WebLogin servers were overwhelmed with connections. WebLogin, due to the way that it involves accepting an Apache connection and then turning around and connecting back to itself to talk to the WebKDC, has a tendency to get into overload deadlocks. Worse, because the WebLogin script runs under FastCGI, when it deadlocks waiting for Apache to reply to it, mod_fastcgi decides that the script isn't responding and starts killing all the scripts. It's an obnoxious meltdown scenario. I want to find some way of getting around this (such as running WebLogin via ngnix and running the WebKDC on another port), but in the short term the best solution is to just increase MaxClients to handle more load. The limitation there is the amount of memory on the servers, which was a problem since they were built 32-bit (for historical reasons). So, this weekend I started working on rebuilding them all 64-bit. Imagine my surprise when I installed WebAuth and WebLogin on the newly-built 64-bit system and it segfaulted. It turns out that when allocating memory for a struct, it's good to allocate enough memory for the whole struct, not just for a pointer to the struct. (This is exactly the reason why I hate typedefs for structs in C, but in this case it was XS code where I don't have a good alternative, and I didn't pay close enough attention.) So, lots of debugging later, along with a detour to set up comprehensive valgrind testing of WebAuth (at least all the parts of it that aren't the Apache modules), this release fixes one uninitialized memory error and one memory allocation size error. The latter is theoretically exploitable, although since it's inside Perl XS code with pretty sharply bounded external inputs, it's going to be pretty complicated to figure out how to exploit it and therefore I decided it didn't warrant an explicit advisory. Obviously, anyone running earlier versions of mod_webkdc or WebLogin (4.2.0 or later, sigh) should upgrade to this release. You can get the latest release from the official WebAuth distribution site or from my WebAuth distribution pages.

4 November 2012

Wouter Verhelst: Windows usability: followup.

Yesterday, I posted a blog post containing a list of issues with Windows, in response to a similar list for Linux, which has received quite some response; I've seen people +1 it on google+, and I've received a number of replies on the blog post itself. One of these replies was by the author of the original list; and as expected, he totally and utterly failed to see the point. He also has the audacity to claim I haven't installed Linux yet what?!? The last time I used anything other than Linux on my personal machine for any real work was on Windows 98. And no, that was not because I was too cheap to buy the latest and greatest. I have used Windows in some cases since then, but never on my own hardware. First of all, I should note that the list which I posted was only semi-serious (with tongue in cheek); the fact that it refers to Windows as the operating system which "attempts to replace Linux as the major operating system" should have been a pretty big clue to that effect. Like the original list that it was a reply to, my list contains things that are only true in a particular context (e.g., the one about requiring recent hardware), it contains things that don't matter for the large majority of Windows users (e.g., the point about focus-follows-mouse) but that are major arguments for why I don't use Windows, it contains things that are so alien to what most of us are used to that even most Linux users don't use it, while on the other hand some swear by it (like the SELinux argument), and it contains some things that a seasoned Windows user will call bullshit on but that are real actual problems for the non-seasoned Windows user that I am (such as the mouse argument). It also contains some actual facts, like the statement about most crashes being a direct result of faulty drivers that's pulled directly from a scientific study on the subject conducted by Microsoft, together with some independent researchers. Like the original list, I don't expect this list to change anything for the better, precisely because it is flawed in exactly the same way. But whether the items on that list were right or wrong isn't the point. This is: Nobody (except nutcases) ever said Linux is perfect, or that it's useful for everyone; and while the issues Artem lists may be bugs or may be design flaws or any of a number of other things that could be real problems, the fact is that creating a long "list of issues" isn't going to solve anything. If you find that there are problems with Linux, you basically have three options: fix the problem, get someone to fix it for you, or stop using Linux. Really. Rather than better or worse, Linux is different. It will have its own set of problems, and it will have its own set of advantages. Some of the design decisions cause problems, but also have serious advantages, and focussing on only these problems (the existence of which I'm not denying) isn't very helpful. A good example is the library model: under Linux, libraries are typically installed in a system-wide location, and use a SONAME which is (part of) the filename of that library and which defines the binary interface exported by that library. In contrary to the situation under Windows, Linux libraries also must use position-independent code. The result is a more robust library model. The downside to that model, however, is that when you install older applications that require older SONAME versions of a particular library, getting those older versions may be a challenge. While in theory, installing an older version of a library is not going to cause any problem if the SONAME is different, in practice you'll often find that you'll need to go to (unsupported) older packages from older versions of the distribution, which may not install due to missing or conflicting dependencies, or which may expect data files in a different format in the same location that newer versions of the same library do, or which may have any number of other issues. When that happens, the solution is to either use an older version of the distribution in a chroot or some such, or to patch and/or compile the needed libraries yourself, and install them alongside the newer version of the same library. That's perfectly possible, after all, precisely because of the SONAME. Another good example is the development model: where Windows has a central authority (Microsoft) which defines what it is to be "Windows" and what is and isn't part of that, the same simply isn't true about Linux. Yes, Linus defines what the official Linux kernel has, but he doesn't have any say about the user interface, the boot process, or any number of other things; other people make decisions there. As a result, in effect, who decides what it means to be a "Linux system" is, really, you, the user. If you believe that the Linux user space is completely wrong, you can just change it to something else entirely; and that's precisely what Google did when they created Android. Since there's no central authority making decisions about what will and won't go into the system, this means that inevitably, at any given time, there will be many competing options for many subsystems. Usually this will be an advantage; e.g., the competition between KDE and Gnome benefits both. Sometimes, however, the competing options are so confusingly interwoven that even those who should know the system inside out can't make heads or tails of it, and debugging issues then becomes quite a challenge. Unfortunately, the Linux sound subsystem is currently an example of the latter. I'm not saying that all the items in Artem's list are invalid; a minority are, in fact, actual problems for a majority of Linux users. Most however, are personal opinions, issues affecting only a small minority of Linux users, or, as above, things where he only talks about the downsides of a design decision that also has major advantages. As an example of something in the second category: it's easy, for example, to list hundreds if not thousands of issues relating to particular pieces of hardware that don't work under Linux, simply because the manufacturer doesn't provide any information, and the people developing drivers for Linux have to reverse engineer how everything works. If you're a bit careful in what you buy, however much like MacOS users need to be careful, too then using Linux isn't an issue. My Lenovo Thinkpad X220, for instance, has no parts that don't work; and this includes things like a docking station, and the fact that my (9-cell) battery life is over fourteen hours, which (I believe) is more than on the same laptop with the same battery under Windows. So how is making a list of issues useful? The answer to that is simply, not at all. If you want to help, don't look for other people's problems; we can all do that. Instead, try to look for solutions to these problems. That's not always easy, and in many cases requires that you first understand why something is done in a particular way. If you don't want to spend time doing that, that's perfectly fine; but then don't expect that open source developers (who are already strapped for time, usually) will suddenly jump up and fix the problems you think are most important, as opposed to what they think should be worked on most urgently. After all, most Open Source developers have their own sets of priorities what they think requires most work, and that's what they'll work on not what's shown in a particular list by someone they don't know and have no reason to respect, somewhere on the Internet. Finally, I'd like to note that any statement which claims that Linux isn't "ready" to be used on the desktop, when hundreds of thousands (if not millions) of people are using it in exactly that way for their day-to-day work, is insulting these people's intelligence. Since that includes me, it's also insulting my intelligence. And I don't like to have my intelligence insulted...

Petter Reinholdtsen: The European Central Bank (ECB) take a look at bitcoin

Slashdot just ran a story about the European Central Bank (ECB) releasing a report (PDF) about virtual currencies and bitcoin. It is interesting to see how a member of the bitcoin community receive the report. As for the future, I suspect the central banks and the governments will outlaw bitcoin if it gain any popularity, to avoid competition. My thoughts go to the W rgl experiment with negative inflation on cash which was such a success that it was terminated by the Austrian National Bank in 1933. A successful alternative would be a threat to the current money system and gain powerful forces to work against it. While checking out the current status of bitcoin, I also discovered that the community already seem to have experienced its first pyramid game / Ponzi scheme. Not very surprising, given how members of "small" communities tend to trust each other. I guess enterprising crocks will try again and again, as they do anywhere wealth is available.

Russ Allbery: Review: Set This House in Order

Review: Set This House in Order, by Matt Ruff
Publisher: Perennial
Copyright: 2003
Printing: 2004
ISBN: 0-06-095485-X
Format: Trade paperback
Pages: 479
A textbook case history would say that Andy Gage has disassociative identity disorder (referred to here as the more common although medically inaccurate "multiple personality disorder" for reasons of clarity and to match casual speech, according to the author). However, that's a completely inadequate description. A better description would be that Andy Gage's body is inhabited by a large set of different personalities, who take turns being in control. The original Andy Gage is dead, shattered by horrific childhood abuse. Aaron became the dominant personality and gave structure to the internal geography of the mind they all share, with the help of extensive counselling. But Aaron was exhausted by being the public face and maintaining control of the shared body. Hence, Andrew. Andrew Gage is the first-person narrator of half of this book. He is twenty-eight, the same age as the body, but he's only existed for two years. He considers Aaron his father, since Aaron called him out of the lake inside their internal metal geography. His purpose is to run the body and be the public face of the family. He is by turns capable and naive, shy on personal experience, but able to get the help of the others. The one heard from most frequently is Adam, a rude and mischevious teenager who also has an uncanny ability to read other people, and who frequently watches through the body's eyes from what the family calls the "pulpit" and gives Andrew advice. Andrew works for Julie Sivik, a woman he met the day he quit his job at a big-box store because he couldn't explain his behavior to his boss without discussing being a multiple. Julie is a wholly impractical, slightly unethical, well-meaning force of chaos who serves as the thread that draws the plot of the book together. She's running a startup on ingenuity and borrowed money that is attempting to create a comprehensive and customizable virtual reality world. She's the one who finds Penny and hires her, in part for her skills, and in part to push Penny and Andrew together. Penny Driver has the same condition as Andrew, but Penny is only vaguely aware of it. The world for her is full of blackouts, lost time, waking up in strange and sometimes horrific situations, and attempting to structure her life based on notes left for herself that she didn't write. When she meets Andrew, she's barely making it through life; thankfully, one of her personalities is remarkably good at computers, and that has been enough to make a living despite the chaos. But Julie is the unintentional catalyst for her losing that job, the best job she's ever had, leaving her with no real alternative but to take the job with Julie's company. There are a few things that are important to say up-front about this book. First, neither I nor the author are actually multiples (or, put another way, neither of us have DID), and writing about atypical mental states is fraught with peril. It's very easy to misrepresent other people's experiences, and multiple personalities in particular are frequent material for dramas of all sorts. There's something people find inherently fascinating about the idea, and it's easy for that fascination to run away with reality and distort the lived experience. I recommend reading Ruff's FAQ about the book, which spells out what his goals are and what approach he took. I did some quick searches and didn't uncover a lot of anger or dislike of the book among either people who have DID or psychiatrists, and it seems true to the descriptions of internal reality I've heard from people who are multiples. But despite being more respectful of real experience than most fictional treatments of DID, this is still a work of fiction written by someone without first-hand experience, and it's worth keeping that in mind. Second, DID is an intensely controversial diagnosis among psychiatrists. There are, generally speaking, two major controversies. One is over whether or not DID can actually be caused by traumatic experiences such as childhood abuse, or whether it is a creation of psychiatrists, a mental model that they've imposed on their patients or convinced their patients to believe. Set This House in Order comes down firmly on the first side of that controversy: that DID can be created by abuse and is not a creation of psychotherapy. (So, I should note, does every multiple that I've ever talked to or read.) Second, there is significant controversy over treatment, with some psychiatrists believing that integration of the personalities into one is the only true treatment, and others believing that multiples can remain stable and functional as multiple personalities. Set This House in Order leans strongly towards the view that multiples can be stable as multiples and that while integration may be appropriate for some, it's not for all. Both of these viewpoints are presented very convincingly by the narrative in the book, and both are consistent with the beliefs of a lot of multiples, but it's worth remembering that these are active and often quite heated controversies and a novel is not evidence. It's good to be aware of both of those points, and to read Put This House in Order as something of a what-if, where much more is clear about DID than is actually clear in our world. But once one has done that, one is in for an excellent book. I loved this book from start to finish. (Well, okay, I felt some surprise at the nature of the ending, but it grew on me after further consideration.) It's simply a brilliant novel, with a little bit of everything: very good characterization, messy love affairs, the satisfying tension of someone overcoming great odds to put themselves together with the help of friends, a startlingly original perspective of the world, some excellent surprises, and some absolutely wonderful characters. Andrew is a good choice of narrator: he brings an earnest calm to a story that could otherwise be quite chaotic and provides a great vantage point for the reader. And I found Penny (and particularly some of her other souls) impossible not to like. Set This House in Order is a tense book in places, and it's a book where people's lives break down before they can get better. It's also a book about rather horrific abuse, and while nearly all of the specifics happen off-stage, imagination fills in some nasty details. But Ruff manages to write it with a structure and with enough calm assurance that I felt just reassured enough without taking away the tension. That's a tricky balance; few books do it better. And the supporting characters are a delight. One might think, with all the complexity happening for both Andrew and Penny, that Ruff wouldn't have room in the narrative for other characters, but one would be wrong. There are very few saints and demons here, just a lot of complex people, with their own problems and their own perspectives, who are generally trying to do the best they can from day to day. There are some clear-cut villains in the past Ruff wisely makes no attempt to add shades of grey to abusers but even there one can see some clear signs of untreated severe mental illness and chaotic lives rather than mustache-twirling pre-meditated villainy. This is, in short, a book full of people, in all their messy complications, which is what makes it so delightful. I'm not going to say anything more about the plot, or even the various personalities, since discovering the direction of the book was so much of the fun. I will say that, despite the subject matter and some of the events, the book is surprisingly gentle. One of the things about it that I loved the most was the emotional tone Ruff gave to being a multiple: the sense of a squabbling but tight-knit family looking out for each other. I can't judge the reality of this impression, but it makes for a wonderful story. Highly recommended. The best book I've read so far this year. Rating: 10 out of 10

Wouter Verhelst: Why Windows is not ready for general usage

There's this new Windows thing that people keep talking about, and which attempts to replace Linux as the major operating system. I've given it a look, but there's a wide range of problems with using it: In short, while this Windows thing has some promise, the above list shows that there's clearly still quite some work to be done before anyone can even think about starting to use it seriously. Hopefully the Windows developers will understand that and work on the above list of serious issues; otherwise, I'm afraid Windows will not be used by many people. Update: before you go all berserk and start commenting on this list, please read my followup to this post, which explains the point I was trying to make.

3 November 2012

Richard Hartmann: Release Critical Bug report for Week 42

Lucas Nussbaum pointed out that the weekly stats which Alexander Reichle-Schmehl used to provide are AWOL so I decided to take over. As I am patently lazy, I hacked together a little helper in perl which outputs the page I am just adding to by writing this. It's a tad hackish inasmuch I don't bother to transform the HTML into anything and simply embedd it in the Markdown, but it's working well enough for now. In the medium term, I am hoping to add some nice graphs as per azhag's suggestion, but they didn't get back to me, yet. A warning to anyone not subscribed to Planet Debian and reading this via a different aggregator: You will still see weekly stats from now on which may or may not be interesting to you. In case you object, please leave a comment or poke me directly. The UDD bugs interface currently knows about the following release critical bugs: How do we compare to the squeeze cycle ?
Week Squeeze Wheezy Diff
43 284 (213+71) 468 (332+136) +184 (+119 ; +65)
44 261 (201+60)
45 261 (205+56)
46 271 (200+71)
47 283 (209+74)
48 256 (177+79)
49 256 (180+76)
50 204 (148+56)
51 124 (124+54)
52 115 (78+37)
1 93 (60+33)
2 82 (46+36)
3 25 (15+10)
4 14 (8+6)
5 2 (0+2)
6 release!

2 November 2012

Neil Williams: Introducing pyBit - Buildd Integration Toolkit

pyBit - cross-platform package building using AMQP


Message queues provide a simple way to create a distributed, cross-platform buildd toolkit to build packages using a collection of buildds, direct from various VCS clients. pyBit is intended to support rapidly evolving software collections and can support multiple VCS frontends and multiple build backends. Cross building is expected to be supported for some backends. The initial backend uses dpkg for Debian with subversion providing the source and sbuild doing the actual build.

pyBit includes support for cancelling selected builds and using multiple buildd clients per architecture, per platform and per suite.

Hooks are available or in development for subversion and git, other VCS hooks can be added. A RESTful web API provides live build reports and can generate build jobs for specific packages using particular VCS branches on selected architectures to support re-building packages at any point in the development process. Build history is stored using postgresql.

Other buildd systems can rebuild long lists of packages or build lots of binary packages from relatively slow moving source packages. PyBit exists to handle much more rapid software development across a wide range of platforms, VCS inputs and architectures. Buildd clients which are under-used can be tasked with building multiple suites or adding cross-build support. Buildd clients which are over-utilised are easily identified and adding new machines to an existing architecture / platform / suite pool should be easy. Hook activation automatically cancels any ongoing build for the same architecture, platform and suite to avoid wasting time on an interim version.

The emphasis in pyBit is to have fast builds with redundant clients, reliable reporting using a flexible and intuitive frontend. To this end, there is no need for a source package to be uploaded. Depending on the VCS hook in use, builds can happen every time a particular file is changed (e.g. debian/changelog for a native Debian package) or at every push (for a distributed VCS) or whatever is appropriate for a particular software collection. Builds are checked for available build-dependencies and packages re-queued if the build-dependencies are not yet met.

In the longer term, it may well be possible to use more than one server / database combination to support more builds and more platforms.

So far, we've got to a working model and tagged 0.1.0 as the first downloadable release. There is a lot more to do, especially adding more documentation, more VCS hooks, support for more VCS methods on the buildd clients and more buildd client scripts for platforms other than Debian. (The git hook and git source client are expected to let pyBit be self-buildable but a certain amount of configuration will be required for the server and each client which makes it not-quite self-hosting.)

pyBit concentrates on preparing collections of binary packages which can be used to build others, rather than trying to rebuild everything every time - this allows more rapid upstream software development and encourages modular, re-usable software. pyBit will also support rebuilding specific versions, architectures, suites or platforms via the RESTful web API. Access to this frontend can be controlled through any of the standard methods.

Components are loosely coupled via JSON encoded messages sent using rabbitMQ and curl. A new client can be added at any time and it will simply pick up buildd jobs from the relevant queue.

Development has now moved to Github and if anyone wants to look at new clients and new hooks, just contact the team by email or via IRC (#pybit on irc.oftc.net). Current development is based on Debian Squeeze 6.0.6 with backports and also on Debian Wheezy - more testing is welcome. Patches are also very welcome, pyBit is licensed under GPL2+.

There is no intrinsic reason why pyBit could not support any buildd platform capable of taking source from a known location and building a set of binaries. Packages are added to the queues whenever the hook is activated, so adding new packages to the collection is simply a case of triggering the existing hook.

Another interesting challenge for pyBit would be to trigger a hook on a new source code repository and just let it work through every package until everything is built. That probably won't work with the current 0.1.0 code but if that is what you'd like to work on, join the team.

Vincent Bernat: Network virtualization with VXLAN

Virtual eXtensible Local Area Network (VXLAN) is a protocol to overlay a virtualized L2 network over an existing IP network with little setup. It is currently described in an Internet-Draft. It adds the following perks to VLANs while still providing isolation:
  1. It uses a 24-bit VXLAN Network Identifier (VNI) which should be enough to address any scale-based concerns of multitenancy.
  2. It wraps L2 frames into UDP datagrams. This allows one to rely on some interesting properties of IP networks like availability and scalability. A VXLAN segment can be extended far beyond the typical reach of today VLANs.
The VXLAN Tunnel End Point (VTEP) originates and terminates VXLAN tunnels. Thanks to a serie of patches from Stephen Hemminger, Linux can now act as a VTEP. Let s see how this works.

About IPv6 When possible, I try to use IPv6 for my labs. This is not the case here for several reasons:
  1. IP multicast is required and PIM-SM implementations for IPv6 are not widespread yet. However, they exist. This explains why I use XORP for this lab: it supports PIM-SM for both IPv4 and IPv6.
  2. VXLAN Internet-Draft specifically addresses only IPv4. This seems a bit odd for a protocol running on top of UDP and I hope this will be fixed soon. This is not a major stopper since some VXLAN implementations support IPv6.
  3. However, the current implementation for Linux does not support IPv6. IPv6 support will be added later.
Once IPv6 support is available, the lab should be easy to adapt.

Lab So, here is the lab used. R1, R2 and R3 will act as VTEPs. They do not make use of PIM-SM. Instead, they have a generic multicast route on eth0. E1, E2 and E3 are edge routers while C1, C2 and C3 are core routers. The proposed lab is not resilient but convenient to explain how things work. It is built on top of KVM hosts. Have a look at my previous article for more details on this. VXLAN lab The lab is hosted on GitHub. I have made the lab easier to try by including the kernel I have used for my tests. XORP comes preconfigured, you just have to configure the VXLAN part. For this, you need a recent version of ip.
$ sudo apt-get install screen vde2 kvm iproute xorp git
$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/shemminger/iproute2.git
$ cd iproute2
$ ./configure && make
You get  ip' as  ip/ip' and  bridge' as  bridge/bridge'.
$ cd ..
$ git clone git://github.com/vincentbernat/network-lab.git
$ cd network-lab/lab-vxlan
$ ./setup

Unicast routing The first step is to setup unicast routing. OSPF is used for this purpose. The chosen routing daemon is XORP. With xorpsh, we can check if OSPF is working as expected:
root@c1# xorpsh
root@c1$ show ospf4 neighbor   
  Address         Interface             State      ID              Pri  Dead
192.168.11.11    eth0/eth0              Full      3.0.0.1          128    36
192.168.12.22    eth1/eth1              Full      3.0.0.2          128    33
192.168.101.133  eth2/eth2              Full      2.0.0.3          128    36
192.168.102.122  eth3/eth3              Full      2.0.0.2          128    38
root@c1$ show route table ipv4 unicast ospf   
192.168.1.0/24  [ospf(110)/2]
                > to 192.168.11.11 via eth0/eth0
192.168.2.0/24  [ospf(110)/2]
                > to 192.168.12.22 via eth1/eth1
192.168.3.0/24  [ospf(110)/3]
                > to 192.168.102.122 via eth3/eth3
192.168.13.0/24 [ospf(110)/2]
                > to 192.168.102.122 via eth3/eth3
192.168.21.0/24 [ospf(110)/2]
                > to 192.168.101.133 via eth2/eth2
192.168.22.0/24 [ospf(110)/2]
                > to 192.168.12.22 via eth1/eth1
192.168.23.0/24 [ospf(110)/2]
                > to 192.168.101.133 via eth2/eth2
192.168.103.0/24        [ospf(110)/2]
                > to 192.168.102.122 via eth3/eth3

Multicast routing Once unicast routing is up and running, we need to setup multicast routing. There are two protocols for this: IGMP and PIM-SM. The former one allows routers to forward multicast datagrams while the first one allows hosts to subscribe to a multicast group.

IGMP IGMP is used by hosts and adjacent routers to establish multicast group membership. In our case, it will be used by R2 to let E2 know it subscribed to 239.0.0.11 (a multicast group). Configuring XORP to support IGMP is simple. Let s test with iperf to have a multicast listener on R2:
root@r2# iperf -u -s -l 1000 -i 1 -B 239.0.0.11
------------------------------------------------------------
Server listening on UDP port 5001
Binding to local address 239.0.0.11
Joining multicast group  239.0.0.11
Receiving 1000 byte datagrams
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
On E2, we can now check that R2 is properly registered for 239.0.0.11:
root@e2$ show igmp group
Interface    Group           Source          LastReported Timeout V State
eth0         239.0.0.11      0.0.0.0         192.168.2.2      248 2     E
XORP documentation contains a good overview of IGMP.

PIM-SM PIM-SM is far more complex. It does not have its own topology discovery protocol and relies on routing information from other protocols, OSPF in our case. I will describe here a simplified view on how PIM-SM works. XORP documentation contains more details about PIM-SM. The first step for all PIM-SM routers is to elect a rendez-vous point (RP). In our lab, only C1, C2 and C3 have been configured to be elected as a RP. Moreover, we give better priority to C3 to ensure it wins. RP election
root@e1$ show pim rps   
RP              Type      Pri Holdtime Timeout ActiveGroups GroupPrefix       
192.168.101.133 bootstrap 100      150     135            0 239.0.0.0/8
Let s suppose we start iperf on both R2 and R3. Using IGMP, they subscribe to multicast group 239.0.0.11 with E2 and E3 respectively. Then, E2 and E3 send a join message (also known as a (*,G) join) to the RP (C3) for that multicast group. Using the unicast path from E2 and E3 to the RP, the routers along the paths build the RP tree (RPT), rooted at C3. Each router in the tree knows how to send multicast packets to 239.0.0.11: it will send them to the leaves. RP tree
root@e3$ show pim join   
Group           Source          RP              Flags
239.0.0.11      0.0.0.0         192.168.101.133 WC   
    Upstream interface (RP):   eth2
    Upstream MRIB next hop (RP): 192.168.23.133
    Upstream RPF'(*,G):        192.168.23.133
    Upstream state:            Joined 
    Join timer:                5
    Local receiver include WC: O...
    Joins RP:                  ....
    Joins WC:                  ....
    Join state:                ....
    Prune state:               ....
    Prune pending state:       ....
    I am assert winner state:  ....
    I am assert loser state:   ....
    Assert winner WC:          ....
    Assert lost WC:            ....
    Assert tracking WC:        O.O.
    Could assert WC:           O...
    I am DR:                   O..O
    Immediate olist RP:        ....
    Immediate olist WC:        O...
    Inherited olist SG:        O...
    Inherited olist SG_RPT:    O...
    PIM include WC:            O...
Let s suppose that R1 wants to send multicast packets to 239.0.0.11. It sends them to R1 which does not have any information on how to contact all the members of the multicast group because it is not the RP. Therefore, it encapsulates the multicast packets into PIM Register packets and sends them to the RP. The RP decapsulates them and sends them natively. The multicast packets are routed from the RP to R2 and R3 using the reverse path formed by the join messages. Multicast routing via the RP
root@r1# iperf -c 239.0.0.11 -u -b 10k -t 30 -T 10
------------------------------------------------------------
Client connecting to 239.0.0.11, UDP port 5001
Sending 1470 byte datagrams
Setting multicast TTL to 10
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
root@e1# tcpdump -pni eth0
10:58:23.424860 IP 192.168.1.1.35277 > 239.0.0.11.5001: UDP, length 1470
root@c3# tcpdump -pni eth0
10:58:23.552903 IP 192.168.11.11 > 192.168.101.133: PIMv2, Register, length 1480
root@e2# tcpdump -pni eth0
10:58:23.896171 IP 192.168.1.1.35277 > 239.0.0.11.5001: UDP, length 1470
root@e3# tcpdump -pni eth0
10:58:23.824647 IP 192.168.1.1.35277 > 239.0.0.11.5001: UDP, length 1470
As presented here, the routing is not optimal: packets from R1 to R2 could avoid the RP. Moreover, encapsulating multicast packets into unicast packets is not efficient either. At some point, the RP will decide to switch to native multicast1. Rooted at R1, the shortest-path tree (SPT) for the multicast group will be built using source-specific join messages (also known as a (S,G) join). Multicast routing without RP From here, each router in the tree knows how to handle multicast packets from R1 to the group without involving the RP. For example, E1 knows it must duplicate the packet and sends one through the interface to C3 and the other one through the interface to C1:
root@e1$ show pim join   
Group           Source          RP              Flags
239.0.0.11      192.168.1.1     192.168.101.133 SG SPT DirectlyConnectedS 
    Upstream interface (S):    eth0
    Upstream interface (RP):   eth1
    Upstream MRIB next hop (RP): 192.168.11.111
    Upstream MRIB next hop (S):  UNKNOWN
    Upstream RPF'(S,G):        UNKNOWN
    Upstream state:            Joined 
    Register state:            RegisterPrune RegisterCouldRegister 
    Join timer:                7
    KAT(S,G) running:          true
    Local receiver include WC: ....
    Local receiver include SG: ....
    Local receiver exclude SG: ....
    Joins RP:                  ....
    Joins WC:                  ....
    Joins SG:                  .OO.
    Join state:                .OO.
    Prune state:               ....
    Prune pending state:       ....
    I am assert winner state:  ....
    I am assert loser state:   ....
    Assert winner WC:          ....
    Assert winner SG:          ....
    Assert lost WC:            ....
    Assert lost SG:            ....
    Assert lost SG_RPT:        ....
    Assert tracking SG:        OOO.
    Could assert WC:           ....
    Could assert SG:           .OO.
    I am DR:                   O..O
    Immediate olist RP:        ....
    Immediate olist WC:        ....
    Immediate olist SG:        .OO.
    Inherited olist SG:        .OO.
    Inherited olist SG_RPT:    ....
    PIM include WC:            ....
    PIM include SG:            ....
    PIM exclude SG:            ....
root@e1$ show pim mfc  
Group           Source          RP             
239.0.0.11      192.168.1.1     192.168.101.133
    Incoming interface :      eth0
    Outgoing interfaces:      .OO.
root@e1$ exit
[Connection to XORP closed]
root@e1# ip mroute
(192.168.1.1, 239.0.0.11)        Iif: eth0       Oifs: eth1 eth2

Setting up VXLAN Once IP multicast is running, setting up VXLAN is quite easy. Here are the software requirements:
  • A recent kernel. Pick at least 3.7-rc3. You need to enable CONFIG_VXLAN option. You also currently need a patch on top of it to be able to specify a TTL greater than 1 for multicast packets.
  • A recent version of ip. Currently, you need the version from git.
On R1, R2 and R3, we create a vxlan42 interface with the following commands:
root@rX# ./ip link add vxlan42 type vxlan id 42 \
>                               group 239.0.0.42 \
>                               ttl 10 dev eth0
root@rX# ip link set up dev vxlan42
root@rX# ./ip -d link show vxlan42
10: vxlan42: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1460 qdisc noqueue state UNKNOWN mode DEFAULT 
link/ether 3e:09:1c:e1:09:2e brd ff:ff:ff:ff:ff:ff
vxlan id 42 group 239.0.0.42 dev eth0 port 32768 61000 ttl 10 ageing 300
Let s assign an IP in 192.168.99.0/24 for each router and check they can ping each other:
root@r1# ip addr add 192.168.99.1/24 dev vxlan42
root@r2# ip addr add 192.168.99.2/24 dev vxlan42
root@r3# ip addr add 192.168.99.3/24 dev vxlan42
root@r1# ping 192.168.99.2                    
PING 192.168.99.2 (192.168.99.2) 56(84) bytes of data.
64 bytes from 192.168.99.2: icmp_req=1 ttl=64 time=3.90 ms
64 bytes from 192.168.99.2: icmp_req=2 ttl=64 time=1.38 ms
64 bytes from 192.168.99.2: icmp_req=3 ttl=64 time=1.82 ms
--- 192.168.99.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 1.389/2.375/3.907/1.098 ms
We can check the packets are encapsulated:
root@r1# tcpdump -pni eth0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
11:30:36.561185 IP 192.168.1.1.43349 > 192.168.2.2.8472: UDP, length 106
11:30:36.563179 IP 192.168.2.2.33894 > 192.168.1.1.8472: UDP, length 106
11:30:37.562677 IP 192.168.1.1.43349 > 192.168.2.2.8472: UDP, length 106
11:30:37.564316 IP 192.168.2.2.33894 > 192.168.1.1.8472: UDP, length 106
Moreover, if we send broadcast packets (with ping -b or ARP requests), they are encapsulated into multicast packets:
root@r1# tcpdump -pni eth0
11:31:27.464198 IP 192.168.1.1.41958 > 239.0.0.42.8472: UDP, length 106
11:31:28.463584 IP 192.168.1.1.41958 > 239.0.0.42.8472: UDP, length 106
Recent versions of iproute also comes with bridge, an utility allowing one to inspect the FDB of bridge-like interfaces:
root@r1# ../bridge/bridge fdb show vxlan42
3e:09:1c:e1:09:2e dev vxlan42 dst 192.168.2.2 self 
0e:98:40:c6:58:10 dev vxlan42 dst 192.168.3.3 self

Demo For a demo, have a look at the following video (it is also available as an Ogg Theora video).
<iframe frameborder="0" height="270" src="http://www.dailymotion.com/embed/video/xusell" width="480"></iframe>

  1. The decision is usually done when the bandwidth used by the follow reachs some threshold. With XORP, this can be controlled with switch-to-spt-threshold. However, I was unable to make this works as expected. XORP never sends the appropriate PIM packets to make the switch. Therefore, for this lab, it has been configured to switch to native multicast at the first received packet.

Erich Schubert: DBSCAN and OPTICS clustering

DBSCAN [wikipedia] and OPTICS [wikipedia] are two of the most well-known density based clustering algorithms. You can read more on them in the Wikipedia articles linked above.
An interesting property of density based clustering is that these algorithms do not assume clusters to have a particular shape. Furthermore, the algorithms allow "noise" objects that do not belong to any of the clusters. K-means for examples partitions the data space in Voronoi cells (some people claim it produces spherical clusters - that is incorrect). See Wikipedia for the true shape of K-means clusters and an example that canot be clustered by K-means. Internal measures for cluster evaluation also usually assume the clusters to be well-separated spheres (and do not allow noise/outlier objects) - not surprisingly, as we tend to experiment with artificial data generated by a number of Gaussian distributions.
The key parameter to DBSCAN and OPTICS is the "minPts" parameter. It roughly controls the minimum size of a cluster. If you set it too low, everything will become clusters (OPTICS with minPts=2 degenerates to a type of single link clustering). If you set it too high, at some point there won't be any clusters anymore, only noise. However, the parameter usually is not hard to choose. If you for example expect clusters to typically have 100 objects, I'd start with a value of 10 or 20. If your clusters are expected to have 10000 objects, then maybe start experimenting with 500.
The more difficult parameter for DBSCAN is the radius. In some cases, it will be very obvious. Say you are clustering users on a map. Then you might know that a good radius is 1 km. Or 10 km. Whatever makes sense for your particular application. In other cases, the parameter will not be obvious, or you might need multiple values. That is when OPTICS comes into play.
OPTICS is based on a very clever idea: instead of fixing MinPts and the Radius, we only fix minpts, and plot the radius at which an object would be considered dense by DBSCAN. In order to sort the objects on this plot, we process them in a priority heap, so that nearby objects are nearby in the plot. this image on Wikipedia shows an example for such a plot.
OPTICS comes at a cost compared to DBSCAN. Largely because of the priority heap, but also as the nearest neighbor queries are more complicated than the radius queries of DBSCAN. So it will be slower, but you no longer need to set the parameter epsilon. However, OPTICS won't produce a strict partitioning. Primarily it produces this plot, and in many situations you will actually want to visually inspect the plot. There are some methods to extract a hierarchical partitioning out of this plot, based on detecting "steep" areas.
The open source ELKI data mining framework (package "elki" in Debian and Ubuntu) has a very fast and flexible implementation of both algorithms. I've benchmarked this against GNU R ("fpc" package") and Weka, and the difference is enormous. ELKI without index support runs in roughly 11 minutes, with index down to 2 minutes for DBSCAN and 3 minutes for OPTICS. Weka takes 11 hours and GNU R/fpc takes 100 minutes (DBSCAN, no OPTICS available). And the implementation of OPTICS in Weka is not even complete (it does not support proper cluster extraction from the plot). Many of the other OPTICS implementations you can find with Google (e.g. in Python or MATLAB) seem to be based on this Weka version ...
ELKI is open source. So if you want to peek at the code, here are direct links: DBSCAN.java, OPTICS.java.
Some part of the code may be a bit confusing at first. The "Parameterizer" classes serve the purpose of allowing automatic UI generation, for example. So there is quite a bit of meta code involved.
Plus, ELKI is quite extensively optimized. For example, it does not use Java Collections much anymore. Java Iterators, for example, require returning an object on next();. The C++ style iterators used by ELKI can have multiple values, and primitive values.
for(DBIDIter id = relation.iterDBIDs(); id.valid(); id.advance())
is a typical for loop in ELKI, iterating over all objects of a relation, but the whole loop requires creating (and GC'ing) a single object. And actually, this is as literal as a for loop can get.
ModifiableDBIDs processedIDs = DBIDUtil.newHashSet(size);
is another example. Essentially, this is like a HashSet<DBID>. Except that it is a lot faster, because the object IDs do not need to live a Java objects, but can internally be stored more efficiently (the only currently available implementation of the DBID layer uses primitive integers).
Java advocates always accuse you of premature optimization when you avoid creating objects for primitives. Yet, in all my benchmarking, I have seen this continuously to have a major impact how many objects you allocate. At least when it is inside a loop that is heavily used. Java collections with boxed primitives just eat a lot of memory, and the memory management overhead does often make a huge difference. Which is why libraries such as Trove (which ELKI uses a lot) exist. Because memory usage does make a difference.
(Avoiding boxing/unboxing systematically in ELKI yielded approximately a 4x speedup. But obviously, ELKI involves a lot of numerical computations.)

Bastian Venthur: Introducing The Art of Asking

Since October 2011 my flatmate and I where quite busy realizing a little pet project of ours called The Art of Asking. The ultimate goal is to visualize the world s opinion in an intuitive fashion and make it easy for everyone to play around with the data. The idea behind The Art of Asking is that users submit interesting questions which are answered by users around the world. But instead of showing only the boring result, we want to provide interesting insights and statistics about the answers given. For now the users can see the results of the question visualized by geographical regions. For example on the page for the question How are you today? you can see the interactive map with the pie chart. The map shows the average/dominating answer for each continent encoded by color, and the pie chart the distribution of the different answers for that region. If you move the mouse over a continent the pie chart updates and shows the distribution of answers for that continent. You can also click to zoom into the map to see the same for countries and regions. This allows you to investigate how the answers are distributed around the world. This is already quite nice and fancy to play around with, but of course we want much more. Right now we re working on a feature which will users allow to combine two arbitrary questions and see how the answers are related. This doesn t sound like much, but it is very addictive to crawl through the list of questions and find interesting correlations. Here is a little plot how it could look like (the data is from the actual data base of answers). For each possible combination of answers it shows the percentage of people who answered in that combination. Of course those plots only make sense when enough users answered for both questions you want to compare which is right now not very often the case, so I guess we ll roll out that feature sometime later when we have more data. But we have lots of ideas and are working on further ways to investigate the answers. Obviously one low hanging fruit for example would be the distribution of the answers over time. We ve been working for over a year now on this project in our spare time and we built it more or less from scratch. I wrote a small WSGI framework in Python and on top of that the WSGI application which runs the site. We use MongoDB for the storage of the data, uWSGI and nginx for the server, Jinja2 and Bootstrap for the HTML and D3.js for the visualization of the data, where Maci did a wonderful job realizing the interactive map and charts. We re running this site since July 2012 now and are already quite satisfied with the number of users, and the quality of the questions. But of course we could always use more (especially more answers). So if you want to try it out, go to theartofasking.com and fill out the blank spots on the map! We re happy about every answer and question we can get and are eager to hear your suggestions.

31 October 2012

Yves-Alexis Perez: Update on OpenPGPv2 smartcards

After some feedback from other people, I have an important update to make on my last post. As I said, what decided me to eventually buy an OpenPGP smartcard was that it supported 4096 bit keys, so it would fit my 4096R/71EF0BA8 key. In the end, it seems it's a little more complicated than that. 4096R keys are indeed supported, as far as signing and authentication are concerned. But encryption keys seem limited to 3072 bits (or maybe more, I didn't test toroughly). When trying to decrypt some stuff encrypted for a 4096R key on the smartcard, gpg fails with some general error . It has already been reported here, but no news since. In my case, it's not that bad, I decided to go for 2048R for all subkeys. But if you desperately need 4096 bit encryption key, OpenPGPv2 smartcards might not be the right solution for you. I have no idea if the problem lies in GnuPG or in the smartcard, and I can't really find much information on this.

Mirco Bauer: Mono 3.0 Preview Packages for Debian and Ubuntu

Just after the Mono 3.0 release was announced I started working on Debian packages and after a couple of days of hard work I already have some preview packages for you. The preview packages are currently limited to the AMD64 architecture but can be installed on Debian/Testing (Wheezy), Debian/Unstable (Sid) and Ubuntu/Quantal (12.10). After these preview packages have received some successful feedback (please leave feedback via comments!), I will upload them to Debian/Experimental. In case you are shocked why I have packages in just 8 days after the official Mono 3.0 release, there are 3 factors that play a big role:
  1. New packaging workflow
    • Downstream patches are management in git branches
    • Release Early, Release Often: I no longer port and test every patch very carefully (especially with architecture specific patches), instead I port them with my best knowledge and upload them to debian.meebey.net and/or Debian/Experimental and use that as test ground. If regressions happen, let them happen, and make further uploads to address them.
  2. New packaging style
  3. Sponsored packaging
    • As my employer needs Mono packages for an upcoming software project, I was granted to work on Mono packages in worktime.
Install instructions OBLIGATORY WARNING: DO NOT USE THESE PREVIEW PACKAGES ON PRODUCTION SYSTEMS NOR BLAME ME IF YOUR MONO SOFTWARE NO LONGER WORKS! Add this line to your /etc/apt/sources.list file:
deb http://debian.meebey.net/experimental/mono /
Now update the APT database and install mono-complete from that repository:
apt-get update
apt-get install mono-complete

Raphael Geissert: rsync is not enough

Ask how can one create a Debian mirror and you will get a dozen different responses. Those who are used to mirroring content, whether it is distribution packages, software in general, documents, etc., will usually come up with one answer: rsync.

Truth is: for Debian's archive, rsync is not enough.

Due to the design of the archive, a single call to rsync leaves the mirror in an inconsistent state during most of the sync process.
Explanation is simple: index files are stored in dists/, package files are stored in pool/. Sort those directories by name (just like rsync does) and you get that the indexes will be updated before the actual packages are downloaded.

There are multiple scripts out there that do exactly that, one of them in Ubuntu's wiki. Plenty more if you search the web.

Now, addressing that issue shouldn't be so difficult, right? after all, all the index files are in dists/, so syncing in two stages should be enough. It's not that simple.

With the dists/ directory containing over 8.5GBs worth of indexes and, erm, installer files, even a two stages sync will usually leave the mirror in an inconsistent state for a while.

How about only deferring to the second stage the bare minimum?, I hear you ask.
That is the current approach, but it leads to some errors when new index files are added and used. The fact that people insist in writing their own scripts doesn't help.

Hopefully, some ideas like moving the installer stuff out of dists/ and
overhauling the repository layout are being considered. An alternative is to make the users of the mirrors more robust and fault-tolerant, but we would be talking about tenths if not hundreds of tools that would need to be improved.

In all cases, the one script that is actively maintained, is rather portable, and improved from time to time is the ftpsync script. Please, do yourself and your users a favour: don't attempt to reinvent the wheel (and forget about calling rsync just once).

Russell Coker: Links October 2012

The F Word has an informative post about men commenting on Feminist blogs [1]. Most of it applies to any situation where a member of a powerful group comments on an issue related to a minority group. Near the end they say: I ll also paraphrase and flesh out the most useful piece of advice I ever read, when I made the effort to research white privilege: don t expect the minority to trust you. Trust is earned, and you re just another commenter who they can t tell apart from any other commenter. You re entering someone else s space, where different rules apply. You get to have the rest of the world for people to assume you re a wonderful person. Here, you re just another one of them , and given the track record of them , it s up to you to listen, learn and prove that you re being thoughtful and honestly trying to examine your privilege. Sociological Images has an interesting article on people s perception of the sky colour it seems that the blue sky meme is modern [2]. Charles Stross wrote an interesting article about the possible uses for future low power computers [3]. He gets a bit over-excited about the possibilities for sensing making a tiny computer that can sense so many things isn t going to be easy (DSLRs are big for a reason). But having lots of powerful computers everywhere does provide lots of interesting and potentially bad opportunities. Martin Bekkelund has an interesting article about Amazon wiping DRM infected books that it had sold to a customer without giving a refund or an explanation [4]. If you want to buy ebooks it seems that the sensible thing to do would be to immediately crack them or download them from The Pirate Bay so Amazon can t steal them back. Rebecca Saxe gave an interesting talk about trying to develop methods for conflict resolution through neuroscience [5]. Barry Eisler has written an interesting article about the corruption of journalists [6]. It s really worth reading, some of the methods of corruption apply to even the more casual bloggers. Krebs on Security has an informative article about the Microsoft Tech Support phone scams [7]

Mark Brown: UK landline non-security (and Orange clue)

Yesterday when I got in from work I got my second letter in as many months through from BT saying that my account was being closed as my landline was being transferred to another provider. This was the first I d heard of this and it causes a cancellation charge so I called to complain; the first time I did this they said they couldn t tell me anything about who the line had been transferred to. They did tell me that there was no equivalent of PAC or MAC for landlines and that the only thing stopping this happening is the two week delay in implementing. This time BT felt able to tell me that the line had been transferred to Orange so I phoned Orange. Orange told me that the phone number had indeed been transferred to them in the name of someone else. They also said that they had no intention of attempting to carry out any authentication that lines being transferred to them are owned by the person they re being transferred to I explicitly asked them if anyone could just do this for any phone number and they confirmed that this is indeed the case. BT claim they can t block transfers for regulatory reasons; Orange claim this is possible and that I should just do that. I ve asked Orange to put a note on the account (which was the best they claimed they could do) and complained to OFCOM (who won t really talk to me without a formal escalation from the phone providers) but none of this really helps given the gaping security holes in the system. You really should need more information than just the phone number itself to transfer a number. On the slightly bright side I still appear to have phone service; presumably currently paid for by whoever is initiating the transfers.

30 October 2012

Tanguy Ortolo: Supporting GNU MediaGoblin

Gavroche, a purple goblin, holding a poster in his arms and giving it a hug You certainly know Picasa Albums Web and Flickr, two commercial picture hosting services. You probably know a few free software projects that allow people to keep control of their own pictures, such as Gallery, Piwigo, Zenphoto or PhotoShow. GNU MediaGoblin is a similar project, but with some interesting characteristics and features: If you find GNU MediaGoblin as promising as I do, please consider donating!

Next.

Previous.