Search Results: "ron"

3 August 2013

Russell Coker: Religious Conversion

Today I was in the center of Melbourne and I saw two stands run by Muslims advocating their faith. They had a selection of DVDs and brochures to give away and seemed friendly enough for anyone who wanted a chat. I think that this is a good thing, if the majority of the Australian population would think of the Muslim minority as people who give away DVDs rather than as potential suicide bombers then it would be a great benefit for society. Then I saw a Christian stand which had placards indicating that Jesus loves Muslims, which is fine. One of the workers on that stand then gleefully told me that there had been a Muslim stand in that area but the Muslims departed when they arrived. It would be good if advocates for the various religions could work together to promote positive things that they all agree on (generally related to being nice to other people). When they seem to be essentially bullying other religious groups into leaving the area they aren t making a good case for a kind and caring god. The Christian who was so happy about the Muslims departing then tried to convert me. He started by talking about a mobile phone not being able to appear from nothing which was the start of an utterly stupid argument about creationism. That was stupid firstly because it s obvious that evolution doesn t apply to consumer electronics, if I leave a phone lying on top of another phone overnight I m not going to get some little phones appearing as a result. But the bigger stupidity is in even promoting creationism in the first place. Advocating creationism is essentially claiming that god is so incapable that ze can t manage an evolutionary process. An all-seeing all-knowing god of infinite intelligence could just create a plasma cloud that will form into a solar system and evolve life. People who argue for creationism don t believe in a capable god, they believe in a being that resembles a god in the way a magician resembles someone who can actually perform magic. Arguing for creationism is arguing for a limited being which probably isn t worthy of worship (why worship space aliens who are probably only a few thousand years ahead of us). Regardless of the scientific evidence (which is strongly in favor of evolution) just logically thinking about the issue in regard to what a being with infinite powers might actually do suggests that ze could just as easily create or evolve all life and probably wouldn t care about what we believe. Genesis should be taken as metaphor because it s obvious that nothing that s taught in a year 7 science class could be taught to your typical stone age goat-herd, but even if it was taken literally it s not specified as a required belief (as opposed to a belief in a single god, repenting sins, and other beliefs which are required). The greatest stupidity however is the fact that even if it made any sense to promote creationism it wouldn t make sense to do so immediately. Any time you want to convert someone to a set of beliefs it s a good strategy to start with the ones which will seem least wacky. I can have have a conversation with Christians about positive things which we generally agree on such as donating to charity and trying to do good things in various ways. But when they start with the stupid stuff I walk away. When the weather is better I plan to interview representatives of the various religious groups who are trying to convert people on the street and rate them according to how wacky they are. I ll be a little limited in my ability to review them by the total lack of any right to free speech in Australia (our laws are much the same as those in the UK). But I should still be able to give them a reasonable review.

Benjamin Mako Hill: Doctor of Philosophy

On Wednesday, I successfully defended my PhD dissertation in front of a ridiculously packed house at the MIT Media Lab. I am humbled by the support shown by the MIT Sloan, Media Lab, and Harvard communities. Earlier today, I finished up paperwork and submitted my archival copies. I m done. Although I ve often heard PhDs described as emotional roller coasters, I feel enormously blessed in that I honestly can t relate. My eight years at MIT and Harvard have been almost universally positive and I have learned and grown indescribably. As excited as I am about my next chapter at the University of Washington, I m going to miss my life here. Deeply. My dissertation was three essays on volunteer mobilization in peer production. Once I have a chance to catch up and recover, I ll be posting the previously unpublished pieces. The Remixing Dilemma was included in the dissertation and is already online. The Media Lab AV team shot professional video of the talk. When I get a copy of the video, I ll post that too. But because I think it s important, I ve formatted and published the acknowledgments section of the dissertation today. Although there are too many folks to thank, I ve highlighted the contributions of my co-authors, and friends, Aaron Shaw and Andr s Monroy Hern ndez and my almost unbelievably incredible group of advisors: Eric von Hippel, Yochai Benkler, Mitch Resnick, and Tom Malone.

2 August 2013

Petter Reinholdtsen: 90 percent done with the Norwegian draft translation of Free Culture

It has been a while since my last update. Since last summer, I have worked on a Norwegian docbook version of the 2004 book Free Culture by Lawrence Lessig, to get a Norwegian text explaining the problems with the copyright law. Yesterday, I finally broken the 90% mark, when counting the number of strings to translate. Due to real life constraints, I have not had time to work on it since March, but when the summer broke out, I found time to work on it again. Still lots of work left, but the first draft is nearing completion. I created a graph to show the progress of the translation:

When the first draft is done, the translated text need to be proof read, and the remaining formatting problems with images and SVG drawings need to be fixed. There are probably also some index entries missing that need to be added. This can be done by comparing the index entries listed in the SiSU version of the book, or comparing the English docbook version with the paper version. Last, the colophon page with ISBN numbers etc need to be wrapped up before the release is done. I should also figure out how to get correct Norwegian sorting of the index pages. All docbook tools I have tried so far (xmlto, docbook-xsl, dblatex) get the order of symbols and the special Norwegian letters wrong. There is still need for translators and people with docbook knowledge, to be able to get a good looking book (I still struggle with dblatex, xmlto and docbook-xsl) as well as to do the draft translation and proof reading. And I would like the figures to be redrawn as SVGs to make it easy to translate them. Any SVG master around? There are also some legal terms that are unfamiliar to me. If you want to help, please get in touch with me, and check out the project files currently available from github. If you are curious what the translated book currently look like, the updated PDF and EPUB are published on github. The HTML version is published as well, but github hand it out with MIME type text/plain, confusing browsers, so I saw no point in linking to that version.

1 August 2013

Julien Danjou: The definitive guide on how to use static, class or abstract methods in Python

Doing code reviews is a great way to discover things that people might struggle to comprehend. While proof-reading OpenStack patches recently, I spotted that people were not using correctly the various decorators Python provides for methods. So here's my attempt at providing me a link to send them to in my next code reviews. :-) How methods work in Python A method is a function that is stored as a class attribute. You can declare and access such a function this way:

>>> class Pizza(object):
...     def __init__(self, size):
...         self.size = size
...     def get_size(self):
...         return self.size
...
>>> Pizza.get_size
<unbound method Pizza.get_size>

What Python tells you here, is that the attribute get_size of the class Pizza is a method that is unbound. What does this mean? We'll know as soon as we'll try to call it:

>>> Pizza.get_size()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unbound method get_size() must be called with Pizza instance as first argument (got nothing instead)

We can't call it because it's not bound to any instance of Pizza. And a method wants an instance as its first argument (in Python 2 it must be an instance of that class; in Python 3 it could be anything). Let's try to do that then:

>>> Pizza.get_size(Pizza(42))
42

It worked! We called the method with an instance as its first argument, so everything's fine. But you will agree with me if I say this is not a very handy way to call methods; we have to refer to the class each time we want to call a method. And if we don't know what class is our object, this is not going to work for very long. So what Python does for us, is that it binds all the methods from the class Pizza to any instance of this class. This means that the attribute get_size of an instance of Pizza is a bound method: a method for which the first argument will be the instance itself.

>>> Pizza(42).get_size
<bound method Pizza.get_size of <__main__.Pizza object at 0x7f3138827910>>
>>> Pizza(42).get_size()
42

As expected, we don't have to provide any argument to get_size, since it's bound, its self argument is automatically set to our Pizza instance. Here's a even better proof of that:

>>> m = Pizza(42).get_size
>>> m()
42

Indeed, you don't even have to keep a reference to your Pizza object. Its method is bound to the object, so the method is sufficient to itself. But what if you wanted to know which object this bound method is bound to? Here's a little trick:

>>> m = Pizza(42).get_size
>>> m.__self__
<__main__.Pizza object at 0x7f3138827910>
>>> # You could guess, look at this:
...
>>> m == m.__self__.get_size
True

Obviously, we still have a reference to our object, and we can find it back if we want. In Python 3, the functions attached to a class are not considered as unbound method anymore, but as simple functions, that are bound to an object if required. So the principle stays the same, the model is just simplified.

>>> class Pizza(object):
...     def __init__(self, size):
...         self.size = size
...     def get_size(self):
...         return self.size
...
>>> Pizza.get_size
<function Pizza.get_size at 0x7f307f984dd0>

Static methods Static methods are a special case of methods. Sometimes, you'll write code that belongs to a class, but that doesn't use the object itself at all. For example:

class Pizza(object):
    @staticmethod
    def mix_ingredients(x, y):
        return x + y
 
    def cook(self):
        return self.mix_ingredients(self.cheese, self.vegetables)

In such a case, writing mix_ingredients as a non-static method would work too, but it would provide it a self argument that would not be used. Here, the decorator @staticmethod buys us several things:

Python doesn't have to instantiate a bound-method for each Pizza object we instiantiate. Bound methods are objects too, and creating them has a cost. Having a static method avoids that:

>>> Pizza().cook is Pizza().cook
False
>>> Pizza().mix_ingredients is Pizza.mix_ingredients
True
>>> Pizza().mix_ingredients is Pizza().mix_ingredients
True

It eases the readability of the code: seeing @staticmethod, we know that the method does not depend on the state of object itself;
It allows us to override the mix_ingredients method in a subclass. If we used a function mix_ingredients defined at the top-level of our module, a class inheriting from Pizza wouln't be able to change the way we mix ingredients for our pizza without overriding cook itself.

Class methods Having said that, what are class methods? Class methods are methods that are not bound to an object, but to a class!

>>> class Pizza(object):
...     radius = 42
...     @classmethod
...     def get_radius(cls):
...         return cls.radius
... 
>>> 
>>> Pizza.get_radius
<bound method type.get_radius of <class '__main__.Pizza'>>
>>> Pizza().get_radius
<bound method type.get_radius of <class '__main__.Pizza'>>
>>> Pizza.get_radius is Pizza().get_radius
True
>>> Pizza.get_radius()
42

Whatever the way you use to access this method, it will be always bound to the class it is attached too, and its first argument will be the class itself (remember that classes are objects too). When to use this kind of methods? Well class methods are mostly useful for two types of methods:

Factory methods, that are used to create an instance for a class using for example some sort of pre-processing. If we use a @staticmethod instead, we would have to hardcode the Pizza class name in our function, making any class inheriting from Pizza unable to use our factory for its own use.

class Pizza(object):
    def __init__(self, ingredients):
        self.ingredients = ingredients
 
    @classmethod
    def from_fridge(cls, fridge):
        return cls(fridge.get_cheese() + fridge.get_vegetables())

Static methods calling static methods: if you split a static methods in several static methods, you shouldn't hard-code the class name but use class methods. Using this way to declare ou method, the Pizza name is never directly referenced and inheritance and method overriding will work flawlessly

class Pizza(object):
    def __init__(self, radius, height):
        self.radius = radius
        self.height = height
 
    @staticmethod
    def compute_circumference(radius):
         return math.pi * (radius ** 2)
 
    @classmethod
    def compute_volume(cls, height, radius):
         return height * cls.compute_circumference(radius)
 
    def get_volume(self):
        return self.compute_volume(self.height, self.radius)

Abstract methods An abstract method is a method defined in a base class, but that may not provide any implementation. In Java, it would describe the methods of an interface. So the simplest way to write an abstract method in Python is:

class Pizza(object):
    def get_radius(self):
        raise NotImplementedError

Any class inheriting from Pizza should implement and override the get_radius method, otherwise an exception would be raised. This particular way of implementing abstract method has a drawback. If you write a class that inherits from Pizza and forget to implement get_radius, the error will only be raised when you'll try to use that method.

>>> Pizza()
<__main__.Pizza object at 0x7fb747353d90>
>>> Pizza().get_radius()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in get_radius
NotImplementedError

There's a way to triggers this way earlier, when the object is being instantiated, using the abc module that's provided with Python.

import abc
 
class BasePizza(object):
    __metaclass__  = abc.ABCMeta
 
    @abc.abstractmethod
    def get_radius(self):
         """Method that should do something."""

Using abc and its special class, as soon as you'll try to instantiate BasePizza or any class inheriting from it, you'll get a TypeError.

>>> BasePizza()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class BasePizza with abstract methods get_radius

Mixing static, class and abstract methods When building classes and inheritances, the time will come where you will have to mix all these methods decorators. So here's some tips about it. Keep in mind that declaring a class as being abstract, doesn't freeze the prototype of that method. That means that it must be implemented, but i can be implemented with any argument list.

import abc
 
class BasePizza(object):
    __metaclass__  = abc.ABCMeta
 
    @abc.abstractmethod
    def get_ingredients(self):
         """Returns the ingredient list."""
 
class Calzone(BasePizza):
    def get_ingredients(self, with_egg=False):
        egg = Egg() if with_egg else None
        return self.ingredients + egg

This is valid, since Calzone fulfil the interface requirement we defined for BasePizza objects. That means that we could also implement it as being a class or a static method, for example:

import abc
 
class BasePizza(object):
    __metaclass__  = abc.ABCMeta
 
    @abc.abstractmethod
    def get_ingredients(self):
         """Returns the ingredient list."""
 
class DietPizza(BasePizza):
    @staticmethod
    def get_ingredients():
        return None

This is also correct and fulfil the contract we have with our abstract BasePizza class. The fact that the get_ingredients method don't need to know about the object to return result is an implementation detail, not a criteria to have our contract fulfilled. Therefore, you can't force an implementation of your abstract method to be a regular, class or static method, and arguably you shouldn't. Starting with Python 3 (this won't work as you would expect in Python 2, see issue5867), it's now possible to use the @staticmethod and @classmethod decorators on top of @abstractmethod:

import abc
 
class BasePizza(object):
    __metaclass__  = abc.ABCMeta
 
    ingredient = ['cheese']
 
    @classmethod
    @abc.abstractmethod
    def get_ingredients(cls):
         """Returns the ingredient list."""
         return cls.ingredients

Don't misread this: if you think this going to force your subclasses to implement get_ingredients as a class method, you are wrong. This simply implies that your implementation of get_ingredients in the BasePizza class is a class method. An implementation in an abstract method? Yes! In Python, contrary to methods in Java interfaces, you can have code in your abstract methods and call it via super():

import abc
 
class BasePizza(object):
    __metaclass__  = abc.ABCMeta
 
    default_ingredients = ['cheese']
 
    @classmethod
    @abc.abstractmethod
    def get_ingredients(cls):
         """Returns the ingredient list."""
         return cls.default_ingredients
 
class DietPizza(BasePizza):
    def get_ingredients(self):
        return ['egg'] + super(DietPizza, self).get_ingredients()

In such a case, every pizza you will build by inheriting from BasePizza will have to override the get_ingredients method, but will be able to use the default mechanism to get the ingredient list by using super().

Russ Allbery: Review: And Playing the Role of Herself

Review: And Playing the Role of Herself, by K.E. Lane

Publisher:	Regal Crest
Copyright:	2006
Printing:	June 2010
ISBN:	1-932300-72-4
Format:	Kindle
Pages:	336

Caidence Harris is an actress who has finally gotten her big shot: a starring role in the hit police drama 9th Precinct. She plays half of a police partnership; the other half is played by her best friend, the considerably more famous Liz Stokley. She's quite happy with her life and her work, and only tentatively pondering what to do about coming out (at least to herself) as a lesbian. But matters get considerably more complicated when she starts falling for Robyn Ward, a star in a parallel legal drama. And the very high-profile girlfriend of one of the best male tennis players in the world. Yes, another lesbian romance. This one has won several awards, including the highest recommendation from my friend who's been prefiltering the genre for me and pointing out only the best stories. And despite being set firmly among the rich and famous and involving TV dramas with the serial numbers filed off, he was right. This is a thoroughly enjoyable story. One of its strongest parts is how Lane develops the romance between Caidence and Robyn. There is a bit of the inevitable physical chemistry so common in romances (I still don't believe kisses work that way), and a few descriptions of physical beauty (after all, there's a built-in excuse for everyone to be TV-star beautiful). But Lane avoids putting either at the center of the relationship. Instead, this is a romance built on trust, tentative at first and then growing but rocky. That speaks to me a lot more than all the descriptions of physical chemistry. For example, we don't see Caidence and Robyn meet for the first time in the novel, as is usually the case. They've been around each other before, with Caidence acting rather tongue-tied and a bit star-struck and Robyn thinking she's rather silly. The story begins when, for whatever reason, Caidence finally manages to relax and become genuine, and then both of them become closer as they find themselves lowering barriers around each other that they normally don't lower. Lane does an excellent job using a few key facial expressions as symbols and hallmarks of that, letting the reader follow the way that trust builds through the story. The other great element of this book is Caidence's friend Liz. She could have been a stereotype of a movie star: drop-dead gorgeous, a new boyfriend each week, but not particularly intelligent. That's how she comes off at first. But as the story progresses, Lane shows more and more of their friendship, and the reader gets to see just how much Liz's first impression is a combination of an act and her unapologetic tendency to do what she wants. Underneath, she's a strong female character and an excellent, if meddling friend; by the end of the book, she was in the running for my favorite character. This is characterization of a quality that I rarely see in any genre of novel. Given that it's a romance that I enjoyed, you can correctly expect there not to be a lot of obnoxious communication breakdowns or chapters full of the main characters refusing to communicate with each other. There are, of course, a wide variety of barriers in the way of a relationship for the main characters, not the least of which being that neither of them was looking for something serious. But while they both do occasionally thoughtless or hurtful things, it's always clear just why they were thoughtless, or where their buttons were accidentally pressed, and few of the character actions are outright stupid. And, delightfully, Lane structures the story so that there are multiple, incremental payoffs. One can not only enjoy the characters getting together in the end, as they always do in this genre, but a lot of important and meaningful steps along the way. Lane does a great job putting the reader in Caidence's head, helped by first-person narration, so that one can feel just how much some moments mean to her emotionally. Add in several other fleshed-out and interesting supporting characters, some incredible architecture that was fun to imagine, and rather more in the way of plot twists and dramatic moments than I usually expect in a romance (and although the book worried me once, none of them go to the dark places that would have been less fun), and one gets a novel that kept me engrossed from start to finish. I'm tempted to read it again, and that's rare for me for a romance novel. If you're in the mood for a heart-warming story about intelligent people learning how to be together, I highly recommend this book. It's a great book to read as a stress relief when you want a story that you know will turn out "right." Rating: 9 out of 10

31 July 2013

Richard Hartmann: vcsh v1.20130724

vcsh has seen a lot of activity in recent times and today's release of vcsh 1.20130724 contains a lot of useful new features. In case you haven't tried vcsh yet, or not recently, now is a very good time to do so. vcsh has been trivial to use for at least a year now, but the documentation did a very poor job of exposing that fact. Which is why there's now a 30 second howto which will get you up to speed and commiting in no time. The rest of the docs have undergone a major rewrite, as well. Feature-wise, there's

vcsh pull, which pulls from all vcsh remotes
vcsh push, which pushes to all vcsh remotes
vcsh status, which lists the status of all files checked into all vcsh repositories
clone hooks, including one with the vcsh environment tore down. This allows you to operate on other repositories, e.g. clone another repository
$VCSH_GITIGNORE == none, which allows you to avoid writing gitignores automagically
comprehensive zsh completion, which will complete on pretty much anything you could wish for
several bug fixes, increased robustness, and more comments

If vcsh pull, vcsh push, and vcsh status sound a bit like a poor man's myrepos (formerly know as mr), that's by design. On the one hand, this enables vcsh to handle pratically all aspects of configuration management. One tool for everything On the other hand, even if you use myrepos (like I still do and will continue to do), this gives you a quick and cheap way to operate on configuration repositories, only. Especially if you're in a hurry or on a cellular connection, you may not want to pull in all changes from all remotes. If the Linux kernel or other large projects are part of your normal myrepos setup, updating them all takes time and bandwidth. Contrary to that, configuration repositories tend to be lightweight. Updating those selectively simply makes sense. Debian unstable carries the current package, Homebrew should carry it soon. Arch AUR and Fedora are getting there. And as vcsh is written in POSIX shell, it does not need to be compiled but can be run directly from a git clone without the need for installation. Did I mention that this is the perfect time to try out vcsh? ;)

Raphael Geissert: Ten years-old ebook reader

I own a ten years-old ebook reader.

Is it a smartphone?
It has features that make it pretty much like one, except that it can't make phone calls. So no. (However, I dare to say that making phone calls is one of the least used features of smartphones nowadays. Perhaps in the feature smartphones won't even be phones anymore, due to atrophy.)

What is it then?
It is a Sony CLI , a PEG-SJ22 to be more precise. A PDA running Palm OS 4.1 that I'm now using as an ebook reader. To my surprise, the Plucker and iSilo readers still exist and at least the latter seems somewhat alive - there is even a version for Android.

Nowadays there are ebook readers with electronic paper displays, wifi or 3G connectivity, and other features but they all come down to the same thing: an ebook reader. Truth be told, the technology is quite old. In 2004, a year after the release of the PEG-SJ22, Sony also released the LIBRI EBR-1000EP in Japan, an ebook reader with a 6" electronic paper display. A few years later is was released in the US as the Sony Reader.

Certainly, there have been advances since those devices were first released, but I have yet to see something that is really innovating. This year we are back to wrist watches, which were first released more than ten years ago - and one of them even ran linux.

Netbooks also had devices like the PEG-UX50 as their ancestor.

And if you were thinking about shoes, you've arrived late: the Puma RS already had some chips in them, and the Adidas 1 were also sported a few years ago. World, it's time to innovate.

Russell Coker: Links July 2013

Wayne Mcgregor gave an interesting TED talk about the creative processes of a choreographer [1]. The dancing in this talk is really good. Melissa McEwan wrote an interesting article on whether being an ally to members of a disadvantaged group is a state or a process [2]. It seems to me that the word ally is a problem here, maybe a word like supporter would be more useful. Ken Murray wrote an insightful article How Doctors Die about the end of life choices that people with medical experience make [3]. He makes a good case for rejecting the type of treatment which has a low probability of success and a certainty of lowering the quality of life. It would be good if health insurance offered patients with terminal illness an option of $1000 per day party funds if they chose to reject the expensive and painful methods that might extend their life, that might even save enough money to allow cheaper health insurance! Rick Falkvinge wrote an interesting post about the copyright to translations of the Bible [4]. I used to think that copyright issues with religious works was only a problem with cults Joshua Foer wrote an interesting article for the New Yorker about the invention of the language Quijada which is designed for maximum precision [5]. It also has a lot of background information on constructed languages and the way that they are used.

29 July 2013

Thomas Goirand: OpenStack Havana b2 available, openstack-debian-images approved

I have finished preparing the beta 2 of the next release of OpenStack. It is currently only available from out Git on Alioth (in /git/openstack), and directly from my jenkins repository, which creates Wheezy backports for it:

deb ftp://havana.pkgs.enovance.com/debian havana main
deb http://archive.gplhost.com/debian havana-backports main

As for every OpenStack release, a large number of Python modules needed to be packaged and are waiting in the FTP master NEW queue to be approved: oslo-sphinx, python-django-discover-runner, python-hacking, python-jsonrpclib, python-lesscpy, python-neutronclient, python-nosehtmloutput, python-requestbuilder, python-termcolor, sphinxcontrib-httpdomain and sphinxcontrib-pecanwsme. Let s hope that they will be approved before the next beta release in september (time where OpenStack Havana will be in feature freeze). The total number of packages maintained by the OpenStack team (on which I really am the only active maintainer for the moment ), there s 53 packages, plus these 11 packages waiting in the NEW queue. That s a big number of package, and I wouldn t mind some help One thing that annoyed all the community is that Quantum, the OpenStack network virtualization module, had to be renamed as Neutron, because of a trademark from Quantum (you probably remember the Quantum Fireball hard drives? Well, it s the same company ). Another good news is that my openstack-debian-images package has just been approved and landed in Sid. With it, you can automate the process of building a Debian image for OpenStack with a simple shell command (there s a man page that I wrote with it: read it if you need to build images). It is made of a single shell script to build the image, using kpartx, parted, mbr, debootstrap, extlinux and friends. I tried to keep it simple, not involving a huge amount of components. With the release of cloud-init 0.7.2-3, I have fixed a few bugs (3 important bugs, out of which 2 where RC bugs), thanks to contributions in the debian-cloud@lists.debian.org mailing list. This includes adding new init.d scripts, so we now have support for user data. This doesn t only benefits OpenStack images, but anyone willing to start virtual machines in the cloud (nowadays, every cloud implementation needs cloud-init installed in the virtual images). This means you can include a script in the metadata of the virtual machine you start it, and it will be executed at startup. If everything goes as planned (that is, no new RC bug), I will upload an update of cloud-init to backports in 5 days (there is already a version there, but it doesn t have the necessary init.d scripts to execute the user data scripts), and openstack-debian-images in 9 days. Then it will be possible to build OpenStack images with absolutely all the tools available from Wheezy (and backports). I hope to be able to discuss that during Debconf 13.

28 July 2013

Daniel Pocock: This blog was slashdotted

My earlier blog about rigged elections was slashdotted over the weekend. I had an email from the slashdot team giving me advance warning, having heard the stories about other sites getting knocked offline, I immediately logged into my Ganglia console to start the tsunami watch. I then set about enabling a reverse proxy using Squid to accelerate web server performance. This took about 30 minutes to set up. I chose Squid 3 so I could use a logging format similar to Apache.

The wave hits A few hours after the warning email, details appeared on slashdot and sure enough, thousands of visitors started coming to this site. As detailed previously, the site is running on Debian servers with XCP. The web site is managed by Drupal/Apache/MySQL. Everything held together very well. Ganglia demonstrated an initial surge lasting about 10 minutes. From there, traffic has tapered off gradually over 24 hours, although it still remains well above average. slashdot comments One of the most interesting things about the slashdot community are the comments. Given that nobody really wanted to defend the indefensible and nothing in my blog surpasses some of what has been in the news recently (150 charges of fraud against a single MP), most of the discussion quickly turned to a comparison of broadband policies. For those not already aware of it, the Australian Government (Labor Party) is currently rolling out fibre to the home. The project is expected to take about 10 years, although it was delivered very quickly in certain areas where the Government needs votes. Opinions are divided: people who got it first think it is great, some people feel it is too expensive (it's much more expensive than a similar service in Switzerland, for example), some people feel it will be too slow when the final stages are completed in 10 years. It appears technically stronger than the solution proposed by the alternative party.

Francois Marier: FISL for foreigners HOWTO

FISL (pronounced FIZZ lay by the locals) is a large Free and Open Source software gathering in Porto Alegre, Brazil. While the primary audience of that conference is the Latin America "libre software" community, a number of overseas speakers also participate in that conference. This is my contribution to them: a short guide for international guests visiting the F rum Internacional do Software Livre.

Planning Before you fly out, make sure you look up the visa requirements for your country of citizenship. Many western countries will require a visa and you will need to visit the local Brazillian embassy ahead of time to get one. Next, have a look at the list of recommended immunizations. As with most destinations, it is recommended that your routine immunizations be up to date, but there are also other specialized ones such as Yellow Fever that are recommended by the Brazillian government. You should therefore visit a travel clinic a few weeks ahead of time. Other than that, I suggest reading up on the country and keeping an eye on the various travel advisories.

Arrival You will be flying at the Porto Alegre airport. If you need to exchange overseas money for Brazilian Reals, you can do that there. You'll probably also want to pick up a power adapter at the airport if you intend to charge your laptop while you're in the country Brazil has both 127V and 220V outlets using Type N sockets. Privacy note: using the free airport wifi will require giving your Passport details as part of the registration process.

Language If you don't speak Portuguese, expect a few challenges since most of the people you'll meet (including taxi drivers, many airport workers, some hotel staff) won't speak English. I highly recommend getting a phrase book before you leave and printing paper maps of where you are planning to go (to show to taxi drivers when you get lost). Native Spanish speakers seem to get by speaking Spanish to Portuguese speakers and understanding enough Portuguese to hold a conversation. I wouldn't count on it unless your Spanish is quite good though. Also, the official conference blog posts get eventually translated to English, but there is a delay, so you may want to subscribe to the Portuguese feed and use Google Translate to keep up with FISL news before you get there.

The conference FISL is a large conference and it has a very "decentralized" feel to it. From the outside, it looks like it's organized by an army of volunteers where everyone is taking care of some portion of it without a whole lot of top-down direction. It seems to work quite well! What this means for you as a foreign speaker however is that you're unlikely to be provided with a lot of information or help finding your way around the conference (i.e. no "speaker liaison"). There is a separate registration desk for speakers but that's about all of the attention you'll receive before you deliver your talk. So make sure you know where to go and show up in your assigned room early to speak with the person introducing you. If your talk is in English, it will be live-transated by an interpreter. It's therefore a good idea to speak a bit more slowly and to pause a bit more. Other than that, the organizers make an effort to schedule an English talk in each timeslot so non-Portuguese speakers should still be able to get a lot out of the conference. FISL was a lot of fun for me and I hope that some of these tips will help you enjoy the biggest FLOSS gathering in the southern hemisphere!

27 July 2013

Julien Danjou: OpenStack Ceilometer Havana-2 milestone released

Last week, the second milestone of the Havana developement branch of Ceilometer has been released and is now available for testing and download. This means the first half of the OpenStack Havana development has passed! New features Ten blueprints have been implemented as you can see on the release page. I'm going to talk through some of them here, that are the most interesting for users.

The Ceilometer API now returns all the samples sorted by timestamp. This blueprint is the first one implemented by Terri Yu, our OPW intern! In the same spirit, I've added the ability to limit the number of samples returned. On the alarming front, things evolved a lot. I've implemented the notifier system that will be used to run actions when alarms are triggered. To trigger these alarms, Eoghan Glynn (RedHat) worked on the alarm evaluation system that will use the Ceilometer API to check for alarm states. I've reworked the publisher system so it now uses URL formatted target for publication. That now allows to publish different meters to different target using the same publishing protocol (e.g. via UDP toward different hosts). Sandy Walsh (RackSpace) have been working on the StackTach like functionality and added the ability for the collector to optionally store the notification events received. Finally, Mehdi Abaakouk (eNovance) implemented a TTL system for the database, so you're now able to expire your data whenever you like. Bug fixes Thirty-five bugs were fixed, though most of them might not interest you so I won't elaborate too much on that. Go read the list if you are curious. Toward Havana 3 We now have 30 blueprints targeting the Ceilometer's third Havana milestone, with some of them are already started.I'll try to make sure we'll get there without too much trouble for the 6th September 2013. Stay tuned!

Petter Reinholdtsen: First beta release of Debian Edu/Skolelinux based on Debian Wheezy

The first wheezy based beta release of Debian Edu was wrapped up today. This is the release announcement: New features for Debian Edu 7.1+edu0~b0 released 2013-07-27 These are the release notes for for Debian Edu / Skolelinux 7.1+edu0~b0, based on Debian with codename "Wheezy". About Debian Edu and Skolelinux Debian Edu, also known as Skolelinux, is a Linux distribution based on Debian providing an out-of-the box environment of a completely configured school network. Immediately after installation a school server running all services needed for a school network is set up just waiting for users and machines being added via GOsa , a comfortable Web-UI. A netbooting environment is prepared using PXE, so after initial installation of the main server from CD, DVD or USB stick all other machines can be installed via the network. The provided school server provides LDAP database and Kerberos authentication service, centralized home directories, DHCP server, web proxy and many other services. The desktop contains more than 60 educational software packages and more are available from the Debian archive, and schools can choose between KDE, Gnome, LXDE and Xfce desktop environment. This is the fifth test release based on Debian Wheezy. Basically this is an updated and slightly improved version compared to the Squeeze release. ALERT: Alpha based installations should reinstall or downgrade the versions of gosa and libpam-mklocaluser to the ones used in this beta release. Software updates

Switched roaming workstation profiles from wicd to network-manager for network configuration, as wicd didn't work any more.
Changed version numbers of patched gosa and libpam-mklocaluser packages to make sure our locally patched versions will be replaced by the official packages when they are released from Debian. Those installing alpha version need to reinstall or manually downgrade gosa and libpam-mklocaluser.
Added bluetooth tools to the default desktop (bluedevil, blueman).
Added tools for sharing the desktop on KDE (krdc, krfb).
Added valgrind to the default installation for easier debugging of crash bugs.

Other changes

Fixed artwork package to work with gnome, no longer break desktop=gnome installations.
Adjusted installer to now work when forced to use a proxy with the netinst CD.
Fixed code detecting and setting/loading hardware specific setup/firmware to work more robust out of the box.
Adjusted Kerberos setup to detect realm and server settings at install time instead of dynamically at run time. This avoid a crash with krb5-auth-dialog on diskless workstations without a DNS name.
Worked around misfeature in network-manager not calling the dhclient exit hooks, causing automatic proxy configuration and automatic host name setting at run time to work again.
Fixed feature setting the default Iceweasel start page from URL fetched from LDAP, to allow schools to set the global default by updating the dc=skole,dc=skolelinux,dc=no LDAP object.
Changed default host name on all networked machines to be unique (generated from MAC or reverse DNS) after boot.
Adjusted partition sizes to make sure they are big enough.

Known issues

Grub is missing the new artwork.
KDE fail to understand the wpad.dat file provided, causing it to not use the http proxy as it should.
Chromium also fail to use the proxy.

Where to get it To download the multiarch netinstall CD release you can use

ftp://ftp.skolelinux.org/skolelinux-cd/wheezy/debian-edu-7.1+edu0~b0-CD.iso
http://ftp.skolelinux.org/skolelinux-cd/wheezy/debian-edu-7.1+edu0~b0-CD.iso
rsync -avzP ftp.skolelinux.org::skolelinux-cd/wheezy/debian-edu-7.1+edu0~b0-CD.iso .

The MD5SUM of this image is: 55d5de9765b6dccd5d9ec33cf1a07109
The SHA1SUM of this image is: 996a1d9517740e4d627d100de2d12b23dd545a3f To download the multiarch USB stick ISO release you can use

ftp://ftp.skolelinux.org/skolelinux-cd/wheezy/debian-edu-7.1+edu0~b0-USB.iso
http://ftp.skolelinux.org/skolelinux-cd/wheezy/debian-edu-7.1+edu0~b0-USB.iso
rsync -avzP ftp.skolelinux.org::skolelinux-cd/wheezy/debian-edu-7.1+edu0~b0-USB.iso .

The MD5SUM of this image is: d8f0818c51a78d357de794066f289f69
The SHA1SUM of this image is: 49185ca354e8d0543240423746924f76a6cee733 How to report bugs http://wiki.debian.org/DebianEdu/HowTo/ReportBugs

Christian Perrier: Developers per country (July 2013)

This is time again for my annual report about the number of developers per country. This is now the fifth edition of this report, after the 2012, 2011, 2010 and 2009 editions. So, here we are with the July 2013 version, sorted by the ratio of *active* developers per million population for each country.

Act: number of active developers
Dev: total number of developers
A/M: number of active devels per million pop.
D/M: number of devels per million pop.
2009: rank in 2009
2010: rank in 2010
2011: rank in 2011 (June)
2011: rank in 2012 (June)
2013: rank now

Code	Name	Population	Act Dev	Dev	Act/Million	Dev/Million	2009	2010	June 2011	June 2012	July 2013
fi	Finland	5259250	19	31	3,61	5,89	1	1	1	1	1
ie	Ireland	4670976	12	15	2,57	3,21	13	9	6	2	2
ch	Switzerland	7870134	20	27	2,54	3,43	2	2	2	3	3
mq	Martinique	396404	1	1	2,52	2,52			3	4	4
se	Sweden	9088728	22	37	2,42	4,07	3	6	7	5	5
no	Norway	4973029	12	15	2,41	3,02	5	4	4	6	6
nz	New Zealand	4331600	10	15	2,31	3,46	4	3	5	7	7
de	Germany	81471834	165	235	2,03	2,88	7	7	9	9	8	*
lu	Luxemburg	503302	1	1	1,99	1,99	8	5	8	8	9	*
at	Austria	8217280	16	26	1,95	3,16	6	8	10	10	10
fr	France	65350000	100	130	1,53	1,99	12	12	11	11	11
au	Australia	22607571	31	59	1,37	2,61	9	10	12	12	12
be	Belgium	11071483	14	17	1,26	1,54	10	11	13	13	13
uk	United-Kingdom	62698362	73	114	1,16	1,82	14	14	14	14	14
nl	Netherlands	16728091	19	40	1,14	2,39	11	13	15	15	15
ca	Canada	33476688	32	61	0,96	1,82	15	15	17	16	16
dk	Denmark	5529888	5	10	0,9	1,81	17	17	16	17	17
es	Spain	46754784	35	56	0,75	1,2	16	16	19	18	18
it	Italy	59464644	36	51	0,61	0,86	23	22	22	19	19
cz	Czech Rep	10190213	6	6	0,59	0,59	21	20	21	21	20	*
lt	Lithuania	3535547	2	2	0,57	0,57	28	19	20	22	21	*
us	USA	313232044	173	378	0,55	1,21	19	21	25	24	22	*
il	Israel	7740900	4	6	0,52	0,78	24	24	24	25	23	*
hu	Hungary	10076062	5	10	0,5	0,99	18	25	26	20	24	*
hr	Croatia	4290612	2	2	0,47	0,47	20	18	18	26	25	*
lv	Latvia	2204708	1	1	0,45	0,45	26	26	27	27	26	*
bg	Bulgaria	7364570	3	3	0,41	0,41	25	23	23	23	27	*
uy	Uruguay	3477778	1	2	0,29	0,58	22	27	28	28	28
jp	Japan	127078679	35	50	0,28	0,39	30	28	29	29	29
pl	Poland	38441588	10	14	0,26	0,36	29	29	30	30	30
cr	Costa Rica	4301712	1	1	0,23	0,23	31	30	31	31	31
by	Belarus	9577552	2	2	0,21	0,21	35	36	39	39	32	*
sg	Singapore	5183700	1	1	0,19	0,19				33	33
pt	Portugal	10561614	2	4	0,19	0,38	27	32	32	34	34
gr	Greece	10787690	2	3	0,19	0,28	33	38	34	35	35
sk	Slovakia	5477038	1	1	0,18	0,18	32	31	33	36	36
ar	Argentina	40677348	6	8	0,15	0,2	34	33	35	32	37	*
rs	Serbia	7186862	1	1	0,14	0,14					38	*
tw	Taiwan	23040040	3	3	0,13	0,13	37	34	37	37	39	*
br	Brazil	192376496	21	23	0,11	0,12	36	35	38	38	40	*
cu	Cuba	11241161	1	1	0,09	0,09		38	41	41	41
kr	South Korea	48754657	4	6	0,08	0,12	39	39	42	42	42
gt	Guatemala	13824463	1	1	0,07	0,07					43	*
ve	Venezuela	28047938	2	2	0,07	0,07	40	45	50	49	44	*
ec	Ecuador	15007343	1	1	0,07	0,07		40	43	43	45	*
co	Colombia	45566856	3	4	0,07	0,09	41	44	46	47	46	*
cl	Chile	16746491	1	2	0,06	0,12	42	41	44	44	47	*
za	South Africa	50590000	3	10	0,06	0,2	38	48	48	48	48
ru	Russia	143030106	7	8	0,05	0,06	43	42	47	45	49	*
mg	Madagascar	21281844	1	1	0,05	0,05	44	37	40	40	50	*
ro	Romania	21904551	1	2	0,05	0,09	45	43	45	46	51	*
my	Malaysia	28250000	1	1	0,04	0,04			49	50	52	*
pe	Peru	29907003	1	1	0,03	0,03	46	46	51	51	53	*
tr	Turkey	74724269	2	2	0,03	0,03	47	47	52	52	54	*
ua	Ukraine	45134707	1	1	0,02	0,02	48	53	58	59	55	*
th	Thailand	66720153	1	2	0,01	0,03	50	50	54	54	56	*
eg	Egypt	80081093	1	3	0,01	0,04	51	51	55	55	57	*
mx	Mexico	112336538	1	1	0,01	0,01	49	49	53	53	58	*
cn	China	1344413526	10	14	0,01	0,01	53	53	57	56	59	*
in	India	1210193422	9	9	0,01	0,01	52	52	56	57	60	*
sv	El Salvador	7066403	0	1	0	0,14			36	58	61	*


			957	1535	62,35%

A few interesting facts:

Rankings are very stable: the first 7 countries are the same
Germany takes Luxemburg over with 10 more active developers than last year
Finland is still ranked 1st but loses two active developers. Still, it would need to lose 6 developers to lose rank 1
By losing one active developer, Hungary drops from 20th to 24th
Similar loss for Bulgaria, dropping from 23rd to 27th
Belarus doubles its number of developers from 1 to 2 and therefore jumps from rank 39 to 32
Serbia gains its first developer (indeed, the person registered with the "yu" ISO code and I arbitrarily decided that would count for Serbia.....please correct me if I'm wrong) and shows up at rank 38
Argentina sadly loses two active developers and rops to rank 37
Venezuela doubles its number of developers from 1 to 2 and slightly climbs up
Even though both of them gained one developer, the two most populated countries in the world, China and India, remain very low. China would indeed need 2730 developers to reach the ratio of Germany and India would need 2470.
Again, one new country : Guatemala. Let's take this as an achievement of another DebConf in Central America, last year.
Ukraine shows up again and I'm partly responsible for that as I processed Boris Pek's application as DD!
This year, DebConf will be hosted in one of the top-ranked countries, namely Switzerland. Guys, 9 more developers and you can overtake Finland!
We have 37 more active Debian developers and 45 more developers overall.
The ratio of active developers increases from 61.74% to 62.35%

26 July 2013

Justus Winter: Bootstrapping the Hurd

Due to popular demand this weeks report contains another awesome ascii screenshot. I added some lines to /etc/hurd/runsystem.sysv to inspect the environment very early in the boot process:

start ext2fs: Hurd server bootstrap: ext2fs[device:hd0s1] exec init proc auth
[...]
+ echo my pid is 1
my pid is 1
+ ps Ax
+ head
USER       PID TT STAT     TIME COMMAND
-            0  ? R<mo  0:00.01 /hurd/proc
root         1  - Sslow 0:00.02 /bin/sh /etc/hurd/runsystem
-            2  - Sp    0:00.00 /hurd/init root=device:hd0s1 console=com0
-            3  ? D<p   0:00.01 root=device:hd0s1 console=com0
-            4  - S<o   0:00.18 ext2fs --readonly --multiboot-command-line=root
-            5  - S<o   0:00.02 /hurd/exec
-            6  - S<o   0:00.00 /hurd/auth
root         7  - S<o   0:00.04 /hurd/term /dev/console device console
root         8  - S<o   0:00.00 /hurd/pflocal
+ pstree -p
sh(1)-+-init(2)-+-auth(6)
                 -ext2fs(4)-+-exec(5)
                             -pflocal(8)
                             -procfs(12)
                             -term(7)
       -pstree(11)
       -root=device:hd0s1(3)

Those who are familiar with the Hurd might note that /etc/hurd/runsystem.sysv runs as PID 1. At the end of this script /sbin/init is exec(2)uted, so sysvinit gets run as PID 1. This is the result of a fairly small patch series. It still took me like two days to come up with it since bootstrapping a Hurd system (or any microkernel OS, or like bootstrapping in general) is complicated. If you look at the screenshot above you will note the /hurd/pflocal process running with PID 8. This is the server providing pipe(2)s and Unix sockets. Before that server is started, one cannot use pipes to connect two processes with pipes like I did with ps and head (I actually lied a little here, because the way Debian/Hurd is set up, the pflocal server is started on demand courtesy of a Hurd feature called "passive translators", very similar to what the cool kids call "socket activation" today). Process 0 is the /hurd/proc server. It maps Mach tasks to the concept of Unix processes. Before that server is started, the notion of a process does not even exist. Pid 4 is the root filesystem. Without that, no other server can be read from the disk. But if the root filesystem is to be started first, it cannot be registered as a Unix process at the proc server, because that has not been started yet. So the first few processes are started in a weird way and order, and since they need to talk to each other, they need to register at each other. No Unix concepts exist yet, only Mach concepts like tasks and messages can be used. The boot sequence is roughly this (the numbers in parenthesis are the PIDs):

Grub loads GNU Mach(3), the statically linked root filesystem translator (4) and the /hurd/exec (5) server (the exec server loads executables into Mach tasks, implementing most parts of execve(2)).
The rootfs translator is started.
The rootfs initiates the Hurd server bootstrap by starting /hurd/exec and /hurd/init (2).
/hurd/init starts /hurd/proc (0) and /hurd/auth (6).
The translators perform several rendezvouses to introduce themselves to each other.
The proc servers state with respect to the already running processes is fixed.
/hurd/init starts /etc/hurd/runsystem (1) which starts the pager (that pages out memory to disk) and later execs /sbin/init.
From this point on the initialization of Debian/Hurd is no different from Debian/Linux.

Besides the pid one issue I spent my time on:

A prototypical patch for the other sysvinit related issue, /sbin/killall5 wreaking havoc on a Hurd system: http://lists.gnu.org/archive/html/bug-hurd/2013-07/msg00226.html
I updated my patch series binding the mtab translator to /proc/mounts: http://lists.gnu.org/archive/html/bug-hurd/2013-07/msg00216.html
I updated my patch series for daemonizing support for console-client: http://lists.gnu.org/archive/html/bug-hurd/2013-07/msg00246.html

Next week I hope to fully address the killall5 issue and to clean up, polish and submit my patch series for sysvinit. I will then revise my mtab translator prototype which is currently in its third iteration. As always, if there is time left, I will look at the network related issues.

25 July 2013

Michael Stapelberg: Looking for a USB WLAN stick

Posting this on behalf of a friend of mine in the hope that you can help: I ve failed several times now to find a suitable WLAN USB dongle that works out of the box on Debian testing. Often manufacturers change the chipsets without changing the version numbers, the product pages are incomplete or even state wrong information. I turn to you, in the hopes that someone can give me helpful pointers. The dongle should have:

USB Dongle, i.e. no external router and the like
5 GHz support and 802.11n ( agn )
work nicely on Debian testing (non-free is okay, but ndiswrapper or manually building modules isn t)
available in Europe/Germany

If you have any hints, please send them directly to breunig AT uni-hd DOT de. Thanks!

John Goerzen: Development Freedom in New Platforms

I started writing code when I was somewhere around 1st grade, hacking some rudimentary BASIC on a TRS-80. Since then, I ve used more programming languages than I can count; everything from C to Prolog, Python to Haskell, and probably a dozen others. I enjoy diversity in programming languages and tools. I ve had an interest in both web and mobile development for some time, but have done little with either. Reflecting on that lately, I realize that both platforms severely restrict my freedom to use the tools I choose. Even on an old x86 with 640K of RAM, I had choices; BASIC, Pascal, and C would work under DOS. (Plus assembler, but I had yet to learn that at the time.) On Android, my choices are Java or Java. (OK, or something that compiles to Java, but those selections are still limited.) Plus it seems like I d have to use Eclipse, which seems to have taken the kitchen sync crown from Emacs long ago, and not in a good way. On iOS, the choices are Objective C. And on the web, it s JavaScript. The problem with all this is that there s little room for language innovation on those platforms. If predictions about the decline of development on the PC pan out, how will the next advance in programming languages gain traction if the hot spots for development are locked down to three main languages? Will we find ourselves only willing to consider languages that can compile to Java/Dalvik bytecode? And won t that put limits on the possible space for innovation to occur within? Yes, I know about things like Mono and the Android Scripting Environment, which offer some potential for hope, but I think it s still safe to say that these platforms are pretty closed to development in unofficial languages. I don t know of major apps in the Android market written in Python, for instance. I know of efforts such as Ubuntu s, but even they simply lock down development to a different toolkit. There is no choice there of GTK vs. Qt, or such things. Sadly, I don t think it has to be this way. My phone has way more power than my computer did just a few years ago, and that computer was capable of not just executing, but even compiling, code written in all sorts of different languages. But I don t know if there s much we can do to change things. Even with native code on Android, plenty still has to go through Dalvik (at least that s my understanding). If you wanted to write a Python app with a full graphical touch interface on Android, I don t think this would be all that easy of a task.

John Goerzen: Spring Always Comes

There will come a time when you believe everything is finished. That will be the beginning. Louis L Amour

Last year, I wrote about the difficult times in my life and hope for future, but never really explained why. I have written little since, because there is little I can write without a bit of explanation. It is partly because of the complexity of the task of telling my story without telling too much of others stories. But it is important I tell this story. So often on the Internet, we hear only the brave face, the positive things that happen. This story involves tears and difficulty. And also we often see only anger and bitterness. This story involves joy and celebration. One afternoon last year, I was working as usual (I work from home) when two sheriff deputies arrived on my doorstep. They gave me paperwork showing that my now ex-wife had filed for divorce, had asked them to serve me the paperwork, and that she had been given temporary possession of the house. I had 2 hours to gather up clothes and a little computer equipment (there was a list) and leave the house. Thus began the most difficult time in my life. I went from reading a bedtime story and singing a bedtime song to my young boys every night to seeing them only a little, from living in the house my grandparents and dad had lived in to having no particular plan for where I d sleep that night, from thinking I had a good idea of what the future held to not knowing when, if ever, I d ever be back home. I worried about how the boys would fare (they have done well so far). And it was incomprehensible; I couldn t find answers to why? . In the time since, the divorce became final, I did return home, the boys spend more time with me, and a new normal emerged. At the time, it seemed like a sudden, deep winter blizzard. I couldn t see very far down the road, spring seemed far off, and I couldn t see very well either forward or backward. But I was determined to find positives in the situation. It started almost immediately; I had never been a person to talk about pain, but just a few hours after the divorce was filed, I knew I needed to talk to someone about it, and did. A week later, I shared about it in church. Amazing friends, locally and all over the world, provided support and encouragement. I had less total time with the boys than before the divorce, but more time with just the three of us, and we used it to play together at home, spend days in town, and even take a train trip to Santa Fe, where none of us had ever been before. I realized how much I could forgive, and that my ex-wife probably did the best she could with the persistent legacy of difficult life events that happened to her long before she met me. I understood this, and was never angry, just sad, for everyone. I have always known nobody is perfect, myself included, but can be hard on myself when I m less than perfect. I forgave myself, too, realizing that I did my best to help in the most unfamiliar of waters, and although I sometimes didn t get it right, my conscience is clear because my heart was in the right place and I tried, very hard. Most incredibly, I became a person with a deep sense of inner peace. I always tried to work hard to set life on a good path; I got good grades in school, am a good employee, and have a strong set of values. But where courts are involved, there s a strong sense of powerlessness. At times, there was nothing I could do to make life better for my boys or for me. I finally had to let go of taking on responsibility for all that on my own shoulders. I simply knew that things would be OK, and in fact were OK, and that there is nothing in life that really deserves worries. That doesn t mean worries are never present, but that mostly they are subdued, like a radio quietly playing in another room. When they aren t, I can sit down at the piano, play my penny whistle, sing, walk to my creek, talk to friends, or any of so many things that let them melt away. I stopped searching for happiness and peace, and let those things find me. In religious terms, my faith became not just an intellectual one, but also a spiritual one. An atheist friend asked me, Just what does religion mean to you anyway? My answer: The certainty that spring always comes, for everyone that understands this.

People speak of misfortunes and sufferings, remarked Pierre, but if at this moment I were asked: Would you rather be what you were before you were taken prisoner, or go through all this again? then for heaven s sake let me again have captivity and horseflesh! We imagine that when we are thrown out of our usual ruts all is lost, but it is only then that what is new and good begins. While there is life there is happiness. There is much, much before us. Tolstoy, War and Peace

By Thanksgiving, I had much to be thankful for. Some of it under my nose waiting to be rediscovered after years of distraction, such as the glorious Kansas sunrises. Some things were new, such as roasting a turkey all by myself (or, rather, with Jacob and Oliver) for the first time, and having it come out absolutely perfectly. And some were just the things of everyday life: that I lived in my own house again, that I could walk out to the creek at the edge of my property whenever I wanted, that I could play piano, that the sounds of laughter and little running feet again could often be heard on my wood floors. That s not to say everything was easy; the courts sometimes made decisions sometimes I didn t think were in the boys best interests, legal things dragged on and on, but in the end, peace endured. Happiness endured. I found myself thinking at Thanksgiving that it was the best Thanksgiving ever. Not much later, I considered myself happier than I ve ever been. I was focusing on the daily gifts of life, marveling at the sunrises, looking forward to life s next adventure, confident that it would be far better than the last. And then, to my complete surprise, I found myself in a relationship again. On top of all the wonderful things happening in my life, I met Laura. I never could have imagined a friend so wonderful, a relationship so loving and joyful, something to treasure so deeply. I can look back at events and shake my head in amazement and wonder, that I found myself happier than I d ever been, and then this wonderful relationship on top of that. I have been blessed to have the life I do, and feel almost embarrassingly lucky. I share this story because friends that had been through divorce years ago shared their stories. They gave me hope. And if I didn t share this story in this public way, I would be squandering an opportunity to find more positives from what happened. I hope that this helps, somehow, someone that is in pain know that there is beauty in the valley, and spring always comes, every single year. And I share it because happiness like this can t be repressed for long. Tolstoy was right. While there is life, there is happiness. I ll end with a story from last Thanksgiving. It was 6:10AM that weekend. I was still asleep, and heard this: Jacob, yelling from his room: Dad? Dad! Me, groggy: Yes, Jacob? Jacob: Can I go down and look at the Christmas tree? Me: Sure, and you can turn it on too. Jacob: Great! At that point, I could have gone back to sleep. I was really short on sleep that morning, and Jacob would have been fine. But I gave him a couple of minutes, then I went downstairs too. He was curled up on the piano bench, looking at the tree. I quietly turned up the downstairs thermostat, got a chair, put it next to the piano bench, and sat down by the tree too. Jacob crawled over onto my lap and snuggled up for awhile. Neither of us said anything. Then: Dad? Yes? This is the best Thanksgiving ever. And he gave me a big hug. And he was right. Yes, he was RIGHT! That will be the beginning. Spring comes!

24 July 2013

Daniel Leidert: Moving from (software) RAID 1 to RAID 5 and testing performance

Finally I finished migrating my new N54L from a 2-disk software RAID 1 (2x2TB WD20EFRX) to 3-disk RAID 5 (3x2TB WD20EFRX) using mdadm. I got an excellent mini HOWTO I followed. Resyncing and reshaping the RAID took several days ... top speed was 110.000K/s ... but left my data untouched. Finally growing the RAID now I have two partitions; one with 0.5GB (md0) and the other with 3.4GB (md1) which both host an EXT4 file system, whereas the first one is encrypted via LUKS and the second one is not. RAID device and file system configuration Below is a summary of the setup and the values I found working well for my system.

3-disk RAID 5 device				file system
device	chunk	stripe_cache_size	read_ahead_kb	type	encryption	stride	stripe-width
/dev/md0	64	4096	32768	ext4	luks	16	32
/dev/md1	512	16384	32768	ext4	no	128	256

I've played around with the system a bit. Changing the chunk size on the fly however took a long time. /dev/md0 will contain some backups, so there probably will be a mixture of small and large files. So I've chosen to only test the values 64K, 128K and 512K (default) for this device. I left the other untouched as it will mainly contain large files. Performance measurement Below are the results using hdparm to measure performance. First lets take a look at the drives ...


$ hdparm -tT /dev/sd[bcd]
/dev/sdb:
 Timing cached reads:   3268 MB in  2.00 seconds = 1634.54 MB/sec
 Timing buffered disk reads: 438 MB in  3.00 seconds = 145.87 MB/sec

/dev/sdc:
 Timing cached reads:   3292 MB in  2.00 seconds = 1646.32 MB/sec
 Timing buffered disk reads: 392 MB in  3.01 seconds = 130.22 MB/sec

/dev/sdd:
 Timing cached reads:   3306 MB in  2.00 seconds = 1653.18 MB/sec
 Timing buffered disk reads: 436 MB in  3.00 seconds = 145.26 MB/sec

$ hdparm --direct -tT /dev/sd[bcd]
/dev/sdb:
 Timing O_DIRECT cached reads:   468 MB in  2.01 seconds = 233.17 MB/sec
 Timing O_DIRECT disk reads: 442 MB in  3.00 seconds = 147.15 MB/sec

/dev/sdc:
 Timing O_DIRECT cached reads:   468 MB in  2.00 seconds = 233.69 MB/sec
 Timing O_DIRECT disk reads: 392 MB in  3.01 seconds = 130.36 MB/sec

/dev/sdd:
 Timing O_DIRECT cached reads:   468 MB in  2.00 seconds = 233.94 MB/sec
 Timing O_DIRECT disk reads: 442 MB in  3.01 seconds = 146.93 MB/sec

... and now at the RAID devices ...


$ hdparm -tT /dev/md?
/dev/md0:
 Timing cached reads:   3320 MB in  2.00 seconds = 1660.37 MB/sec
 Timing buffered disk reads: 770 MB in  3.01 seconds = 256.05 MB/sec

/dev/md1:
 Timing cached reads:   3336 MB in  2.00 seconds = 1668.07 MB/sec
 Timing buffered disk reads: 742 MB in  3.01 seconds = 246.89 MB/sec

$ hdparm --direct -tT /dev/md?
/dev/md0:
 Timing O_DIRECT cached reads:   974 MB in  2.00 seconds = 487.08 MB/sec
 Timing O_DIRECT disk reads: 770 MB in  3.01 seconds = 256.17 MB/sec

/dev/md1:
 Timing O_DIRECT cached reads:   784 MB in  2.00 seconds = 391.18 MB/sec
 Timing O_DIRECT disk reads: 742 MB in  3.01 seconds = 246.42 MB/sec

... and now lets see, which actual speed we reach using dd. First lets check the encrypted device:


RAID-5 /dev/md0 (LUKS encrypted EXT4): chunk=64K, stripe_cache_size=4096,
   readahead(blockdev)=65536, stride=16, stripe-width=32 ...

$ dd if=/dev/zero of=/mnt/md0/10g.img bs=1k count=10000000
10000000+0 Datens tze ein
10000000+0 Datens tze aus
10240000000 Bytes (10 GB) kopiert, 64,1227 s, 160 MB/s

$ dd if=/mnt/md0/10g.img of=/dev/null bs=1k count=10000000
10000000+0 Datens tze ein
10000000+0 Datens tze aus
10240000000 Bytes (10 GB) kopiert, 85,768 s, 119 MB/s

Well, read speed is consistently lower than write speed for the encrypted file system. Lets take a look at the non-encrypted device:


RAID-5 /dev/md1 (EXT4): chunk=512K, stripe_cache_size=16384,
   readahead(blockdev)=65536, stride=128, stripe-width=256 ...

$ dd if=/dev/zero of=/mnt/md1/10g.img bs=1k count=10000000
10000000+0 Datens tze ein
10000000+0 Datens tze aus
10240000000 Bytes (10 GB) kopiert, 37,0016 s, 277 MB/s

$ dd if=/mnt/md1/10g.img of=/dev/null bs=1k count=10000000
10000000+0 Datens tze ein
10000000+0 Datens tze aus
10240000000 Bytes (10 GB) kopiert, 33,5901 s, 305 MB/s

Looks nice to me. How to set these values I use a mixture of udev, util-linux and e2fsprogs to set the values. First I checked, which values for stripe_cache_size and read_ahead_kb are working best for me. For the LUKS encrypted EXT4 I got varying results showing best performances with values of 4096, 8192 and 16384 for stripe_cache_size. I decided for the first value, because it appeared more often with the best performance than the others.


$ less /etc/udev/rules.d/90-local-n54l.rules   grep stripe_cache
SUBSYSTEM=="block", KERNEL=="md0", ACTION=="add", TEST=="md/stripe_cache_size", TEST=="queue/read_ahead_kb", ATTR md/stripe_cache_size ="4096", ATTR queue/read_ahead_kb ="32768", ATTR bdi/read_ahead_kb ="32768"
SUBSYSTEM=="block", KERNEL=="md1", ACTION=="add", TEST=="md/stripe_cache_size", TEST=="queue/read_ahead_kb", ATTR md/stripe_cache_size ="16384", ATTR queue/read_ahead_kb ="32768", ATTR bdi/read_ahead_kb ="32768"

The read_ahead_kb value can also be set using blockdev. Note that this command expects a value of 512-byte sectors whereas read_ahead_kb is the size in kbyte. Therefor the difference in values:


$ blockdev --setra 65536 /dev/md[01]

Tuning the EXT4 file system performance with calculated values using tune2fs:


$ tune2fs -E stride=16,stripe-width=32 -O dir_index /dev/mapper/_dev_md0
$ tune2fs -E stride=128,stripe-width=256 -O dir_index /dev/md1

Disabling NCQ reduced the speed a lot for me, so I left the values as is and did not struggle with it:


$ cat /sys/block/sd[bcd]/device/queue_depth 
31
31
31

Gergely Nagy: My git tagging convention

Zigo posted an article earlier, one which I disagree with strongly. First of all, something that has been in use ever since git was invented is hardly a new fashion. Something that is used by the very same person who invented git and by git itself, is hardly surprising to be found so widespread. Especially not when it is used as an example in git's very own documentation! Writing it off as something silly, because of the way GitHub works, without even trying to understand the reason behind it is just plain wrong. Just because GitHub names tarballs the way it does, does not mean that we should change the way we tag.For example, cgit (used by, among others, kernel.org), uses the tag name as-is for its tarballs, unlike GitHub which prepends the project name, so one will have to mangle the filename in a debian/watch file anyway, when downloading from a host that is not GitHub, therefore his arguments for a raw version-only tag are bogus, unless your upstream is using GitHub.But I do not want to criticise only, rather, to provide a reason why a prefix is used, and why I chose the "worst" prefix: the project name itself.One very strong reason to use a prefix, either v or any other prefix is to ease tab completion. If you have a tag that is a bare number, you type the first, press tab, and get a mixture of commits and tags. You can't easily tell your completion system that you want commits or tags. With a prefix outside of the hex range, which v is, you can do that, and that makes working with it a lot easier.Is it for convenience? Yes. But ask yourself this: how many times do you have to write a debian/watch file? Once per package. How much time do you spend looking for a tag, or a commit? A lot more. So which one is most important? The convenience of someone who has to work around the tarball naming of the hosting service once, or the developer who works with the software daily? Obviously the latter!But that is not all. I'm an advocate of prefixing the tag with the project name, actually, and I've been doing that for all new projects for a while now, and I'm not going to change that, because it serves a very practical purpose: if you have a parent repository, with a lot of submodules, if git tag tells you the project name too, that makes it much easier to navigate. I don't have to bake submodule support into my shell prompt (that would be costy), and I won't find it surprising that if I enter a subdirectory, git tag fails to list the version I know I'm working on. Just because I happened to end up in a submodule, which is something I end up doing often, as I have many repositories that have submodules, which in turn have other submodules, and so on and so forth through many layers.A raw version number as a tag name is insufficient for my needs, simple as that. And I, as upstream, don't care that whoever packages my thing, will have to add a line to a watch file. That only needs to be done once, while I work with tags and commits daily.I'm sorry, but calling this naming convention a disease just because GitHub's tarball naming is what it is, without even considering that there may be other reasons behind it than legacy, and that there are hosting sites outside of GitHub, is a mistake. Dear package maintainers, please do not annoy us upstreams with requests to cripple our daily work. Thank you.

Next.

Previous.