Doing code reviews is a great way to discover things that people might
struggle to comprehend. While proof-reading
OpenStack patches recently, I spotted that
people were not using correctly the various decorators Python provides for
methods. So here's my attempt at providing me a link to send them to in my
next code reviews. :-)
How methods work in Python
A method is a function that is stored as a class attribute. You can declare
and access such a function this way:
>>> class Pizza(object):
... def __init__(self, size):
... self.size = size
... def get_size(self):
... return self.size
...
>>> Pizza.get_size
<unbound method Pizza.get_size>
What Python tells you here, is that the attribute
get_size of the class
Pizza is a method that is unbound. What does this mean? We'll know as
soon as we'll try to call it:
>>> Pizza.get_size()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unbound method get_size() must be called with Pizza instance as first argument (got nothing instead)
We can't call it because it's not bound to any instance of
Pizza. And a
method wants an instance as its first argument (in Python 2 it must be
an instance of that class; in Python 3 it could be anything). Let's try to
do that then:
>>> Pizza.get_size(Pizza(42))
42
It worked! We called the method with an instance as its first argument, so
everything's fine. But you will agree with me if I say this is not a very
handy way to call methods; we have to refer to the class each time we want
to call a method. And if we don't know what class is our object, this is not
going to work for very long.
So what Python does for us, is that it binds all the methods from the class
Pizza to any instance of this class. This means that the attribute
get_size of an instance of
Pizza is a bound method: a method for
which the first argument will be the instance itself.
>>> Pizza(42).get_size
<bound method Pizza.get_size of <__main__.Pizza object at 0x7f3138827910>>
>>> Pizza(42).get_size()
42
As expected, we don't have to provide any argument to
get_size, since
it's bound, its
self argument is automatically set to our
Pizza
instance. Here's a even better proof of that:
>>> m = Pizza(42).get_size
>>> m()
42
Indeed, you don't even have to keep a reference to your
Pizza object. Its
method is bound to the object, so the method is sufficient to itself.
But what if you wanted to know which object this bound method is bound to?
Here's a little trick:
>>> m = Pizza(42).get_size
>>> m.__self__
<__main__.Pizza object at 0x7f3138827910>
>>> # You could guess, look at this:
...
>>> m == m.__self__.get_size
True
Obviously, we still have a reference to our object, and we can find it back
if we want.
In Python 3, the functions attached to a class are not considered as
unbound method anymore, but as simple functions, that are bound to an
object if required. So the principle stays the same, the model is just
simplified.
>>> class Pizza(object):
... def __init__(self, size):
... self.size = size
... def get_size(self):
... return self.size
...
>>> Pizza.get_size
<function Pizza.get_size at 0x7f307f984dd0>
Static methods
Static methods are a special case of methods. Sometimes, you'll write code
that belongs to a class, but that doesn't use the object itself at all. For
example:
class Pizza(object):
@staticmethod
def mix_ingredients(x, y):
return x + y
def cook(self):
return self.mix_ingredients(self.cheese, self.vegetables)
In such a case, writing
mix_ingredients as a non-static method would work
too, but it would provide it a
self argument that would not be used. Here,
the decorator
@staticmethod buys us several things:
- Python doesn't have to instantiate a bound-method for each Pizza object
we instiantiate. Bound methods are objects too, and creating them has a
cost. Having a static method avoids that:
>>> Pizza().cook is Pizza().cook
False
>>> Pizza().mix_ingredients is Pizza.mix_ingredients
True
>>> Pizza().mix_ingredients is Pizza().mix_ingredients
True
-
It eases the readability of the code: seeing @staticmethod, we know that
the method does not depend on the state of object itself;
-
It allows us to override the mix_ingredients method in a subclass. If we
used a function mix_ingredients defined at the top-level of our module, a
class inheriting from Pizza wouln't be able to change the way we mix
ingredients for our pizza without overriding cook itself.
Class methods
Having said that, what are class methods? Class methods are methods that are
not bound to an object, but to a class!
>>> class Pizza(object):
... radius = 42
... @classmethod
... def get_radius(cls):
... return cls.radius
...
>>>
>>> Pizza.get_radius
<bound method type.get_radius of <class '__main__.Pizza'>>
>>> Pizza().get_radius
<bound method type.get_radius of <class '__main__.Pizza'>>
>>> Pizza.get_radius is Pizza().get_radius
True
>>> Pizza.get_radius()
42
Whatever the way you use to access this method, it will be always bound to
the class it is attached too, and its first argument will be the class
itself (remember that classes are objects too).
When to use this kind of methods? Well class methods are mostly useful for
two types of methods:
- Factory methods, that are used to create an instance for a class using for
example some sort of pre-processing. If we use a @staticmethod instead, we
would have to hardcode the Pizza class name in our function, making any
class inheriting from Pizza unable to use our factory for its own use.
class Pizza(object):
def __init__(self, ingredients):
self.ingredients = ingredients
@classmethod
def from_fridge(cls, fridge):
return cls(fridge.get_cheese() + fridge.get_vegetables())
- Static methods calling static methods: if you split a static methods in
several static methods, you shouldn't hard-code the class name but use class
methods. Using this way to declare ou method, the Pizza name is never
directly
referenced and inheritance and method overriding will work flawlessly
class Pizza(object):
def __init__(self, radius, height):
self.radius = radius
self.height = height
@staticmethod
def compute_circumference(radius):
return math.pi * (radius ** 2)
@classmethod
def compute_volume(cls, height, radius):
return height * cls.compute_circumference(radius)
def get_volume(self):
return self.compute_volume(self.height, self.radius)
Abstract methods
An abstract method is a method defined in a base class, but that may not
provide any implementation. In Java, it would describe the methods of an
interface.
So the simplest way to write an abstract method in Python is:
class Pizza(object):
def get_radius(self):
raise NotImplementedError
Any class inheriting from
Pizza should implement and override the
get_radius method, otherwise an exception would be raised.
This particular way of implementing abstract method has a drawback. If you
write a class that inherits from
Pizza and forget to implement
get_radius, the error will only be raised when you'll try to use that
method.
>>> Pizza()
<__main__.Pizza object at 0x7fb747353d90>
>>> Pizza().get_radius()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in get_radius
NotImplementedError
There's a way to triggers this way earlier, when the object is being
instantiated, using the
abc
module that's provided with Python.
import abc
class BasePizza(object):
__metaclass__ = abc.ABCMeta
@abc.abstractmethod
def get_radius(self):
"""Method that should do something."""
Using
abc and its special class, as soon as you'll try to instantiate
BasePizza or any class inheriting from it, you'll get a
TypeError.
>>> BasePizza()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class BasePizza with abstract methods get_radius
Mixing static, class and abstract methods
When building classes and inheritances, the time will come where you will
have to mix all these methods decorators. So here's some tips about it.
Keep in mind that declaring a class as being abstract, doesn't freeze the
prototype of that method. That means that it must be implemented, but i can
be implemented with any argument list.
import abc
class BasePizza(object):
__metaclass__ = abc.ABCMeta
@abc.abstractmethod
def get_ingredients(self):
"""Returns the ingredient list."""
class Calzone(BasePizza):
def get_ingredients(self, with_egg=False):
egg = Egg() if with_egg else None
return self.ingredients + egg
This is valid, since
Calzone fulfil the interface requirement we defined
for
BasePizza objects. That means that we could also implement it as being
a class or a static method, for example:
import abc
class BasePizza(object):
__metaclass__ = abc.ABCMeta
@abc.abstractmethod
def get_ingredients(self):
"""Returns the ingredient list."""
class DietPizza(BasePizza):
@staticmethod
def get_ingredients():
return None
This is also correct and fulfil the contract we have with our abstract
BasePizza class. The fact that the
get_ingredients method don't need to know
about the object to return result is an implementation detail, not a
criteria to have our contract fulfilled.
Therefore, you can't force an implementation of your abstract method to be a
regular, class or static method, and arguably you shouldn't. Starting with
Python 3 (this won't work as you would expect in Python 2, see
issue5867), it's now possible to use the
@staticmethod and
@classmethod decorators on top of
@abstractmethod:
import abc
class BasePizza(object):
__metaclass__ = abc.ABCMeta
ingredient = ['cheese']
@classmethod
@abc.abstractmethod
def get_ingredients(cls):
"""Returns the ingredient list."""
return cls.ingredients
Don't misread this: if you think this going to force your subclasses to
implement
get_ingredients as a class method, you are wrong. This simply
implies that your implementation of
get_ingredients in the
BasePizza
class is a class method.
An implementation in an abstract method? Yes! In Python, contrary to methods
in Java interfaces, you can have code in your abstract methods and call it
via
super():
import abc
class BasePizza(object):
__metaclass__ = abc.ABCMeta
default_ingredients = ['cheese']
@classmethod
@abc.abstractmethod
def get_ingredients(cls):
"""Returns the ingredient list."""
return cls.default_ingredients
class DietPizza(BasePizza):
def get_ingredients(self):
return ['egg'] + super(DietPizza, self).get_ingredients()
In such a case, every pizza you will build by inheriting from
BasePizza
will have to override the
get_ingredients method, but will be able to use
the default mechanism to get the ingredient list by using
super().
Well, my git-annex crowdfunding campaign is
half way to its August 15th conclusion. So far it's raised more than five times
what I hoped it would. I wish I could say I'm like some canny NASA engineer
who intentionally sets low expectations for their Mars rover, but in
both the previous kickstarter and this campaign I've really had no idea how
far it'd go. I'm glad that I'll be working on
git-annex for another year.
I was particularly unsure if it'd be successful to move off Kickstarter.
During the git-annex assistant Kickstarter campaign, I saw many small
contributions from people who learned of it due to it being a successfully
funded project, a staff pick, etc. Losing that easy network effect is a
gamble.
So far I've had only half the number of contributors that I got on
Kickstarter. I've basically missed out entirely on the $5 level casual
contributors. On the other hand, my backers have generally been more
generous (and some have been exceedingly generous). And I've avoided
rewards that will cost much money, so I may end up in the same ballpark
funding level in the end!
Incidentially, I'm really enjoying getting in
touch to let people know when
I
make
their
sponsored
commits.
There's still time to sponsor one of your own ;)
Bitcoin
I also was curious to experiment with Bitcoin in this campaign.
Partly because Paypal isn't available everywhere internationally, and takes
really obnoxious percentages of transactions (though probably not as bad
as Kickstarter taking its percentage followed by Amazon payments taking its
percentage..) and partly because there seem to be interesting possibilities for
supporting free software with Bitcoin. (Especially if any of the
microtransactions on top of Bitcoin take off.)
So far 5% of backers have used Bitcoin. It's been quite strange to actually
have significant amounts of bitcoins in my wallet.
Wordpress
has had 94 bitcoin payments over 9 months since starting accepting them.
I've had 47 payments in the two weeks my campaign has run so far. Wow!
Most of the bitcoin payments have come in via Coinbase (a few people have
found my direct
payment address), but of those very few were using bitcoin purchased on
Coinbase. Most are probably transfers of bitcoin they already had, or
perhaps bitcoin purchased on other sites.
The one technical issue I've had with using bitcoin is that Coinbase
has not provided details about who sent most of the donations. Probably
some of them are intentionally anonymous, but I suspect Coinbase's interface
to claim incoming bitcoin transactions failed for some of them.
(If you donated bitcoin and want to actually get a reward, please email me.)
By the way, I'm converting most of the bitcoins back to USD pretty quickly.
I'm not interested in speculating on currency exchange rates with money
that has been donated so I can accomplish a particular task..
DIY
I put up the campaign website without any
means in place to handle updating it. This is because I never automate
anything until I've done it at least 10 times by hand. ;) After the first
trickle of donations became a flood, I quickly realized I needed at least
something to handle keeping the numbers straight.
What I whipped up in an hour of coding is a system where I enter incoming
payments into a hledger file and a small haskell
program parses that and writes out various files that are included into the
website. Amusingly the percentage calculation and display code was copied
from git-annex, so part of git-annex is helping run its own fundraising
campaign. The campaign video is itself hosted in a public git-annex
repository, come to think of it.
The rest of the site is built using ikiwiki. Given that it's hosted at
Branchable, this is a high level of dogfooding and DIY. There are certianly
better crowdfunding platforms, but all I miss in this one is automated
transaction entry. And I have total flexability, double entry accounting,
and a powerful static website generator that handled being on the top of
Hacker News without a sweat. Oh, and some money. What's not to like?
Shipping a UEFI-bootable Linux distribution is a touch awkward[1], with the main sticking point being the necessity to produce boot media with multiple El-Torito images. An El-Torito image is either an image of a floppy disk or a small hard drive, embedded into the ISO. This allows BIOS systems to look for an El Torito image, hook some interrupts and then boot without the BIOS having to care about the fact that the image is embedded in an ISO-9660 filesystem. UEFI systems will look for an El-Torito image with a special tag - if they find it they'll mount it as a FAT filesystem and look for a bootloader, and if not they'll fall back to BIOS compatibility.
Continuing my 
Following 
Today I've made the 0.15 release of 
Did you ever had the need in a git repository to change to another branch without altering AT ALL the working copy and ever wondered how that's done?
It seems to be a new fashion. Instead of tagging software with a normal version number, many upstream adds a one letter prefix. Instead of version 0.1.2, it becomes version v0.1.2.
This sickness has spread all around in Github (to tell only about the biggest one), from one repository to the next, from one author to the next. It has consequences, because Github (and others) conveniently provides tarballs out of Git tags. Then the tarball names becomes packagename-v0.1.2.tar.gz instead of package packagename-0.1.2.tar.gz. I ve even seen worse, like tags called packagename-0.1.2. Then the tarball becomes packagename-packagename-0.1.2. Consequently, we have to go around a lot of problems like mangling in our debian/watch files and so on (probably the debian/gbp.conf if you use that ). This is particularly truth when upstream doesn t make tarball and only provides tags on github (which is really fine to me, but then tags have to be made in a logical way). Worse: I ve seen this v-prefixing disease as examples in some howtos.
What s wrong with you guys? From where is coming this v sickness? Have you guys watch too much the v 2009 series on TV, and you are a fan of the visitors? How come a version number isn t just made of numbers? Or is this just a v like the virus of prefixing release names with a v ?
So, if you are an upstream author, reading debian planet, with your software packaged in Debian, and caught the bad virus of prefixing your version numbers with a v, please give-up on that. Adding a v to your tags is meaningless anyway, and it s just annoying us downstream.
Edit: Some people pointed to me some (IMO wrong) reasons why to prefix version numbers. My original post was only half serious, and responding with facts and common sense breaks the fun! :) Anyway, the most silly one being that Linus has been using it. I wont comment on that one, it s obvious that it s not a solid argumentation. Then the second one is for tab completion. Well, if your bash-completion script is broken, fix it so that it does what you need, rather than going around the problem by polluting your tags. Then the 3rd argument was if you were merging 2 repositories. First this never happened to me to merge 2 completely different repos, and I very much doubt that this is an operation that you have to do often. Second, if you merge the repositories, the tags are loosing all meanings, and I don t really think you will need them anyway. Then the last one would be working with submodules. I haven t done it, and that might be the only case where it makes sense, though this has nothing to do with prefixing with v (you would need a much smarter approach, like prefixing with project names, which in that case makes sense). So I stand by my post: prefixing with v makes no sense.
Present Extension Redirection
Multi-buffered applications have always behaved poorly in the presence
of the Composite extension:
It's almost one year since 
It makes me wonder, if Mr Assange is genuinely guilty of rape despite his alleged victim 
reSIProcate is quite a large project involving multiple shared libraries, some 
To gain some confidence in this progress, I had already run the full reSIProcate build, including test cases,