Search Results: "nate"

2 November 2008

Decklin Foster: Tap that class

And now, something entirely impractical. I picked up Beginning Ruby by Peter Cooper the other day to look for some teaching material. Flipping through, I came across a basic feature that had heretofore escaped my notice: the Struct class. If you want to use a class for nothing more than bundling together a few values, you can create a Struct instead of writing out initialize and attr_accessor yourself. The idiom is like this:
class Server < Struct.new(:host, :port, :password)
  def to_s
    port == 6600 ? host : "# host :# port "
  end
end
That particular one's from Njiiri (which may clue you in that it took me several months to get around to writing this post). For educational purposes, it's nice: you can point out how we give a name to a class (I've also tried to explain assignment as "naming" rather than "storing"), that there's an unnamed class (classes are also objects!), and overriding a method (which uses the dynamically defined stuff), without any boilerplate to get in the way. So it's tempting to use this in a lesson. Unfortunately, Ruby is a little bit weird here, and you run into those distracting "practical" issues about the particular language you're working in. Your classes can't subclass Class, and thus you can't say Foo.new like you can Struct.new. Struct is actually implemented in C (or Java, or whatever's native). So you have to do some handwaving. This is because the "metaclasses" we have in Ruby are implemented, so to speak, as singleton classes of objects (including class objects). I do not mean this as a criticism of Matz at all, but they seem like more of a serendipitous thing than an intentional design -- "hey, if we implement classes in this particular way, we get this sort of metaclassing automatically [1]." (The clearest explanation of why this is that I've found is in the first chapter of Advanced Rails by Brad Ediger.) I wanted to be able to just subclass Class, rather than have all that fun power that we normally get to abuse in Ruby, only because I think this is the clearest way to explain the abstraction. Even I still have trouble describing singletons in plain English. Struct is intuitively a class of classes, and factors out similar/boring stuff -- a good practice. It could be an example of how to refactor some simple classes, if only we could follow it. I decided to give up on the idea for my tutorial, but it kept me up at night. Ruby metaclasses clearly can do anything that the sort of Struct-like metaclasses I have in my mind -- "parametric" classes, if you will -- can do, and I can dynamically define just about anything; why not make it happen? We can instantiate Class itself, but the tool we have to shape that class is singleton methods. This can certainly be abstracted away. So, I whipped something up. Before getting to it, though, let's visit a new construct in Ruby 1.9 and 1.8.7: Object#tap. Apart from the obvious debugging use described in its documentation, it makes it quite easy to factor out this pattern:
def gimme_a_thing
  thing = Thing.new
  thing.do_stuff_to_it
  thing
end
Into something closer to the style of functional or declarative programming:
def gimme_a_thing
  Thing.new.tap  thing 
    thing.do_stuff_to_it
  end
end
(Well, "do stuff" is obviously still procedural, but A for effort.) Which one is "better" could probably be the subject of much debate, but: I really prefer the second one; even though it has exactly the same effect, it looks like "what" rather than "how" (something else I try to beat into impressionable young heads. :-)), which is I think easier to write tests for. I'm going to use it here, because it's shorter. Now, Object#tap passes the object as an argument to the block; in our case, we are going to define a metaclass, so we want to work with self, instead. So we define a new version, class_tap -- by analogy with class_def, I suppose --- which class_evals the block rather than simply evaluating it:
class Class
  def class_tap(&blk)
    tap  _self  _self.class_eval &blk  
  end
end
And to do the following trickery, we make use of MetAid, written by why the lucky stiff -- it's very small, so we could always just incorporate the bits we want into the code here, but this short file provides a common vocabulary for talking about metaclass stuff which is quite valuable. Now, we can write a method to create our new classes on the fly. Here's what I came up with:
require 'metaid'
class << Class
  def meta(_super=Object, &blk)
    new.class_tap do                     # 1
      meta_def :new do  *args            # 2
        Class.new(_super).class_tap do   # 3
          class_exec *args, &blk         # 4
        end
      end
    end
  end
end
(Note the "class << Class", opening the singleton, rather than "class Class", opening Class itself. Also, the distinction between new and Class.new -- they are the same method, but from inside the meta_def we're no longer in the Class class.) The lines of meta itself mean:
  1. The value of this thing is a generated class, which we will describe thusly:
  2. Its singleton class has a new method, which gives you a value that is:
  3. Another generated class, which is defined by:
  4. the original block, which now gets run with new's arguments.
Nary an assignment in sight! The (embarrassing) caveat is that methods in the block that defines the new class instance cannot use def to create methods as normal. A def is evaluated in a totally fresh scope (I don't think I get the opinion behind this decision, to be honest), so we need to use class_def instead. This is, I must admit, rather hideous. Perhaps someday the language will change. But, first things first: now we can implement Struct in pure Ruby.
class MyStruct < Class.meta \
  do  *args 
    attr_accessor *args
    class_def :initialize do  *instance_args 
      args.zip(instance_args).each do  attr, val 
        instance_variable_set :"@# attr ", val
      end
    end
  end
end
The real Struct class does a few other nice things for you, but this is the heart of it; I can go back to that Njiiri example and just swap in MyStruct for Struct (Providing an instant performance gain of -200%... :-)). Here's a "shapes" example from my abandoned OO lesson:
class Polygon < Struct.new(:sides)
  def perimeter
    @sides.inject(&:+)
  end
end
class RegularPolygonClass < Class.meta(Polygon) \
  do  n_sides 
    class_def :initialize do  side_length 
      @sides = [side_length] * n_sides
    end
    class_def :area do
      @sides.size * @sides.first**2 / Math.tan(Math::PI / @sides.size) / 4
    end
  end
end
class Square < RegularPolygonClass.new(4); end
class Pentagon < RegularPolygonClass.new(5); end
(Note that area has no free variables and thus could actually be defined with def. It would just look funny.) You only have to change one number here to make new polygon classes, rather than accumulating parameter lint or explicitly subclassing and redefining something implicit just for the derived class [2]. In a way it the exact analogue of the imperative vs. tap style described above. There is quite a bit of aesthetics involved; one way is not "right". Apart from the syntactical wart, I like being able to do things this way. Ruby is, as they say, optimized for programmer happiness and the principle of least surprise. Still, I'm sure it is quite slow, and I don't particularly need it for any real-world application right now. For an intro to OO it's way too much complexity to have lurking unexplained beneath the surface and still requires getting bogged down in the language you happen to be using. But hey! It's neat.
[1]I basically try to do this whenever possible, anyway. Design decisions are like sex: they're better when they're free.
[2]Of course, if Ruby class variables were not leaked across the entire inheritance hierarchy, something as trivial as this one-variable example would be... trivial. At least, few people would actually miss metaprogramming features. Blub blub blub.

1 November 2008

Thomas Girard: Playing with C++0x -- tuple

C++0x is approaching, and it comes with a lot of new stuff, so I've decided to play with it. Some C++0x features are already available in g++ 4.3. Others will be available in the 4.4 release -- see this page for a summary of C++0x language support in g++. What is a tuple? It's a std::pair on stero ds: it can contain any number of elements. For the impatient, the whole sample program is available from here. It can be compiled with: g++-4.3 --std=c++0x tuple.cpp -o tuple. Now let's see how to create a tuple:
#include <string>
#include <tuple>
typedef std::tuple<int, double, std::string> my_tuple_t;
my_tuple_t a_tuple(1, 5.0);
Every element of the tuple gets initialized to its default value if it's not specified. Indeed displaying the third element:
std::cout << std::get<2>(a_tuple) << std::endl;
outputs nothing, because the string element was default-constructed to an empty string. The std::make_tuple function, like std::make_pair for std::pair, can be used to create a tuple using type deduction:
std::cout << std::get<1>(std::make_tuple(false, std::string("see"))) << std::endl;
A tuple element can be changed using the same get function:
std::get<2>(a_tuple) = "a string";
Of course, a tuple is type safe. Trying to put a string at the first element:
std::get<0>(a_tuple) = "another string";
does not compile. But there's more: you also get a compilation error when trying to access an element that does not exist. For example trying to display fourth element does not compile:
std::cout << std::get<3>(a_tuple) << std::endl;
What else can you do with tuples? You can test them for equality, or assign them:
my_tuple_t another_tuple = std::make_tuple(2, 10.0, std::string("nothing"));
std::cout << (a_tuple == another_tuple) << std::endl;
another_tuple = a_tuple;
std::cout << (a_tuple == another_tuple) << std::endl;
You can also retrieve multiple elements at once, possibly ignoring some of them:
std::string a_string;
std::tie(std::ignore, std::ignore, a_string) = another_tuple;
Last, you can concatenate two tuples into a new tuple:
std::tuple<int, double, std::string, bool, std::string> cat =
    std::tuple_cat(a_tuple, std::make_tuple(false, std::string("see")));

31 October 2008

Gunnar Wolf: How (and how not) to create cyclist awareness

Michael blogs about Critical Mass.
What is Critical Mass in case you are too lazy to go to Michael's or to the Wikipedia? A cyclist-awareness movement, showing how bikers are safer (i.e. more visible) when there are more of them. The (dis)organizational nature of this movement (at least according to Wikipedia's information) seems quite fun and interesting.
However, I have to oppose what Michael says is a strong point of CM in Austria - At least, given my country's culture.
People who don't bike often say we have the least bike-aware culture, and that this city must be like hell to cyclists. It is not - Mexicans tend to believe this is the worst place possible on many accounts, and I'm happy to prove the contrary. I do feel that bringing people out to the streets, as our local government's Mu vete en bici program pushes, is completely right: It shows people how fast they can move in real streets in the city, taking away the fear of being run over by a bus, and people will gradually understand we can all excercise the right to use the streets without polluting, and gives us a better idea on how to behave with traffic (even if the traffic is made of fellow bikers) around us. I know this program works - At least, because it worked on me. Whenever possible, I go everywhere in this (not small or friendly by any measure) city by bike.
Anyway - What criticism do I have for Michael's post? That he states they:
Bicycling ludicrously slowly for a good hour, we managed to claim some fairly busy streets while many people tried to get home by car.
IMHO, what we should be doing is to demand the drivers to respect us, not to make them want to run us over. They should not be driven into hating bikers because of the chaos they generate - It is terrible to sit in a car for two hours when you usually take 30 minutes. There is simply no justification for that.
When I took part of the World Naked Bike Ride, some people wanted to take all four lanes of Reforma. Fortunately, reason prevailed, and we took only the lane we were assigned. And we should keep that in mind! Whenever possible, we should protest and make ourselves heard, but without interfering, without damaging, other people's lives!
Oh, by the way: If anybody in Mexico wants to have some nice hours of healthy fun: I am still pondering whether to join, as it is a huge effort, but I am very inclined to do so. Next week, November 9, we will go by bike from Mexico City's Z calo to Pachuca. We face 95Km and close to four hours. The road to Pachuca is basically flat, and going in a large group is a great experience. Hope to see you there!

30 October 2008

Miriam Ruiz: Homophobic words of the Queen of Spain?

“I can understand, accept and respect that there are people with different sexual trends, but that they feel proud to be gay? That they go out on parades and demonstrations?” , Said the Queen of Spain to Pilar Urbano, the author of a biography about her (”La Reina muy de cerca“). “If these people want to live together, dress up for a wedding and to marry, they might have right to do so or not, according to their country’s laws: but they shouldn’t call that marriage, because it’s not. There are many possible names: social contract, union contract.“, she added. “Puedo comprender, aceptar y respetar que haya personas con otra tendencia sexual, pero que se sientan orgullosos por ser gays? Que se suban a una carroza y salgan en manifestaciones?“, dijo la Reina de Espa a a Pilar Urbano, autora de una biograf a sobre ella (”La Reina muy de cerca“). “Si esas personas quieren vivir juntas, vestirse de novios y casarse, pueden estar en su derecho, o no, seg n las leyes de su pa s: pero que a eso no le llamen matrimonio, porque no lo es. Hay muchos nombres posibles: contrato social, contrato de uni n.“, a adi . She has also spoken against abortion and euthanasia and in favor of forcing all the children to learn religion (catholic) at school. The Royal Family is supposed not to have a public political opinion, as they should theoretically represent every Spanish person. Fortunately many of us don’t believe in blue blood, kings, queens and all that stuff, so she definitely doesn’t represent us :) God shave the Queen!

Michael Banck: 30 Oct 2008

Systems Expo 2008 As the last years, Debian was offered a booth at last week's Systems expo here in Munich again. However, this year the Free Projects area was not organized by Rosa Riebl from C&L publishing, but by Wolfgang Drotschmann from LinuxTag e.V.. This made some things a bit more difficult, e.g. we did not know our exact booth number until a couple of days before the expo and the small conference programme was made up in an ad-hoc fashion after the expo started, but in the end most things worked out fine in some way or the other. While the booth (a demo-point, really) was as big (or rather small) as last year, there was much more space around the booth this year (something which seemed to apply to all of Systems), so things did not get too crowded even when a handful of visitors approached the booth at once. Also, the visibility was much improved as our demo point was visible by strolling visitors this year (last year, our demo point was just facing the wall). As nobody else stepped up, I had to organize the booth again. Fewer people than last year were around; only Robert Grimm, Arne Wichmann, Franziska Lichtblau, Johannes Wiedersich, Andreas Barth and I were able to commit to staffing the booth; other people were busy over the week or moved away from Munich since last year, like Robert Lemmen or Wolfgang Lonien. Luckily, we still had the computer the GNOME project donated to us last year, and I took a TFT, keyboard and mouse from the university along. This year, I decided to not show up for the booth build-up the day before Systems starts. On the one hand, I had made the experience that there is not much one can do then anyway, and would have to put the computer/TFT into the central locker room overnight anyway. On the other hand, there was as always a very low attendence in the first few hours of the expo so building up the booth in the morning turned out to be no problem. The Credativ people again provided us with merchandise (due to some miscommunication on my part the package had to arrive directly at the expo, but in the end I was glad about this as I had enough trouble carrying the computer and TFT to the expo). This year, thanks to Credativ, we were able to provide some t-shirts for the first time, something quite some visitors had requested over the years. Besides t-shirts, we had some swirl stickers and the popular Debian keychains provided by Joerg Jaspert through Credativ. In the end the booth mostly consisted of the computer (demonstrating Lenny most of the time) and some A4 sized Swirls I had printed out the day before. From Wednesday on we were able to provide Lenny Beta2 CDs as well thanks to Johannes Wiedersich who organized them. Initially, I asked ADR whether they would produce some CDs for us again as they did last year, but they did not bring the appropriate hardware this time. But thanks to Johannes we were still able to provide interested visitors with CDs through LSK. In the end, we ran out of most t-shirts at some point on Thursday and managed to sell the two remaining Lady shirts on Friday. All the other merchandise was gone by the end of the show as well so the way back was not that difficult, even more so as Andreas Barth helped me carry the computer to the subway and my car. The days I was at Systems (first and last day) the attendance was rather low, so not that many interesting discussions happened. Almost everybody who stepped by knew Debian already and the majority was using it themselves as well, at least on their servers. Those people were also really quite happy, we rarely heard much critizism, even after asking people for some. The most frequently asked question was undoubtly "when will Lenny release?", followed by "do you have that cool t-shirt in L or XL as well?". Overall it was a pretty good experience, albeit slightly stressful organizing it. In any case, this was probably the last time I had to do this as the Systems organizers announced they will rethink their concept and there will be no Systems 2009.

Marc 'Zugschlus' Haber: Nagios, Parent Hosts, and traceroute on the Internet

Nagios has the - very useful - feature of “parent hosts”. If it deems a host A being down, it first checks its parent host, B, and reports A only as down if B is up. This goes back recursively until a host with state “up” is found and only the first “down” host is actually reported. This keeps on-call people from being bombed with alerts in case of major network outages and makes sure that the alerts that are actually sent out do reasonably accurately describe the actual outage. As an individual who has some “external” servers in various data centers on the Internet, I would like to not be alerted multiple times that my servers at ISP C, D, and E are down if there is an outage at the ISP F hosting my Nagios installation or at one of the various exchange points temporarily rendering the servers unreachable (without me being able to do anything). The solution sounds easy but is surprisingly hard. How one could solve the issue For each host in the manual nagios configuration, do a traceroute and generate host stanzas for each of the hosts found:
$ sudo traceroute -I icmp -n -A 213.239.240.200
traceroute to 213.239.240.200 (213.239.240.200), 64 hops max, 28 byte packets
 1  <snip>
 2  217.243.221.150 [AS3320]  1 ms  0 ms  0 ms
 3  62.154.10.106 [AS3320]  1 ms  2 ms  2 ms
 4  217.239.40.226 [AS3320]  51 ms 217.239.40.234 [AS3320]  3 ms 217.239.40.226 [AS3320]  60 ms
 5  193.159.226.2 [AS3320]  3 ms  3 ms  3 ms
 6  213.239.240.200 [AS24940]  6 ms  7 ms (TOS=238!)  8 ms
$
This sounds easy and surely can be accomplished in a few hundred lines of perl. However, this makes it necessary to parse Nagios configuration if one wants to obtain the list of configured hosts directly from there. Unfortunately, the syntax of Nagios configuration is “historically grown” which makes it quite illogical and awfully hard to parse - this would easily double the size of the traceroute-based configuration generator. Additionally, the (real) traceroute given above shows three other issues that might arise from such a setup. Dynamic Routing Network paths are not as static as a traceroute suggests. On the Internet, the ISPs use BGP to connect their networks together and to ensure connectivity. In case of network reconfiguration (which may be caused by desired technical or policy changes or actual failures), the traceroute to a given network host may change. Considerable experience is needed to judge whether a change in the traceroute is only short-term while one looks at it, or long term meant to stay. Of course, only long term changes should be reflected to Nagios configuration. The possibility of changes makes it necessary to distinguish between “manually” and “automatically” generated Nagios host entries and to have a mechanism to quickly re-generate the “automatic” entries according to the current situation found on the network. Alternative Network Paths If you look at hop 4 of the traceroute given above, you see two IP addresses. This is a common “one packet left, one packet right” setup where two identical lines are sharing the load and this one sees different interfaces of the next hop router. Which IP address should one put into the parent host declaration then? Later Nagios versions supposedly are able to handle multiple parents to a host, but the docs of course don’t cover this complex configuration and I didn’t yet find the time to find out how this mechanism works and whether it can be used to represent a network layout like that in Nagios. Unpingable Routers For some network operators, ICMP echo request packets (ping) pose a significant load on their network equipment. To remedy that load, they have disabled ICMP echo reply transmission on these machines to keep the load down. This makes the router still show up in a traceroute (as this is done by sending packets with short TTL values and making them expire in transit, resulting in an ICMP TTL exceeded message being sent back), but it cannot be pinged. Hop Number 5 on the traceroute above is such a box: It shows up fine in the traceroute, but pinging it will always return “DOWN”. Currently, I simply omit such hosts from the Nagios configuration, but this may result in extra and inaccurate alerts once the unpingable host itself fails. A different possibility would be to check the host’s availabilty by sending an appropriately TTLed packet either to the host itself or to a host that is “behind” that host. The latter introduces even more complexity since one needs to find out a host “behind” the unpingable box, but this may be necessary since in the case of Hop Number 5 above, the IP address of the hop does not even seem to be in AS3320’s internal routing and tracing to the hop itself stops well before the expected place of the host regardless of from where one traces. Conclusion This is a surprisingly hard issue if one wants to do network monitoring while generating accurate alerts. Surely, it’s a topic that needs to be addressed in network monitoring, so I am quite interested in how other people tackle this. Please comment! Thanks in advance.

Wouter Verhelst: Reading the "Metro" on my Hanlin V3

It was recently brought to my attention that the free newspaper "Metro" has an online version, where you can download the entire newspaper in PDF form; or use it in an online version, which consists of a series of images of each page, where if you click on an article, the usemap of that image is a link to the relevant article in HTML form. Of course my hanlin doesn't have a webbrowser (it has a somewhat limited HTML parser, but that does not understand even hyperlinks, let alone usemaps), so the PDF version is what I need. Unfortunately, the only option which "Metro" provides is a set of one-page PDF files. They are somewhat readable on my hanlin (someone with worse eyes than my own would probably have trouble reading the small letters, but for me it's not a problem); however, the fact that rather than just using the 'page turn' buttons on my hanlin I have to close the current file, open the next, and reconfigure the zoom all over, means that this multi-PDF thing isn't exactly great, and that I'd prefer just getting one PDF file instead. So I got to work, and tried out a few things. There doesn't appear to be anything to merge multiple PDF files into one. There is 'psmerge', but that only does PostScript files, not PDF ones. But that's not a problem, because PDF files can easily be converted to postscript and back, right? Well, no. The first pdf-to-postscript converter that I tried was 'pdf2ps', a wrapper script around ghostscript. Unfortunately, using that results in some ugliness: original PDF
source, as rendered by Xpdf output of
pdf2ps, as rendered by gv As you can see in the second image, using ps2pdf results in some quality loss. Ghostscript apparently doesn't understand the letter 'A' very well, and (more importantly) loses the anti-aliasing that is part of the pdf file. Converting this file back to PDF and storing it on the hanlin results in an unreadable text. But it gets worse. Once I had converted all those PDF files to postscript using pdf2ps and merged them with psmerge, the output was gibberish. Or Klingon, take your pick: psmerge
output, as rendered by gv In case you were wondering, yes, that is the exact same fragment (counting lines sucks) at the exact same zoom level. Clearly this was a dead end. I found that there was a second pdf-to-ps converter, called 'pdftops', which uses the xpdf code base to do its thing. This converter produced clearly better PDF output; I could not see any difference between the Xpdf rendering of the original file, and the gv rendering of the pdftops output. Also, the psmerge output does not garble things as badly: better psmerge
output, as rendered by gv Sadly, it still loses anti-aliasing, and thereby the readability of the document on my ebookreader device. Moreover, the pdftops output seems to confuse psmerge, to the extent that it hangs on some pages. Another approach I tried was to import the pdftops output into scribus, and create a multi-page PDF file with that program. Unfortunately, however, scribus does not seem to like the pdftops output. Xpdf also has a 'pdftoppm' converter. With that, I was able to create a multi-page .pdf file that was not garbled, and did still contain anti-aliasing. Unfortunately, since the PPM format is a raster image format, the PDF file that is created in this way does not scale very well, resulting in artifacts and, again, an unreadable PDF file on the hanlin device. So it appears that for now I'll be stuck with storing multiple files on my device. Sigh. I wish that wouldn't be necessary... Update: So I needed pdftk.

29 October 2008

David Welton: Slicehost + Ubuntu Intrepid Upgrade

As I wrote previously, I started looking around for a new hosting provider, as Layered Tech is jacking up their prices too much. This isn't something I consider 'fun', as it will involve a lot of time swapping things over, with the potential for problems, but the amount of money disappearing every month is too much to ignore at this point. After some dithering, I finally went ahead and spent some money for a minimal slice at Slicehost. Linode seems pretty competitive too, but I more or less flipped a mental coin. I was actually ready to go ahead the other day, but I stopped to think some more after seeing that Slicehost has been acquired by Rackspace. In terms of seeing hard working people rewarded for their efforts, I am of course happy for the guys at Slicehost, but it also leaves a few questions:
  • Sometimes, perhaps not immediately, but after a while, there is less incentive to bust your butt if you cashed in in a big way.
  • Rackspace is not my kind of hosting outfit: they don't even list their prices where people can see them, like Slicehost, Layered Tech, and most everyone else does. I've heard they're good, but quite expensive. I don't need 'managed' servers - I can do that myself.
  • The FAQ says that Slicehost will continue as a going concern in its own right, but there's always the chance that they were acquired for their tech, and that the FAQ just says that they will continue to operate to keep their current clients from bolting.
Of course, good things could come out of the deal too, so I decided to put my paranoia aside and test the waters. There are a lot of things to like about a system like Slicehost's: it's relatively quick and easy to set up new slices, fiddle with the resources that each slice has, and so on, compared with a physical server. Furthermore, they have a reputation for not 'overbooking' their servers, which is why I wanted to get away from a VPS in the first place, many years ago. Setting up a slice was very quick and easy, but unfortunately got off to a rough start: their "fraud detection" system had a problem with me being in a different place than my billing address. Luckily, they were very fast at correcting the problem. As an aside, I think that fraud detection is probably best left to the credit card companies, although of course I don't really know their business, so maybe that particular check has saved them some money/frustration in the past. Upgrading the Slicehost Ubuntu to Intrepid In any case, once I got it up and running, I wanted to test out the upgrade to Intrepid, as that's going to be out in less than a week, and I might as well upgrade while I have 'fresh' servers to play with, and prior to installing a bunch of stuff. It went very well. I did a do-release-upgrade -d, hit 'yes' to everything, rebooted, and everything was fine! Admittedly, I don't have anything running on the server yet, so that kept things short and sweet, and didn't require the upgrade of various systems which might have had troubles, but so far so good If you'd like to sign up for Slicehost, it looks like they give you a referral code. Mine can be utilized by going here.

David Welton: Oslo - October 20-22

A quick note: I'm going to be in Oslo from the 20th to the 22nd of October. I probably won't have time to meet anyone, unfortunately, but if you are up for it, send me an email with a cell phone number and we'll see what can be arranged. Also, I'm trying out Dopplr. I'm not a frequent traveller, but I really like to meet people when I do go someplace new: http://www.dopplr.com/traveller/davidnwelton

David Welton: Oslo - October 20-22

A quick note: I'm going to be in Oslo from the 20th to the 22nd of October. I probably won't have time to meet anyone, unfortunately, but if you are up for it, send me an email with a cell phone number and we'll see what can be arranged. Also, I'm trying out Dopplr. I'm not a frequent traveller, but I really like to meet people when I do go someplace new: http://www.dopplr.com/traveller/davidnwelton

David Welton: Making Applets Suck Less

With all the effort being poured into Javascript lately, it's pretty evident that that's the way forward. However, there are still some neat things you can do with applets that you can't do with Javascript. For instance, I use the microemulator applet to let people play with a simulated phone on the Hecl web site. Unfortunately, the user experience hasn't that good. While playing around with the code for that site, Firefox crashed a number of times, and I've heard other browsers have trouble as well. No fun. So it is with some satisfaction that I found this: https://jdk6.dev.java.net/plugin2/ It's an updated Java plugin that seems to work much, much better than the old one. Unfortunately it doesn't appear as if it will be in the new Ubuntu, as it was released recently, but I'm quite happy to have something a bit more stable to develop with. You can download that version of the Java SDK or JRE here: http://download.java.net/jdk6/ It's good to see this effort, even if it may be a bit late.

David Welton: Slicehost + Ubuntu Intrepid Upgrade

As I wrote previously, I started looking around for a new hosting provider, as Layered Tech is jacking up their prices too much. This isn't something I consider 'fun', as it will involve a lot of time swapping things over, with the potential for problems, but the amount of money disappearing every month is too much to ignore at this point. After some dithering, I finally went ahead and spent some money for a minimal slice at Slicehost. Linode seems pretty competitive too, but I more or less flipped a mental coin. I was actually ready to go ahead the other day, but I stopped to think some more after seeing that Slicehost has been acquired by Rackspace. In terms of seeing hard working people rewarded for their efforts, I am of course happy for the guys at Slicehost, but it also leaves a few questions: Of course, good things could come out of the deal too, so I decided to put my paranoia aside and test the waters. There are a lot of things to like about a system like Slicehost's: it's relatively quick and easy to set up new slices, fiddle with the resources that each slice has, and so on, compared with a physical server. Furthermore, they have a reputation for not 'overbooking' their servers, which is why I wanted to get away from a VPS in the first place, many years ago. Setting up a slice was very quick and easy, but unfortunately got off to a rough start: their "fraud detection" system had a problem with me being in a different place than my billing address. Luckily, they were very fast at correcting the problem. As an aside, I think that fraud detection is probably best left to the credit card companies, although of course I don't really know their business, so maybe that particular check has saved them some money/frustration in the past. Upgrading the Slicehost Ubuntu to Intrepid In any case, once I got it up and running, I wanted to test out the upgrade to Intrepid, as that's going to be out in less than a week, and I might as well upgrade while I have 'fresh' servers to play with, and prior to installing a bunch of stuff. It went very well. I did a do-release-upgrade -d, hit 'yes' to everything, rebooted, and everything was fine! Admittedly, I don't have anything running on the server yet, so that kept things short and sweet, and didn't require the upgrade of various systems which might have had troubles, but so far so good If you'd like to sign up for Slicehost, it looks like they give you a referral code. Mine can be utilized by going here.

28 October 2008

Russell Coker: The Latest Dick Smith Catalogue

I was just reading the latest catalogue from Dick Smith Electronics (a chain of computer stores in Australia). The first interesting thing that I noticed is that laptops are cheaper than desktops in all categories. For any combination of CPU power and RAM in a desktop system I can see a laptop advertised with similar specs at a lower price. Of course you won’t get such a big display in a laptop, but big displays don’t always work well. I just read an interesting review of LCD display technology [1] which states (among other things) that TN panels (which provide poor colors and a limited viewing angle) are used in all current 22 inch monitors! They state that the Dell 2007WFP (which I own) comes in two versions, I was fortunate to get the one that doesn’t suck. Based on that review I think I’ll refrain from all further monitor purchases until the technology gets sorted out and it becomes possible to reliably buy the better monitors at a decent price. The most expensive desktop system that Dick Smith advertised in their catalogue has a 22 inch monitor. It seems that with desktop systems being more expensive an increasing number of home users will use laptops instead, which will of course change the economics of manufacture. Maybe the desktop computer is about to die out and be replaced by laptops, PDAs, and mobile phone type devices (blackberries etc). Another interesting thing is an advert for a LASER pointer (it seems that they haven’t been banned as “terrorist weapons” yet). Being on special for a mere $27 is not the interesting thing, what is interesting is that the advert claims “projects up to 500m indoors“. I’m sure it will be handy if I ever have to give a presentation at the Airbus factory. But otherwise it seems quite unlikely that I will ever get an opportunity for a 500m indoor space. The prices on digital cameras have been dropping consistently for some time. Now they are selling a Samsung S860 (8.1MP with 3* optical zoom) for $98. This is (according to the specs at least) a very powerful camera for a price that most people won’t think twice about. I expect that an increasing number of people will buy new digital cameras every year the way white-box enthusiasts buy new motherboards! Hopefully people will use services such as Freecycle [2] to dispose of all their old cameras, to both avoid pollution and get cameras into the hands of more people. Very few monitors are being sold with resolutions greater than 2MP (1680*1050 is the highest you can get for a reasonable price). So an 8MP camera allows significant scope for cropping and resizing an image before publishing it on the web. Even the 4MP cameras that were on sale a few years ago (and which are probably being discarded now) are more than adequate for such use.

Ond&#345;ej &#268;ert&iacute;k: Google Mentor Summit III

On Sunday I checked out from the hotel and enjoyed the breakfast, basically the same as the first picture here. Which is cool, the best way to start a day is with a good breakfast in the morning.



At Google I finally met with Bart Massey, who works at the Portland State University where I was an intern (in 2005) and also he was sponsoring one SymPy GSoC student in 2007.


Then we started to port SymPy to Jython with Philip Jenvey. I first installed openjdk-6, which is in Debian main, then checked out Jython svn, typed "ant" and it just worked. This is so awesome, that we finally have a truly opensource and fully working java implementation in Debian. Philip fixed some bugs already, so SymPy imports just fine. Most of the tests run, but also quite a lot still fail.

We managed to isolate two bugs: issue 1158 and 1159. Then there was an annoying recursion bug, that took us several hours to dig into and I then had to leave at 6pm without a clue. But Philip kept saying, I just need to look more at the java stacktraces that Jython was generating and I'll figure it out. And he did! The next morning he filed an issue 412 in the PyPy tracker, because they seem to have the same problem. There are subtle differences in how to handle __mul__ and __rmul__ methods that are not well documented, so it works in CPython but fails in Jython/PyPy. I think we can fix that in SymPy too in the meantime, so I am very excited, because it seems SymPy will work on top of Jython soon.

There was also a git vs hg session with both hg and git developers. Since I recently switched from hg to git for all stuff where I can choose the vcs, it was very interesting for me. And I must say I am glad I switched. Mercurial works fine enough and with the recent rebase feature I think it is usable, but git just has a bigger momentum at the moment and it just doesn't get in my way.

During the day Steve sent an email to the GSoC list that Dirk forgot his jacket in the hotel room, so I said to myself, haha, that's a pity. And then I realized, oh shit, I left my jacket there as well. They didn't have it in the evening when I stopped there, but fortunately they sent me an email yesterday that they found it, so I'll stop there on my way back.

We also took couple of group pictures, here is just my lame attempt (I am sure people will soon post better ones):


On Monday morning I met with Fernando Perez, the author of ipython, he know works with Jarrod. Then I rented a car and came to visit Brian Granger and his family in San Luis Obispo.

We went to the beach in the evening:





And had a dinner together. Brian showed me his options pricing example he did with SymPy, so I made him create an issue 1175 with the code.
Then we played with the parallel stuff that Brian contributed to ipython and we tried sympy with it and to our disappointment, we found pickling bugs in sympy, it works with protocol 0, but not protocol 2, see the issue 1176 and 1177. This almost smells like a bug in Python itself, but I need to investigate that later more deeply.

Today is Tuesday and we'll work on doing atomic physics calculations with sympy. See also Brian's email to the sympy list about it.

27 October 2008

Romain Beauxis: About democracy (revisited)

After my last post about the ongoing debate on the new process to accept and handle debian project members, Rapha l Hertzog proposed an alternative organisation which could be a solution, but misses the most important fact: how the people in charge of the process would be chosed and dismissed. Rapha l Hertzog wants that we consider a group of skilled and trustfull people in charge of giving and revoking powers to lower class of developers, on a per-event basis. Their procedure would be an internal vote amongst this group, and they would be called the "Debian Community Managers". Although it could be a good idea, it really misses the MOST IMPORTANT FACT: how they are chosed and to whom are they responsible. As a reply to my comment, Rapha l propose that they would be elected at first, and then nominated/revoked on the basis of the decision procedure mentioned above. Not only is it exactly what have been in place for the last years in the various core teams, but also it is a dangerous naive vision of how powers and regulations must happen. Before the Debian project started to consider these issues, the various ideas to build a system where the majority is ruled by a minority and the means to prevent the most important issues has already been studied, in the field of Law and how to write a Constitution for a democratic system. The goal of a democratic system is to put some higher power in the hand of several, but prevent as much as possible any issue. And preventing means act before shit happens, not after. Furthermore, power has to be divided into several domains, where the people in charge of these domains are strongly seperated and share as less common interest as possible. Hence, the goal of a constitution is to weaken power and put barreers, even if this leads to slow and frustrating reform processes. In the same idea, elections, which redraw the positions, are planed on a regular basis, and if the previously elected people were good, they are relected, not the opposite, in which if they aren't good they are dismissed. Again, we prevent issues, so the requirement to be responsible for what you did is implicit. And, last, it is well known that the less direct a decision process is, the less democratic it is. Hence, one should always prefer direct elections to indirect nominations. Now, coming back to Debian, we can have a very similar distinction in the various powers available: Now, the propositions, and all the various discussions are motivated by either the fact that there are issues with some developper's behaviour, which falls into the justice side and for which we already have a wide history of long debates before we act whatever the act is, or the executive power to give rights and introduce a new member in the project. Yes, we clearly miss a role in the project for applying rules and sanctions and deciding about these issues. However, we really ought to fix it the right way. In particular, never should we, in any way, give any justice power to someone in charge of any of the other power, in particular core teams. And the same for the right to appoint new members. For instance, to name an issue, the Frans vs. Sven issue would have been really different if there wasn't any kind of personal interest and power in the people in charge of the decisions. In real social systems, the right to apply a sanction or give new rights is a very dangerous power. It suppose a high level of separation between your personal interest, and the decision you have to make, as well as a high level of responsability against the people who trust you. We used to claim that Debian was not a democracy, but a meritocracy where technical skills are the ultimate ranking for deciding who's good or bad when giving reponsabilities. I have never been convinced by this approach, which I consider, just as anarchy, as valid for a small group of developers, but completely naive for a big group. Now, it is clear that the issues that we are dealing with in this situation have nothing to do with technical skills. In particular, the original proposition creates a class of non technical contributors, documentation writers, or translators etc... Furthermore, if you look back in the past, there have been a lot of situation where the issue, and its solution had nothing to do at all with technical issues, like issue mentioned above, or the Duck Tank or whatever it was called. We really should drop this naive idea that technicals skills are the only way to put on decision on the project. That is no longer how it works, nor a solution to the human issues that we face now.

Rapha&#235;l Hertzog: Debian membership reform

Following Ganneff’s post to debian-devel-announce, several discussions have again started on the topic of Debian’s membership and several proposals have been made. Unfortunately none of these proposals try to resolve the underlying trust problem that has been growing over the years. Despite the NM process (or maybe due to it), we managed to give DD status to people who are motivated but whose technical skills are doubtful (at that point people ask for an example, and as much as I hate fingerpointing, here’s an example with #499201. The same maintainer created troubles with libpng during the etch release cycle and tried to take over a base package like mawk recently). With our current model, all DD can sponsor, NMU, introduce/adopt/hijack packages without review. This is fine as long as we trust the body of DD to contain only skilled and reasonable people. I believe that premise to be somewhat broken since Debian has become too big for people to know everybody and since the NM process had no way to grant partial rights to volunteers who were motivated but that clearly had not shown their ability to handle more complex stuff than what they had packaged during their NM period (like some trivial perl modules for example). Thus I strongly believe that any membership reform must provide a convincing answer to that trust problem before being implemented. I took several hours to draft a proposal last Friday and I’ve been somewhat disappointed that nobody commented on it. I hope to draw some attention on it with this blog post. The proposal builds on the idea that we should not have classes of contributors but simply two: a short-term contributor and a long-term contributor (those are called Debian Developers and have the right to vote). But all contributors can be granted privileges as they need them for their work and each privilege requires the contributor to fulfill some conditions. The set of privileges and the conditions associated all need discussions (but I have personal opinions here, see below). There’s however one privilege that is somewhat particular: it’s the right to grant privileges to other contributors. Handling it as a privilege like another is on purpose: it makes it clear that anyone can try to get that privilege and the procedure is clear. In practice, imagine that set of people as a big team encompassing the responsibilities split over DAM/AM/FD/DM-team and where all members can do all the steps required to grant/retire a privilege provided that 2 or 3 members agrees and that nobody opposes (in case of opposition a specific procedure is probably needed). I called that set of people the Debian Community Managers. It should contain only skilled and dedicated developers. One of their main duties would be to retain the trust that the project as a whole must have in all its members. They would have the powers to retire privileges if they discover someone that has not acted according to the (high) expectations of the project. Among the privileges would be limited upload rights (like DM have currently), full upload rights (like DD have currently although it might be that we want to split that privilege further in right to sponsor, right to package new software, right to maintain a package of priority > standard, etc.) and developer status (email + right to vote, once you can prove 6 months of contribution). There’s lots of stuff to discuss in such a proposal (like how to decide who gets what privileges among existing DD) but I think it’s a good basis and need some serious consideration by all the project members. The NM process is there only so that we can collectively trust that new members are as good as we expect them to be and trust can only be built over time so it’s good that we can grant privileges progressively. Some people believe that I’m reinventing a new NM process that will end up to be very similar to the current one. My answer is that the conditions associated to each privilege should be based on the work done by the contributor and the advocations that he managed to collect. It should not be a questionnaire like Task and Skills . This, together with the distribution of the power/work on many people, would render this system very different from today’s NM process. Some people believe that I’m copying Ubuntu when designing this since it’s somewhat similar to the process to become MOTU and/or get upload right to Ubuntu’s main component. Let me say that I’m not copying deliberately at least, I simply took the problem from the most important side. But remember that many aspects of Ubuntu have been designed by Debian developers that tried to avoid known pitfalls of Debian, and maybe they got some things right (or better at least) while doing this. Partagez cet article / Share This

25 October 2008

David Welton: Slicehost + Ubuntu Intrepid Upgrade

As I wrote previously, I started looking around for a new hosting provider, as Layered Tech is jacking up their prices too much. This isn't something I consider 'fun', as it will involve a lot of time swapping things over, with the potential for problems, but the amount of money disappearing every month is too much to ignore at this point. After some dithering, I finally went ahead and spent some money for a minimal slice at Slicehost. Linode seems pretty competitive too, but I more or less flipped a mental coin. I was actually ready to go ahead the other day, but I stopped to think some more after seeing that Slicehost has been acquired by Rackspace. In terms of seeing hard working people rewarded for their efforts, I am of course happy for the guys at Slicehost, but it also leaves a few questions: Of course, good things could come out of the deal too, so I decided to put my paranoia aside and test the waters. There are a lot of things to like about a system like Slicehost's: it's relatively quick and easy to set up new slices, fiddle with the resources that each slice has, and so on, compared with a physical server. Furthermore, they have a reputation for not 'overbooking' their servers, which is why I wanted to get away from a VPS in the first place, many years ago. Setting up a slice was very quick and easy, but unfortunately got off to a rough start: their "fraud detection" system had a problem with me being in a different place than my billing address. Luckily, they were very fast at correcting the problem. As an aside, I think that fraud detection is probably best left to the credit card companies, although of course I don't really know their business, so maybe that particular check has saved them some money/frustration in the past. Upgrading the Slicehost Ubuntu to Intrepid In any case, once I got it up and running, I wanted to test out the upgrade to Intrepid, as that's going to be out in less than a week, and I might as well upgrade while I have 'fresh' servers to play with, and prior to installing a bunch of stuff. It went very well. I did a do-release-upgrade -d, hit 'yes' to everything, rebooted, and everything was fine! Admittedly, I don't have anything running on the server yet, so that kept things short and sweet, and didn't require the upgrade of various systems which might have had troubles, but so far so good If you'd like to sign up for Slicehost, it looks like they give you a referral code. Mine can be utilized by going here.

Ond&#345;ej &#268;ert&iacute;k: Google Mentor Summit I

I flew via Atlanta and had only about an hour to my next flight to San Francisco, so after my last experience, when I went to the immigration, got stuck in the line for more than an hour and then had to run to catch my flight, I decided to try a different strategy this time: first run to the immigration and then walk to my gate. Unfortunately I was sitting near the back of the plane and I got out among the last ones. Fortunately, it was several hundreds meters to the immigration, so I run as fast as I could and I managed to get there as the first one and everything took about 5 minutes. That was just awesome, I finally figured this out.

Jarrod was waiting for me at the airport, went to his place. Here's his cat:



On Friday we did some work and then went to the Golden Gate, the traffic was quite dense:




Alcatraz:


San Francisco:




Then we had a cofee in San Francisco and went to Silicon Valey, Jarrod drove me around a little bit to see SLAC, Stanford campus and other things. In the evening we went to the common pub with other mentors and Google guys, where we for example met with Robert Bradshaw.

Today I am looking forward to meet with all the people I know from mailinglists, Debian and other places.

24 October 2008

Jon Dowland: dissident

The "Art is Resistance" Alternate Reality Game recently posted a list of "dissidents, including myself (as jmtd). My case number is apparently 24.24.2.1447.

Lucas Nussbaum: Developer status: problems and solutions

For a minute, I’d like to think about The Announcement in terms of problems and solutions. Problem: We don’t have enough non-developing contributors in Debian
Solution in The Announcement: Give them special, official, statuses in the Debian community, so they are rewarded for their work.
Alternate solution: Are they really looking forward this status? Aren’t we just thinking that they are just as power-hungry as we are? Last time I checked, the Debian community wasn’t really welcoming non-developers. It isn’t a problem of official status. We have to work on ourselves to make Debian a better place to be for non-developing contributors. Also, giving those people a second-class citizenship, that won’t be widely recognized — “Debian Member” instead of “Debian Developer”, isn’t going to help. (or even worse, “Debian Contributor”, which doesn’t give any right except an email address). Problem: We have a trust and security problem. It’s difficult to give upload rights and access to Debian machines to more than 1000 people, with some of them not being active anymore, or not having much experience with security (in the case of non-developing contributors).
Solution in The Announcement: Remove rights based on classes of developers, so the group of people having full access stays within manageable boundaries.
Alternate solution: Have fine-grained control on who can do want. Problem: DMs can’t get access to Debian resources because their keys aren’t managed by the Debian keyring team, but by another team.
Solution in The Announcement: Keyring managers will take care of the Debian Maintainers keyring too. In the past, the keyring managers were the cause for huge delays in the NM process. It has improved a lot recently, but it doesn’t mean that it will always be like that.
Alternate solution: Make the DM keyring team, the keyring managers, and DSA, work together on a solution. it doesn’t sound impossible. If that’s necessary, merge the DM keyring team and the keyring managers. Problem: DM was done without the blessing of most of the members of the loosely defined group of powerful DDs (sometimes described as “cabal”).
Solution in The Announcement: Drop it, replace it with something very similar, but originating from the cabal.
Alternate solution: Why not keep it? Problem: Clueless DDs advocate clueless contributors for DMs. Then those DMs upload crap to the archive. (Note: I don’t necessarily agree with that problem, that’s just a problem that the announcement tries to solve)
Solution in The Announcement: Require answering a few questions before becoming a DM. Remove the right for every DD to decide which DM can upload which package. Give that control only to the NM committee.
Alternate solution: If we can’t trust one DD to take the good decision about advocating someone for DM, we could require that each person needs to be advocated by two different DDs. Or three, four, five. Asking the DM to answer a few questions, whose answers he can probably get by asking around on IRC, is not going to improve the average level of our DMs or DDs.
I believe that there are real problems that need to be solved, but that the decisions announced don’t solve the real underlying problem. Here is what I’d like to see: All rights would default to “NO”. DDs would be allowed to change their rights for everything, while DMs would only be allowed to change specific rights. Of course, we need a secure way to change rights (HTTPS+confirmation using GPG, like the current “sudo password” thing?). But that would help with the security problem (you wouldn’t have access to all Debian hosts without enabling it). In fact, we already have that: many hosts require a switch to be enabled in LDAP before you can login, and we have tons of Unix groups to restrict access to specific areas of our infrastructure. I’m just proposing to extend that to basically everything, and to allow people to grant themselves some rights without going through DSA.
  • Modified NM process for non-developing contributors, that would still include many P&P and T&S questions, but would give “Debian Developer” status without any upload rights. That way, non-developing contributor would have an official status, and will still be able to write “Debian Developer” in their resume. Giving a different status to those people just because we don’t think that they should have the right to upload any package doesn’t feel right.
  • Is that enough? Probably not. We need to be more welcoming towards non-developing contributors, which is a social problem, not a technical one. But we are not going to solve that in a GR.

    Next.

    Previous.