Search Results: "ema"

27 January 2012

Raphaël Hertzog: People Behind Debian: Josselin Mouette, founder of the Debian GNOME team

Josselin Mouette is one the leaders of the pkg-gnome team, he takes sound technical decisions and doesn t fear writing code to work-around upstream issues. He deserves kudos for the work he has put into packaging GNOME over the years. He can also be very sarcastic (sometimes he even enjoys participating to flamewars on debian lists), and there are quite a few topics where we have long agreed to disagree. But this kind of diversity is also what makes Debian a so interesting place Read on to learn more about the pkg-gnome team, its plans for Wheezy, Josselin s opinion on the GNOME 3 switch, and much more. Raphael: Who are you? Josselin: I am a 31 years old Linux systems engineer. I started in life with physics, which I studied at the ENS Lyon. I started a thesis on experimental and numerical models for optoelectronics, but when it became clear that research was not for me, I abandoned it and accepted a job at the CEA, which holds the largest computing center in Europe. Working on these machines has been the most awesome job ever (except for it being near Paris). After that I worked a bit on system monitoring technologies. I am married, currently living in Lyon, and working for EDF (the French historical electricity company) on scientific workstations using Debian. EDF is using Debian on more than a thousand workstations and holds the fastest Debian supercomputer in the world (200 Tflops), which makes it another obvious place for Debian developers. Raphael: How did you start contributing to Debian? Josselin: I discovered Debian in 1999 while studying at the ENS, which is one of the biggest nests of Debian developers while being a small place, it is producing almost one Debian developer per year on average. After wondering for a while what it could be useful for, hacking on a slink snapshot made me think that it was for, well, everything except for gaming. Later, in 2002, when I was working on optoelectronics computing codes, I started to package them for Debian in order to make them easier to install, for us as well as other labs over the world. I started the NM process, and it was going smoothly but also going to take time. However, at that moment, the frozen-bubble game went out and made quite some buzz. Since I knew a guy who knew the game s developer, he asked me to package it. The package found 3 sponsors in a very short time and was fast-tracked into the archive at a speed that was unseen before. After which the NM process was completed very quickly. At that time, I was a heavy WindowMaker user, but I didn t like the direction the project was taking (actually, I wonder if there was one). GNOME was starting to become attractive, but its packaging in Debian was very ineffective, with many inconsistent packages maintained by people who didn t ever talk to each other some of them didn t speak English, and some of them didn t talk at all. Together with awesome people, among which Jordi Mallach, Gustavo Noronha Silva, JHM Dassen, Ross Burton and S bastien Bacher, we started the GNOME team in 2003, introducing consistent packaging practices, and initiating synchronized uploads. Releasing a completely integrated GNOME 2.8 in sarge was a considerable achievement; proving (together with the Perl team) that a team was the best way to maintain large package sets changed the way people work on Debian.
Proving [ ] that a team was the best way to maintain large package sets changed the way people work on Debian.
Raphael: You re one of the most active contributors of the team which is packaging GNOME for Debian. What would you suggest to a new contributor who would like to help the team? Josselin: There are several ways to contact the team, but the recommended one has always been IRC. We hang on #debian-gnome on the OFTC network, so just come around and ask for us. The real question is what you want to do in the team. Of course, most new volunteers want to help packaging the latest and greatest version of GNOME into unstable as soon as possible, but unless they already have Debian background, this is not the easiest task. Since there are already people working on this, the big packages are usually waiting on dependencies. I used to direct newcomers towards bug triage, but it is a tedious task and I m now convinced that our huge bug backlog will never be dealt with. The most useful thing to do for newcomers now is probably to find a GNOME or GNOME-related package that needs improvement or is lagging behind, and simply try to work on it. You can also come and fix the bugs you find annoying. Find a patch on the GNOME bugzilla, or cook it yourself, propose it, and if it s worthy enough you ll soon get commit access.
Our huge bug backlog will never be dealt with.
At this point I feel worth mentioning that if no one answers in 10 minutes, it doesn t mean that no one will answer in 2 hours, so please stay on the channel after asking. Raphael: There s been some controversy about GNOME 3 and the direction that the project is taking. What s your personal stance on GNOME 3? And what s the position of the pkg-gnome team? Josselin: The controversy is not new to GNOME 3, but the large-scale changes made with it have put it more prominently. The criticism usually boils down to a few categories:
  1. General lack of configurability
  2. Strange design decisions
  3. Red Hat centric development
  4. Hardware requirements
  5. Change resistance
The lack of configuration options has been an ongoing criticism since GNOME 2.0 has decided to rip off most of them. Of course, when the control center was redesigned again for 3.0, there was a surge of horrified exclamations from people who missed their favorite buttons. On this topic, I fully concur with GNOME developers. The configuration option that is useful for you is not necessarily useful for someone else. Of course, sometimes developers go a bit too far, but the general direction is right. At work, we found that only a minority of users actually configure anything on their desktops: they just want something that works to launch their applications. Apple and Google have sold millions of devices by making them the simplest possible and without any configuration. Design decisions are, on the contrary, individual decisions, and each of them, while having reasons behind it, can be questioned. I remember seeing a lot of complaints when the OK and Cancel buttons were reversed in dialog boxes, something that nobody questions anymore. GNOME Shell is full of such changes; some are easy to get accustomed with, some others just make eyebrows raise. The most obvious example is the user menu in GNOME 3.2, which contains an entry to configure your Google account, but no entry to shutdown the computer. Both decisions were taken independently, each of them with (good or bad) reasons, but the result is simply ridiculous. The default configuration in Debian will contain an extension to make it a bit better, but on the whole we don t intend to diverge from the upstream design, on which a lot of good work has been done.
On the whole we don t intend to diverge from the upstream design, on which a lot of good work has been done.
Point 3 is more complex. Red Hat being the company spending the most on GNOME, it is obvious that their employees work on making things work for their distribution. An example is the recurring discussions about relying on system services that are currently only implemented by systemd. Since there is a lot of (mostly unjustified) resistance against systemd in Debian, and since it won t work on kFreeBSD anyway, someone needs to develop an alternative implementation of these services for upstart and sysvinit. Everything is in place for someone else to do the job but it has to be done, and this can be frustrating. Especially since it can also be hard to integrate changes needed for other distributions . Hardware requirements are mostly a consequence of the previous criticism: there s hardware that most distributions just don t want to bother supporting. We ve seen it in squeeze with the introduction of a hard dependency on PulseAudio. The Debian GNOME team (together with the Gentoo maintainers) made this dependency optional, carrying heavy patches, in order to cover the cases where it does not work. Now that it has gained more maturity, making this effort obsolete, the new tendency is to require 3D acceleration. For various reasons, it is not available to everyone . On this matter, the position of the Debian GNOME team has always been to support as much different configurations as possible with reasonable effort. Thanks to efforts from the incredible Vincent Untz, upstream supports a so-called fallback mode , which is the GNOME panel from 2.x with a lot of its bugs fixed. We intend to support this mode for as long as reasonably possible in Debian, possibly even after upstream ends up dropping it. However, other applications are going to require 3D because GStreamer is moving to clutter too, affecting video playback performance on non-accelerated systems . For epiphany this is not a problem; only embedded video will be affected. But for totem, this is a major issue; because of that we will probably keep totem 3.0 in wheezy. Finally, there is a natural human tendency to dislike change (I have it too), and it applies a lot to desktop users habits. Needless to say a change of such a scale as introducing GNOME Shell can trigger reactions. However, I don t think it is reasonable, because of this resistance, to keep gnome-panel 2.x in Debian. This would be a lot of work on obsolete technology, and would prevent the upcoming removal of a lot of deprecated libraries. This time is much better spent improving gnome-panel 3.x in Debian and keeping the fallback mode great. One of the change that was made in Debian was to make it easier to find, being available as GNOME Classic directly from the login manager, instead of having to find it in an obscure configuration panel. In all cases, I would recommend to actually try GNOME Shell for a few hours before ditching it. I had never been accustomed to a new environment as quickly ever before.
In all cases, I would recommend to actually try GNOME Shell for a few hours before ditching it.
Having seen several of my GDM patches reverted without a warning, I know we are not finished with carrying patches in Debian packages.
Scientific workstations are a non-trivial example, since there is a measurable effect of using 3D in the window manager on heavy 3D applications.
On the other hand, on accelerated systems, this feature should end up improving performance a lot. Raphael: What are your plans for Debian Wheezy? Josselin: The first goal of the GNOME team is, of course, to provide again a great desktop environment to work on. For wheezy it will probably be based on GNOME 3.4. There also needs to be some work on package management interfaces. Upstream bases everything on PackageKit, but it is not as featureful as the aptdaemon Ubuntu technology. If I have time, I would also like to improve HTTP proxy support, since currently it is based on a stack of terrible hacks. Raphael: If you could spend all your time on Debian, what would you work on? Josselin: Obviously I would like to make GNOME in Debian even better. That would imply working on underneath dependencies (what we now like to call plumbing) to make sure everything is working great. This would also imply working more as GNOME upstream to make it more suitable for our needs. I would also work on large-scale improvements on the distribution, like conditional recommends which I d love to see implemented , or automatic build-dependency generation. I would also work on the installer to make it better for desktops machines. The idea is to automatically install language packs, or glues between two packages when both packages are installed. Raphael: What s the biggest problem of Debian? Josselin: The obvious answer is the same as the one most people you interviewed before gave: not enough members in core teams. A lot of developers join Debian to work on a small number of pet packages, and don t necessarily want to be involved with existing teams. It is probably still not obvious enough that the primary way to start contributing to Debian is to join an existing team. But if there is one thing that is preventing Debian from gaining more momentum now, it is a completely different one: the too short support timeframe. 3 years is really not enough for corporate users. One year to migrate from one version to another is too short, and it is not possible to skip a release. It is definitely possible to change that with reasonable effort: the long-term support after 3 years doesn t have to cover the same perimeter as the short-term one. For example, we could upgrade the kernel to the version in the current stable release, and stop fixing all non-remote security holes. The important thing is to cover the most basic needs: companies are ready to take the risk of having less support if it allows skipping a version, but not the risk of having no support at all. And even more important is to say that you do something. Red Hat says they support a release for 10 years, but of course after 5 years the supported perimeter is extremely small.
3 years [of support] is really not enough for corporate users.
Long-term support will not magically fix all problems in Debian, but it will bring more corporate users into the picture. And with corporate users come paid Debian developers, who can work on critical pieces of the system. Debian was built on the synergy between individuals and companies, and in recent years perhaps as a reaction against what happened with Ubuntu we ve kind of forgot the latter. A lot of individuals have joined the project, and they are actively working, for example, on shortening the release cycle, which goes against the interest of professionals. We should embrace again such users and developers, and that means adapting to the current needs of larger entities. Raphael: You re the maintainer of python-support, a packaging helper that was competing with python-central. Both helpers are now deprecated in favor of dh_python2. Does this mean that the situation of Python in Debian is now sane? Or are there remaining problems? Josselin: dh_python2 (and the Python3 version, dh_python3) has a sane enough design. It fixes a lot of issues in python-central and also python-support, at the expense of somehow reduced functionality for developers. However, just like the previous tools, it merely works around design mistakes in the Python interpreter. For example it is not possible to split binary modules, pure-Python modules and byte-compiled modules in different directory trees, like Perl does although PEP 3147 introduces a way to do so. There is still no sane and standardized way to deal with module versions. There is no difference made between the module (which is a part of language semantics) and the file containing it (an information which depends on the implementation). Developers heavily rely on introspection features and make assumptions based on the implementation, that make it impossible to work around problems with module files. Such problems are not restricted to Python. Those who fought against Ruby gems could tell even worse stories. While introducing GObject introspection packages in Debian (they can be used in JavaScript and Python to provide modules based on GObject libraries), I was pleased to see a clear distinction between file and module, but I was again struck by the fact you are not forced to declare API versions in your Python/JS code. In all cases, there is no reliable way to detect runtime dependencies in a given Python or JavaScript file, which leaves the maintainer to declare them by hand, and of course, often be wrong about them. Add to that the fact that most errors cannot be detected before runtime. For all these reasons, and while still being fond of Python for scripts and prototyping, I ve become really skeptical of using purely interpreted languages to write real applications. Some GNOME developers are moving away from Python and JavaScript, mostly towards Vala; I can only approve of that move and hope the same happens to other projects. Raphael: Is there someone in Debian that you admire for their contributions? Of course there is the never-sleeping, never-stopping, Michael Biebl who can upload a whole GNOME release in a single week-end. But there are a lot of awesome people who make Debian something that simply works. I could talk about Cyril Brulebois from the X strike force, Julien Cristau from the release team, Sjoerd Simons for his sound advice and work on plumbing, Luca Falavigna who is so fast at processing NEW, to quote only a few of those I work with frequently. And of course, Jordi and Sam for their humor.
Thank you to Josselin for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Note that you can find older interviews on http://wiki.debian.org/PeopleBehindDebian.

Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Google+, Twitter and Facebook .

8 comments Liked this article? Click here. My blog is Flattr-enabled.

Craig Small: Unlucky sometimes

Sometimes life throws little curves at you to see if you are still awake, today has been one of those days. fglrx is (apparently) fixed I ve had a long-running problem with fglrx on my laptop. The problem stems from ATI closed-source drivers with one of those laptops that has an ATI and Intel driver. It means I am basically using the slow Intel chip only. This morning I had enough and backed up my home and started to rebuild the laptop with debian 6.0.3. So I kicked off the very very slow process of reformatting the crypto drive (it has taken 5 hours and still going) let it gurgle on its merry way and started to read my email. One of the emails was that my bug about fglrx not working is closed, apparently it is fixed. If I had read that 10 minutes earlier, a simple apt-get install fglrx-driver would of perhaps fixed it; oh well. My problem is now is do I move to the latest driver and hope their fix is my fix or leave it with some ancient version? My preference is the former; I only hope it works! psmisc 22.15 and buffer overflows psmisc has a program called pstree which prints the set of processes in a tree fashion. It hasn t changed much for quite a while. I released version 22.15 and the Debian package 22.15-1. 22.15-1 I also adopted the harden CFLAGS as suggested for procps. I was a little surprised that I received an important bug. The report was saying I had a buffer overflow introduced in 22.15-1, but no relevant code had changed. The compiler options had done their job and stopped a buffer being overflowed. But where exactly was the overflow? Running gdb on pstree quickly showed that it was line 267 of pstree.c which uses strcpy(). That function set off warning bells. The relevant code is:
    PROC *new;
 
    if (!(new = malloc(sizeof(PROC))))  
        perror("malloc");
        exit(1);
     
    strcpy(new->comm, comm);
Now comm is the short command name you find in /proc//stat. It is fixed in the kernel at 16 characters. The PROC structure has this field as 17 characters long, one extra for the NUL. I went and checked the Linux source and yes, it is still 16 characters long. The clue was in the name of the program that it died on.
#6  new_proc (comm=0x6111b0 " console-kit-dae ", pid=1571, uid=0)
  at pstree.c:267
That string is 17 characters long. The problem is that 16 characters is for the name only. If the name is in brackets or braces, then that 16 character limit doesn t apply. The buffer overflow bug has been there for a long time, but only with the compiler flags did it become visible. Given you need to read names out of the /proc filesystem and if someone can fiddle with that you have bigger problems it doesn t seem to be too much of an issue. It should be (and is in Debian 22.15-2) fixed but is a nice example of the compiler catching bad things.

26 January 2012

Aigars Mahinovs: Zombies. Wait, don t run away, yet

I never got the craze for the zombie this and zombie that. I saw the premise as thinner than vampire storyline which in my book only had a good day with Buffy franchise. But then I got Kindle and run out of stuff to read and decided to pick up something less lively and more shambling. And you know what I found there are a few great books hiding behind the zombie label. I read them in the worng order, so to help you guys out, here is the way I would recommend to check out if the zombie genre is interesting to you: Start with World War Z: An Oral History of the Zombie War. It is written as a historical book documenting the memories of people that have survived the world wide zombie apocalipse is a series of escalating short stories. This is a perfect bathroom or bus reading. Each story focuses on how people react (or fail to react) to a radical change in their environment. Zombies are a tool to expose the human condition for this book. The next you could read (or you could even skip this if it get a bit boring for you) is The Zombie Survival Guide: Complete Protection from the Living Dead this is a problem solving book, it basically takes a problem ( ZOMBIES! ) a goal ( I want to survive! ) and describes the logical steps needed to reach the goal in the context of the problem. It is very educational on the issue of problem solving, especially with unusual problems. But it can get boring for people if so, don t give up, but just skip this book for now and move on the the best of them all The third is a trilogy, two books are already published, the third is coming soon first books is Feed (Newsflesh, Book 1) and the second is Deadline (Newsflesh, Book 2). This is a rare gem of modern literature. Basically this is a book set in the world long time after a zombie outbreak (with a good reason why and how it happened) where people have already lived a few decades with the problema and the kids are very used to it. So the book is written from a perspective of a young woman and her brother, who were born after Rising and are now into one of the most dangerous profession journalism. :) Early on they become the first bloggers to be picked to follow a presidential election campagn (book was written before Obama campaign really took off) and it only get wilder from there. I really liked these books. So much so, that I will now explore the other zombie fiction books to see if there possibly is something good there as well. Any recommendations?

Ian Wienand: pylint and hiding of attributes

I recently came across the pylint error:
E:  3,4:Foo.foo: An attribute affected in foo line 12 hide this method
in code that boiled down to essentially:
class Foo:
    def foo(self):
        return True
    def foo_override(self):
        return False
    def __init__(self, override=False):
        if override:
            self.foo = self.foo_override
Unfortunately that message isn't particularly helpful in figuring out what's going on. I still can't claim to be 100% sure what the message is intended to convey, but I can construct something that maybe it's talking about. Consider the following using the above class
foo = Foo()
moo = Foo(override=True)
print "expect True  : %s" % foo.foo()
print "expect False : %s" % moo.foo()
print "expect True  : %s" % Foo.foo(foo)
print "expect False : %s" % Foo.foo(moo)
which gives output of:
$ python ./foo.py 
expect True  : True
expect False : False
expect True  : True
expect False : True
Now, if you read just about any Python tutorial, it will say something along the lines of:
... the special thing about methods is that the object is passed as the first argument of the function. In our example, the call x.f() is exactly equivalent to MyClass.f(x). In general, calling a method with a list of n arguments is equivalent to calling the corresponding function with an argument list that is created by inserting the method s object before the first argument. [Official Python Tutorial]
The official tutorial above is careful to say in general; others often don't. The important point to remember is how python internally resolves attribute references as described by the data model. The moo.foo() call is really moo.__dict__["foo"](moo); examining the __dict__ for the moo object we can see that foo has been re-assigned:
>>> print moo.__dict__
 'foo': <bound method Foo.foo_override of <__main__.Foo instance at 0xb72838ac>> 
Our Foo.foo(moo) call is really Foo.__dict__["foo"](moo) -- the fact that we reassigned foo in moo is never noticed. If we were to do something like Foo.foo = Foo.foo_override we would modify the class __dict__, but that doesn't give us the original semantics. So I postulate that the main point of this warning is to suggest to you that you're creating an instance that now behaves differently to its class. Because the symmetry of calling an instance and calling a class is well understood you might end up getting some strange behaviour, especially if you start with heavy-duty introspection of classes. Thinking about various hacks and ways to re-write this construct is kind of interesting. I think I might have found a hook for a decent interview question :)

Russ Allbery: Do symbols files make sense for C++?

I also posted this to debian-devel, which is where the discussion will be, but I wanted to post it on my journal as well so that others who may be interested will see it on Planet Debian. For those not familiar with Debian, this concerns the implications of a new feature for how Debian tracks shared library dependencies. I'm currently working on the Policy modification to document (and recommend) use of symbols instead of shlibs, but I'd only personally used symbols with C libraries. Today I decided that I should try adding a symbols file to a C++ library, particularly if I'm going to recommend everyone do it. I tried this exercise with xml-security-c, which is, I think, a reasonably typical C++ library. Not the sort of core C++ library that would sit at the center of the distribution, but a random software package that's in Debian because other things use it. The experience was rather interesting, and I ended up uploading the new version without a symbols file and continuing to just use shlibs. That's for the following reasons:
  1. The generated symbols file was HUGE. Hundreds of lines. This is a marked difference from the typical C symbols file, which is of quite manageable size. Some of that is that the library provides a lot of different classes, but some of it is that C++ just generates a lot of exported symbols. There's no way that I could do what I would do with a C library and understand those symbols, why they're there, and whether they are likely to have changed between revisions.
  2. Generating a reasonable symbols file was a pain. Generating an unreasonable symbols file that just contains all of the mangled symbols is largely mechanical and uninteresting, but that symbols file doesn't seem to me to convey useful information. So I did some scripting to translate the symbols back with c++filt, and add (c++) tags, and then try to understand what I was looking at and figure out whether I should sort the symbols list because the default sort is by mangled name, which is meaningless. This is a rather unappealing process. It's not particularly difficult, but it's very awkward and feels like it's missing vital tools.
  3. The resulting symbols file is incomprehensible to someone without strong knowledge of C++. It's full of opaque entries that don't make sense to the non-C++ programmer, which I suspect is a substantial number of people who package C++ libraries for Debian. I know enough C++ from school that I can evaluate security fixes, make simple patches, and review upstream changes, and I think that's all that should be needed to package things for Debian. But I'm deeply uncomfortable producing a symbols file on my own that contains entries for things that I know nothing about and cannot evaluate when they've last changed, like "non-virtual thunk to FooClass::~FooClass@Base".
  4. Once I had a symbols file that resulted in a successful build and that I could have uploaded, I started thinking about how I was going to maintain it. With a C program, I would change the symbols file versions when the underlying function implementation changes in a way that may not offer eqiuvalence, similar to bumping shlibs. I realized that I was going to have no idea when that happened, and the only way that I would maintain the symbols file would be to either trust upstream to maintain ABI equivalence and therefore only change the symbols file when upstream changes the SONAME, or not trust upstream to maintain ABI equivalence and therefore change all the versions with each new upstream release. That gives me exactly the same semantics as a shlibs file, so what's the point in having a symbols file?
  5. The exported symbols of the library contained many symbols that obviously weren't really from that library, but instead were artifacts of the C++ compilation process, things like instantiations of std::vector. Do those go into the symbols file? Do they change from architecture to architecture? If they disappear again, is that actually an ABI break? How do I know? It's all very mysterious, and while shlibs provides the same semantics as just ignoring this, at least I'm not then including in the package data, generated by me, things that I'm just blindly ignoring.
I came away from this experience thinking that I should revise the Policy amendment to say that symbols files are really for C libraries and for C++ libraries with either a tightly maintained symbol export list or maintained by a C++ expert, and that most C++ library maintainers should just not bother with this and use shlibs, bumping the shlibs version or not based on their impression of how good upstream is at maintaining ABI equivalence. But that feels like a result contrary to what I had previously thought was the intended direction, so I wanted to ask the Debian development community as a whole: am I missing something? Are these symbols files actually useful? Am I missing some trick to make them useful?

25 January 2012

Phil Hands: The future arrived

... a week ago, in the shape of Alexandra Daphne Scholz, my enchanting daughter. A proud father shows the world his adorable 6 day old daughter She seems to have inherited my hairstyle. Gundemarie Scholz (my wife) and she are both doing remarkably well, to the extent that they both keep surprising medical staff with their rapid progress. For those of you that care about such things: She was 2.9kg (6lb 6oz) at birth, and 49cm (1'7") tall ... or should that be "long", given that she's not doing so much standing up as yet? What she is doing is sleeping, breast feeding, and excreting ... rather a lot. Between times, it's mostly gurgling cheerfully, looking around at the world, and attempting to study her own hands -- I find her completely fascinating. That being the case, don't expect anything very sensible out of me for the foreseeable future (so no change there, eh? ;-) ) and because I cannot resist the urge, here are a few more snaps of her: Alexandra in the Neonatal unit, having a well earned nap when a day old Alexandra in the Neonatal unit, looking around at the world Alexandra snuggled up in bed this morning As you can see, at only one day old, it seems that she'd already managed to get hold of the Amulet of Yendor, and was having a well earned nap ;-)

Pietro Abate: new thinkpad x220

This summer my beloved thinkpad x301 died in a cloud of smoke. It was exactly 3 years and 20 days old while my warranty was valid only for 3 years. Now, don't tell me this is a coincidence. Anyway. After about 5 months, I finally managed to convince my employer to get me a new thinkpad, the x220. My specs includes a 128G SSD , 4G of RAM, 2.4Gz processor, camera and fingerprint reader. It's a pity that the x300 series is not in production anymore. They were light, with a solid battery and a large screen. The X1 just don't cut it. Even though it can be consider the successor of the x300, its battery life is just not enough for my needs. On the other hand, the x220 is a very nice machine : the screen is a bit smaller then the X300, but it is light, with a very powerful processor, good battery and it feels very solid. In my opinion lenovo should have packed the new hw of the x220 in the chassis of the x300, maybe with small compromise on the battery life (I got the big battery and I can squeeze almost 7 hs with a single charge) but clearly this was not a good business choice... Installing debian on this laptop is not immediate because none of the official debian installers are shipped with a recent kernel (as in 3.x series). Since with the official debian installer I cannot have neither the driver for the Ethernet card or the driver for the wirelles card, I opted to use a custom installer built by kmuto (http://kmuto.jp/debian/d-i/ ) . Using this installer the ethernet card is recognized immediately and it's easy to proceed with the installation as usual. Another option would have been to add the binary blog for the wireless chip, but apparently the deb installer supports only WEP auth, while all my access point are WPA. I didn't spend too much time on the wireless setup, so it might well be that is indeed possible to install using a WPA access point. Last time I installed a laptop, I used the automatic partition option to have lvm on top of a lucks encrypted partition, only to discover later that the default dimensions of the partitions were a bit too small for me. For example, by default the boot partition is only 256Mb. This is plenty if you want to have only one kernel image installed at each given time, but if you want more then one kernel, memtest and for example a grml rescue iso, it's easy to run out of space. So I proceed to manually partition the disc creating a boot partition of 512M, and using the rest as a luks encrypted device with lvm on top and 3 logical volumes : sys (15G), swap (4G) and home (the rest). For my daily use having 15G on my laptop for /usr, /var, etc should more the enough... Next step was to install the system. Since in recent times I got extremely pissed off with gnome 3, I've decided to dump it completely and go back to awesome. But since awesome all by itself is a bit sad, I paired it up with xfce. Everything works, except the automount, and I'm still trying to figure out how to make it work. Apparently is a consolkit problem... I'll write another post about the xfce4 + awsome setup soon... Today I've also started playing with the finger print reader. It seems working, but I haven't managed yet to use it in conjunction with pam for authentication ... more to come. And... On last closing remark : during the last 5 months I've used a dell latitude e6410 ... Gosh. I feel I'm on anther planet. The keyboard of a thinkpad give you pleasure, not pain, from 2 to 4G of RAM is a big jump and from a conventional HD to a SSD ... well... it seems I'm flyinggggg :) I've the impression my productivity just went up 50% !!! If you work with your laptop everyday get a good laptop. It is well worth the investment ...

24 January 2012

Michal &#268;iha&#345;: It's too early for GSoC 2012

It looks like with start of the new year, people started to ask for phpMyAdmin's projects for GSoC 2012. Given that we have no information whether there will be Google Summer of Code this year and whether we will be chosen to participate, these questions look quite premature. On the other side you can start contributing any time and if you are already known in the community, you might have better chance being accepted. So forget speaking about hypothetical GSoC projects for now and start fixing some bugs, which is something much more needed right now (given that 3.5 is in alpha state). If you are looking for more information, we have basic facts for developers on our website, there is also a wiki page mentioning some tasks for junior developers.

Filed under: English Phpmyadmin 0 comments Flattr this!

Charles Plessy: Strange Megumi

It is since two days that all my emails, @debian.org included, are in parked somewhere between the sender and my MX, as my server broke at a moment where I strictly had no time (dealine for a grant application). It is an OpenRD Ultimate with a second hand SSD. I was quite happy to have an ARM system available to test some packages like T-COFFEE, but it looks like I will have to do without. Apparently, the machine permanently reboots. I did not manage to access to the USB serial port; there are no 10 seconds where the USB port is not disconnected / reconnected. Both ethernet port LEDs blink together synchronously with the disconnections. Removing the drive did not solve the problem. The SSD itself is fine, and I did not find in the logs hints for software problems. Strange coincidence, the last log file that was modified is daemon.log, which indicates the connection of so-called megumi-PC to our unprotected, but to my knowledge never visited, wireless network. After this, nothing.

Gunnar Wolf: BugCon friends, are you trying to scare away 50%+ of the target audience?

You are scaring away much more than that. I just came across an invitation for BugCon 2012. BugCon is a Mexican conference devoted to computer security I cannot comment on its level or value because, although it's a topic that has long interested me, I must recognize each day I feel less of an expert, nowadays finding myself at the level of a "sysadmin who tries not to be too dumb for his own job security". Oh, and also because it would be completely off-topic for this post. If you look at Vendetta's (the main organizer) blog post, it will probably give you the impression that the conference is just an excuse for the afterparty: Lets go see some b00bs! Do you think your fellow female hackers will have any interest in joining a bunch of sex-starved, hormone-infested teenagers who only want to pwn a website and grab more pr0n? Do you think females will feel welcome (or even mildly safe) between you? I would not think so. And I also think you are alienating any professional who might have any interest in joining your community, be it as a member, as a mentor, or whatnot. I cannot right now do a coherent post on this topic, but I can reference you to what I have seen (and read) over the last almost 10 years, when the issue was first brought up to our attention. I am very glad to see that, at least in the Free Software area, there has been a real change of mindset. I hope you are in time to think about it and rectify. Oh, and not the description of an incident, but a very interesting and thoughtful take on this: [pdf] Interesting analysis by Hannah Wallach on the numbers and motivations of women in Free Software groups. I don't know if Hannah has published this in article form, but many interesting points can be understood by looking at the presentation. My good friend Vendetta: I don't mean this post (longer than what I originally intended) as a way to say you and the conference you are organizing for the third year (IIRC) already is unprofessional or targetted to pimply teenagers. I know the work you have put in it. I hope you see the points I'm trying to drive You are of course free to have whatever afterparty you have. But, if as the main organizer, you are giving the images of nice chicks at Hooters more weight and relevance than to the conference itself... you are doing yourself a disservice. I hope you can rectify it, and make BugCon attractive to hacker women as well.

Rapha&#235;l Hertzog: Answering questions of Debian users on various support channels

When you start your journey with Debian, you tend to have lots of questions. You ll find some answers in various documentations but there always are remaining questions. Those can be asked on various support channels: Those are the places where you can also start your journey as a Debian contributor instead of asking questions, you just have to answer questions of other users! Let me share some advice if you want to do some user support. User support is difficult It s not always an easy task. Some users are more skilled than others and there might be difficulties related to the language, English is not always the native language of a user who asks a question in English. Be respectful and courteous when you answer user questions, even if they made mistakes. You re effectively representing Debian and you should give out a good image of the project. If you don t have the patience or the time needed to do a good answer, don t reply and let someone else take care of this user. I invite you to read (and follow!) the Debian Community Guidelines. Avoid RTFM answers, instead you should show the users how they could have found (alone) the solution to their problem. We don t want to scare people away, we want to grow our community. But it s also rewarding In some cases, the problem reported by the user will be a real problem and you ll have an opportunity to file a good bug report, thus helping to improve Debian for everybody. Often, you don t even have the answer to the user s question. But you re more skilled than him/her to do researches on the web, or you know of a good documentation that might contain the relevant bits of information, in any case you re doing further research to help this user. In this process, you also grow your own skills since you re learning stuff that you didn t know yet. At least that s how I learned many things during my first year in the Debian community there s no reason why you couldn t learn lots of stuff that way, in particular if you also read the answers of other skilled people on those channels (it takes a bit of training to learn who are the skilled people though). I still believe that doing user support is one of the best ways to join the Debian community and to start contributing. It helps you to grow your skills, and to slowly progress from average user to advanced user .
If you want to start contributing to Debian, click here to subscribe to my newsletter and get future updates for new contributors. You can also follow me on Identi.ca, Google+, Twitter and Facebook.

5 comments Liked this article? Click here. My blog is Flattr-enabled.

C.J. Adams-Collier: SOPA response from Representative Rick Larsen, WA 2nd District

Below is an email I received from Representative Rick Larsen s office. I don t recall ever having identifying myself as PVT Adams-Collier in any communications with his office, though I could have done so at some point. Maybe. I also don t recall contacting his office directly concerning SOPA. I blogged a few things and mentioned him on facebook a couple of times, though. It is good to know that I don t always have to go directly to my representative to have my voice heard. For those playing along at home, I think the best course of action would be for producers to perform searches for their work on peer-to-peer file sharing search engines such as http://thepiratebay.org/ and http://scrapetorrent.com/. If producers find that their work is being distributed in a way which is contrary to their wishes, I advise the producer or their agent to contact the individuals operating seeds with high share ratio in order to ensure that they perform AAA before accepting new peers into the swarm. Those seed operators who refuse to negotiate reasonable AAA terms should be issued C&D orders, and legal action should be taken, should they fail to comply with the request. [edit 2012-01-24T05:27:44] It would probably be wise to impose a minimum fee of $5 for seed operators per content package they distribute. This will encourage them to pass this fee on to their subscribers in a legal and responsible manner. I recommend that it be distributed in the same fair and concise way in which the tickets are distributed for toll violations on the Tacoma Narrows Bridge. Ask me how I know. This will provide content producers a valuable distribution channel which is closely in touch with the consumers of the producers media. I believe that if the producer puts forth effort in good faith to develop and maintain relationships with key distribution points, we will see a situation where both content producers and consumers get what they want. The authorization phase of the AAA system could charge a Payment Card Industry or bitcoin account a reasonable fee. My employer produces content delivery appliances which would serve quite well the role of AAA gateway. For an example of this use case in action (sans account billing), I recommend contacting seeding participants of the Debian CD/DVD swarms. I will proxy such requests if there is interest. An alternative might be a tool developed for the online gaming community with a strange name: http://www.evenbalance.com/

Dear Pvt. Adams-Collier: Thank you for contacting me about the Stop Online Piracy Act (SOPA) and the Protect IP Act (PIPA). I wanted to update you on my views on this important issue. I am opposed to SOPA and PIPA in their current forms. I believe that these bills create unacceptable threats to free speech and free access to the internet. I have heard from many of you in Northwest Washington who are deeply concerned about the potential impacts of SOPA and PIPA. Online piracy is a serious problem that costs U.S. businesses billions of dollars. Government agencies must be empowered to stop and prosecute intellectual property thieves. But in doing so we cannot undermine freedom of speech or jeopardize the free flow of information on the internet. I will work with my colleagues to see that any final anti-online piracy legislation protects the internet and does not encroach on free speech rights. Please be assured that I will keep your thoughts in mind should I have the opportunity to vote on any legislation that would impact online piracy and internet freedom on the House floor. Again, thank you for contacting me. I encourage you to contact me in the future about this or any other issue of importance to you. Sincerely, Rick Larsen
United States Representative
Washington State, 2nd District
[edit 2012-01-24T05:28:09] Sincerely, C.J. Adams-Collier
San Juan County Democratic Central Committee (*whew*) PCO
San Juan County, Washington State, Orcas 3 District

21 January 2012

Lior Kaplan: PHP 5.4 @ Debian

PHP 5.4.0 is around the corner, with RC6 released this weekend. With the courtesy of Ond ej Sur it s already available in experimental. Earlier this week, Raphael Geissert, tested some PHP extensions with the new version and reported 16 bugs (severity: important) against those who failed to build with PHP 5.4. As they will become RC bugs when 5.4.0 will enter unstable (probably around mid February), I preferred to handle them sooner than later. The result is 3 patches sent (one already uploaded) and 5 NMUs done. During the process I poked through a lot of upstream SCMs. In the way I found out a trivial change was done in PECL s SVN two years ago for all the extensions located there, which got me suspicious regarding some extensions we have. For non PECL hosted extensions, I had to track if and when a fix was done and use it for building or do a trivial fix myself. Packages using the 3.0 (quilt) source format were extremely easy to fix, and I was quite happy it generally went fast. In general I m a passive subscriber to the Debian PHP Maintainers mailing list, and finally could help actually and not just reply emails. For me it s also doing some Debian work after a few months of focusing on the coming LibreOffice 3.5 release. update [26/1/2012]: From the 16 issues, only 2 aren t fixed already or will be on the next upstream release. As their upstreams are dead, this is in the hands or the debian people, at least till they ll FTBFS on unstable (instead of experimental).
Filed under: Debian GNU/Linux, PHP

Steve Kemp: So mega-upload is gone

So the site http://megaupload.com/ has been taken offline, amidst allegations of knowingly conducting in piracy. There are probably a lot of legitimate users who have lost access to their uploaded files, even if they were offsite backups you can imagine a user owning a website which now has a million dead-links. This reminds me of a conversation I overheard on Jon Dowlands blog - the summary is that he'd written a (useful) tool to extract attachments from Maildir folders and was wondering how to store and access those attachments. The upshot seemed to be magical URLs of the form:
  • https://file.example.com/sha1/509c2fe2eba509e93987c3024a74d74583c274bd
The comments covered an alternative which was hash:///sha1/xxxxxxxxxxxxxxxx, which then becomes close to the magnet:// schema. I've not yet thought things through, but I can't help thinking that with the redundency already present in the internet we should be looking at non-server-specific links. Yes there are times right now when you might want to address a specific file on a specific server - but otherwise? Wouldn't it be nice if you could just access a file from "anywhere" which happened to have the right contents? Already my nonporn-but-definitely-adult-site makes its images available as /img/$md5sum.jpg - and similarly the storage at the back-end of my random image upload site uses SHA1 hashes to store the actual files. To make this more complete what we need is something that crawls the internet to find files by hash; then add support in browsers. Obviously this must be async and could introduce timing issues, but fundamentally it seems like a reasonable approach to the problem of a single host going offline. (Consider what happens if imgur.com disappears. All those links would die, yet 99% of the images would still be available somewhere.) I'm tempted to suggest microformat format but I need to consider the matter. Right now I'm going to immediately update my current image hosts to use, at the very least:
 <a href="/foo" rel="sha1:xxxxx md5sum:xxxx">
  <img src="foo.jpg" alt="img name">
 </a>
The unfortunate thing is you cannot have a 'rel="xx"' attribute for an image. So you either have to encode it in the parent link, or add it to the alt attribute which is suboptimal. ObQuote: "Now, they tell me I paid my debt to society." - Oceans Eleven (2001)

20 January 2012

Alexander Reichle-Schmehl: Release Critical Bug report for Week 03

The bug webinterface of the Ultimate Debian Database currently knows about the following release critical bugs:
In Total:1563
Affecting Wheezy:928
Wheezy only:192
Remaining to be fixed in Wheezy:736
Of these 736 bugs, the following tags are set:
Pending in Wheezy:51
Patched in Wheezy:116
Duplicates in Wheezy:46
Can be fixed in a security Update:22
Contrib or non-free in Wheezy:12
Claimed in Wheezy:0
Delayed in Wheezy:3
Otherwise fixed in Wheezy:41
Ignoring all the above (multiple tags possible) 495 bugs need to be fixed by Debian Contributors to get Debian 7.0 Wheezy released. However, with the view of the Release Managers, 827 need to be dealt with for the release to happen. Please see Interpreting the release critical bug statistics for an explanation of the different numbers.

19 January 2012

Rapha&#235;l Hertzog: Debian related goals for 2012

Like last year, here s a list of Debian related goals that I d like to achieve this year. I might not have the time to implement all the projects, but I like to write them down to keep me motivated. And maybe it can inspire other people to implement some of them (or to help me).
  1. Finish the translation of the Debian Administrator s Handbook The target is to have the book available in April. It would be nice to complete the liberation fund until then so that the book is immediately made available under a DFSG-free license.
  2. Update the Debian Administrator s Handbook for Wheezy
  3. Translating the book in English is only the start of the journey. The real challenge is to keep the book up-to-date with each subsequent release of Debian. And Wheezy should hopefully be released in 2012 since the freeze is in June.
  4. Design and implement the Debian Package Maintenance Hub It s an ambitious project that aims to merge and replace the PTS, the DDPO and their respective mail variants. It should also standardize the flow of information directed towards package maintainers. I m going to use the DEP process to drive this project. This could easily take most of the year, but hopefully I ll motivate other people to chime in and help.
  5. Implement dpkg --check-db and dpkg --repair-db While dpkg is fairly reliable, it s not exempt of bugs and more annoyingly, harddrives/filesystems are not 100% reliable either, thus it happens that some internal database files get corrupted. Given that most files are text based, advanced users can manually fix them but many less skilled users are just left with a broken system that they tend to reinstall. To avoid this, we could provide a command that would try to automatically bring back the internal database to a sane state by looking for a working backup to restore (while at the same time marking some packages as requiring re-installation since we have some indications that they were present).
  6. Implement storage of dpkg s internal files in Git This would be an extension of the former idea. Installing a package dpkg-db-history (any idea for a better name?) would setup dpkg hooks that would record every database change in a git repository. This repository could then be used to restore the last working version of the database.
Besides those concrete projects, I want to do better than last year on the topic of funding my Debian work. I will thus reiterate some objectives:
  1. Write useful articles for Debian users and Debian contributors. They should complete the pages Mastering Debian, Contributing to Debian 101, Debian Packaging Tutorials, and help me increase the audience of this blog.
  2. Write at least one Debian-related ebook (different from the Debian Admin Handbook) and sell it. It could be an ebook targetting testing users since I believe that many more users could benefit from it if they had some better knowledge of the limitations and of the way to mitigate the problems that arise from time to time. Or maybe it could be an ebook for people who want to start contributing to Debian, it could even be bundled with a few hours of mentoring.
  3. By the end of the year, have at least 1/3 of my time funded by donations and/or earnings of my information products. This means doing 3,5 times better than in 2011. It should be doable given that the sales of the Debian Administrator s Handbook will contribute to this goal (once the translation is over).
That makes up lots of challenges for this year. Feel free subscribe to my newsletter to stay up-to-date with my progress and to get my monthly summary of the Debian/Ubuntu news. It s also a good way to help me reach those goals since you will be informed of all my new projects.

11 comments Liked this article? Click here. My blog is Flattr-enabled.

18 January 2012

John Sullivan: Sent to the ACLU today

I was on the brink of mindlessly clicking through the ACLU action center as usual to send an email opposing SOPA. But then I read their boilerplate text, and ended up cancelling the letter to my rep and instead sending this quick note to the ACLU:
Your SOPA suggested letter text supports current copyright law, and also backhandedly supports PIPA (the Senate version of the bill). This is far too weak of a position. As a donor, I ask you to take a stronger position that current copyright law unjustly restricts free speech, and that no further enforcement measures should be instituted until that fundamental problem is addressed. At least take on BOTH of these bills strongly. Most of the significant Internet is blacked out today to oppose both bills -- why would you cede so much ground to copyright maximalists? We have the support to oppose and defeat both bills.
For reference, here was their text:
While I believe it's important to protect copyrighted material online, the language of the Stop Online Piracy Act (SOPA) is flawed and will lead to the blocking of lawful content. Unlike the Senate version of the bill, SOPA eliminates the concept of sites 'dedicated to infringing activity' and enables law enforcement to target all sites that contain some infringing content -- no matter how trivial. The potential for impact on non-infringing content is much greater under SOPA than under other versions of this bill. Sites with user-generated content, like YouTube, Twitter, and Facebook, would be especially vulnerable, as one small piece of infringing content could lead to blocking the entire site. Even though proposed changes would narrow the amount of lawful content impacted, the changes don't go far enough. It is still likely that search engines will end up blocking access to perfectly legal online content. Congress should focus not just on the goal of protecting copyright owners, but also protecting the speech rights of consumers and providers who are reading and producing wholly non-infringing content. Congress must eliminate the collateral damage to protected non-infringing content. Only in that way will Congress truly achieve its goal of protecting authors while respecting the constitutional right to free speech.
Maybe I'm overreacting, but I dislike it when good organizations take weak positions unnecessarily. Usually this is not a problem with the ACLU, for me. It doesn't help that I keep seeing this meme everywhere in the anti-SOPA/PIPA conversation: "I agree we need to do something about piracy, but not this..." I don't think we need to do anything to fix violations of an extraordinarily unjust law until the law itself is fixed. I don't find that to be a very radical position.

Matthew Garrett: Why UEFI secure boot is difficult for Linux

I wrote about the technical details of supporting the UEFI secure boot specification with Linux. Despite me pretty clearly saying that this was ignoring issues of licensing and key distribution and the like, people are now using it to claim that Linux could support secure boot with minimal effort. In a sense, they're right. The technical implementation details are fairly straightforward. But they're not the difficult bit.
Secure boot requires that all code that can touch hardware be trustedRight now, if you can run unstrusted code before the OS then you can subvert the OS. Secure boot gives you a mechanism for making sure you only run trusted code, which protects against that. So your UEFI drivers have to be signed, your bootloader has to be signed, and your bootloader must only load a signed kernel. If you've only booted trusted code then you know that your OS is safe. But, unlike trusted boot, secure boot provides no way for you to know that only trusted code was executed. That has to be ensured by OS policy.

This doesn't sound like too much of a problem. But it is. Imagine we have a signed Linux bootloader and a signed Linux kernel, and that these signatures are made with a globally trusted key. These will boot on any hardware using secure boot. Now imagine that an attacker writes a kernel module that sets up a fake UEFI environment, stops the kernel from running code and then executes the Windows bootloader - kind of like kexec, but a little more fiddly. The Windows bootloader believes that it's performing a secure boot, but in fact everything that it believes is trustworthy is the attacker's fake UEFI code. The user's encryption passphrase is logged, the Windows kernel is modified to hide the Linux code and despite everything looking fine your credit card details are on their way to China. In this scenario, the signed Linux kernel is simply used as a malware loader. The only sign that anything is wrong is that boot will be slightly slowed down.

Signing the kernel isn't enough. Signed Linux kernels must refuse to load any unsigned kernel modules. Virtualbox on Linux? Dead. Nvidia binary driver on Linux? Dead. All out of tree kernel modules? Utterly, utterly dead. Building an updated driver locally? Not going to happen. That's going to make some people fairly unhappy.

(The same applies to Windows, of course. Windows 7 allows you to disable driver integrity checks. Windows 8 will have to forbid that when the system's using secure boot)
LicensingGPLv3 has various requirements for signing keys to be available. Microsoft's new requirement that systems support the installation of user keys would let users boot their own modified bootloaders, so that may end up being sufficient to satisfy the license. But we're then beholden on Microsoft - if they remove that requirement then users lose that freedom, and suddenly we're in an awkward licensing situation. There are ongoing conversations about exactly what we're able to do here, but it's not a solved problem.
Key distributionThe UEFI spec doesn't describe or mandate a central certifying authority. Microsoft require that everyone carry their key. We could generate our own, but we have much less sway with vendors. There's no way to guarantee that all hardware vendors will generate our key. And, obviously, if we generate a key, we can't just hand the private half out to others. That means that it becomes impossible for people to produce derivative versions of Linux distributions without getting their own key. The kind of identity verification that would be required for getting such a key is likely to be expensive, and also fairly likely to require that the distribution have a legally registered company in order to facilitate the identity verification. Think Extended Validation certificates, not Startssl Free. Hobbyist Linux distributions will be a thing of the past.
Doesn't custom mode fix this?Microsoft's certification requirements now state that all systems must support a custom mode, implying that it will be possible for a user to install their own keys. Linux vendors would then be able to ship with their own keys on the install media and impose their own policies. Everyone's happy. It's not really good enough, though. People have spent incredible amounts of time and effort making it easy to install Linux by doing little more than putting a CD in a drive. Asking them to go into the firmware and reconfigure things adds an extra barrier that restricts the ability to install Linux to more technically skilled users. And it's even worse than that. This is the full description of the requirement for custom mode:
  1. It shall be possible for a physically present user to use the Custom Mode firmware setup option to modify the contents of the Secure Boot signature databases and the PK.
  2. If the user ends up deleting the PK then, upon exiting the Custom Mode firmware setup, the system will be operating in Setup Mode with Secure Boot turned off.
  3. The firmware setup shall indicate if Secure Boot is turned on, and if it is operated in Standard or Custom Mode. The firmware setup must provide an option to return from Custom to Standard Mode which restores the factory defaults.

There's a few things missing from this, namely:

So no, custom mode doesn't make everything ok. Custom mode with a mandated UI and a documented key format would be much closer, but it wouldn't solve the problem of unattended automated installs.
SummaryWe can write the code required to support secure boot on Linux in a minimal amount of time - in fact, most of it's now done. But significant practical problems remain, and so far we have no workable solutions for any of them.

comment count unavailable comments

15 January 2012

Roland Mas: Looking for the ultimate distributed filesystem

(This is not quite a Dear lazyweb post. If anything, it's Dear lazyweb, I've done my homework, now what? ) I'm looking for the ultimate distributed filesystem. Something that's simple to use, redundant, fault-tolerant, and still smart enough to avoid the more obvious performance chokepoints. Ideally, it should work for all of the following sets of files: The requirements on the ultimate distributed filesystem (which I'll call UDFS for short, otherwise you'll get bored and go look at pictures of kittens) are as follows: Quite a few constraints, eh? I have a feeling they are not mutually incompatible though, so I had a look at several candidates. I started with Wikipedia, and I followed the links to Tahoe-LAFS, XtreemFS and Ceph. The following is my evaluation of these candidates based on much reading of docs and websites and wikis, some questioning on IRC, and very little testing. The overall picture seems full of good things scattered across different solutions, but unfortunately none of the existing ones seems to address the whole problem; at least, not my whole problem. It would be good if each focused on one layer and did that layer well, but that seems not to be the case either, so they can't be combined to get the best of all worlds. It may be that I'm missing something, or that I failed to read some docs properly, or that I misunderstood the docs, or that the docs themselves are simply lacking; but my ideal UDFS currently doesn't seem to exist as a turn-key solution. However, the main pieces are available, and implementing the remaining parts may be doable. My humble idea of a way forward would be based on Tahoe-LAFS, with the following three changes: Also nice to have would be a way to work with multiple introducer nodes, but that seems to be in the works already. This would be pretty damn close to my UDFS; read/write performance would certainly be far from what can be obtained on native filesystems stored on local disks, but my use cases involve reasonably small files for which instant access is not compulsory, and the filesystem cache would probably absorb most of the access times. In case anyone is looking for ideas of things to do in their spare time, here are rough sketches of other possible UDFS implementations I thought of. These are wild ideas, and I'm not even sure they could be doable in practice: Such is the state of my research so far. I would welcome feedback, pointers to things I neglected to read, corrections for things I misread or misunderstood, comments on the ideas and so on. I'll probably post an update if my search goes significantly forward. Update: I've already received two pieces of feedback, including a lengthy one with corrections about Tahoe-LAFS. For the sake of fairness, I'll solicit (and wait for) the same from the other candidates I looked at. I was also pointed at HekaFS, GlusterFS and git-annex, which I'll have to look at in more details. Other suggestions are still welcome, but the more I get, the more the full update will be delayed. Thanks already!

13 January 2012

Jon Dowland: vi

As seen on reddit: an old Bill Joy quote often banded about as part of the (largely synthetic) vi-vs.-emacs holy war starts as follows:
vi was written for a world that doesn't exist anymore
The context of that quote is:
I was trying to make it usable over a 300 baud modem. That's also the reason you have all these funny commands. It just barely worked to use a screen editor over a modem. It was just barely fast enough. 9600 baud is faster than you can read. 1200 baud is way slower. So the editor was optimized so that you could edit and feel productive when it was painting slower than you could think.
(source: the register). Furthermore:
unless you decide to get a satellite phone and use it to connect to the Net at 2400 baud, in which case you'll realize that the Net is not usable at 2400 baud.
You don't need anything as esoteric as a satellite phone: a cellular mobile phone will suffice. I've no idea what the effective/equivalent baud rate of GPRS is (or EDGE), but I have a new-found appreciation for vi's design goals having tried to run an SSH session over such a connection.

Next.

Previous.