Search Results: "nate"

19 April 2008

Chris Lawrence: Principals, agents, and Debian

I’ve noted in the past that Debian has deliberately enshrined in its constitution some rather serious principal-agent problems. By and large this isn’t a bad thing, since there isn’t the consensus within the Debian community to support the “benevolent dictator for life” model of decision-making—if you want that, well that’s what Ubuntu and Daddy Warbucks is for. But it does mean that sometimes the caca hits the fan when a Debian project leader does exercise his powers, as our now-former DPL did earlier this week just before the end of his term of office (by my estimate, just over one hour and 27 minutes before). John Adams would be proud. So we have three related issues in my mind:

As a matter of general principle, lame-duck DPLs shouldn’t be making appointments. This issue is ameliorated somewhat because DPL delegations—unlike “midnight judges”—can be revoked at any time, but it strikes me that whatever legitimacy a DPL has from the developers evaporates once a new DPL-elect has been designated. I can only speculate why this happened in this case, so I won’t bother.
Second, while Debian has a very strong tradition of developer sovereignty, with many aspects of the project being self-organized rather than originating with appointments from upon high, it seems to me that certain aspects of core infrastructure can’t be managed in this way.
Third, the appointment does little to relieve the excessive concentration of power in the core of Debian; if anything, Anthony Towns’ apparent resignation in the wake of Jörg’s appointment worsens the situation. Ensuring there is proper vetting of people with access to important infrastructure is important, but at the same time I find it difficult to believe that there are only a half-dozen or so Debian developers who are trustworthy enough to be system administrators, account managers, or archive maintainers (several of them occupying overlapping positions). That, rather than a lack of technical tools, has been a problem of note within Debian since, oh, the days of my youthful vigor within Debian (which are long since past).

In any event, congratulations to all the new Debian developers—and I’ll avoid pondering for too long why one person’s appointment to an unrelated group would suddenly break the logjam of developer application approvals.

18 April 2008

Steve Kemp: That wasn't true. Made it up. Shouldn't have done that. Sorry.

Chronicle

My blog compiler received a bit of love recently, primarily because MJ Ray wanted to use it. As mentioned before I've added a simple spooling system, and the mercurial repository now contains a simple RSS importer.

Debian Work

In other news I've been working on various Debian packages, here is a brief summery:

bash-completion

After seeing a RFH bug I closed a few bash-completion bugs, and submitted patches for a couple more. I was intending to do more, but I'm still waiting for the package code to be uploaded to the the alioth project.

javascript work

I've updated the jquery package I uploaded to follow the new "Javascript standard" - in quotes only because it is both minimal and new. Once the alioth project has been configured I'll upload my sources.

Apache2

I've agreed to work on a couple of SSL-related bugs in the Apache 2.x package(s) - time disappeared but I hope to get that done this weekend. Initially that was because I was hoping I could trade a little love for getting a minor patch applied to mod_vhost_alias - instead I've now copied that module into libapache2-mod-vhost-bytemark and we'll maintain our own external module.

Hardware

I've been loaned a Nokia 770 which is very nice. Having used it with vim, ssh & etc I think that I'd rather have a device with a real keyboard. The Nokia 810 looks pretty ideal for me. I'm going to be asking around to see if I can get a donated/loaned device to play with for a while before I take the plunge and pay for one of my own.

I've got a couple more things on the go at the moment, but mostly being outdoors is more interesting to me than the alternative. Hence the downturn in writing and releasing security advisories. I'll pick things up more fully over the coming weeks I'm sure. ObQuote: Shaun of the Dead

MJ Ray: Explaining web site improvements: what's important to you?

[Photo of Some Traffic]
Is this traffic or congestion? This is one of those Friday Afternoon Projects - it's been put off all through a busy week because it's unpaid, I'm not completely sure how to approach it and now my arms hurt like hell from travel jabs which are making it hard to concentrate! So I'm going to float it on here... I've been asked to brief a meeting next week about that group's current web site and its problems. I'm not linking it yet to avoid insulting/embarrassing them. The site looks OK, but doesn't rank well on search engines and doesn't allow much member participation. I need to explain why that's a bad thing and how the site's technical choices have led to that. I'm not directly pitching for TTLLP to get any work (because I'm a member of that group, it might be a conflict of interest and we're pretty busy anyway - even our own site needs work on some of the points I'm going to mention), but I don't want to be unhappy if we're asked to implement my recommendations. I've got a usual outline that I follow, but my presentation's time is limited, so I'd like to ask you: what about this is important and what isn't? If you give me useful feedback, I'll put you in the Acknowledgements with a backlink and I hope the briefing will be shared pretty widely over the next few months. The current plan is to start with a basic explanation of how search engines rank pages, as far as we can tell, referring to PageRank Explained Correctly with Examples, by Ian Rogers as well as the shorter official summaries from the dominant search sites at Yahoo, Microsoft and Google. Then I go through a quick evaluation of the site against the basics of validation, accessibility and robot-friendliness, followed by a couple of SEO-style checks of its current rankings and inbound links. Next is a bit different because I have access to some of their web access stats: I summarise what we know and suggest some other stats they've probably not considered and why they're useful, along the lines of Dave Briggs's measures of blog success. Finally, I suggest ways to improve the site. The top tip will be to take control of the site hosting and stop using the cheap and cheerful donated server that makes all links except the front page point to another domain. I'll probably suggest a mix of free and open source software tools to power it. If they don't want to move it all yet, I'll suggest running a second site for member participation, using tools like Wordpress, NoseRub and so on. What do you think? Plan for success, am I missing some tricks, or am I setting myself up for a lynching? Let me know with a comment or email, please.

Julien Danjou: Updating my .plan

I've been on holidays for one week now, breaking my usual workflow. Well, that allowed me to rest and to think about what I'd like to do and things I need to handle during next weeks. The urgent things this next days will be my primary server replacement. It is currently dying, and I already had to change its power supply twice in a month. Unfortunately, I'm now at a point where I do not have any spare piece so if things go wrong again, I'm screwed. I need to collect some money and buy a new server, or maybe get a server if someone have an old or spare one to give me, I do not know yet. On the awesome front, I'm about to release awesome 2.3, which will be the final minor release of the major branch 2. This will lead me to work on awesome 3 at a slower rate and a cooler pace. Then, the thing I do not have to hurry for is awesome 3. There's no big problems in awesome 2, and the xcb-util stuff are not stabilized yet. After my gentle yelling on XCB mailing list, it seems that things will move but will slowly. So I do have time to make things right and do what I want on that branch, making bugfix release on awesome 2 if needed. You can read more about futur on my last post about awesome. All this should give me some more spare time to work on the upcoming Debian release, lenny, which I'd like to work on. Two years ago (my god, time flies), we've done good work with the french cabal squashing critical bugs and I'd like to go back on this and squash asses again.

Lucas Nussbaum: 19 new Debian Developers! \o/

I am very happy that 19 contributors who were waiting for their accounts, sometimes for a very long time, became Debian Developers today. This is great news for them, and for the project as a whole. Many thanks to all people involved for making this possible, including Joerg Jaspert, Steve McIntyre and James Troup. And congratulations to (using their account names) kibi, plessy, gregoa, goneri, tincho, akumar, filipe, miriam and the others I haven’t had the chance to work with yet. It also seems that the various pending issues (updating keys that expired, etc.) have been resolved, which is great news for several of our current DDs. But this doesn’t solve the DAM problem on a permanent basis. Something interesting about today’s events is that the account manager asked the system administrators to create the accounts, which is a nice way to offload part of the process. But the keyring maintainance is still a SPOF. A tool has been developed to allow multiple people to collaboratively edit the same keyring (and it’s used to maintain the Debian Maintainers keyring), but I’ve heard that some people weren’t satisfied with it, unfortunately. Let’s hope that this is solved soon, so the next ones to go through NM won’t have to wait that long!

17 April 2008

Russell Coker: Solar Powered PC

I’ve just read an interesting post on TomsHardware.com about a solar powered PC [1]. It describes all the steps involved in creating a modern high-performance low-power computer. They have a lot of interesting information. One surprising fact (from page 3) is that the PSUs tested (both for AC and DC input) were more efficient when idle (I expected the greatest efficiency to be when under load). An AMD processor was chosen due in large part to the fact that chipsets in suitable motherboards used less power. For the CPU itself Intel had a competitive offering but no matching motherboard was power efficient enough (from page 7). Page 8 documents how using a cooling fan (instead of passive cooling) reduced the power requirements of the CPU to such a degree that it always saved power use overall. Why do CPUs take less power when they are cooler? Page 9 mentions that a small passively cooled video card can draw 88.5W when idle! That sucks pretty badly, it seems that having a video controller integrated with the motherboard is the way to go if you want to save power. It’s interesting to note how much energy can be used by RAM. Page 13 shows that the difference between 2*1G and 2*512M can be as much as 3.4W and that the difference between different brands of RAM for the 2*1G can make as much as 1.2W difference. Their final system drew 61W when idle, my latest 64bit system takes 52W when idle [2] (which compares to the 38W of their system without a monitor), so we are talking about 9% of system power being saved by using less RAM or 3% being saved by using a different brand of RAM. The summary of hard drive power use on page 14 is interesting, the fact that 2.5 inch laptop disks use less power than 3.5 inch desktop disks is hardly surprising, but the difference when idle is very surprising (apparently one of the 3.5 inch disks spends 8W on turbulence and friction in the bearings). It’s unfortunate that they didn’t compare any of the server-class 2.5 inch disks, it was about 6 months before the article was written that HP announced that in future they would cease shipping 3.5 inch disks and only use 2.5 inch disks (I wonder if this is related to all HP’s recent work on server cooling). Rumor has it that many server class 3.5 inch disks have platters that would fit into a 2.5 inch case because at high rotational speeds a larger diameter platter would not be strong enough. The information on DVD power use on page 15 is quite shocking. From now on when I install machines as servers which don’t have a need for for a CD-ROM drive I’ll remove the drive prior to deployment. Even if it saves only 0.47W then it’s still worth doing on a machine which uses less than 40W! An additional benefit of this is that it might speed up the boot process as the system won’t need to check for a bootable CD. It’s unfortunate that most computer parts don’t have published documentation on how much power they draw. Even if you don’t want to run on solar power there are still significant benefits to saving electricity (including reducing the noise from cooling fans and heat problems in summer). If technical data was published then people could make informed decisions about which parts to buy. Update: Changed the percentage savings for different types of RAM to be based on the system power use without the monitor. I’m most interested in saving power for servers and for idle desktops (running a desktop machine 24*7 is pretty common) so most of the time the monitor will be turned off. It’s interesting to note that they power their system uses is about the same as a P3 system and could be less if they used a different hard drive.

[1] http://www.tomshardware.com/2007/09/13/hardware_components/

[2] http://doc.coker.com.au/environment/computer-power-use/

Share This

David Pashley: Using In-memory tarballs with Archive::Tar

Archive::Tar is a useful library for working with tar archives from Perl. Unfortunately, one thing it doesn't allow is using data from memory as the archive. From the TODO section:

Allow archives to be passed in as string Currently, we only allow opened filehandles or filenames, but not strings. The internals would need some reworking to facilitate stringified archives.

Fortunately, it does allow you to use a filehandle. I've previously mentioned about how useful the IO::Handle subsystem in perl is. And we should be able to use it in this case. The module we'll want is IO::String, which is a IO::Handle over a perl scalar. We can use it:

my $tar = new Archive::Tar(new IO::String($data));

Unfortunately when we run this now we get:

Cannot read compressed format in tar-mode at Foo.pm line 41
No data could be read from file at Foo.pm line 41

It turns out that this is because Archive::Tar uses IO::Zlib internally if the file isn't uncompressed, but this doesn't provide the ability to uncompress from a filehandle. The answer is to uncompress the data before passing it to Archive::Tar and the easiest way to do this is to use the IO::Uncompress::Gunzip module, so we can rewrite our code to:

my $tar = new Archive::Tar(new IO::Uncompress::Gunzip(new IO::String($data)));

Now when you run the script, Archive::Tar has an uncompressed tar stream. Yet another situation where IO::Handles comes to the rescue.

Read Comments (1)

Joerg Jaspert: debadmin delegation - should I accept or deny it?

As you might have read on debian-devel-announce lately, Sam delegated the ftpmaster rights to me. Steve, our new DPL, did state his full support in an IRC query minutes later. Now I got a query from someone who is, pretty strongly, argumenting that I should decline this delegation. Mostly on the grounds that it is not nice to force new team members into an existing team, and that it is not a good environment to work in, if a DPL can do that if he feels like proving a point. Leaving alone the fact that delegations are one (central) part of the DPL job, and that the constitution leaves a way if you do not like those delegations (GR), I’m now unsure what to do. (Just for the record - i once was bitten by a DPL using his powers in a way I absolutely did hate (and still do). Some of you might remember the fun around the DebConf7 venue decision). I could do

decline the delegation, stay ftp assistant only. Which I am for a long time now. Possibly see some flame on the lists when I do this.
do nothing, ie. accept the delegation. That will get at least one of the long time ftpmasters upset, possibly more of them. Maybe with a resign from that job. Or Debian. But it would enable me to do the work only ftpmasters can do, like changes to the code, changes to the archive directly, etc. Without delays while waiting for a master to comment on something I prepared. Good example here is the current situation with the release teams transitions: They are 90% done and merged, just one little more merge is waiting to happen before they are ready to use… And whenever I tried to get a master (mostly Aj, as he was working with me on this) to do the final steps I got nothing but silence back, even when he was otherwise active…

Bah. I hate this. The thing is - I really do want to have debadmin rights, and am working in that direction for a long time now. Not publically, but by talking with James in the past months, … As Sams mail showed - that did go nowhere, unfortunately. Now, could readers of my blog (and I know there are people reading this, my traffic stats tell me so :) ) please comment on this and give me advice? I just want some opinions about this. (Yes, my blog does understand comments since some time :) ) UPDATE: Thanks to everyone who commented on this, either by using the comment form here, by mail or via IRC. I really appreciate your input. I’ve decided to not decline the delegation and one DSA member already added my account to the debadmin group. (Thanks).

Russell Coker: Making Linux DVDs

Anthony Towns writes about using an improved version of jigdo to download CD/DVD images [1]. His improvement is basically to pipeline operation for better performance. Jigdo (the Jigsaw download) is a tool to download a set of files and then use them to create a CD or DVD image [2]. The idea is that most web sites that have CD or DVD images also have a collection of files which comprise the DVD image available. This removes the need to store the data twice (wasting disk space on mirrors and in some situations breaking web caching). I have never used jigdo, and for all Debian installations in recent times (the last few years at least) I download a small net-inst CD image and then have it download the other files from the net. I have Squid set up to cache large objects so this doesn’t waste too much of my precious network bandwidth (which is limited and expensive in Australia). Now I’m thinking about what the optimum method for doing installs might be. One thing that would be good would be to support multiple repositories, the packages have unique file names and checksums so it should be possible to check one repository and then check another if it’s not working. I don’t mean multiple “deb” lines in the APT configuration. What I would like to do is to have an NFS file server or web server with an archive of downloaded packages and have APT check there first before downloading a file. So APT could get a list of packages from the net and then get the actual files locally if they are available. The next thing that would be good is the ability to create a CD or DVD image dynamically and to store all temporary files. So I could download files from the repository and create a DVD image with just the packages that I need. Every time I create a DVD image my sub-set of the Debian archives would increase and the number of files actually downloaded in the creation process would be reduced. The effect would be to incrementally create a local mirror of the Debian repository. Then I would like to see a two-stage DVD install process. I would like to boot from a CD or DVD and start the install and then have it give a list of files needed (which could be stored on a USB device or floppy) to create further CDs or DVDs for installation. One situation where this could have been beneficial was when I was doing an emergency install of CentOS. I did the first part of the install (selecting packages etc) to determine which CDs were needed. It turned out that almost all CDs were needed even though some of the CDs had only a few files that I was installing. If the installer could have written a list of packages to a USB device then I could have downloaded just those packages and got the install working a lot sooner. It seems to me that it’s fairly common to do one test install and then do some dozens of other installs with the same settings. So the ability to create a DVD of exactly the needed files for the other dozens of installs would be a great benefit. Now this is just random commentary, unfortunately don’t have time to do any coding on this. But it seems obvious that something has to be done to improve the situation for developers and IT staff who need some degree of mirroring of the Debian package pool but who can’t do a full mirror. Back in 1996 I was able to mirror Debian over a 28K8 modem link and fit it on what was a reasonable hard drive by the standards of the day (incredibly tiny by today’s standards). Now I can’t practically mirror Debian over an Australian cable broadband connection and even by the standards of about 4 years ago (the age of most of my hard disks) the space requirements are significant. I hope this post helps inspire some interest in developing these features. As delays in joining Debian [3] is the topic of the day it should be noted that work on preparing DVD images can easily be done by people who are not DDs. Such software should work from Debian archives without requiring any changes to them, and thus nothing special is needed from the Debian project to start work.

[1] http://azure.humbug.org.au/~aj/blog/2008/04/15#2008-04-15-jigdodl

[2] http://atterer.net/jigdo/

[3] http://etbe.coker.com.au/2008/04/17/motivation-and-perspective/

Share This

Kyle McMartin: debugging strace

A bug was reported on linux-parisc@ last week, that “ls -l” was whinging about “Operation not supported” once per file. Easy, I thought, I’ll just break out strace, figure out which syscall it is, and Bob’s your uncle, we’ll have it fixed. Not so, unfortunately, as strace was bloody broken. upeek: ptrace(PTRACE_PEEKUSER,2606,4294967292,0): Input/output error Again, no big deal, some kernel change must have broken ptrace(2) on me. git logs didn’t shed any light on it, so I pulled an old strace binary out of an etch chroot and tried that, which failed as well. Wonderful. But… curious… it still works inside the chroot. At this point I was a little bit beyond annoyed, but the fact that it worked in the chroot was a big clue. Anyway, long story short, strace decides to assume that you can have up to 32 possible syscall arguments (wtf?#1) which is all well and good and generally harmless. However, it also decides to yank MAX_ARGS of them out with ptrace(PEEK_USER, …) if strace is unaware of the syscall prototype. Again, this would be mostly harmless, on every other architecture. On parisc though, argument registers count down from %r26 through %r20, which is the syscall number. Instrumenting the PEEK_USER path through sys_ptrace in the kernel made it pretty obvious when a consecutive request was suddenly 0xfffffffc (which is 0-4, especially suspicious given the previous values of 4, 8, 16…) that I needed to find a loop in ptrace. Searching through all the HPPA specific syscall code found it quickly,



for (i = 0; i < tcp->u_nargs; i++)  

    if (upeek(pid, PT_GR26-4*i, &tcp->u_arg[i]) < 0)

        return -1;

where “i” is 0 .. u_nargs, which is set to either the known number of syscall args, or MAX_ARGS (which, remember, was 32.) There, the culprit found, and with a simple one line fix, strace works again. In case you’re curious, the reason things worked inside the chroot, is because the syscall being strace’d was dependant on libselinux being loaded.



syscall_298(0x400e0fbc, 0x58, 0xfb444310, 0, 0x409878ac, 0x409878ac) = -1 (errno 2)

Which is statfs64. The moral of the story here? strace is out of date and doesn’t yet support recently added syscalls. Also, the code quality is pretty atrocious. Oh, and the problems with ls? getxattr was returning -EOPNOTSUP because the filesystem doesn’t support
xattrs. Or, more correctly, because that config option was disabled. This seems to be causing laughs on the buildds too. Doesn’t anyone check errno anymore? Now back to our regularly scheduled hacking. –
wtf?#1: Generally speaking, more than 5 uint32_t syscall args is frowned upon, since that makes the poor register-starved i386 pass args on the stack.

16 April 2008

Simon McVittie: Implementing D-Bus services with telepathy-glib

16:46 < epmf> tp-glib is made of magic
—#telepathy, 2008-04-15

Rob pointed out today that I'd been promising to blog about telepathy-glib for several months and still haven't done so. I think there's too much to cover in one blog post, so I'll start with the oldest part - implementing a service (typically a Telepathy connection manager).

While telepathy-glib is (obviously) intended for Telepathy implementors, I think the ideas we've been implementing are likely to be useful for any D-Bus API, particularly flexible/complex APIs where an object implements many interfaces.

Introduction to telepathy-glib telepathy-glib started out as a collection of code extracted from our XMPP connection manager, telepathy-gabble, but at some point it grew beyond that into a wrapper around/partial replacement for dbus-glib. We've been quite pleased with it, really :-) My first major round of work on telepathy-glib, during the first half of 2007, was to split it out from telepathy-gabble (versions up to 0.5.9 were released as part of Gabble 0.5.x tarballs, in fact), leading to the release of the 0.6.x stable branch in late September. This was very helpful for implementing connection managers - Salut, Idle and telepathy-sofiasip (our connection managers for link-local XMPP, IRC and SIP) all started off by forking/cargo-culting from Gabble, and achieved huge code reductions by porting to telepathy-glib. Also, Will Thompson used it in his Summer of Code project, telepathy-haze, to go from nothing to a working and useful Telepathy implementation based on libpurple in a matter of a few months. The second major round of work, which I'll discuss in a later article, added client code to telepathy-glib, obsoleting the older/worse libtelepathy and allowing client programs like Empathy and telepathy-stream-engine to be written using telepathy-glib. In the process, I accidentally reinvented DBusGProxy :-) More details to come later!

Some background: service-side code generation

The secret Collabora code-generation process

(credit: daf)

Supporting D-Bus in the GObject world has always involved quite a lot of code generation. The core API of dbus-glib is heavily reliant on varargs functions, which aren't type-safe and are easy to get wrong. dbus-glib contains a program called dbus-binding-tool, which is meant to generate reasonably sane GObject APIs for D-Bus objects. Unfortunately, it seems to be intended to generate a whole GObject at a time. Early versions of telepathy-gabble used dbus-binding-tool plus a script called gengobject.py to generate API stubs for the exported objects; developers then filled in the blanks, and hey presto, we had a GObject on D-Bus. This was fine up to a point, but had a couple of major drawbacks. Whenever we changed the D-Bus API (quite common during the early development of Telepathy), there was a very laborious and error-prone merging process. We ended up with the following process:

copy the D-Bus introspection XML from the telepathy-spec repository into a directory xml/pristine (actually, the canonical form for the spec was Python for a while, and we had a script that exported a contrived object onto D-Bus and introspected itself to get the XML - but that's another story!)
preprocess the introspection XML to add the dbus-glib CFunctionName annotation, resulting in a directory xml/modified
run dbus-binding-tool and a Python script gengobject.py to generate C source in xml/src
three-way-diff the old version of xml/src, the new version of xml/src, and the real C code in src to create a new version of src
pray that the diff process hadn't randomly exchanged functions' implementations
update the resulting code in src so it worked
check the whole mess back into darcs
hope that nobody else had made changes that would result in darcs conflicts

If you haven't yet gathered, this was a bit of a nightmare. Also, it was very easy to return the wrong thing from a method without noticing - because all the APIs were varargs, the compiler didn't notice, and the first we'd know about it was a mysterious segfault under certain circumstances.

How telepathy-glib fixes code generation telepathy-glib improves on this by taking advantage of this innocuous-looking feature in dbus-glib:

commit 355ef78d98d6fc65715845d56232199162cab12a
Author: Steve Frécinaux <steve istique net>
Date:   Thu Aug 17 11:48:20 2006 +0200
    Interface support for bindings.

Instead of running dbus-binding-tool or creating GObjects with D-Bus interfaces, we put the entire Telepathy spec through glib-ginterface-gen.py, a distant descendant of Gabble's gengobject.py. For each interface in the spec, we generate a GInterface that mirrors the D-Bus interface. Any GObject that implements the GInterface automatically gets the D-Bus interface if it's exported onto D-Bus. There are a few non-obvious refinements, though:

The signature of the method implementation is always in the mode that dbus-glib calls "asynchronous", where the method implementation can either send a reply message before or after it returns. For "slow" methods, this is the only thing you can do. For instance, many Telepathy D-Bus methods can't return anything until some TCP round-trips to the server have happened. This is also the only thing you can do if you want to extract extra information, like the sender's unique name, from the method-call message (dbus-glib's "synchronous" API doesn't allow this to be done). For "fast" methods, sending a reply message before returning is just as easy as using the "synchronous" API (it's just a difference of syntax), and has an API consistent with that of the "slow" methods, making it easier for service authors.
The layout of the GInterface vtable is private, and (auto-generated) accessor functions are used to fill in implementations. This means our ABI doesn't change just because we re-ordered functions in the spec.
If no implementation is provided for a method, we just raise an appropriate error (org.freedesktop.Telepathy.Errors.NotImplemented), rather than suffering an assertion failure. This means we can safely add methods to an interface.
While we're generating code anyway, we generate some static inline wrapper functions which wrap dbus_g_method_return(), to have type-safe method returns. You can easily check that a method replies with correct types, by checking that the implementation of Foo() replies by calling tp_svc_some_interface_return_from_foo().
Similarly, we generate wrapper functions to emit signals, to get type-safe signal emission.

Not all interfaces are stable enough to be included in telepathy-glib's stable API and ABI, so some of our other projects include a copy of the telepathy-glib code-generation tools, and generate their own "mini-telepathy-glib" (traditionally in a /extensions/ directory) for their implementation-specific (drafts of) interfaces. This isn't yet as polished as it ought to be (mainly because we don't want to freeze the augmented-introspection-XML format that we write the Telepathy spec in, because it's rather ad-hoc and hackish in places) but it works quite well in practice. To see what this looks like in practice, have a look at the examples in telepathy-glib, or at a Telepathy connection manager like telepathy-gabble.

Mixins and base classes Another feature of telepathy-glib which makes life easier for connection manager authors is that it provides "mixins" and base classes which implement some of the GInterfaces. There's absolutely nothing magic about the base classes - they don't have any access to private implementation details of the GInterfaces, they just implement them like everyone else does. The "mixins" are only slightly more magical - they store some extra state inside the structures representing GObjects and their classes, and use it to provide default implementations of some or all of the methods on a particular interface (or two interfaces, in the case of the new TpMessageMixin). I'll leave it up to Rob to explain those in greater detail, since I seem to remember that he invented them :-P The examples and regression tests in telepathy-glib are quite good examples of how these work in practice.

Neil McGovern: Be careful what you post

In follow-up to my previous post, in which the phrase "google for gerri bird cambridge" (which turned up a rather poor lack of reply to a survey) was printed on some campain leaflets, I've come across a slightly different example of how traditional PR ideas may not work online if the people who are doing it don't understand the technology.

The Cambridgeshire County Council have placed on their transport page a link inviting people to view some videos of their new mis^WGuided Busway on YouTube. They've even gone so far as to create a YouTube account for the purpose. Unfortunately, the comments on their first video aren't too favourable, with the vast majority of people hating the idea. Not content with this, a second video was also posted, with exactly the same effect. In retaliation, it seems that their PR department has tried to fake some good reviews, but have been caught out.

So, some simple lessons:

Don't trust user generated content to do your job for you
Don't trust sites you have little control over
Don't try and stuff the ballot by writing like a PR person

Russell Coker: Resizing the Root Filesystem

Uwe Hermann has described how to resize a root filesystem after booting from a live-cd or recovery disk [1]. He makes some good points about resizing an LVM PV (which I hadn’t even realised was possible). The following paragraph is outdated, see the update at the end:
Incidentally it should be noted that if your root filesystem is an LVM logical volume then it can’t be resized without booting from a different device because the way LVM appears to work is that the LV in question is locked, then files under /etc/lvm/ are written, and then the LV is unlocked. If the LV in question contains /etc/lvm then you deadlock your root filesystem and need to press reset. Of course if your root filesystem is on an LV which has been encrypted via cryptsetup (the LV is encrypted not the PV) then a live resize of the root filesystem can work as locking the LV merely means that write-backs from the encryption layer don’t get committed. I’m not sure if this means that data written to an encrypted device is less stable (testing this is on my todo list). If your root filesystem is on a partition of a hard drive (such as /dev/hda2) then it is possible to extend it without booting from different media. There is nothing stopping you from running fdisk and deleting the partition of your root filesystem and recreating it. When you exit fdisk it will call an ioctl() to re-read the partition table, the kernel code counts the number of open file handles related to the device and if the number is greater than 1 (fdisk has one open handle) then it refuses to re-read the table. So you can use fdisk to change the root partition and then reboot to have the change be noticed. After that ext2online can be used to take advantage of the extra space (if the filesystem has a recent enough version of the ext3 disk format). One thing he didn’t mention is that if you do need to boot from another device to manipulate your root filesystem (which should be quite rare if you are bold and know the tricks) then you can always use your swap space. To do this you simply run swapoff and then run mkfs on the device that had been used for swap (incidentally there is nothing really special about the swap space, but it does tend to often be used for such recovery operations simply because it has no data that persists across a reboot). The minimum set of files that need to be copied to a temporary filesystem is usually /bin, /sbin, /dev, /lib (excluding all directories under /lib/modules apart from the one related to the kernel you are using), and /etc. Also you should make directories /proc, /sys, and /selinux (if the machine in question runs SE Linux). The aim is not to copy enough files for the machine to run in a regular manner, merely enough to allow manipulating all filesystems and logical volumes. Often for such recovery I boot with init=/bin/bash as a kernel parameter to skip the regular init and just start with a shell. Note that when you use init=/bin/bash you end up with a shell that has no job control and ^C is not enabled, if you want to run a command that might not terminate of it’s own accord then the command “openvt /bin/bash” can be used to start another session with reasonable terminal settings. I recommend that anyone who wants to work as a sys-admin experiment with such procedures on a test machine. There are lots of interesting things that you can learn and interesting ways that you can break your system when performing such operations. Update: Wouter points out that the LVM bug of deadlocking the root filesystem has been fixed and that you can also use ext2online to resize the mounted filesystem [2].

[1] http://www.hermann-uwe.de/blog/resizing-a-dm-crypt-lvm-ext3-partition

[2] http://www.grep.be/blog/en/computer/cluebat/actually_you_can

Share This

Miriam Ruiz: More on OOXML: ISO tries to recover lost credibility through press notes

After the unfortunate show that Microsoft and ISO provided us recently, ISO’s response is a FAQ note about ISO/IEC 29500 (OOXML, that’s it) in which they threaten Microsoft with the most severe of penalties if they’re tricking on as. Funny. In that statement, they try to answer all the issues that initially were concerns but that now has turned into proven facts about the OOXML standard and it’s approval, and to try to recover some of their lost credibility. It’s sad to be commenting on this, words don’t come easy, as if to say. Microsoft has already proven who’s the one in charge here, who owns the committees, who takes the decisions and in which terms, and nothing happens. It’s gonna take more than to publish a FAQ note saying how well you did it and to threaten Microsoft to behave or else. No one seriously believe you at the moment, you know? A funny thing is that ISO dismisses any responsibility about standardization, and relies on the market taking the decisions for them. Microsoft has said many times that they consider something positive and healthy to have multiple standards for the same, and let the market decide. Now ISO declares to think along the same lines: “After a period of co-existence, it is basically the market that decides which survives”. We don’t really need an organization for standardization for that, do we? We already have the market for that. Will ISO apply the same policy for every standard from now on? Like, say, having a couple of competing standards at least for every need, and let the market decide? ISO tries to recover from their credibility loss with this note (“the standards development process is credible, works well and is delivering the standards needed, and widely implemented, by the market“), without much success at least in what I’m concerned. If they keep trying to convince with press notes instead of with real acts, no one is gonna believe them ever again.

15 April 2008

Russell Coker: Software Development is a Team Sport

Albert writes about software development and how much teamwork is used [1]. He makes an interesting clash of analogies by suggesting that it’s not a “team sport” because “its not like commercial fishing where many hands are used to pull in the net at the same time“. I think that software development for any non-trivial project is a team sport. You don’t have the same level of direct coordination as required for pulling in a net (or the rugby scrum [2] to use a sporting analogy), but that doesn’t stop it being a team effort. Some parts of team development projects are like a relay event, in corporate environments the work is done in parallel simply because everyone is working the same hours but in free software development projects the work is often serialised. I think that it’s often more effective to serialise some work, if someone is actively working on one sections of code it may save time to avoid working in that area until they are finished. There is little benefit in writing code to old interfaces. Some parts of team projects have specialised skill areas (EG debugging, skills in particular programming languages, and graphical design). Soccer is one sport where different rules apply to different players (the goal keeper can use their hands). In ice-hockey the protective clothing used by the goal keeper is considerably different from that used by other players. In most team sports where the aim is to put a ball through a goal at one end (EG basketball and all versions of football) there seems to be some degree of specialisation, some players are dedicated to scoring goals while others are dedicated to defense. The fielding team in cricket has every player assigned to a different part of the field - with slight differences in the skills required. Then there is the issue of large projects such as Linux distributions. It seems to me that a Linux distribution will always comprise multiple teams as well as some individual projects. Maybe we could consider Linux distributions (and distributions of the other free OSs) to be similar to countries that compete in the Olympics. The culture of the Free Software (or Open Source if that’s your idea) community can be compared to the Olympic Spirit. Of course the Olympic idea that people should come together in peace for the Olympic Games and that it’s about honor not money is pretty much dead. Maybe the Free Software development processes should be compared to an ideal of what sporting contests would be if there weren’t unreasonable amounts of money (and therefore corruption) involved. Of course no analogy is perfect and there are many ways in which this one breaks down. One of which is the cooperation between distributions. There is a lot of private discussion between developers of various distributions and upstream developers about how to plan new features. It’s not uncommon for developers to announce certain development decisions as soon as they are made to help other distributions make decisions - for a developer in a distribution project if there is an issue which doesn’t matter much to you or your users then it’s often good to strive for compatibility with other distributions. When users advocate new features or changes they sometimes try multiple distributions. It’s not uncommon for a feature request to be rejected by one distribution and then accepted by another. Once a feature is included in a major distribution the upstream developer is more likely to accept it due to it’s wide testing. Then when the feature is in upstream it’s almost certain to be included in all other distributions. I often recommend that when someone disagrees with one of their bugs being closed as “not a bug” that they try reproducing it in another distribution and reporting it there. As a side note, the criteria for reporting a bug in any free software distribution is that you can describe it in a way that allows other people to reproduce it - whether it’s a bug that afflicts you every day or whether you installed the distribution for the sole purpose of reporting the bug in a new forum is not relevant. As a general rule I recommend that you not have the same bug report open in more than one distribution at any time (if you notice a bug reported in multiple distributions then please add a note to each bug report so that work can be coordinated). As a general rule the only situation where I will open the same bug in multiple forums is if I have been told that the responsible person or people in one forum are unwilling or unable to fix it. Finally, the people who consider that they don’t need to be a team player because they do their coding alone might want to consider Qmail. Dan Bernstein is a great coder and Qmail is by most metrics a fine piece of software, in terms of security Qmail is as good as it gets! If Dan was more of a team player then I believe that his mail server would have been much more successful (in terms of the number of sites using it). However I do understand his desire to have a great deal of control over his software.

[1] http://www.docunext.com/blog/2008/04/13/the-people-who-live-on-planet-debian/

[2] http://en.wikipedia.org/wiki/Scrum_(rugby)

Share This

Lucas Nussbaum: 4 months and 10 days without any new Debian developer. Is Debian dying?

It has now been more than 4 months since the last Debian developer account was created. 18 contributors have been through all steps, and are simply waiting for this simple administrative task to be done. We are sending a terrible message to potential contributors. We have strong requirements on the technical level of our developers. During the new maintainer process, we ask them to answer about 80 questions about Debian. We ask them to do grunt work. We review their reports twice (New Maintainers’ Front Desk, then Debian Account Manager). But even after we are totally satisfied about what they did, even after they became more qualified than many of our current DDs, we ask them to wait for months, so that the only person allowed to create accounts can finally do his “job”. It discourages the contributors currently in the NM process. I’ve seen several signs of frustration, or even depression. Some of them reduce their involvement in Debian, so we lose them before they even became Debian developers. Some of them consider resigning from the NM process. We should all feel guilty about that. But it also discourages people from joining Debian. Instead, they go to other more welcoming projects, which is totally understandable. Debian isn’t the only distribution with developers from the community those days. There’s Gentoo, Ubuntu, Fedora, openSUSE. Some of those have nice programs for new contributors, like “school” sessions. Sure, Debian is the “biggest” distribution without a company behind it. But is independance worth all the trouble? Of course, we have Debian Maintainers. DM is great for people who want to work on their packages. But, when we are trying to release lenny, we need more: people who are going to go through RC bugs, submitting patches. Who are going to do QA work. In short: people who care more about the whole distribution. Can we afford not recruiting anybody? Can we run Debian with the current manpower? I don’t think so. There are more than 550 RC bugs in lenny, many packages are currently blocked from migrating to lenny because they are RC-buggy, and many packages are orphaned or neglected. There are also a lot of bugs which haven’t been filed yet (I asked for help with running piuparts, which would probably result in 100-200 new RC bugs, but nobody had time to help). Most of the work that needs to be done is not rocket science. We could use a lot more manpower. Currently, the same small set of developers is doing most of the grunt work. They will get tired too. So, what can we do, as simple developers? There’s no magic solution, but we can try a few things.

It seems that some people disagree that there’s a problem. Let’s prove them wrong: we could start a blog meme with “I, too, agree that the Debian accounts and keyring situation is severely hurting Debian, and that a solution needs to be found RSN.” It’s not going to solve the problem by itself, but it will at least show that we consider it very important. Pressure could help.
We could start discussing solutions together. Our newly elected DPL said that he would talk with the problematic teams to determine how the situation could be improved. Unfortunately, this has been tried in the past (and failed). It might work this time, of course, but we could prepare a backup plan. So let’s find one or two good plans, and vote on them. (I liked the idea of giving accounts creation/management to DSA. After all, it’s only an sysadmin task once the report has been approved by FD and DAM.)
We could push forward Josip Rodin’s proposal about infrastructure teams. It might not solve the DAM problem immediately, but would probably help avoid similar problems in the future.

Notes:
1. Maybe the 18 waiting accounts will be created today or tomorrow. Even if that happens, it won’t solve anything. Waiting 4 months for a simple administrative task is not acceptable, and we need to fix that problem anyway.
2. Account creation is not the only problem. Some people have been unable to upload packages or to vote for the DPL election, because their PGP key expired, and nobody updated it even if they have been asking for more than 4 months.

Russell Coker: Buying Old PCs

I install quite a number of internet gateway machines for one of my clients. While eventually he will probably move to using an ASUS EeePC [1] or something similar, the current plan is to keep using desktop PCs (unfortunately server-class machines make too much noise). P4 machines use an unreasonable amount of power and don’t seem worth getting second-hand. P3 machines are the most power efficient machines that have been commodity PCs (with keyboard and monitor), see my computer power use document for an example [2]. So what I would like is to get a dozen name-brand P3 systems at a reasonable price. Currently the only companies I can find selling such machines are charging ridiculous prices (such as $300) with the aim of reaming corporations who want to complete a set of old machines rather than upgrading them all. Most online auctions etc are selling P4 and Celeron 2GHz as the minimum hardware. The Celeron machines are better than P4 (at around 50W) but not as good as P3 (at less than 40W). If you read this post and are in Melbourne, Australia and have a name-brand P3 desktop system that you want to sell then let me know. If you have any suggestions of how to buy such machines then I’m interested in advice.

[1] http://etbe.coker.com.au/2008/01/23/asus-eeepc-as-a-router/

[2] http://doc.coker.com.au/environment/computer-power-use/

Share This

14 April 2008

Anthony Towns: A New DPL...

In a couple of days, DPL-elect Steve McIntyre takes over as DPL, after being elected by around four hundred of his peers… Because I can’t help myself, I thought I might poke at election numbers and see if anything interesting fell out. First the basics: I get the same results as the official ones when recounting the vote. Using first-past-the-post, Steve wins with 147 first preference votes against Raphael’s 124, Marc’s 90 and NOTA’s 19 (with votes that specify a tie for first dropped). Using instant-runoff / single transferable vote, the winner is also Steve, with NOTA elimited first and Marc collecting collecting 5 votes, Steve 4 and Raphael 2, followed by Marc getting eliminated with Steve collecting 50 votes, against Raphael’s 26. So, as usual, different voting systems would have given the same result, presuming people voted in basically the same way. NOTA really didn’t fare well at all in this election, with a majority of voters ranking it beneath all candidates (268 of 401, 53.5%). For comparison, only 18 voters ranked all candidates beneath NOTA, with 9 of those voters then ranking all candidates equally. (For comparison, in 2007, 312 of 482 voters (about 65%) ranked some candidate below NOTA, though that drops to 225 voters (47%) if you ignore voters that just left some candidates unranked. Only 98 voters (20%) voted every candidate above NOTA) With NOTA excluded from consideration, things simplify considerably, with only 13 possible different votes remaining. Those come in four categories: ranking everyone equal (17 votes, 9 below NOTA as mentioned above, and 8 above NOTA), ranking one candidate below the others (13 votes total, 7 ranking Raphael last, 3 each for Steve and Marc), ranking one candidate above the others (66 votes; 30 ranking Steve first, 18 each ranking Raphael and Marc first), and the remainder with full preferences between the candidates:

The most interesting aspect of that I can see is that of the people who ranked Raphael first, there was a 1.8:1 split in preferring Steve to Marc, and for those who preferred Marc first, there was a 2:1 split preferring Steve to Raphael. For those who preferred Steve, there was only a 1.1:1 split favouring Raphael over Marc. I think it’s fair to infer from that that not only was Steve the preferred candidate overall, but that he’s considered a good compromise canidate for supporters of both the alternative candidates (though if all the people who ended up supporting Steve hadn’t been voting, Raphael would have won by something like 26 votes (129:103) with a 1.25:1 majority; if they had been voting, but Steve hadn’t been a candidate, Raphael’s margin would’ve increased absolutely to 33 votes (192:159) but decreased in ratio to a 1:1.2 majority.

James Morrison: Niti days

I don't think any of these are mine, but this needs to be published to the general public (at least until it rests on AlumNit).

Poo Alliterations
- The Earl of Excrement
- The Duke of Dung
- The Prince of Poop
- The Sultan of Shit
- The Father of Feces
- The Matriarch of Merde
- The Steward of Stink
- The Dignitary of Dangleberries
- The Grand Poohbah of Grand Poo
- The Tyrant of Turd
- The Shogun of Scheisse
- The Baron of Bowels
- The Cardinal of Crap
- The Director of Dump
- The Surveyor of Stool
- The Lord of the Log
- The Crown Prince of Cow Patties
- The Royal Archbishop of Road Apples
- The Duchess of Droppings
- The Princess of Poodie
- The Legionnaire of Loaf
- The Supervisor of Stinkers
- The Pharaoh of Floaters
- The Magnate of Manure
- The Tycoon of TCPDumps
- The Czar of Czesspools
- The Mogul of Bowel Movements
- The Ruler of the Runs
- The Dean of Diarrhea
- The Royal Ringleader of Rump Raisins
- The Denizen of Doo-doo
- The Emperor of Evacuations
- The Headmaster of Horrible Odors
- The Titan of the Trots
- The King of Klingons
- The Big Boss of Bum Beans
- The Taskmaster of Turtleheads
- The Supervising Sovereign of Sewer Serpents
- The Colonel of Colon Cobras
- The Nobleman of Number Two
- The Top Dog of Toilet Twinkies
- The Kingpin of Keester Cakes
- The Shepherd of Sea Pickles
- The Head Navigator of Heinie Nuggets
- Rectal Representative
- The Commander of Cow Pie
- The First Mate of Fecal Matter
- The Creator of Cloth Touchers
- The Protector of Porcelan
- The Theologian of the Throne
- The God of Grunts
- The Small Insignificant Yodeling President of Shit in your Pants
- The Diaper Destroyer
- The Loader of Loincloths
- The Pontiff of Plum Pudding
- The Daiymo of Dysentery
- The Master of Montezuma's revenge
- The Squire of Squirts
- The [Weaver] of [WvLog]s
- The Primary Contact of Pudding Cookies
- The Best Producer of Brown Puddles
- The Senior Partner of Steaming Piles
- The Fine Baker of Fudge Brownies
- The Sheriff of Stench
- The Don of the Deuce
- The Torrid Putrid Tainted Pervert of Two-Ply Toilet Paper
- The Sacred Monarch of Skid Marks
- The Main Scientist of Microfeceological Studies
- The G.I. Joe of Gastro Intestinal Journeys
- The Premier of Polyps
- The Brigadier General of Bedpan Gold
- The Plunger Pilot
- The Specialist of Splashers
- The Foreman of Fake Farts
- The Ambassador's Under-Secretary of Astounding Unearthly Scents
- The Old Man and the Sea of Odiferous Monsters in the Sink
- The Bafflingly Blunt and Blasphemous Bearer of Big Bad Beastly Booty Burdens
- The Crisp Pungence of Chain Pooping
- The Goddess of Groundhogging
- The Burly Bobby's Brief Battle with Brim Blasting Bowel Bombs
- The Nasty Neighbour's Noisy and Nefarious Neglect of Normal Nasal Niceties
- The Judge Jury and Executor of Jolly Jumper Excrement
- The Forwarder of the Forwarded
- The Wizard of the Water-closet
- The Grunting Lobber of Great Logs
- The Annoyingly Arrogant Architect of Awful and Abhorrent yet Amazingly Aerodynamic Anal Atrocities
- The Dark, Dangerous and Delusional Dealer of Death-Defying Doses of Disturbingly Decadent Dung
- The Shakespeare of Sinfully Smelly and Supremely Scary Scripture with Soggy Stains and Screaming Splashes
- The Wise but Weakened Warlord Whose Willful Wrath Was Wholly Wrought With Wicked and Wretched Winds from Within
- The Languid and Lackluster Legend of the Little Log from the Loo and its Lavish Life of Luxury in a Lasagna
- The Kahuna of Kaka
- The Emergency Evacuator of Enraged Entrails
- The Squire of Sultry Stinknuggets
- The Deadening Discomfort of a Dearth of Dumps
- The Far From Favourable Fragrance of Four Fragments of Fantastically Fresh Feces
- The Principal of Prodigiously Putrid Pellets
- The Markedly Morose Maintenance of Maniacally Mounded Midden Movements and Mostly Meaty Meals Mysteriously Made Mushy
- The Crass and Cranky Coroner Connecting Clues Concerning a Cornucopia of Casualties Caused by a Crude Cluster of Completely Contemptible Colon Concoctions
- The Thinking Thoughts of a Thinker
- The Horrible, Hubristic Heathen Hoarding Hoary Humps of Hideousness Hailing from His Huge Hairy Heinie in His Harmfully Heady Hovel
- Inane Writings on Irritated Wreathing of the Incubator of Wrath
- That Special Numbness from Sniffing Noodled Stink Nougat
- The Moral Sanctity of the Mystery Shit
- The Ornery Opressor's Opulent Outpost Overseeing the Ordinary Occupants' Outrageous and Ongoing Observance Of an Occasional but Offensive Orgy Of Offal
- The Rector of Relief
- The Rectal Regurgitator
- The Shit Squat
- The Smelly Surprise
- The World Wide Wisdom
- The Slayer Supreme of System Shock

Christian Perrier: IP over DNS for dummies

Once again, I'm sitting in an airport (CDG) and enjoying the benefits of IP over DNS to bypass the outraegous rates for wireless access (about 10 euros for 1h if that didn't change since last year). Actually, having IP over DNS work is pretty simple. What you need is:

A domain that you control (ie you can create en entry in it)
A server running DNS services on the Internet, preferrably with a static IP address, preferrably with no need of external DNS services
Appropriate software

"Appropriate software" here is, IMHO, the iodine package in Debian. There's nothing as easy to setup as it is. There is no package for etch, AFAIK, but backporting it is straightforward. You then need to create an entry in your foo.bar domain, such as "tunnel.foo.bar" and have it point to your static address. Then you run iodine on that server, listening to port 53 (with a configured password). That creates a virtual interfaces with something like 10.0.0.1 as address. When you need IP over DNS access with your roaming laptop, all you need is to connect to the expensive network, then launch iodine and provide it two parameters: the "tunnel.foo.bar" address and a local address (for instance 10.0.0.2). If everything works well, you'll be prompted for the password and voil . You have a tunnel between your local 10.0.0.2 address and the remote 10.0.0.1. The rest is a matter of network routing and you can now access the entire Internet through your remote server with no charge. The entire traffic goes through DNS requests to the local DNS server. That can be fairly slow but all a geek needs is receiving/sending mail and remote SSH access, right? This doesn't work in every place, unfortunately. Many wireless providers block such traffic in some way. But, at least in A roports de Paris, I can promise that will work..:-)

Next.

Previous.