It looks like some of us might be a little busy this weekend. If all goes well, then it looks like next weekend may be busy too. Yay! UPDATE: The party happened yesterday, and we had more than 20 people for the BBQ. Yay! :-) I was too busy cooking and drinking to be bothered with photos, but I expect others will oblige at some point - check the wiki page above for updates.

I was recently (at FOSDEM) made aware of the fact that apart from the client-side software, Zetes also provides some server-side software to allow one to verify electronic ID card certificates on an HTTP server. I didn't know about that, since nobody ever told me about it; and none of the eID sites that I know of mention it, either. So today I had a first look at the code to see what I'd need to do to properly package it. The code does contain a few WTFs, though.

-    if (!r->proxyreq   !r->filename   strncmp(r->filename, "proxy:", 6) != 0)
+    if (!r->proxyreq)
+        return DECLINED;
+
+    if (!r->filename)
+        return DECLINED;
+
+    if (strncmp(r->filename, "proxy:", 6) != 0)
         return DECLINED;

This one is especially silly if you know that the above is the only change to mod_proxy.c. They do extensive changes to apache's mod_ssl, since that is where most of the needed work is done. The changes include the following stuff:

 #ifndef BOOL
-#define BOOL unsigned int
+#define BOOL int
 #endif

It's a boolean. By definition, that needs to keep only 2 values. The ability to set it to negative values doesn't matter. But, well; the above two could be explained by compilers having strange ideas about what should be warned about. So let's ignore those. The code should be sane then, right? Well, almost.

--- httpd-2.0.54/modules/ssl/mod_ssl.h  2005-02-04 21:21:18.000000000 +0100
+++ httpd-2.0.54-beid-proxy/modules/ssl/mod_ssl.h       2007-04-15 13:45:27.0000
00000 +0200
@@ -74,7 +74,7 @@
 #include "apr_global_mutex.h"
 #include "apr_optional.h"
 
-#define MOD_SSL_VERSION AP_SERVER_BASEREVISION
+#define MOD_SSL_VERSION AP_SERVER_BASEREVISION "+OCSP (1.0.8)"
 
 #ifdef HAVE_SSLC
   
@@ -152,7 +152,7 @@

That's okay, sure? If you change mod_ssl to include OCSP support, you may want to change its version number to reflect that? Yes, indeed. But if you set the version number like above, and spread the rest of the mod_ssl code with patches like...

+#ifndef NOOCSP

(...)

+#endif

...and...

+ifndef NO_OCSP

(...)

+#endif

... then my guess is something's wrong with your processes. Anyway. The modified SSL code for apache will be coming to a Debian mirror near you some time. But not soon.

So, today I tried to sign up for a service online (unidentified for reasons that will become obvious briefly). This is a service that, had I successfully signed up, would be withdrawing monthly subscription fees from my bank account and handling some data that's fairly important to me. But when I hit the button to start the registration process, I got back...

[TCX][MyODBC]You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near ''7GDiSp=E8-Mgvgo\', Now() )' at line 9

SQL = "INSERT INTO [REDACTED] ([REDACTED], FieldName, FieldValue, update_dt ) VALUES ( 10724, 'PASSWORD', '7GDiSp=E8-Mgvgo\', Now() )"

The password I tried to enter, for the record, was 7GDiSp=E8-Mgvgo\#dEwb8m7Ec_e~z0myAAj. This is at best inexcusable sloppiness and at worst a security hole, and I don't want a company that does either anywhere near my money and data. (sadly, there probably are already many such companies near my money and data, but I try to avoid the ones I know about)

Whilst downloading, installing and running Debian packages is easy, fun and “free”, most Web applications aren’t. I can’t be the only one that thinks there is a problem somewhere as for example, where are the all the Web applications listed on Freshmeat? I have an idea for a Web application though I thought it would be cool to make it as Free software though it can be a pain.

• Getting a reliable host
• Host has to be Debian and probably unstable to get the cool stuff :)
• You probably want to have root, hence dedicated
• It is probably expensive and you might have to watch bandwidth
• You have to create accounts, where your colleagues also want root.
• You could use sudo though you’ll probably get frustrated with it and end up with “ALL = NOPASSWD: ALL” and no one checks the logs ;)
• You’re probably want Xen and a few IPs
• Since Web application packaging with current tools is near impossible with all the different httpds and DBMSes, you’ll probably have to use some odd hand crafted setup.
• You will probably want a DBA sooner or later, to figure WTF is going on with your DBMS ;)
• You’ll probably end up with a several step build&deployment process (that’s bad)
• Testing (if you ever get that far) will probably be a kludge
• To lower the barrier of entry, you will probably go for PHP which is great to deploy. Scary to secure.
• Though non-PHP avenues will probably require FastCGI. And FastCGI on Apache last time I checked is PAIN.
• Your developers probably can’t run the Web application locally as it such a pain to setup (esp. forget offline)
• You might have some “privacy policy” that makes sharing log or database data difficult amongst developers.
• Your Web application will probably need mail server access at all the pain that comes with that.
• Your application will have to deal with difficult abuse problems like spam that ordinary applications don’t usually deal with.
• If your project’s site becomes popular you’ll have a ton of maintenance and scalability head aches that will probably require money to be thrown at the problem.
• Someone will think your Web design stinks. Then if you use some “designer”, he creates some awful XHTML template and some fixed px style sheet with images replacing text. ;)

I think there is a few more “barriers to entry” problems running a Web application as a free software project. Web hosting is generally quite poor and a lot of the Web centric deployment/packaging tools aren’t up to scratch. Though there are a couple of amazing examples of “open source” Web sites, like the stuff My Society do. Nonetheless I would like to see Debian as an even better Web application platform.

Whilst downloading, installing and running Debian packages is easy, fun and “free”, most Web application services aren’t. I can’t be the only one that thinks there is a problem somewhere as for example, where are the all the free software Web services listed on Freshmeat? I have an idea for a Web application though I thought it would be cool to make it as Free software though it can be a pain.

1. Getting a reliable host
2. Host has to be Debian and probably unstable to get the bleeding edge stuff :)
3. You probably want to have root, hence dedicated
4. It is probably expensive and you might have to watch bandwidth
5. You have to create accounts, where your colleagues also want root.
6. You could use sudo though you’ll probably get frustrated with it and end up with “ALL = NOPASSWD: ALL” and no one checks the logs ;)
7. You’re probably want Xen and a few IPs
8. Since Web application packaging with current tools is near impossible with all the different httpds and DBMSes, you’ll probably have to use some odd hand crafted setup.
9. You will probably want a DBA sooner or later, to figure WTF is going on with your DBMS ;)
10. You’ll probably end up with a several step build&deployment process (that’s bad)
11. Testing (if you ever get that far) will probably be a kludge
12. To lower the barrier of entry, you will probably go for PHP which is great to deploy. Scary to secure.
13. Though non-PHP avenues will probably require FastCGI. And FastCGI on Apache last time I checked is PAIN.
14. Your developers probably can’t run the Web application locally as it such a pain to setup (esp. forget offline)
15. You might have some “privacy policy” that makes sharing log or database data difficult amongst developers.
16. Your Web application will probably need mail server access and all the pain that comes with that.
17. Your application will have to deal with difficult abuse problems like spam that ordinary applications don’t usually deal with.
18. If your project’s site becomes popular you’ll have a ton of maintenance and scalability head aches that will probably require money to be thrown at the problem.
19. Someone will think your Web design stinks. Then if you use some “designer”, he creates some awful XHTML template and some fixed px style sheet with images replacing text. ;)

I think there is a few more “barriers to entry” problems running a Web application service as a free software project. Web hosting is generally quite poor and a lot of the Web centric deployment/packaging tools aren’t up to scratch. Though there are a couple of amazing examples of “open source” Web services, like the stuff My Society do. Any other good examples? Nonetheless I would like to see Debian serve as an even better Web application platform. Update: I think “services” makes more sense than “application” in this context… I hope you’ll agree.

I've been meaning to attend the ACCU Conference for some years and finally decided to pay my own way for a couple of days this year. It was an opportunity to hear world experts on C++ and other programming topics and to a smaller extent to meet and socialise with other professional programmers. I attended the following sessions: Keynote: Improving Collaboration in Open Source Projects, Mark Shuttleworth This was a somewhat surprising choice of speaker and subject since I think most attendees work on proprietary software, but I think any sensible developer in the proprietary world will try to build on free libraries under non-copyleft licences, where appropriate. Mark talked about governance and leadership in "open source" projects (actually praising Debian for having a clear constitution) and about the need for good communication between projects, for example about release schedules. He seems to have accepted that Launchpad cannot be a single hub, and called for standard formats for, for example, exchanging information about related bugs between bug trackers. But it doesn't sound like he's ready to take the lead on that. Choose your Poison: Exceptions or Error Codes, Andrei Alexandrescu Andrei looked at the strengths and weaknesses of various error reporting mechanisms in C++ and introduced a technique for making error handling more flexible. Normally a function is designed either to indicate errors through its return value or to throw an exception on error. If a function returns an error indicator, it's easy for callers to ignore it. If it throws an exception on error, then a caller that provokes errors more often than the called function's author anticipated pays a high cost in processing time for those errors. Andrei's solution involves a class template Likely<typename T> whose instances hold either a return value of type T or an exception. A function that can fail has a specialisation of this template as its return type. The caller can check for failure if it knows how to handle it, or it can ignore it - in which case the conversion to T or the destructor will throw the exception. This is clever but doesn't seem to be entirely satisfactory. The destructor which throws does first check that no exception is already being processed, avoiding collision of the two exceptions and consequent abrupt termination of the program, but this can result in errors being unexpectedly ignored. Introduction to Component-Level Testing, John Lakos A marathon 3-hour talk on designing and testing software, particularly C++ programs (the subject of his well-known book) as a set of well-defined components. Lakos first defined these components as being "physical" i.e. sets of source files - normally a header file defining an interface, an implementation file, and a test driver file. They may also include metadata for the build system. He talked about the value of reusable components (as if this wasn't obvious!) and the risk of introducing circular dependencies - which essentially turn many components into one and reduce reusability - if we fail to keep track of dependencies. Software built out of components can be viewed as a hierarchy or stack defined by dependencies. In a large system, components with similar dependencies and dependents can be grouped together into packages or package groups to simplify a view of these dependencies. He explained that the test driver's dependencies count just as much as the implementation's, because we cannot reasonably test a component until we have tested its dependencies. Test drivers should also be written to avoid depending on environmental settings and configuration, so far as possible. Finally, and perhaps most importantly, he spent a long time talking about how to construct test cases and test data. I might try to summarise this in a later entry if anyone's interested. The Appliance of Science: Things in Computer Science that every practitioner should know, Andrei Alexandrescu Andrei began by complaining that OpenOffice Impress was even worse than PowerPoint, but being able to use Linux (specifically Ubuntu) more than made up for the pain. He briefly presented four key ideas that he believes programmers should understand and be ready to make use of:

• Dynamic programming. The Wikipedia article summarises this as "a method of solving problems exhibiting the properties of overlapping subproblems and optimal substructure (described below) that takes much less time than naive methods". As a simple example, Andrei presented a Fibonacci number generator. But since no-one really needs Fibonacci numbers, he also showed its application to calculation of the Levenshtein distance between words which is important in spell-checking and fuzzy string matching. A naive approach to either of these problems is so inefficient as to be useless.
• Garbage collection and its connection to type-safety. Any object model can only provide 2 of the following 3 features without losing type-safety: mutable objects, aliasing (multiple pointers/references to the same objects), and explicit reuse of memory (via free, delete or similar). C and C++ have all three of these. Garbage collection eliminates the need for the third. However it does require more virtual memory and/or more processing time (there is a trade-off between these). (He quoted numbers from Hertz and Berger, Quantifying the Performance of Garbage Collection vs. Explicit Memory Management, 2005. However, these numbers show the cost of GC in Java, specifically Jikes, not as an addition to C or C++.)
• Machine learning. Sometimes we don't know how to solve a problem, but we can have the program gather feedback that will allow it to learn how to do it better. This depends on the "smoothness assumption": small changes in inputs have a small effect on the result, rather than making sudden changes. Examples: image recognition, deciding whether changes to web page layout are helpful, disambiguating search terms.
• Transactional memory is the most promising approach to achieving scalability across massively parallel systems. Most hardware performance improvements over the next 10 years will come from greater parallelism, not higher clock speeds.

Writing for ACCU (BoF) I think I've been convinced to write an article about standardisation of multithreading in C++. C++ Threads, Lawrence Crowl Crowl gave an overview (similar to this) of the extensive work that's been done on standardisation of multithreading in the C++ language and library. This is one of the most important features in C++0x due to the increased parallelism mentioned above. The current semantics of multithreaded programs are not standardised, resulting in subtle but potentially serious differences between current C++ implementations that support multiple threads, and in the need for assembly-language code in programs that use lockless synchronisation. Much of this is also relevant to C and is being considered by the C standard committee. C++0x Initialisation: Lists and a Uniform Syntax, Bjarne Stroustrup Stroustrup recapped the 4 forms of syntax for initialising objects in C++, some inherited from C and some necessarily invented for C++, none of which are entirely generic. He noted the lack of a simple syntax to initialise the elements of containers, which violates the general principle that class types should have the same status as built-in types in C++. He then introduced a new initialiser syntax that he and others have defined for C++0x, which is a brace-enclosed comma-separated list of values, similar to that used for arrays and C-style structures (aggregates). Container types can define a constructor with a parameter of type const std::initializer_list<T> & and when constructed with this new form of initialiser they will receive this wrapper for a constant array of element values. For types that do not, the values in the list are treated as arguments to some other constructor, if one matches. Otherwise, for compatibility, this may be treated as aggregate initialisation. C++ Standard Library report, Alisdair Meredith There was far too much information here for me to even begin to summarise. Suffice to say there are lots of good things coming bu the library committee will have its work cut out to get them all properly specified in time for the Committe Draft coming late this year.

---

I also met old friends and new, put faces to names from newsgroups and books, and got too little sleep.

Ok, so Xfce 4.4.1 (bugfix release) is out since some days. I've packaged this new version, which is mostly ready in our svn. So 4.4.0 won't be uploaded to unstable at all, and we will upload directly 4.4.1 in sid in some days. The thing which blocked us to upload to unstable was gtk2.10, and it has been uploaded yesterday, so nothing is blocking now and we should be able to upload really soon.

The packages are available from my repository for ppc and sparc. I've changed the layout to add an UNRELEASED branch, so people using it are warned that those packages are even more unstable than unstable. Those packages are packages which have not been uploaded to debian, and thus may be unusable.

Alway prefer packages coming from pure Debian repositories, but if you want to test and help us correct mistakes, you can use the repository (and report bugs to our mailing list).

For etch users, I'll prepare 4.4.1 backports, but as I don't have hardware access, they will be only ppc and sparc binaries. Maybe other will be able to build backports for other arches.

I've decided I wanted to try running Debian on nc4000 (laptop PC by HP).I didn't want to erase what is already there, so I needed some external USB HDD to do that.Since I didn't feel like hunting around for a USB CD-ROM drive and creating bootable USB, or configuring PXE, I decided to install using qemu to install to the USB HDD and use that installation on the machine.It was surprisingly simple.I connected my USB-HDD to my desktop machine, which recognized the disk as /dev/sda, and I used qemu to install Debian on it. qemu -hda /dev/sda -cdrom XXXX -boot d.Note that you really need kqemu or kvm for speed.Then, I selected LVM, and installed.After initial reboot within qemu, I changed the kernel to a 686 kernel since a k7 kernel was installed, but that's the only package I changed.After reconnecting the USB HDD to nc4000, I managed to boot from the USB disk.Initially, it failed to boot since vgscan ran before USB disk was recognized, and could not find the root device.Looking at the source, I found initramfs will probably re-try after 3 minutes. But reading further, I found that the timing can be improved by specifying rootdelay= boot option to the kernel.I specified rootdelay=15 for the kernel option in grub,and voila, it found the root filesystem on LVM.The /boot filesystem was /dev/hda1 inside qemu, but it is now /dev/sda1 on the nc4000; and boot stopped at fsck on /dev/hda1.It was trivial to fix after dropping into the command-line.LVM is able to work around device name change, which is nice to know.For the graphics card, qemu and nc4000 have different cards, and I had to dpkg-reconfigure xserver-xorg.Okay, so I have Debian running on my notebook via USB HDD, now what ?

Debian Etch has been released lately, as you may know, and the work on Debian Lenny is already starting. For some of the DDs the work is way much different than for the rest of the DDs: the good ol' brave porters of m68k.
Stephen Marenka asked on debian-release what is needed to get back and be able to release with Lenny?
The answers are shaking:

1. To get back to testing: Basically, the release team didn't mind the
state we had with m68k in testing recently - basically, we just ignored
it, but it was still there. The release team didn't asked for full
dropping from testing, we just didn't mind either way. We also wouldn't
mind for m68k to be back in testing - as long as we can just continue to
ignore it.

2. To get released (or a realistic chance to): Basically, make sure that
the toolchain works (see the other mails), and get really fast buildds
(otherwise, the security team would probably veto the inclusion). The
second might happen with Coldfire, but - the how is your part, not ours
(i.e. if you do it some other way, that's ok as well of course).

Having said that, I can understand if the ftp-masters don't want to have
an architecture be in testing unless there is reason to believe it is
possible to include it in the next stable release, but that's not the
domain of the release team.

ad 1)
Ah, ok... as long as we are ignored, everything is fine. But how will testing m68k get updated when the testing scripts ignore m68k? Pardon me, but isn't this a little bit strange, at least?

ad 2)
Getting really fast buildds - good joke, indeed! Or is this a Friday, 13th habit like April Fools Day?

Conclusion)
Let's summarize: we can release with Lenny, when we get back in shape - but testing is needed for this, where we're ignored for. And without faster buildds for an arch such as m68k, there's simply no way back to make it into a release again - because of this.
Describing Coldfire as such a possibility for faster machine seems a little bit strange to me. CF is totally new, there's not progress yet on the CF front to my knowledge and a combined m68k/CF port may introduce other drawbacks, which nobody can predict for sure today. Do you really believe that ftp-masters or release team will allow such a port to get released? I don't.
Getting faster buildd in a reasonable amount of time is illusory for m68k. We can get more buildds online, but not "really fast buildds". Despite one older blog post of me, I'll receive just another '060 m68k soon (to be hosted by a fellow DD). A few days ago, two other m68k buildds were added to the m68k buildd rotation list. That will make 3 more buildds for the m68k.

Anyway, in most cases your first impression is the correct one and my first impression of the Vancouver Massacre/Proposal was: this is a plan to drop m68k!
Well, the lastest incidents confirm this impression, IMHO.
After all, I think that the m68k port would be best served by dropping Debian instead of being dropped by the project. The m68k port - de facto - has been dropped already. I'm disappointed.

Waking up at 5am (I know, not a decent hour, though I don't choose when a plane leaves), cycling to the railway station with all my luggage (no buses drive that early...), taking the train (actually 3 trains as there are 2 stops) for 1.5h, arriving at the airport, hearing about some strike, checking in and being told strike will be over when my plane takes off (9am), waiting, plane being delayed till 10am, waiting, plane being delayed till 11am, waiting, plane being delayed to 1pm, eating, waiting, plane being delayed till 3pm, queueing for information (which was going to take ages), plane being cancelled. Apparantly some firemen didn't like that they didn't get food tickets nor paid overhours I should have known that taking a plane to Spain is risky. My previous attempt: ... arriving in the airport, trying to be checked in, being told that tickets have been cancelled, making calls, being told that travel agency forgot to confirm a second time and that new tickets have been booked for 2 days later, going home, night before take off seeing that ticket to Spain has been cancelled (ticket back still valid), going to the airport anyway, being called that everything is ok, arriving at the airport, being told that I have 2 tickets for the same flight... Sorry to the people who expected me to be in Spain, I'm not going to try my luck again tomorrow, though I'll try to be happy on IRC

I just read about this really cool hack: He installed the well known Unix-Tool xeyes in his window no, not Windows, window. The thing to look out of your room if the shades are not down. (Video available)The effect is that the innocent bywalkers on the street will feel, well, watched. Not like otherwise where they probably are watched and don t know about it. Might be a very cool gimmic for privacy related PR actions, or just to generally protest against state surveilliance...

OpenedHand have announced Pimlico, an umbrella project for our PIM applications: Contacts, Dates, Tasks and Sync (in progress). Released at the same time was Dates 0.4: this release is a vast improvement on the previous release and now available in three flavours: desktop, Maemo, and OpenMoko. At the same time we sneaked out an updated set of evolution-data-server packages for Maemo 3.0. As far as the user is concerned the only changes are that addressbook syncing will work (using OpenSync or SyncEvolution), and that WebCal is supported (as used by Dates). NP: Urban Biology, Machine Drum

In response to Stefano's reply to my post about Coq:

toy-ness

Usually such pieces of software are more seen as academic toys, seeing it promoted as a geeky software is a joy to me.

I really hate to detract from your happiness -- but I definitely had the word "math" implicitly stuck before "geek toy". :) Most coders that I've met are, at best, indifferent to mathematics, outright hostile at worst, so I doubt that coq will get far outside of small niches where the benefits are irresistable (security software, maybe? -- and of course pure math). I read up on this stuff despite it being utterly useless for anything I'm likely to be paid to do (unless I return to academia) because I'm a hopeless weenie loser who likes math. :P

programming-language-ness

Coq is not really a programming language (even if the CoqArt book says so, to attract the developer community), it is mainly a proof assistant; i.e. a software in which you give definitions, state theorems and prove them.

Well, this all depends on how you define programming language, which is one of those never-ending definitional questions. But using my general conception of what a programming language is, I actually have noticed two programming languages inside Coq.

1. The more interesting one (IMO) is the proof language they call "Gallina" and the "Vernacular", in which you construct proofs by reducing your goal to a tautology. The goal here seems to be to compute a well-typed term in Coq's core calculus, and you generally construct programs interactively: Coq shows you the current program state (expressed as one or more goals and a set of local hypotheses) and you manipulate it by (e.g.) applying lemmas to yield new known facts.
   
   I think this language is fairly weak viewed as a programming language. It's completely unreadable and the proofs generated don't correspond to the reasoning process one normally follows when working out a proof. I am less annoyed by it now that I've learned how to do forward reasoning :), but it seems like backwards reasoning is really pushed, and that just seems .. well .. backwards to me. There's also a strong tendency to use implicitly generated names for things and to just "know" what the state of the system after issuing a command will be; as far as I can tell, the only way to understand a Coq proof is to step through it in coqide.
   
   While it's nice to be able to "trust the kernel", it would also be nice to be able to "read a proof" directly. Among other things, their inscrutability makes me think that Coq proofs would be annoying to maintain over time. This is not necessary for stuff that's part of Principia Mathematica, but seems likely if you're developing a new theory -- in particular, I would shudder (theatrically) at the mere thought of including embedded Coq proofs in my programs.
   
   Some additional brief comments on this were written by Nick Benton and put on the Web under the title of Machine Obstructed Proof.
   
   
2. The other sense is that Coq directly embeds a strongly normalizing variant of System F (I think? -- at least the simply typed LC with some type parametricity) with primitive recursion over inductive structures. This isn't Turing-complete (of course!), but I would certainly call it a real programming language.
   
   As you note, though, this is less interesting unless you want to prove facts about your programs.
   

OCaml-ness

Regarding the synergy of Coq with OCaml: yes, you can prove in Coq the correctness of OCaml program, but actually not more than you can do with programs written in other programming languages

I was specifically thinking about Concoqtion; I have no idea how other projects compare, and I apologize if I incorrectly summarized it!

Perhaps the most accessible documentation on what PIC code is and how an ELF dynamic linker works is John Levine s Linkers and Loaders (and it has amazing sketches, too!). The Gentoo documentation also has an Introduction to Position Independent Code. I d like to give a few hints on how to fix the shlib-with-non-pic-code lintian error caused by inline assembly on the i386 and amd64 platforms, as well as build errors that may occur due to inline assembly being used. I m not going to cover the trivial all objects were not built using gcc s -fPIC flag problem. It usually requires a fix to the build system, not to the code. gcc can t find a register (i386) PIC on i386 uses a register to store the GOT (global offset table) address. This register is usually %ebx, making it unavailable for use by inline assembly (and also restricting the compiler's register usage when compiling C or C++ code). So the following perfectly valid code will not build with the -fPIC flag:

 void cpuid(uint32_t op, uint32_t reg[4])
  
    asm volatile("cpuid"
                 : "=a"(reg[0]), "=b"(reg[1]), "=c"(reg[2]), "=d"(reg[3])
                 : "a"(op)
                 : "cc");
  

Using -fPIC, gcc will say something around the lines of error: can't find a register in class BREG while reloading asm . Several things need to be done to fix this:

• use a register other than %ebx
• save %ebx if it risks being clobbered by the assembly code, and don't tell gcc about %ebx at all (it doesn't need to know anyway)
• if we saved %ebx by pushing it on the stack, make sure the inline assembly code takes the new stack offset into account

And here is the PIC-compliant version:

 void cpuid(uint32_t op, uint32_t reg[4])
  
    asm volatile("pushl %%ebx      \n\t" /* save %ebx */
                 "cpuid            \n\t"
                 "movl %%ebx, %1   \n\t" /* save what cpuid just put in %ebx */
                 "popl %%ebx       \n\t" /* restore the old %ebx */
                 : "=a"(reg[0]), "=r"(reg[1]), "=c"(reg[2]), "=d"(reg[3])
                 : "a"(op)
                 : "cc");
  

using variables from assembly code (i386) Directly using variables from inline assembly always creates non-PIC code even if -fPIC is being used. There are at least three strategies to consider when trying to fix this. Here is an example of PIC-incompatible code:

 extern uint32_t sym1, sym2, sym3;
 
 void store_in_symbols(uint32_t x)
  
     asm volatile("movl %0, sym1   \n\t"
                  /* ... */
                  "movl %0, sym2   \n\t"
                  /* ... */
                  "movl %0, sym3   \n\t"
                  : : "r"(x));
  

The first strategy is to pass the variable through the usual gcc inline assembly syntax. This is not always possible because there might be a shortage of registers, but here is what it looks like:

 extern uint32_t sym1, sym2, sym3;
 
 void store_in_symbols(uint32_t x)
  
     asm volatile("movl %3, %0   \n\t"
                  /* ... */
                  "movl %3, %1   \n\t"
                  /* ... */
                  "movl %3, %2   \n\t"
                  : "=r"(sym1), "=r"(sym2), "=r"(sym3) : "r"(x));
  

If there are too many variables and not enough registers, one can use the second strategy: put all required variables (or addresses) in a table, thus requiring only one extra register:

 extern uint32_t sym1, sym2, sym3;
 
 void store_in_symbols(uint32_t x)
  
     uint32_t tab[3];
     asm volatile("movl %1, (%0)   \n\t"
                  /* ... */
                  "movl %1, 4(%0)  \n\t"
                  /* ... */
                  "movl %1, 8(%0)  \n\t"
                  : : "r"(tab), "r"(x));
     sym1 = tab[0];
     sym2 = tab[1];
     sym3 = tab[2];
  

If the second method happens to be unsuitable because variable types differ too much, there is a third, more complicated strategy: retrieve the GOT address using some linker-specific magic, then use @GOT addressing to access variables:

 extern uint32_t sym1, sym2, sym3;
 
 void store_in_symbols(uint32_t x)
  
     uint32_t got;
     asm volatile("call get_got                               \n\t"
                  "get_got:                                   \n\t"
                  "popl %0                                    \n\t"
                  "addl $_GLOBAL_OFFSET_TABLE_ - get_got, %0  \n\t"
                  : "=r"(got));
     /* ... */
     asm volatile("pushl %%ebx                \n\t" /* save ebx */
                  "movl sym1@GOT(%0), %%ebx   \n\t" /* retrieve sym1 address */
                  "movl %1,( %%ebx)           \n\t"
                  /* ... */
                  "movl sym2@GOT(%0), %%ebx   \n\t" /* retrieve sym2 address */
                  "movl %1, (%%ebx)           \n\t"
                  /* ... */
                  "movl sym3@GOT(%0), %%ebx   \n\t" /* retrieve sym3 address */
                  "movl %1, (%%ebx)           \n\t"
                  "popl %%ebx                 \n\t" /* restore ebx */
                  : : "r"(got), "r"(x));
  

It is of course possible to merge the two above asm statements into one. However the performance implications of the GOT address retrieval should not be overlooked, especially if the assembly code takes place inside a critical loop, which means the loop might have to be unrolled and the GOT address retrieval done outside the loop. Note also that _GLOBAL_OFFSET_TABLE_ is ELF-specific. On BSD and a.out systems you should use __GLOBAL_OFFSET_TABLE_. I am not aware of a way to do the same on Darwin / Mac OS X, which is a shame because it is the only widespread i386 platform where shared objects cannot be non-PIC. gcc silently generates non-PIC code (amd64) The amd64 architecture has the same problem as i386 when trying to access variables from assembly code. However, the solution is a lot easier: the GOT address can always be retrieved from the special %rip register. This is called rip-relative addressing:

 extern uint32_t sym1, sym2, sym3;
 
 void store_in_symbols(uint32_t x)
  
     asm volatile("pushl %%rax                       \n\t" /* save rax */
                  "movq sym1@GOTPCREL(%rip), %%rax   \n\t" /* retrieve sym1 address */
                  "movl %0,(%%rax)                   \n\t"
                  /* ... */
                  "movq sym2@GOTPCREL(%rip), %%rax   \n\t" /* retrieve sym2 address */
                  "movl %0,(%%rax)                   \n\t"
                  /* ... */
                  "movq sym3@GOTPCREL(%rip), %%rax   \n\t" /* retrieve sym3 address */
                  "movl %0,(%%rax)                   \n\t"
                  "popl %%rax                        \n\t" /* restore rax */
                  : : "r"(x));
  

amd64 also has more registers than i386, which makes the first discussed technique less prone to register shortage. Why bother? Good question. Currently many shared libraries simply disable entire chunks of inline assembly on the amd64 architecture. Whether these routines bring any performance benefit compared to the C alternative is uncertain, but without being able to build the code there is no way to tell. Even fewer developers care about i386, because this architecture can handle non-PIC shared objects quite fine (but then they are no longer really shared). I am in favour of making all shared objects PIC-friendly (ie. not necessarily build them as PIC if there is a valid reason not to, but at least don t make it too difficult to build PIC versions) because that makes it easier to port the code to Darwin. I also believe the memory gain and especially the cache gain when using real shared libraries is underestimated.

Gunnar Wolf commented on my recent series of init script posts. Thanks for the feedback.The thing is that I've been doing a lot of Debian stuff these days (such as our GSoC coordination, and lots of stuff you saw in this blog). But I shouldn't... instead I should be trying to finally get my diploma thesis done.I had my exams last july. If I had been focused, I could have been finished with my degree last christmas. Now it will likely by this fall. :-( All that is remaining is my final thesis. It should be written in 6 months, but I still havn't submitted my topic yet. So if you are wondering what took me 8 months already: I investigated two topics, but they didn't work out. Then I started researching for my 'current' topic, and I have a vision for it, but I still havn't managed to convince my professor (which of course is largely due to me not being able to convey this idea, and that the topic is in information management and representation, where it's probaly often hard to find a solid theoretical part. I have examples I'd like to do, and I hope they are actually convincing, but I have no idea how to 'prove' something here; it's too much about social aspects and human perception I guess, and I'm not really qualified or capable to do solid research in that field.)(And as you can probably imagine, I'm pretty fed up with studying now. A year ago, I'd have considered to go for a PhD afterwards, but now I just want to get out, and work on my projects.)Anyway, I got distracted. Back to the init topic.I already posted this link: Init Hackfest, as well as a pointer to the initscripts-ng mailing list. I'd really like to get this topic on it's way for lenny. So please, join this effort. While I'd like to push it, I shouldn't; on contrary, I should suspend my Debian efforts.As for your (Gunnar) suggestion on how to handle multiple inits: what's wrong with just including multiple init scripts in one package? They are not particularly large and compress well. A single init script package might already be bigger than all the init scripts.The problem is more on how to write these init scripts; by the time we're done with this project, we might actually have hit 1000 init scripts. Yes; I didn't just make up this number: packages.debian.org has 890 hits for /etc/init.d; the init-like scripts in /etc/dbus-1/event.d not included. So if just want to add support to 5 alternate init systems, this means we would need to write 5000 new init scripts. Ouch. So we maybe should convert them to 1000 meta-init-scripts instead, and generate the 6000 real init scripts from them.Please join the discussion, and lets get this project going, despite having to face such scaringly high numbers...

A lot of people have asked me about the Tim Wu piece in Slate about Wikitravel. I've commented already about the article in Wikitravel's Logbook: 8 April 2007. The upshot: Wu got a lot of things really wrong about Wikitravel (like saying we didn't have any hotel listings for Bangkok). But if our readers get things wrong, that means Wikitravellers need to do some more work to help them get things right. User errors are developer errors writ large; we can do better. But I think Tim Wu missed the main reason Wikitravel guides will always be better than proprietary guides: they are Free Cultural Works. Where I come from, that means a lot. I wrote to Tim about his article, and he responded really kindly. The thing I liked most is that when he got home, he updated Wikitravel with the information he picked up on his trip to Thailand. That is pretty cool. I'm not taking others' responses to the article too seriously. Wikitravel got a Webby Award nomination for Best Travel Site the same week. As Tom Wolfe said, "The lead dog is the one they always try to bite in the ass." And Oscar Wilde: "The only thing worse than being talked about is not being talked about." (Thanks to Hugh!) tags: wikitravel travel tim wu slate

Off to San Juan We're getting prepared this morning to head out to Puerto Rico for the first annual World-wide Wikitravel Get-together. Our flight leaves at 2:20PM, so we've got kind of a leisurely morning to get ready. As usual, we're pretty tense: what are we forgetting? And I usually get hung up on unnecessary stuff to deal with. I should probably print out the Spanish phrasebook from Wikitravel, because my Spanish is pretty rusty. And I should download all the WT pages about Puerto Rico destinations to my laptop, just to know where we should fill in. And we can't find the charger for our camera's battery. And I want to bring a certain pair of cargo pants, but there's a button missing. Do I have time to sew on another? Of course, all these anxieties melt once we get out of the house and into a taxi to the airport. Once you can't do things, you stop worrying about what you should do. I think. The more I learn about Puerto Rico, the more fascinated I am. San Juan (Puerto Rico)'s Old Town is a UNESCO World Heritage site. The giant telescope at Arecibo is world-renowned as the SETI site. The islands of Vieques and Culebra are isolated and fun, and Fajardo and Rincon has world-class surfing. My friend Biella Coleman gave me a great list of travel tips for San Juan and Culebra, which have been a big help in our planning. OK, enough blogging: gotta get to work sewing on that button. (I've been stalling long enough for Maj to have found the camera charger under a baseball hat. Yay!) tags: get-together07 wikitravel travel Puerto+Rico biellacoleman

I just upgraded my home server (kheops) to etch. I just had an issue with lifelines-report, which the former maintainer should be ashamed of. Apart from that glitch, the upgrade went well, thanks to the excellent upgrade advices in the Release Notes. Funnily, I got warned that kheops still has the "base" package installed which I didn't notice in former upgrades. This is pretty expected as kheops has never been reinstalled, is my home server since october 1996 when I installed Debian Buzz on it. I think that I'll just avoid following Santiago Vila's advice and leave "base" installed on my system, indeed. Just for fun. This is the 6th server I upgrade as of now. All other servers were pretty minor servers at work. The big issue is now upgrading the file and print servers and particularly the samba file server which hosts files for 250 users. This is probably the one and only server for which we will setup a clone and play the upgrade scenario a few times. Etch rocks, as expected.

Today's post (previous parts) tries to show what benefits other init systems can offer.I'm picking 'minit' as an example; not because it's the ultimate solution or anything, but because I happen to know it quite well. I used to be the package maintainer for some time, and I've also used it on an embedded system (where it was a perfect match). However, it's not very distribution-friendly; it's great when you can invest some time into configuring it (such as when building an embedded system) but not when it should ideally work automatically out of the box. The Debian package is still there, but it's orphaned. Still it's well written, clean code; there shouldn't be any bigger bugs in there.The first thing to notice is that minit is tiny. It's statically linked, but only about 6.5 kb. Regular init is 86 kb plus some libraries (despite actually not doing much during init!). This makes it very well-suited for embedding.But now for the benefits of minit over regular init:1. Service monitoring. This is the biggest point: minit will keep an eye on the services and restart them when needed. (Another thing very useful in embedded situations, btw.)Minit doesn't use the start/stop paradigm of sysv/shell-style-init. Instead it has a "run" script for every service. If a service is flagged as 'respawn', it will automatically be restarted when the run script quits; therefore the applications should not background themselves. This offers some benefits, for example the init process will be notified by the kernel of a termination, and can immediately respawn the service; in contrast to other monitoring systems suchas monit it doesn't rely on 'busy polling'.2. Dependency handling. Minit handles startup dependencies. If a service lists another service as a dependency, it will bring that service up first.3. Parallel startup. Unless an ordering is given via dependencies, minit will start the services in parallel.4. Control apps. In minit, you don't start a service by running some random shell script. Instead you call "msvc -u servicename", which will signal minit to bring up the service (and it's dependencies). This would (there currently is no policy for minit) be especially useful in SELinux, but it also allows to e.g. give a user the permission of running "sudo msvc -u" without introducing a huge security hole, since this cannot be used to execute arbitrary scripts. "msvc" can also be used to check if a service is up. (Another benefit for embedded systems, since that removes the need for pid files, which then need a ram (writeable) filesystem)I'll also give an example for a minit service:A service is represented by a directory in /etc/minit which can contain various files. I'm going to look at getty/1 example; which will bring up a console login on the first terminal. There are three files:

• a symlink named 'run' to /sbin/getty, which is the daemon we want to bring up
• a file named 'params' containing the parameters for the getty application, one per line
• an empty file named 'respawn', since we want the login screen to reappear

If the service would need it, there could also be a file called "depends" listing service dependencies.This allows minit to start the getty with no overhead. There is no shell script being executed or anything (note: sysvinit also does this for gettys, but it's not being used much for other services).Of course minit isn't perfect. I've already said that it isn't particularly distribution friendly. This is because it uses a lot of magic files (e.g. an empty file named 'respawn' to flag a service as needing to be respawned), and it's configuration files aren't easy to handle automatically. The often frowned-upon 'symlink farms' of sysv-shell-style init are much better here.Another thing that is missing from minit is an automatic rollback of dependencies. If we told minit to bring up service 'a', which lists dependencies 'b' and 'c', minit will bring these three up. If we tell it to shutdown 'a', it will however not shutdown 'b' or 'c'. And unfortunately, even if we shutdown 'b', it won't shutdown 'a'. If you want a more complex dependency handling (or things such as runlevels), you need to do this externally. (An example is my 'enitdir' app, it adds runlevel support to minit, and allows switching of runlevels by replacing a symlink; Services are added and removed from runlevels by adding a symlink to the service into the runlevels directory. By using dnotify, this also takes virtually no CPU time.)Minit also lacks a differentiation between a 'started' and a 'running' service. It has support for 'sync' services, such as e.g. mounting a filesystem. It will wait for this operation to complete before it starts services depending on it. However, there are services that will continue to run, while not being instantly available when the exec() was called. Sometimes it would be nice if a service could go into a 'started' state, then do some cleanup work, and then when actually running the real service signal minit that it's now 'running'.As a last malus, minit won't print progress messages to the console. A minit boot is pretty much silent, directing it's output to some logging processes. There are good reasons for this (since services are started in parallel!), but it makes debugging harder, and users might be wondering what their system is actually doing (though some users prefer to see a colored 'busy' bar)...I guess this concludes this series of blog posts. Maybe someone can follow up with some details on upstart (which seems to be the most promising init replacement)? I hope to have given an impression on where there is room for improvement in our current init to one or the other. Now we just need some people to actually work on a better support for other inits in Debian...

Carpenter Mukti Mitchell has set off on a tour of the coastline of Britain in a zero-emission microyacht promoting low carbon lifestyles. During the course of his voyage, he will be stopping at 40 ports and towns around the country with events organised to promote low carbon products and services. He doesn't call at Clevedon or King's Lynn (I guess they're too far inland) but he's currently in South Wales and should be calling at Wells-next-the-Sea on 21 July and Sheringham on 23. More dates The Phone Co-op (agent 471) is a co-sponsor of this event with, among others, Ecotricity, Resurgence and Co-operative membership so at least two of my co-operatives are supporting it.

First post for Debian Planet. It was about time that I decided to finally post here :) But now, I have a good excuse: I've been selected to work in the Google Summer of Code for Debian! Yay!! So this feed will be used to post about it and keep the community informed. Stay tuned!