It seems that after my last post talking about the abusive position of teosto regarding the songs in the game Frets on Fire, even when the authors of the game and songs were willing to put them under a free (DFSG) license, many people is asking about whether FoF could enter Debian even without songs. I don’t really see any problem with that, in fact the game is already packaged in our subversion server (Debian Games Team). In my opinion the game itself could be into main, and the songs, well, I guess the only possibility would be to put them in non-free. There are free and non-free songs all around the net, and in fact it’s quite easy to make them, so I don’t think the lack of songs would be a problem. The game doesn’t need to depend at all in the original songs. The problem is that most of the songs out there are based in music protected by copyright, and/or they’re not easy enough for beginners. We should need to have at least a replacement of the tutorial, so that people could downoad the game and try it.

It’s been a crazy few weeks. I’ve heard the phrase “drinking from the fire hose” many times, and while I’ve never actually tried to do that, I suspect the experience is something like this. I’m having a blast though. A few quick first impressions:

1. First things first: The lead up to the announcement was remarkable—not at all what I expected from a big company. As Jim Grisanzio pointed out, the PR people did a great job coordinating everything, but the bloggers were very much in the lead here. In fact, I was given so little (OK, no) guidance on what “the message” was supposed to be that I sent a draft of what I was going to post to Jonathan, and he replied, no one is allowed to run their posts by me, just speak your mind, that’s what we all do. The message: Sun really is as transparent as it appears from the outside perspective.
2. In the other direction, the people here are not only open to the outside perspective, they want to hear it. That’s a big part of the reason I’m here now. Solaris has lost a lot of developer mindshare to Linux over the past 5-10 years. There are important lessons to be learned from that, and we (yes, I feel like it’s “we” already) are intent on doing just that.
3. Maybe it’s just the people I’m working with, but this place feels like a startup. There’s a lot of positive energy, a real sense of urgency, and people genuinely seem to love what they’re doing. The difference: It’s a startup with actual resources. You know, like 30,000+ people. A potent combination indeed.

Regular blogging should resume shortly.

We are having a release party on Saturday the 14th of April. We meet at mid-day under the clocks at Flinders Street Station and then go somewhere convenient and not too expensive for lunch.

All welcome.

Update:

The event was moderately successful. There were only six people including me - that was quite a bit smaller than the Debian 10th birthday party we had in Melbourne, but it was still enough to have fun.

Everyone there had a good knowledge of Linux and Debian and many interesting things were discussed. We had lunch at a Japanese stone-grill restaurant - their specialty is serving raw ingredients along with a stone that's at 400C (or so they claim - I would expect a 400C stone to radiate more heat than I experienced on my previous visit). As it was a warm day we skipped the stone grill and ordered from the lunch menu (which was also a lot cheaper). Some of the guys had never tried Sake or Plum Wine before, they seemed to like it. Strangely the waitress always wanted to deliver alcohol to a 15yo in preference to almost anyone else.

One of the topics of discussion was Linux meetings and the ability to attend them. A point was made that if you are 18yo and rely on your parents' permission to do things then a meeting that finishes at 9PM isn't a viable option. It has previously been noted that for people from regional areas an evening meeting is also inconvenient.

Maybe we should have occasional LUG meetings on a Saturday afternoon to cater for the needs of such people?

So, this is my last evening (and when I remember correctly the first one I’m at home) of my holidays. All began last Friday called “Karfreitag”, a national holiday. To be correct, the first party was already on Thursday and they continued daily till Monday evening. The best event in this time was Sunday afternoon, some minutes after I waked up Debian GNU/Linux 4.0 aka Etch was released, what a feeling. It was especially interesting because it’s the first stable release with some of my packages and it was a great experience to see how releasing and planning works. I often think back to the two bug squashing parties in Zurich, one in September and one in October iirc. Back to my holidays: First it was planned that I go with my sister to Zurich from Monday evening till Thursday evening, but then I got a appalling call from madduck who told me that he is in the hospital in Zurich with several broken bones. We planned to sleep in madducks flat and so we couldn’t get the key and went Tuesday morning. Walking through Zurich, getting some informational material from the University Zurich for my Sister (she maybe wants to study there) and the visiting madduck. After the visit we went again to the center of the city and walked without a real destination. In the evening we finally went to Oerlikon and went to “our” flat. After installing our sleep places (goddamn, I want to have a waterbed) we went to Migros to buy some food and drinks for the evening and the next day. The next day we really got up quite early and visited Zurich with a city guide, a lot of sightseeing and hanging in some bars. In the afternoon Daniel Baumann also was in Zurich and we went to have a drink together and also Tobias Ast came and staid with us till the next evening. We really had fun together and the main thing we did on Thursday was relaxing on the lake of Zurich and in the evening we went home. Then it was finally friday, I looked forward to this day for about one week, why? The release party in Zurich!! It was my first release party on the Irchel area in Zurich, we grilled, had some beer together and talked about god and the world. It also was the first time i met Adrian von Bidder and Mathias St rmer in person, especially with Adrian i had a longer interesting discussion about network devices from Cisco and their special prices. Saturday was already the next party day. When I was 13 we had a supply teacher for our sick teacher for 2 months, I was her horror pupil. Fine - some weeks ago I met her again in a Pub in the evening and we staid together the whole evening and became good friends in a way, so I was also invited (as the only of her pupils) to her flat share party. I met several “new” people and some of them (mostly IT students) recognized my Debian t-shirt and I had some interesting discussions about free software and had to explain the philosophy of Debian to some of them. So yes, now it’s sunday, I’m more tired than before my holidays but I’m really happy, my holidays were great because of some adventures. Regards to all to my friends… Tomorrow I will continue working and have several tasks open, especially upgrading some Debian servers from Sarge to Etch (Spamfilters, LDAP, Proxy servers…) and I hope I won’t have many troubles. In addition I have also some tasks open for Debian which I’d like to complete this week.

So, today I tried to sign up for a service online (unidentified for reasons that will become obvious briefly). This is a service that, had I successfully signed up, would be withdrawing monthly subscription fees from my bank account and handling some data that's fairly important to me. But when I hit the button to start the registration process, I got back...

[TCX][MyODBC]You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near ''7GDiSp=E8-Mgvgo\', Now() )' at line 9

SQL = "INSERT INTO [REDACTED] ([REDACTED], FieldName, FieldValue, update_dt ) VALUES ( 10724, 'PASSWORD', '7GDiSp=E8-Mgvgo\', Now() )"

The password I tried to enter, for the record, was 7GDiSp=E8-Mgvgo\#dEwb8m7Ec_e~z0myAAj. This is at best inexcusable sloppiness and at worst a security hole, and I don't want a company that does either anywhere near my money and data. (sadly, there probably are already many such companies near my money and data, but I try to avoid the ones I know about)

I've been meaning to attend the ACCU Conference for some years and finally decided to pay my own way for a couple of days this year. It was an opportunity to hear world experts on C++ and other programming topics and to a smaller extent to meet and socialise with other professional programmers. I attended the following sessions: Keynote: Improving Collaboration in Open Source Projects, Mark Shuttleworth This was a somewhat surprising choice of speaker and subject since I think most attendees work on proprietary software, but I think any sensible developer in the proprietary world will try to build on free libraries under non-copyleft licences, where appropriate. Mark talked about governance and leadership in "open source" projects (actually praising Debian for having a clear constitution) and about the need for good communication between projects, for example about release schedules. He seems to have accepted that Launchpad cannot be a single hub, and called for standard formats for, for example, exchanging information about related bugs between bug trackers. But it doesn't sound like he's ready to take the lead on that. Choose your Poison: Exceptions or Error Codes, Andrei Alexandrescu Andrei looked at the strengths and weaknesses of various error reporting mechanisms in C++ and introduced a technique for making error handling more flexible. Normally a function is designed either to indicate errors through its return value or to throw an exception on error. If a function returns an error indicator, it's easy for callers to ignore it. If it throws an exception on error, then a caller that provokes errors more often than the called function's author anticipated pays a high cost in processing time for those errors. Andrei's solution involves a class template Likely<typename T> whose instances hold either a return value of type T or an exception. A function that can fail has a specialisation of this template as its return type. The caller can check for failure if it knows how to handle it, or it can ignore it - in which case the conversion to T or the destructor will throw the exception. This is clever but doesn't seem to be entirely satisfactory. The destructor which throws does first check that no exception is already being processed, avoiding collision of the two exceptions and consequent abrupt termination of the program, but this can result in errors being unexpectedly ignored. Introduction to Component-Level Testing, John Lakos A marathon 3-hour talk on designing and testing software, particularly C++ programs (the subject of his well-known book) as a set of well-defined components. Lakos first defined these components as being "physical" i.e. sets of source files - normally a header file defining an interface, an implementation file, and a test driver file. They may also include metadata for the build system. He talked about the value of reusable components (as if this wasn't obvious!) and the risk of introducing circular dependencies - which essentially turn many components into one and reduce reusability - if we fail to keep track of dependencies. Software built out of components can be viewed as a hierarchy or stack defined by dependencies. In a large system, components with similar dependencies and dependents can be grouped together into packages or package groups to simplify a view of these dependencies. He explained that the test driver's dependencies count just as much as the implementation's, because we cannot reasonably test a component until we have tested its dependencies. Test drivers should also be written to avoid depending on environmental settings and configuration, so far as possible. Finally, and perhaps most importantly, he spent a long time talking about how to construct test cases and test data. I might try to summarise this in a later entry if anyone's interested. The Appliance of Science: Things in Computer Science that every practitioner should know, Andrei Alexandrescu Andrei began by complaining that OpenOffice Impress was even worse than PowerPoint, but being able to use Linux (specifically Ubuntu) more than made up for the pain. He briefly presented four key ideas that he believes programmers should understand and be ready to make use of:

• Dynamic programming. The Wikipedia article summarises this as "a method of solving problems exhibiting the properties of overlapping subproblems and optimal substructure (described below) that takes much less time than naive methods". As a simple example, Andrei presented a Fibonacci number generator. But since no-one really needs Fibonacci numbers, he also showed its application to calculation of the Levenshtein distance between words which is important in spell-checking and fuzzy string matching. A naive approach to either of these problems is so inefficient as to be useless.
• Garbage collection and its connection to type-safety. Any object model can only provide 2 of the following 3 features without losing type-safety: mutable objects, aliasing (multiple pointers/references to the same objects), and explicit reuse of memory (via free, delete or similar). C and C++ have all three of these. Garbage collection eliminates the need for the third. However it does require more virtual memory and/or more processing time (there is a trade-off between these). (He quoted numbers from Hertz and Berger, Quantifying the Performance of Garbage Collection vs. Explicit Memory Management, 2005. However, these numbers show the cost of GC in Java, specifically Jikes, not as an addition to C or C++.)
• Machine learning. Sometimes we don't know how to solve a problem, but we can have the program gather feedback that will allow it to learn how to do it better. This depends on the "smoothness assumption": small changes in inputs have a small effect on the result, rather than making sudden changes. Examples: image recognition, deciding whether changes to web page layout are helpful, disambiguating search terms.
• Transactional memory is the most promising approach to achieving scalability across massively parallel systems. Most hardware performance improvements over the next 10 years will come from greater parallelism, not higher clock speeds.

Writing for ACCU (BoF) I think I've been convinced to write an article about standardisation of multithreading in C++. C++ Threads, Lawrence Crowl Crowl gave an overview (similar to this) of the extensive work that's been done on standardisation of multithreading in the C++ language and library. This is one of the most important features in C++0x due to the increased parallelism mentioned above. The current semantics of multithreaded programs are not standardised, resulting in subtle but potentially serious differences between current C++ implementations that support multiple threads, and in the need for assembly-language code in programs that use lockless synchronisation. Much of this is also relevant to C and is being considered by the C standard committee. C++0x Initialisation: Lists and a Uniform Syntax, Bjarne Stroustrup Stroustrup recapped the 4 forms of syntax for initialising objects in C++, some inherited from C and some necessarily invented for C++, none of which are entirely generic. He noted the lack of a simple syntax to initialise the elements of containers, which violates the general principle that class types should have the same status as built-in types in C++. He then introduced a new initialiser syntax that he and others have defined for C++0x, which is a brace-enclosed comma-separated list of values, similar to that used for arrays and C-style structures (aggregates). Container types can define a constructor with a parameter of type const std::initializer_list<T> & and when constructed with this new form of initialiser they will receive this wrapper for a constant array of element values. For types that do not, the values in the list are treated as arguments to some other constructor, if one matches. Otherwise, for compatibility, this may be treated as aggregate initialisation. C++ Standard Library report, Alisdair Meredith There was far too much information here for me to even begin to summarise. Suffice to say there are lots of good things coming bu the library committee will have its work cut out to get them all properly specified in time for the Committe Draft coming late this year.

---

I also met old friends and new, put faces to names from newsgroups and books, and got too little sleep.

After wearing heavy shoes and army boots all winter, the first day wearing regular chucks again feels like walking barefoot.

It's spring! :)

I was a long-time hold-out user of standalone RSS aggregators (SharpReader for Windows and Straw on GNU/Linux, both of which have been somewhat stagnant over the last couple of years). I’ve finally thrown in the towel and switched to Google Reader. It was a great decision. The key factor is mobility. I check email and blogs from my personal laptop; our kitchen computer; my office workstation; and, perhaps most significantly for Google Reader, from my cell phone. The Google Reader cell phone app is quite good (handy “keyboard” shortcuts, efficient presentation), and allows me to catch up during time that is difficult to use for other purposes: waiting for the train, standing in line at Starbucks, on the elevator. The only real downside is the lack of offline support (although I don’t think it would take a rocket scientist to create it), but this is far outweighed by the mobile functionality. As “access anywhere” becomes more important than “offline access,” I predict we’ll see more users of all applications make this sort of switch.

Just yesterday I mentioned that DBus contains an own set of init-like scripts. From todays new DBus package in unstable:

  * debian/dbus.init
    + Add functionality to start/stop dbus dependent services based on their
      LSB header.
      Instead of installing an init script into /etc/dbus-1/event.d, services
      depending on dbus should from now on install a regular sysv init script
      and add an LSB header with "Required-Start: dbus".

So the dbus init script now adds a 'workaround' (likely, since this is done in DBus, not in sysvinit) for the missing dependency handling in sysv-style init, by taking care of the restarting itself. This is good, because init scripts aren't distributed into two places anymore (soon).It probably highlights why we need to work on our init.

In his “Init followup” post, Erich posted a reference to his Debian-Wiki post about a proposed Debian HackFest after Etch was released (which would be about 5 days ago<smile />). Anyway, since that page at the wiki is protected, which means I can’t edit it, I’m posting my thoughts here. I agree with Erich and some other commentors at Gunnar Wolf’s blog post that it doesn’t make sense to add X init-script-only-packages for any daemon to support X init systems. Instead, each package containing one or more daemons should contain the meta information needed to create all init scripts. Basically, this would probably include script snippets which do the following operations:

• Pre-check: Check wether the package actually is installed rather than in removed/configured status.
• start: Start the daemon in background mode.
• run: Start a daemon in foreground mode (so that an exiting daemon can immediately be recognized by the init system).
• stop: Stop a daemon running in background mode.
• stop-running: Stop a daemon running in foreground mode. This should only be needed if the daemon doesn’t shut down nicely on a SIGTERM but instead needs some sort of message/semaphore to do so.
• status: Return current running status.
• reload: Reload configs, reopen logs (if supported by daemon).

Additionally, some control information would be needed including, but probably not limited to:

• Dependencies: Should allow for alternatives, lists daemons which need to be running before this daemon is started.
• Provides: Similar to package-provides, this should be used to allow things like replacing inetd with xinetd or something else providing similar functionality.
• Short Description: For init systems that are capable of showing a short description next to the name of configured daemons
• Run-Sets: Not strictly like sysvinit-runlevels, but similar, should allow the admin/user to configure different sets of daemons to start, selecting at boot by a kernel cmdline-parameter or at runtime through some command (”init 1″) which set should be running.

The script snippets listed above should allow basically any operation on a daemon, including the force-reload and try-restart commands not yet implemented by all current sysv-initscripts.

I just rewrote my aggregator (schcyrssmerge2) which still works roughly on this algebra from 2004 (still the basic set theory rule of blog aggregate generation: aggregate_i+1 = (blogs/aggregate_i) u (aggregate_inblogs)) but now calculates agg1 a different way. First, there's agg0 - the items from the input RSS file - and I store only the post ids of agg0 as agg0i for speed. Previously, I was reading in all the blogs and doing the above set theory calculation in one go. With a couple of ill-behaved blogs (full-content feeds of a few hundred kb), mzscheme's memory usage was just getting silly, topping out at hundreds of megabytes to merge 4Mb of feeds into a 3Mb aggregate. I think some of the problems are PLT's - reading a 3Mb RSS file in with

"(define a (call-with-input-file "friends.rss" read-xml))"

seems to result in mzscheme (v352) holding 100Mb. What's with that? Anyway, now I've written a newitems function which extracts new items not listed in agg0i and old items which still appear in a blog and adds them to lists. Then, there's a function which tail-recurses over all the blog filenames, parsing the RSS, calling newitems on each one and building the lists of old and new items. Finally, agg1 is the new items appended to the old items. That is only calculated as it gets written out as a RSS file. I'll give it a few days of testing, then I'll publish a tarball. I use this to build my all.rss feed from my various source feeds, in case you didn't realise, as well as to run some planets for public and private use. Anyone got a suggestion how best to reduce the read-xml memory consumption? Upgrade? Switch dialect? I'd like some XML read/write functions and the SRFI list functions.

In response to Stefano's reply to my post about Coq:

toy-ness

Usually such pieces of software are more seen as academic toys, seeing it promoted as a geeky software is a joy to me.

I really hate to detract from your happiness -- but I definitely had the word "math" implicitly stuck before "geek toy". :) Most coders that I've met are, at best, indifferent to mathematics, outright hostile at worst, so I doubt that coq will get far outside of small niches where the benefits are irresistable (security software, maybe? -- and of course pure math). I read up on this stuff despite it being utterly useless for anything I'm likely to be paid to do (unless I return to academia) because I'm a hopeless weenie loser who likes math. :P

programming-language-ness

Coq is not really a programming language (even if the CoqArt book says so, to attract the developer community), it is mainly a proof assistant; i.e. a software in which you give definitions, state theorems and prove them.

Well, this all depends on how you define programming language, which is one of those never-ending definitional questions. But using my general conception of what a programming language is, I actually have noticed two programming languages inside Coq.

1. The more interesting one (IMO) is the proof language they call "Gallina" and the "Vernacular", in which you construct proofs by reducing your goal to a tautology. The goal here seems to be to compute a well-typed term in Coq's core calculus, and you generally construct programs interactively: Coq shows you the current program state (expressed as one or more goals and a set of local hypotheses) and you manipulate it by (e.g.) applying lemmas to yield new known facts.
   
   I think this language is fairly weak viewed as a programming language. It's completely unreadable and the proofs generated don't correspond to the reasoning process one normally follows when working out a proof. I am less annoyed by it now that I've learned how to do forward reasoning :), but it seems like backwards reasoning is really pushed, and that just seems .. well .. backwards to me. There's also a strong tendency to use implicitly generated names for things and to just "know" what the state of the system after issuing a command will be; as far as I can tell, the only way to understand a Coq proof is to step through it in coqide.
   
   While it's nice to be able to "trust the kernel", it would also be nice to be able to "read a proof" directly. Among other things, their inscrutability makes me think that Coq proofs would be annoying to maintain over time. This is not necessary for stuff that's part of Principia Mathematica, but seems likely if you're developing a new theory -- in particular, I would shudder (theatrically) at the mere thought of including embedded Coq proofs in my programs.
   
   Some additional brief comments on this were written by Nick Benton and put on the Web under the title of Machine Obstructed Proof.
   
   
2. The other sense is that Coq directly embeds a strongly normalizing variant of System F (I think? -- at least the simply typed LC with some type parametricity) with primitive recursion over inductive structures. This isn't Turing-complete (of course!), but I would certainly call it a real programming language.
   
   As you note, though, this is less interesting unless you want to prove facts about your programs.
   

OCaml-ness

Regarding the synergy of Coq with OCaml: yes, you can prove in Coq the correctness of OCaml program, but actually not more than you can do with programs written in other programming languages

I was specifically thinking about Concoqtion; I have no idea how other projects compare, and I apologize if I incorrectly summarized it!

Perhaps the most accessible documentation on what PIC code is and how an ELF dynamic linker works is John Levine s Linkers and Loaders (and it has amazing sketches, too!). The Gentoo documentation also has an Introduction to Position Independent Code. I d like to give a few hints on how to fix the shlib-with-non-pic-code lintian error caused by inline assembly on the i386 and amd64 platforms, as well as build errors that may occur due to inline assembly being used. I m not going to cover the trivial all objects were not built using gcc s -fPIC flag problem. It usually requires a fix to the build system, not to the code. gcc can t find a register (i386) PIC on i386 uses a register to store the GOT (global offset table) address. This register is usually %ebx, making it unavailable for use by inline assembly (and also restricting the compiler's register usage when compiling C or C++ code). So the following perfectly valid code will not build with the -fPIC flag:

 void cpuid(uint32_t op, uint32_t reg[4])
  
    asm volatile("cpuid"
                 : "=a"(reg[0]), "=b"(reg[1]), "=c"(reg[2]), "=d"(reg[3])
                 : "a"(op)
                 : "cc");
  

Using -fPIC, gcc will say something around the lines of error: can't find a register in class BREG while reloading asm . Several things need to be done to fix this:

• use a register other than %ebx
• save %ebx if it risks being clobbered by the assembly code, and don't tell gcc about %ebx at all (it doesn't need to know anyway)
• if we saved %ebx by pushing it on the stack, make sure the inline assembly code takes the new stack offset into account

And here is the PIC-compliant version:

 void cpuid(uint32_t op, uint32_t reg[4])
  
    asm volatile("pushl %%ebx      \n\t" /* save %ebx */
                 "cpuid            \n\t"
                 "movl %%ebx, %1   \n\t" /* save what cpuid just put in %ebx */
                 "popl %%ebx       \n\t" /* restore the old %ebx */
                 : "=a"(reg[0]), "=r"(reg[1]), "=c"(reg[2]), "=d"(reg[3])
                 : "a"(op)
                 : "cc");
  

using variables from assembly code (i386) Directly using variables from inline assembly always creates non-PIC code even if -fPIC is being used. There are at least three strategies to consider when trying to fix this. Here is an example of PIC-incompatible code:

 extern uint32_t sym1, sym2, sym3;
 
 void store_in_symbols(uint32_t x)
  
     asm volatile("movl %0, sym1   \n\t"
                  /* ... */
                  "movl %0, sym2   \n\t"
                  /* ... */
                  "movl %0, sym3   \n\t"
                  : : "r"(x));
  

The first strategy is to pass the variable through the usual gcc inline assembly syntax. This is not always possible because there might be a shortage of registers, but here is what it looks like:

 extern uint32_t sym1, sym2, sym3;
 
 void store_in_symbols(uint32_t x)
  
     asm volatile("movl %3, %0   \n\t"
                  /* ... */
                  "movl %3, %1   \n\t"
                  /* ... */
                  "movl %3, %2   \n\t"
                  : "=r"(sym1), "=r"(sym2), "=r"(sym3) : "r"(x));
  

If there are too many variables and not enough registers, one can use the second strategy: put all required variables (or addresses) in a table, thus requiring only one extra register:

 extern uint32_t sym1, sym2, sym3;
 
 void store_in_symbols(uint32_t x)
  
     uint32_t tab[3];
     asm volatile("movl %1, (%0)   \n\t"
                  /* ... */
                  "movl %1, 4(%0)  \n\t"
                  /* ... */
                  "movl %1, 8(%0)  \n\t"
                  : : "r"(tab), "r"(x));
     sym1 = tab[0];
     sym2 = tab[1];
     sym3 = tab[2];
  

If the second method happens to be unsuitable because variable types differ too much, there is a third, more complicated strategy: retrieve the GOT address using some linker-specific magic, then use @GOT addressing to access variables:

 extern uint32_t sym1, sym2, sym3;
 
 void store_in_symbols(uint32_t x)
  
     uint32_t got;
     asm volatile("call get_got                               \n\t"
                  "get_got:                                   \n\t"
                  "popl %0                                    \n\t"
                  "addl $_GLOBAL_OFFSET_TABLE_ - get_got, %0  \n\t"
                  : "=r"(got));
     /* ... */
     asm volatile("pushl %%ebx                \n\t" /* save ebx */
                  "movl sym1@GOT(%0), %%ebx   \n\t" /* retrieve sym1 address */
                  "movl %1,( %%ebx)           \n\t"
                  /* ... */
                  "movl sym2@GOT(%0), %%ebx   \n\t" /* retrieve sym2 address */
                  "movl %1, (%%ebx)           \n\t"
                  /* ... */
                  "movl sym3@GOT(%0), %%ebx   \n\t" /* retrieve sym3 address */
                  "movl %1, (%%ebx)           \n\t"
                  "popl %%ebx                 \n\t" /* restore ebx */
                  : : "r"(got), "r"(x));
  

It is of course possible to merge the two above asm statements into one. However the performance implications of the GOT address retrieval should not be overlooked, especially if the assembly code takes place inside a critical loop, which means the loop might have to be unrolled and the GOT address retrieval done outside the loop. Note also that _GLOBAL_OFFSET_TABLE_ is ELF-specific. On BSD and a.out systems you should use __GLOBAL_OFFSET_TABLE_. I am not aware of a way to do the same on Darwin / Mac OS X, which is a shame because it is the only widespread i386 platform where shared objects cannot be non-PIC. gcc silently generates non-PIC code (amd64) The amd64 architecture has the same problem as i386 when trying to access variables from assembly code. However, the solution is a lot easier: the GOT address can always be retrieved from the special %rip register. This is called rip-relative addressing:

 extern uint32_t sym1, sym2, sym3;
 
 void store_in_symbols(uint32_t x)
  
     asm volatile("pushl %%rax                       \n\t" /* save rax */
                  "movq sym1@GOTPCREL(%rip), %%rax   \n\t" /* retrieve sym1 address */
                  "movl %0,(%%rax)                   \n\t"
                  /* ... */
                  "movq sym2@GOTPCREL(%rip), %%rax   \n\t" /* retrieve sym2 address */
                  "movl %0,(%%rax)                   \n\t"
                  /* ... */
                  "movq sym3@GOTPCREL(%rip), %%rax   \n\t" /* retrieve sym3 address */
                  "movl %0,(%%rax)                   \n\t"
                  "popl %%rax                        \n\t" /* restore rax */
                  : : "r"(x));
  

amd64 also has more registers than i386, which makes the first discussed technique less prone to register shortage. Why bother? Good question. Currently many shared libraries simply disable entire chunks of inline assembly on the amd64 architecture. Whether these routines bring any performance benefit compared to the C alternative is uncertain, but without being able to build the code there is no way to tell. Even fewer developers care about i386, because this architecture can handle non-PIC shared objects quite fine (but then they are no longer really shared). I am in favour of making all shared objects PIC-friendly (ie. not necessarily build them as PIC if there is a valid reason not to, but at least don t make it too difficult to build PIC versions) because that makes it easier to port the code to Darwin. I also believe the memory gain and especially the cache gain when using real shared libraries is underestimated.

Today's post (previous parts) tries to show what benefits other init systems can offer.I'm picking 'minit' as an example; not because it's the ultimate solution or anything, but because I happen to know it quite well. I used to be the package maintainer for some time, and I've also used it on an embedded system (where it was a perfect match). However, it's not very distribution-friendly; it's great when you can invest some time into configuring it (such as when building an embedded system) but not when it should ideally work automatically out of the box. The Debian package is still there, but it's orphaned. Still it's well written, clean code; there shouldn't be any bigger bugs in there.The first thing to notice is that minit is tiny. It's statically linked, but only about 6.5 kb. Regular init is 86 kb plus some libraries (despite actually not doing much during init!). This makes it very well-suited for embedding.But now for the benefits of minit over regular init:1. Service monitoring. This is the biggest point: minit will keep an eye on the services and restart them when needed. (Another thing very useful in embedded situations, btw.)Minit doesn't use the start/stop paradigm of sysv/shell-style-init. Instead it has a "run" script for every service. If a service is flagged as 'respawn', it will automatically be restarted when the run script quits; therefore the applications should not background themselves. This offers some benefits, for example the init process will be notified by the kernel of a termination, and can immediately respawn the service; in contrast to other monitoring systems suchas monit it doesn't rely on 'busy polling'.2. Dependency handling. Minit handles startup dependencies. If a service lists another service as a dependency, it will bring that service up first.3. Parallel startup. Unless an ordering is given via dependencies, minit will start the services in parallel.4. Control apps. In minit, you don't start a service by running some random shell script. Instead you call "msvc -u servicename", which will signal minit to bring up the service (and it's dependencies). This would (there currently is no policy for minit) be especially useful in SELinux, but it also allows to e.g. give a user the permission of running "sudo msvc -u" without introducing a huge security hole, since this cannot be used to execute arbitrary scripts. "msvc" can also be used to check if a service is up. (Another benefit for embedded systems, since that removes the need for pid files, which then need a ram (writeable) filesystem)I'll also give an example for a minit service:A service is represented by a directory in /etc/minit which can contain various files. I'm going to look at getty/1 example; which will bring up a console login on the first terminal. There are three files:

• a symlink named 'run' to /sbin/getty, which is the daemon we want to bring up
• a file named 'params' containing the parameters for the getty application, one per line
• an empty file named 'respawn', since we want the login screen to reappear

If the service would need it, there could also be a file called "depends" listing service dependencies.This allows minit to start the getty with no overhead. There is no shell script being executed or anything (note: sysvinit also does this for gettys, but it's not being used much for other services).Of course minit isn't perfect. I've already said that it isn't particularly distribution friendly. This is because it uses a lot of magic files (e.g. an empty file named 'respawn' to flag a service as needing to be respawned), and it's configuration files aren't easy to handle automatically. The often frowned-upon 'symlink farms' of sysv-shell-style init are much better here.Another thing that is missing from minit is an automatic rollback of dependencies. If we told minit to bring up service 'a', which lists dependencies 'b' and 'c', minit will bring these three up. If we tell it to shutdown 'a', it will however not shutdown 'b' or 'c'. And unfortunately, even if we shutdown 'b', it won't shutdown 'a'. If you want a more complex dependency handling (or things such as runlevels), you need to do this externally. (An example is my 'enitdir' app, it adds runlevel support to minit, and allows switching of runlevels by replacing a symlink; Services are added and removed from runlevels by adding a symlink to the service into the runlevels directory. By using dnotify, this also takes virtually no CPU time.)Minit also lacks a differentiation between a 'started' and a 'running' service. It has support for 'sync' services, such as e.g. mounting a filesystem. It will wait for this operation to complete before it starts services depending on it. However, there are services that will continue to run, while not being instantly available when the exec() was called. Sometimes it would be nice if a service could go into a 'started' state, then do some cleanup work, and then when actually running the real service signal minit that it's now 'running'.As a last malus, minit won't print progress messages to the console. A minit boot is pretty much silent, directing it's output to some logging processes. There are good reasons for this (since services are started in parallel!), but it makes debugging harder, and users might be wondering what their system is actually doing (though some users prefer to see a colored 'busy' bar)...I guess this concludes this series of blog posts. Maybe someone can follow up with some details on upstart (which seems to be the most promising init replacement)? I hope to have given an impression on where there is room for improvement in our current init to one or the other. Now we just need some people to actually work on a better support for other inits in Debian...

hair transplant commented:

"to blog in different languages, we need and on line translator, I think"

in reply to my old question about multilingual blogging. Like many multilanguage speakers I know, I don't hold out much hope of machine translation becoming useful any time soon. Systems like systrans can be useful for getting the basics of a message when you're totally stuck, but babelfishisms are a well-known danger (or comedy source, depending on what you're doing with it). So we're left with the question of how to blog in multiple languages without annoying the hell out of everyone. Do aggregators support content-negotiation? Should we use bertilow-style consecutive translation? Or simply run multiple blogs? I'm still only blogging in English. I probably should experiment. What's the state of this art now?

The cardinal reason for the switch is collaboration: git scales. During my bzr usage time of the last year i observed the following shortcomings of bzr:

Speed - This improved a bit lately but not enough for my q&d usage.

Documentation - Beside the obvious "svn-style" commands the usage of the other commands isn't obvious and the man pages lack examples. git is not perfect in that regard, but you quickly find typical usage pattern on wiki tutorials or on many google hits.

Pager - On big diff output bzr doesn't pipe that to pager^Wless. I consider that as a big usability bug.

Branches - I recently learned that a bzr repository corresponds to a git repo with just one branch. This seems strangely complicated. I really love to have different branches ready to checkout in my pwd . This short-coming blocks cooperation as i'm not confident enough to fetch random changes from bug reporters in my own branch.

Minor Nitpicks:
* Push command - Defaults on the non-obvious old sftp protocol. Worked with rsync on some very ancient version.
* Repo format - Changes on every major upgrade.
* Patchlevel - Probably for svn compat reason the chosen default patchlevel is -p0 not -p1.

Latest initramfs-tools repo is available at:
git clone git://git.debian.org/git/kernel/initramfs-tools.git Thanks to tailor it is now possible to move from different version control systems. For git as target repository it is recommended to use latest upstream either directly via darcs or at least install the unstable version. The etch version throws strange python backtraces.

I've changed the urls of everything in my website, including the blog. Feeds are limited to displaying new items to avoid flooding aggregators. Old urls will continue to work thanks to rewriting, and the rss feed url didn't change at all for the main blog (though some of the minor feeds urls did change and will redirect). Nothing you haven't seen a million times before, move along..

If you use grep-dctrl, sort-dctrl or tbl-dctrl, I’d like to see your use cases! I’m building a system test automaton for dctrl-tools, and I’d like to include as much real-world examples as possible so that any regressions get found before uploading. You can send example command lines (and preferably also a test file, unless it’s one of the standard Debian files like Packages) to me by email at <ajk@debian.org>, as a wishlist bug report against dctrl-tools or by a Darcs patch against the experimental repository (see instructions).

I'm really excited this morning. Wikitravel has been nominated for a Webby Award in the Travel category. It's amazing to get this kind of recognition for the work that our community has done over the last four years. I think that Wikitravel has a good shot at winning the award. I also hope that the Open Source and Open Content community, as well as wiki enthusiasts, will turn out to vote for us in the People's Voice Awards (open to voting by the entire Web community). tags: wikitravel wiki webbyawards webby awards webbies nominee travel

AboutUs.org has OpenID Looks like Portland (Oregon)-area startup AboutUs has implemented the OpenID extension for MediaWiki. Meaning that anyone can log into AboutUs with an OpenID! This moves AboutUs into a nice single-signon network with Wikitravel, Wikevent, and wikiHow, as well as a number of other OpenID-enabled sites. Great for them. tags: aboutus openid mediawiki

One version of pilot-qof went into Etch (0.1.1-3) with 0.1.2-1 uploaded to unstable after the freeze and just migrated into Lenny. A new release had sat in the NEW queue for a month due to the final stage of the Etch freeze. Finally, it is moving into unstable. 0.1.3-1 separates the pilot-qof binary from the perl, XSL and documentation making four packages in all, three under the new title: datafreedom. See data-freedom.org.

Hopefully, splitting existing Debian packages will not require a spell in the NEW queue in future - or at least a reduced queue, after all these packages have been checked once already. (Hence Sam was my first choice as DPL - congratulations again, Sam.)

If you previously used the XSL stylesheets from pilot-qof, you will now need the datafreedom-qsfxsl package. Note that the location of the XML catalog (and the XSL stylesheets) has changed too. See NEWS.Debian in the pilot-qof package. (zless /usr/share/doc/pilot-qof/NEWS.Debian.gz).

The intention is that the datafreedom packages will now mature to support a wider range of small utilities like pilot-qof - there is already support for calcurse, contacts, dates, gpe-expenses, gpe-calendar and a few others - justifying the package name change.

The XSL is (predictably) limited to converting from XML so if XSL is used to convert to a non-XML format, you'll also need the datafreedom-perl package to support conversion of non-XML data back into XML for inclusion into pilot-qof or conversion to other formats via XSL. The perl scripts also include support for aggregating data sources to create new data sets, e.g. combining addressbook, expenses and contacts data sources into a simple invoice. If, like me, you enter new data into your Palm whilst travelling around on business, you can HotSync the Palm and generate all your invoices in one script!

The perl package includes a number of new scripts and brings in a few new perl module packages and this is where the majority of updates are likely to occur in future.

Enjoy!