Both my daughters have been playing soccer for a while now. And for another little while, I had been promising that if they ever scored three goals in a game, I'd shave my head. As the attentive reader may have guess by now, that day finally came. This weekend saw a suburban tournament in nearby Oak Brook, and lo and behold Anna scored three goals in the first game! So home we went, out came the tool and she rather professionally separated me from my hair. So today on day two of the new look, a friend took this picture of me (scaled down from 2.4mb to around 80kb) at the same tournament: They actually played just about the best soccer I have seen them play, won their group (with three shutouts!) and lost a hard-fought and well-played final 2:4. And today the weather even cooperated as one can see from the photo. Nice weekend, all told. And yes, the head feels kinda nice ;-)

JROBI, a chess blogger, on energy policy:

A large study in Europe concluded that it takes more gas and oil to produce a bottle of bio-fuel than it does to produce a bottle of gas. What does this mean? It means that Bio-Fuel is more damaging to the environment in the long run, and on top of that it is driving up the cost of basic food supplies. Millions and millions around the world in a number of countries are unable to afford the rising food costs for basic staples like Corn, and for what? If Bio-Fuel is not better for the environment, why are politicians and environmentalists getting behind this growing industry? I think it’s because it seems to be the “trendy” thing to do, and we all know what happens when the media promotes a new trend. We get tons of media coverage telling us why it’s a good thing, and hardly any coverage of the negative impacts. Already people from the Bio-Fuel industry are getting on television shouting out that there are many factors contributing to rising food prices, trying to deflect the fact that their destruction of food to fuel vehicles is the main culprit.

Actually, I suspect the emphasis on biofuels in the USA and Europe has to do with the fact that it’s the only alternative to fossil-based motor fuels proven to be sustainable and scalable:

The success of FFFVs, together with the mandatory use of E25 blend of gasoline throughout the country, allowed Brazil to get more than 40% of its automobile fuels from sugar cane-based ethanol in 2007.

I see no link to the European study in question, but previous studies have suffered from various faults; for example, the assumption that trucks transporting fuel cannot themselves shift to biofuels. I’m sure better analysis of the study is on its way. But that’s not the most interesting thing, to me. More interesting: my general impression that a lot of the climate-change hysteria is just that. If we hear what science seems to be telling us about the environment, and we think that something needs to be done, then we should do things that will actually work. One thing that really works is conservation: use less of the bad stuff we’re using. But we’ve done quite a bit on that front, only to hear that much, much more is required to make a difference. I’m not sure there’s much, much more benefit for us to realize in conservation, at least in the short term. So, to make a real difference, we have to make more radical changes. Can we change our motor fuel? Sure; starting with something that pollutes less, and that even absorbs some of that same pollutant in its production, sounds like a winner. JROBI, again:

It makes no sense whatsoever to create Bio-Fuel when there are much better options on the table - for instance Hydrogen vehicles. When was the last time you heard someone on the news talk about Hydrogen initiatives?

I hear it every so often. But most talk, today, focuses on the very real problems with hydrogen as a motor fuel. There are many; just look at the discussions of hydrogen fuel tank technology for a sample. But one of the biggest problems is that of developing an infrastructure for delivering fuel to the customer. No one talks about the problems of setting up an ethanol infrastructure. We already have it. Brazil has demonstrated that the current gasoline infrastructure can easily be adapted to deliver ethanol instead, and that there is a viable migration plan for gradually moving people off fossil fuels. Now, this isn’t to say that the world of ethanol is hunky-dory. It’s arguable that, while ethanol may be sustainable, the corn-based system the USA has adopted isn’t. Some people are talking about sensible tweaks that may solve the food problems while continuing to support biofuels–removing our silly tariff on Brazilian ethanol, for example, or developing alternative feedstocks for ethanol production. The problem is that hysteria seems to be breeding hysteria. Global warming is so severe, we are told, that we need solutions, and we need them immediately. So we develop solutions we can use immediately. But no! These solutions cost; we need something else, and we need it immediately, and we need it cost-free. Practically, this kind of insistence on perfection–that we deploy solutions with no drawbacks, only benefits–has the effect of dampening our enthusiasm for environmental solutions. We tried, our leaders will tell us, but nothing was good enough, so we gave up. And so, rather than do something that helps, or even something that lays the foundation for helping, we continue our use of fossil fuels. Perhaps ethanol is the wrong solution. But if it is, we should resign ourselves to the inevitability of the future, as foretold by science, or fervently hope that the global warming deniers are right, because other solutions will arrive too late to do much good.

Lets go on your nerves again with a ftpmaster post about work I’ve done in the recent past.

• The already mentioned move of britney. And it seems that this did help, so something that should have been done way earlier already… Oh well.
• Enabled a few triggers for the qa user on merkel. While I already mentioned them in my first post, there is a third one now. The qa user now gets notified at the beginning of the dinstall run, so the pages can show why packages might not be visible in the archive and in incoming right now.
• Split up the insanely huge mbox files we had as the ftpmaster mail archive. They got that big because the monthly cronjob rotating them got disabled when the ftp-master archive went away from raff, and noone bothered to reenable that job. Done now, so we are back to a manageable size. (Even if 90% of the content is spam, no matter how much greylisting and blacklists are activated :( ).
• Just committed a largish change thats nothing more than a cleanup. Someone in the past did love to not care about tab versus spaces for the indentation, so dak code ended up with a mess of tab/space mix for indentation. Which is not nice, one style is way better. Fixed that with the help of reindent.py, which one can find in Pythons examples.
• Got the code on security.debian.org to use the latest dak revision from ftpmaster. It was a few commits behind, but worse - it did have one dak tool changed in a way that it only worked on security, no longer on ftpmaster. Also some files that clearly have been copied by scp -r or rsync, but not using the VCS that should be used for it. Args. Brrrr. The “fix” was a simple check if that value the code wants to use from the config file actually exists, instead of blindly assuming it will always be there.
• Generally looked around on security.debian.org, to get more familiar with the setup there, so I can work on a request I got from some members of the security teams: Run a queue daemon that manages the upload of the packages from the security archive to the normal archive (so they end up in proposed-updates). Currently it is done with some lftp calls within the script that does the security install, and that seems to break every now and then. Which means security uploads to proposed-updates randomly go MIA. Using a queue daemon and then only locally copy files into its dir should make that way more reliable.
• Used a tool from Dato and enabled bzr commit mails for the ftpmaster dak tree, they are now going to the dak list.
• Modified the cronjobs that are expected to have output to log that into files in a new logdir. Also modified the cron.daily AKA dinstall cronjob to be more verbose about the actions done. The directory is synced to merkel too (log/cron in the ftp.debian.org tree), so every DD who wants to can look at them now.
• I also merged a small patch from Thomas, fixing/unifying some wording in reject messages.

Comments: 3

... does anyone know what would be the expected behaviour if I returned a "multipart/related" document from an HTTP request?

Hey'all! Another update of the moment:

Physio is going amazingly well. I feel like my neck / shoulder mobility is close to what it was pre-surgery. My strength is still lagging behind, and I'm doing hefty 3 pound bicep curls! Woohoo! Not so much for my arms, but for my back, which still has to support my arms doing this. I'm also doing 45 pound dips, and some exercises with a yellow theraband. Walking is really tiring me out at all anymore, no matter what distance I walk. As I'm reducing my pain meds, I am starting to get sore, though. Hopefully in the next week that will stabilise. Next Monday I get an ultrasound of some internal organs (I can't remember which, sorry) that will see if I have a non-drug related cause for the ongoing nausea. But so far a pill every day or two seems to be keeping it at bay.

I've posted two more videos of Leif up on YouTube. I have a bunch to sift through, but hopefully there's at least one more good one in the set. I haven't uploaded the pictures yet because I hate f-spot so much. (Ask my sometime why I think it was called f-spot)

I got a fun chain letter from someone asking everyone to boycott Esso and Shell for the rest of the year to force gas prices to come back down the sub dollar-per-litre. This was my reply:

I like high gas prices, though. I want the prices to go much higher.

Every time the prices go up, there are more cyclists on the road, and
the air gets a bit clearer.

If it were $10/litre, then almost noone would drive. We'd all be on
bicycles. Then it we be so much safer to be on a bike.

I can't wait for the day.

What do y'all think?

And on the hacking front, I actually dusted off CVS and fixed some autotools bits in GNU Sysutils so that David could fix a compile bug. I don't know if this round of activity will spur us into doing more. But it's possible. So maybe in another.. Oh, I dunno. decade or so we'll have a full GNU replacement for util-linux that's portable and everything. =)

I might get off my ass and hack on my reimplementation of Galactic Bloodshed through the month of May. I miss that game in a strange pathetic way.

Heading into my last month of off-work recovery time, I'm surprised at how good I feel. I haven't replied to alot of emails I've gotten from folks, even from before the surgery. I'm starting to make headway on the emails, though. Thanks for your patience!

Hugs all!

When I got in to my hotel room last night, it was brightly illuminated and the TV was on with music and a message to greet me. The first thing I did was turn it off, along with most of the lights. Later, I asked at the reception when they turn on the lights. I was expecting them to have a central switch for each room that the receptionist could flip when the guest finished checking in and has started to make way towards the room. Not so. The guy behind the desk told me that housekeeping sets it all up. So my lights have been on and the TV blasting for the entire afternoon and evening. That did not make me feel at home. It made me cross. NP: The Pineapple Thief: 8 Days Later

As mentioned a few weeks ago, I had submitted two entries as possible topics for this year's version of the Google Summer of Code. And lo and behold, the response was very, very good and both applications were slotted! For the topic 'create a PostgreSQL package for R that uses the standard DBI interface', a number of interested students contacted me, and a total of three applications were submitted. And while the R Foundation was only able to allocate four topics among a number of really good applications, Sameer Prayaga was our pick for this topic. It would be nice to fill this gap among the existing database connection methods for R, and I feel that Sameer can pull this off. For the second topic of 'create a cran2deb tool for converting CRAN sources into Debian package' which I had submitted within Debian, Charles Blundell wrote an excellent application. In a way, this topic is a '2.0' version of our previous attempts of a 'top-down' set of tools in the pkg-bioc project on Alioth. This time, we will try something smaller, maybe more modular and lighter and see how far we get there if we try it 'bottom-up'. And as we are currently in the community bonding phase, say Hi to Sameer or Charles when you come across them these days. Lastly, I'd like to thank everybody who submitted an entry at Debian or R, or who contacted me about one of the topics I posted. The respone was very humbling, many of you were imminently qualified and seemingly very motivated -- but even Google's pockets can only pay for a finite number of projects. Sorry if yours did not get picked.

It seems existing versions of gcc don't warn when you use an assignment as a truth value as the first operand to the conditional operator. For example: Presumably you meant ==, but unfortunately that does not warn with gcc (maybe it will one day). We can use the tree dumping mechanisms (previously mentioned here) to confirm our problem. For the following function gcc creates: As you can see, this isn't the result we wanted. This can be quite nasty if you use the conditional operator in a hidden fashion; for example behind an assert. A common idiom is: You'll currently get no warning you've slipped up and used the assert wrong. Moral of the story: a quick audit of your code might turn up some surprising uses! However, in this case, the Intel compiler picks this one up: Although it's often not trivial to move to another compiler, as a rule I would recommend trying alternative compilers for your code as it's a great sanity check.

For a university seminar I m currently writing a GUI program to view fractals based on simple iterated function systems (only similaritudes allowed). It supports three different drawing algorithms and you can edit the IFS by dragging squares around on the screen. But this post is not about this program (I might present it later), but how Haskell allowed me to solve a problem very nicely:The first instances of the code had a problem that a lot of GUI programmers know: The drawing of the fractal took quite some time, and during that time the gtk main loop is blocked and the program becomes unresponsive. At first I avoided this problem by manually splitting the drawing function (e.g. by repeatedly increasing the resolution) and kept re-drawing it at higher resolutions in an idle handler, so at least I could interact whenever one resolution has finished drawing. It worked somewhat but it was not easily done for the other, not pixel based, algorithms and it was ugly.So I wanted a way to (a) pause the drawing at any convenient point, to resume it later, and (b) safely abort the drawing if what I m drawing has changed and I need to restart.A common solution to this would be to do the drawing in a separate thread, so (a) is actually not needed, but I did not know if I can safely do (b), and I have heard that threads cause problems with gtk.So I tried to dig deeper for the hidden treasures of advanced haskell programming: I need a monad transformer! I expected the infamous ContT monad transformer to help, but I couldn t figure out how, and I started to create my own monad transformer, called CoroutineT.I tried to figure out what an action of type (CoroutineT IO a) should do, and I came up with this type signature:

pausingAction :: IO (Either (CoroutineT IO a) a)

which means that after the pausingAction is done, it is Either paused (and I get back another pausingAction to run when I want it), or it is done (and I get the result). Note that I m writing IO here, but it can be any monad.The definition of the datatype and the monad instance came mostly from trying to make this type work (yay to haskell s type system, less thinking required), and looks like this:

data CoroutineT m a = CoroutineT  unCoroutineT :: (m (Either (CoroutineT m a) a))  

instance (Monad m) => Monad (CoroutineT m) where
    return v = CoroutineT (return (Right v))
    a >>= b = CoroutineT $ do
          r <- unCoroutineT a
          case r of
            Left  paused   -> return $ Left (paused >>= b)
            Right unpaused -> unCoroutineT (b unpaused)

This translates to english like this: A call to return is not paused. When an action is already paused, further actions should be run after the paused action is resumed. When an action is not paused, further actions can happen now.Like every well behaving monad transfomer, I also need a "runCoroutineT" function to start the coroutine. I probably could have used unCoroutineT directly, but for my use case (GUI drawing) I did not need a return value, so this function is more handy:

runCoroutineT :: Monad m => CoroutineT m () -> m (Maybe (CoroutineT m ()))
runCoroutineT a = either (Just) (const Nothing)  liftM  unCoroutineT a

Nothing surprising here, basically just turning the Either into a Maybe. So it becomes clear how to do (b): We can just throw away the resume action returned by runCoroutineT (if any). The more interesting thing is how we do (a): We need a pause action of type (Coroutine m ()). But how should it work? I did not really try to understand why it works, but by looking at the types, I came up with this:

pause :: Monad m => CoroutineT m ()
pause = CoroutineT (return (Left (CoroutineT (return (Right ())))))

Yes, it sounds like some dance step instructions (read the second line out aloud!), but it works somehow.So here is some example code: I have a pausable IO action that counts from one to ten, pausing after each number. I also have function that resumes an pausable action up to n times:

example n = keepGoingFor n $ do
        liftIO $ putStrLn "This is the coroutine"
        forM_ [1..10] $ \i -> do
            liftIO $ putStrLn $ "Counting to "++ show i ++" while you keep calling it"
            pause
  where --keepGoing :: Monad m => CoroutineT m () -> m ()
      keepGoingFor 0 _   = putStrLn "Here I just abort the run"
      keepGoingFor n cor = do
        resume <- runCoroutineT cor
        case resume of
            Just runAgain -> keepGoingFor (n-1) runAgain
            Nothing       -> putStrLn "Finally stopped"

And here is the output of two different runs:

*CouroutineT> example 5
This is the coroutine
Counting to 1 while you keep calling it
Counting to 2 while you keep calling it
Counting to 3 while you keep calling it
Counting to 4 while you keep calling it
Counting to 5 while you keep calling it
Here I just abort the run
*CouroutineT> example 14
This is the coroutine
Counting to 1 while you keep calling it
Counting to 2 while you keep calling it
Counting to 3 while you keep calling it
Counting to 4 while you keep calling it
Counting to 5 while you keep calling it
Counting to 6 while you keep calling it
Counting to 7 while you keep calling it
Counting to 8 while you keep calling it
Counting to 9 while you keep calling it
Counting to 10 while you keep calling it
Finally stopped

So it does really works fine, and it proved very useful in my GUI drawing problem. For that, I created this nice control structure which works like mapM_, but calls pause every n iterations, and therefore hides the pausing stuff almost completely:

pausingForM_ :: Monad m => Int -> [a] -> (a -> CoroutineT m ()) -> CoroutineT m ()
pausingForM_ period list action = pausing' 0 list
  where pausing' _ []     = return ()
        pausing' n (x:xs) = do action x
                       if n==period then pause >> pausing' 0     xs
                                    else          pausing' (n+1) xs

I have put the complete module (including instances omitted here) in the darcs repository that might later also contain the fractal drawing program.

I've implemented last night's design, in debhelper 7.0.0. Currently fully supported are packages that use the autotools, or perl's MakeMaker, or python setup.py, or a simple Makefile. The other classes of packages will be added as bugs are filed about them. I've uploaded a few of my own packages converted to use debhelper 7 with minimised rules files, so there are a range of examples in incoming:

• mr uses a very simple makefile, and a 3 line debian/rules
• aalib uses automake and autoconf, and is a library package of average complexity. The rules file is minimised where possible using dh, but still does complex things in its build and install targets. It's also a good example of how to set up a -dbg package when using dh, and of how to set up /usr/share/doc symlinks between packages when using dh.
• libaudio-mpd-perl is an example of a perl module package using dh. It shows how to override dh default behavior in two ways: Skipping running dh_auto_test, and passing a changelog filename to dh_installchangelogs.

Early adopters, have at it!

There has been ongoing debate in the Debian community for a number of years about what standards of behavior should be expected. Matthew Garrett sets a new low by making a joke about Jesus being molested as a child [1]. While I believe that debate and discussion about religion is a good thing, such comments about someone who is widely regarded as a God (part of the Holy Trinity) seems to provide no value and just needlessly offends people. I used to be a Christian, and while I have great disagreements with most Christians about issues of religion I still believe that Jesus (as described in the bible) was a good person and deserves some respect. I don’t believe that blasphemy should be illegal, but some minimum standards should be observed when discussing religion. Next there is the issue of child molesting, most people agree that there’s nothing amusing in that - so I hope that nothing more needs to be said about violating babies. Finally there is the issue of rape in general often being treated as a joke in the computer industry (I am not sure how prevalent this is in the wider community). One example that is from a Wired article: “We were raped by Microsoft. Bill Gates did it personally. Introducing Gates to the president of a small company is like introducing Mike Tyson to a virgin” [2]. I admit that finding examples of this on the web is not easy, part of it is due to such slang use being more common in spoken communication than in written communication, another is the vast number of slang terms that are used. A Google search for “male rape” [3] turns up some informative articles. One FAQ suggests that 3% of men will be raped as an adult [4] - I expect that some guys will decide that it’s not so funny when they realise that it could happen to them. For people who’s knowledge of the English language is not as good as that of Matthew and I, here is the dictionary definition of the word “violated” [5].

• [1] http://www.advogato.org/person/mjg59/diary/114.html


• [2] http://www.wired.com/wired/archive/2.04/gates_pr.html


• [3] http://www.google.com/search?hl=en&c2coff=1&safe=off&q=male+rape&btnG=Search


• [4] http://www.aest.org.uk/survivors/male/myths_about_male_rape.htm


• [5] http://en.wiktionary.org/wiki/violated

Share This

anjuta is my favourite IDE for C work - the configurable symbol browser/highlighter, the ability to right click on a function name or typedef and go straight to that part of the relevant source file somewhere else in the project (or outside), the debug support, mostly sane syntax highlighting (i.e. when it works it is perfect, when it doesn't work it doesn't try and make things worse). geany is very competent too, although possibly better suited to small tasks. (The compile and build menu options in geany don't always use the existing Makefile.)

Recently anjuta has developed an annoying habit of complaining about filetypes it claims not to understand (e.g. configure.in ac, Makefile.am - quite important files in the general C workload), geany (my favourite text editor, especially for perl) does those well but doesn't seem to support highlighting ChangeLog or debian/changelog - a task left to gedit but I can't use gedit for source code because the highlighting and general support of normal source code is truly awful, especially for perl.

(I got comments last time I rated geany higher than gedit - it's a question of focus. gedit cannot do everything - at least not well - but it tries anyway and therefore fails badly. Geany has far, far superior source code highlighting and an incredibly useful symbol browser because it has a more "source code" focus. I'm not going to go scrabbling around in gedit to create a customised highlighting scheme and only have two or three colours for perl is completely unacceptable in gedit (and changing colour scheme as no effect), as is the lack of a symbol browser.) Besides, geany loads and runs much faster than gedit.

Compare for yourself: Geany:
geany.png>

or gedit:
gedit.png

(inlined images removed - originals behind the links are quite large)

I mean, WTF? No separate colour to distinguish interpreter keywords or function names from plain text strings in gedit?? Duh? Badly broken regexp highlighting that highlights only the $ at the end? The entire file is like that - there are three colours in all. 99.9% is puce, variable names are olive and characters like line endings or tab markers are blue. Yes, I can change those colours but what's the point of changing everything that is now in puce into blue when what I need is some parts to be blue, some parts to be light blue, some parts to be red and some parts to be yellow. The gedit perl highlighting has to be the worst I've seen in any editor. It's almost easier in monochrome. Perl is by far the worst but gedit C highlighting isn't much better.

Compare with geany where keywords are always identifiable, regexps are marked up clearly, no silly marking of line ending characters inside a string, sane marking up of pod content and HERE documents and variable highlighting - and it never gets confused.

Needless to say, vi (or more precisely vim) does all the highlighting with aplomb, even though asking for a symbol browser in vim is a bit excessive, even for me.

vi.png

Why can't I have a single GUI that does highlighting as well as vim ? Geany is the closest of the three - lacking only ChangeLog and debian/changelog (and debian/control) highlighting, anjuta is almost there (once that annoying bug is fixed - already filed by someone else) but gedit? What? Has groundhog day been changed to 1st April somewhere? They're having a laugh, must be.

simple case Imagine a very simple debian/rules, that should work for most packages w/o modification:

#!/usr/bin/make -f
build:
    dh build
clean:
    dh clean
binary-arch:
    dh binary-arch
binary-indep:
    dh binary-indep
binary: build binary-arch binary-indep

absurdly simple case

AKA debian/rules zen guru level.
#!/usr/bin/make -f
%:
    dh %@

the complex cases Often you'll want to handle configuring by hand. (And maybe, run automake too.)

build:
    dh build --before configure
    #aclocal
    #autoheader
    #automake --add-missing
    #autoconf
    ./configure --kitchen-sink=yes
    dh build --after configure

Here's how to skip two steps in a row (configure and build):

build:
    dh build --before configure
    ./mondoconfig
    make universe-explode-in-delight GROOVINESS=420
    dh build --after build

Another common case is wanting to run some code manually after a particular debhelper command is run.

binary: build
    dh binary --until dh_fixperms
    # dh_fixperms has run, now override it for one program
    chmod 4755 debian/foo/usr/bin/foo
    # and continue
    dh binary

Suppose you hate dh_strip, and just don't want it to run. (This might be a contrived example; if people frequently don't want to run a few commands in a set, it may be better to have -X exclude them.)

binary: build
    dh binary --before dh_strip
    dh binary --after dh_strip

Maybe you want to run a command out of order. This should be no problem:

binary: build
    dh binary --until dh_fixperms
    dh_install foo usr/bin
    dh binary --after dh_fixperms

A separate install target can be added if you like. And maybe you want to run make install by hand in it. (This is tricky because dh binary needs to avoid running dh_auto_install which was skipped over in the install target.)

install: build
    dh install --before install
    $(MAKE) install PREFIX=debian/tmp
    dh install --after install
binary: install
    dh binary

Once you have an install target, you can do this to run some commands manually after the install target runs, and before the remaining steps are run to build a binary.

binary: install
    dh_strip -X foo
    dh_fixperms -X bar
    dh binary --remaining

If the --remaining was left off, dh binary would only run commands in its list after dh_fixperms. With --remaining, it runs commands that haven't run before, avoiding running dh_strip and dh_fixperms again. A more elaborate way to do the same thing, without needing a separate install target is this.

binary: build
    dh binary --before dh_strip
    dh_strip -X foo
    dh binary --after dh_strip --before dh_fixperms
    dh_fixperms -X bar
    dh binary --after dh_fixperms

the dh_auto_* commands dh_auto_* are thin wrappers, with enough intelligence to handle 90% of cases (autofoo, perl/python packages, etc) automatically. These will rarely be run by hand, instead if they don't do what is needed, they can just be skipped.

• dh_auto_configure: Figures out how to run autofoo, ./configure, Makefile.PL, or whatever.
• dh_auto_build: runs make
• dh_auto_test: runs make test or the like (may have a config file specifying if test failures should fail the build).
• dh_auto_clean: runs "make clean" etc
• dh_auto_install: runs "make install" or similar

the dh command dh runs a set of other debhelper commands in order (and prints their names as it does so). The set that it runs is determined by its parameter. The sets of commands are:

• build: dh_auto_configure, dh_auto_build, dh_auto_test
• clean: dh_testdir, dh_testroot(?), dh_auto_clean, and finally dh_clean.
• install: Everything in the build list, plus dh_installdirs, dh_install, dh_auto_install (and probably dh_install* too)
• binary: Everything in the install list, and all debhelper commands used in binary targets.
• binary-arch: Like binary, but runs commands with -s.
• binary-indep: Like binary, but does not run dhstrip or dhshlibdeps or dh_makeshlibs, and runs commands with -i.

Possibly dh does smart optimisations to avoid running lesser-used commands if they will do nothing. (Or not.) Each debhelper command will record when it's successfully run in debian/package.log.debhelper. (Which dh_clean deletes.) So dh can tell which commands have already been run, for which packages, and skip those. Options passed to dh are passed on to each command it runs, which is useful for common options like -v, -X, and the like. For that matter, you can use it for uncommon options as well, since commands will ignore options that don't apply to them. dh has a few options of its own:

• --until cmd: Run commands until and including cmd, then stop.
• --before cmd: Run commands before cmd, then stop.
• --after cmd: Run commands that are listed after cmd.
• --remaining: Run all listed commands that have yet to be run.

cmd can be a full name of a debhelper command, or a substring. It'll first search for a command exactly matching the name, to avoid any ambiguity. For example, "--before dh_installdeb" does not refer to dh_installdebconf. Otoh, "--before build" is much easier to type, and clearer, than "--before dhautobuild". If there are multiple substring matches, the last in the command list is taken. So "--before install" stops before dh_auto_install, and not before the earlier dh_install. Each time dh is run, it examines the log, and finds the last logged command that is in its list of commands to run. It then continues with the following command in its list. (Or the one after --after, if that was specified. Or if --remaining was specified, start at the first command in the list that is not in the log.) So... If you run debian/rules build twice, and it just has "dh build" in it, the second run will be a no-op. The same as if the build target touched a build-stamp. This is generally considered a feature, so I'll consider it a feature in dh too. (To force a rebuild, run dh_clean to remove the log.) If you run debian/rules binary, and a build hasn't happened yet, it will run the build steps first. Even if the rules file is a bit lame (or over-golfed), and doesn't have the binary target explicitly depend on the build target. Ditto for the install target. If a rules file runs a command out of order, it'll need to use --after to get dh back on track. If you manually run a debhelper command at the command line, running debian/rules binary will skip forward until after that command. Similarly, if debian/rules binary is interrupted (or fails), running it again will resume after the last debhelper command run. This is a bit of a behavior change, and could be considered a bug, but is easy enough to work around by running debian/rules clean binary. (Or by manually deleting the log.)

...is just not enough time to see the place properly. I took Friday and Monday off, and Thursday night a week ago, Sarah and I flew down to San Diego for a bit of a "Yay, we've achieved normality" escape. We'd intended to go to SeaWorld on Friday, vege out for two days, go to San Diego Zoo on Monday, then fly home on Monday night. Well we did SeaWorld and the zoo as planned, but lazing around the pool never happened, we ended up running ourselves ragged exploring San Diego all day Saturday and Sunday instead. We stayed at The Dana on Mission Bay, which we picked because it was the closest to SeaWorld. The accommodation was more like a motel than a hotel. There were numerous two-storey buildings across a fairly sprawling area. It had frontage onto what I presume was Mission Bay, and also had a Marina attached to it, so the outlook was quite nice. It also had free WiFi, so we could upload photos throughout the stay. The food offerings were pretty good, and reasonably priced as well. SeaWorld was really good. It's been a long time since I've been to the Australian equivalent, but I don't remember it as being as interactive. San Diego's SeaWorld had a Bay Ray feeding area, where you could purchase food (small whole fish), and hand-feed the Bat Rays. That was pretty cool. It took a bit of a courage to leave your hand in the water, palm up, with something the size of sardine dangling between your fingers and let these huge rays literally swim right over your hand so they could suck the fish from between your fingers. There was also a dolphin feeding area, where you could purchase some small fish again, and give the dolphins a pat on the head and then throw them a fish. I really love dolphins, and one day I'd like to be able to swim with them. The main attraction of SeaWorld is the various shows that they do. I think we caught all of the different ones. The killer whales are just amazing. There were a few different programs where for an additional fee, you could swim with various animals. Dolphins and Beluga whales seemed to be the ones we noticed in particular. There were a few oddities, though: The park is owned by a beer company, so there was a part where you could go and get free samples. Two per person per day (I think on the honour system). At least at the food outlet where we bought lunch (they don't allow you to bring in any food or drink from the outside "for the safety of the animals", so they have a nice monopoly on catering) every kid's meal came in a commemorative Shamu blue plastic lunch pail - whether you wanted it or not. There were two empty ones left on the table that Sarah and I sat down at. The lady cleaning up the tables asked us if we wanted them, and we said no, and she promptly chucked them out with the trash. This just struck me as a terrible waste. They obviously cost money and energy to produce, and they're just going to end up in landfill. They could have been washed and reused instead. I felt this was very hypocritical for a park that was trying to send people away with a message about conservation. Other than this nitpick, I thought SeaWorld was really great. It was a good size, and it was doable in one day, in their normal opening hours. I guess it'd be a bit slower with kids. Photos from the day are here. The next two days we spent exploring San Diego. The hotel was conveniently located on an MTS loop, which ran surprising frequently for a weekend (at least compared to public transit in the Bay Area), so we bought day passes on Saturday and Sunday and used it to get to the Old Town transit centre. On Saturday we explored Old Town, which was a historic preservation of how San Diego looked "back in the day". In the afternoon, we also bought 48-hour tickets for the red double-decker "hop on, hop off" tour bus, and did one of the two loops that it offered. The next day, we caught the loop in the opposite direction, and checked out Mission Beach, which was probably the best beach I've seen in California so far (although the water was still far too cold), saw some pretty cool alternative accommodation (possibly for next time), and did part of the other loop on the tour bus, getting off at the USS Midway. This massive retired aircraft carrier is permanently moored at San Diego, and for a fee, you can crawl all over most of it. It was very interesting, and gave a good insight into the life of a sailor. We blew a good 4 hours or so here, and ran out of time to see the entire thing (we didn't make it onto the "island" part of the ship, which we were a bit bummed about). We then caught the trolley back to Old Town, and grabbed a beer at a pub that sold it by the yard glass and half yard glass. We opted for the half yard glass, since the full yard glass seemed a bit unwieldly, and we thought it looked like a hell of a lot of beer, but the half yard glass, whilst also looking like a lot of beer, was only about one and a half pints. We then grabbed some dinner at one of the very authentic looking Mexican restaurants (they had women out the front, almost on the sidewalk making fresh tortillas on the spot) and headed back home. Photos from our exploration are here. On the last day, we went to the San Diego Zoo (which every time I read the URL for, I read it as "Sandie go Zoo!"). This zoo is purportedly the best zoo in the country, but it really didn't blow my socks off. We seemed to have a really hard time navigating the place, and spent a lot of the day walking around in circles trying to find various exhibits. It's also in a bit of a valley, so the circles tended to be up and down hills, which was tiring. I really don't think we were very efficient at all in our coverage of the place. The other thing that I personally found annoying was the cages. The wire was very close together, which made it really hard to take photos, because the camera would keep focusing on the wire, instead of what was behind it. I guess this is why God invented manual focus, but that made it very hard to photograph big cats stalking their cages. The photos we did manage to get are here. Overall, we had a great time in San Diego, and it seems we didn't really scratch the surface. I'd like to go back again and see Balboa Park, the Gaslamp district, and I thought there were more naval vessels that you could look at, but I might be mistaken. Definitely a very nice city.

Section 5.7.2 of the ACPI specification provides for a method called _OSI. Platform firmware can query for various interfaces, which allow it to tweak its behaviour depending on the interfaces that the OS supports. Linux claims to support the following interfaces:

• Windows 2000
  
• Windows 2001
  
• Windows 2001 SP1
  
• Windows 2001 SP2
  
• Windows 2001.1
  
• Windows 2001.1 SP1
  
• Windows 2006

which all correspond (unsurprisingly) to different versions of Windows. Linux used to be in there, but was removed for reasons I'll get into later.

The typical way in which this feature is used by a platform is by querying each of the interfaces in turn, starting with the most recent a firmware is aware of. After the first match, a value will be stashed somewhere and used elsewhere in the firmware when making certain decisions. An example of this may be only enabling certain devices (such as the HPET) if the OS claims to support Windows Vista.

As I mentioned, recent versions of Linux no longer claim to support the Linux interface. There's a couple of reasons for this. Firstly, some platforms break if they're told that they're running on Linux. It's usually an entirely untested codepath. Secondly, the whole point of the _OSI method is to provide the platform with information about the interface supported by an OS. "Linux" is too coarse grained - we make no guarantees that the interface between the OS and the platform will remain constant. "Linux 2.6.25" isn't all that much better, since distribution patches may alter the behaviour and BIOSes would become specific to a given kernel version. We could have something like "Linux ABI 1" and then bump that every time we altered the behaviour of the kernel in this respect, but this would become unwieldy fairly quickly (what would count as an ABI breaking change?)

So what kind of interface can we claim to provide? The sad truth of the matter is that most vendors will only test their code against Windows, and that as a result Windows provides the de-facto specification for OS/platform interaction. We could skip claiming to be Windows, but (in the best case) that means that the system will assume it's working with an OS that supports no advanced features. Rather than getting support for brightness changes being handled by the OS, we get stuck with SMM traps and unexpectedly missing time. If we want hardware to work to its full potential, we need to claim to be a recent version of Windows.

Does this mean the entire _OSI mechanism is useless? Not really. It's still useful for providing information about supported functionality. One example used in the spec is whether or not the OS implements the ACPI 3.0 thermal model. Another obvious (but unspecced) example might be whether or not the video opregion spec is implemented in the OS. _OSI strings which explicitly advertise the existence of advanced functionality are something that should be specified and encouraged. What they shouldn't be used for is as a way of advertising lacking functionality, which is effectively the only way that Linux can be usefully interpreted today. We don't want machines to automatically repost their video if they detect they're running Linux. It makes it harder to implement proper kernel-level graphics resume code. We don't want them to suddenly alter their keyboard behaviour, because that probably makes it harder to support older machines that don't have these workarounds.

Message to vendors: If we're not behaving in the same way as Windows does, let us know. We'll fix it. This has the benefit of fixing not only your latest and greatest platform, but also all the older ones that expected the same behaviour. Please don't try to work around our bugs in your BIOS - it just means that we need to special case you forever, and in the long run your hardware won't work as well as it would if we just fixed things properly.

This week I'm in Mountain View for the X Developers' Conference, where I gave a presentation on things graphics drivers have to care about in order to make suspend/resume work reliably. Various people suggested that I write up an explanation of the current situation, along with some best practices for manufacturers.

One of the major differences between the ACPI specification and the older APM specification is that saving and restoring hardware state is generally left up to the operating system, with the firmware playing a much more passive role. Linux is now able to handle this state restoration for the majority of hardware, but we still have issues with graphics. Modern graphics hardware can be hugely complicated, which to a large extent we've been able to ignore because the platform will initialise it on boot. Many X drivers know how to switch a card from text mode to graphics mode (and vice-versa), but that's not adequate.

What can make this situation even more awkward is that the ACPI specification makes no guarantees about what state the hardware comes up in. It may still be in PCI D3. It may be powered up, but the registers may all be zeroed. It's even valid for the hardware to come up in text mode. A driver has to be able to deal with all of these situations.

So far, we've mostly tried to deal with this situation by running chunks of the video BIOS with applications like vbetool. This is hacky and often works badly - in some cases, it won't work at all (nvidia rewrite the entry point to their video BIOS after boot, in order to prevent situations where someone attempts to POST the card and it tries to jump to bits of the system BIOS that no longer exist). The only long-term option is for the kernel to handle the graphics suspend/resume. There's several levels at which this can be supported:

• Able to restore text mode. This is basically adequate right now, since the kernel enforces a virtual terminal switch around suspend/resume.
  
• Able to restore arbitrary video mode. In the kernel modesetting world, we want to be able to go straight into the mode that we're running X in, in order to reduce screen flicker. This would probably still be surrounded with a VT switch out of and into X.
  
• Able to jump straight into X. Right now, on x86 the process freezer will prevent X from running until after all of the devices have been resumed. As we move to a model where we remove the freezer and expect drivers to be able to block userspace tasks, we hit a pretty obvious problem with X - right now it hits the graphics hardware directly, rather than going via the kernel. With nothing to block it, it'll hit the hardware while it's still in an undefined state and then become very unhappy. This can't really be fixed until X does all its hardware access through the kernel. For now, we can inhibit X by leaving the VT switching in place.

Since 2.6.25, i915 and later Intel hardware can manage basic mode reprogramming without requiring any external assistance. The radeon kernel modesetting code is also heading in this direction, and we ought to be able to manage something there before too long. The nouveau developers are heading towards the stage of being able to do native mode programming, so there's some hope of progress there as well.

It's not all plain sailing, though. Vendors can wire up hardware in different ways. AMD and Nvidia hardware provides scripts in the ROM that, in theory, abstract this away. The downside to this is that we'll have to put the interpreters for these scripts in kernel space, though this is probably needed for kernel modesetting work anyway. Even then, it's possible that we'll need workarounds for specific pieces of hardware that don't quite behave as we expect them to.

As far as making graphics work over suspend/resume, we have a pretty good idea where we are now (failing on anything other than modern Intel hardware), where we need to be in the future (triumph, huge success, cake) and how to get there (?). However, hardware manufacturers can still help.
Chipset vendors:Tell us how to reinitialise your hardware. If that involves parsing BIOS tables, then please let us know how to do that. Expecting us to jump into your ROM and execute x86 code is ok if that's genuinely the only way to do it, but for any modern hardware that's clearly not the case.
System vendors:Try not to alter your system behaviour too wildly. Requiring magic register writes in order to light up the VGA output does not classify as value add. Please don't have your firmware program text mode directly - we're trying to get away from that. And do not attempt to detect that you're running on Linux and alter your system behaviour. If it's possible for you to tell that you're not running on Windows, that's a bug in Linux. If you rely on that bug, you will break. I'm not joking here in the slightest. The closest thing to a "standard" interface between an OS and the hardware is whatever Windows does, and we're basically aiming for bug-for-bug compatibility there.

After the unfortunate show that Microsoft and ISO provided us recently, ISO’s response is a FAQ note about ISO/IEC 29500 (OOXML, that’s it) in which they threaten Microsoft with the most severe of penalties if they’re tricking on as. Funny. In that statement, they try to answer all the issues that initially were concerns but that now has turned into proven facts about the OOXML standard and it’s approval, and to try to recover some of their lost credibility. It’s sad to be commenting on this, words don’t come easy, as if to say. Microsoft has already proven who’s the one in charge here, who owns the committees, who takes the decisions and in which terms, and nothing happens. It’s gonna take more than to publish a FAQ note saying how well you did it and to threaten Microsoft to behave or else. No one seriously believe you at the moment, you know? A funny thing is that ISO dismisses any responsibility about standardization, and relies on the market taking the decisions for them. Microsoft has said many times that they consider something positive and healthy to have multiple standards for the same, and let the market decide. Now ISO declares to think along the same lines: “After a period of co-existence, it is basically the market that decides which survives”. We don’t really need an organization for standardization for that, do we? We already have the market for that. Will ISO apply the same policy for every standard from now on? Like, say, having a couple of competing standards at least for every need, and let the market decide? ISO tries to recover from their credibility loss with this note (“the standards development process is credible, works well and is delivering the standards needed, and widely implemented, by the market“), without much success at least in what I’m concerned. If they keep trying to convince with press notes instead of with real acts, no one is gonna believe them ever again.

It has now been more than 4 months since the last Debian developer account was created. 18 contributors have been through all steps, and are simply waiting for this simple administrative task to be done. We are sending a terrible message to potential contributors. We have strong requirements on the technical level of our developers. During the new maintainer process, we ask them to answer about 80 questions about Debian. We ask them to do grunt work. We review their reports twice (New Maintainers’ Front Desk, then Debian Account Manager). But even after we are totally satisfied about what they did, even after they became more qualified than many of our current DDs, we ask them to wait for months, so that the only person allowed to create accounts can finally do his “job”. It discourages the contributors currently in the NM process. I’ve seen several signs of frustration, or even depression. Some of them reduce their involvement in Debian, so we lose them before they even became Debian developers. Some of them consider resigning from the NM process. We should all feel guilty about that. But it also discourages people from joining Debian. Instead, they go to other more welcoming projects, which is totally understandable. Debian isn’t the only distribution with developers from the community those days. There’s Gentoo, Ubuntu, Fedora, openSUSE. Some of those have nice programs for new contributors, like “school” sessions. Sure, Debian is the “biggest” distribution without a company behind it. But is independance worth all the trouble? Of course, we have Debian Maintainers. DM is great for people who want to work on their packages. But, when we are trying to release lenny, we need more: people who are going to go through RC bugs, submitting patches. Who are going to do QA work. In short: people who care more about the whole distribution. Can we afford not recruiting anybody? Can we run Debian with the current manpower? I don’t think so. There are more than 550 RC bugs in lenny, many packages are currently blocked from migrating to lenny because they are RC-buggy, and many packages are orphaned or neglected. There are also a lot of bugs which haven’t been filed yet (I asked for help with running piuparts, which would probably result in 100-200 new RC bugs, but nobody had time to help). Most of the work that needs to be done is not rocket science. We could use a lot more manpower. Currently, the same small set of developers is doing most of the grunt work. They will get tired too. So, what can we do, as simple developers? There’s no magic solution, but we can try a few things.

1. It seems that some people disagree that there’s a problem. Let’s prove them wrong: we could start a blog meme with “I, too, agree that the Debian accounts and keyring situation is severely hurting Debian, and that a solution needs to be found RSN.” It’s not going to solve the problem by itself, but it will at least show that we consider it very important. Pressure could help.
2. We could start discussing solutions together. Our newly elected DPL said that he would talk with the problematic teams to determine how the situation could be improved. Unfortunately, this has been tried in the past (and failed). It might work this time, of course, but we could prepare a backup plan. So let’s find one or two good plans, and vote on them. (I liked the idea of giving accounts creation/management to DSA. After all, it’s only an sysadmin task once the report has been approved by FD and DAM.)
3. We could push forward Josip Rodin’s proposal about infrastructure teams. It might not solve the DAM problem immediately, but would probably help avoid similar problems in the future.

Notes:
1. Maybe the 18 waiting accounts will be created today or tomorrow. Even if that happens, it won’t solve anything. Waiting 4 months for a simple administrative task is not acceptable, and we need to fix that problem anyway.
2. Account creation is not the only problem. Some people have been unable to upload packages or to vote for the DPL election, because their PGP key expired, and nobody updated it even if they have been asking for more than 4 months.