* Long time, no see! Well, I am busy with ‘real’ life Kavin is fine and growing like increasing number of Linux users * Firefox (ah, IceWeasel!) team is coming to Mumbai (See ILUG-BOM website/mailing list for more details), as a contributor to Gujarati Localization, I got this cool Tshirt. * Installed Debian ‘Etch’ on my office machine! Earlier, I was responsible for installing Ubuntu here It took around 20 minutes for fresh install (1st CD). I would love if we can fit OO.o in first CD! Everything went fine. Smooth migration from Thunderbird mail box to IceDove. And, I love this cool sw seahorse.

The Cyber Law Center has blogged about the Phonographic Performance Company of Australia Limited being permitted to charge night-club owners $1.05 per person per night for playing commercial music up from $0.07. The number of people is calculated on the maximum licensed capacity of the venue - not good if the venue isn’t full. Of course given the huge prices of drinks in clubs this probably makes no difference to the profit margins. One thing that hasn’t been mentioned in any of the articles I’ve read is whether this applies to clubs that don’t play music from artists represented by the PPCA. If a club can freely play music produced under a Creative Commons license that permits commercial use then the fee increases would be good for the production of music. Also encouraging clubs to pay musicians to perform live is a good thing, most of the best musicians started out with live performances in bars. The recording industry is the enemy of music. They under-pay artists and lock out competition giving us a succession of talent-less wage-slaves on the radio. It’s difficult to imagine any change other than Digital Restriction Management (DRM) that could make things worse.

Share This

I've been playing some computer games this week, which I don't often do. I finally tried a modern version of this Wesnoth game that everyone in Debian was playing a while ago, and have been enjoying working through the stories in it. I played glest for a few hours, and really enjoyed the 3d graphics, especially when it rained. I'm no good at real time strategy games though, and ordering units around contantly becomes wearisome. Globulation 2 might partially manage to avoid that problem, I'll have to try it a bit more sometime. Transcend is an amazing thing. After finishing a level, I found myself avoiding going on to the next level for a long time so I could spend time messing around with the thing I constructed while playing the game -- and wished there was a way to save my creation. The graphics are reminiscent of kalidoscopes and flowers. It subverts tradition, it has the trappings of a spacewar descendent, but instead of Star Control, the game turns out to be about music growing. Like tetris, it's abstract enough that I can zone out while playing it and get into a different mental state; and I can even pick which kind by making different kinds of music as I play. Wow, wow, wow.

I now have a working system for creating minimal installations of Fedora Core (6), and CentOS (4.5 & 5). I've updated xen-tools to make use of this, via --install-method=rinse - and now I can create new Xen guests :) There were several false starts: <dl>

Have a static list of .rpm names + versions

This is precisely what rpmstrap does, and it is a broken approach. I'm happy to have a static list of packages, but restricting the packages to specific versions is fragile and broken.

Use Yum

yum is used by many RPM based distributions, but unfortunately each distro uses slightly different versions to the one in Debian - so whilst I could run:

mkdir /tmp/fc6
yum -c /tmp/yum.conf --installroot /tmp/fc6 install yum

This results in a system which cannot be updated:

chroot /tmp/fc6 /usr/bin/yum update
[errors about yum database formats]

I suspect this is because the versions of the Berkeley database are different .. but I'm not sure.

Parse Dependencies

This would be a neat approach, however it is difficult. Too difficult. Abandoned after several fruitless hours.

Cheating

I cheated in the end.

• Download a list of packages which are known to be required to produce a working yum, and unpack them.
• Copy those downloaded packages into the yum cache tree.
• Run yum install yum - to make the local Yum + RPM database match what we fetched manually
  • Unfortunately "rpm rebuilddb" couldn't do that for us.

Worst case? I download each package twice. Once for the initial download, and once again when the Yum process runs. I can live with that.

</dl> I'll talk about something else next time. Promise.

On several occasions I have watched part of a TV documentary on gifted children, but I have never been able to watch one completely because every one that I have seen has been offensively wrong. One thing that they always seem to do is say that gifted children have special needs and often claim that they have problems socialising. This sounds quite reasonable, but if that’s the case then why would you make such children perform tricks on TV? Putting the children on TV is a very poor example of journalism and often of parenting - the parents’ desire to boast about their children’s performance (and implicitly their own parenting skills) is apparently more important than protecting the children. I think that the only situation in which gifted children should have their talents demonstrated on TV is if their skill is related to the performing arts. Not that TV coverage is necessarily good for children who have abilities related to performing (a casual scan of the news regarding adults in Hollywood shows the problems that people have dealing with fame), but it’s something that they will be driven to anyway. Also if they are going to demonstrate the intelligence of a child on TV then they really should make sure that they don’t demonstrate a lack of intelligence (on the part of the child as well as the producers). Asking a child to provide a definition of a word is often used as an example of intelligence (when it’s really an example of vocabulary). In one documentary a child defined a philanthropist as “someone who has a lot of money” (according to dict on my system it is “Love to mankind; benevolence toward the whole human family; universal good will; desire and readiness to do good to all men“). In another a child defined a genius as “someone who knows a lot“, while the definition of genius is not clear and there is some disagreement about what it is, most people agree that it’s about ability not knowledge. Being able to recognise when you don’t know something and admit would surely be correlated with intelligence… Telling a child that they are a genius or telling them their IQ seems like a bad idea at the best of times, and doing so in front of a documentary camera crew isn’t the best of times. Children are able to determine how their skills compare to others, there are more than enough attitude problems in schools related to skill comparisons without encouragement from adults. When I was in high school a friend who studied a martial art refused to tell me which belt he wore - he had been taught that such things shouldn’t be discussed outside the dojo to avoid creating a hierarchy based on belt rating in the community. I think that the same thing could be applied to IQ ratings. The term genius is grossly overused generally, I believe that showing greater than average ability in one area is not enough to qualify. I think that the minimum criteria would be to produce dramatic new developments/inventions in one area of research (EG Albert Einstein or Stephen Hawking) or to advance multiple fields of science or art (EG Leonardo da Vinci) would be the minimum criteria. I had people call me a genius when I was young because I won prizes in maths competitions, was good at programming computers, playing chess, etc. The main criteria for achieving such things is to avoid wasting time on sport - in a school run by idiots with a focus on sport this was more a tribute to being stubborn than being smart! The TV documentaries mention many things that gifted children supposedly require, but strangely I don’t recall any of them mentioning the need to meet adults who are significantly smarter than average! You might think that this is the most obvious thing. The Big Brother and Big Sister programs of mentoring children who are at risk of crime and drugs apparently provide significant benefits, something similar for gifted children might do some good, probably more based on meetings of a few intelligent adults with a small group of intelligent children instead of the BB-BS model of one-on-one meetings. Washingtonienne seems like a good example of the need for this. There are some useful print articles however, this article in The Age makes some good points, it’s particularly interesting to note that some schools lie about special programs to support gifted children to attract students (someone should name the schools that do this). But then you get awful ones like this, naming the child in such a situation is irresponsible journalism. The girl in question may be forced to change her name to escape google when she’s older. It’s strange that it’s illegal to name a child who is involved in a court case but it’s not illegal to name them in such an article (let’s hope that the journalist used pseudonyms). Also you might expect an organisation such as Mensa to have someone smart enough to realise that bringing such attention on a 2yo is not in the child’s best interests. Does anyone know of a good TV documentary about gifted children? I guess that it would be extremely difficult to make one without showing the children on TV so that would restrict the film maker to children who’s abilities are related to the performing arts.

Share This

A few people commented on my last post asking for details about how I configured exim and dovecot to have the fake scott+canonical account and separate Maildir tree. exim4 router configuration The first key part of the configuration is to configure exim4 to split the local part into a user name and a suffix. This allows a local part such as “scott+canonical” to be split into the user name “scott” and the suffix “+canonical”. This is configured by adding the following two options to the appropriate routers in your exim4.conf:

local_part_suffix = +*
local_part_suffix_optional

The first option defines the suffix; normal practice appears to be to use both “+” and “-“, but since I’d previously used “-” with qmail I opted to use “+” only so that I didn’t confuse myself during the transition. The second option allows the suffix to be optional, so that mail to “scott” is still delivered normally. Now mail to “scott+canonical” will be delivered to the “scott” user. Forwarding configuration The next task is to ensure that mail is actually forwarded to this address; for me this was a configuration performed by the Canonical sysadmins to ensure that my work e-mail is actually delivered to scott+canonical on my own mail server. Filtering configuration Since both personal and work e-mail are now both being delivered to the same user account on my home mail server, I need to filter the mail into separate folders. This can be done by checking the $localpartsuffix variable in Exim filter .forward files, e.g.:

# Exim filter
if $local_part_suffix is "+canonical"
then
    save Maildir/Canonical/
endif

Now incoming work e-mail is filtered into a different mail folder, while personal mail is delivered into the primary one. I’ve used similar filter instructions for mailing lists, mailman messages, Launchpad mails, etc. to filter into appropriate folders. Where the mail is personal, it is filtered into (e.g.) Maildir/.Lists/upstart-devel/ Where the mail is for work, I add /Canonical/ to the path, (e.g.) Maildir/Canonical/.Lists/linux-hotplug-devel/ This means that the Maildir/ directory is my personal INBOX, with sub-folders immediately under that and beginning with a period; and the Maildir/Canonical/ directory is my work INBOX, with sub-folders immediately under that and beginning with a period. This defines two trees in a manner compatible with dovecot. There’s no particular reason that Maildir/Canonical/ has to be under Maildir/, it could have been Maildir-Canonical/ and this would still work. I simply wanted them in one place to ease backups. Dovecot configuration Now we need to configure dovecot to permit login by a fake (“virtual”) user, with a different Maildir tree, so that I can configure them as two separate accounts. The first set of changes is to dovecot.conf to add an additional authentication source. Modify the “auth default” block to add a new “passwd-file” passdb in addition to the “pam” passdb (or whatever your system is using).

passdb passwd-file  
  args = /etc/dovecot/passwd
 

This lets us authenticate virtual users, but we also want to set their attributes, so we can use the same file as a userdb in the same “auth” block.

userdb passwd-file  
  args = /etc/dovecot/passwd
 

Dovecot will now check both PAM and this file for user information. We now simply need to add a line to this file to specify the virtual user and set up the alternate Maildir tree.

scott+canonical:PASSWORD:1000:1000::/home/scott::userdb_mail=maildir:/home/scott/Maildir/Canonical

The format is that of an ordinary passwd file; the first two parts give the passdb authentication credentials and the rest give the userdb information. I’ve set this user to have the same uid, gid and home directory as my real “scott” user. The final part changes the mail environment for this virtual user, instead rooting it at Maildir/Canonical/ Client configuration The mail client will need two accounts adding; one for “scott” and the other for “scott+canonical”. It will see two separate folder trees for each account. An unexpected bonus is that the reply account is now automatically set for me, since I’m replying from the specific account rather than from a single general one.

A hot topic this past DebConf7 was git. I've bought into the hype, so I decided to take the plunge and move my Subversion repositories to it. A lot of people seem to be willing to part with their history when making the move, but I'm some what irrationally attached to it.

Most of my Debian packages are in one big SVN repo, which used to be one big CVS repo (I used cvs2svn to move). So I decided to pull apart each source package into its own git repository. I started with git svn clone like so:

$ git svn clone -A authors --trunk=trunk/libclass-makemethods-perl \
  --branches=branches/libclass-makemethods-perl/upstream \
  --tags=tags/libclass-makemethods-perl --no-metadata \
  file:///home/svn/debian libclass-makemethods-perl

And this gives a git repo that can track an SVN repo. But I want to go all the way, no turning back. So I made a little script to clean up this repo called clean-git-svn:

#!/bin/sh -e
tags= git branch -r   egrep 'tags/[^@]+$' 
echo "$tags"   perl -ne \
  'chomp($_); my $head = $_; s,\s+tags,debian,g; print "git tag $_ $head^\n"'
for i in $tags ; do echo "git branch -r -d $i"; done
upstream_tags= git branch -r   egrep ' +[0-9][^@]+$' 
echo "$upstream_tags"   perl -ne \
  'chomp($_); s,\s+,,g; print "git tag upstream/$_ $_^\n"'
for i in $upstream_tags ; do echo "git branch -r -d $i"; done
echo "git branch upstream current"
echo "git branch -r -d current"
echo "git branch -r -d trunk"

This will output a bunch of git commands that will make this a more "gitified" repo. I make no guarantees this will work, you should probably hand check your history with gitk to make sure this won't screw anything up. If your the part you're converting also came up from CVS, you'll probably have a bit of a mess on your hands that will need more manual cleanup. gitk is really an amazing visualization tool and can help a lot here. Once you're satisfied just pipe it into /bin/sh. Since git svn mostly can't track where you merged in your SVN repo, make sure you merge the upstream branch into master as soon as you can. It will save you a lot of conflicts later, trust me.

One thing I miss from svn is svn export. Does an equivalent command exist in git? You would think there would be an option you could pass to git clone to do this but I can't seem to find anything.

There are lots of packages I’ve been doing since I firstly started developing stuff for Debian, that I just didn’t do anything with (apart from using them myself, that’s it). Some are lacking man pages, icons, or things like that. At the time I just didn’t feel like it was too urgent to finish them, because usually finding a sponsor for them would take more effort than making the package itself, but I plan to finish them some time and putting them into the repositories. In any case, I thought that it might be nice to clean them up a bit, compile them against SID, and putting them available somewhere, if someone was interested. Right now I’ve put there:

• construo: Construo is a program for constructing 2D wire-frame objects and let them react on physical forces, its neither a real simulation nor a real game, its just a little toy app which can be a nice way to waste time.
• xspringies: XSpringies is quite similar to construo in its purpose. It’s a 2D mass and spring simulation system. It’s intended use is more like that of a game, than some design package.
• wok: It is an old game made by Kenta Cho. The game objective is very simple. Catch the falling balls by the wok (a flying pan) and throw them to the right. You can control the wok by mouse. To earn more score, you should throw many balls continuously. Collect many balls on the wok, and throw them at the same time. If a ball falls to the lower end of the screen, the game is over.

I plan to keep uploading some of my old packages that are worth it, are in good state, can be compiled against SID right now, and might be of interest to someone, at least until I find some time to finish them properly and upload them to the archive. If anyone is interested in them, they’re welcome!

Started with a cleanup of the account database, to get the official Developers count back to some reality. The first run did include about 400 people, where one or more of the listed categories applied. It started very good - my first run of my script just generated a bounce for every single mail I tried to sent. Turns out its not ok to have “To: user@debian.org user@debian.org>” in your mail text, the setup on master.debian.org doesn’t like this. Oh yes, it really helps to use the right variable for the username part, which doesn’t have the domain appended. Oh well. Then I got that part right, and master did accept all my mails. Just so that I could discover another mistake in my script. Only a cosmetical one, but it helps if you set the From address to the wat alias, not your own, after you finished testing, in case you want to have replies go there directly. Tsss. Now I have to bounce those mails there. Anyway, those are minor things noone, especially the people who got a mail, need to care about, it all just works for them if they reply. And I already got about 40 bounces back, so already nearly 40 accounts to disable. Not all 40, because a few contain enough information so i can try a resend of the mail to a little modified recipient address. The rest is just so that I can’t do anything about them. Well. Ok. There are two people that are using one braindead idiot wanna-be antispam solution which requires you to send some token somewhere to get your mail delivered. Usually I just delete mails then and dont care if they lose, as they obviously do not want to receive mail. I am not sure if I should do it in this case, even if I would like to. After all its their lack of sanity that they do not receive mails… I also got around 60 replies to the pings, with a good number of them resigning from Debian and some others that reconfirm they are active (and mostly point me to their activities). The next WaT run will start about 4 months after this one. Now - 4 months after the previous is over (which would be 6 months from now) or after it started? Probably the first and then have 3 runs a year or something similar. I’m still looking at my scripts to gather the data to enhance them in a way that makes it easy. Like - If someone replies with “Im active” in this run I would like to recheck this if the same user is still a possible WaT target a few runs later.

I can successfully boot Fedora Core 6 & 7 - to the extent that RPM and Yum both work. This takes in the order of 70Mb and around 2 minutes - assuming the caching mechanism works. Now I need :

1. A cute name.
2. To abstract the common parts of the code somehow.
3. To get started on SuSE + CentOS.

Today: Hacking. Tomorrow: Security work.

So we're here in sunny Vilnius sprinting on bzr. I thought I'd write up some of what we've achieved. On Thursday Wouter got most of the nested-trees branch merged up with bzr.dev, but about 500 tests failing ;). Jelmer has introduced a new parameter to the sprout api call on BzrDir called 'limit_to_revisions' which if supplied is a set of revisions which forms a strict limit on the amount of data to be fetched. This is intended to be the API by which the desire for limiting history - to achieve shallow branches/history horizons - is communicated at the 'bzr branch' or 'bzr checkout' stage. This API is tested so everything passes, but nothing chooses to implement it yet. I was hacking on commit (somewhat indirectly) by working on the repository-wide indexing logic we will require if we wish to move to a more free-form database with semi-arbitrary keys mapping to opaque data (at the lowest level). I got a core GraphIndex up at this point. Once complete this should drop the amount of I/O we perform during commit by half. There was some other hacking going on too - Tim Hatch did some tweaks to bzr-hg, Jelmer some bzr-svn, and so on. On Friday we carried on on the same basic targets: Wouter fixed the 500+ failing tests so now its only in the 5-10 range, and has been debugging them one by one since. Jelmer implemented the limit_to_revisions parameter all the way down to the knit level, for non-delta compressed knits (e.g. revisions.knit), which made knit1 repositories support the parameter and got the 'supports' branches of the interface tests being executed. I developed CombinedGraphIndex, and have sent the low level Index stuff up for review, and followed that up by implementing a KnitIndex implementation that layers on the GraphIndex API to allow the use of .knit files without .kndx. This new index API allows us to switch out the indexing code and format much more easily than previously - the knit index logic is decoupled from the generic indexing facilities. It also allows us to record arbitrary-parent compression chains. Finally Jelmer implemented the wishlist item for bzr-gtk of a notification area widget that can be used to enable bzr-avahi watching, and share commits with the local LAN/get notified of commits on the local LAN. Today is the last day of the sprint, and I hope that we'll get the nested tree branch passing all tests, the limit_to_revisions parameter causing delta-compressed knits to only copy data outside the listed parameters to the first full text, and have an experimental repository format that uses the new index layer for at least a couple of its knits (e.g. revisions and inventory, or revisions and signatures).

Check out this black and white digital back for medium-format cameras. Why aren't there any dedicated black-and-white sensors available in compact digicams or DSLRs? The sensor would have much more light to work with than a color sensor with a Bayer filter. It would also be able to use colored filters without throwing away so much information. It would be a niche product for sure, but it would be perfect in something like the Ricoh GR-D or the Sigma DP1. The people buying those cameras are already sacrificing flexibility (both have fixed prime lenses), and many of them are street photographers and shoot primarily black and white already. Wouldn't a dedicated black and white sensor improve dynamic range and sensitivity for available-light shooting? Add this to my DMD wishlist. Not that I could actually afford it if it existed.

So. In a month's time I become unemployed. Which is something I couldn't have predicted a year ago. This is going to mean a lot of changes for me, most of which I haven't really been able to even start to plan until the news was out in the open. Scary stuff.

Early this year I deployed a server. As part of my normal procedure I ran the Memtest86+ memory test program (which showed no errors) before deploying it. After some time running the machine started to become unreliable, yesterday it crashed twice and I had to replace it. I ran Memtest86+ before removing it from where it was installed and found several memory errors. When a server crashes I highly recommend running Memtest86+ before removing it so that you at least know the cause of the problems. As I want to use the machine elsewhere I want to discover the cause of the problem. The machine has two DIMM sockets (I’ll call them A and B) and two DIMM modules (again I’ll call them A and B). After getting the machine home I first tested the machine with DIMM A in socket A (and DIMM B removed) which passed, then I tested it with DIMM A in socket B which also passed. I removed DIMM A and tested DIMM B in each socket and those tests passed. Then I installed both DIMMs and again the test passed! I now realise that I made a mistake in removing a DIMM when I got the machine home. I should have tested it again with the DIMMs in place. If the problem was due to heat or a poor contact made worse by vibration then the problem might have gone away during the trip home - it would have been handy to know that I would be unable to reproduce the problem! My mistake here was to change multiple factors at the same time. When diagnosing faults you should try to change one thing at a time so that you will know what fixes it! Now, I am wondering what I should do next? Assume that it was just a bad contact and put the machine back in service? Suggestions appreciated.

Share This

One of the coolest features of grml IMO is grml-tips. grml-tips is a collection of tips you can request via a keyword. For example if you want to know how to use grml-x just run ‘grml-tips grml-x’:

mika@meilenschwein ~ % grml-tips grml-x
Start X window system (XFree86 / Xorg / X.org):
% grml-x $WINDOWMANAGER
Usage examples:
% grml-x fluxbox
% grml-x -mode '1024x768' wmii
% grml-x -nosync wm-ng
--
mika@meilenschwein ~ %

So as soon as you invoke ‘grml-x $keyword’ and if the grml-tips collection includes something with $keyword inside you’ll get it. I use that on daily base because all the hard-to-remember command sequences or dangerous commands like mdadm, sfdisk,… are easy to use this way. But how about keywords not being availble in grml-tips collection yet? What about keywords other users request but don’t get a match? Thanks to formorer we have a submit feature within grml-tips since version 0.4.1:

% grml-tips nfs
Sorry, could not find a tip for 'nfs'.  
Do you want to submit the keyword 'nfs' to grml's keyword database?
The grml team will write tips for the most requested and useful keywords.
To use and contribute to this feature you'll need a working networking connection.
No personal data will be transmitted to the database.
Send "nfs" to grml's keyword database? [y N]
Keyword 'nfs' has been submitted to grml's keyword database.
Thanks.
%

Now being the maintainer of grml-tips I can check what users want to get through querying the sqlite database which stores the submitted keywords. Dear paranoid users: by default the keyword is not transfered to the database of course, you have to manually press ‘y’ to submit it. It’s not the default behaviour but an option. And there’s really no personal information stored inside the database, all I can query looks like the following:

$ID nfs grml 1.0-1 Release Codename Sonnenbrand [2007-07-13] 1184261151

Oh and thanks to Nico we also have our 24/7 support slave within the #grml channel:

19:37 <     mika > grml-tips: grml-x
19:37 < grml-tips> matches: http://www.grml.org/tips/index.html#23

And if our support guy does not work as expected:

19:38 <     mika > grml-tips: does-not-match
19:38 < grml-tips> Sorry, could not find a tip for 'does-not-match'.
If you want to submit a tip please mail it to tips (at) grml.org - thank you!

… and once a week I get a cronjob-triggered mail sending me all the keywords without a match. Cool, ey? All what’s missing now is a bot that writes all the missing tips on its own. Conclusion: make it easy for your users to submit information you might be interested in - to help your users.

We've had so many great Lightning Talk submissions this year that we're going to have to split the session into two. There is still the original session at 15:00, but there is now another session at 17:00 for any talks that don't fit into the first session. If you are doing a lightning talk, please remember to either check that your laptop works with the projector in the hall perfectly before the talks start, or put your talk on a USB memory stick -- something portable such as PDF or S5 please -- so you can use the provided laptop (a ThinkPad T43, 1024x768).

The Debian Description Translation Project is a very old project which had many adventures in the past. Indeed, working on packages descriptions translations has been my very first work in Debian l10n back in....long ago. Unfortunately, because of the servers compromise and also because of the lack of support for translated descriptions in the archive and package tools such as APT, the project slowly lost interest from translators. The revival may be dated back in 2005 when Michael Bramer revived the project and even more in 2006 when experimental support was added in APT as well as some support in the archive. While the framework is still quite crude (no offense intended for grisu who did a great job maintaining it) and not very opened to collaborative work and peer review, a few teams did a great job translating the zillions of package descriptions. And we still expect to use Pootle soon. So, as we now have a big enough handicap, it's time for the French team to beat them all in the friendly competitions we all enjoy..:-). For that reason, I began to translate dozens of packages descriptions last week and a few other French translators joined me. I somewhat expect us to slowly put all of our (wo)manpower in this and, therefore, translate as many packages descriptions as possible. The count begins with 1896 active translations, French being ranked 5th after German, Japanese, Czech and Brazilian Portuguese, even with Korean. So, the next target is the Brazilian team. Watch out your back, fellows..:-)

For a long time I've wanted to know how to embed binary blobs into executables. This would be most useful for files like Glade and and UI Manager definitions, which are required for a given program to work at all but either cannot be embedded as a string literal (Glade) or can be but is annoying (UI Manager). I finally asked the Interweb, and Daniel Jacobowitz replied with some pointers. It turns out that doing this is remarkable simple. First, a caveat. This probably requires GNU ld, which may or may not be a deal breaker for many people. First, create a data file. Let's call it foo.txt, and put some text in it.

Hello, World!

Using ld this can be read in as a plain binary blob, and then written as a standard relocatable ELF object.

ld -r -b binary -o foo.o foo.txt

Now we have a standard ELF object with the data and some useful symbols defined. objdump will show you the contents.

$ objdump -x foo.o 
foo.o:     file format elf32-i386
Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 .data         0000000d  00000000  00000000  00000034  2**0
                  CONTENTS, ALLOC, LOAD, DATA
SYMBOL TABLE:
00000000 l    d  .data  00000000 .data
0000000d g       .data  00000000 _binary_foo_txt_end
0000000d g       *ABS*  00000000 _binary_foo_txt_size
00000000 g       .data  00000000 _binary_foo_txt_start

Here we see 13 bytes of data, and a symbol which contains the address of the data. This is all we need to access it from a C program.

#include <stdio.h>
extern char _binary_foo_txt_start[];
int main (void)  
  puts (_binary_foo_txt_start);
  return 0;
 

Now if we compile this and link it against the generated object, we'll have a binary.

$ gcc -o test test.c foo.o
$ ./test
Hello, World!

Hooray! One small problem which alert people should have noticed: the string itself is in the .data section, which is read/write. For my use, I want it to be read-only data in the .rodata section so that it isn't copied for every instance of the application. As far as I know, this isn't possible with ld but objcopy will let us rename sections on the fly.

$ objcopy --rename-section .data=.rodata,alloc,load,readonly,data,contents foo.o foo.o
$ objdump  -h foo.o
...
  0 .rodata       0000000d  00000000  00000000  00000034  2**0

Excellent, problem solved. If you want to download this sample, I have a tarball. Many thanks to Daniel Jacobowitz for pointing out how to achieve this. Update: note that any data embedded in the binary like this won't be terminated with a NULL. This is obvious in hindsight, but due to luck my example still worked. There might be a way of asking objcopy to append a 0 to the end of the data, but if not always remember to use the start and end pointers or size instead of just the start, or append a NULL yourself before converting to an ELF. NP: (), Sigur Rós

#!/bin/bash
# keeps updating le tour on the top left of the root window
# Depends: bash, netpbm
while true ; do
  xsetroot -bitmap <(
    curl -s 'http://live.cyclingnews.com/?id=latest' \
      sed -e 's/<br[^>]*>/\n/g;s/<[^>]*>//g;s/&nbsp;/ /g;s/ *$//;1,/Tour de France/d;/\(rovisional\ start.*prev\)/,$d' \
      cat -s   fmt \
      pbmtext -builtin fixed   pnminvert \
      pnmflip -tb   pnmcut -h 620 -pad   pnmflip -tb \
      pnmpad -black -w 1024 -h 768 -b 620 -l 0 \
      pbmtoxbm )
  sleep 300
done

Mono/C♯ has this nice feature where arbitrary files can be linked into the final binary, and you can programmatically access them. I'd like to be able to do that in C too, I'm sure it is possible, I just don't know an easy way. I know that if you have a section foo, then ld will create __start_foo and __stop_foo symbols which point to the start and end of the section, so all I really want is an easy way to get ld to use the contents of an arbitrary file (say, ui.xml) as a section. Anybody know how to do this? Update: thanks to Daniel Jacobowitz for giving enough clues to a working, and clean, solution. I'll blog this shortly. NP: The Sound Of A Handshake, cLOUDDEAD