(this post was going to be about something else, but after getting
this far, I think it stands on its own as an introduction to dynamic
linking)
The shared library is an integral part of a modern system, but
often the mechanisms behind the implementation are less well
understood. There are, of course, many guides to this sort of thing.
Hopefully this adds another perspective that resonates with
someone.
Let's start at the beginning - relocations are
entries in binaries that are left to be filled in later -- at link
time by the toolchain linker or at runtime by the dynamic linker. A
relocation in a binary is a descriptor which essentially says
"determine the value of X, and put that value into the binary at
offset Y" each relocation has a specific type, defined
in the ABI documentation, which describes exactly how "determine the
value of" is actually determined.
Here's the simplest example:
$ cat a.c
extern int foo;
int function(void)
return foo;
$ gcc -c a.c
$ readelf --relocs ./a.o
Relocation section '.rel.text' at offset 0x2dc contains 1 entries:
Offset Info Type Sym.Value Sym. Name
00000004 00000801 R_386_32 00000000 foo
The value of
foo is not known at the time you make
a.o, so the compiler leaves behind a relocation (of
type
R_386_32) which is saying "in the final binary, patch
the value at offset 0x4 in this object file with the address of symbol
foo". If you take a look at the output, you can see at
offset 0x4 there are 4-bytes of zeros just waiting for a real
address:
$ objdump --disassemble ./a.o
./a.o: file format elf32-i386
Disassembly of section .text:
00000000 <function>:
0: 55 push %ebp
1: 89 e5 mov %esp,%ebp
3: a1 00 00 00 00 mov 0x0,%eax
8: 5d pop %ebp
9: c3 ret
That's link time; if you build another object file with a value of
foo and build that into a final executable, the relocation can go
away. But there is a whole bunch of stuff for a fully linked
executable or shared-library that just can't be resolved until
runtime. The major reason, as I shall try to explain, is
position-independent code (PIC). When you look at an
executable file, you'll notice it has a fixed load address
$ readelf --headers /bin/ls
[...]
ELF Header:
[...]
Entry point address: 0x8049bb0
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
[...]
LOAD 0x000000 0x08048000 0x08048000 0x16f88 0x16f88 R E 0x1000
LOAD 0x016f88 0x0805ff88 0x0805ff88 0x01543 0x01543 RW 0x1000
This is not position-independent. The code section (with
permissions
R E; i.e. read and execute) must be loaded at
virtual address
0x08048000, and the data section
(
RW) must be loaded above that at exactly
0x0805ff88.
This is fine for an executable, because each time you start a new
process (
fork and
exec) you have your own fresh
address space. Thus it is a considerable time saving to pre-calculate
addresses from and have them fixed in the final output (you can make
position-independent executables, but that's another story).
This is not fine for a shared library (
.so). The whole
point of a shared library is that applications pick-and-choose random
permutations of libraries to achieve what they want. If your shared
library is built to only work when loaded at one particular address
everything may be fine until another library comes along that
was built also using that address. The problem is actually somewhat
tractable you can just enumerate every single shared library
on the system and assign them all unique address ranges, ensuring that
whatever combinations of library are loaded they never overlap. This
is essentially what prelinking does (although that is a hint,
rather than a fixed, required address base). Apart from being a
maintenance nightmare, with 32-bit systems you rapidly start to run
out of address-space if you try to give every possible library a
unique location. Thus when you examine a shared library, they do not
specify a particular base address to be loaded at:
$ readelf --headers /lib/libc.so.6
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
[...]
LOAD 0x000000 0x00000000 0x00000000 0x236ac 0x236ac R E 0x1000
LOAD 0x023edc 0x00024edc 0x00024edc 0x0015c 0x001a4 RW 0x1000
Shared libraries also have a second goal code sharing. If
a hundred processes use a shared library, it makes no sense to have
100 copies of the code in memory taking up space. If the code is
completely read-only, and hence never, ever, modified, then every
process can share the same code. However, we have the constraint that
the shared library must still have a unqiue data instance in each
process. While it would be possible to put the library data anywhere
we want at runtime, this would require leaving behind relocations to
patch the code and inform it where to actually find the data
destroying the always read-only property of the code and thus
sharability. As you can see from the above headers, the solution is
that the read-write data section is always put at a known offset from
the code section of the library. This way, via the magic of
virtual-memory, every process sees its own data section but can share
the unmodified code. All that is needed to access data is some simple
maths; address of thing I want = my current address + known fixed
offset.
Well, simple maths is all relative! "My current address" may or
may not be easy to find. Consider the following:
$ cat test.c
static int foo = 100;
int function(void)
return foo;
$ gcc -fPIC -shared -o libtest.so test.c
So
foo will be in data, at a fixed offset from the code in
function, and all we need to do is find it! On amd64, this
is quite easy, check the disassembly:
000000000000056c <function>:
56c: 55 push %rbp
56d: 48 89 e5 mov %rsp,%rbp
570: 8b 05 b2 02 20 00 mov 0x2002b2(%rip),%eax # 200828 <foo>
576: 5d pop %rbp
This says "put the value at offset 0x2002b2 from the current
instruction pointer (
%rip) into
%eax. That's it
we know the data is at that fixed offset so we're done. i386,
on the other hand, doesn't have the ability to offset from the current
instruction pointer. Some trickery is required there:
0000040c <function>:
40c: 55 push %ebp
40d: 89 e5 mov %esp,%ebp
40f: e8 0e 00 00 00 call 422 <__i686.get_pc_thunk.cx>
414: 81 c1 5c 11 00 00 add $0x115c,%ecx
41a: 8b 81 18 00 00 00 mov 0x18(%ecx),%eax
420: 5d pop %ebp
421: c3 ret
00000422 <__i686.get_pc_thunk.cx>:
422: 8b 0c 24 mov (%esp),%ecx
425: c3 ret
The magic here is
__i686.get_pc_thunk.cx. The
architecture does not let us get the current instruction address, but
we can get a known fixed address the value
__i686.get_pc_thunk.cx pushes into
cx is the return
value from the
call, i.e in this case
0x414. Then
we can do the maths for the
add instruction;
0x115c +
0x414 = 0x1570, the final move goes
0x18 bytes past that
to
0x1588 ... checking the disassembly
00001588 <global>:
1588: 64 00 00 add %al,%fs:(%eax)
i.e., the value 100 in decimal, stored in the data section.
We are getting closer, but there are still some issues to deal
with. If a shared library can be loaded at any address, then how does
an executable, or other shared library, know how to access data or
call functions in it? We could, theoretically, load the library and
patch up any data references or calls into that library; however as
just described this would destroy code-sharability. As we know, all
problems can be solved with a layer of indirection, in this case
called global offset table or GOT.
Consider the following library:
$ cat test.c
extern int foo;
int function(void)
return foo;
$ gcc -shared -fPIC -o libtest.so test.c
Note this looks exactly like before, but in this case the foo is
extern; presumably provided by some other library. Let's
take a closer look at how this works, on amd64:
$ objdump --disassemble libtest.so
[...]
00000000000005ac <function>:
5ac: 55 push %rbp
5ad: 48 89 e5 mov %rsp,%rbp
5b0: 48 8b 05 71 02 20 00 mov 0x200271(%rip),%rax # 200828 <_DYNAMIC+0x1a0>
5b7: 8b 00 mov (%rax),%eax
5b9: 5d pop %rbp
5ba: c3 retq
$ readelf --sections libtest.so
Section Headers:
[Nr] Name Type Address Offset
Size EntSize Flags Link Info Align
[...]
[20] .got PROGBITS 0000000000200818 00000818
0000000000000020 0000000000000008 WA 0 0 8
$ readelf --relocs libtest.so
Relocation section '.rela.dyn' at offset 0x418 contains 5 entries:
Offset Info Type Sym. Value Sym. Name + Addend
[...]
000000200828 000400000006 R_X86_64_GLOB_DAT 0000000000000000 foo + 0
The disassembly shows that the value to be returned is loaded from
an offset of
0x200271 from the current
%rip;
i.e.
0x0200828. Looking at the section headers, we see that
this is part of the
.got section. When we examine the
relocations, we see a
R_X86_64_GLOB_DAT relocation that says
"find the value of symbol
foo and put it into address
0x200828.
So, when this library is loaded, the dynamic loader will examine
the relocation, go and find the value of
foo and patch the
.got entry as required. When it comes time for the code
loads to load that value, it will point to the right place and
everything just works; without having to modify any code values and
thus destroy code sharability.
This handles data, but what about function calls? The indirection
used here is called a procedure linkage table or PLT. Code
does not call an external function directly, but only via a PLT
stub. Let's examine this:
$ cat test.c
int foo(void);
int function(void)
return foo();
$ gcc -shared -fPIC -o libtest.so test.c
$ objdump --disassemble libtest.so
[...]
00000000000005bc <function>:
5bc: 55 push %rbp
5bd: 48 89 e5 mov %rsp,%rbp
5c0: e8 0b ff ff ff callq 4d0 <foo@plt>
5c5: 5d pop %rbp
$ objdump --disassemble-all libtest.so
00000000000004d0 <foo@plt>:
4d0: ff 25 82 03 20 00 jmpq *0x200382(%rip) # 200858 <_GLOBAL_OFFSET_TABLE_+0x18>
4d6: 68 00 00 00 00 pushq $0x0
4db: e9 e0 ff ff ff jmpq 4c0 <_init+0x18>
$ readelf --relocs libtest.so
Relocation section '.rela.plt' at offset 0x478 contains 2 entries:
Offset Info Type Sym. Value Sym. Name + Addend
000000200858 000400000007 R_X86_64_JUMP_SLO 0000000000000000 foo + 0
So, we see that function makes a call to code at
0x4d0.
Disassembling this, we see an interesting call, we jump to the value
stored in
0x200382 past the current
%rip
(i.e.
0x200858), which we can then see the relocation for
the symbol
foo.
It is interesting to keep following this through; let's look at the
initial value that is jumped to:
$ objdump --disassemble-all libtest.so
Disassembly of section .got.plt:
0000000000200840 <.got.plt>:
200840: 98 cwtl
200841: 06 (bad)
200842: 20 00 and %al,(%rax)
...
200858: d6 (bad)
200859: 04 00 add $0x0,%al
20085b: 00 00 add %al,(%rax)
20085d: 00 00 add %al,(%rax)
20085f: 00 e6 add %ah,%dh
200861: 04 00 add $0x0,%al
200863: 00 00 add %al,(%rax)
200865: 00 00 add %al,(%rax)
...
Unscrambling
0x200858 we see its initial value is
0x4d6 i.e. the next instruction! Which then pushes
the value
0 and jumps to
0x4c0. Looking at that
code we can see it pushes a value from the GOT, and then jumps to a
second value in the GOT:
00000000000004c0 <foo@plt-0x10>:
4c0: ff 35 82 03 20 00 pushq 0x200382(%rip) # 200848 <_GLOBAL_OFFSET_TABLE_+0x8>
4c6: ff 25 84 03 20 00 jmpq *0x200384(%rip) # 200850 <_GLOBAL_OFFSET_TABLE_+0x10>
4cc: 0f 1f 40 00 nopl 0x0(%rax)
What's going on here? What's actually happening is lazy
binding by convention when the dynamic linker loads a
library, it will put an identifier and resolution function into known
places in the GOT. Therefore, what happens is roughly this: on the
first call of a function, it falls through to call the default stub,
which loads the identifier and calls into the dynamic linker, which at
that point has enough information to figure out "hey, this
libtest.so is trying to find the function
foo". It
will go ahead and find it, and then patch the address into the GOT
such that the next time the original PLT entry is called, it
will load the actual address of the function, rather than the lookup
stub. Ingenious!
Out of this indirection falls another handy thing the
ability to modify the symbol binding order.
LD_PRELOAD, for
example, simply tells the dynamic loader it should insert a library as
first to be looked-up for symbols; therefore when the above binding
happens if the preloaded library declares a
foo, it will be
chosen over any other one provided.
In summary code should be read-only always, and to make it
so that you can still access data from other libraries and call
external functions these accesses are indirected through a GOT and PLT
which live at compile-time known offsets.
In a future post I'll discuss some of the security issues around
this implementation, but that post won't make sense unless I can refer
back to this one :)
I started doing daily builds of the Debian Installer
in February 2004. (Before me, Martin Sj gren did them for a while.)
Seven years of building d-i every single day, twice a day in recent years,
and also twice a day for armel. Well over ten thousand builds total. That
would have been tedious, but thanks to cron it was instead seven years of
keeping the machines running and upgraded, dealing with things when they
broke, and getting highlighted on IRC whenever someone mentioned
"people.debian.org/~joeyh/d-i". Still a bit tedious, but those bits were
used to install a lot of machines.
The rest of the d-i daily builds moved onto the autobuilders a while back,
but I guess there was some reluctance to mess with my institution. I
finally convinced them to take over my builds too. :)
http://d-i.debian.org/daily-images/
Long time no blogging.
Earlier, I wrote a blog post about 
Note that the syntax of the overrides should be backwards compatible, so unlike the 
I'm playing with btrfs to get a feel for what's coming up in linux filesystems. To be daring, i've configured a test machine using only btrfs for its on-disk filesystems. I really like some of the ideas put forward in the btrfs design. (i'm aware that btrfs is considered experimental-only at this point). I'm happy to report that despite several weeks of regular upgrade/churn from unstable and experimental, i have yet to see any data loss or other serious forms of failure. Unfortunately, i'm not impressed with the performance. The machine feels sluggish in this configuratiyon, compared to how i remember it running with previous non-btrfs installations. So i ran some benchmarks. The results don't look good for btrfs in its present incarnation. UPDATE: see the comments section for revised statistics from a quieter system, with the filesystems over the same partition (btrfs is still much slower). The simplified test system i'm running has Linux kernel 2.6.39-rc6-686-pae (from experimental), 1GiB of RAM (no swap), and a single 2GHz P4 CPU. It has one parallel ATA hard disk (WDC WD400EB-00CPF0), with two primary partitions (one btrfs and one ext3). The root filesystem is btrfs. The ext3 filesystem is mounted at /mnt I used 
It looks like I am going to Debconf11! Vacation time is booked, registration is in, only some final approvals inside Accenture are pending to pay my flights. I must say after working in Finland for the whole winter, I need to get out and get to friendly (and relatively chatty) Debian people more than ever.
Almost a year ago I put together 
Finally some outdoor morning photography from spring. Taken at fields near Hovo any.
Taken again using PENTAX K7 + PENTAX-DA 70mm F2.4 Limited.
I was amazed this morning that it's only been a month since I moved
house. So much has happened since.
For reasons that I won't go into, I've had to move house. Since I
like living in Mechelen, not too far from work, however, I decided to
search for something close by. And I did find something close by; I
moved to "just around the corner". Which is a bit silly, but then blame
the bureaucrats.
At any rate, I'm starting to feel at home in the new spot. It's a bit
smaller and a bit more expensive than the old place, but it looks a lot
better, and that more than makes up for it. And it's not like I needed
the extra space in the old place -- I kept having this big empty spot in
the middle of my living room, anyway.
To make matters more interesting, I've been on stage three times and
out of the country once in this past month: One wedding, two concerts,
and a work trip to Cork. I guess that's why it seemed like so much more
time.
But that's over now, and recently I finally managed to make some time
working on other projects. Today it was incorporating some patches
(which turned out not to work entirely) and doing a release for nbd.
That release was overdue by quite a bit; there were a few fairly serious
bugs outstanding that I had commits for in the git repository already, I
just had to get them out there. Oh well.
A few weeks back I've also managed to work a bit on 
As 
Yes, finally, I decided to upload my Debconf 10 pictures. Not to 
I ve been a little sick the last days and decided to be productive instead of just drinking tea all the time.The result are some (10) squashed bugs from pkg-games, and one long standing nmu.adonthell
Today we had a very interesting demonstration. People who know me for some time know I like to be a part of demonstrations, adding my voice to the people. And even though I know most demonstrations, as massive as they might be, have no real and tangible effect... Well, I try to be there.
Today we had a large demonstration in Mexico City of people angry with the government's tactics on the war on drugs Have you been to Mexico before 2007? You will remember we have always had our problems, but violence was basically limited to keeping an eye open, not flashing your money/jewels, and not looking too touristy. We were, all in all, a quiet country Known even as "the country where nothing happens, even if it happens".
Five years later, a good portion of the country lives really in war-like conditions, and close to 45,000 people have been killed. At some point about a year ago (where the body count was at ~22,000), our oh-so-bright de-facto president said that "only" 10% of them were civilians. If that is still the case, and if proportions are held, we can now talk about ~4,500 people killed just because they were there Oh, but remember the country is not at war, and death penalty has very long been abolished, so we cannot give the other 90% a different status. We have had over 45,000 killed people. That means at least ~200,000 torn lifes.
About six weeks ago, the cry from a father who lost his son started getting heard. And no, I do not want to talk about specific names or anything like that It is just the voice-bearer of many people who are fed up. Four days ago, he and a group of people left Cuernavaca city on a caravan and walked to Mexico city. Yesterday evening they arrived to the National University (UNAM), and were greeted by an amazing open-air concert: Mozart's Requiem.
Today, our day started very early. My mother wanted to help, as many more anonymous citizens did, so we started making and packing sandwiches and apples at ~5:30AM, delivered them to the camp where they were staying and grouping with other people, and came back home for our breakfast. Later, at around 9AM, as the demonstration started going by our house (we live a block from UNAM), my girlfriend, father and me started walking with them.
Why do I say this was a different demonstration than others I have been to?
After numerous contributions from many of you, the DebConf team has finally released the DebConf10 Final Report. It s a 46-page document which
gives the reader an idea about the conference as a whole. It includes descriptions of talks, DebCamp and Debian Day activities, personal impressions, attendee and budgeting numbers, the work of various teams, social events, funny pictures and so on. If you attended Debconf10, the report will refresh some of your remarkable memories and bring you closer to the organization team work. If not, it will certainly encourage you to be part of future Debian events :)
The report is available in two PDF sizes and can be downloaded from 
This should probably be an official FAQ, but a) I wanted to rant a bit more than is probably acceptable for something "official" and b) the sort of person this information is directed at never bloody reads 
