When building a user space binary, the
-lc that gcc
inserts into the final link seems pretty straight forward link
the C library. As with all things system-library related there is
more to investigate.
If you look at
/usr/lib/libc.so; the "library" that gets
linked when you specify
-lc, it is not a library as such, but
a link script which specifies the libraries to link, which also
includes the dynamic linker itself:
$ cat /usr/lib/libc.so
/* GNU ld script
Use the shared library, but some functions are only in
the static library, so try that secondarily. */
OUTPUT_FORMAT(elf64-x86-64)
GROUP ( /lib/libc.so.6 /usr/lib/libc_nonshared.a AS_NEEDED ( /lib/ld-linux-x86-64.so.2 ) )
On
very
old glibc's the
AS_NEEDED didn't appear; so every binary
really did have a
DT_NEEDED entry for the dynamic linker
itself. This can be somewhat confusing if you're ever doing forensics
on an old binary which seems to have these entries for not apparent
reason. However, we can see that that
/lib/libc.so itself
does actually require symbols from the dynamic linker:
$ readelf --dynamic /lib/libc.so.6 grep NEEDED
0x0000000000000001 (NEEDED) Shared library: [ld-linux-x86-64.so.2]
Although you're unlikely to encounter it, a broken toolchain can
cause havoc if it gets the link order wrong here, because the glibc
dynamic linker defines
minimal
versions of
malloc and friends -- you've got a
chicken-and-egg problem using libc's
malloc before you've
loaded it! You can simulate this havoc with something like:
$ cat foo.c
#include <stdio.h>
#include <syslog.h>
int main(void)
syslog(LOG_DEBUG, "hello, world!");
return 0;
$ cat libbroken.so
GROUP ( /lib/ld-linux-x86-64.so.2 )
$ gcc -o -Wall -Wl,-rpath=. -L. -lbroken -g -o foo foo.c
$ ./foo
Inconsistency detected by ld.so: dl-minimal.c: 138: realloc: Assertion ptr == alloc_last_block' failed!
Depending on various versions of things, you might see that assert
or possibly just strange, corrupt output in your logs as
syslog calls the wrong
malloc. You could debug
something like this by asking the dynamic linker to show you its
bindings as it resolves them:
$ LD_DEBUG_OUTPUT=foo.txt LD_DEBUG=bindings ./foo
Inconsistency detected by ld.so: dl-minimal.c: 138: realloc: Assertion ptr == alloc_last_block' failed!
$ cat foo.txt.11360 grep "\ malloc'"
11360: binding file /lib/libc.so.6 [0] to /lib64/ld-linux-x86-64.so.2 [0]: normal symbol malloc' [GLIBC_2.2.5]
11360: binding file /lib64/ld-linux-x86-64.so.2 [0] to /lib64/ld-linux-x86-64.so.2 [0]: normal symbol malloc' [GLIBC_2.2.5]
11360: binding file /lib/libc.so.6 [0] to /lib64/ld-linux-x86-64.so.2 [0]: normal symbol malloc' [GLIBC_2.2.5]
Above, because the dynamic loader comes first in the link order,
libc.so.6's
malloc has bound to the minimal
implementation it provides, rather the full-featured one it provides
internally.
As an aside, AFAICT, there is really only one reason why a normal
library will link against the dynamic loader -- for the thread-local
storage support function
__tls_get_addr. You can try this
yourself:
$ cat tls.c
char __thread *foo;
char* moo(void)
return foo;
$ gcc -fPIC -o libtls.so -shared tls.c
$ readelf -d ./libtls.so grep NEED
0x00000001 (NEEDED) Shared library: [libc.so.6]
0x00000001 (NEEDED) Shared library: [ld-linux.so.2]
0x6ffffffe (VERNEED) 0x314
0x6fffffff (VERNEEDNUM) 2
Thread-local storage is worthy of a
book of its own,
but the gist is that this support function says "hey, give me an
address of
foo in
libtls.so", the magic being that
if the current thread has never accessed
foo then it may not
actually have any storage for
foo yet, so the dynamic linker
can allocate some memory for it lazily and then return the right
thing. Otherwise, every thread that started would need to reserve
room for
foo "just in case", even if it never cares about
moo.
But looking a little closer at the symbols of
libc.so is
also interesting.
libc.so doesn't actually have many
functions you can override. You can see what it is possible to
override by checking the relocations against the procdure-lookup table
(PLT).
Relocation section '.rela.plt' at offset 0x1e770 contains 8 entries:
Offset Info Type Sym. Value Sym. Name + Addend
00000035b000 084600000007 R_X86_64_JUMP_SLO 00000000000a2100 sysconf + 0
00000035b008 02e000000007 R_X86_64_JUMP_SLO 0000000000075e50 calloc + 0
00000035b010 01dd00000007 R_X86_64_JUMP_SLO 0000000000077910 realloc + 0
00000035b018 029300000007 R_X86_64_JUMP_SLO 00000000000661c0 feof + 0
00000035b020 046f00000007 R_X86_64_JUMP_SLO 00000000000768c0 malloc + 0
00000035b028 000400000007 R_X86_64_JUMP_SLO 0000000000000000 __tls_get_addr + 0
00000035b030 01b400000007 R_X86_64_JUMP_SLO 0000000000076dd0 memalign + 0
00000035b038 086000000007 R_X86_64_JUMP_SLO 00000000000767e0 free + 0
i.e. instead of jumping directly to the
malloc defined in
the
libc code section, any internal calls will jump to this
stub which, the first time, asks the dynamic linker to go out and find
the address of
malloc (it then saves it, so the second time
the stub just jumps to the saved location).
This is an interesting list, seemingly without much order.
feof, for example, stands out as worth checking out a bit
closer why would that be there when
fopen isn't, say?
We can track down where it comes from with a bit of detective work;
knowing that the value of the symbol
feof will be placed into
0x35b018 we can disassemble
libc.so to see that this
address is used by the
feof PLT stub at
0x1e870
(luckily,
objdump has done the math to offest from the
rip for us; i.e.
0x1e876 + 0x33c7a2 = 0x35b018)
000000000001e870 <feof@plt>:
1e870: ff 25 a2 c7 33 00 jmpq *0x33c7a2(%rip) # 35b018 <_IO_file_jumps+0xb18>
1e876: 68 03 00 00 00 pushq $0x3
1e87b: e9 b0 ff ff ff jmpq 1e830 <h_errno+0x1e7dc>
From there we can search for anyone jumping to that address, and
find out the caller:
$ objdump --disassemble-all /lib/libc.so.6 grep 1e870
000000000001e870 <feof@plt>:
1e870: ff 25 a2 c7 33 00 jmpq *0x33c7a2(%rip) # 35b018 <_IO_file_jumps+0xb18>
f19f7: e8 74 ce f2 ff callq 1e870 <feof@plt>
$ addr2line -e /lib/libc.so.6 f19f7
/home/aurel32/eglibc/eglibc-2.11.2/sunrpc/bindrsvprt.c:70
Which turns out to be part of a
local
patch which probably gets things a little wrong, as described
below. The
sysconf relocation is from a similar add-on patch
(being used to find the page size, it seems).
libc, like all sensible libraries, uses the
hidden
attribute on symbols to restrict what is exported by the library.
The benefit of this is that when the linker knows you are referencing
a hidden symbol it knows that the value can never be overridden, and
thus does not need to emit extra code to do indirection just in case
you ever wish to redirect the symbols. In the above, it appears that
feof has never been marked as hidden, probably because no
internal glibc functions used it until that add-on patch, and since it
is not considered an internal function the linker must allow for the
possibility that it will be overridden at run time and provide a PLT
slot for it. There are consequences; if this was on a fast-path then
the extra jumps required to go via the PLT may matter for performance
and it may also cause strange behaviour if somebody had preloaded
something that took over
feof.
Note, this is different from saying that your library can
override symbols provided by
libc.so; such as when you
LD_PRELOAD a library to wrap an
open call. What you
can not override is the
open call that say, the internal
libc.so function
getusershell does to read
/etc/shells.
Having the
malloc related calls as preemptable seems
intentional and sane; although I can not find a comment to the effect
of "we deliberately leave these like this so that users may use
alternative malloc implementations", it makes sense so that
libc.so is internally using the same malloc as everything
else if you choose to use something such as
tc-malloc.
tl;dr? Digging into your system libraries is always interesting.
Be careful with your link order when creating toolchains, and be
careful about symbol visibility when you're working with
libraries.
This June, soon after my seventh anniversary with Canonical, I will finish my job as Ubuntu CTO at Canonical. You can read my official announcement on the Canonical blog.
It s sometimes hard to believe that it s been that long since we first envisioned a Linux desktop for human beings . When I look at how far we ve come, at the difference we ve made to so many people, it s easy to see how the time passed so quickly.
It s been an incredible experience for me to play a part in building Ubuntu and Canonical. Whether building a decentralized company of hundreds of people, to a global community project of tens of thousands of members, I m grateful for all of the learning opportunities along the way. It has been a privilege to work with so many brilliant, passionate and thoughtful people under such auspicious circumstances. There is much I will miss, and I have many memories to enjoy, from all-night global hacking sessions driving toward a ship date, to casual singing and playing music at our many face-to-face events
Nonetheless, it is time for me to seek out new challenges and stretch myself in new ways. I ll be moving back to the US, closer to old friends and family, and starting a new job with a different type of company. I am leaving behind a capable and dedicated team at Canonical, who I am confident will achieve even greater things in the years to come.
I will remain active in the free software community as a volunteer. I intend to continue to participate in Ubuntu, and to serve on the Technical Board. I will also be continuing my work with Debian and the DEX project. This year, I ve accepted advisory positions with the Ada Initiative and the Freedom Box Foundation, and will continue to support those organizations and their missions.
I ll be in Budapest at the Ubuntu Developer Summit for the next week, and look forward to seeing my friends in Canonical and the Ubuntu community.
I ll have more to say soon about what I ll be doing next professionally. Watch this space for updates!
I got in early this time and started working on this post a day early.
Well, we've made it. We've all managed to survive parenting to the 1 year
mark. Zoe continues to do fabulously. Today she had her 12 month
paediatrician appointment, and weighed in at 19 pounds and 8 ounces (8.8 kg),
which has brought her up from the 25th percentile for weight to around the
50th percentile. She's also up from the 25th to the 50th percentile for
height. She got four shots today, which she was singularly unimpressed
about even before she was jabbed.
Somewhat surprisingly (to us) Zoe still isn't walking. She'll stand on her
own if you distract her with something, but otherwise she'll fall down and
prefers to crawl everywhere. She'll cruise around with the help of
furniture, and she'll happily walk while pushing something. Go figure. I
really thought she'd be walking into her 12 month appointment at the
paediatrician.
Developmentally, she's waving and pointing now. I'll usually get a wave
goodbye out of her when I head out the door to work in the mornings. She
seems to be picking up things pretty quickly now. One of her traits has been
she'd always toss her sippy cup on the floor when she was done taking a
drink from it. She's been doing this for months. In one sitting, I taught
her to hold it out to me and say "ta!" instead, which is great, because now
at mealtimes, I'm not picking up the sippy cup off the floor a dozen times.
We had another trip back to Australia last month, and this one was purely
vacation, so we had a nice couple of weeks of pottering around with family.
Zoe again handled the flights very well, although the jet lag upon returning
to the US seemed a lot worse than I remembered after our first trip. The
pilot was pretty punchy with the seat belt sign on the way to Australia, so
we had to keep taking Zoe out of the bassinet while she was asleep.
Incredibly we were able to do this about four times before she woke up.
She had a bit of a cold before we left for Australia, and that turned into a
nasty cough, which was diagnosed as bronchitis the first week we were there.
We got some antibiotics for her, and that cleared the cough up. She also had
a couple of spontaneous vomiting incidents, which were out of character for
her (but Sarah contracted some kind of tummy bug herself, so it wasn't
entirely surprising). Despite these health issues, she was still in
excellent spirits the whole time, and I think she thoroughly enjoyed
herself.
We went to 
Yay!
Mostly everything in life boils down to the same troublesome issue: people are reading different things into what they read, and interpret them regularly in a way it wasn't meant to. It seems that in certain areas a culture of interpreting things in a bad way instead of good or asking how they were actually meant has established the rules of (not) working together but rather against each other and around each other. At times I would like to account it to language barriers, or cultural differences, but it happens with people from all areas so that reasoning would be too easy.
Even artists manage to do that, and in that certain area it creates something extremely creative and thoughtful. This blog entry thus contains three songs and six videos: Two different interpretations of the same lyrics. Maybe this is able to stir some thinking process whether the interpretation that one found for a given situation might be biased or even just looking from the wrong angle.
Since this has been a major request from users for a long time, I can only cool with the idea of seeing the Debian project support a rolling release. However I m not pleased with the 
This is my monthly summary of my Debian related activities. If you re among the people who 
GNOME 3 packaging
Right after the GNOME 3 release, I was eager to try it out so I 
I ve been running this code for more than 2 months and it works fine. I want to see it merged. I m ready to update my code should anything need to be changed to please Guillem. But without any feedback we re in a deadlock.
Misc dpkg work
While fixing a bug in update-alternatives (found in one of the valid reports on launchpad), I noticed that there was room for improvements in the error messages output by update-alternatives. I changed them to reuse the same strings that were already used in other parts of dpkg. The result is that there are a few strings less to translate (always a nice thing for the poor translators who have to deal with the thousands of strings that dpkg contains).
I also tried to fix some of the most cryptic error messages in dpkg (see 
Here is my attempt at a summary of the rolling discussion currently happening on debian-devel@. It might not be complete, it s probably a bit biased, but I hope that it s still better than nothing. It was also posted on 
This week I have a shot I took while visiting Beeston Castle. This is the start of the path leading up to the castle itself.
Branches are marked with green rectangles, and tags with yellow arrows.
What we have here (expected given our configuration of the tool) are
branches (e.g. 
Branches that were not merged using svn merge look like they were not
merged at all.
Before being in SVN, the repository was stored in CVS. When it was imported
into SVN, no special attention was given to the commit author. Hence I
got credited for changes I did not write.
The tool leaves all branches, including removed ones (with tag on their end)
so that you can decide what to do with them.
The branch 
This week, I finally switched to use pytrainer to track my running data.
Up to now, I was still using Garmin proprietary software on
Elizabeth's Windows copmuter for this. Mostly because my GPW watch is
a Forerunner 405, that uses a bloody crappy stupid USB stick with
proprietary communication protocol (ANT+).
So, the use of proprietary crappy software was kinda mandatory. There
is however some code floating around to handle this device in Linux,
named "Gant". This is far from being very polished, but I finally
managed to make it work more or less reliably to get the GPS traces
from my watch to my Debian laptop.
So, during the week , I brought all my old running data back from
"Garmin Training Center" (the Windows software provided by Garmin)
back to my laptop and imported it in pytrainer. And I imported new
data from my daily runs with help of that "gant" software.
So, this is now time to really jump into maintenance of pytrainer and
help my friend No l (that will be easy as pytrainer is kinda well
maintained), then package "gant" for Debian, as part of the work in
the "pkg-running" maintenance team (Ralf Treinen, No le K the and /me).
New challenge for me: learn how to create a git public repo and
maintain a package in it (I already participate to maintenance work
with git for several things, but I never initiated any).
Other plans: some bug triage on pytrainer (if some is needed, I indeed
don't know yet) and eventually deal with things that have to be
reported upstream. Looking at Launchpad might also be a good idea as I
suspect that many are using pytrainer with Ubuntu.