Torsten Landschoff: Hitting the dynamic linker wall
I was working on replacing some mockup code for testing an internal library with python. Basically, the C code is incredibly big and I would rather mock the 3rd party API we are using in python. I wrote a generator to create wrapping code that forwards the C API calls to my python module.
Now that most of the work is finished, I get the following error message from my python mock:
1 2 3 4 5 6 | Traceback (most recent call last): File "simple_mockup.py", line 2, in <module> import threading File "/usr/lib/python2.5/threading.py", line 11, in <module> from time import time as _time, sleep as _sleep ImportError: /usr/lib/python2.5/lib-dynload/time.so: undefined symbol: PyExc_ValueError |
- demo.c
-
1
2
3
4
5
6
7#include <Python.h>
void test_python()
Py_Initialize();
PyRun_SimpleString("import threading\n");
- main.c
-
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21#include <stdio.h>
#include <dlfcn.h>
int main(void)
const char *error;
void *handle = dlopen("./demo.so", RTLD_LAZY EXTRA_RTLD_FLAGS);
void (*test_python)();
*(void**) &test_python = dlsym(handle, "test_python");
if ((error = dlerror()))
fprintf(stderr, "%s\n", error);
return 1;
printf("Calling test_python @ %p in shared object @ %p.\n", test_python, handle);
(*test_python)();
dlclose(handle);
printf("Feels fine, finishing.\n");
return 0;
- run_it.sh
1 2 3 4 5 6 7 8 9 10 11 12 13 | #! /bin/sh gcc -shared python-config --includes -o demo.so demo.c python-config --ldflags echo "Running example with default RTLD flags:" gcc -DEXTRA_RTLD_FLAGS=0 -o main main.c -ldl ./main echo echo "Passing RTLD_GLOBAL in addition:" gcc -DEXTRA_RTLD_FLAGS=RTLD_GLOBAL -o main main.c -ldl ./main echo |
torsten@pulsar:~/sh_bug$ ./run_it.sh
Running example with default RTLD flags:
Calling test_python @ 0xb77084bc in shared object @ 0x96bc018.
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib/python2.5/threading.py", line 11, in <module>
from time import time as _time, sleep as _sleep
ImportError: /usr/lib/python2.5/lib-dynload/time.so: undefined symbol: PyExc_ValueError
Feels fine, finishing.
Passing RTLD_GLOBAL in addition:
Calling test_python @ 0xb783f4bc in shared object @ 0x95e1018.
Feels fine, finishing.
Sucky. So embedding Python into an application is easy but if you want to use the interpreter and its modules from a plugin, you are out of luck. Somebody knows a way around this problem? The only solution I can think of is to link libpython.so into each of the python plugins.
Update: Work around
Small update: For my current problem, the work around is to run the application with the python library preloaded: LD_PRELOAD=/usr/lib/libpython2.5.so.1.0 app. Back to adding functionality
This poem is only in German language, but I hope you can forgive me to run it in my English language feed nevertheless. I send you the best season greetings, have a nice time, use it well, relax and think about it. :)
Over the summer, there was a bit of a tussle at the highest level of
Ubuntu governance over whether or not 
My personal site s copy of this blog and some of my other sites moved servers last week. A few differences in Apache configuration between 1.3 on the old server and 2.2 on the new server tripped me up and it was Monday morning before some missing pages were back where they should be. By the time you read this, the dynamic pages should also be back. Sorry to anyone whose viewing was interrupted.
The old server was the last one built by 
He might too shy to tell everybody, but the world must know. Luca Capello successfully defended his PhD thesis today at Universit de Gen ve, getting a Tr s Bien mention.
I must admit that my last biology lesson was 13 years ago, so I m not sure I can really comment on his work on Vomeronasal Receptors: from Monogenic Expression to Axon Guidance. But I was very impressed by the quality of the experimental process, especially compared to what we do in computer science.
Debian was well represented at the defense, since Axel Beckert, Didier Raboud and the Debian kilt were also there.
Welcome to the list of Debian Developers holding PhD degrees, Luca!
The WebKit hackfest is now over, and I think it was a very productive week. Thank you very much to all who attended, to Igalia for organizing the hackfest, and hosting us so well, to Collabora for having sponsored the event, and allowed me to spend the week working on it, and to the GNOME foundation for having payed all of my costs!
The passwords are saved in the GNOME Keyring. It s interesting to point out that GNOME Keyring seems to be unhappy with the number of passwords a browser stores - Xan s daemon was hanging, crashing, and spawning a large number of threads. My daemon decided to take up some 300MB of RAM at one point. It s somewhat funny to see how much a browser pushes the limits of our platform. We are hoping this will improve with the new keyring APIs, and the rewrite that is ongoing. It s nice to see my browser form passwords in seahorse, though, and be able to manage them like any other.
One more thing worth of notice, although this post is already a bit too big: one of the main concerns people had during the Hackfest was on making build time smaller. Touching a single file in WebCore causes a debug build of 10 minutes on my laptop. Evan Martin and Benjamin Otte made a push at removing unnecessary includes from WebCore, and WebKitGTK+ files, which brough the build time down a bit. They end up inspiring Aroben, from Apple, to go even further into this, and remove many includes from files all over WebKit.
Evan was also able to bring linking time down by making it possible to link libwebkit without having to build all the intermediate libraries, which brought build time down to 1 minute, when touching a single file in WebCore. Behdad and I also started looking into breaking WebCore up into lots of shared libraries for Debug builds, since we don t care too much about speed penalties in those. None of these experiments got committed yet, but I am hoping we will be having a better time hacking on WebKitGTK+ in the near future.
It was awesome meeting everyone, by the way! See you around =).
For the first time, I guess, XCB has beat Xlib support! 
We're going to have a 
Just had some time in the last two days, and here's the list of uploaded NMUs. So far, I had only one maintainer complaining (again, sorry!), but I must admit that on Sunday I just didn't follow the correct NMU procedure 
Anyhow, here's what I have:
If you want to participate in this great game too, and improve our loved distribution, you can have a look at this list of 
