
I had cause to try and get a core dump from a segfaulting process at work the
other day, and I wanted to figure out if fiddling with
/etc/security/limits.conf was going to do the trick (I didn't) or if I
had to modify the initscript to include a
ulimit -c unlimited call.
Of course, on a modern system (>= 2.6.24), one would just take a look at
/proc/PID/limits and get on with life, but unfortunately the
system in question was running 2.6.18, so more fiddling around was required.
I'd found something once before that told me how to do it with GDB, but all I
could find this time around was a
rather
over-complicated Knol article, which made a bunch of assumptions (well
mainly that the binary in question wasn't stripped). So with some help from the
Knol article, I muddled through it.
Disclaimer: I don't profess to be an expert on system internals like this, so
if this eats your first-born, don't come crying to me.
Firstly, you need to know that it's the
getrlimit(2) system call that
you want to be using, and then you need to figure out the number for the
resource limit you want to retrieve. The man page for
getrlimit(),
tells you it's defined in
/usr/include/sys/resource.h, but I've found
that the actual useful bits end up being in
/usr/include/bits/resource.h
I wanted the resource limit for the maximum core dump size, which is
RLIMIT_CORE and has a value of 4.
Next, you need to know that the
getrlimit() system call takes an
integer and a pointer to an
rlimit structure as arguments. We've just
figured out the value for the integer, but we're also going to need to pass a
pointer as the second parameter. A pointer to enough memory to hold an
rlimit structure. Fortunately, the
rlimit structure is pretty
simple:
struct rlimit
rlim_t rlim_cur; /* Soft limit */
rlim_t rlim_max; /* Hard limit (ceiling for rlim_cur) */
;
After a bit of grepping around in
/usr/include, I determined that an
rlim_t is essentially an
unsigned long int, so we need to
allocate a pointer big enough to hold two of them.
Note that if we had an unstripped binary, we could have saved a lot of faffing
around by just going
print sizeof(struct rlimit)
in GDB (assuming that the binary has the
getrlimit symbol in it)
The sure-fire way of figuring out how much memory we need for this pointer is to go
print sizeof(unsigned long int)
in GDB, and then double that (since we want two of them). On my system an
unsigned long int is four bytes, so I'm going to want to allocate
enough memory for 8 bytes.
Now it's time to attach GDB to the offending process and see what the resource
limit currently is. (
gdb -p PID)
(gdb) print malloc(8)
$1 = 152186904
(gdb) print getrlimit(4, $1)
$2 = 0
(gdb) x/2xw $1
0x9123018: 0x00000000 0x7fffffff
(gdb) quit
So in this particular case, we've retrieved the soft and hard limits of the
RLIMIT_CORE resource limit, and you can see that the soft limit is zero, and
the hard limit is unlimited. Note that the
getrlimit() function
returns an integer as its return code, which is what the
$2 = 0 is
above.
Now it's just a case of altering the the resource limits via the preferred
mechanism, restarting the process and then repeating this GDB examination of
the process to check they were changed successfully.
Circumstances where this can all fall down would appear to be ones where the
getrlimit symbol isn't present in the binary, or the binary is compiled as a
position-independent executable. I'd think that in the latter case, the system
is probably modern enough to be running a kernel that supports directly
examining the resource limits via the
/proc filesystem.