Re: mmap'ed memory in core files ?

[Rob Mueller]

> > This is particularly interesting on a 64 bit kernel where a bug in
> > your code causes you to try to read something about 2Gb into your
> > alleged mmaped file (actual size ~500 bytes) and the segfault causes
> > a coredump.
>
> Do you have a ssimple example program for this?

Trying to reproduce the problem, I think it's actually related to sparse 
files.

$ cat a.c
#include <stdlib.h>
int main() {
 unsigned int i;
 char * a = malloc(3000000000u);
 *(int *)0 = 0;
}
$ gcc a.c
$ ulimit -c 10240 && ./a.out
$ ls -l
-rw------- 1 root root 3000082432 Jul  2 02:23 core.7761
$

It's clearly sparse, but slightly unintuitive that the ulimit doesn't 
actually limit the filesize, just the size of the data written to the file.

If I change the code to include this line after the malloc():

 for (i = 0; i < 3000000000u; i++) a[i] = i % 256;

I get:

-rw------- 1 root root   10485760 Jul  2 02:25 core.8992

More what you'd expect.

One interesting side effect of running a 64-bit kernel + 32-bit userland is 
that previously bugs that might have previously caused malloc() to fail (eg 
underflowing integer to a huge value), now succeed and allocate a huge chunk 
of memory rather than failing and causing the program to bailout/crash on 
dereference.

Rob

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/