flexible mmap layout and vdso mapping

[Daniel Forrest]

I am having a problem with memory allocation in a program when run
under Linux 2.6.18.

The program is statically linked and uses its own malloc library based
on brk().  Under previous versions of Linux it was able to allocate up
to ~3GB of memory before failing, now it fails at ~1GB.

The problem is that the vdso is mapped at 0x40000000 which limits the
heap to addresses less than 1GB.

I have tracked this down to the point that I see it is dependent on
the function arch_pick_mmap_layout(), in particular:

	if (!(2 & exec_shield) && (sysctl_legacy_va_layout ||
			(current->personality & ADDR_COMPAT_LAYOUT) ||
			current->signal->rlim[RLIMIT_STACK].rlim_cur == RLIM_INFINITY)) {
		mm->mmap_base = TASK_UNMAPPED_BASE;
		mm->get_unmapped_area = arch_get_unmapped_area;
		mm->unmap_area = arch_unmap_area;
	} else { ...


And the vdso is apparently mapped at "mm->mmap_base".


The program I have is a compiled binary and it is running through a
queueing system so modifying "/proc/sys/kernel/exec-shield" or
changing the stack limit are problematical.

Everything worked fine under Linux 2.6.9, except I notice in this case
that "cat /proc/sys/kernel/vdso" returns 0, whereas under 2.6.18 it
returns 1.


Are there any options on a per-program basis to disable vdso mapping?
Or any per-program options to change the location?  This queueing
system supports checkpointing and process migration, but that requires
that the binary not be changed so recompiling is not desirable.

-- 
Daniel K. Forrest	Laboratory for Molecular and
forrest@xxxxxxxxxxxxx	Computational Genomics
(608) 262 - 9479	University of Wisconsin, Madison
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/