[PATCH] mmap PROT_NONE fix (was Re: serious performance regressiondue to NX patch)

[Daniel McNeil]

This patch fixes mmap PROT_NONE from elf binaries that do not
have the PT_GNU_STACK so that the do not have execute permission.

Before this fix, calling access() with a PROT_NONE page
was returning ENOENT instead of EFAULT.

BEFORE: 
$ x.gcc322
p2=0x40013000
pid=1982
 
access 0 ptr (nil) return code -1 errno 14
access result:: Bad address
access 1 ptr 0xffffffff return code -1 errno 14
access result:: Bad address
access 2 ptr 0x40013000 return code -1 errno 2
access result:: No such file or directory

AFTER:

$ ./x.gcc322
p2=0x40017000
pid=2492
                                                                                
access 0 ptr (nil) return code -1 errno 14
access result:: Bad address
access 1 ptr 0xffffffff return code -1 errno 14
access result:: Bad address
access 2 ptr 0x40017000 return code -1 errno 14
access result:: Bad address


Program available at
http://developer.osdl.org/daniel/mmap.PROT_NONE.bug/

Here is the patch.

Signed-off-by: Daniel McNeil <daniel@xxxxxxxx>


===== mm/mmap.c 1.133 vs edited =====
--- 1.133/mm/mmap.c	2004-06-29 07:43:12 -07:00
+++ edited/mm/mmap.c	2004-07-14 09:56:37 -07:00
@@ -792,6 +792,12 @@
 	vm_flags = calc_vm_prot_bits(prot) | calc_vm_flag_bits(flags) |
 			mm->def_flags | VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC;
 
+	/*
+	 * mm->def_flags might have VM_EXEC set, which PROT_NONE does NOT want.
+	 */
+	if (prot == PROT_NONE)
+		vm_flags &= ~VM_EXEC;
+
 	if (flags & MAP_LOCKED) {
 		if (!capable(CAP_IPC_LOCK))
 			return -EPERM;


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/