[PATCH] kernel/fork.c: VM accounting bugfix (2.6.11-rc3-bk5)

[Mark F. Haigh]

[Aargh!  Missing Signed-off-by.]

Unless I'm missing something, in kernel/fork.c, dup_mmap():

			if (security_vm_enough_memory(len))
				goto fail_nomem;
/* ... */
fail_nomem:
	retval = -ENOMEM;
	vm_unacct_memory(charge);
/* ... */

If security_vm_enough_memory() fails there, then we vm_unacct_memory()
that we never accounted (if security_vm_enough_memory() fails, no memory
is accounted).

If it is in fact a bug, a simple but largely untested patch (against
2.6.11-rc3-bk5) is included.


Mark F. Haigh
Mark.Haigh@xxxxxxxxxxxxxx

Signed-off-by: Mark F. Haigh  <Mark.Haigh@xxxxxxxxxxxxxx>

--- linux-2.6.11-rc3-bk5/kernel/fork.c.orig	2005-02-08 19:12:26.254589504 -0800
+++ linux-2.6.11-rc3-bk5/kernel/fork.c	2005-02-08 19:16:30.756419576 -0800
@@ -193,8 +193,10 @@
 		charge = 0;
 		if (mpnt->vm_flags & VM_ACCOUNT) {
 			unsigned int len = (mpnt->vm_end - mpnt->vm_start) >> PAGE_SHIFT;
-			if (security_vm_enough_memory(len))
-				goto fail_nomem;
+			if (security_vm_enough_memory(len)) {
+				retval = -ENOMEM;
+				goto out;
+			}
 			charge = len;
 		}
 		tmp = kmem_cache_alloc(vm_area_cachep, SLAB_KERNEL);