Arithmetic overflow in may_expand_vm()
From: Earl Chew
Date: Thu Oct 15 2009 - 13:26:56 EST
This code currently reads:
int may_expand_vm(struct mm_struct *mm, unsigned long npages)
{
unsigned long cur = mm->total_vm; /* pages */
unsigned long lim;
lim = current->signal->rlim[RLIMIT_AS].rlim_cur >> PAGE_SHIFT;
if (cur + npages > lim)
return 0;
return 1;
}
If npages is stupendously large, the failure predicate may
return a false negative due to (cur + npages) overflowing and
wrapping.
I think it's more robustly written as:
if (npages > lim - cur)
return 0;
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/