a.out issue

[Florian Heinz]

Hi ppl,

there seems to be a bug related to a.out-binfmt.

try executing this binary:
perl -e'print"\x07\x01".("\x00"x13)."\xc0".("\x00"x16)'>eout
(it may be neccessary to turn memory overcommit on before)

This should result in a kernel-oops.
Doing this in a loop will eat fd's and memory.

seems like find_vma_prepare does not what insert_vm_struct expects when
the whole addresspace is occupied.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/