Andrea fixed the invocation problem, which also handles the reentrancy
problem in a clean way. It get's us rid of the ugly count, time,
whatever mechanisms in out_of_memory which was designed to cover the
invocation problem but was not able to prevent reentrancy and the
resulting overkill (kill a random amount of processes even if enough
memory is available).
I added the "Take child processes into account" modification for the
whom to kill selection on top of that and I was not able to make it
missbehave with my different test scenarios.
The patches are available in parts in this thread and the final combined
patch is there:
http://marc.theaimsgroup.com/?l=linux-kernel&m=110269783227867&w=2
2.6.10-rc3 contains a partial fix for the erroneous invocation problem,
but it is not as effective as Andrea's solution and it still runs into
overkill once the oom mechanism is invoked.
Andrea's fix and the selection changes should go into 2.6.10, but I
suspect that the VM gurus havent still reached a point, where they
agree. I also have the feeling that the problem is partially ignored.
Obviously has everybody plenty of memory in his boxes. </rant off>