Linda Walsh writes:
> Richard Gooch wrote:
> > Each of these options is flawed:
> >
> > 1&2) you have to do this for all processes
[bugger, you've removed the context]
> You have to assign process ID's, memory mappings, etc to every
> process. So what's your point? Having a default guaranteed stack
No, *I* don't have to allocate PIDs. The kernel does that for me.
To avoid overcommit, I have to limit the stacksize for all
processes. I either set a global value which is big enough for my
monster simulation, or I go and hand-tune a pile of exceptions. And
from time to time get a segfault because I set the limit too low.
Right now I can safely set the global limit to 128 MiB, and not expect
any process to exceed that. If we abolish overcommitting, I've got to
reserve an awful lot of swap space now. 128 MiB * nproc mounts up
*very* quickly.
> > 3) you can't handle a signal caused by stack exhaustion
>
> Why not? If the default action is to kill the process running out
> of stack. Again, no worse than current and it affects the current process,
> not some unrelated process.
That's why I said "handle". If I meant "killed by" I would have said
"killed by". For a stack exhaustion signal to be useful to my
application as a memory management tool (like you argue NULL return
from malloc(3) can be), I have to *handle* that signal and *do
something constructive*.
If the default (i.e. the only possibility) is to kill the process,
then we may as well admit defeat and just use the OOM killer. At least
the OOM killer is less likely to kill us (or anything), and even if it
does, it will do it later (hopefully after processing has finished).
> > 4) deadlock.
>
> Deadlock is a risk on any system. There are no guarantees
> that resource exhaustion won't lead to deadlock -- in fact, I'd call
> this a predictable failure. Deadlock is not guaranteed, however --
> processes *may* 'exit' or release memory. The behavior of the
> machine on lack of resources is a deterministic failure. Killing
> random processes is a non-deterministic failure.
Deadlock is not a risk on any system. It happens due to bugs in the
OS. It's no use to me if the application hangs (or worse, the system
hangs) rather than gets killed. It's a useless corpse in cyberspace
either way. You may as well argue the difference between process
"crash" or "emergency exit". Either way, it's dead, and it didn't get
a chance to save state.
> And with regards to algorithms to select processes to kill.
> From the perspective of any user or running process it *is* random
> -- since no user or process is going to exactly know the cpu time,
> run time, memory usage and memory priority of every other process
> is. It is, more precisely, pseudo-random. I maintain that random
> results of an OOM condition are worse than predictable results even
> if the predictable result group includes deadlock.
As a user, it's just as random to me if my application dies/hangs
because I ran out of stack space. I'm running a big job and it did
because it was greedy. The OOM killer will do the same thing, it just
reduces the chance of death.
> > That was 4 null options. I'd settle for one that could be shown to
> > work, or at least reasonably expected to.
>
> My intention is for the state of the system after an OOM
> failure to be as predictable as possible. Ideally, completely
> predictable. I would say all of those achieve that end. So what is
> your definition of "null" and what is your proposal for
> deterministic failure that maintains the integrity of the system?
By "null" I mean pseudo-"solutions" which aren't shown to be correct,
may be harmful (deadlock), and don't really solve the problem, they
just move it around a bit.
At least be honest about the solutions you're proposing and say that
they move the failure mode over a few steps, but don't fundamentally
change the nature of the problem: we can die when we run out of
memory.
I'm not proposing something that will give "deterministic failure" (in
your terms). I'm not even suggesting it's a tractable problem. It may
not be (it looks that way at the moment), and we should at least be
prepared to admit that.
If you can come up with a programming model such that stack exhaustion
results in error codes that can be handled by the application, rather
than killing/hanging, then you'll have made an important first step,
and maybe then the word "deterministic" can be used with meaning. But
then, we're not talking POSIX anymore.
> > Because there isn't always syscall interface for getting memory.
>
> But if you apply user and/or programmer defaults for stack,
> there is for for every other access. Thus you have a predictable
> result for both stack and other memory access.
I don't think you can reasonably say that stack requirements can be
predicted. Who knows what's happening down there in the C library?
> > It's better not to claim something at all than claim it
> > and people find out later it's not true.
>
> How is the above scheme untrue in any of it's claims?
It should be clear.
Regards,
Richard....
Permanent: rgooch@atnf.csiro.au
Current: rgooch@ras.ucalgary.ca
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Fri Mar 31 2000 - 21:00:19 EST