Well, if someone implements a true 'vfork' for Linux, then this argument is
moot. Similarly, if overcommitting is on, this argument is moot. So there's
only an issue if you make those assumptions.
> _The main point I've been trying to make_ is that having *both* overcommit
> and no-overcommit behaviors available *simultaneously* is a GOOD THING.
I could not disagree more. If a person turns overcommitting off, he does so
because he doesn't want processes killed randomly for reasons beyond his
control.
> > Given these two assumptions, I believe it is disastrous to
> allow 'vfork' to
> > succeed if there aren't enough swap pages to back every single
> copy-on-write
> > page.
>
> So, with the second assumption "that overcommiting is turned
> off", it would
> be disastrous if overcommiting were turned on. I never would have thought
> of that. :-)
Exactly. If overcommitting is off, the system should not let itself get
into a case where dirtying a copy-on-write page requires it to kill a
process.
> Since you have a firm grasp of why overcommiting can be bad, I'll give you
> a real world example of why it can be good. In microelectronics, database
> sizes -- in RAM -- frequently exceed 1GB. Sometimes, they even
> exceed 4GB.
> (I've personally had to build special versions of Perl for
> processing these
> monsters.) Because of the expense, hosts processing these databases are
> only rarely configured with _more than twice the amount of RAM_
> it takes to
> process one of of these large databases at a time. Programs processing
> this data *frequently* run for more than 10 hours. Sometimes they run for
> several *days*. If this process starts paging because of lack of RAM, it
> can increase the run time by factors of 10. At the completion of
> processing, these programs sometimes call external programs. Frequently,
> external programs are a required part of printing the result. Sometimes,
> external programs are a required part of saving the result to disk in a
> useful format. Saving the result to disk in a useful format is usually
> _the entire purpose_ of running the program in the first place.
This is why you should either:
1) Turn overcomitting on, or
2) Configure the machine with lots of swap.
> popen(), ENOMEM, bummer! (Similar applies to Perl in additional
> contexts.)
Okay, that's bad. But if you turn overcommitting off and don't have enough
swap, that's your decision.
> Now imagine yourself as a Developer or Sys Admin, explaining to a
> room full
> of engineers and project managers why a machine that costs more money than
> you make in a year needs twice the huge amount of RAM it already has when
> they can log in, run "top", and observe that the total amount of RAM used
> never exceeds 90%, *hundreds* of MB RAM free, right up to the very end of
> processing. You can bet some of the engineers already know the external
> filter program never uses more than 12MB RAM, ever. Alternately, imagine
> explaining why you need to allocate 8GB of swap on a particularly fast and
> expensive disk -- when it will *never* be used -- by design. Imagine
> explaining why the project schedule just slipped the better part
> of a week.
I'd rather do that than explain why even though overcomitting is off, the
system still overcomitted, and had to kill an innocent process.
> While plotting world domination:-), please consider the many engineers who
> work in environments like these who would *love* to see Linux
> play a larger
> role.
Yes, that's one of the main reasons we have the ability to turn off
overcommitting.
DS
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/