Re: A true story of a crash.

Theodore Y. Ts'o (tytso@mit.edu)
Sun, 16 Aug 1998 18:04:49 -0400


Date: Sun, 16 Aug 1998 08:19:36 -0400 (EDT)
From: "Stephen M. Benoit" <benoits@servicepro.com>

Am I missing the point? I was under the impression that there is a
well-defined behavior when a Un*x reaches memory resource limits.
Additional requests for resources are simply denied: currently
executing processes don't get axed until they segfault, get signalled
or exit().

It's very simple --- it's called swap space overcommit. The issue here
is that with any modern VM system, it's possible to be very efficient
with how you use data pages. For example, when a process forks, you
don't need to copy all of its pages; instead you can mark them as "copy
on write", and only duplicate its pages if they are actually modified.

The problem with using stract allocation is that you will start
returning ENOMEM while there is still plenty of memory available, since
most of the time the maximum theoretical memory that might be used by a
process (if you have to allocate pages for all of itw COW pages, etc.)
is nil. For example, if you have a huge executable, but the way you use
the program, most of it never gets used, a strict allocation system
might not let you run the executable, even though there might be plenty
of space and no problems since the executable might not use all of its
text pages (especially true with shared libraries). Strict allocation
demands that you reserve a page for each processes's use of a text or
shared library memory page, since there's a chance it might get
duplicated because of debugger setting a break point (for example).

The problem with overcommit is that you don't necessary know when a user
space process calls malloc(), fork(), etc. whether or not you're going
to run into trouble, since the pages don't actually get allocated until
a page fault happens (either because the page needs to be paged in for
the first time, or because a COW page needs to be copied).

I've tried using a system with strict allocation, and it runs out of
memory *really* quickly. In fact, it usually runs out of memory so
quickly that it's frequently not very useful.

- Ted

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.altern.org/andrebalsa/doc/lkml-faq.html