Re: Avoiding OOM on overcommit...?

From: Olaf Weber (olaf@infovore.xs4all.nl)
Date: Sun Mar 26 2000 - 13:35:28 EST


Khimenko Victor writes:
> Olaf Weber (olaf@infovore.xs4all.nl) wrote:
>> Sandy Harris writes:

>>> Earlier in this thread, though, someone suggested replacing malloc()
>>> with calloc() to ensure that space actually gets allocated. That should
>>> not be necessary.

>> Nor should it work, because the implementation of calloc relies on the
>> kernel zeroing new pages, and therefore won't touch new pages.

> It WORKS. Not perfectly but works. If calloc returned something it's your
> memory but it can just kill program... For buffer prealloction in deamons
> it should be enough though.

You're right. I'd forgotten that calloc on linux is special in that
sense nowadays. It is a useful distinction.

This seems as good a place as any, so...

>From the point of view of an application, it does not matter much
whether the system as a whole is OOM or the user memory quota limit
has been reached. In both cases a well-behaved application will need
to cope in the least destructive manner possible, which often means
"save and exit". In both cases, it would probably be better if an
application that would otherwise drive the system/user into OOM would
not have started at all.

When dealing with systems that do not allow overcommitting memory,
there is a problem with applications are those that allocate far more
address space than memory, because they cause needless failures,
either of themselves, or of other processes.

For the fork/exec case of a "large" application, this can probably be
fixed using a suitable clone/exec.

For other applications, one possible solution would be to split
"allocate address space" (AS) and "allocate pages in address space"
(AM). On many systems, AS is done with brk/sbrk and mmap. AM is
often either a side effect of using memory, or is done unconditionally
as a part of AS.

If the calls were separate, then you could program an application with
sparse array by grabbing a large address space, and allocate bunches
of pages on an at need basis. Even an overcommitting system could
make use of this information: if the application asks for 100 pages
when the system/user has only 50 available, fail the request before
the system is completely unable to page or grow a stack.

-- 
Olaf Weber

Do not meddle in the affairs of sysadmins, for they are quick to anger and have no need for subtlety.

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Fri Mar 31 2000 - 21:00:17 EST