On Fri, 17 Mar 2000, Olaf Weber wrote:
>James Sutherland writes:
>> On 15 Mar 2000, Rask Ingemann Lambertsen wrote:
>
>>> Not at all. COW is a performance optimisation which does not depend on
>>> overcommitment of memory in any way. Why would you want to turn it off?
>
>> Because it *IS* overcommitment of memory. You can have two processes, each
>> with their 200Mb of data, in a machine with 256Mb RAM+swap, quite happily
>> - until they start writing to it, at which point you discover you have
>> overcommitted your memory, and things go wrong.
>
>You're conflating two things: the COW optimization and whether or not
>virtual memory is actually reserved. For example, in a system that
>doesn't overcommit, suppose you have process that forks: at that
>point, the kernel reserves enough pages of virtual memory to be able
>to give the new process unique pages if it needs them. COW means that
>those reserved pages are only pressed into service when they are
>actually written to.
>
>How many pages is enough? In the case of a fork, you only need to
>reserve pages for the writable pages of the old process. The
>read-only pages (the program text segment) can be shared (and have the
>binary as backing store to boot). On an exec, the kernel will of
>course reset the count of reserved pages to match the new executable.
>(And the exec could fail if it tries to start a new program that
>requires a larger data segment than available memory allows.)
FINALLY. One thing I thought of that would help reduce the effect
of fork/exec sequence on the reservation - If the fork only reserved
say 5-10 pages - then if the new process exceeded this reserve then
entire amount should be reserved. This would reduce the peak reservation
in the case of a fork followed by an exec. This delayed reservation
should give nearly the same peak as the current method, and protect
against the worst case.
>The thing about fork/exec is that the requirement for extra virtual
>memory when a large process forks a small program (emacs forks ls) is
>short-lived.
This is where the 5-10 page reservation would come in usefull.
>Read-only data is not a problem, so apart from fork/exec, how many
>cases are there where you have processes sharing large numbers of
>writable pages? Note that for overcommitment to actually "work" in
>those cases, those pages should hardly ever be written to: if they are
>all touched in the long run, then you do really need the extra memory,
>and reserving it now will prevent nasty surprises later. And if the
>pages are de-facto read-only, would it not be better if the
>application marked them as such before forking?
>
>
>I have some experience with the pros and cons of overcommitment on
>IRIX workstations, where you can specify how many pages the kernel is
>allowed to overcommit. When the system is stressed and overcommitment
>isn't allowed, the first sign is typically that you cannot print from
>netscape or something similarly irritating. When overcommitment is
>allowed, the first sign is processes dying at random, with the X
>server usually among the first to go. I don't overcommit at all.
I was getting inetd killed, and cron, followed by sendmail, sshd,
and init.
>If during normal work you get processes killed due to overcommitment,
>or unable to fork, exec, or malloc due to memory shortage, you need to
>either get more (virtual) memory or lessen the workload.
>
>One thing that irks me about the current discussion is the complete
>lack of data: I would be interesting to know how much additional VM a
>sane non-overcommitting regime requires when compared with the
>overcommitting case? It seems no-one actually knows.
It is also very difficult to get since there is no real resource
accounting available.
-------------------------------------------------------------------------
Jesse I Pollard, II
Email: pollard@cats-chateau.net
Any opinions expressed are solely my own.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Thu Mar 23 2000 - 21:00:22 EST