On Tue, 21 Mar 2000, James Sutherland wrote:
[...]
> >1) process A malloc()s 1MB. It DOES touch them, right after malloc().
> >
> >2) process B malloc()s 128MB. It does NOT touch all of them. Only some.
> >
> >3) process C malloc()s 128MB. It does NOT touch all of them.
> >
> >4) process C starts accessing its pages sequentially. RAM gets allocated.
> >
> >5) RAM is short (suppose 64MB). the system starts paging-out A, B, C pages.
> > Here are you sure that A swap space is already allocated? I'm not. Anyway,
> > let's assume it is. As long as the systems pages out B and C pages,
> > it allocates swap space. Sooner or later we're short of swap, because
> > both B and C have address spaces larger than available swap (128MB - 1MB
> > taken by A).
> >
> >6) A (who's been sleeping all the time) wakes up, accesses one of its pages.
> > No page-frame is available, so the system tries to free one used by C
> > or B. But no swap space is available. Page fault for process A fails
> > to provide a free page-frame to A. Boom.
>
> That's a fault in the virtual memory system in Linux, if that happens.
> Nothing to do with memory allocation, the malloc() implementation or
> overcommit - it's a broken VM system.
Why? Please explain what should happen instead. Process A is faulting,
and so requesting for RAM. No RAM is available and no RAM can be freed
because swap is full. The swap space process A "owns" is not there
to fulfill its memory requirements, is there to honor *other* processes
requests. When process B requests a page-frame, the kernel may select one
of A pages as a victim. A pages get paged-out to free RAM in order to give
it to B. The same can happen with A and B swapped. But it's A request for
RAM when no RAM and no swap is available that triggers the problem.
So A gets hit.
I'm 100% in favor of overcommitting. But touching every page at malloc()
time won't save you from causing OOS. I'll have to use your own "swapping",
mlock() you own RAM, and handle your very own OOM conditions.
[...]
> That's nothing to do with memory allocation - that's a faulty VM
> subsystem. A badly broken implementation, if that can happen.
>
> >> If YOU touch memory when malloc()ed (i.e. you REALLY allocate memory,
> >> not just address space to put memory in) you're fine. Only the other
> >> processes can be hit.
> >
> >No. Only if ALL processes touch ALL pages after malloc() you are
> >(partially) safe. They also need NOT to grow their stacks, for example.
>
> OK, disabling overcommit is difficult, and would break all your
> existing code. You still haven't shown anything it ever improves,
> though.
Nothing. I'm against non-overcommiting. It's just that touching every page
isn't a "safe" malloc().
>
>
> James.
>
.TM.
--
____/ ____/ /
/ / / Marco Colombo
___/ ___ / / Technical Manager
/ / / ESI s.r.l.
_____/ _____/ _/ Colombo@ESI.it
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Thu Mar 23 2000 - 21:00:35 EST