On Tue, 21 Mar 2000, James Sutherland wrote:
> On Tue, 21 Mar 2000 13:57:07 +0100 (CET), you wrote:
> >On Tue, 21 Mar 2000, James Sutherland wrote:
> >
> >[...[
> >> So? If you touch the memory, overcommit doesn't come into play.
> >
> >All processes should do that for it to work. Example (you have 128MB of swap):
> >
> >you malloc() 1MB of space, and touch every page on it. Does swap gets
^^^^^^^^^^^^^^^^^^^^^^^^^
> >allocated at this time? Assume it is (I'm not sure: the pages are dirty,
> >but get written on swap when needed, at later time).
> >
> >Another process malloc 128MB of space. It does not touch the pages,
> >overcommiting allows malloc() to succeed.
> >
> >RAM gets used up by some one else. Your pages get swapped out.
>
> No - you haven't allocated (touched) the pages yet. They are purely
> imaginary pages until then.
Please read again. I've already touched all the pages, as you suggested.
Maybe i'm not been clear:
1) process A malloc()s 1MB. It DOES touch them, right after malloc().
2) process B malloc()s 128MB. It does NOT touch all of them. Only some.
3) process C malloc()s 128MB. It does NOT touch all of them.
4) process C starts accessing its pages sequentially. RAM gets allocated.
5) RAM is short (suppose 64MB). the system starts paging-out A, B, C pages.
Here are you sure that A swap space is already allocated? I'm not. Anyway,
let's assume it is. As long as the systems pages out B and C pages,
it allocates swap space. Sooner or later we're short of swap, because
both B and C have address spaces larger than available swap (128MB - 1MB
taken by A).
6) A (who's been sleeping all the time) wakes up, accesses one of its pages.
No page-frame is available, so the system tries to free one used by C
or B. But no swap space is available. Page fault for process A fails
to provide a free page-frame to A. Boom.
> >Later, you access one page of yours. It's not in RAM. The system has to
> >fetch it from swap. First, it has to locate a free page-frame. There is none.
> >So, it has to swap-out some other page. But, wait, swap space is full.
> >What happens? It's your process causing the page fault.
> >
> >Even if your process has all swap space allocated, it's not enough.
> >Overcommiting for other processes still hits you.
>
> You seem to misunderstand. Basically, malloc() allocates ADDRESS
> SPACE, not memory. USING that address space then allocates VM (RAM or
> swap). Problems only arise if you haven't really allocated your memory
> yet, you only think you have (because you misunderstand the malloc()
> implementation.)
That's what *I*'ve been saying in previous messages. I'm not misunderstanding
malloc(). You are misunderstanding that your 'safe' process may still
page-fault. Even if its own pages are on swap, you may need to free
other processes pages, and you can't be sure *they* can be swapped out.
> If YOU touch memory when malloc()ed (i.e. you REALLY allocate memory,
> not just address space to put memory in) you're fine. Only the other
> processes can be hit.
No. Only if ALL processes touch ALL pages after malloc() you are
(partially) safe. They also need NOT to grow their stacks, for example.
>
> James.
>
.TM.
--
____/ ____/ /
/ / / Marco Colombo
___/ ___ / / Technical Manager
/ / / ESI s.r.l.
_____/ _____/ _/ Colombo@ESI.it
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Thu Mar 23 2000 - 21:00:33 EST