On 21 Mar 2000, Olaf Weber wrote:
> Marco Colombo writes:
> > On 18 Mar 2000, Mirian Crzig Lennox wrote:
>
> >> Overcommitting memory is the moral equivalent of writing bad checks and
> >> praying there will be money to cover them before they are cashed. It's
> >> completely irresponsible-- and when it fails, it really bites down hard.
>
> > I must be missing something. How can you compare the two things?
> > brk() is just extending your virtual address space. Your VA space
> > has little to do with RAM or swap, just because is *virtual*. It just
> > means that 0xe0000000 is not a *real* address, just a translated ones...
>
> The debate is (in part) about whether brk(2) should just extend the
> address space of a process, or also arrange for virtual memory for
> that address range to be reserved at that point of time.
>
> At this point, the kernel would not need to decide which particular
> pages of memory or swap are to be allocated, just that the known
> reserve is descreased by the requested amount.
>
> The underlying assumption is that processes do indeed use the address
> space they request, which implies that the virtual memory would have
> to be made available anyway in the long run.
>
> > As someone already pointed out, i may need it just to implement some
> > sparse algorithm without having to handle a translation table and
> > a cache myself.
>
> Granted, but is this the norm or the exception? I believe the latter.
> Note that if you want to use this technique, you'd better _know_ that
> the system you're on overcommits, or you end up wasting gobs of
> memory. It is therefore not something a reasonably portable program
> can do.
>
> On ther other hand, most programs do not use sbrk(2) to obtain memory,
> they use malloc(3). The difference is that (following the C standard)
> there is a guarantee that if malloc succeeds, then the memory can be
> used. On a system that unconditionally overcommits, this guarantee
> doesn't hold.
Does the standard say something about paging? (I don't know, but i think no)
Paging is transparent, until you *time* what you're doing. Then you'll
see what the difference is between RAM and disk access time.
Isn't malloc() supposed to return "memory"? What is this thing that under
some some (uncontrollable, unpredictable) conditions this "memory" behave
just like disk? What does the C standard say about my process being
put to sleep in the middle of a for(;;); ? Or being interrupted
by ^C (while doing no terminal I/O: I've seen too many DOS programmers
implement this interrupt "feature" by having a program read stdin
- as soon as they discovered the way to do a nonblocking reads - in its
core loop, just to allow themselves stop runaway programs).
And who's this "root" who keeps killing my wonderful memory-hog fork-bomb
C exercise (a perfectly legal "C" program, it compiles just fine):
the Standard says nothing about fascist sysadms B-)
> To put it in a different way: if a program crashes because it
> dereferences a NULL returned by malloc, the program has a bug. If it
> crashes because on dereferencing a non-NULL pointer returned by malloc
> there is no virtual memory available, then the C runtime is at fault.
Put it another way: what did wrong my C program that received a nasty
SIGTERM (and as soon as I learned how to catch it, they sent me SIGKILL
- still studing they way to catch it...)? Of course, root just decided
it was shutdown time...
At shutdown time, program gets SIGTERMed and SIGKILLed, no matter if
the follow the C Standard or not. At OOS time, they get SIGKILLed.
What the C Standard has to do with that?
malloc() "allocates" memory only on its own opaque pool. It has no
OS meaning. On systems with no protected memory (yes, you compile C
programs on them) you can access every address you like. No need to
malloc. You may write you own heap management routine. libc provides one.
On system with memory protection, you can still write your own
heap management routine, which uses whatever system call to do its jobs.
malloc is a "standard" way to manage a heap. Just like the standard
buffered I/O functions. They are a way to use buffered I/O. Those I/O
buffers are all a user-land thing. On Linux, even write(2) is "buffered".
What malloc() does is to give you an address which is "valid" for a
following free(), and which is not overlapping with following malloc()s.
It's a general purpose "memory" allocator which is supposed to be:
1) bug - free;
2) efficient (with an average use).
But here "memory" is an abstraction. Not real RAM. Nor paged RAM+swap.
How this abstraction is mapped to system resources its up to the kernel.
It may give you real RAM. Or set the MMU up with permissions and handle
page faults.
Whether *a = 0; will cause an error or not it's all a OS matter, not a C
language issue. No matter what the value of "a" is. Even NULL is an
abstraction. It may be implemented being not zero (but in C i think
that NULL == 0 is always true no matter its real value, so that
if (node->next) {...} works).
What malloc() returns is a valid "C" address, and a valid OS virtual
address. When you access it, if the kernel cannot map it to a valid
object, because of resource shortage, you will get an OS "error", which
is not a "C" error. The kernel is allowed to post you a signal whenever
it thinks it should to. The C standard has nothing to say about it.
> In any case, it seems people are now mostly shooting the same
> arguments at each others. More constructive (but definitely more
> work) would be to instrument a kernel to actually keep track of memory
> in sufficient detail, and (for example) log messages when it is
You're missing that an overcommitting MM system is more advanced than
a nonovercommitting one. The kernel "keeps track of memory" the right way.
> overcommitting. This would give actual information about the costs
> and exactly how often overcommitment occurs in normal use. If only I
> had the time...
>
> --
> Olaf Weber
>
> Do not meddle in the affairs of sysadmins,
> for they are quick to anger and have no need for subtlety.
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.rutgers.edu
> Please read the FAQ at http://www.tux.org/lkml/
>
.TM.
--
____/ ____/ /
/ / / Marco Colombo
___/ ___ / / Technical Manager
/ / / ESI s.r.l.
_____/ _____/ _/ Colombo@ESI.it
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Thu Mar 23 2000 - 21:00:36 EST