Marco Colombo <marco@esi.it>:
> On Tue, 4 Apr 2000, Jesse Pollard wrote:
>
> > Marco Colombo wrote:
> > >
> > > On Mon, 3 Apr 2000, Linda Walsh wrote:
> > >
> > > > Marco Colombo wrote:
> > > > > It's not a matter of how I call it. The kernel gives you addresses, which
> > > > > you are able to use. You read at a given address, you get data. You
> > > > > write to an address, and you ask the kernel to allocate memory. That
> > > > > request may fail.
> > > > ---
> > > > This would be fine for an "address manager", but the Linux kernel has
> > > > something called a "memory manager". What kernel are you talking about?
> > >
> > > Nice game. Once more "names" are different from "objects".
> > > Whatever you name it brk() just modifies the address space. No memory
> > > management is performed. So it's not part of Linux mm. How it's
> > > implemented in other ancient OSes, I don't care...
> > > What kernel are *you* talking about?
> >
> > Actually, we are talking about posix. I know POSIX doesn't include brk -
> > it doesn't
> > to allow the OS (any OS) to define its own method of allocating memory.
> > That doesn't
> > eliminate the fact that malloc allocates memory for a process to use.
>
> Last time: malloc() allocates memory (from the malloc heap) in a C *program*
> A *process* knows nothing of malloc().
So .. If malloc (the binary) invokes brk (the binary) then memory is allocated
to the process ???? Make up your mind.
> [...]
> > > If there's no enough memory to *run* the process, it will fail.
> > > Why should the system consider the worst (uncommon) case and allocate
> > > the whole address space?
> >
> > It doesn't fail. the system killed itself.
> >
> > Irrelevent - malloc is the defined API, and the API is defined to return
> > null on allocation
> > failure. It doesn't do that in a consistant manner.
>
> brk() is the defined Linux API. Malloc() means nothing if you're not
> using C.
I can call malloc (the binary) from assembler if I feel like it. It isn't
C dependant. Fortran I/O uses the same allocation when it uses the runtime
library that opens files. malloc is used. It is the defined API. malloc
uses brk. Now just what are you smoking...
> > > > > Use another system call. Implement your own memory management.
> > > > > Or just show a single example of an application, *written following the docs*,
> > > > > that will fail.
> > > > ---
> > > > This is the point -- why use a different system call? Why not just
> > > > allow the end user to configure whether or not the Linux kernel does address management
> > > > or memory management? What is your problem with configurability?
> > >
> > > It should be done by the application. This is Linux programming, not
> > > system management.
> >
> > The application cannot depend on the system to supply resources.
>
> So write your own application with an OS in it and boot it out of LILO.
> "The application cannot depend on the system to supply resources" is
> just nonsense, BTW.
>
> [...]
> > > Anyway, my opinion doesn't count. Most of the main "large commercial apps"
> > > vendors are moving to Linux. This is a fact. Either directly or not,
> > > they're driven by technical reasons, since the Linux kernel has no
> > > marketing dept.
> >
> > Many of them are having to rewrite parts of linux to do so. You seem to
>
> Really? Why don't they post patches, so? I'm sure Linus will be glad to
> accept them, if they're valid.
>
> And BTW, why don't YOU post a non-overcommitting patch?
>
> > be stuck in
> > a single user environment. As long as you remain there you will not
> > recognize the need
> > for proper resource management.
> ^^^^^^^^^^^^^^^^^^^
> Already seen. Nothing do to with overcommit.
You really are blind.
> >
> > > > runs this way, deal with it" won't fly if someone requires no-overcommit. You can
> > >
> > > Only buggy applications require no-overcommit.
> >
> > Only buggy systems mandate overcommit. Besides- init was killed, (or
> > sendmail/syslog/some
> > random user/bash/...) Not the small program that used the last available
> > page.
>
> If you run your system in constant OOM conditions, you should either:
> 1) use Rik's patch;
> 2) avoid that anyway;
It just shows that you don't understand resources.
>
> > > > argue that they are wrong until you are blue in the face, but they won't care -- they
> > > > will just run NT or Solaris. The idea is to run with a kernel that performs resource
> > >
> > > And they will pay that choice. This is NOT a commercial vendor.
> > > This is the kernel development list. (Not speaking for others) I don't
> > > care a bit if someone else, in order to support badly written applications,
> > > uses another OS. They will pay that choice. Someone sooner or later
> > > will do the same job at half the costs (and the price) using Linux.
> > > I've seen this happen. I've seen companies not listening to my advices
> > > (at times when there was no hype around Linux) and choosing another,
> > > zero-management, OS, and later employing (full time) an administrator
> > > just to get a decent uptime. But this is OT.
> > >
> > > > management. Linux doesn't, as you have pointed out. It simply manages virtual address
> > > > page tables -- not real memory. Some people want a system to actually manage *memory*
> > > > not slots in a page table.
> > >
> > > Never said that. Your deliberately FUDing me (and Linux). I've never said
> > > there's no mm in Linux. I've just said that brk() does not allocate
> > > memory. The Linux kernel knows perfectly where RAM goes.
> >
> > So it is lying to the process ?
>
> The process asks the kernel to extend its address space with brk().
> The kernel does what the process asks.
> The process does not ask to allocate memory.
> The kernel does not allocate memory.
> So simple. It's the defined brk() behaviour. And AFAIK, doesn't break
> POSIX.
It does because malloc (the defined API) uses brk.
>
> > > > > brk() is the normal API. And it works as expected.
> > > > ---
> > > > Not if I come from a system that manages memory.
> > >
> > > You don't like your application shows its bugs when run on a modern OS?
> > > Should I care?
> >
> > A correctly functioning POSIX compliant program should work the same on
> > Linux as it
> > does on Solaris/Irix/UNICOS... Unfortunately they don't. Nothing wrong
> > with the program.
>
> Of course the program is wrong. It's using some non-POSIX implementation
> details of Solaris/Irix/UNICOS. It's implementation dependant. And NOT
> POSIX. So why should it not fail under Linux?
No it isn't. It matches all posix requirements. (after all it could be the
validation test for posix compliency... These other systems have passed, Linux
doesn't.)
> > the system keeps killing random or sometimes the wrong programs. The OOM
> > killer is only
> > a band-aid to hide the fact that the kernel has problems in a multiuser
> > environment.
>
> Multiuser? You keep confusing your user-level memory quotas with Linux
> ovecommitting.
Nope. This is resource control.
> > > > getting back a *usable* pointer. Please -- don't try to argue that users really
> > > > don't care if malloc returns a bogus pointer. It won't fly.
> > >
> > > Never said that.
> >
> > Sure you did. "malloc doesn't allocate memory..."
>
> "users really don't care if malloc returns a bogus pointer" - I have
> never said that.
>
> "malloc doesn't allocate memory..." - I've said: "brk() doesn't allocate
> memory...".
malloc can't allocate memory unless brk gives it.
Besides (and I give the full quote, apoligies for paraphrasing):
Date: Mon, 20 Mar 2000 19:16:26 +0100 (CET)
From: Marco Colombo <marco@esi.it>
To: Jesse Pollard <pollard@cats-chateau.net>
Cc: linux kernel <linux-kernel@vger.rutgers.edu>
Subject: Re: Some questions about linux kernel.
On Sun, 19 Mar 2000, Jesse Pollard wrote:
[...]
> NOPE - if the process requests it and is granted it, then it should have
> access to it. It is not up to the system to say "here it is, but don't use
> that part of it, because I really didn't give it".
malloc() does grant you what you (the process) asked for: it extends your
valid address space. It does not grant you any bit of RAM (page-frames
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
will be allocated when you access them, if available, or the process will be
put to sleep) or swap space (with overcommitting, it will be allocated when
the pages are paged-out). If you want the kernel to *grant* something,
you'll have to ask for it. You want RAM, not just VM, use mlock(). You don't
need RAM, but you need safe backing store, just create a file, fill it, and
mmap() it. If applications die because they treat malloc() as mlock(), it's
programmer's fault, not a kernel issue. malloc() it's just an interface to
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
brk(): it does not "allocate" anything (despite of the name) (kernel Gurus:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
maybe it allocates PTEs and other kernel resources, ok...).
So when you access that part of your address space, you should be prepared
in doing I/O instead of memory access (most applications are unaware of it,
just because they need not to know... but applications depending on
performances, such as benchmarks, WILL notice it), or even expect a failure.
Therefore you are saying that the pointer is not usable... and are arguing
that users don't care if malloc returns a bogus pointer.
-------------------------------------------------------------------------
Jesse I Pollard, II
Email: pollard@navo.hpc.mil
Any opinions expressed are solely my own.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Fri Apr 07 2000 - 21:00:11 EST