Re: Avoiding *mandatory* overcommit...

From: Marco Colombo (marco@esi.it)
Date: Tue Apr 04 2000 - 07:15:45 EST


On Tue, 4 Apr 2000, Jesse Pollard wrote:

> Marco Colombo wrote:
> >
> > On Mon, 3 Apr 2000, Linda Walsh wrote:
> >
> > > Marco Colombo wrote:
> > > > It's not a matter of how I call it. The kernel gives you addresses, which
> > > > you are able to use. You read at a given address, you get data. You
> > > > write to an address, and you ask the kernel to allocate memory. That
> > > > request may fail.
> > > ---
> > > This would be fine for an "address manager", but the Linux kernel has
> > > something called a "memory manager". What kernel are you talking about?
> >
> > Nice game. Once more "names" are different from "objects".
> > Whatever you name it brk() just modifies the address space. No memory
> > management is performed. So it's not part of Linux mm. How it's
> > implemented in other ancient OSes, I don't care...
> > What kernel are *you* talking about?
>
> Actually, we are talking about posix. I know POSIX doesn't include brk -
> it doesn't
> to allow the OS (any OS) to define its own method of allocating memory.
> That doesn't
> eliminate the fact that malloc allocates memory for a process to use.

Last time: malloc() allocates memory (from the malloc heap) in a C *program*
A *process* knows nothing of malloc().

[...]
> > If there's no enough memory to *run* the process, it will fail.
> > Why should the system consider the worst (uncommon) case and allocate
> > the whole address space?
>
> It doesn't fail. the system killed itself.
>
> Irrelevent - malloc is the defined API, and the API is defined to return
> null on allocation
> failure. It doesn't do that in a consistant manner.

brk() is the defined Linux API. Malloc() means nothing if you're not
using C.

> > > > Use another system call. Implement your own memory management.
> > > > Or just show a single example of an application, *written following the docs*,
> > > > that will fail.
> > > ---
> > > This is the point -- why use a different system call? Why not just
> > > allow the end user to configure whether or not the Linux kernel does address management
> > > or memory management? What is your problem with configurability?
> >
> > It should be done by the application. This is Linux programming, not
> > system management.
>
> The application cannot depend on the system to supply resources.

So write your own application with an OS in it and boot it out of LILO.
"The application cannot depend on the system to supply resources" is
just nonsense, BTW.

[...]
> > Anyway, my opinion doesn't count. Most of the main "large commercial apps"
> > vendors are moving to Linux. This is a fact. Either directly or not,
> > they're driven by technical reasons, since the Linux kernel has no
> > marketing dept.
>
> Many of them are having to rewrite parts of linux to do so. You seem to

Really? Why don't they post patches, so? I'm sure Linus will be glad to
accept them, if they're valid.

And BTW, why don't YOU post a non-overcommitting patch?

> be stuck in
> a single user environment. As long as you remain there you will not
> recognize the need
> for proper resource management.
             ^^^^^^^^^^^^^^^^^^^
Already seen. Nothing do to with overcommit.

>
> > > runs this way, deal with it" won't fly if someone requires no-overcommit. You can
> >
> > Only buggy applications require no-overcommit.
>
> Only buggy systems mandate overcommit. Besides- init was killed, (or
> sendmail/syslog/some
> random user/bash/...) Not the small program that used the last available
> page.

If you run your system in constant OOM conditions, you should either:
1) use Rik's patch;
2) avoid that anyway;

> > > argue that they are wrong until you are blue in the face, but they won't care -- they
> > > will just run NT or Solaris. The idea is to run with a kernel that performs resource
> >
> > And they will pay that choice. This is NOT a commercial vendor.
> > This is the kernel development list. (Not speaking for others) I don't
> > care a bit if someone else, in order to support badly written applications,
> > uses another OS. They will pay that choice. Someone sooner or later
> > will do the same job at half the costs (and the price) using Linux.
> > I've seen this happen. I've seen companies not listening to my advices
> > (at times when there was no hype around Linux) and choosing another,
> > zero-management, OS, and later employing (full time) an administrator
> > just to get a decent uptime. But this is OT.
> >
> > > management. Linux doesn't, as you have pointed out. It simply manages virtual address
> > > page tables -- not real memory. Some people want a system to actually manage *memory*
> > > not slots in a page table.
> >
> > Never said that. Your deliberately FUDing me (and Linux). I've never said
> > there's no mm in Linux. I've just said that brk() does not allocate
> > memory. The Linux kernel knows perfectly where RAM goes.
>
> So it is lying to the process ?

The process asks the kernel to extend its address space with brk().
The kernel does what the process asks.
The process does not ask to allocate memory.
The kernel does not allocate memory.
So simple. It's the defined brk() behaviour. And AFAIK, doesn't break
POSIX.

> > > > brk() is the normal API. And it works as expected.
> > > ---
> > > Not if I come from a system that manages memory.
> >
> > You don't like your application shows its bugs when run on a modern OS?
> > Should I care?
>
> A correctly functioning POSIX compliant program should work the same on
> Linux as it
> does on Solaris/Irix/UNICOS... Unfortunately they don't. Nothing wrong
> with the program.

Of course the program is wrong. It's using some non-POSIX implementation
details of Solaris/Irix/UNICOS. It's implementation dependant. And NOT
POSIX. So why should it not fail under Linux?

> the system keeps killing random or sometimes the wrong programs. The OOM
> killer is only
> a band-aid to hide the fact that the kernel has problems in a multiuser
> environment.

Multiuser? You keep confusing your user-level memory quotas with Linux
ovecommitting.

> > > getting back a *usable* pointer. Please -- don't try to argue that users really
> > > don't care if malloc returns a bogus pointer. It won't fly.
> >
> > Never said that.
>
> Sure you did. "malloc doesn't allocate memory..."

"users really don't care if malloc returns a bogus pointer" - I have
never said that.

"malloc doesn't allocate memory..." - I've said: "brk() doesn't allocate
memory...".

>
> --
> -------------------------------------------------------------------------
> Jesse I Pollard, II
> Email: pollard@cats-chateau.net
>
> Any opinions expressed are solely my own.
>

.TM.

-- 
      ____/  ____/   /
     /      /       /			Marco Colombo
    ___/  ___  /   /		      Technical Manager
   /          /   /			 ESI s.r.l.
 _____/ _____/  _/		       Colombo@ESI.it

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Fri Apr 07 2000 - 21:00:11 EST