Re: Memory use: returning freed memory to the pool

Theodore Y. Ts'o (tytso@MIT.EDU)
Tue, 28 Jan 1997 22:40:49 -0500


From: ganesh@cse.iitb.ernet.in
Date: Wed, 29 Jan 1997 07:04:32 +0530 (IST)

There was a thread in the newsgroups some time back about the X server not
returning memory but always growing. From memory, the basic points were :

1. X does use free (), but the malloc/free system never returns virtual
memory to the OS. It merely increases brk when needed.

It depends on the malloc/free implementation. In fact the malloc()
which is in the Linux libc will return virtual memory to the OS.

HOWEVER, malloc() can only return memory if if there is a contiguous
chunk of memory just below brk that it can give back to the system.
With X, sometimes this just doesn't happen.

4. Xinside's server uses their own version of malloc/free, which actually
unmaps memory which has been freed, once it exceeds a certain limit or
something.

If I understand things correctly, Xinside does more than this --- they
actually use a garbage collecting-like scheme where they will actually
rearrange allocated memory (by mucking with pointers) so that there's a
chunk of memory they can give back.

IMHO this sucks badly. Surely malloc/free could check whether there is
a fairly large block between the last allocated chunk and brk ? If
efficiency is the problem then might it be better to have another
function in libmalloc like free_virtual() which can be called explicitly
by those programs which know they need it ?

malloc/free already do check to see if there's a fiarly large block
between the last allocated chunk and brk, and they will give that back.
If you run e2fsck -tt, you will see the memory used go negative during
some of the passes. That reflects malloc() giving back memory by moving
the brk.

At one point there was someone who was experimenting with a version of
malloc() which used mmap in a clever way to try to make it more easy to
release memory back to the system. There are some real framentation
overhead and page boundary overheads that have to be considered,
though. It's not a trivial problem.

It's much easier when you put a lot of cleverness into the application
program, which is what X Inside does with its server.

- Ted