Re: Ideas for memory management hackers.

Rik van Riel (H.H.vanRiel@fys.ruu.nl)
Fri, 14 Nov 1997 13:32:50 +0100 (MET)


On Fri, 14 Nov 1997, Mitch Davis wrote:

> Dear Rick,
>
> I'm writing to you with some ideas for Linux memory management.
> There are two things which I think would be really nice.
>
> 1. Imagine you're running a Linux binary off a umsdos or nfs or
> smbfs (read *slow*) volume. When memory gets tight, the kernel
> starts looking for pages to throw away to meet requests. For
> ordinary binaries, the first thing it throws away are the read-only
> program text pages. This is because they don't need to be written
> - they can just be re-read. But if you're running off a slow
> volume, the cost of rereading the page is a lot higher - often
> higher in fact than if that read-only page had been swapped to a
> (raw, local) swap partition and re-read quickly from there.

This will be more than just a trivial hack, because programs
are MMAPed. For this to change I think we'll have to hack
fs/exec.c and the fs/binfmt_xxx.c files to check for a
file / fs option and read the program in memory completely.

Than there's the possibility of sharing these pages...
Arghh. I'll take a look after the weekend.

> 2. I know free RAM is bad, but....
>
> Why should a file page I read in ten seconds ago need to be reread,
> just because a 10-minute-old program page took up RAM that could
> have been used as file cache?
>
My patch has solved these issues. The third version of my
patch even does page-aging for shared pages...
I can see my system's using more swap than it used to, and
I also notice decreased disk activity.
The patch (against linux-2.1.63) should be on linux mama
anytime now... <a href="http://www.huwig.de/linux/mama/">mama</a>
I think I'll put them on my homepage when I get to it...
<a href="http://www.fys.ruu.nl/~riel/">me</a> you don't want to see it now,
only the scouting part is up :-(

> In practise, when I'm low on real RAM, I start Emacs then quit it.
> This does as we described above, and the performance benefit
> afterwards is really worth it.

Not needed anymore, not with my patch that is.
I'm just not sure about CPU usage of my patch (I think
I might be seeing some CPU usage spikes every once in
a while). This could be far worse on large-memory
systems (>128MEG) with loads of I/O going on.
I still need feedback from people with machines like that...
PLEASE, test the patch and give mee feedback. I know
that it is stable on my (24MEG) machine, but I need
to know side-effects on large-memory and loads-of-IO
machines. I'll submit it to Linus as soon as I know
it works for large-mem systems as well...

Rik.

PLEASE, test vhand-2.1.63. I know it is stable, I
just don't know how well it scales...
----------
Send Linux memory-management wishes to me: I'm currently looking
for something to hack...