Re: Gigantic memory leak in linux-2.6.[789]!

[David Lang]

On Sun, 24 Oct 2004, Bill Davidsen wrote:

> David Lang wrote:
>
> > This puts the cost of zeroing out and freeing memory on new programs that 
> > are allocating memory, which tends to scatter the work over time rather 
> > then having a large burst of work kick in when a program exits (it seems 
> > odd to think that if a large computer exits the machine would be pegged 
> > for a little while while it frees up and zeros the memory, not exactly 
> > what you would expect when you killed a program :-)
>
> Any this partially explains why response is bad every morning when starting 
> daily operation. Instead of using the totally unproductive time in the idle 
> loop to zero and free those pages when it would not hurt response, the kernel 
> saves that work for the next time the memory is needed lest it do work which 
> might not be needed before the system is shutdown.

actually, what useually has happened is that updatedb ran overnight and 
used all your memory for it's work so all your application stuff got 
thrown away or swapped out as it appeared to be less useful then the 
then-active process. so first thing in the morning you need to do a lot of 
disk reads to get your desktop working set into memory. the cost of 
zeroing the pages is minor compared to the disk IO

> With all the work Nick, Ingo,Con and others are putting into latency and 
> responsiveness, I don't understand why anyone thinks this is desirable 
> behavior. The idle loop is the perfect place to perform things like this, to 
> convert non-productive cycles into performing tasks which will directly 
> improve response and performance when the task MUST be done. Things like 
> zeroing these pages, perhaps defragmenting memory, anything which can be done 
> in small parts.
>
> It would seem that doing things like this in small inefficient steps in idle 
> moments is still better than doing them efficiently while a process is 
> waiting for the resources being freed.

the problem is that you don't know that you need to throw away the data. 
the next thing that you try to do could re-use the data that's in the ram, 
how can the system know?

David Lang


--
There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies.
 -- C.A.R. Hoare
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/