Re: Linux 2.6.29

[Linus Torvalds]

On Thu, 26 Mar 2009, Linus Torvalds wrote:
> 
> The only times tunables have worked for us is when they auto-tune. 
> 
> IOW, we don't have "use 35% of memory for buffer cache" tunables, we just 
> dynamically auto-tune memory use. And no, we don't expect user space to 
> run some "tuning program for their load" either.

IOW, what we could reasonably do is something along the lines of:

 - start off with some reasonable value for max background dirty (per 
   block device) that defaults to something sane (quite possibly based on 
   simply memory size).

 - assume that "foreground dirty" is just always 2* background dirty.

 - if we hit the "max foreground dirty" during memory allocation, then we 
   shrink the background dirty value (logic: we never want to have to wait 
   synchronously)

 - if we hit some maximum latency on writeback, shrink dirty aggressively 
   and based on how long the latency was (because at that point we have a 
   real _measure_ of how costly it is with that load).

 - if we start doing background dirtying, but never hit the foreground 
   dirty even in dirty balancing (ie when a writer is actually _writing_, 
   as opposed to hitting it when allocating memory by a non-writer), then 
   slowly open up the window - we may be limiting too early.

.. add heuristics to taste. The point being, that if we do this based on 
real loads, and based on hitting the real problems, then we might actually 
be getting somewhere. In particular, if the filesystem sucks at writeout 
(ie the limiter is not the _disk_, but the filesystem serialization), then 
it should automatically also shrink the max dirty state. 

The tunable then could become the maximum latency we accept or something 
like that. Or the hysteresis limits/rules for the soft "grow" or "shrink" 
events. At that point, maybe we could even find something that works for 
most people.

			Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/