Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plansfor 2.6.23]

[Rene Herman]

On 07/29/2007 01:41 PM, david@xxxxxxx wrote:

> > And now you do it again :-) There is no conclusion -- just the 
> > inescapable observation that swap-prefetch was (or may have been) 
> > masking the problem of GNU locate being a program that noone in their 
> > right mind should be using.
>
> isn't your conclusion then that if people just stopped useing that 
> version of updatedb the problem would be solved and there would be no 
> need for the swap prefetch patch? that seemed to be what you were 
> strongly implying (if not saying outright)

No. What I said outright, every single time, is that swap-prefetch in itself 
seems to make sense. And specifically that even if the _direct_ problem is a 
crummy program, it _still_ makes sense generally. Every single time.

But see -- you failed to notice this because you guys are stuck in this dumb 
adversary "us against them" thing so inherent of (online) communities, where 
you sit around your own habitats patting each other on the back for extended 
periods of time and then every once a while go out clinging on to each other 
vigorously and going "boo! hiss!" at the big bad outside world.

I already got overly violent at one point in this thread so I'll leave out 
any further references to sense-deprived fanboy-culture but please, I said 
every single time that I'm not against swap-prefetch. I cannot communicate 
when I'm not being read.

> I agree that tinkering with the core VM code should not be done lightly, 
> but this has been put through the proper process and is stalled with no 
> hints on how to move forward.

It has not. Concerns that were raised (by specifically Nick Piggin) weren't 
being addressed.

> forget the nightly cron jobs for the moment. think of this scenerio. you 
> have your memory fairly full with apps that you have open (including 
> firefox with many tabs), you receive a spreadsheet you need to look at, 
> so you fire up openoffice to look at it. then you exit openoffice and try
>  to go back to firefox (after a pause while you walk to the printer to
> get the printout of the spreadsheet)

And swinging a dead rat from its tail facing east-wards while reciting 
Documentation/CodingStyle.

Okay, very very sorry, that was particularly childish, but that "walking to 
the printer" is ofcourse completely constructed and this _is_ something to 
take into account. Swap-prefetch wants to be free, which (also again) it is 
doing a good job at it seems, but this also means that it waits for the VM 
to be _very_ idle before it does anything and as such, we cannot just forget 
the "nightly" scenario and pretend it's about something else entirely. As 
long as the machine's being used, swap-prefetch doesn't kick in.

Which is a good feature for swap-prefetch, but also something that needs to 
weighed alongside its other features in a discussion of alternatives, where 
for example something like a larger swap granularity would not have anything 
of the sort to take into account. If it were about walks to the printer, we 
could shelve the issue as being of too limited practical use for inclusion.

> > -- 2: no serious investigation into possible downsides
> >
> > Swap-prefetch tries hard to be as free as possible and it seems to 
> > largely be succeeding at that. Thing that (obviously -- as in I 
> > wouldn't want to state it's the only possible worry anyone could have 
> > left) remains is the "papering over effect" it has by design that one 
> > might not care for.

Arjan van de Ven made another point here about seeking away due to 
swap-prefetch (just) before the next request comes in, but that's probably a 
bit of a non-issue in practice with the "very idle" precondition.

> > -- 3: no serious consideration of possible alternatives
> >
> > Tweaking existing use-oce logic is one I've heard but if we consider 
> > the i/dcache issue dead, I believe that one is as well. Going to 
> > userspace is another one. Largest theoretical potential. I myself am 
> > extremely sceptical about the Linux userland, and largely equate it 
> > with "smallest _practical_ potential" -- but that might just be me.
> >
> > A larger swap granularity, possible even a self-training granularity. 
> > Up to now, seeks only get costlier and costlier with respect to reads 
> > with every generation of disk (flash would largely overcome it though) 
> > and doing more in one read/write _greatly_ improves throughput, maybe 
> > up to the point that swap-prefetch is no longer very useful. I myself 
> > don't know about the tradeoffs involved.
>
> larger swap granularity may help, but waiting for the user to need the 
> ram and have to wait for it to be read back in is always going to be 
> worse for the user then pre-populating the free memory (for the case 
> where the pre-population is right, for other cases it's the same). so I 
> see this as a red herring

I saw Chris Snook make a good post here and am going to defer this part to 
that discussion:

http://lkml.org/lkml/2007/7/27/421

But no, it's not a red herring if _practically_ speaking the swapin is fast 
enough once started that people don't actually mind anymore since in that 
case you could simply do without yet more additional VM complexity (and 
kernel daemon).

> there are fully legitimate situations where this is useful, the 'papering
> over' effect is not referring to these, it's referring to other possible
> problems in the future.

No, it's not just future. Just look at the various things under discussion 
now such as improved use-once and better swapin.

Rene.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/