Re: [resent PATCH] Re: very slow parallel read performance

From: Oliver.Neukum@lrz.uni-muenchen.de
Date: Mon Aug 27 2001 - 17:10:45 EST


Hi,

> > If we are optimising for streaming (which readahead is made for) dropping
> > only one page will buy you almost nothing in seek time. You might just as
> > well drop them all and correct your error in one larger read if necessary.
> > Dropping the oldest page is possibly the worst you can do, as you will need
> > it soonest.
>
> Yes, good point. OK, I'll re-examine the dropping logic. Bear in mind,
> dropping readahead pages is not supposed to happen frequently under
> steady-state operation, so it's not that critical what we do here, it's going
> to be hard to create a load that shows the impact. The really big benefit
> comes from not overdoing the readahead in the first place, and not underdoing
> it either.

OK, so you have four reasons for dropping a readahead page

1) File was closed - trivial I'd say
2) Memory needed for other readahead
3) Memory needed for anything but readahead
4) The page won't be needed - some kind of timeout ?

Cases 3 and 2 are hard. Should replacement policy vary ?

> > > > If you are streaming dropping all should be no great loss.
> > >
> > > The quesion is, how do you know you're streaming? Some files are
> > > read/written many times and some files are accessed randomly. I'm trying
> > > to avoid penalizing these admittedly rarer, but still important cases.
> >
> > For those other case we have pages on the active list which are properly
> > aged, haven't we ?
>
> Not if they never get a change to be moved to the active list. We have to be
> careful to provide that opportunity.

I see no reason a change to readahead would affect that. Maybe I am dense.
Relevant seems to be rather whether you use read-once or put them on the
active list right away.

Could information on the use pattern be stored in the dentry ?
E.g. If we opened it twice in the last 30 seconds we put all referenced
pages into the active list unaged to ensure that they'll stay in core.

> > And readahead won't help you for random access unless you can cache it all.
> > You might throw in a few extra blocks if you have free memory, but then you
> > have free memory anyway.
>
> Readahead definitely can help in some types of random access loads, e.g.,
> when access size is larger than a page, or when the majority of pages of a
> file are eventually accessed, but in random order. On the other hand, it

Only if they fit into ram. There's nothing wrong with reading the whole
accessed file if it's small enough and you have a low memory pressure.
It might cut down on netscape launch times drastically on large machines.
But if there's memory pressure you can hardly evict pages on the active
list for readahead, can you ?

> will only hurt for short, sparse, random reads. Such a pattern needs to be
> detected in generic_file_readahead.

Interresting, how do you do that, keep a list of recent reads ?

        Regards
                Oliver

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Fri Aug 31 2001 - 21:00:26 EST