I see a misunderstanding here. Jay refers to pre-fetching (from disk),
while Dave refers to pre-faulting (setting up page tables and waiting
for the page data to be present).
These are very different.
> The problematic case (and a real life one) is when all of libc has
> been faulted into main memory by various processes, when you start one
> up do you map in all of libc when it gets mmap'd by the application?
> If not, then which if any pages do you choose?
You don't map any more than you do know. What you _do_ do is readahead
pages into the page cache, based on when pages are mapped into the
process. If you just map the lot, (a) it's inefficient (as you know);
(b) you don't get told when to initiate further asynchronous readahead
_before_ the pages are needed synchronously.
Anyway, don't the latest VM clustered page-in/swap-in changes have some
of the sort of effect we're talking about? However, I don't think they
implement genuine predictive readahead, which could be done like this:
Process faults on page 0.
Kernel reads pages 0..15 into page cache (one cluster).
Kernel maps page 0 into process.
Process faults on page 1.
Kernel maps page 1 into process.
Process faults on page 2.
Kernel maps page 2 into process.
Process faults on page 3.
Kernel maps page 3 into process.
Process faults on page 4.
Kernel maps page 4 into process.
Process faults on page 5.
Kernel maps page 5 into process.
Process faults on page 6.
Kernel maps page 6 into process.
Process faults on page 7.
Kernel maps page 7 into process.
Process faults on page 8.
Kernel maps page 8 into process.
Kernel initiates asynchronous readahead of pages 16..31 (next cluster).
Process faults on page 9.
Kernel maps page 9 into process.
[Repeat until page 23]
Process faults on page 24
Kernel maps page 24 into process.
Kernel initiates asynchronous readahead of pages 32..47 (next cluster).
See the difference? This way, the process _never_ blocks waiting for data.
The existing heuristics don't have this property, as far as I know.
-- Jamie
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/