Re: [RFC PATCH 0/3] Neural Storage Driver - learning page cache prefetcher

[ayhan aydın]

Hi Honza, Hi Willy,

Thanks for the review and for the questions — they pushed us to go back
and measure this properly. Since the last exchange we ran a full
window-size matrix with complete latency and overhead numbers, and it
answers directly what both of you asked. Let me start with the data.

These results come from our current in-development tree. We wanted to
share them with you before pushing anything, so the tree referenced in
the RFC (upstream-prep-v1, 2026-08-17) is still the earlier snapshot.
Since that snapshot we made three changes that led to these numbers:

  1. Jump handling — when a prediction lands outside the kernel's
     readahead window we now always issue WILLNEED|NOREUSE explicitly,
     instead of leaving the strategy branch to decide.
  2. Kernel-window awareness — when the kernel's own readahead window
     is already >= 1MB the module detects this and completely silences
     itself (confirmed by its own counters), so it never competes with
     or disturbs a kernel window that is already sufficient.
  3. Full instrumentation — per-run p50/p95/p99/p999 latency, per-pass
     usr/sys CPU overhead, and disk-I/O accounting.

We'll push the updated tree to GitHub once this discussion settles.

Full metrics matrix — methodology: interleaved ON/OFF on the same
machine, drop_caches before every run, 3 runs per configuration, 6.2 GB
SQLite database and an 8 GB sequential file. Disk I/O is byte-identical
between ON and OFF in all 96 runs — NSD never issues extra I/O.

seq64 — 8 GB sequential file, 64k requests, n=131K/run:

  kb    | OFF avg  ON avg   diff  | p50 OFF->ON  | p99 OFF->ON   |
overhead (usr+sys)
  128   | 22.02s   17.11s   -22%  | 50->23 us     | 812->805 us   | 5.94->4.33s
  256   | 17.08s   17.22s   +1%   | 20->15 us     | 824->1129 us  | 4.07->4.13s
  512   | 16.81s   16.96s   +1%   | 16->14 us     | 1092->1903 us | 4.24->3.60s
  1024  | 17.25s   17.03s   -1%   | 15->16 us     | 1903->1883 us | 3.67->3.74s

SQLite Q2 full-table scan — 6.2 GB:

  kb    | OFF avg  ON avg   diff  | overhead (usr+sys)
  128   | 24.51s   20.48s   -16%  | +0.95s
  256   | 20.05s   16.60s   -17%  | +1.26s
  512   | 15.31s   12.18s   -20%  | +1.06s
  1024  | 12.22s   12.09s   -1%   | +0.26s

random4k — 100k random 4k requests:

  kb    | OFF avg  ON avg   diff  | p99
  128   | 25.0s    25.5s    +2%   | ~0.98ms
  256   | 26.9s    26.0s    -3%   | ~0.99ms
  512   | 25.7s    25.5s    -1%   | ~0.98ms
  1024  | 25.4s    24.7s    -3%   | ~0.99ms

random_repeat — 100k offsets x 3 passes, n=300K:

  kb    | OFF avg  ON avg   diff  | p99
  128   | 26.8s    25.1s    -6%   | 0.39ms
  256   | 25.1s    24.0s    -4%   | 0.38ms
  512   | 24.5s    24.4s    0%    | 0.37ms
  1024  | 25.0s    24.7s    -1%   | 0.39ms

Honza, you asked whether the difference goes away at 1M — and yes, it
does, for both workloads. The 128k default is the dominant factor; at
1M the kernel alone already reaches this SSD's bandwidth ceiling, and
NSD recognises that and silences itself entirely at >= 1M windows.

But the 512k row is the one I'd really like your view on. The kernel
with a 512k window alone (15.31s) trails the kernel with 1M (12.22s)
by about 25%, yet NSD with a 512k window reaches 12.18s — the same
throughput at half the window. Our block-layer traces show OFF@512k
keeps issuing ~512k requests, so the device is not the limit. It looks
like the kernel's window growth is capped at ra_pages, while the
fadvise-driven expansion NSD triggers is not — which, incidentally, is
why distros end up hand-tuning this value. Is that cap deliberate, for
latency or cache-pollution control, or is there room for the growth
policy to self-adapt?

Willy, on the architecture: hooking only vfs_read and skipping the
fault path was a deliberate choice to bound the first iteration, but
you're right that it's a gap — filemap_fault bypasses vfs_read
entirely, so we're currently blind to mmap'd workloads. Hooking it
with the same fprobe is the next step. On replacing the PG_readahead
folio marking: the matrix above convinced us this is the right
direction. The kernel's growth policy, not the device, is the real
bottleneck at default settings, so we're moving toward a design that
manages ra_state directly and measures its own waste — per-file
tracking of consumed versus expired prefetches with adaptive
aggressiveness, which is really just replacing the implicit
assumptions behind PG_readahead marking with explicit measurement.
Backward access patterns are already handled by the stride predictor
(negative deltas, forward/backward counted separately); a dedicated
backward-pattern benchmark will be added to the suite.

Two things we're still working on, for honesty: at 256-512k the
sequential p99 regresses (~0.8ms -> ~1.1ms and ~1.1ms -> ~1.9ms) as
the prefetch regions push the device queue — the usage-tracking work
above targets exactly that. And the random_repeat gains (4-6% at
128-256k) suggest the predictor does help beyond what window size can
fix; p95 there improves from 296 to 243 us.

Questions we'd value your input on:
  1. Is fadvise-driven expansion beyond ra_pages acceptable as a
     mechanism, or should the growth policy itself change?
  2. What would a minimal integrated design look like — replacing
     PG_readahead marking with direct ra_state control?
  3. If we iterate: fault path first, or usage-tracking first?

Per your suggestion, we'll cc the page cache maintainers from
MAINTAINERS on the next revision.

Best regards,
Ayhan