[PATCH v2 0/3] mm/filemap: read ahead for mmap readers that move backwards

[Qiliang Yuan]

A reader that walks a file mapping towards lower offsets gets almost no
readahead from mmap faults when the mapping asks for it. glibc's
memmove() and memcpy() copy backwards when the destination lies above
the source, so this happens to loaders that copy large files out of a
mapping, such as LLM inference engines copying model weights out of a
safetensors mapping. Both MADV_HUGEPAGE and MADV_SEQUENTIAL read ahead
only towards higher offsets, so each fault of such a reader reads just
the part it is missing.

MADV_SEQUENTIAL was slow for this before 7.2 as well. MADV_HUGEPAGE
became much slower in 7.2: since commit 0b9c0aeba938 ("mm/filemap:
count only the faulting address as a mmap hit"), the PTEs mapped around
a fault no longer count as hits and PMD mappings never did, so a
backward reader's mmap_miss passes MMAP_LOTSAMISS after about a hundred
folios and readahead stops. Each page after that is read on its own.

Patch 1 moves the mmap hit check of filemap_map_pages() into helpers.
Patch 2 fixes the regression: on a descending VM_HUGEPAGE fault it also
reads the folio below, and it counts a PMD mapping of the faulting
address as a hit, as a PTE mapping is. Patch 3 reads the window that
ends at the fault on a descending VM_SEQ_READ fault.

Reading one byte from each page of a 2 GiB file, last page first, with
a cold page cache, read_ahead_kb 8192, in a 32 vCPU VM:

                       v7.4-rc2      patched
  MADV_HUGEPAGE          42.23 s      0.58 s
  MADV_SEQUENTIAL        52.42 s      0.96 s

The same walk took 0.53 s with MADV_HUGEPAGE just before
0b9c0aeba938.

Forward walks take the same time, and a fault on an uncached 2 MiB hole
between two cached ranges of a MADV_HUGEPAGE mapping is still mapped
with a PMD, 300 out of 300 times with and without the series. Megabytes
read and major faults for other access patterns over a 16 GiB file:

                                    v7.4-rc2           patched
  1% random, MADV_NORMAL         1002 MB, 39617      1013 MB, 39617
  1% random, MADV_HUGEPAGE       1584 MB, 38760      1587 MB, 38756
  1% stride 64 pages            10503 MB, 2          10496 MB, 2
  5% random in 2 GiB memcg,
    MADV_HUGEPAGE                2205 MB, 187416     2225 MB, 187402

The walk above, run as "revwalk FILE" for MADV_HUGEPAGE and "revwalk FILE
seq" for MADV_SEQUENTIAL, is below. I can turn it into a selftest if that
is wanted.

  /* revwalk FILE [seq]: read each page of FILE, last page first */
  #include <fcntl.h>
  #include <stdio.h>
  #include <string.h>
  #include <sys/mman.h>
  #include <sys/resource.h>
  #include <sys/stat.h>
  #include <unistd.h>

  int main(int argc, char **argv)
  {
  	long ps = sysconf(_SC_PAGESIZE);
  	int fd = open(argv[1], O_RDONLY);
  	struct rusage ru;
  	struct stat st;
  	size_t i;
  	char *p;

  	fstat(fd, &st);
  	posix_fadvise(fd, 0, 0, POSIX_FADV_DONTNEED);
  	p = mmap(NULL, st.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
  	madvise(p, st.st_size, argc > 2 && !strcmp(argv[2], "seq") ?
  		MADV_SEQUENTIAL : MADV_HUGEPAGE);
  	for (i = st.st_size / ps; i > 0; i--)
  		(void)*(volatile char *)(p + (i - 1) * ps);
  	getrusage(RUSAGE_SELF, &ru);
  	printf("major faults: %ld\n", ru.ru_majflt);
  	return 0;
  }

Signed-off-by: Qiliang Yuan <qiliang.yuan@xxxxxxxxx>
---
V1 -> V2:
- Split the mmap hit helpers out of the fix into patch 1
- Patch 2: take a fault as descending only when the folio below it is
  not cached either, so a hole between two cached ranges is still read
  as one large folio (Sashiko review)
- Rebase onto v7.4-rc2 and remeasure
- Send from my linux.dev address

v1: https://lore.kernel.org/r/20261022-bug-mm-mmap-readahead-descending-v1-0-6d08b34862f2@xxxxxxxxx

---
Qiliang Yuan (3):
      mm/filemap: move the mmap hit check of filemap_map_pages() into helpers
      mm/filemap: read ahead downwards on descending VM_HUGEPAGE faults
      mm/filemap: extend VM_SEQ_READ readahead below descending faults

 mm/filemap.c | 119 ++++++++++++++++++++++++++++++++++++++++++++---------------
 1 file changed, 89 insertions(+), 30 deletions(-)
---
base-commit: 7475ea8b58616b33c04afb976cf0fbaad9dd63fb
change-id: 20261022-bug-mm-mmap-readahead-descending-9cab86703bbb

Best regards,
-- 
Qiliang Yuan <qiliang.yuan@xxxxxxxxx>