Re: [BUG] shmem: FALLOC_FL_PUNCH_HOLE vs fault-around race corrupts page cache / rss counters

[Pedro Falcato]

(please use the email I actually use for work, thanks; not sure how
you got to that one)

Hi,

On Thu, Sep 24, 2026 at 06:16:21AM +0000, Ayush Ranjan wrote:
> Hi,
> 
> We are seeing shmem/tmpfs page cache corruption on production hosts
> running a workload that punches holes in a memfd (hole-punch based
> memory reclaim) while other threads and forked children fault the
> same MAP_SHARED mapping. The kernel taints but does not oops:
> 
>   BUG: Bad page cache in process ...  pfn:...
>   page dumped because: still mapped when deleted
>   ...
>   dentry name(?): "memfd:..."
> 
> and, more frequently, a paired rss-counter imbalance when the mm is
> torn down, always exactly one PMD-order folio (512 pages):
> 
>   BUG: Bad rss-counter state mm:... type:MM_FILEPAGES  val:-512
>   BUG: Bad rss-counter state mm:... type:MM_SHMEMPAGES val:512
> 
> Seen on 6.12 and 6.18, x86_64, bare metal and VM, with
> /sys/kernel/mm/transparent_hugepage/shmem_enabled = always.
> 
> This looks like the same corruption Gregory reported in March, which
> as far as I can tell stalled without a fix; that report needed ~100
> ballooning VMs to reproduce:

Well, I thought (from the lack of replies) that it was probably a bug on
their side. Perhaps that's not true :/

> 
>   https://patchew.org/linux/20260326162611.693539-1-gourry@xxxxxxxxxx/
> 
> The reproducer at the end of this mail trips it with a single memfd,
> no VMs or ballooning, within a few minutes on a large machine, so
> hopefully it makes the race easier to confirm (and to test a fix
> against).
> 
> Here is my best understanding of the race -- corrections welcome:
> 
> shmem guards faults against an in-progress hole punch with
> inode->i_private: shmem_fault() -> shmem_falloc_wait() waits while
> shmem_fallocate(PUNCH_HOLE) holds i_private. But shmem's .map_pages
> is the generic filemap_map_pages() (shmem_vm_ops /
> shmem_anon_vm_ops), which does not consult i_private and does not
> take invalidate_lock, and shmem does not use invalidate_lock to
> serialize faults against truncation the way regular filesystems do --
> the i_private + waitq scheme stands in for it, but only shmem_fault()
> participates in that scheme.
> 
> So while shmem_fallocate(PUNCH_HOLE) is between
> unmap_mapping_range() and shmem_truncate_range(), a concurrent
> fault-around can (re-)install PTEs for folios that are about to be
> truncated:
> 
>   - filemap_map_pages() samples mm_counter_file(folio) once per batch
>     and applies it with add_mm_counter() after mapping; if the
>     folio's swapbacked state changes while it is concurrently torn

But that cannot happen? We hold the folio lock in filemap_map_pages().
The folio (naturally) cannot be torn down while we have the folio lock.

>     down, the map-time counter (MM_FILEPAGES) and the zap-time
>     counter (MM_SHMEMPAGES) disagree by exactly one folio -- the
>     +/-512 imbalance above.
> 
>   - a folio re-mapped in this window (by fault-around directly, or
>     via a child VMA whose PTEs copy_page_range() installs after
>     unmap_mapping_range() has already walked the i_mmap tree -- the
>     dup_mmap() variant discussed in the earlier thread) can be
>     deleted from the page cache while still mapped -> "still mapped
>     when deleted".

No, I don't think this paragraph is true. Page cache truncation (via
truncate, or fallocate PUNCH_HOLE) takes the folio lock for each folio
that is about to be truncated out. Mapping folios takes the folio lock
as well, except in the fork() case where a myriad of weird interval tree
+ PTE lock interactions make it safe (AIUI).

> 
> Reproducer
> ----------
> 
> The race is on PMD-order folios, so khugepaged needs to scan
> aggressively (with the default 10s scan interval the punched ranges
> are not re-collapsed fast enough to reproduce quickly):
> 
>   echo always > /sys/kernel/mm/transparent_hugepage/shmem_enabled
>   cd /sys/kernel/mm/transparent_hugepage/khugepaged
>   echo 1    > scan_sleep_millisecs
>   echo 4096 > pages_to_scan
>   echo 511  > max_ptes_none
> 
>   cc -O2 -pthread -o repro repro_shmem_punch_race.c
>   for i in $(seq $(( $(nproc) / 3 ))); do ./repro 60 & done; wait
>   # watch: dmesg -w
> 
> On a 112-CPU host this trips within ~2-5 minutes; this capture is
> from 6.12.0-204.92.4.4.3.el9uek.x86_64:

Awesome that you have a reproducer! Have you reproduced this on a mainline
kernel? Enterprise kernels are not supported upstream.

In any case, I'll take a closer look ASAP.
 

-- 
Pedro