Re: [PATCH] mm/lruvec: preemptively free dead folios during lru_add drain

[Barry Song]

On Fri, Apr 24, 2026 at 12:43 AM JP Kobryn (Meta) <jp.kobryn@xxxxxxxxx> wrote:
>
> Of all observable lruvec lock contention in our fleet, we find that ~24%
> occurs when dead folios are present in lru_add batches at drain time. This
> is wasteful in the sense that the folio is added to the LRU just to be
> immediately removed via folios_put_refs(), incurring two unnecessary lock
> acquisitions.
>
> Eliminate this overhead by preemptively cleaning up dead folios before they
> make it into the LRU. Use folio_ref_freeze() to filter folios whose only
> remaining refcount is the batch ref. When dead folios are found, move them
> off the add batch and onto a temporary batch to be freed.
>
> During A/B testing on one of our prod instagram workloads (high-frequency
> short-lived requests), the patch intercepted almost all dead folios before
> they entered the LRU. Data collected using the mm_lru_insertion tracepoint
> shows the effectiveness of the patch:
>
> Per-host LRU add averages at 95% CPU load
> (60 hosts each side, 3 x 60s intervals)
>
>             dead folios/min  total folios/min   dead %
> unpatched:        1,297,785        19,341,986  6.7097%
> patched:                 14        19,039,996  0.0001%
>
> Within this workload, we save ~2.6M lock acquisitions per minute per host
> as a result.
>
> System-wide memory stats improved on the patched side also at 95% CPU load:
>  - direct reclaim scanning reduced 7%
>  - allocation stalls reduced 5.2%
>  - compaction stalls reduced 12.3%
>  - page frees reduced 4.9%
>
> No regressions were observed in requests served per second or request tail
> latency (p99). Both metrics showed directional improvement at higher CPU
> utilization (comparing 85% to 95%).
>
> Signed-off-by: JP Kobryn (Meta) <jp.kobryn@xxxxxxxxx>

Hi JP,
I’m seeing a large number of "BAD page" bugs.
Not sure if it’s related, but reverting this patch
seems to fix the issue.

[ 2869.365978] BUG: Bad page state in process uname  pfn:3a5417
[ 2869.365981] page: refcount:0 mapcount:0 mapping:0000000000000000
index:0x724884c20 pfn:0x3a5417
[ 2869.365983] flags:
0x17ffffc0020908(uptodate|active|owner_2|swapbacked|node=0|zone=2|lastcpupid=0x1fffff)
[ 2869.365985] raw: 0017ffffc0020908 0000000000000000 dead000000000122
0000000000000000
[ 2869.365986] raw: 0000000724884c20 0000000000000000 00000000ffffffff
0000000000000000
[ 2869.365986] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[ 2869.366037] CPU: 11 UID: 0 PID: 179413 Comm: uname Tainted: G S  B
             7.0.0mglruswappiness+ #401 PREEMPT(full)
[ 2869.366039] Tainted: [S]=CPU_OUT_OF_SPEC, [B]=BAD_PAGE
[ 2869.366039] Hardware name: To Be Filled By O.E.M. To Be Filled By
O.E.M./H470D4-P1, BIOS P2.00 04/01/2021
[ 2869.366040] Call Trace:
[ 2869.366041]  <TASK>
[ 2869.366041]  dump_stack_lvl+0x76/0xa0
[ 2869.366043]  dump_stack+0x10/0x20
[ 2869.366044]  bad_page+0x79/0x120
[ 2869.366046]  free_unref_folios+0x85b/0x960
[ 2869.366048]  ? __pfx_lru_add+0x10/0x10
[ 2869.366049]  folio_batch_move_lru+0x260/0x2d0
[ 2869.366051]  __folio_batch_add_and_move+0x7f/0x110
[ 2869.366052]  folio_add_lru+0x49/0x60
[ 2869.366053]  folio_add_lru_vma+0x7b/0xb0
[ 2869.366054]  map_anon_folio_pte_nopf+0xb8/0x170
[ 2869.366055]  do_anonymous_page+0x5fa/0x960
[ 2869.366056]  ? __pte_offset_map+0x1c/0x140
[ 2869.366058]  __handle_mm_fault+0xbb6/0x1010
[ 2869.366059]  ? mt_find+0xe8/0x560
[ 2869.366061]  handle_mm_fault+0x1b0/0x370
[ 2869.366063]  do_user_addr_fault+0x2c8/0x870
[ 2869.366064]  exc_page_fault+0x7d/0x1d0
[ 2869.366065]  asm_exc_page_fault+0x27/0x30
[ 2869.366066] RIP: 0010:rep_movs_alternative+0x11/0x90
[ 2869.366068] Code: c3 cc cc cc cc 0f 1f 40 00 90 90 90 90 90 90 90
90 90 90 90 90 90 90 90 90 48 83 f9 40 73 44 83 f9 08 73 25 85 c9 74
0f 8a 06 <88> 07 48 ff c7 48 ff c6 48 ff c9 75 f1 c3 cc cc cc cc 66 66
2e 0f
[ 2869.366069] RSP: 0018:ffffccb46a837a78 EFLAGS: 00050202
[ 2869.366070] RAX: 0000000000000078 RBX: 00007ffee44638d0 RCX: 0000000000000007
[ 2869.366071] RDX: 0000000000000000 RSI: ffffffff8d0d1d90 RDI: 00007ffee44638c9
[ 2869.366072] RBP: ffffccb46a837a80 R08: 00007ffffffff000 R09: 0000000000000000
[ 2869.366072] R10: ffff8c7769873600 R11: 0000000000000000 R12: ffff8c7786563800
[ 2869.366073] R13: ffff8c76c6668480 R14: ffff8c76cfe3ba40 R15: ffff8c770f419c00
[ 2869.366074]  ? arch_align_stack+0x3f/0x60
[ 2869.366076]  ? _copy_to_user+0x31/0x60
[ 2869.366078]  load_elf_binary+0xda1/0x1800
[ 2869.366080]  bprm_execve+0x2d7/0x590
[ 2869.366083]  do_execveat_common.isra.0+0x15c/0x1c0
[ 2869.366084]  __x64_sys_execve+0x3e/0x70
[ 2869.366085]  x64_sys_call+0xd91/0x26e0
[ 2869.366087]  do_syscall_64+0xe7/0x560
[ 2869.366088]  ? folio_add_lru+0x49/0x60
[ 2869.366089]  ? folio_add_lru_vma+0x7b/0xb0
[ 2869.366090]  ? set_ptes.isra.0+0x3b/0x90
[ 2869.366091]  ? do_wp_page+0x85d/0xed0
[ 2869.366092]  ? __handle_mm_fault+0xafe/0x1010
[ 2869.366094]  ? do_sigaction+0x165/0x4c0
[ 2869.366096]  ? _copy_to_user+0x31/0x60
[ 2869.366098]  ? __x64_sys_rt_sigaction+0xb7/0x110
[ 2869.366100]  ? x64_sys_call+0x880/0x26e0
[ 2869.366101]  ? do_syscall_64+0x11b/0x560
[ 2869.366102]  ? do_syscall_64+0x9c/0x560
[ 2869.366103]  ? clear_bhb_loop+0x30/0x80
[ 2869.366104]  entry_SYSCALL_64_after_hwframe+0x76/0x7e

Thanks
Barry