Re: [PATCH v6 0/5] mm: batch TLB flushing for dirty folios in vmscan
From: Zhang Peng
Date: Mon Nov 16 2026 - 08:00:33 EST
On Mon, Nov 16, 2026 at 07:02:31AM +0800, Barry Song (Xiaomi) wrote:
> On Sat, Oct 31, 2026 at 6:35 PM Zhang Peng <zippermonkey@xxxxxxxxxx> wrote:
> >
> > This series introduces batch TLB flushing optimization for dirty folios
> > during memory reclaim, aiming to reduce IPI overhead on multi-core systems.
> >
> > Background
> > ----------
> > Reclaim calls try_to_unmap_flush_dirty() before writing out each dirty
> > folio. When writable TLB entries are pending, this triggers a flush.
> > On systems that use IPIs for TLB shootdowns, frequent flushes can add
> > significant overhead.
> >
> > Approach
> > --------
> > Collect dirty folios into a folio_batch, releasing their locks while
> > collecting the batch so swap faults can progress. Before writeout, try
> > locking each folio again and recheck writeback, mappings and DMA pins.
> > Keep eligible folios locked, flush pending writable TLB entries once,
> > then page them out. Process a final partial batch at the end of the scan.
> >
> > Changes
> > -------
> > Patch 1: Extract the folio activation block at activate_locked into
> > folio_activate_locked().
> > Patch 2: Extract the folio-freeing path (buffer release, lazyfree,
> > __remove_mapping, folio_batch drain) into folio_try_reclaim_free().
> > Group retained folios, deferred frees and the reclaimed count
> > in reclaim_folio_state, local to each shrink_folio_list() call.
> > Patch 3: Extract the pageout() dispatch state machine into folio_try_pageout().
> > Patch 4: Extract the TTU setup and try_to_unmap() block into folio_try_unmap().
> > Patch 5: Implement batch TLB flushing logic. Dirty folios are accumulated in
> > batches and a single TLB flush is performed for each batch before
> > calling pageout.
> >
> > Testing
> > -------
> > The benchmark script uses stress-ng to compare TLB shootdown behavior before and
> > after this patch. It constrains a stress-ng workload via memcg to force reclaim
> > through shrink_folio_list(), reporting TLB shootdowns and IPIs.
> >
> > Core benchmark command: stress-ng --vm 16 --vm-bytes 2G --vm-keep --timeout 60
> >
> > ==========================================================================
> > batch_dirty_tlb_flush Benchmark Results
> > ==========================================================================
> > Kernel: 7.0.0-rc1+ CPUs: 16
> > MemTotal: 31834M SwapTotal: 8191M
> > memcg limit: 512M alloc: 2G workers: 16 duration: 60s
> > --------------------------------------------------------------------------
> > Metric Before After Delta (abs / %)
> > --------------------------------------------------------------------------
> > bogo ops/s 28238.63 35833.97 +7595.34 (+26.9%)
> > TLB shootdowns 55428953 17621697 -37807256 (-68.2%)
> > Function call IPIs 34073695 14498768 -19574927 (-57.4%)
> > pgscan_anon (pages) 52856224 60252894 7396670 (+14.0%)
> > pgsteal_anon (pages) 29004962 34054753 5049791 (+17.4%)
> > --------------------------------------------------------------------------
> >
> > Suggested-by: Kairui Song <kasong@xxxxxxxxxxx>
> > Signed-off-by: Zhang Peng <bruzzhang@xxxxxxxxxxx>
> > ---
>
> Hi Peng,
>
> Just a quick update: this patch is still on my radar. I ran a
> quick perf test while building the kernel. Without your patch,
> try_to_unmap_flush_dirty() accounted for 1.75% of CPU cycles.
>
> I guess there could be workloads where try_to_unmap_flush_dirty()
> accounts for an even larger share of CPU cycles.
>
> # Children Self Command Shared Object Symbol
> # ........ ........ ....... ................. ............................
> #
>
> 1.75% 0.01% cc1 [kernel.kallsyms] [.] try_to_unmap_flush_dirty
> |
> --1.74%--try_to_unmap_flush_dirty
> try_to_unmap_flush (inlined)
> |
> --1.73%--arch_tlbbatch_flush
> |
> --1.70%--flush_tlb_multi (inlined)
> __flush_tlb_multi (inlined)
> |
> --1.69%--native_flush_tlb_multi
> |
> --1.68%--on_each_cpu_cond_mask
> |
> --1.62%--smp_call_function_many_cond
> |
> --1.23%--csd_lock_wait (inlined)
>
>
> With your patch, the CPU overhead drops by roughly 7x:
>
>
> # Children Self Command Shared Object Symbol
> # ........ ........ ....... ................. ............................
> #
> 0.25% 0.00% cc1 [kernel.kallsyms] [.] try_to_unmap_flush_dirty
> |
> --0.24%--try_to_unmap_flush_dirty
> try_to_unmap_flush (inlined)
> |
> --0.24%--arch_tlbbatch_flush
> |
> --0.24%--flush_tlb_multi (inlined)
> __flush_tlb_multi (inlined)
> |
> --0.24%--native_flush_tlb_multi
> |
> --0.23%--on_each_cpu_cond_mask
> |
> --0.21%--smp_call_function_many_cond
> |
> --0.14%--csd_lock_wait (inlined)
>
> 0.00% 0.00% fixdep [kernel.kallsyms] [.] try_to_unmap_flush_dirty
>
>
> Overall, I think this is a good approach. I'll be traveling soon
> and will return to this patchset for more testing and a thorough
> review in about two and a half weeks.
>
> Thanks
> Barry
>
Hi Barry,
Thanks for the update and for taking the time to run the perf test.
The breakdown is helpful, and it's encouraging to see that
try_to_unmap_flush_dirty() accounts for a significant share of the
cycles in shrink_folio_list().
Safe travels!
Thanks,
Peng