Re: [PATCH mm-unstable v1 2/3] mm/migrate.c: Prevent folio splitting from interacting with KSM

[Lorenzo Stoakes]

On Tue, Jun 09, 2026 at 05:46:14AM -0600, Nico Pache wrote:
> Since commit b1f202060afe ("mm: remap unused subpages to shared zeropage
> when splitting isolated thp"), splitting an anonymous THP remaps all
> zero-filled subpages to the shared zeropage via TTU_USE_SHARED_ZEROPAGE.
> This flag is set unconditionally for every anonymous folio split,
> including splits triggered by KSM.
>
> When KSM is enabled with THP=always, this causes two regressions:
>
> 1. use_zero_pages=1: KSM calls try_to_merge_one_page() which triggers
>    split_huge_page(). The split remaps all 512 zero-filled subpages to
>    the shared zeropage at once, freeing the entire 2MB THP when KSM only
>    intended to process a single 4KB page. This bypasses KSM's
>    pages_to_scan rate limiting, causing ~1GB to be freed almost
>    instantly.
>
> 2. use_zero_pages=0: The same split side-effect occurs through the
>    stable/unstable tree merge paths. Each pages_to_scan iteration
>    triggers an expensive split_huge_page() that silently frees 2MB,
>    while the scanner wastes cycles on tree searches for zero-filled
>    pages that were already freed as a side-effect.
>
> Fix this by restricting TTU_USE_SHARED_ZEROPAGE being set in the case that
> KSM is running and the VMA has VM_MERGEABLE.
>
> Fixes: b1f202060afe ("mm: remap unused subpages to shared zeropage when splitting isolated thp")
> Signed-off-by: Nico Pache <npache@xxxxxxxxxx>
> ---
>  mm/migrate.c | 9 +++++++++
>  1 file changed, 9 insertions(+)
>
> diff --git a/mm/migrate.c b/mm/migrate.c
> index d9b23909d716..f410f972fc5e 100644
> --- a/mm/migrate.c
> +++ b/mm/migrate.c
> @@ -304,6 +304,15 @@ static bool try_to_map_unused_to_zeropage(struct page_vma_mapped_walk *pvmw,
>  	if (PageCompound(page) || PageHWPoison(page))
>  		return false;
>
> +	/*
> +	 * Let KSM handle the zero-filled page deduplication according to its
> +	 * own rate limit (pages_to_scan) and policy (use_zero_pages). Without
> +	 * this, a KSM-triggered THP split would remap all zero-filled subpages
> +	 * to the shared zeropage as a side effect.
> +	 */
> +	if (ksm_is_running() && (pvmw->vma->vm_flags & VM_MERGEABLE))

Please use the new VMA flag API.

This would be vma_test(pvmw->vma, VMA_MERGEABLE_BIT).

> +		return false;
> +
>  	VM_BUG_ON_PAGE(!PageAnon(page), page);
>  	VM_BUG_ON_PAGE(!PageLocked(page), page);
>  	VM_BUG_ON_PAGE(pte_present(old_pte), page);
> --
> 2.54.0
>

Thanks, Lorenzo