Re: [PATCH v6 2/5] mm/page_alloc: only free healthy pages in high-order has_hwpoisoned folio

[Miaohe Lin]

On 2026/7/6 2:07, Jiaqi Yan wrote:
> At the end of dissolve_free_hugetlb_folio(), a free HugeTLB folio
> becomes non-HugeTLB, and it is released to buddy allocator
> as a high-order folio, e.g. a folio that contains 262144 pages
> if the folio was a 1G HugeTLB hugepage.
> 
> This is problematic if the HugeTLB hugepage contained HWPoison
> subpages. In that case, since buddy allocator does not check
> HWPoison for non-zero-order folio, the raw HWPoison page can
> be given out with its buddy page and be re-used by either
> kernel or userspace.
> 
> Memory failure recovery (MFR) in kernel does attempt to take
> raw HWPoison page off buddy allocator after
> dissolve_free_hugetlb_folio(). However, there is always a time
> window between dissolve_free_hugetlb_folio() frees a HWPoison
> high-order folio to buddy allocator and MFR takes HWPoison
> raw page off buddy allocator.
> 
> Another similar situation is when a transparent huge page (THP)
> runs into memory failure but splitting failed. Such THP will
> eventually be released to buddy allocator when owning userspace
> processes are gone, but with certain subpages having HWPoison.
> 
> One obvious way to avoid both problems is to add page sanity
> checks in page allocate or free path. However, it is against
> the past efforts to reduce sanity check overhead [1,2,3].
> 
> Introduce free_has_hwpoisoned() to only free the healthy pages
> and to exclude the HWPoison ones in the high-order folio.
> The idea is to iterate through the sub-pages of the folio to
> identify contiguous ranges of healthy pages.
> 
> free_has_hwpoisoned() is added at the end of __free_pages_prepare()
> as a shortcut and only if PG_has_hwpoisoned indicates HWPoison page
> exists and after checks and preparations in __free_pages_prepare()
> all succeeded. It then use __free_prepared_contig_range() to
> decompose healthy range into the largest possible chunks of
> different orders, then freed via __free_frozen_pages().
> 
> free_has_hwpoisoned() has linear time complexity wrt the number
> of pages in the folio. While the power-of-two decomposition
> ensures that the number of calls to the buddy allocator is
> logarithmic for each contiguous healthy range, the mandatory
> linear scan of pages to identify PageHWPoison() defines the
> overall time complexity. For a 1G hugepage having 8 HWPoison
> pages, free_has_hwpoisoned() takes around 1ms on average on
> a system having 56 Intel Skylake physical cores. This is
> 15x to the case of freeing no HWPoison page. The cost is far
> from triggering soft lockup, and fair for handling exceptional
> hardware memory errors.
> 
> [1] https://lore.kernel.org/linux-mm/1460711275-1130-15-git-send-email-mgorman@xxxxxxxxxxxxxxxxxxx
> [2] https://lore.kernel.org/linux-mm/1460711275-1130-16-git-send-email-mgorman@xxxxxxxxxxxxxxxxxxx
> [3] https://lore.kernel.org/all/20230216095131.17336-1-vbabka@xxxxxxx
> 
> Signed-off-by: Jiaqi Yan <jiaqiyan@xxxxxxxxxx>

Reviewed-by: Miaohe Lin <linmiaohe@xxxxxxxxxx>

Nits:

> ---
>  mm/page_alloc.c | 165 ++++++++++++++++++++++++++++++++++++++++--------
>  1 file changed, 140 insertions(+), 25 deletions(-)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 7d27aff48b15..6418896c9df6 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -93,6 +93,13 @@ typedef int __bitwise fpi_t;
>  /* free_pages_prepare() has already been called for page(s) being freed. */
>  #define FPI_PREPARED		((__force fpi_t)BIT(3))
>  
> +/*
> + * Page(s) needs to go through free_pages_sanitize(), for example, because
> + * free_pages_prepare() cannot sanitize a high-order page block due to
> + * hardware error in some page(s).
> + */
> +#define FPI_SANITIZE		((__force fpi_t)BIT(4))
> +
>  /* prevent >1 _updater_ of zone percpu pageset ->high and ->batch fields */
>  static DEFINE_MUTEX(pcp_batch_high_lock);
>  #define MIN_PERCPU_PAGELIST_HIGH_FRACTION (8)
> @@ -210,6 +217,8 @@ gfp_t gfp_allowed_mask __read_mostly = GFP_BOOT_MASK;
>  unsigned int pageblock_order __read_mostly;
>  #endif
>  
> +static void free_has_hwpoisoned(struct page *page, unsigned int order,
> +				fpi_t fpi_flags);
>  static void __free_pages_ok(struct page *page, unsigned int order,
>  			    fpi_t fpi_flags);
>  static void reserve_highatomic_pageblock(struct page *page, int order,
> @@ -1311,14 +1320,73 @@ static inline void pgalloc_tag_sub_pages(struct alloc_tag *tag, unsigned int nr)
>  
>  #endif /* CONFIG_MEM_ALLOC_PROFILING */
>  
> +/*
> + * Sanitize, which requires writing, a block of pages at the last moment of
> + * preparing to freeing them, i.e. __free_pages_prepare().

s/freeing/free/ ?

> + */
> +static void free_pages_sanitize(struct page *page, unsigned int order)
> +{
> +	bool init = want_init_on_free();
> +	/*
> +	 * __kasan_unpoison_pages() sets kasan tag on every tail page, so
> +	 * it is fine to use should_skip_kasan_poison() when pages here
> +	 * were a set of tail pages from a compound folio.
> +	 */
> +	bool skip_kasan_poison = should_skip_kasan_poison(page);
> +
> +	kernel_poison_pages(page, 1 << order);
> +
> +	/*
> +	 * As memory initialization might be integrated into KASAN,
> +	 * KASAN poisoning and memory initialization code must be
> +	 * kept together to avoid discrepancies in behavior.
> +	 *
> +	 * With hardware tag-based KASAN, memory tags must be set before the
> +	 * page becomes unavailable via debug_pagealloc or arch_free_page.
> +	 */
> +	if (!skip_kasan_poison) {
> +		kasan_poison_pages(page, order, init);
> +
> +		/* Memory is already initialized if KASAN did it internally. */
> +		if (kasan_has_integrated_init())
> +			init = false;
> +	}
> +	if (init)
> +		clear_highpages_kasan_tagged(page, 1 << order);
> +
> +	/*
> +	 * arch_free_page() can make the page's contents inaccessible.  s390
> +	 * does this.  So nothing which can access the page's contents should
> +	 * happen after this.
> +	 */
> +	arch_free_page(page, order);
> +
> +	debug_pagealloc_unmap_pages(page, 1 << order);
> +}
> +
> +/*
> + * Returns
> + * - true: checks and preparations all good, caller can proceed freeing.
> + * - false: do not proceed freeing for one of the following reasons:
> + *   1. Some check failed so it is not safe to proceed freeing.
> + *   2. A compound page has some HWPoison pages. The healthy pages
> + *      are already safely freed, and the HWPoison ones isolated.
> + */
>  static __always_inline bool __free_pages_prepare(struct page *page,
>  		unsigned int order, fpi_t fpi_flags)
>  {
>  	int bad = 0;
> -	bool skip_kasan_poison = should_skip_kasan_poison(page);
> -	bool init = want_init_on_free();
>  	bool compound = PageCompound(page);
>  	struct folio *folio = page_folio(page);
> +	/*
> +	 * When dealing with compound page, PG_has_hwpoisoned is cleared
> +	 * with PAGE_FLAGS_SECOND. So the check must be done first.
> +	 *
> +	 * Note we can't exclude PG_has_hwpoisoned from PAGE_FLAGS_SECOND.
> +	 * Because PG_has_hwpoisoned == PG_active, free_page_is_bad() will
> +	 * confuse and complaint that the first tail page is still active.
> +	 */
> +	bool should_fhh = compound && folio_test_has_hwpoisoned(folio);
>  
>  	if (fpi_flags & FPI_PREPARED)
>  		return true;
> @@ -1416,34 +1484,19 @@ static __always_inline bool __free_pages_prepare(struct page *page,
>  					   PAGE_SIZE << order);
>  	}
>  
> -	kernel_poison_pages(page, 1 << order);
> -
>  	/*
> -	 * As memory initialization might be integrated into KASAN,
> -	 * KASAN poisoning and memory initialization code must be
> -	 * kept together to avoid discrepancies in behavior.
> +	 * After breaking down compound page and dealing with page metadata
> +	 * (e.g. page owner and page alloc tags), take a shortcut if this
> +	 * was a compound page containing certain HWPoison subpages.
>  	 *
> -	 * With hardware tag-based KASAN, memory tags must be set before the
> -	 * page becomes unavailable via debug_pagealloc or arch_free_page.
> +	 * FPI_SANITIZE to remember free_pages_sanitize() healthy pages.
>  	 */
> -	if (!skip_kasan_poison) {
> -		kasan_poison_pages(page, order, init);
> -
> -		/* Memory is already initialized if KASAN did it internally. */
> -		if (kasan_has_integrated_init())
> -			init = false;
> +	if (should_fhh) {
> +		free_has_hwpoisoned(page, order, fpi_flags | FPI_SANITIZE);
> +		return false;
>  	}
> -	if (init)
> -		clear_highpages_kasan_tagged(page, 1 << order);
> -
> -	/*
> -	 * arch_free_page() can make the page's contents inaccessible.  s390
> -	 * does this.  So nothing which can access the page's contents should
> -	 * happen after this.
> -	 */
> -	arch_free_page(page, order);
>  
> -	debug_pagealloc_unmap_pages(page, 1 << order);
> +	free_pages_sanitize(page, order);
>  
>  	return true;
>  }
> @@ -6869,7 +6922,14 @@ static void __free_prepared_contig_range(struct page *page,
>  		/*
>  		 * Free the chunk as a single block. Our caller has already
>  		 * called free_pages_prepare() for each order-0 page.
> +		 *
> +		 * If the original compound page has HWPoison page,
> +		 * free_pages_prepare() has to skip sanitize at that time,
> +		 * but now it is good time to do that.
>  		 */
> +		if (fpi_flags & FPI_SANITIZE)
> +			free_pages_sanitize(page, order);
> +
>  		__free_frozen_pages(page, order, fpi_flags | FPI_PREPARED);
>  
>  		pfn += 1UL << order;
> @@ -6956,6 +7016,61 @@ void __free_contig_range(unsigned long pfn, unsigned long nr_pages)
>  	__free_contig_range_common(pfn, nr_pages, /* is_frozen= */ false);
>  }
>  
> +/*
> + * Given some contiguous pages that have certain number of HWPoison page(s),
> + * free only the healthy ones.
> + *
> + * Used at the end of __free_pages_prepare(). Even if having HWPoison pages,
> + * breaking down compound page and clearing metadata (e.g. page owner, alloc
> + * tag) can be done together during __free_pages_prepare(), which simplifies
> + * the splitting here: unlike __split_unmapped_folio(), there is no need to
> + * turn split pages into a compound page or to carry metadata.
> + *
> + * It scans every raw page of the compound page and causes nontrivial overhead.
> + * So only use this when the compound page contains HWPoison page(s).
> + *
> + * It also works when order == 0, regardless of PageHWPoison() or not.

It works but won't be called when order == 0?

Thanks.
.