Re: [PATCH v3 1/6] mm: khugepaged: ignore MADV_HUGEPAGE for shadow stacks
From: Yeoreum Yun
Date: Mon Nov 09 2026 - 10:30:26 EST
> On 11/9/26 13:32, Yeoreum Yun wrote:
> > commit 3efb9800557b ("mm: do not map the shadow stack as THP") makes
> > shadow stack mappings default to VM_NOHUGEPAGE. However, userspace can
> > still clear this flag and set VM_HUGEPAGE through madvise().
> >
> > Allowing large folios in x86 shadow stacks exposes a limitation in
> > folio_pte_batch_flags(). For example:
> >
> > 1. create a shadow stack mapping.
> > 2. apply MADV_HUGEPAGE and populate the mapping with a PTE-mapped
> > large folio.
> > 3. apply MADV_DONTFORK to the first base page of that folio.
> > 4. call fork().
> >
> > In the parent, the first page remains writable and anonymous-exclusive,
> > while the remaining pages shared with the child become write-protected
> > and lose PageAnonExclusive(). These pages can still belong to the same
> > large folio.
> >
> > On x86, writable shadow-stack PTEs have _PAGE_RW clear and _PAGE_DIRTY
> > set. Write-protected PTEs have both bits clear, with the dirty state
> > preserved in _PAGE_SAVED_DIRTY.
> >
> > When FPB_RESPECT_WRITE is set without FPB_RESPECT_DIRTY,
> > folio_pte_batch_flags() normalizes PTEs using pte_mkclean(), which clears
> > both dirty bits. This loses the distinction between writable
> > shadow-stack PTEs and write-protected PTEs, allowing them to be grouped
> > into the same batch. GUP-fast can encounter such a batch when walking
> > across the VMA boundary created by MADV_DONTFORK.
> >
> > Since there is no practical use-case shadow stack with large folio,
> > ignore MADV_HUGEPAGE for shadow stack mappings, preserving their
> > VM_NOHUGEPAGE setting and preventing large-folio allocation.
> >
> > Fixes: 3efb9800557b ("mm: do not map the shadow stack as THP")
> > Suggested-by: David Hildenbrand (Arm) <david@xxxxxxxxxx>
> > Signed-off-by: Yeoreum Yun <yeoreum.yun@xxxxxxx>
> > Cc: <stable@xxxxxxxxxxxxxxx>
> > ---
> > mm/khugepaged.c | 3 +++
> > 1 file changed, 3 insertions(+)
> >
> > diff --git a/mm/khugepaged.c b/mm/khugepaged.c
> > index 40fad28c52f6f..51ec043fa26c3 100644
> > --- a/mm/khugepaged.c
> > +++ b/mm/khugepaged.c
> > @@ -367,6 +367,9 @@ int hugepage_madvise(struct vm_area_struct *vma,
> > {
> > switch (advice) {
> > case MADV_HUGEPAGE:
> > + /* PTE batching doesn't work reliably. */
> > + if (vma_test_single_mask(vma, VMA_SHADOW_STACK))
> > + break;
> > *vm_flags &= ~VM_NOHUGEPAGE;
> > *vm_flags |= VM_HUGEPAGE;
> > break;
> >
>
> Ack, I prefer this solution for now until we know how to make pte_mkclean() on
> x86 shadow stack PTEs not result in an implicit pte_wrprotect().
Yeap. As we discussed in offline, just skip clearing dirty bit for
shstk in pte_mkclear() wouldn't work since it would make a another
regression -- wouldn't be page-outed.
Might it would be good to use another soft-bit for shstk in x86.
But, for now, it seems enough as you said.
>
> Acked-by: David Hildenbrand (Arm) <david@xxxxxxxxxx>
Thanks!
--
Sincerely,
Yeoreum Yun