Re: [PATCH] KVM: guest_memfd: Disable VMA merging with VM_DONTEXPAND

[Deepanshu Kartikey]

On Sun, Feb 8, 2026 at 11:04 PM Ackerley Tng <ackerleytng@xxxxxxxxxx> wrote:
>

> Since this also affects secretmem, I think thp_vma_allowable_order() is
> the best place to intercept the collapsing flow for both secretmem and
> guest_memfd.
>
> Let me know if you have any ideas!
>

Hi David, Ackerley,

I have been looking into this bug and I think the root cause is in
file_thp_enabled(). When CONFIG_READ_ONLY_THP_FOR_FS is enabled,
guest_memfd and secretmem inodes pass the S_ISREG() and
!inode_is_open_for_write() checks, so file_thp_enabled() incorrectly
returns true. This allows khugepaged and MADV_COLLAPSE to create large
folios in the page cache.

I sent a patch that fixes this at the source by explicitly rejecting
GUEST_MEMFD_MAGIC and SECRETMEM_MAGIC in file_thp_enabled():

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 40cf59301c21..4f57c78b57dd 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -93,6 +93,9 @@ static inline bool file_thp_enabled(struct
vm_area_struct *vma)
  return false;

  inode = file_inode(vma->vm_file);
+ if (inode->i_sb->s_magic == GUEST_MEMFD_MAGIC ||
+     inode->i_sb->s_magic == SECRETMEM_MAGIC)
+ return false;

  return !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode);
 }

I have tested this and confirmed the warning no longer triggers. This
approach covers both guest_memfd and secretmem in one place without
needing separate VMA flag changes in each subsystem. I have sent the
patch.

Please have a look and let me know your thoughts.

Thanks,
Deepanshu