Re: [PATCH] mm/migrate: fix hugetlbfs deadlock by respecting lock ordering

[David Hildenbrand (Red Hat)]

On 1/9/26 16:32, Jinchao Wang wrote:
> On Fri, Jan 09, 2026 at 03:18:37PM +0100, David Hildenbrand (Red Hat) wrote:
> > On 1/9/26 15:16, Jinchao Wang wrote:
> > > On Fri, Jan 09, 2026 at 02:39:08PM +0100, David Hildenbrand (Red Hat) wrote:
> > > > On 1/9/26 04:47, Jinchao Wang wrote:
> > > > > Fix an AB-BA deadlock between hugetlbfs_punch_hole() and page migration.
> > > > >
> > > > > The deadlock occurs because migration violates the lock ordering defined
> > > > > in mm/rmap.c for hugetlbfs:
> > > > >
> > > > >      * hugetlbfs PageHuge() take locks in this order:
> > > > >      * hugetlb_fault_mutex
> > > > >      * vma_lock
> > > > >      * mapping->i_mmap_rwsem
> > > > >      * folio_lock
> > > > >
> > > > > The following trace illustrates the inversion:
> > > > >
> > > > > Task A (punch_hole):             Task B (migration):
> > > > > --------------------             -------------------
> > > > > 1. i_mmap_lock_write(mapping)    1. folio_lock(folio)
> > > > > 2. folio_lock(folio)             2. i_mmap_lock_read(mapping)
> > > > >       (blocks waiting for B)           (blocks waiting for A)
> > > > >
> > > > > Task A is blocked in the punch-hole path:
> > > > >      hugetlbfs_fallocate
> > > > >        hugetlbfs_punch_hole
> > > > >          hugetlbfs_zero_partial_page
> > > > >            folio_lock
> > > > >
> > > > > Task B is blocked in the migration path:
> > > > >      migrate_pages
> > > > >        unmap_and_move_huge_page
> > > > >          remove_migration_ptes
> > > > >            __rmap_walk_file
> > > > >              i_mmap_lock_read
> > > > >
> > > > > To fix this, adjust unmap_and_move_huge_page() to respect the established
> > > > > hierarchy. If i_mmap_rwsem is acquired during try_to_migrate(), hold it
> > > >
> > > >
> > > > I'm confused. Isn't it unmap_and_move_huge_page() that grabs the
> > > > i_mmap_rwsem during hugetlb_page_mapping_lock_write() (where we do a
> > > > try-lock)?
> > > Yes, but the lock is released before remove_migration_ptes().
> > >
> > > Task A can enter the race window between
> > > 	i_mmap_unlock_write(mapping)
> > > and
> > > 	remove_migration_ptes() -> i_mmap_lock_read(mapping).
> > >
> > > This window was introduced by the change below:
> > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/diff/mm/migrate.c?id=336bf30eb765
> >
> > try_to_migrate() is not the problem, but remove_migration_ptes() ?
> >
> > Anyhow, I saw that Willy sent out a version.
> Thank you for letting me know.

For reference:

https://lkml.kernel.org/r/20260109041345.3863089-1-willy@xxxxxxxxxxxxx

--
Cheers

David