Re: [RFC PATCH 0/5] iommupt: Introduce IO page table shrinker

[Jason Gunthorpe]

On Thu, Oct 01, 2026 at 11:02:14PM +0000, Pranjal Shrivastava wrote:
> Introduce a lockless, deferred reclamation framework for IOMMU page tables
> built on the generic_pt library. As VMMs and userspace drivers map and
> unmap large, sparse IOVA regions through VFIO and iommufd, page table
> directories are often left allocated but completely empty. generic_pt
> frees a table when a single unmap covers it entirely, but tables that
> empty through a series of partial unmaps stay allocated until the domain
> is destroyed. Under memory pressure, this *stranded* memory cannot be
> reclaimed and has resulted in OOMs.
> 
> This series refcounts leaf directories natively in struct ioptdesc and
> registers a domain-aware MM shrinker that prunes empty directories under
> system memory pressure.

We had talked about doing it this way

But I had proposed a different, and possibly simpler, solution that
addresses *just* the iommufd use case.

After unmapping something have iommufd compute the gap in IOVA that
contains what was unmapped and then issue a 'clean(gap)' operation to
generic_pt.

This is the same operation as unmap, except we know now that the gap
has no PTEs so all it does is clean up the table pointers.

This requires no special refcounting or anything difficult beyond
some locking in iommufd to hold the gap stable while we clean it.

Would it work for you? It seems substantially simpler, but I never
tried to implement it.

An alternative version is closer to what you have here, somehow
connect iommufd to the shrinker and have it lock and walk the gaps
cleaning them on shrink requests?

iommufd has no trouble walking the gaps in the interval tree, there is
a helper that does that computation.

Jason