Re: [RFC PATCH v2 00/32] Add live update state preservation

[Samiullah Khawaja]

On Wed, Jan 28, 2026 at 11:59 AM Jason Gunthorpe <jgg@xxxxxxxx> wrote:
>
> On Tue, Dec 02, 2025 at 11:02:30PM +0000, Samiullah Khawaja wrote:
> >
> > Samiullah Khawaja (26):
> >   iommu: Add liveupdate FLB for IOMMU state preservation
> >   iommu: Register IOMMU FLB with iommufd file handler
> >   iommu: Implement IOMMU LU FLB preserve/unpreserve callbacks
> >   iommu: Add iommu_domain ops to preserve, unpreserve and restore
> >   iommu/pages: Add APIs to preserve/unpreserve/restore iommu pages
> >   iommupt: Implement preserve/unpreserve/restore callbacks
> >   iommu: Add APIs to preserve/unpreserve iommu domains
> >   iommufd: Use the iommu_domain_preserve/unpreserve APIs
> >   iommu: Add API to keep track of iommu domain attachments
> >   iommu: Add API to preserve/unpreserve a device
> >   iommu/vt-d: Implement device and iommu preserve/unpreserve ops
> >   iommufd: Add APIs to preserve/unpreserve a vfio cdev
> >   vfio/pci: Preserve the iommufd state of the vfio cdev
> >   iommu: Add APIs to get preserved state of a device
> >   iommu/vt-d: Clean the context entries of unpreserved devices
> >   iommu: Implement IOMMU FLB retrieve and finish ops
> >   iommu: Add an API get the preserved state of an IOMMU
> >   iommu/vt-d: restore state of the preserved IOMMU
> >   iommu: Add helper APIs to fetch preserved device state
> >   iommu/vt-d: reclaim domain ids of the preserved devices
> >   iommu: restore preserved domain and reattach
> >   iommu/vt-d: reuse the preserved domain id for preserved devices
> >   iommufd: Handle the iommufd can_finish properly
> >   iommu: Transfer device ownership after liveupdate
> >   iommu: Allow replacing restored domain
> >   iommufd/selftest: Add test to verify iommufd preservation
> >
> > YiFei Zhu (6):
> >   iommufd: Allow HWPTs to have a NULL IOAS
> >   iommufd: split alloc and domain assign from iommufd_hwpt_paging_alloc
> >   iommufd: Add basic skeleton based on liveupdate_file_handler
> >   iommufd-lu: Implement basic prepare/cancel/finish/retrieve using
> >     folios
> >   iommufd-lu: Implement ioctl to let userspace mark an HWPT to be
> >     preserved
> >   iommufd-lu: Persist iommu hardware pagetables for live update
>
> This really needs to be made smaller and more focused to be
> reviewable and mergable. Try to stick to 15-ish patches.

Agreed. I will restructure this into a more focused series.
>
> There is a lot giong on here, I suggest focusing this only on the main
> iommu core, a vt-d driver implementation of the bare minimum, and an
> iommufd function to trigger preservation of a domain.

The preservation is done into FLB and that is bound to the iommufd
file handler. So the phase 1 series will need some boiler plate logic
to trigger preserve/unpreserve. It will also need checking that memfds
are preserved and SEALd to make it robust. But I will try to keep it
to a minimum.
>
> Start off by just making the successor kernel fail to accept any
> drivers at all because the iommu_domain was preserved. ie restore the
> domain and set it as the default_domain and then fail in
> iommu_device_use_default_domain() and related functions.

Right. In this model, the device would remain unusable after
liveupdate as you cannot do session finish. So the only way to recover
would be to do a proper reboot. But I think this is a great
intermediate step.
>
> All it should do is convey an iommu_domain and the pasid table entry
> unchanged without hit from prior to successor kernel. That's the bare
> minimum.
>
> Then a second series which picks up from there and feeds the sucessor path
> through iommufd/vfio.

Yes this sounds like a great split of series. I experimented with this
a little and it is workable. But like I mentioned earlier, some boiler
plate will be needed.
>
> Jason