Re: [RFC PATCH v1 0/1] vfio/pci: Revoke BARs and DMABUFs during sysfs-triggered PCI reset

[Pranjal Shrivastava]

On Mon, Aug 10, 2026 at 10:03:52AM -0600, Alex Williamson wrote:

Hi Alex,

> On Fri,  7 Aug 2026 20:14:04 +0000
> Pranjal Shrivastava <praan@xxxxxxxxxx> wrote:
> 
> > Introduce PCI .reset_prepare and .reset_done handlers to safely revoke
> > active userspace mappings and exported DMABUFs during sysfs-triggered
> > device resets.
> > 
> > We are seeing a situation where system health and monitoring daemons 
> > (at times erroneously) issue device resets via sysfs for devices bound
> > to vfio-pci:
> > 
> >   echo 1 > /sys/bus/pci/devices/0000:01:00.0/reset
> > 
> > However, because vfio-pci does not implement the .reset_prepare and
> > .reset_done error handlers, this hardware reset occurs completely unnoticed
> > by the VFIO driver.
> > 
> > Consequently, active traditional userspace BAR mappings and exported DMABUFs
> > are never zapped or revoked. Importers of the DMABUFs (e.g., RDMA drivers)
> > continue to issue DMAs (such as PCIe Memory Writes) toward the Endpoint. 
> > These transactions are silently dropped by the root port or trigger CTOs 
> > while higher-level actions (e.g., RDMA reg_mr) continue to succeed.
> > 
> > We'd like to fix this by implementing the PCI reset ops for vfio-pci
> > that revoke the DMABUFs and zap the BARs while holding the memory lock
> > allowing concurrent user accesses to sleep and fault back in once the reset
> > completes.
> 
> That sounds like a nice, serene solution, but that's not actually what
> happens.  Due to the write vs read memory_lock semaphore, CPU faults
> are stalled.  On the other hand, DMA mappings via IOMMUFD/dmabuf are
> lost.  They require the userspace driver to be involved to perform the
> unmap/remap.
> 
> Potentially this is all better than letting the device generate a
> machine check as it's still trying to run across the reset, but let's
> not pretend this is just a hiccup for the device that will continue
> running after the rogue reset.  Thanks,
> 

I tend to agree. My intention is definitely not to pretend this is a seamless
hiccup or allow the device/user to carry on as if nothing happened. In fact,
the very problem today with exported DMABUFs is that the user/importer *does*
silently continue to register DMABUFs with the RDMA subsystem and attempts
issuing DMAs to a reset device because nothing told them the state was gone.

I don't mind permanently tearing down the CPU mappings as well along with
revoking the DMABUFs. That way, we fail loudly and force userspace to unmap
and re-initialize (with a dev_warn() explaining that an out-of-band reset
occurred).

What do you think about that approach?

Thanks,
Praan