Re: [RFC PATCH 1/5] PCI: Refuse function reset of an SR-IOV PF with enabled VFs

[Samiullah Khawaja]

On Tue, Aug 18, 2026 at 02:39:06PM -0600, Alex Williamson wrote:
> On Tue, 18 Aug 2026 11:03:04 -0300
> Jason Gunthorpe <jgg@xxxxxxxx> wrote:
>
> > On Tue, Aug 18, 2026 at 07:07:20AM +0000, Tian, Kevin wrote:
> > > 3) resetting a PF due to ad-hoc FW error
> > >
> > > e.g. pdsc_check_pci_health() calls pci_reset_function() when FW
> > > reports broken PCI. Sounds a similar condition to AER.
> > >
> > > but as it calls pci_reset_function() this series causes a regression
> > > in case of active VFs. Is it desired?
> >
> > mlx5 does something similar. RAS flows require detecting the PF has
> > blown up and resetting it to bring it back.
> >
> > This has to work while VFs exist and bound to kernel drivers or vfio.
> >
> > In the VFIO KVM case I would expect the VM to experiance a RAS error
> > on its vPCI device and trigger a VF flr
>
> This would need to be a device level RAS flow as we're sorely lacking
> on forwarding reset notification to VF drivers or providing any
> mechanism for recovery from the vfio error eventfd.
>
> > But we have to sort all this out through the hypervisor to not crash
> > the machine or otherwise :)
>
> In general guarding the high level pci_reset_function() family on
> (pci_num_vfs() == 0) seems to have more pros than cons and the
> restriction matches the scoped intent of the interface.
>
> The low level __pci_reset_function_locked() interface remains
> untouched, and as the prefix implies, relies on the caller to do
> necessary verification and preparation.  A "reset the PF regardless of
> VFs" seems like an underscore prefix sort of operation.  We'll need to
> export the pci_dev_save_and_disable() and pci_dev_restore() functions
> to provide handlers the full suite of tools they need to replace the
> existing function and port those we find (maybe also questioning the
> author's intent on VF reset and recovery).
>
> Of course making the guard parameterized with wrappers to expand this
> family of calls is also an option,
> pci_reset_function_locked_with_vfs()...
>
> The mlx5 driver doesn't actually seem to be affected by these changes,
> doing its own link toggle or a PCI bus reset, not a
> pci_reset_function() call.
>
> On the other improvements, triggering a pci_reset_function_locked() on
> unbind after SR-IOV is disabled, when needs_reset is indicated, looks
> pretty trivial.
>
> The blocking domain operation looks like it might be simplest to
> implement in the IOMMU core.  We can set a flag for a default blocking
> domain on the IOMMU group when we take_dma_ownership of the group.  Then
> release_dma_ownership picks the blocking rather than default domain.
>
> This is then unwound in use_default_domain, called via dma_configure,
> attaching the device to the default domain in probe of the next driver.
> Therefore until probe by another driver, a device used by vfio would
> remain in a blocking domain even while unused and unbound.

The devices are expected to be attached to the default_domain even when
these are unbound and the use_default_domain assumes that, and it only
checks the ownership and doesn't switch the domain to default_domain. I
guess we should add a WARN in use_default_domain() if that is not true.
I will probably send out a patch for that separately.

I think we can move the device back to default_domain after reset after
unbind, maybe it can be done in pci_dma_cleanup() based on
driver_managed_dma?
> Does that seem reasonable (Cc +iommu)?  Thanks,
>
> Alex

Thanks,
Sami