[PATCH 0/5] vfio/pci: Fix power state race with SR-IOV enablement

[David Matlack]

This series fixes a race in VFIO's handling of device power state
transitions and a related missing data race annotation, both found while
reviewing Raghavendra's sriov_active patch [1]. It also adds a selftest
to exercise VFIO's handling of device power state transitions.

Patch 1 fixes a race between vfio_pci_core_disable() and
vfio_pci_core_sriov_configure(), both of which transition the device to
D0, but only the latter holds memory_lock. If userspace puts the device
into D3hot via config space and then closes the device while
concurrently enabling SR-IOV, both threads can transition the device
from D3hot to D0 at the same time. For devices with NoSoftRst-, this can
result in VFIO restoring the PF's BARs after VFs have been enabled
(triggering a WARN in pci_iov_update_resource()), or in a double-free of
vdev->pm_save. The fix is to hold memory_lock for all calls to
vfio_pci_set_power_state() and to assert it unconditionally.

Patch 2 annotates the intentional lockless accesses to
vdev->sriov_active with READ_ONCE() and WRITE_ONCE().

Patches 3-5 add vfio_pci_power_state_test, which covers closing a device
in D3hot, enabling SR-IOV while the PF is in D3hot, and racing the two.
Since power state bugs often only manifest as kernel warnings, the test
fails if /sys/kernel/warn_count changes during a test case. Patches 3
and 4 add the library helpers the test needs.

Testing
-----------------------------------------------------------------------

Tested in a QEMU VM with an emulated igb device (NoSoftRst-, SR-IOV
capable) bound to vfio-pci:

 - Without patches 1-2 (KASAN kernel), close_in_d3hot_races_sriov_enable
   fails reliably, hitting the pci_iov_update_resource() WARN on 9/10
   iterations.

 - With patches 1-2 (KASAN and lockdep kernels), all VFIO selftests pass
   with no warnings, KASAN reports, or lockdep splats.

[1] https://lore.kernel.org/r/20260514173449.3282188-1-rananta@xxxxxxxxxx

David Matlack (5):
  vfio/pci: Hold memory_lock for all calls to vfio_pci_set_power_state()
  vfio/pci: Annotate lockless accesses to vdev->sriov_active
  vfio: selftests: Add vfio_pci_find_capability()
  vfio: selftests: Add sysfs_kernel_warn_count_get()
  vfio: selftests: Add test for device power state transitions

 drivers/vfio/pci/vfio_pci_core.c              |  19 +-
 tools/testing/selftests/vfio/Makefile         |   1 +
 .../vfio/lib/include/libvfio/sysfs.h          |   7 +
 .../lib/include/libvfio/vfio_pci_device.h     |   7 +
 tools/testing/selftests/vfio/lib/sysfs.c      |  19 +-
 .../selftests/vfio/lib/vfio_pci_device.c      |  25 +++
 .../vfio/vfio_pci_power_state_test.c          | 189 ++++++++++++++++++
 7 files changed, 257 insertions(+), 10 deletions(-)
 create mode 100644 tools/testing/selftests/vfio/vfio_pci_power_state_test.c


base-commit: 076d7e989cb69f6ded7d60f2a080eb909930061b
-- 
2.56.0.317.g454f8302e1-goog