[RFC PATCH 10/19] vfio/pci: Serialize hot reset with recovery

[Shameer Kolothum]

Refuse a user hot reset while a recovery transaction blocks access to any
device in the set. Between the AER callbacks memory_lock is not held, so
access_blocked is the only thing marking the device as unavailable.

A device which has failed for good is let through. Nothing is running on
its behalf, and a hot reset covers the whole set, so refusing there would
stop a healthy sibling from being reset because an unrelated device
errored. The failed device stays blocked either way, since the reset does
not clear access_blocked once FAILED is set.

Do not hold recovery_lock while resetting. pci_reset_bus() reaches
pci_bridge_wait_for_secondary_bus(), which takes pci_bus_sem, and AER
already holds pci_bus_sem when it enters the driver and takes
recovery_lock. Holding recovery_lock across the reset would invert that
order for every device in the set.

Assisted-by: Claude:claude-opus-5
Signed-off-by: Shameer Kolothum <skolothumtho@xxxxxxxxxx>
---
 drivers/vfio/pci/vfio_pci_core.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 876ff51d6987..bd3d79d28f27 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -2937,6 +2937,27 @@ static int vfio_pci_dev_set_hot_reset(struct vfio_device_set *dev_set,
 			break;
 		}
 
+		/*
+		 * Between the AER callbacks memory_lock is not held, so refuse
+		 * the reset on access_blocked as well. Read it rather than
+		 * take recovery_lock, which would have to be released before
+		 * pci_reset_bus() anyway since that reaches pci_bus_sem.
+		 *
+		 * Let a device which has failed for good through. Nothing is
+		 * running on its behalf, and this reset covers the whole set,
+		 * so refusing there would stop a healthy sibling from being
+		 * reset because an unrelated device errored. The failed one
+		 * stays blocked, since the reset does not unblock a device
+		 * with FAILED set.
+		 */
+		if (vdev->pci_recovery_supported &&
+		    READ_ONCE(vdev->pci_recovery_access_blocked) &&
+		    !(READ_ONCE(vdev->pci_recovery_flags) &
+		      VFIO_PCI_RECOVERY_FAILED)) {
+			ret = -EBUSY;
+			break;
+		}
+
 		/*
 		 * Take the memory write lock for each device and zap BAR
 		 * mappings to prevent the user accessing the device while in
-- 
2.43.0