[RFC PATCH 01/19] vfio/pci: Add PCI error recovery support state

[Shameer Kolothum]

Add the state, the lock and the wait queue that the rest of the series
needs. Nothing uses any of it yet. The lifecycle and the access guards
come next, the recovery callbacks later.

The flags word records what happened in one event. IN_PROGRESS while
recovery is running, FROZEN if the channel was frozen, RESET if the host
reset the device, and FAILED if it did not recover. These are internal.
A later patch reports the same set to userspace along with a sequence
number, so an event can be told from the one before it.

Where both locks are held, recovery_lock goes outside memory_lock.
Recovery has to shut out new device access and wait for whatever is
already running before it can take memory_lock and revoke the BAR
mappings. The other way round deadlocks. An access path sits on
memory_lock waiting for recovery_lock, while recovery sits on
recovery_lock waiting for memory_lock. So nothing may take recovery_lock
when it already holds memory_lock.

Only the generic vfio-pci driver advertises support, so variant drivers
carry on as they do now. Userspace activation is a separate switch, so
even on generic vfio-pci nothing changes until userspace asks for it.

Signed-off-by: Shameer Kolothum <skolothumtho@xxxxxxxxxx>
---
 include/linux/vfio_pci_core.h    | 59 ++++++++++++++++++++++++++++++++
 drivers/vfio/pci/vfio_pci.c      |  1 +
 drivers/vfio/pci/vfio_pci_core.c |  2 ++
 3 files changed, 62 insertions(+)

diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index 9a1674c152aa..42a77ed6b93c 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -95,6 +95,11 @@ static inline int vfio_pci_core_get_dmabuf_phys(
 }
 #endif
 
+#define VFIO_PCI_RECOVERY_IN_PROGRESS	BIT(0)
+#define VFIO_PCI_RECOVERY_FROZEN	BIT(1)
+#define VFIO_PCI_RECOVERY_RESET		BIT(2)
+#define VFIO_PCI_RECOVERY_FAILED	BIT(3)
+
 struct vfio_pci_core_device {
 	struct vfio_device	vdev;
 	struct pci_dev		*pdev;
@@ -129,6 +134,7 @@ struct vfio_pci_core_device {
 	bool			disable_idle_d3:1;
 	bool			nointxmask:1;
 	bool			disable_vga:1;
+	bool			pci_recovery_supported:1;
 	/* Flags modified at runtime - dedicated storage unit */
 	bool			needs_reset;
 	bool			pm_intx_masked;
@@ -147,7 +153,60 @@ struct vfio_pci_core_device {
 	struct list_head		sriov_pfs_item;
 	struct vfio_pci_core_device	*sriov_pf_core_dev;
 	struct notifier_block	nb;
+	/*
+	 * Serializes host PCI error recovery with device access and the
+	 * open/close lifecycle. recovery_lock nests outside memory_lock.
+	 */
+	struct rw_semaphore	recovery_lock;
 	struct rw_semaphore	memory_lock;
+	/*
+	 * PCI error recovery state, written under recovery_lock held for
+	 * writing except where noted.
+	 *
+	 * Some readers cannot take recovery_lock. An interrupt handler cannot
+	 * sleep. The ioeventfd write runs on a workqueue which is flushed with
+	 * the lock held, so it would block behind a queued writer. A
+	 * wait-queue condition cannot take it either. Some checks would refuse
+	 * work against ordinary device traffic if they took it. Work deferred
+	 * past the guard has none to take, since it reaches pci_bus_sem.
+	 *
+	 * Those readers use READ_ONCE() on pci_recovery_flags,
+	 * pci_recovery_enabled, pci_recovery_access_blocked and
+	 * pci_recovery_device_open instead. All of them fail safe. A stale
+	 * read costs an extra refusal or retry, never an unguarded access.
+	 *
+	 * recovery_lock does not exclude those readers, so publish each field
+	 * with a single store of its final value rather than clearing and
+	 * then setting. That keeps the states a reader can observe to ones
+	 * which are meaningful on their own.
+	 *
+	 * device_open and access_blocked mean different things. device_open
+	 * says the device is open and its per-open state, vconfig included,
+	 * is allocated. access_blocked says a recovery or reset is blocking
+	 * access right now.
+	 *
+	 * access_blocked is only ever set while device_open is set. Nothing
+	 * sets it without testing device_open first under recovery_lock, and
+	 * close clears access_blocked before it clears device_open, so a
+	 * block never outlives the open which created it. A reader which
+	 * finds access_blocked set can rely on the per-open state being
+	 * there with it.
+	 */
+	u32			pci_recovery_flags;
+	u64			pci_recovery_sequence;
+	/* PCI_COMMAND value saved before recovery quiesces the device. */
+	u16			pci_recovery_command;
+	/* Userspace enabled recovery for this device open. */
+	bool			pci_recovery_enabled;
+	/* pci_recovery_command contains a restorable value. */
+	bool			pci_recovery_command_valid;
+	/* A recovery or reset transaction is blocking physical access. */
+	bool			pci_recovery_access_blocked;
+	/* Device initialization completed and close teardown has not started. */
+	bool			pci_recovery_device_open;
+	/* May be set while recovery_lock is held for reading during ROM unmap. */
+	bool			pci_recovery_rom_disable;
+	wait_queue_head_t	pci_recovery_wait;
 	struct list_head	dmabufs;
 };
 
diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
index 830369ff878d..46544dbe70d0 100644
--- a/drivers/vfio/pci/vfio_pci.c
+++ b/drivers/vfio/pci/vfio_pci.c
@@ -139,6 +139,7 @@ static int vfio_pci_init_dev(struct vfio_device *core_vdev)
 	 */
 	vdev->nointxmask = nointxmask;
 	vdev->disable_idle_d3 = disable_idle_d3;
+	vdev->pci_recovery_supported = true;
 #ifdef CONFIG_VFIO_PCI_VGA
 	vdev->disable_vga = disable_vga;
 #endif
diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 6757054e9d87..e0be5ddf7039 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -2196,7 +2196,9 @@ int vfio_pci_core_init_dev(struct vfio_device *core_vdev)
 	if (ret && ret != -EOPNOTSUPP)
 		return ret;
 	INIT_LIST_HEAD(&vdev->dmabufs);
+	init_rwsem(&vdev->recovery_lock);
 	init_rwsem(&vdev->memory_lock);
+	init_waitqueue_head(&vdev->pci_recovery_wait);
 	xa_init(&vdev->ctx);
 
 	return 0;
-- 
2.43.0