[PATCH v2 0/2] PCI: Wait after FLR for devices that don't honor Immediate Readiness

[Alexander Gruhlke]

Since v7.2, VFIO passthrough of a Samsung 990 PRO NVMe [144d:a80c] fails
on the second VM start, and the device is inaccessible until reboot.  The
drive advertises Immediate Readiness but returns ~0 for up to 7 ms after
FLR, so since 10baa9b4df40 ("PCI: Drop unnecessary retries when restoring
BARs") the config restore after FLR is lost.

Patch 1 ignores Immediate Readiness on that drive.  A 990 EVO Plus
[144d:a80d] reports the same errors [1], but it isn't confirmed to
advertise Immediate Readiness, so it's not included.

Patch 2 is independent and keeps pci_dev_wait() from treating ~0 as
ready with RRS Software Visibility, which also affects the Intel
[8086:0a54] in the open d591f6804e7e regression.  Hui, Linux-Fan: testing
patch 2 on that drive would be appreciated.

Tested on v7.2.9 (Ryzen 9 7950X, X670E, drive firmware 1B2QJXD7): with
patch 1, three VM starts in a row work, and every FLR reports "ready 0ms
after FLR".  After updating the drive to firmware 8B2QJXD7, it still
advertises Immediate Readiness but responds right after FLR, and plain
v7.2.9 works, so only older firmware is affected.

Patch 2 in this form is only tested for regressions, as the drive no
longer shows the problem.  v1's variant, which read the Command register,
was tested against it, and the drive's Vendor ID read ~0 while it wasn't
ready.  Applies to pci/next.

Changes since v1:
- Replace the pcie_flr() change with a quirk for the 990 PRO (Lukas)
- pci_dev_wait(): check the Vendor ID for ~0 except on VFs instead of
  reading the Command register, and "may return" in the comment (Lukas)
- Swap the patch order, as the quirk alone fixes the regression

v1: https://lore.kernel.org/r/20261010160529.27974-1-gruhlke@xxxxxxxxxxx

[1] https://bbs.archlinux.org/viewtopic.php?id=314833

Alexander Gruhlke (2):
  PCI: Ignore Immediate Readiness on Samsung 990 PRO NVMe
  PCI: Don't treat ~0 as ready in pci_dev_wait() with RRS SV

 drivers/pci/pci.c    |  8 +++++---
 drivers/pci/quirks.c | 11 +++++++++++
 2 files changed, 16 insertions(+), 3 deletions(-)

-- 
2.56.0