Re: [PATCH] PCI: Avoid bus reset on NVIDIA GB202 and GB203 GPUs

[William Woodstock]

Resending because html was rejected, and I also tried kernel 7.3, with
no success.

> What's your RTX 5080 vBIOS version and system information?  Is it the
> same Asus X870E as the other 5090 reports or something else?

vBIOS: 98.03.3b.c0.0b
Card:  Zotac GeForce RTX 5080 (10de:2c02, subsystem 19da:1765),
       primary PCIEX16 slot, bifurcation Auto
Board: Gigabyte X870E Aorus Master, BIOS F14d (not the Asus board)
CPU:   AMD Ryzen 9 9950X3D2, GPU on root port 00:01.1 [1022:14db]
Host:  Proxmox VE, kernel 7.0.14-20-pve; guest Windows 11 Pro,
       GPU passed through with vfio-pci

It reproduces without a VM. On the host, with the link at 32GT/s x16:

  echo 1 > /sys/bus/pci/devices/0000:00:01.1/reset_subordinate

brings it back at x8. A bwctrl retrain restores the speed but not the
width; only a cold boot gets x16 back.

> Is the issue still present on kernel v7.3?

Yes. I built v7.3 from kernel.org with no patches, using the Proxmox
7.0.14-20-pve config (make olddefconfig, LOCALVERSION=-stock), and ran
the same host-only test with no VM running:

  uname -r: 7.3.0-stock

  before: 01:00.0 LnkSta: Speed 2.5GT/s (downgraded), Width x16
  after:  01:00.0 LnkSta: Speed 32GT/s, Width x8 (downgraded)

  [  211.617016] vfio-pci 0000:01:00.0: resetting
  [  211.617225] vfio-pci 0000:01:00.1: resetting
  [  212.898377] vfio-pci 0000:01:00.0: not ready 1023ms after bus
reset; waiting
  [  279.586064] vfio-pci 0000:01:00.0: not ready 65535ms after bus
reset; giving up
  [  279.587022] vfio-pci 0000:01:00.0: reset done
  [  279.589284] vfio-pci 0000:01:00.1: reset done

The 65 s "not ready" wait didn't happen on 7.0.14, where the reset
finished in about a second. On this run the guest had been
force-stopped just before the reset, so that may explain it rather
than 7.3. The x8 result is the same either way.

> Disabling bus reset entirely should be a last resort, it has
> implications for device isolation when no other reset mechanism exists.

Understood. What I measured:

- 01:00.0 (GPU) reports reset_method "flr bus"; 01:00.1 (HDA) none.
- With only .0 assigned, FLR is used and the link stays x16.
- With .1 assigned too, vfio falls back to a bus reset and the link
  drops to x8.

So the bus reset is only reached because the audio function has no
reset of its own. Would you rather see a quirk that gives the HDA
function a usable reset, or keep the bus reset and restore the link
width afterwards? Happy to test either.
Thanks,
mark-ship-it


On Mon, Oct 19, 2026 at 1:11 PM Alex Williamson <alwilliamson@xxxxxxxxxx> wrote:
>
> On Sun, 18 Oct 2026 13:17:27 -0700
> mark-ship-it <williamwoodstock79@xxxxxxxxx> wrote:
>
> > NVIDIA GB202 (GeForce RTX 5090, RTX PRO 6000) and GB203 (GeForce RTX
> > 5080/5070 Ti/5070) GPUs train their link at x16 at boot, but after a
> > Secondary Bus Reset the link comes back at x8 and stays there until the
> > next cold boot.
> >
> > The GPU's HD Audio function (e.g. 10de:22e9 on GB203) has no reset method,
> > so when both functions are assigned to a VM, vfio-pci falls back to a bus
> > reset at VM start and again at release, and the GPU then runs at half
> > width. Assigning only function 0 avoids the bus reset because the GPU
> > itself supports FLR, which is why the problem is usually reported as
> > "passing the audio function breaks x16".
> >
> > Observed on an RTX 5080 (10de:2c02) behind an AMD Root Port, kernel
> > 7.0.14, Proxmox VE host with a Windows 11 Pro guest:
> >
> >   boot:                         LnkSta: Speed 32GT/s, Width x16
> >   echo 1 > /sys/bus/pci/devices/<root port>/reset_subordinate
> >                                 LnkSta: Speed 32GT/s, Width x8 (downgraded)
> >
> > No VM is involved in that reproduction. Stepping the link speed through
> > bwctrl (Gen5 -> Gen4 -> Gen5) retrains the link and the speed follows,
> > but the width stays at x8. With a VM started frozen (-S), the link is
> > already at x8 before any guest code has run, so the guest driver is not
> > involved.
> >
> > With this patch applied, reset_method for the GPU is "flr" only, both
> > functions can be assigned, and the link stays at x16 Gen5 across VM
> > start, shutdown and restart, with the Windows guest driver working.
> >
> > The same symptom has been reported with an RTX 5090 (10de:2b85) on an
> > AMD X870E board, where passing only function 0 was the only thing that
> > kept x16.
> >
> > Link: https://forum.proxmox.com/threads/asus-x870e-pcie-x8-bifurcation-bug-with-rtx-5090-vfio-passthrough.178461/
> >
> > Signed-off-by: mark-ship-it <williamwoodstock79@xxxxxxxxx>
> > ---
> >  drivers/pci/quirks.c | 9 ++++++++-
> >  1 file changed, 8 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
> > index 1aa87a3ef..a4adcc3e7 100644
> > --- a/drivers/pci/quirks.c
> > +++ b/drivers/pci/quirks.c
> > @@ -3769,10 +3769,17 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_NVIDIA, 0x22D0, quirk_no_bus_reset);
> >  /*
> >   * Some NVIDIA GPU devices do not work with bus reset, SBR needs to be
> >   * prevented for those affected devices.
> > + *
> > + * GB202 (0x2b80-0x2bbf) and GB203 (0x2c00-0x2c7f) GPUs train at x16 at
> > + * boot, but after a Secondary Bus Reset the link comes back at x8 and
> > + * stays there until the next cold boot; a link retrain restores the speed
> > + * but not the width.  Function Level Reset does not have this problem.
> >   */
> >  static void quirk_nvidia_no_bus_reset(struct pci_dev *dev)
> >  {
> > -     if ((dev->device & 0xffc0) == 0x2340)
> > +     if ((dev->device & 0xffc0) == 0x2340 ||
> > +         (dev->device & 0xffc0) == 0x2b80 ||
> > +         (dev->device & 0xff80) == 0x2c00)
> >               quirk_no_bus_reset(dev);
> >  }
> >  DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_NVIDIA, PCI_ANY_ID,
>
> What's your RTX 5080 vBIOS version and system information?  Is it the
> same Asus X870E as the other 5090 reports or something else?
>
> Is the issue still present on kernel v7.3?
>
> Disabling bus reset entirely should be a last resort, it has
> implications for device isolation when no other reset mechanism exists.
> Thanks,
>
> Alex