Re: [PATCH v10 0/7] VMD add second root bus support

From: Vladimir Nedoshivin

Date: Tue Nov 10 2026 - 19:33:45 EST


Hello Szymon, Bjorn, and PCI maintainers,

I have successfully tested the complete VMD Second Root Bus v10
patch series on my MSI Vector 17 HX AI laptop with Intel Arrow Lake
and VMD controller 8086:ad0b.

I would also like to provide some background on the original problems,
the previously developed workaround, and the relationship between this
series and the ARL004 interrupt-ordering fix.

The original investigation is documented here:

https://bugs.launchpad.net/bugs/2166325

Hardware:
- Laptop: MSI Vector 17 HX AI A2XWIG
- CPU: Intel Core Ultra 9 275HX
- VMD controller: Intel 8086:ad0b
- VMD PCI address: 0000:00:0e.0
- NVMe: Phison 1TB ESR01TBYCCA4-EDJ-2MS
- Operating system: Ubuntu 26.04
- Secure Boot: enabled

1. Original problem: NVMe device not detected

With the stock Ubuntu 7.0.0-30/31 VMD driver, the internal NVMe SSD
was not detected.

The kernel reported:

vmd 0000:00:0e.0: Unknown Bus Offset Setting (3)

As a result, the NVMe device behind Intel VMD was not enumerated.

The laptop firmware does not provide a usable option to disable
Intel VMD/RST.

I initially integrated an earlier revision of your Second Root Bus
patch series into the Ubuntu VMD driver.

This successfully restored NVMe enumeration, but exposed a second
hardware-related problem.

2. Second problem: ARL004 interrupt-ordering erratum

After restoring NVMe enumeration, storage operations experienced
repeated stalls of approximately 30 seconds.

The kernel reported:

nvme nvme0: I/O tag ... timeout, completion polled

A diagnostic sequential read of 1 GiB required approximately
121.339 seconds, corresponding to about 8.8 MB/s.

The investigation identified the Intel Arrow Lake ARL004 erratum:

MSI from VMD-Owned Device May Pass Memory Write

An MSI may reach the processor before preceding posted writes from
a VMD-owned device become visible in memory.

This can cause the NVMe interrupt handler to observe stale
completion-queue data.

An experimental workaround was implemented by performing a PCI
configuration-space read from the MSI-initiating device before
dispatching the corresponding interrupt.

With this workaround, the same diagnostic read completed in
approximately 0.882 seconds.

A subsequent 10 GiB read also completed normally, without the
previously observed NVMe completion timeouts.

3. Upstream ARL004 fix

Following the initial investigation, I submitted an upstream patch
enabling the existing VMD interrupt-ordering quirk for Intel
Arrow Lake VMD device 0xad0b.

The patch was accepted by Bjorn Helgaas under the title:

PCI: vmd: Flush DMA writes before handling MSI on Arrow Lake

Upstream commit:

2c88c2dc1852

The accepted implementation uses the existing
VMD_FEAT_INTERRUPT_QUIRK mechanism.

This is separate from the Second Root Bus feature.

The original Ubuntu investigation and test results are available
in Launchpad bug #2166325.

4. Testing the Second Root Bus v10 series

I have now tested all seven patches from your v10 series,
based on Linux 7.4-rc1.

To preserve my working Ubuntu installation, I compiled the patched
vmd.c as an external module against the Ubuntu 7.0.0-43-generic
kernel headers.

The only source compatibility adjustment was replacing
PCI_DEVICE_ID_INTEL_VMD_28C1 with its numeric value, 0x28c1,
because the identifier was unavailable in the Ubuntu headers.

The resulting module was signed using an enrolled MOK and loaded
through a separate test initramfs.

The test therefore used:
- Ubuntu kernel 7.0.0-43-generic
- VMD source from Linux 7.4-rc1
- All seven Second Root Bus v10 patches
- The upstream ARL004 interrupt-ordering quirk already present
in the 7.4-rc1 base

No additional local ARL004 workaround was applied to the v10 source.

The loaded module was verified using srcversion:

45EF68AF5A6E01AAA2882CF

5. Test results

The system booted successfully.

Both VMD PCI domains were created:

vmd 0000:00:0e.0: PCI host bridge to bus 10000:e0
vmd 0000:00:0e.0: PCI host bridge to bus 10001:e1
vmd 0000:00:0e.0: Bound to PCI domains 10000 and 10001

The NVMe controller was successfully detected behind BUS1:

nvme nvme0: pci function 10001:f1:00.0

The NVMe controller initialized successfully, and the Ubuntu
root filesystem mounted normally.

Sequential read performance:

Command:
dd if=/dev/nvme0n1 of=/dev/null bs=4M count=1024 iflag=direct

Data read: 4 GiB
Duration: 4.16695 seconds
Throughput: approximately 983 MiB/s

This is comparable to the approximately 984 MiB/s previously
observed with my working ARL004-specific driver.

Hardware MSI-X interrupts were active.

During a separate 1 GiB direct-read test, the aggregate VMD
interrupt count increased:

Before: 134377
After: 138680
Difference: +4303

One observation worth mentioning:

The virtual NVMe interrupt counters (nvme0q*) remained at zero
in /proc/interrupts, while the underlying physical VMD MSI-X
counters were active.

Despite this, NVMe I/O completed successfully, and no VMD/NVMe
errors or timeouts were found in the checked kernel logs.

6. Test limitations

This was a test of the v10 VMD source built as an external module
against Ubuntu 7.0.0-43, not a boot test of the complete
Linux 7.4-rc1 kernel.

The completed testing covered boot, device enumeration,
read-only NVMe I/O, and interrupt activity.

Dedicated write stress testing, suspend/resume, and long-term
stability testing have not yet been performed.

7. Conclusion

The Second Root Bus v10 series successfully resolves the NVMe
enumeration problem on this Intel Arrow Lake VMD 8086:ad0b platform.

Combined with the previously accepted upstream ARL004
interrupt-ordering fix, the system boots successfully, detects
the NVMe controller behind BUS1, and achieves normal sequential
read performance.

No functional regressions were observed during the completed tests.

Thank you for your continued work on Second Root Bus support.

Tested-by: Vladimir Nedoshivin <gvozd188@xxxxxxx>

Best regards,
Vladimir Nedoshivin