Re: Regression Re: [PATCH v3 1/4] PCI: Enable ACS only after configuring IOMMU for OF platforms

[Manivannan Sadhasivam]

On Fri, Apr 03, 2026 at 12:45:29PM -0400, Jon Kohler wrote:

[...]

> Howdy folks,
> Writing to report a regression from this patch. While attempting to
> rebase our 6.12.y-based internal tree past 6.12.75, we encountered a
> behavior change in our internal systems, wherein we have a platform
> that uses vfio-pci passthrough for a series of PCIe devices (SAS HBAs,
> NVMe SSDs, NIC passthrough, etc) to a given service VM.
> 
> On some, but not all, of these systems, we noticed that the iommu
> group topology completely changes with this commit. Before this commit,
> a given system may have ~90-ish iommu groups. After this commit, we see
> ~60-ish iommu groups.
> 
> The net result is that some of the devices marked for passthrough get
> clubbed together with devices that are not marked for passthrough. As
> such, libvirt refuses to start the VM, like so: 
> 
> [root@demo-system-here ~]# virsh start demo-service-vm
> error: Failed to start domain 'demo-service-vm'
> 2026-04-02T03:52:17.098998Z qemu-kvm: -device {"driver":"vfio-pci","host":"0000:41:00.0","id":"ua-cb173399-0c46-5b7a-b3e6-fe2fb5f9509c","bus":"pci.0","addr":"0x7","rombar":0}: vfio 0000:41:00.0: group 8 is not viable
> Please ensure all devices within the iommu_group are bound to their vfio bus driver.
> 
> [root@demo-system-here ~]# lspci |grep 41:00
> ...
> 41:00.0 Serial Attached SCSI controller: Broadcom / LSI Fusion-MPT 12GSAS/PCIe Secure SAS38xx
> Subsystem: Super Micro Computer Inc AOC-S3816L-L16iT (NI22) Storage Adapter
> Kernel driver in use: vfio-pci
> Kernel modules: mpt3sas 
> 
> [root@demo-system-here ~]# ls -l /sys/kernel/iommu_groups/8/devices
> total 0
> lrwxrwxrwx. 1 root root 0 Apr 2 07:33 0000:40:01.0 -> ../../../../devices/pci0000:40/0000:40:01.0
> lrwxrwxrwx. 1 root root 0 Apr 2 07:33 0000:40:01.1 -> ../../../../devices/pci0000:40/0000:40:01.1
> lrwxrwxrwx. 1 root root 0 Apr 2 07:33 0000:40:01.2 -> ../../../../devices/pci0000:40/0000:40:01.2
> lrwxrwxrwx. 1 root root 0 Apr 2 07:33 0000:41:00.0 -> ../../../../devices/pci0000:40/0000:40:01.1/0000:41:00.0
> lrwxrwxrwx. 1 root root 0 Apr 2 07:33 0000:42:00.0 -> ../../../../devices/pci0000:40/0000:40:01.2/0000:42:00.0
> 
> The tricky bit here is 41 is the SAS controller we're trying to pass
> through, and 42 is the local NVMe m.2 boot disk that the server itself
> is booting off of.
> 
> Before this patch, they are put in separate groups, and there is not
> a problem.
> 
> I’ve only tried this on our 6.12.y tree, but not yet our 6.18 and 6.6
> trees, so I’m not sure if this problem exists there yet, but this
> commit is in those trees in general in 6.18.16 and 6.6.128 respectively
> 
> Happy to provide any other details you might like, as this is 100%
> reproducible on a variety of systems here.
> 

Sorry for the breakage. This issue was reported and a revert has been submitted:
https://lore.kernel.org/linux-pci/20260320172335.29778-1-john@xxxxxxxxxxxxxxxx/

The stable maintainers are yet to pick it. And I've submitted similar reverts
for all pre v6.15 kernels where this patch is not applicable.

https://lore.kernel.org/linux-pci/20260331091009.19536-1-manivannan.sadhasivam@xxxxxxxxxxxxxxxx/
https://lore.kernel.org/linux-pci/20260331091306.25210-1-manivannan.sadhasivam@xxxxxxxxxxxxxxxx/
https://lore.kernel.org/linux-pci/20260331091455.30394-1-manivannan.sadhasivam@xxxxxxxxxxxxxxxx/
https://lore.kernel.org/linux-pci/20260331091727.33552-1-manivannan.sadhasivam@xxxxxxxxxxxxxxxx/

- Mani

-- 
மணிவண்ணன் சதாசிவம்