[PATCH v9 0/7] VMD add second root bus support

[Szymon Durawa]

This series adds support for a second VMD root bus.

Today the VMD driver can enumerate and manage devices only behind the
first VMD root bus. Starting with Intel Arrow Lake, VMD may expose a
second root bus (BUS1), and devices behind that bus are not discovered
by the current driver.

The hardware model for Intel Arrow Lake VMD is slightly different
from earlier VMD generations:

- CFGBAR, MEMBAR1, and MEMBAR2 are shared by both VMD root buses.
- Root ports on BUS1 have PCI_PRIMARY_BUS hardwired to 0x80.
- Interrupt handling remains based on the existing VMD MSI/MSI-X and
  irq_domain model; this series does not introduce a separate interrupt
  domain for BUS1.

To support that layout, the series first refactors the current code so
resource setup, bus creation, and enumeration are split into helpers.
It then partitions the shared CFGBAR/MEMBAR apertures between BUS0 and
BUS1 and creates a second root bus in its own emulated PCI domain.
Reads of PCI_PRIMARY_BUS from BUS1 root port bridges are emulated to
return the actual root bus number, so PCI core bridge scanning does not
detect a mismatch and lose BUS1 subordinate device discovery. This
replaces the separate workaround patch used in v8.

Patch overview:

- Patches 1-4 extract helper functions for bus enumeration, CFGBAR and
  MEMBAR configuration, and root-bus creation.
- Patch 5 replaces hardcoded resource indices and constants with enums
  and defines.
- Patch 6 converts single-bus state (bus, busn_start, sysdata) into
  per-root-bus arrays.
- Patch 7 adds BUS1 resource partitioning, a separate PCI domain for
  BUS1, PCI_PRIMARY_BUS read emulation for BUS1 root ports, and
  second-root-bus support.

Tested on an Arrow Lake platform with NVMe devices present behind both
VMD root buses. With the series applied, devices behind BUS0 and BUS1
are enumerated and become visible to the kernel.

Changes from v1:
- splitting series into more commits, requested by Bjorn
- adding helper functions, suggested by Bjorn
- minor typos and unclear wording updated, suggested by Bjorn

Changes from v2:
- wording update in commit logs, suggested by Bjorn

Changes from v3:
- using GENMASK() instead of manual bits shifting, suggested by Bjorn
- converting decimal number to hex representation, suggested by Bjorn
- wording update in commit logs, suggested by Bjorn

Changes from v4:
- Update Dan's email address
- Resending the whole series to the correct mailing list

Changes from v5:
- Fixed correctness issues in BUS/resource layout in patches 7 and 8
- Fixed lifetime and use-after-free issues in patches 3 and 7
- Fixed resource tree integrity issues in patch 7
- Fixed error-path leaks across patches 3, 4, and 5
- Fixed allocation failure handling in patch 3
- Note on Sashiko feedback: Pre-existing warnings (unrelated to
  this feature series) were left untouched.

Changes from v6:
- Refactoring the series to align with recent changes in link below:
 https://patchwork.kernel.org/project/linux-pci/patch/20260629165025.268836-1-nirmal.patel@xxxxxxxxxxxxxxx/
- Fixed double free of the IRQ domain and emulated domain number
- Fixed a race with concurrent readers when modifying sibling pointers
- Fixed the MEMBAR2 split equations in
  vmd_configure_membar1_membar2()
- Fixed hardcoded resource indices in vmd_configure_membar()

Changes from v7:
- wording update in commit logs in patch 8, suggested by Bjorn

Changes from v8:
- Dropped patch 8 ("PCI: vmd: Workaround for hardwired BUS1 bus
  number") along with the issues Sashiko reported against it. The
  hardwired BUS1 primary bus number is now handled in patch 7 by
  emulating PCI_PRIMARY_BUS reads for BUS1 root port bridges in
  vmd_pci_read(), and BUS1 is assigned its own emulated PCI domain.
- Patch 6: sysdata is converted to a per-root-bus array as well.
- Patch 7: Fixed a data race reported by Sashiko where the MEMBAR
  .sibling pointers were assigned manually while the resource tree
  could be walked concurrently. BUS0/BUS1 MEMBAR resources are now
  inserted and removed with request_resource()/release_resource().
- Patch 7: Fixed an issue reported by Sashiko where resizing the CFGBAR
  start address for BUS1 caused vmd_domain_reset() to skip the root
  buses. The reset now starts from busn_start[VMD_BUS_0] and covers
  the whole CFGBAR aperture when BUS1 is present.
- Patch 7: BUS1 setup failures (domain allocation or bus creation) now
  release BUS1 resources and fall back to BUS0 only.

Suggested-by: Nirmal Patel <nirmal.patel@xxxxxxxxxxxxxxx>
Reviewed-by: Asutosh Pathak <asutosh.pathak@xxxxxxxxxxxxxxx>
Signed-off-by: Szymon Durawa <szymon.durawa@xxxxxxxxxxxxxxx>

Szymon Durawa (7):
  PCI: vmd: Add vmd_bus_enumeration() helper function
  PCI: vmd: Add vmd_configure_cfgbar() helper function
  PCI: vmd: Add vmd_configure_membar() and
    vmd_configure_membar1_membar2()
  PCI: vmd: Add vmd_create_bus()
  PCI: vmd: Replace hardcoded values with enum and defines
  PCI: vmd: Convert bus, busn_start and sysdata to an array
  PCI: vmd: Add support for second rootbus under VMD

 drivers/pci/controller/vmd.c | 727 ++++++++++++++++++++++++++++-------
 1 file changed, 592 insertions(+), 135 deletions(-)

-- 
2.43.0