Re: [PATCH v3 8/8] PCI/quirks: Avoid certain address on Genoa systems
From: Bjorn Helgaas
Date: Tue Oct 13 2026 - 19:49:46 EST
[+cc Hans, Kai-Heng, Andy, linux-acpi]
On Tue, Oct 13, 2026 at 01:59:23PM -0500, Mario Limonciello wrote:
> On 10/13/26 12:26, Ilpo Järvinen wrote:
> > On Tue, 13 Oct 2026, Mario Limonciello wrote:
> > > On 10/13/26 12:00, Mario Limonciello wrote:
> > > > On 10/13/26 08:09, Ilpo Järvinen wrote:
> > > > > While testing the resource placement changes, tests hit a case where
> > > > > igb fails to probe when BAR 0 is placed at 0x9c000000:
> > > > >
> > > > > 90000000-9cffffff : PCI Bus 0000:a0
> > > > > - 90000000-902fffff : PCI Bus 0000:a1
> > > > > - 90000000-900fffff : 0000:a1:00.0
> > > > > - 90000000-900fffff : igb
> > > > > - 90100000-901fffff : 0000:a1:00.0
> > > > > - 90200000-90203fff : 0000:a1:00.0
> > > > > - 90200000-90203fff : igb
> > > > > + 9be00000-9c0fffff : PCI Bus 0000:a1
> > > > > + 9be00000-9befffff : 0000:a1:00.0
> > > > > + 9bf00000-9bf03fff : 0000:a1:00.0
> > > > > + 9c000000-9c0fffff : 0000:a1:00.0
> > > > > 9c100000-9c17ffff : amd_iommu
> > > > > 9c180000-9c1803ff : IOAPIC 8
> > > > >
> > > > > - Region 0: Memory at 90000000 (32-bit, non-prefetchable)
> > > > > [size=1M]
> > > > > - Region 3: Memory at 90200000 (32-bit, non-prefetchable)
> > > > > [size=16K]
> > > > > - Expansion ROM at 90100000 [disabled] [size=1M]
> > > > > + Region 0: Memory at 9c000000 (32-bit, non-prefetchable)
> > > > > [size=1M]
> > > > > + Region 3: Memory at 9bf00000 (32-bit, non-prefetchable)
> > > > > [size=16K]
> > > > > + Expansion ROM at 9be00000 [disabled] [size=1M]
> > > > >
> > > > > igb 0000:a1:00.0 0000:a1:00.0 (uninitialized): PCIe link lost
> > > > > ------------[ cut here ]------------
> > > > > igb: Failed to read reg 0x18!
> > > > > WARNING: drivers/net/ethernet/intel/igb/igb_main.c:724 at
> > > > > igb_rd32.cold+0x3c/0x4f [igb], CPU#32: kworker/32:1/706
> > > > > ...
> > > > > igb_get_invariants_82575+0xff/0xf00 [igb]
> > > > > igb_probe+0x3c8/0x1190 [igb]
> > > > > local_pci_probe+0x3b/0x80
> > > > >
> > > > > It turns out there is a 64kB iomem black hole at 9c000000 that returns
> > > > > ~0 and this is where igb's BAR 0 resides. If another BAR of the same
> > > > > card is placed into that address, it is similarly black holed.
> > > > >
> > > > > Mark the problematic 64kB range reserved using a quirk bound to the
> > > > > bridge found in the problematic system.
> > > > >
> > > > > Closes: https://bugzilla.kernel.org/show_bug.cgi?id=222074
> > > > > Suggested-by: Mario Limonciello <mario.limonciello@xxxxxxx>
> > > > > Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@xxxxxxxxxxxxxxx>
> > > > > ---
> > > > > Cc: Borislav Petkov <bp@xxxxxxxxx>
> > > > >
> > > > > As an alternative to this, I've also tested reverting the commit
> > > > > 07eab0901ede ("efi/x86: Remove EfiMemoryMappedIO from E820 map") and
> > > > > that also resolves the issue by retaining "9c000000-9cffffff : Reserved"
> > > > > entry in the resource tree.
> > > > >
> > > > > ---
> > > > > arch/x86/pci/fixup.c | 36 ++++++++++++++++++++++++++++++++++++
> > > > > 1 file changed, 36 insertions(+)
> > > > >
> > > > > diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c
> > > > > index b301c6c8df75..cc463bf492b6 100644
> > > > > --- a/arch/x86/pci/fixup.c
> > > > > +++ b/arch/x86/pci/fixup.c
> > > > > @@ -822,6 +822,42 @@ DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1571,
> > > > > pci_amd_enable_64bit_bar);
> > > > > DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x15b1,
> > > > > pci_amd_enable_64bit_bar);
> > > > > DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1601,
> > > > > pci_amd_enable_64bit_bar);
> > > > > +/*
> > > > > + * TODO: Remove when/if root cause is found.
> > > > > + *
> > > > > + * Genoa appears to have a 64kB iomem black hole starting at 0x9c000000
> > > > > + * address for which all reads return ~0 blocking an overlapping BAR from
> > > > > + * working.
> > > > > + *
> > > > > + * Work around the problem by reserving the space prior to making any
> > > > > iomem
> > > > > + * allocations that could overlap with the black hole.
> > > > > + */
> > > > > +static struct resource genoa_black_hole =
> > > > > DEFINE_RES_MEM_NAMED(0x9c000000,
> > > > > + SZ_64K,
> > > > > + "reserved");
> > > > > +
> > > > > +static void genoa_iomem_black_hole(struct pci_dev *dev)
> > > > > +{
> > > > > + struct resource *r;
> > > > > +
> > > > > + pci_bus_for_each_resource(dev->bus, r) {
> > > > > + if (!r || !r->flags || !resource_assigned(r))
> > > > > + continue;
> > > > > +
> > > > > + if (!__resource_contains_unbound(r, &genoa_black_hole))
> > > > > + continue;
> > > > > +
> > > > > + if (resource_assigned(&genoa_black_hole))
> > > > > + pci_dbg(dev, "iomem black hole workaround already
> > > > > applied\n");
> > > > > + else if (!insert_resource(r, &genoa_black_hole))
> > > > > + pci_info(dev, "iomem black hole workaround enabled\n");
> > > > > + else
> > > > > + pci_dbg(dev, "iomem black hole workaround add failed\n");
> > > > > + return;
> > > > > + }
> > > > > +}
> > > > > +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_AMD, 0x14ab,
> > > > > genoa_iomem_black_hole);
> > > > > +
> > > > > #define RS690_LOWER_TOP_OF_DRAM2 0x30
> > > > > #define RS690_LOWER_TOP_OF_DRAM2_VALID 0x1
> > > > > #define RS690_UPPER_TOP_OF_DRAM2 0x31
> > > >
> > > > I guess it's good we know the root cause is 07eab0901ede. If we're going
> > > > down the path of making holes in that we may well put the hole in
> > > > efi_remove_e820_mmio() like this (attached).
> > >
> > > I don't really like that though. If Genoa is affected I would expect plenty
> > > of other machines are too they just haven't been found yet.
> >
> > That's what I'm worried as well. My own testing can only cover very little
> > ground here.
> >
> > > Can we roll back 07eab0901ede and add quirks for the busted machines instead?
>
> I had an agent dig through all the reports linked in 07eab0901ede and
> drafted up what should be the remaining quirks for a revert.
>
> https://git.kernel.org/pub/scm/linux/kernel/git/superm1/linux.git/log/?h=superm1/genoa-mess
>
> Bjorn, thoughts on this? I'll send it out if that's the way we want
> to go (or Ilpo feel free to subsume into your series as it's only
> really exposed by your series at the moment)
I would LOVE it if we can use those patches because it's way less
magic than 07eab0901ede is.
But I'm frankly terrified to touch it because I'm skeptical that two
more quirks are enough. I just expect that whatever strategy BIOS
uses to build these maps is likely to be used on many models. I do
hate 07eab0901ede though, so I would love to see it gone.
I do think the description [1] is overly specific because AFAIK
there's nothing that connects this to AMD EPYC 9354 or igb other than
the fact that it was seen there. It could happen to any PCI device on
any system with similar memory maps.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/superm1/linux.git/commit/?h=superm1/genoa-mess&id=0a49c704477153d6597d34b3c6e3141007ff058c