Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery

From: Shawn Jin

Date: Fri Mar 13 2026 - 12:48:34 EST

Hi Ilpo and Bjorn,

A quick update on the test result with the kernel 6.19.6-1-default in openSUSE Tumbleweed.

Summary:
1. With PCI realloc off, the bridge windows were reassigned as before, the ending addresses look normal, all ending with 0xffff.
2. With PCI realloc on, the non-pref window ending address is still incorrect: 0xe1a55554. But the pref window looks right.
[ 482.943430] [ T1435] pci 0000:99:01.0: bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]: assigned
[ 482.943431] [ T1435] pci 0000:99:01.0: bridge window [mem 0xe1a00000-0xe1a55554]: assigned

Details:
Focused topology:
-[0000:94]---01.0-[95-9e]----00.0-[96-9e]----01.0-[98-9c]----00.0-[99-9c]--+-00.0-[9a]--
+-01.0-[9b]----00.0 Device 1e3e:0002
\-02.0-[9c]--
Test 1: realloc OFF
// Cold boot
[ 2.132565] [ T1] pci 0000:99:00.0: PCI bridge to [bus 9a]
[ 2.133202] [ T1] pci 0000:99:01.0: PCI bridge to [bus 9b]
[ 2.133413] [ T1] pci 0000:99:01.0: bridge window [mem 0xe9600000-0xe96fffff]
[ 2.133555] [ T1] pci 0000:99:01.0: bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
[ 2.133827] [ T1] pci 0000:99:02.0: PCI bridge to [bus 9c]
[ 2.134453] [ T1] pci 0000:98:00.0: PCI bridge to [bus 99-9c]
[ 2.134565] [ T1] pci 0000:98:00.0: bridge window [mem 0xe9600000-0xe96fffff]
[ 2.134647] [ T1] pci 0000:98:00.0: bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
[ 2.134809] [ T1] pci 0000:96:01.0: PCI bridge to [bus 98-9c]
[ 2.134830] [ T1] pci 0000:96:01.0: bridge window [mem 0xe9600000-0xe96fffff]
[ 2.134845] [ T1] pci 0000:96:01.0: bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
// after SBR
[ 202.341320] [ T1430] pci 0000:99:01.0: PCI bridge to [bus 9b]
[ 202.341539] [ T1430] pci 0000:99:01.0: bridge window [mem 0xe9600000-0xe96fffff]
[ 202.341680] [ T1430] pci 0000:99:01.0: bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
[ 202.341950] [ T1430] pci 0000:99:02.0: PCI bridge to [bus 9c]
[ 202.342573] [ T1430] pci 0000:98:00.0: PCI bridge to [bus 99-9c]
[ 202.342685] [ T1430] pci 0000:98:00.0: bridge window [mem 0xe9600000-0xe96fffff]
[ 202.342766] [ T1430] pci 0000:98:00.0: bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]
[ 202.342929] [ T1430] pcieport 0000:96:01.0: PCI bridge to [bus 98-9c]
[ 202.342951] [ T1430] pcieport 0000:96:01.0: bridge window [mem 0xe9600000-0xe96fffff]
[ 202.342964] [ T1430] pcieport 0000:96:01.0: bridge window [mem 0x13b000000000-0x13b7ffffffff 64bit pref]

// although the bridges don't have IO windows, the dbg message showing the window extension may be a bit worrisome.
[ 202.340536] [ T1430] pci 0000:99:00.0: bridge window [io size 0x0000 disabled] extended by 0x0000000000000555
[ 202.340538] [ T1430] pci 0000:99:01.0: bridge window [io size 0x0000 disabled] extended by 0x0000000000000555
[ 202.340538] [ T1430] pci 0000:99:02.0: bridge window [io size 0x0000 disabled] extended by 0x0000000000000555

Test 2: realloc ON
// code boot
[ 2.146003] [ T1] pci_bus 0000:96: resource 1 [mem 0xe1800000-0xe1dfffff]
[ 2.146004] [ T1] pci_bus 0000:96: resource 2 [mem 0x134000000000-0x1348004fffff 64bit pref]
[ 2.146007] [ T1] pci_bus 0000:98: resource 0 [io size 0x1000]
[ 2.146008] [ T1] pci_bus 0000:98: resource 1 [mem 0xe1a00000-0xe1bfffff]
[ 2.146009] [ T1] pci_bus 0000:98: resource 2 [mem 0x134000000000-0x1347ffffffff 64bit pref]
[ 2.146010] [ T1] pci_bus 0000:99: resource 1 [mem 0xe1a00000-0xe1afffff]
[ 2.146011] [ T1] pci_bus 0000:99: resource 2 [mem 0x134000000000-0x1347ffffffff 64bit pref]
[ 2.146012] [ T1] pci_bus 0000:9b: resource 1 [mem 0xe1a00000-0xe1afffff]
[ 2.146013] [ T1] pci_bus 0000:9b: resource 2 [mem 0x134000000000-0x1347ffffffff 64bit pref]

// after SBR
// USP 98:00.0's windows still look correct
[ 482.943427] [ T1435] pci 0000:98:00.0: bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]: assigned
[ 482.943429] [ T1435] pci 0000:98:00.0: bridge window [mem 0xe1a00000-0xe1bfffff]: assigned
// DSP 99:01.0 pref window is good
[ 482.943430] [ T1435] pci 0000:99:01.0: bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]: assigned
// DSP 99:01.0 non-pref window ending addr is wrong: 0xe1a55554
[ 482.943431] [ T1435] pci 0000:99:01.0: bridge window [mem 0xe1a00000-0xe1a55554]: assigned
[ 482.943432] [ T1435] pci 0000:99:00.0: PCI bridge to [bus 9a]
[ 482.944063] [ T1435] pci 0000:9b:00.0: BAR 0 [mem 0x134000000000-0x1347ffffffff 64bit pref]: assigned
[ 482.944164] [ T1435] pci 0000:9b:00.0: BAR 2 [mem 0xe1a00000-0xe1a3ffff]: assigned
[ 482.944202] [ T1435] pci 0000:99:01.0: PCI bridge to [bus 9b]
[ 482.944424] [ T1435] pci 0000:99:01.0: bridge window [mem 0xe1a00000-0xe1a55554]
[ 482.944565] [ T1435] pci 0000:99:01.0: bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]
[ 482.944837] [ T1435] pci 0000:99:02.0: PCI bridge to [bus 9c]
[ 482.945463] [ T1435] pci 0000:98:00.0: PCI bridge to [bus 99-9c]
[ 482.945575] [ T1435] pci 0000:98:00.0: bridge window [mem 0xe1a00000-0xe1bfffff]
[ 482.945657] [ T1435] pci 0000:98:00.0: bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]
[ 482.945820] [ T1435] pcieport 0000:96:01.0: PCI bridge to [bus 98-9c]
[ 482.945842] [ T1435] pcieport 0000:96:01.0: bridge window [mem 0xe1a00000-0xe1bfffff]
[ 482.945856] [ T1435] pcieport 0000:96:01.0: bridge window [mem 0x134000000000-0x1347ffffffff 64bit pref]
[ 482.945883] [ T1435] PCI: No. 2 try to assign unassigned res
[ 482.945885] [ T1435] pci 0000:99:00.0: disabling bridge window [mem size 0x00000000 64bit pref disabled] to [bus 9a] (unused)
// DSP 99:00.0 doesn't have any EP attached. But the mem size 0x55555 looks very suspicious.
[ 482.945886] [ T1435] pci 0000:99:00.0: disabling bridge window [mem size 0x00055555 disabled] to [bus 9a] (unused)
[ 482.945887] [ T1435] pci 0000:99:02.0: disabling bridge window [mem size 0x00000000 64bit pref disabled] to [bus 9c] (unused)
// Same for DSP 99:02.0
[ 482.945888] [ T1435] pci 0000:99:02.0: disabling bridge window [mem size 0x00055555 disabled] to [bus 9c] (unused)

Hope the above info gives you some hints.

Thanks,
Shawn.
________________________________________
From: Ilpo Järvinen <ilpo.jarvinen@xxxxxxxxxxxxxxx>
Sent: Thursday, March 12, 2026 10:48 AM
To: Shawn Jin <shawn.jin@xxxxxxxxxxxxxx>
Cc: Bjorn Helgaas <helgaas@xxxxxxxxxx>; Bjorn Helgaas <bhelgaas@xxxxxxxxxx>; linux-kernel@xxxxxxxxxxxxxxx <linux-kernel@xxxxxxxxxxxxxxx>; linux-pci@xxxxxxxxxxxxxxx <linux-pci@xxxxxxxxxxxxxxx>
Subject: Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery

[EXTERNAL EMAIL]

On Thu, 12 Mar 2026, Shawn Jin wrote:

> Hi Ilpo,
>
> Thanks for looking into the issue. Just a quick question. So the patch
> you linked and the patch you attached is for the latest kernel, not 6.8.
> Right? I'll try to install the latest kernel first and rerun my test and
> keep you updated.

Yes, they are for the latest trees but I can look into backporting them if
needed.

--
i.

> ________________________________________
> From: Ilpo Järvinen <ilpo.jarvinen@xxxxxxxxxxxxxxx>
> Sent: Thursday, March 12, 2026 6:24 AM
> To: Shawn Jin <shawn.jin@xxxxxxxxxxxxxx>
> Cc: Bjorn Helgaas <helgaas@xxxxxxxxxx>; Bjorn Helgaas <bhelgaas@xxxxxxxxxx>; linux-kernel@xxxxxxxxxxxxxxx <linux-kernel@xxxxxxxxxxxxxxx>; linux-pci@xxxxxxxxxxxxxxx <linux-pci@xxxxxxxxxxxxxxx>
> Subject: Re: [BUG] PCIe bridge resource allocation creates invalid limit addresses after Secondary Bus Reset recovery
>
> [EXTERNAL EMAIL]
>
> Hi,
> First of all, many of the changes related to this additional resource
> distribution do not really make sense to me.
>
> There's one fix that tries to rectify to most gross wrongness in the
> adjustment itself because it caused a regression (not yet applied):
>
> https://lore.kernel.org/linux-pci/20260219153951.68869-1-ilpo.jarvinen@xxxxxxxxxxxxxxx/
>
> It probably doesn't address the bug you see here though it may hide it.
>
> > [ 796.604869] pcieport 0000:96:01.0: distributing available resources
> > [ 796.604872] pci 0000:98:00.0: bridge window [mem 0x00100000-0x001fffff] extended by 0x0000000000100000
> > [ 796.604876] pci 0000:99:00.0: bridge window [??? 0x00000000 flags 0x0] extended by 0x0000000000055555
> > [ 796.604880] pci 0000:99:01.0: bridge window [mem 0x00100000-0x001fffff] shrunken by 0x00000000000aaaac
> > [ 796.604883] pci 0000:99:01.0: bridge window [mem 0x800000000-0xfffffffff 64bit pref] shrunken by 0x0000000000000001
> > [ 796.604886] pci 0000:99:02.0: bridge window [??? 0x00000000 flags 0x0] extended by 0x0000000000055555
>
> Note how it's not just -1 that seems wrong, the other numbers too seem
> non-round (to 1M) as well.
>
> There seems to be a number of problems with this entire algorithm.
> I think the -1 comes from remove_dev_resource() that is called for a
> resource which does have !res->flags (+ res->start=res->end=0 ->
> resource_size(res) == 1).
>
> So could you please try the patch below.
>
> Not that the fix makes everything okay IMO but perhaps it addresses this
> particular case. It's not even clear to me how some of the cases with not
> valid bridge windows should be dealt with while distributing the
> additional memory between the bridges.
>
> The 55555 cases probably come from align == 0 which makes
> ALIGN_DOWN_IF_NONZERO() to produce non-aligning sizes. The fix patch below
> might address it.
>
> I don't (yet) understand how that aaaac came to be, I suppose it somehow
> relates to this doing something unexpected so if you want to check that
> out, it would be helpful:
> align = pci_resource_alignment(bridge, res);
> if (!resource_assigned(res) && align)
> available[i].start = min(ALIGN(available[i].start, align),
> available[i].end + 1);
>
> Another option is remove_dev_resource() doing something that leads to it.
>
> It might be worth adding debug traps for available[i] getting non-1M
> aligned but it's not very clear to me how and where.
>
> From: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= <ilpo.jarvinen@xxxxxxxxxxxxxxx>
> Date: Thu, 12 Mar 2026 14:59:43 +0200
> Subject: [PATCH 1/1] PCI: Skip not valid bridge windows while distributing
> resources
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
>
> pci_bus_distribute_available_resources() distributes available
> resources between downstream bridges, however, it does not take
> into account cases where a bridge window is not valid.
>
> Skip touching bridge windows that are not valid.
>
> Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@xxxxxxxxxxxxxxx>
> ---
> drivers/pci/setup-bus.c | 7 +++++++
> 1 file changed, 7 insertions(+)
> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> index 61f769aaa2f6..4fcb5e0c44b4 100644
> --- a/drivers/pci/setup-bus.c
> +++ b/drivers/pci/setup-bus.c
> @@ -1868,6 +1868,9 @@ static void remove_dev_resource(struct resource *avail, struct pci_dev *dev,
> {
> resource_size_t size, align, tmp;
>
> + if (!res->flags)
> + return;
> +
> size = resource_size(res);
> if (!size)
> return;
> @@ -1922,6 +1925,8 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus,
> available[i] = available_in[i];
> + if (!res->flags)
> + continue;
> /*
> * The alignment of this bridge is yet to be considered,
> * hence it must be done now before extending its bridge
> @@ -1993,6 +1998,8 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus,
> for (i = 0; i < PCI_P2P_BRIDGE_RESOURCE_NUM; i++) {
> res = pci_resource_n(dev, PCI_BRIDGE_RESOURCES + i);
>
> + if (!res->flags)
> + continue;
> /*
> * Make sure the split resource space is properly
> * aligned for bridge windows (align it down to
> base-commit: 6de23f81a5e08be8fbf5e8d7e9febc72a5b5f27f
> --
> 2.39.5
>