Re: [RFC] PCI: Fix kernel panic of root-port-less PCIe enum due to ASPM
From: Serge Semin
Date: Fri Nov 25 2016 - 09:03:27 EST
On Tue, Nov 08, 2016 at 05:29:57PM -0600, Bjorn Helgaas <helgaas@xxxxxxxxxx> wrote:
Here are the answers on your questions inlined in the text.
> Hi Serge,
> On Thu, Oct 06, 2016 at 12:34:15PM +0300, Serge Semin wrote:
> > Hello linux folks,
> > Sometime ago I discovered a kernel panic popping up when PCI subsystem was
> > trying to enumerate PCI express bus with ASPM service enabled. Here it is:
> > [ 5.089667] CPU 0 Unable to handle kernel paging request at virtual
> > address 00000060, epc == 80317004, ra == 80316ac8
> > [ 5.120952] Oops[#1]:
> > ...
> > [ 5.528438] Call Trace:
> > [ 5.535640] [<80317004>] pcie_aspm_init_link_state+0x6c0/0x814
> > [ 5.552843] [<80300c44>] pci_scan_slot+0x140/0x148
> > [ 5.566957] [<80301dcc>] pci_scan_child_bus+0x50/0x1b0
> > [ 5.582096] [<80301944>] pci_scan_bridge+0x25c/0x694
> > [ 5.596724] [<80301e78>] pci_scan_child_bus+0xfc/0x1b0
> > [ 5.611862] [<80301944>] pci_scan_bridge+0x25c/0x694
> > [ 5.626488] [<80301e78>] pci_scan_child_bus+0xfc/0x1b0
> > [ 5.641628] [<8030215c>] pci_scan_root_bus+0x64/0x124
> > [ 5.656528] [<804ca298>] pcibios_scanbus+0xa8/0x188
> > I more than sure you are familiar with the issue, since I've found the
> > mailing discussion: "PCI: avoid NULL deref in alloc_pcie_link_state"
> > https://patchwork.kernel.org/patch/2751651/
> > https://bugzilla.kernel.org/show_bug.cgi?id=60111
> I'm trying to puzzle out a few things here. Maybe you can help me out?
> - Does this issue exist in current upstream kernels? Your dmesg shows a
> v3.19-based kernel. c8fc9339409d ("PCI/ASPM: Use dev->has_secondary_link
> to find downstream links"), which appeared in v4.2, fixes a problem very
> similar to what you're reporting.
I saw that fix, but alas it hasn't fixed the issue. I've tested kernel 4.4.24
without my patch applied and the problem with ASPM-related kernel panic still
exists (see the stack-trace above).
> - When we dereference the NULL pointer, which device did we call
> pcie_aspm_init_link_state() for?
My suggestion was that the problem arised in the framework of bus 2 enumeration.
Since there was no root bus on my architecture, the pci_link_state structure was
not created. So when the algorithm tried to enumerate the second bus, it needed
actual pci_link_state structure of parental bus, which hadn't been created.
That's how the NULL-dereference happened.
> - https://bugzilla.kernel.org/attachment.cgi?id=240981 is the failing dmesg
> log, and it shows "vgaarb: device added: PCI:0000:04:00.0".
> Your lspci output (https://bugzilla.kernel.org/attachment.cgi?id=241001)
> shows 04:00.0 is a downstream port, but vga_arbiter_add_pci_device() only
> prints that message for VGA class devices.
> https://bugzilla.kernel.org/attachment.cgi?id=240991, the successful
> dmesg log, shows "vgaarb: device added: PCI:0000:06:00.0". That makes
> more sense because 06:00.0 is class 0300, which is a VGA device.
I can't be sure about the reason of that strange enumeration. But I can assure
you, that that bus confusion isn't the reason of the ASPM panicing. So I can
just guess, that the misleading BDF can be caused by SMP (I've got a processor
with two cores) and ASPM panic. VGA driver initialization may happen
concurrently with PCI bus enumeration.