Re: [GIT PULL] PCI changes for v4.20

From: Lendacky, Thomas
Date: Tue Nov 13 2018 - 14:48:05 EST

On 11/13/2018 08:41 AM, Lendacky, Thomas wrote:
> On 11/13/2018 04:20 AM, Borislav Petkov wrote:
>> On Tue, Nov 13, 2018 at 08:17:12AM +0100, Ingo Molnar wrote:
>>>
>>> * Bjorn Helgaas <helgaas@xxxxxxxxxx> wrote:
>>>
>>>> PCI changes:
>>>>
>>>> - Pay attention to device-specific _PXM node values (Jonathan Cameron)
>>>
>>> There's a new boot regression, my AMD ThreadRipper system (MSI X399 SLI
>>> PLUS (MS-7B09)) hangs during early bootup, and I have bisected it down to
>>> this commit:
>>>
>>> bad7dcd94f39: ACPI/PCI: Pay attention to device-specific _PXM node values
>>>
>>> Reverting it solves the hang.
>>>
>>> Unfortunately there's no console output when it hangs, even with
>>> earlyprintk. It just hangs after the "loading initrd" line.
>>>
>>> Config is an Ubuntu-ish config with PROVE_LOCKING=y and a few other debug
>>> options.
>>>
>>> All my other testsystems boot fine with similar configs, so it's probably
>>> something specific to this system.
>>
>> Lemme add Tom, he might have an idea.
>
> I'm not seeing any issues on my EPYC system. Let me see if I can locate a
> Threadripper system to test on.

Based upon the link that Bjorn referenced in another email, I was able to
re-create the problem by having my EPYC system return early from
acpi_numa_init() with a -ENOENT (skipping the SRAT table). This resulted
in the following GPF:

[ 11.157840] general protection fault: 0000 [#1] SMP NOPTI
[ 11.158785] CPU: 3 PID: 1 Comm: swapper/0 Not tainted 4.20.0-rc2-zp-linux #3
[ 11.158785] Hardware name: ******
[ 11.158785] RIP: 0010:get_partial_node.isra.76+0x33/0x2b0
[ 11.158785] Code: 89 e5 41 57 41 56 41 55 41 54 53 48 83 e4 f0 48 83 c4 80 48 85 f6 48 89 7c 24 30 48 89 54 24 10 89 4c 24 0c 0f 84 d5 00 00 00 <48> 83 7e 08 00 0f 84 ca 00 00 00 48 89 f7 48 89 74 24 38 e8 95 5e
[ 11.158785] RSP: 0018:ffffc900001078b0 EFLAGS: 00010002
[ 11.158785] RAX: 0000000000000000 RBX: 0000000000000202 RCX: 00000000006080c0
[ 11.158785] RDX: ffff889ffdae7150 RSI: 4c7a584873359cf2 RDI: ffff888107c07000
[ 11.158785] RBP: ffffc90000107958 R08: ffff888107c07000 R09: 0000000000000001
[ 11.158785] R10: 00000000006080c0 R11: 0000000000000002 R12: ffff889ffdae7140
[ 11.158785] R13: ffff888107c07000 R14: ffff888107c07000 R15: 0000000000000002
[ 11.158785] FS: 0000000000000000(0000) GS:ffff889ffdac0000(0000) knlGS:0000000000000000
[ 11.158785] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 11.158785] CR2: 0000000000000000 CR3: 00008014bc20a000 CR4: 00000000003406e0
[ 11.158785] Call Trace:
[ 11.158785] ? acpi_os_release_object+0xa/0x10
[ 11.158785] ? acpi_ds_result_pop+0xf8/0x10c
[ 11.158785] ? acpi_ds_create_operand+0x227/0x24e
[ 11.158785] ___slab_alloc+0x100/0x540
[ 11.158785] ? acpi_ds_create_operands+0x72/0xd7
[ 11.158785] ? alloc_desc+0x35/0x210
[ 11.158785] ? acpi_ns_check_object_type+0x123/0x1c0
[ 11.158785] ? alloc_desc+0x35/0x210
[ 11.158785] __slab_alloc+0x1c/0x33
[ 11.158785] kmem_cache_alloc_node_trace+0xac/0x210
[ 11.158785] alloc_desc+0x35/0x210
[ 11.158785] __irq_alloc_descs+0x1c4/0x230
[ 11.158785] __irq_domain_alloc_irqs+0x54/0x2e0
[ 11.158785] mp_map_pin_to_irq+0x2cf/0x330
[ 11.158785] acpi_register_gsi_ioapic+0x78/0x170
[ 11.158785] ? mmio_resource_enabled.part.0+0x60/0x60
[ 11.158785] acpi_pci_irq_enable+0xcd/0x280
[ 11.158785] ? mmio_resource_enabled.part.0+0x60/0x60
[ 11.158785] ? mmio_resource_enabled.part.0+0x60/0x60
[ 11.158785] do_pci_enable_device+0x5b/0x100
[ 11.158785] ? pci_bus_read_config_word+0x56/0x70
[ 11.158785] pci_enable_device_flags+0xe0/0x130
[ 11.158785] pci_enable_bridge+0x52/0x90
[ 11.158785] pci_enable_device_flags+0x8c/0x130
[ 11.158785] quirk_usb_early_handoff+0x63/0x6b0
[ 11.158785] ? bus_find_device+0x87/0xd0
[ 11.158785] ? mmio_resource_enabled.part.0+0x60/0x60
[ 11.158785] pci_fixup_device+0xe8/0x1a0
[ 11.158785] pci_apply_final_quirks+0x68/0x127
[ 11.158785] ? pci_proc_init+0x68/0x68
[ 11.158785] do_one_initcall+0x4b/0x1cb
[ 11.158785] ? init_setup+0x1b/0x28
[ 11.158785] kernel_init_freeable+0x1be/0x26b
[ 11.158785] ? loglevel+0x5b/0x5b
[ 11.158785] ? rest_init+0xb0/0xb0
[ 11.158785] kernel_init+0xa/0x110
[ 11.158785] ret_from_fork+0x22/0x40
[ 11.158785] Modules linked in:
[ 11.158785] ---[ end trace ba1c80a146740c8b ]---
[ 11.158785] RIP: 0010:get_partial_node.isra.76+0x33/0x2b0
[ 11.158785] Code: 89 e5 41 57 41 56 41 55 41 54 53 48 83 e4 f0 48 83 c4 80 48 85 f6 48 89 7c 24 30 48 89 54 24 10 89 4c 24 0c 0f 84 d5 00 00 00 <48> 83 7e 08 00 0f 84 ca 00 00 00 48 89 f7 48 89 74 24 38 e8 95 5e
[ 11.158785] RSP: 0018:ffffc900001078b0 EFLAGS: 00010002
[ 11.158785] RAX: 0000000000000000 RBX: 0000000000000202 RCX: 00000000006080c0
[ 11.158785] RDX: ffff889ffdae7150 RSI: 4c7a584873359cf2 RDI: ffff888107c07000
[ 11.158785] RBP: ffffc90000107958 R08: ffff888107c07000 R09: 0000000000000001
[ 11.158785] R10: 00000000006080c0 R11: 0000000000000002 R12: ffff889ffdae7140
[ 11.158785] R13: ffff888107c07000 R14: ffff888107c07000 R15: 0000000000000002
[ 11.158785] FS: 0000000000000000(0000) GS:ffff889ffdac0000(0000) knlGS:0000000000000000
[ 11.158785] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 11.158785] CR2: 0000000000000000 CR3: 00008014bc20a000 CR4: 00000000003406e0
[ 11.158785] Kernel panic - not syncing: Fatal exception
[ 11.158785] ---[ end Kernel panic - not syncing: Fatal exception ]---

In acpi_get_node(), if I replace "return acpi_map_pxm_to_node(pxm);" with
"return acpi_map_pxm_to_online_node(pxm);" then the system successfully
boots. I'm just not sure if that should be the proper approach or if
NUMA_NO_NODE should be returned if the _PXM value is outside the defined
entries.

I was also able to trigger this GPF by returning a bogus _PXM value on the
EPYC system that had a valid SRAT table. So it definitely would be worth
validating the PXM value before returning it.

Thanks,
Tom

>
> It seems very strange that the commit in question would cause a hang so
> early. Do you have a serial console hooked up for the earlyprintk? Is the
> serial port set up in legacy mode (e.g. 0x3f8 as opposed to being an MMIO
> device that would require a driver)?
>
> Can you dump the ACPI tables / run them through iasl to see what the _PXM
> values are in the DSDT table?
>
> Thanks,
> Tom
>
>>