Re: [3.11.4] Thunderbolt/PCI unplug oops in pci_pme_list_scan

[Yinghai Lu]

On Thu, Oct 24, 2013 at 10:13 PM, Yinghai Lu <yinghai@xxxxxxxxxx> wrote:
> On Thu, Oct 24, 2013 at 8:33 PM, Bjorn Helgaas <bhelgaas@xxxxxxxxxx> wrote:
>>
>>>>> Bisection points to 928bea964827d7824b548c1f8e06eccbbc4d0d7d .
>>>>
>>>> This is "PCI: Delay enabling bridges until they're needed" by Yinghai.
>>>
>>> that double disabling should be addressed by:
>>>
>>> https://lkml.org/lkml/2013/4/25/608
>>>
>>> [PATCH] PCI: Remove duplicate pci_disable_device for pcie port
>>
>> I'll look at that patch again.  I had some questions about it the
>> first time, but perhaps it makes more sense after 928bea9648 has been
>> applied.
>>
>> Andreas originally reported a GPF oops in pci_pme_list_scan().  I
>> posted a refcounting patch, which made the problem go away, but I
>> can't explain why, and I don't want to apply it without understanding
>> that.  Decoding his oops shows this:
>>
>>   24: 0f 1f 00             nopl   (%rax)
>>   27: 48 8b 50 10           mov    0x10(%rax),%rdx
>>   2b:* 48 8b 52 38           mov    0x38(%rdx),%rdx <-- trapping instruction
>>   2f: 48 85 d2             test   %rdx,%rdx
>>
>> %rax is the pci_dev pointer, so 0x10(%rax) is the dev->bus pointer,
>> which we put in %rdx.  The oops says %rdx = 6b6b6b6b6b6b6b6b, which is
>> POISON_FREE, so I think we loaded dev->bus out of a struct pci_dev
>> that has already been freed.
>>
>> pci_pme_list_scan() holds pci_pme_list_mutex while it traverses
>> pci_pme_list, and the pci_stop_and_remove_bus_device() path removes
>> the pci_dev by calling pci_pme_active(), which also holds
>> pci_pme_list_mutex, so I don't understand how pci_pme_list_scan() can
>> see a pci_dev that has already been freed.
>>
>> If I understand Andreas correctly, 928bea9648 also fixes the crash,
>> even without my refcounting change.  Can you explain why?
>
> 928bea will make the dev->enable_cnt increase wrongly, as we have
> pci_enable_device for child
>    pci_enable_bridge for parent
>      pci_enable_bridge for grandparent
>        pci_enable_device for grandparent
>    pci_enable_device for parent
>        pci_enable_brdige for grandparent
>          pci_enable_device for grandparent.  ===> looks like i read the code wrongly. This one is skipped as we have pci_is_enabled checking.

928bea delay bridge enabling, and it's pci_is_enabled checking will prevent
pci bridge get enabled second time, so enable_cnt will be only 1. instead of
2. if we enable bridge at first and then pcie port driver.

Yinghai
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/