Re: [PATCH] pciehp: fix a race between pciehp and removing operations by sysfs

[Lukas Wunner]

On Fri, Aug 02, 2019 at 04:23:33PM +0800, Xiongfeng Wang wrote:
> If I use a global flag to mark if any pci device is being rescaned or
> removed, the problem is that we can't remove two devices belonging to
> two root ports at the same time.
> Since a root port produces a pci tree, so I was planning to make the
> flag per root port slot. I mean add the flag in 'struct slot'.
>  But in some situation, the root port doesn't support hotplug and the
> downport below the root port support hotplug. I am not sure if it's
> better to add the flag in 'struct pci_dev' of the root port.

We're susceptible to deadlocks if at least two hotplug ports are removed
simultaneously where one is a parent of the other.

What you're witnessing is basically a variation of that problem wherein
a hotplug port is removed while it is simultaneously removing its
children.

pci_lock_rescan_remove(), which was introduced by commit 9d16947b7583
to fix races (which are real), at the same time caused these deadlocks.
The lock is too coarse-grained and needs to be replaced with more
fine-grained locking.

Specifically, unbinding PCI devices from drivers on removal need not
and should not happen under that lock.  That will fix all the deadlocks.

I've submitted a patch last year to address one class of those deadlocks
but withdrew it as I realized it's not a proper fix:

https://patchwork.kernel.org/patch/10468065/

What you can do is add a flag to struct pci_dev (or the priv_flags
embedded therein) to indicate that a device is about to be removed.
Set this flag on all children of the device being removed before
acquiring pci_lock_rescan_remove() and avoid taking that lock in
pciehp_unconfigure_device() if the flag is set on the hotplug port.

But again, that approach is just a band-aid and the real fix is to
unbind devices without holding the lock.

Thanks,

Lukas