Re: [PATCH 1/8] pci: Make sriov work with hotplug removal

[Bjorn Helgaas]

On Mon, Oct 17, 2011 at 4:24 PM, Yinghai Lu <yinghai.lu@xxxxxxxxxx> wrote:
> On 10/17/2011 03:12 PM, Bjorn Helgaas wrote:
>>
>> Maybe this is the best we can do, but it still doesn't seem ideal, and
>> it's certainly not obvious when reading the code.  It doesn't seem
>> right for the driver ->remove() method to be calling
>> pci_destroy_dev().   Won't the core data structures be corrupted if a
>> defective driver doesn't call pci_disable_sriov()?  Seems like we
>> could end up with a device that's been physically removed, but still
>> has pci_dev structs hanging around.
>
> i did add some print out in
>        pci_stop_bus_device
> when stop PF, that function is called for those VFs.
>
> also driver have to call pci_disable_sriov() and that is current design.

Yep.  But I don't have to like the current design :)  It doesn't seem
as robust as it could be.

It took me a long time to puzzle out what was happening here.  Here's
some possible changelog text that would have saved me a lot of time:

    The PCI hot-remove path calls pci_stop_bus_devices() via
    pci_remove_bus_device().

    pci_stop_bus_devices() traverses the bus->devices list (point A below),
    stopping each device in turn, which calls the driver remove() method.  When
    the device is an SR-IOV PF, the driver calls pci_disable_sriov(), which
    also uses pci_remove_bus_device() to remove the VF devices from the
    bus->devices list (point B).

        pci_remove_bus_device
          pci_stop_bus_device
            pci_stop_bus_devices(subordinate)
              list_for_each(bus->devices)             <-- A
                pci_stop_bus_device(PF)
                  ...
                    driver->remove
                      pci_disable_sriov
                        ...
                          pci_remove_bus_device(VF)
                              <remove from bus_list>  <-- B

    At B, we're changing the same list we're iterating through at A, so when
    the driver remove() method returns, the pci_stop_bus_devices() iterator has
    a pointer to a list entry that has already been freed.

    This patch avoids the problem by building a separate list of all PFs on
    the bus and traversing that at A instead of the bus->devices list.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/