Re: [PATCH v2 1/2] PCI/ERR: Fix fatal error recovery for non-hotplug capable devices

[Yicong Yang]

Hi Jay,

I've tested the patches on my board, and they work well.

Thanks,
Yicong


On 2020/6/25 2:52, Jay Vosburgh wrote:
> Jay Vosburgh <jay.vosburgh@xxxxxxxxxxxxx> wrote:
>
>> sathyanarayanan.kuppuswamy@xxxxxxxxxxxxxxx wrote:
>>
>> From: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@xxxxxxxxxxxxxxx>
>>> Fatal (DPC) error recovery is currently broken for non-hotplug
>>> capable devices. With current implementation, after successful
>>> fatal error recovery, non-hotplug capable device state won't be
>>> restored properly. You can find related issues in following links.
>>>
>>> https://lkml.org/lkml/2020/5/27/290
>>> https://lore.kernel.org/linux-pci/12115.1588207324@famine/
>>> https://lkml.org/lkml/2020/3/28/328
>>>
>>> Current fatal error recovery implementation relies on hotplug handler
>>> for detaching/re-enumerating the affected devices/drivers on DLLSC
>>> state changes. So when dealing with non-hotplug capable devices,
>>> recovery code does not restore the state of the affected devices
>>> correctly. Correct implementation should call report_slot_reset()
>>> function after resetting the link to restore the state of the
>>> device/driver.
>>>
>>> So use PCI_ERS_RESULT_NEED_RESET as error status for successful
>>> reset_link() operation and use PCI_ERS_RESULT_DISCONNECT for failure
>>> case. PCI_ERS_RESULT_NEED_RESET error state will ensure slot_reset()
>>> is called after reset link operation which will also fix the above
>>> mentioned issue.
>>>
>>> [original patch is from jay.vosburgh@xxxxxxxxxxxxx]
>>> [original patch link https://lore.kernel.org/linux-pci/12115.1588207324@famine/]
>>> Fixes: 6d2c89441571 ("PCI/ERR: Update error status after reset_link()")
>>> Signed-off-by: Jay Vosburgh <jay.vosburgh@xxxxxxxxxxxxx>
>>> Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@xxxxxxxxxxxxxxx>
>> 	I've tested this patch set on one of our test machines, and it
>> resolves the issue.  I plan to test with other systems tomorrow.
> 	I've done testing on two different systems that exhibit the
> original issue and this patch set appears to behave as expected.
>
> 	Has anyone else (Yicong?) had an opportunity to test this?
>
> 	Can this be considered for acceptance, or is additional feedback
> or review needed?
>
> 	-J
>
>>> ---
>>> drivers/pci/pcie/err.c | 24 ++++++++++++++++++++++--
>>> 1 file changed, 22 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/drivers/pci/pcie/err.c b/drivers/pci/pcie/err.c
>>> index 14bb8f54723e..5fe8561c7185 100644
>>> --- a/drivers/pci/pcie/err.c
>>> +++ b/drivers/pci/pcie/err.c
>>> @@ -165,8 +165,28 @@ pci_ers_result_t pcie_do_recovery(struct pci_dev *dev,
>>> 	pci_dbg(dev, "broadcast error_detected message\n");
>>> 	if (state == pci_channel_io_frozen) {
>>> 		pci_walk_bus(bus, report_frozen_detected, &status);
>>> -		status = reset_link(dev);
>>> -		if (status != PCI_ERS_RESULT_RECOVERED) {
>>> +		/*
>>> +		 * After resetting the link using reset_link() call, the
>>> +		 * possible value of error status is either
>>> +		 * PCI_ERS_RESULT_DISCONNECT (failure case) or
>>> +		 * PCI_ERS_RESULT_NEED_RESET (success case).
>>> +		 * So ignore the return value of report_error_detected()
>>> +		 * call for fatal errors. Instead use
>>> +		 * PCI_ERS_RESULT_NEED_RESET as initial status value.
>>> +		 *
>>> +		 * Ignoring the status return value of report_error_detected()
>>> +		 * call will also help in case of EDR mode based error
>>> +		 * recovery. In EDR mode AER and DPC Capabilities are owned by
>>> +		 * firmware and hence report_error_detected() call will possibly
>>> +		 * return PCI_ERS_RESULT_NO_AER_DRIVER. So if we don't ignore
>>> +		 * the return value of report_error_detected() then
>>> +		 * pcie_do_recovery() would report incorrect status after
>>> +		 * successful recovery. Ignoring PCI_ERS_RESULT_NO_AER_DRIVER
>>> +		 * in non EDR case should not have any functional impact.
>>> +		 */
>>> +		status = PCI_ERS_RESULT_NEED_RESET;
>>> +		if (reset_link(dev) != PCI_ERS_RESULT_RECOVERED) {
>>> +			status = PCI_ERS_RESULT_DISCONNECT;
>>> 			pci_warn(dev, "link reset failed\n");
>>> 			goto failed;
>>> 		}
>>> -- 
>>> 2.17.1
> ---
> 	-Jay Vosburgh, jay.vosburgh@xxxxxxxxxxxxx
> .
>