Re: [PATCH 1/2] PCI: Clear LBMS on resume to avoid Target Speed quirk

[Bjorn Helgaas]

On Mon, Jan 29, 2024 at 01:27:09PM +0200, Ilpo Järvinen wrote:
> While a device is runtime suspended along with its PCIe hierarchy, the
> device could get disconnected. Because of the suspend, the device
> disconnection cannot be detected until portdrv/hotplug have resumed. On
> runtime resume, pcie_wait_for_link_delay() is called:
> 
>   pci_pm_runtime_resume()
>     pci_pm_bridge_power_up_actions()
>       pci_bridge_wait_for_secondary_bus()
>         pcie_wait_for_link_delay()
> 
> Because the device is already disconnected, this results in cascading
> failures:
> 
>   1. pcie_wait_for_link_status() returns -ETIMEDOUT.
> 
>   2. After the commit a89c82249c37 ("PCI: Work around PCIe link
>      training failures"),

I this this also depends on the merge resolution in 1abb47390350
("Merge branch 'pci/enumeration'").  Just looking at a89c82249c37 in
isolation suggests that pcie_wait_for_link_status() returning
-ETIMEDOUT would not cause pcie_wait_for_link_delay() to call
pcie_failed_link_retrain().

>      pcie_failed_link_retrain() spuriously detects
>      this failure as a Link Retraining failure and attempts the Target
>      Speed trick, which also fails.

Based on the comment below, I guess "Target Speed trick" probably
refers to the "retrain at 2.5GT/s, then remove the speed restriction
and retrain again" part of pcie_failed_link_retrain() (which I guess
is basically the entire point of the function)?

>   3. pci_bridge_wait_for_secondary_bus() then calls pci_dev_wait() which
>      cannot succeed (but waits ~1 minute, delaying the resume).
> 
> The Target Speed trick (in step 2) is only used if LBMS bit (PCIe r6.1
> sec 7.5.3.8) is set. For links that have been operational before
> suspend, it is well possible that LBMS has been set at the bridge and
> remains on. Thus, after resume, LBMS does not indicate the link needs
> the Target Speed quirk. Clear LBMS on resume for bridges to avoid the
> issue.
> 
> Fixes: a89c82249c37 ("PCI: Work around PCIe link training failures")
> Reported-by: Mika Westerberg <mika.westerberg@xxxxxxxxxxxxxxx>
> Tested-by: Mika Westerberg <mika.westerberg@xxxxxxxxxxxxxxx>
> Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@xxxxxxxxxxxxxxx>
> ---
>  drivers/pci/pci-driver.c | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
> index 51ec9e7e784f..05a114962df3 100644
> --- a/drivers/pci/pci-driver.c
> +++ b/drivers/pci/pci-driver.c
> @@ -574,6 +574,12 @@ static void pci_pm_bridge_power_up_actions(struct pci_dev *pci_dev)
>  {
>  	int ret;
>  
> +	/*
> +	 * Clear LBMS on resume to avoid spuriously triggering Target Speed
> +	 * quirk in pcie_failed_link_retrain().
> +	 */
> +	pcie_capability_write_word(pci_dev, PCI_EXP_LNKSTA, PCI_EXP_LNKSTA_LBMS);
> +
>  	ret = pci_bridge_wait_for_secondary_bus(pci_dev, "resume");
>  	if (ret) {
>  		/*
> -- 
> 2.39.2
>