Re: [PATCH v2] [RFC PATCH] PCI: Update LTR threshold based on LTRME bit

[Bjorn Helgaas]

[+cc Vidya, Kenny]

On Tue, Mar 08, 2022 at 12:29:09AM +0530, Prasad Malisetty wrote:
> Update LTR threshold scale and value based on LTRME (Latency
> Tolenrance Reporting Mechanism) from device capabilities.
> 
> In ASPM driver, LTR threshold scale and value is updating
> based on tcommon_mode and t_poweron values. In kioxia NVMe,
> L1.2 is failing due to LTR threshold scale and value is
> greater values than max snoop/non snoop value.
> 
> In general, updated LTR threshold scale and value should be
> less than max snoop/non snoop value to enter the device
> into L1.2 state.

Interesting that you mention an L1.2 issue on a KIOXIA NVMe device.

Kenny also reported an L1 Substates issue related to a KIOXIA NVMe
device at [1].  That issue happened when saving/restoring the L1 SS
state for suspend/resume.

We ended up reverting 4257f7e008ea to avoid the problem, but when he
tested that commit later, the issue did not occur [2].

I don't know if there's a connection here, so this is just a heads-up
in case there is.

[1] https://lore.kernel.org/r/20201228040513.GA611645@bjorn-Precision-5520
[2] https://lore.kernel.org/r/3ca14a7-b726-8430-fe61-a3ac183a1088@xxxxxxxxx

> Signed-off-by: Prasad Malisetty <quic_pmaliset@xxxxxxxxxxx>
> 
> ---
> Changes since v1:
> 	- Added missing variable declaration in v1 patch.
> ---
>  drivers/pci/pcie/aspm.c | 12 +++++++++---
>  1 file changed, 9 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
> index a96b742..a67746c 100644
> --- a/drivers/pci/pcie/aspm.c
> +++ b/drivers/pci/pcie/aspm.c
> @@ -463,6 +463,7 @@ static void aspm_calc_l1ss_info(struct pcie_link_state *link,
>  	u32 val1, val2, scale1, scale2;
>  	u32 t_common_mode, t_power_on, l1_2_threshold, scale, value;
>  	u32 ctl1 = 0, ctl2 = 0;
> +	u32 cap;
>  	u32 pctl1, pctl2, cctl1, cctl2;
>  	u32 pl1_2_enables, cl1_2_enables;
>  
> @@ -499,9 +500,14 @@ static void aspm_calc_l1ss_info(struct pcie_link_state *link,
>  	 * Table 5-11.  T(POWER_OFF) is at most 2us and T(L1.2) is at
>  	 * least 4us.
>  	 */
> -	l1_2_threshold = 2 + 4 + t_common_mode + t_power_on;
> -	encode_l12_threshold(l1_2_threshold, &scale, &value);
> -	ctl1 |= t_common_mode << 8 | scale << 29 | value << 16;
> +	pcie_capability_read_dword(child, PCI_EXP_DEVCAP2, &cap);
> +	if (!(cap & PCI_EXP_DEVCAP2_LTR)) {
> +		l1_2_threshold = 2 + 4 + t_common_mode + t_power_on;
> +		encode_l12_threshold(l1_2_threshold, &scale, &value);
> +		ctl1 |= scale << 29 | value << 16;
> +	}
> +
> +	ctl1 |= t_common_mode;
>  
>  	/* Some broken devices only support dword access to L1 SS */
>  	pci_read_config_dword(parent, parent->l1ss + PCI_L1SS_CTL1, &pctl1);
> -- 
> QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member 
> of Code Aurora Forum, hosted by The Linux Foundation
>