Re: [PATCH v2 5/6] pci: dwc: pci-exynos: rework the driver to support Exynos5433 variant

[Marek Szyprowski]

Hi

On 27.10.2020 13:04, Marek Szyprowski wrote:
> On 26.10.2020 20:14, Rob Herring wrote:
>> On Fri, Oct 23, 2020 at 2:58 AM Marek Szyprowski
>> <m.szyprowski@xxxxxxxxxxx> wrote:
>>> From: Jaehoon Chung <jh80.chung@xxxxxxxxxxx>
>>>
>>> Exynos5440 SoC support has been dropped since commit 8c83315da1cf 
>>> ("ARM:
>>> dts: exynos: Remove Exynos5440"). Rework this driver to support DWC 
>>> PCIe
>>> variant found in the Exynos5433 SoCs.
>>>
>>> The main difference in Exynos5433 variant is lack of the MSI support
>>> (the MSI interrupt is not even routed to the CPU).
>>>
>>> Signed-off-by: Jaehoon Chung <jh80.chung@xxxxxxxxxxx>
>>> [mszyprow: reworked the driver to support only Exynos5433 variant,
>>>             simplified code, rebased onto current kernel code, added
>>>             regulator support, converted to the regular platform 
>>> driver,
>>>             removed MSI related code, rewrote commit message]
>>> Signed-off-by: Marek Szyprowski <m.szyprowski@xxxxxxxxxxx>
>>> Acked-by: Krzysztof Kozlowski <krzk@xxxxxxxxxx>
>>> ---
>>>   drivers/pci/controller/dwc/Kconfig      |   3 +-
>>>   drivers/pci/controller/dwc/pci-exynos.c | 358 
>>> ++++++++++--------------
>>>   drivers/pci/quirks.c                    |   1 +
>>>   3 files changed, 145 insertions(+), 217 deletions(-) 
...
>>> +static int __maybe_unused exynos_pcie_suspend_noirq(struct device 
>>> *dev)
>>> +{
>> Why noirq variant needed? Lot's of PCI host drivers do this and I've
>> yet to get a reason...
> Frankly, I have no idea, but switching to SET_LATE_SYSTEM_SLEEP_PM_OPS 
> breaks system suspend/resume operation - the board doesn't resume from 
> suspend. If this is really important I will add some more logs and try 
> to find what happens between late/early and noirq phases.

It looks that PCI framework does something with the device or controller 
in noirq phase, so the driver cannot shutdown the controller earlier. 
Here is a relevant part from the kernel log after system suspend/resume 
cycle captured with init_calldebug enabled:

$ dmesg | grep pci
brcmfmac 0000:01:00.0: calling pci_pm_suspend+0x0/0x248 @ 96, parent: 
0000:00:00.0
brcmfmac 0000:01:00.0: pci_pm_suspend+0x0/0x248 returned 0 after 650 usecs
pcieport 0000:00:00.0: calling pci_pm_suspend+0x0/0x248 @ 7, parent: 
pci0000:00
pcieport 0000:00:00.0: pci_pm_suspend+0x0/0x248 returned 0 after 85 usecs
exynos-pcie 15700000.pcie: calling platform_pm_suspend+0x0/0x68 @ 447, 
parent: soc@0
exynos-pcie 15700000.pcie: platform_pm_suspend+0x0/0x68 returned 0 after 
4 usecs
exynos_pcie_phy 15680000.pcie-phy: calling platform_pm_suspend+0x0/0x68 
@ 447, parent: soc@0
exynos_pcie_phy 15680000.pcie-phy: platform_pm_suspend+0x0/0x68 returned 
0 after 3 usecs
brcmfmac 0000:01:00.0: calling pci_pm_suspend_late+0x0/0x50 @ 448, 
parent: 0000:00:00.0
brcmfmac 0000:01:00.0: pci_pm_suspend_late+0x0/0x50 returned 0 after 4 usecs
pcieport 0000:00:00.0: calling pci_pm_suspend_late+0x0/0x50 @ 449, 
parent: pci0000:00
pcieport 0000:00:00.0: pci_pm_suspend_late+0x0/0x50 returned 0 after 4 usecs
exynos-pcie 15700000.pcie: calling exynos_pcie_suspend_late+0x0/0x30 @ 
447, parent: soc@0
exynos-pcie 15700000.pcie: exynos_pcie_suspend_late 439
exynos-pcie 15700000.pcie: exynos_pcie_suspend_late+0x0/0x30 returned 0 
after 17 usecs
brcmfmac 0000:01:00.0: calling pci_pm_suspend_noirq+0x0/0x278 @ 449, 
parent: 0000:00:00.0
brcmfmac 0000:01:00.0: pci_pm_suspend_noirq+0x0/0x278 returned 0 after 
24272 usecs
pcieport 0000:00:00.0: calling pci_pm_suspend_noirq+0x0/0x278 @ 448, 
parent: pci0000:00
pcieport 0000:00:00.0: pci_pm_suspend_noirq+0x0/0x278 returned 0 after 
196 usecs
exynos-pcie 15700000.pcie: calling exynos_pcie_suspend_noirq+0x0/0x40 @ 
447, parent: soc@0
exynos-pcie 15700000.pcie: exynos_pcie_suspend_noirq+0x0/0x40 returned 0 
after 653 usecs
exynos-pcie 15700000.pcie: calling exynos_pcie_resume_noirq+0x0/0x38 @ 
447, parent: soc@0
exynos-pcie 15700000.pcie: Link up
exynos-pcie 15700000.pcie: exynos_pcie_resume_noirq+0x0/0x38 returned 0 
after 91433 usecs
pcieport 0000:00:00.0: calling pci_pm_resume_noirq+0x0/0x140 @ 96, 
parent: pci0000:00
pcieport 0000:00:00.0: pci_pm_resume_noirq+0x0/0x140 returned 0 after 
316 usecs
brcmfmac 0000:01:00.0: calling pci_pm_resume_noirq+0x0/0x140 @ 143, 
parent: 0000:00:00.0
brcmfmac 0000:01:00.0: pci_pm_resume_noirq+0x0/0x140 returned 0 after 
25470 usecs
exynos-pcie 15700000.pcie: calling exynos_pcie_resume_late+0x0/0x30 @ 
447, parent: soc@0
exynos-pcie 15700000.pcie: exynos_pcie_resume_late 445
exynos-pcie 15700000.pcie: exynos_pcie_resume_late+0x0/0x30 returned 0 
after 24 usecs
pcieport 0000:00:00.0: calling pci_pm_resume_early+0x0/0x48 @ 449, 
parent: pci0000:00
pcieport 0000:00:00.0: pci_pm_resume_early+0x0/0x48 returned 0 after 3 usecs
brcmfmac 0000:01:00.0: calling pci_pm_resume_early+0x0/0x48 @ 448, 
parent: 0000:00:00.0
brcmfmac 0000:01:00.0: pci_pm_resume_early+0x0/0x48 returned 0 after 3 usecs
exynos_pcie_phy 15680000.pcie-phy: calling platform_pm_resume+0x0/0x60 @ 
447, parent: soc@0
exynos_pcie_phy 15680000.pcie-phy: platform_pm_resume+0x0/0x60 returned 
0 after 4 usecs
exynos-pcie 15700000.pcie: calling platform_pm_resume+0x0/0x60 @ 447, 
parent: soc@0
exynos-pcie 15700000.pcie: platform_pm_resume+0x0/0x60 returned 0 after 
4 usecs
pcieport 0000:00:00.0: calling pci_pm_resume+0x0/0xe0 @ 96, parent: 
pci0000:00
pcieport 0000:00:00.0: pci_pm_resume+0x0/0xe0 returned 0 after 54 usecs
brcmfmac 0000:01:00.0: calling pci_pm_resume+0x0/0xe0 @ 142, parent: 
0000:00:00.0
brcmfmac 0000:01:00.0: pci_pm_resume+0x0/0xe0 returned 0 after 554 usecs


Best regards
-- 
Marek Szyprowski, PhD
Samsung R&D Institute Poland