Re: [PATCH 24/25] PCI: Perform reset_resource() and build fail list in sync

From: Ondřej Jirman
Date: Mon Apr 14 2025 - 10:18:32 EST

Next message: Konrad Dybcio: "Re: [PATCH v1 2/2] arm64: dts: qcom: sa8775p: Add support for camss"
Previous message: kernel test robot: "Re: [PATCH v7 3/4] drm/panthor: Label all kernel BO's"
In reply to: Ilpo Järvinen: "Re: [PATCH 24/25] PCI: Perform reset_resource() and build fail list in sync"
Next in thread: Ilpo Järvinen: "Re: [PATCH 24/25] PCI: Perform reset_resource() and build fail list in sync"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On Mon, Apr 14, 2025 at 04:15:01PM +0300, Ilpo Järvinen wrote:
> On Mon, 14 Apr 2025, Ondřej Jirman wrote:
> > On Mon, Apr 14, 2025 at 12:52:15PM +0300, Ilpo Järvinen wrote:
> > > On Fri, 11 Apr 2025, Ondřej Jirman wrote:
> > >
> > > > Hello Ilpo,
> > > >
> > > > On Tue, Apr 01, 2025 at 08:38:48PM +0300, Ilpo Järvinen wrote:
> > > > > That log wasn't taken from a bad case but it doesn't matter anymore as I
> > > > > could test this with qemu myself, thanks for providing enough to make it
> > > > > easy to reproduce & test it locally :-).
> > > > >
> > > > > The problem is caused by assign+release cycle being destructive on start
> > > > > aligned resources because successful assigment overwrites the start field.
> > > > > I'll send a patch to fix the problem once I've given it a bit more thought
> > > > > as this resource fitting code is somewhat complex.
> > > >
> > > > BTW, same thing here on a different SoC:
> > > >
> > > > https://lore.kernel.org/lkml/hrcsm2bo4ysqj2ggejndlou32cdc7yiknnm5nrlcoz4d64wall@7te4dfqsoe3y/T/#u
> > > >
> > > > There are kernel logs there, too, although I don't have dynamic debug enabled
> > > > in the kernel.
> > > >
> > > > Interestingly, bisect pointed me initially to a different commit. Reverting
> > > > it helped, but just on one board (QuartzPro64).
> > >
> > > Hi,
> > >
> > > Since you didn't mention it, I guess you haven't tried the fix:
> > >
> > > https://patchwork.kernel.org/project/linux-pci/patch/20250403093137.1481-1-ilpo.jarvinen@xxxxxxxxxxxxxxx/
> >
> > This patch works. Thank you.
>
> Thanks for testing. If you feel confident enough, please send your
> Tested-by tag to its thread (not this one).
>
> > One difference compared to 6.14 that I'm noticing
> > in the kernel log is that "save config" sequences now are printed twice for
> > unpopulated port. Not sure if it's related to your patches. Previously it was
> > printed just once.
>
> I don't think it is related. The resource fitting/assignment changes
> should not impact PCI save/restore AFAIK (except by preventing device from
> working in the first place if necessary resources do not get assigned).
>
> PCI state saving has always seemed like that in most logs I've seen with
> dyndbg enabled, that is, the state is seemingly saved multiple times,
> often right after the previous save too. So TBH, I'm not convinced it's
> even something new. If you have backtraces to show those places that
> invoke pci_save_state() I can take a look but I don't expect much to
> come out of that.

You're right. It's unrelated. The traces:

[ 1.878732] show_stack+0x28/0x80 (C)
[ 1.878751] dump_stack_lvl+0x58/0x74
[ 1.878762] dump_stack+0x14/0x1c
[ 1.878772] pci_save_state+0x34/0x1e8
[ 1.878783] pcie_portdrv_probe+0x330/0x6a8
[ 1.878794] local_pci_probe+0x3c/0xa0
[ 1.878804] pci_device_probe+0xac/0x194
[ 1.878813] really_probe+0xbc/0x388
[ 1.878825] __driver_probe_device+0x78/0x144
[ 1.878837] driver_probe_device+0x38/0x110
[ 1.878850] __device_attach_driver+0xb0/0x150
[ 1.878862] bus_for_each_drv+0x6c/0xb0
[ 1.878873] __device_attach+0x98/0x1a0
[ 1.878886] device_attach+0x10/0x20
[ 1.878897] pci_bus_add_device+0x84/0x12c
[ 1.878911] pci_bus_add_devices+0x40/0x90
[ 1.878924] pci_host_probe+0x88/0xe4
[ 1.878933] dw_pcie_host_init+0x258/0x714
[ 1.878945] rockchip_pcie_probe+0x2f0/0x3f8
[ 1.878956] platform_probe+0x64/0xcc
[ 1.878965] really_probe+0xbc/0x388
[ 1.878977] __driver_probe_device+0x78/0x144
[ 1.878989] driver_probe_device+0x38/0x110
[ 1.879001] __device_attach_driver+0xb0/0x150
[ 1.879014] bus_for_each_drv+0x6c/0xb0
[ 1.879025] __device_attach+0x98/0x1a0
[ 1.879037] device_initial_probe+0x10/0x20
[ 1.879049] bus_probe_device+0xa8/0xb8
[ 1.879061] deferred_probe_work_func+0xa4/0xf0
[ 1.879072] process_one_work+0x13c/0x2bc
[ 1.879084] worker_thread+0x284/0x460
[ 1.879094] kthread+0xe4/0x1a0
[ 1.879107] ret_from_fork+0x10/0x20
[ 1.879121] pcieport 0002:20:00.0: save config 0x00: 0x35881d87
[ 1.879158] pcieport 0002:20:00.0: save config 0x04: 0x00100507
[ 1.879168] pcieport 0002:20:00.0: save config 0x08: 0x06040001
[ 1.879177] pcieport 0002:20:00.0: save config 0x0c: 0x00010000
[ 1.879221] pcieport 0002:20:00.0: save config 0x10: 0x00000000
[ 1.879232] pcieport 0002:20:00.0: save config 0x14: 0x00000000
[ 1.879242] pcieport 0002:20:00.0: save config 0x18: 0x00212120
[ 1.879251] pcieport 0002:20:00.0: save config 0x1c: 0x000000f0
[ 1.879260] pcieport 0002:20:00.0: save config 0x20: 0x0000fff0
[ 1.879270] pcieport 0002:20:00.0: save config 0x24: 0x0001fff1
[ 1.879279] pcieport 0002:20:00.0: save config 0x28: 0x00000000
[ 1.879289] pcieport 0002:20:00.0: save config 0x2c: 0x00000000
[ 1.879298] pcieport 0002:20:00.0: save config 0x30: 0x00000000
[ 1.879307] pcieport 0002:20:00.0: save config 0x34: 0x00000040
[ 1.879316] pcieport 0002:20:00.0: save config 0x38: 0x00000000
[ 1.879325] pcieport 0002:20:00.0: save config 0x3c: 0x00020194
[ 1.879376] pcieport 0002:20:00.0: vgaarb: pci_notify

second time it's from:

[ 2.004478] Workqueue: pm pm_runtime_work
[ 2.004488] Call trace:
[ 2.004491] show_stack+0x28/0x80 (C)
[ 2.004500] dump_stack_lvl+0x58/0x74
[ 2.004506] dump_stack+0x14/0x1c
[ 2.004511] pci_save_state+0x34/0x1e8
[ 2.004516] pci_pm_runtime_suspend+0xa8/0x1e0
[ 2.004521] __rpm_callback+0x3c/0x220
[ 2.004527] rpm_callback+0x6c/0x80
[ 2.004532] rpm_suspend+0xec/0x674
[ 2.004537] pm_runtime_work+0x104/0x114
[ 2.004542] process_one_work+0x13c/0x2bc
[ 2.004548] worker_thread+0x284/0x460
[ 2.004552] kthread+0xe4/0x1a0
[ 2.004559] ret_from_fork+0x10/0x20
[ 2.004567] pcieport 0002:20:00.0: save config 0x00: 0x35881d87
[ 2.004578] pcieport 0002:20:00.0: save config 0x04: 0x00100507
[ 2.004583] pcieport 0002:20:00.0: save config 0x08: 0x06040001
[ 2.004587] pcieport 0002:20:00.0: save config 0x0c: 0x00010000
[ 2.004591] pcieport 0002:20:00.0: save config 0x10: 0x00000000
[ 2.004596] pcieport 0002:20:00.0: save config 0x14: 0x00000000
[ 2.004600] pcieport 0002:20:00.0: save config 0x18: 0x00212120
[ 2.004604] pcieport 0002:20:00.0: save config 0x1c: 0x000000f0
[ 2.004608] pcieport 0002:20:00.0: save config 0x20: 0x0000fff0
[ 2.004613] pcieport 0002:20:00.0: save config 0x24: 0x0001fff1
[ 2.004617] pcieport 0002:20:00.0: save config 0x28: 0x00000000
[ 2.004621] pcieport 0002:20:00.0: save config 0x2c: 0x00000000
[ 2.004625] pcieport 0002:20:00.0: save config 0x30: 0x00000000
[ 2.004629] pcieport 0002:20:00.0: save config 0x34: 0x00000040
[ 2.004633] pcieport 0002:20:00.0: save config 0x38: 0x00000000
[ 2.004638] pcieport 0002:20:00.0: save config 0x3c: 0x00020194
[ 2.004662] pcieport 0002:20:00.0: PME# enabled

on 6.15-rc:

cat /sys/class/pci_bus/0002:20/device/0002:20:00.0/power/runtime_status OK
suspended

on 6.14:

cat /sys/class/pci_bus/0002:20/device/0002:20:00.0/power/runtime_status OK
active

So some other unrelated change in 6.15-rc just enabled RPM to suspend the unused
port.

Thnak you,
o.

> --
> i.
>
> > Kind regards,
> > o.
> >
> > rockchip-dw-pcie a40800000.pcie: PCI host bridge to bus 0002:20
> > pci_bus 0002:20: root bus resource [bus 20-2f]
> > pci_bus 0002:20: root bus resource [io 0x300000-0x3fffff] (bus address [0xf2100000-0xf21fffff])
> > pci_bus 0002:20: root bus resource [mem 0xf2200000-0xf2ffffff]
> > pci_bus 0002:20: root bus resource [mem 0x980000000-0x9bfffffff] (bus address [0x40000000-0x7fffffff])
> > pci_bus 0002:20: scanning bus
> > pci 0002:20:00.0: [1d87:3588] type 01 class 0x060400 PCIe Root Port
> > pci 0002:20:00.0: ROM [mem 0x00000000-0x0000ffff pref]
> > pci 0002:20:00.0: PCI bridge to [bus 01-ff]
> > pci 0002:20:00.0: bridge window [io 0x0000-0x0fff]
> > pci 0002:20:00.0: bridge window [mem 0x00000000-0x000fffff]
> > pci 0002:20:00.0: bridge window [mem 0x00000000-0x000fffff 64bit pref]
> > pci 0002:20:00.0: supports D1 D2
> > pci 0002:20:00.0: PME# supported from D0 D1 D3hot
> > pci 0002:20:00.0: PME# disabled
> > pci 0002:20:00.0: Adding to iommu group 9
> > pci 0002:20:00.0: vgaarb: pci_notify
> > pci_bus 0002:20: fixups for bus
> > pci 0002:20:00.0: scanning [bus 01-ff] behind bridge, pass 0
> > pci 0002:20:00.0: Primary bus is hard wired to 0
> > pci 0002:20:00.0: bridge configuration invalid ([bus 01-ff]), reconfiguring
> > pci 0002:20:00.0: scanning [bus 00-00] behind bridge, pass 1
> > pci_bus 0002:20: bus scan returning with max=21
> > pci 0002:20:00.0: ROM [mem 0xf2200000-0xf220ffff pref]: assigned
> > pci 0002:20:00.0: PCI bridge to [bus 21]
> > pci_bus 0002:20: resource 4 [io 0x300000-0x3fffff]
> > pci_bus 0002:20: resource 5 [mem 0xf2200000-0xf2ffffff]
> > pci_bus 0002:20: resource 6 [mem 0x980000000-0x9bfffffff]
> > pcieport 0002:20:00.0: vgaarb: pci_notify
> > pcieport 0002:20:00.0: assign IRQ: got 148
> > pcieport 0002:20:00.0: PME: Signaling with IRQ 157
> > pcieport 0002:20:00.0: AER: enabled with IRQ 158
> > pcieport 0002:20:00.0: bwctrl: enabled with IRQ 157
> > pcieport 0002:20:00.0: save config 0x00: 0x35881d87
> > pcieport 0002:20:00.0: save config 0x04: 0x00100507
> > pcieport 0002:20:00.0: save config 0x08: 0x06040001
> > pcieport 0002:20:00.0: save config 0x0c: 0x00010000
> > pcieport 0002:20:00.0: save config 0x10: 0x00000000
> > pcieport 0002:20:00.0: save config 0x14: 0x00000000
> > pcieport 0002:20:00.0: save config 0x18: 0x00212120
> > pcieport 0002:20:00.0: save config 0x1c: 0x000000f0
> > pcieport 0002:20:00.0: save config 0x20: 0x0000fff0
> > pcieport 0002:20:00.0: save config 0x24: 0x0001fff1
> > pcieport 0002:20:00.0: save config 0x28: 0x00000000
> > pcieport 0002:20:00.0: save config 0x2c: 0x00000000
> > pcieport 0002:20:00.0: save config 0x30: 0x00000000
> > pcieport 0002:20:00.0: save config 0x34: 0x00000040
> > pcieport 0002:20:00.0: save config 0x38: 0x00000000
> > pcieport 0002:20:00.0: save config 0x3c: 0x00020194
> > pcieport 0002:20:00.0: vgaarb: pci_notify
> > pcieport 0002:20:00.0: save config 0x00: 0x35881d87
> > pcieport 0002:20:00.0: save config 0x04: 0x00100507
> > pcieport 0002:20:00.0: save config 0x08: 0x06040001
> > pcieport 0002:20:00.0: save config 0x0c: 0x00010000
> > pcieport 0002:20:00.0: save config 0x10: 0x00000000
> > pcieport 0002:20:00.0: save config 0x14: 0x00000000
> > pcieport 0002:20:00.0: save config 0x18: 0x00212120
> > pcieport 0002:20:00.0: save config 0x1c: 0x000000f0
> > pcieport 0002:20:00.0: save config 0x20: 0x0000fff0
> > pcieport 0002:20:00.0: save config 0x24: 0x0001fff1
> > pcieport 0002:20:00.0: save config 0x28: 0x00000000
> > pcieport 0002:20:00.0: save config 0x2c: 0x00000000
> > pcieport 0002:20:00.0: save config 0x30: 0x00000000
> > pcieport 0002:20:00.0: save config 0x34: 0x00000040
> > pcieport 0002:20:00.0: save config 0x38: 0x00000000
> > pcieport 0002:20:00.0: save config 0x3c: 0x00020194
> > pcieport 0002:20:00.0: PME# enabled
> >
> >
> > > --
> > > i.
> > >
> > > > And this is iomem:
> > > >
> > > > 0010f000-0010f0ff : 10f000.sram sram@10f000
> > > > 00200000-e2bbffff : System RAM
> > > > 02010000-0474ffff : Kernel code
> > > > 04750000-0498ffff : reserved
> > > > 04990000-0508ffff : Kernel data
> > > > daa00000-e29fffff : reserved
> > > > e2bc0000-ecbbffff : reserved
> > > > e2bc0000-ecbbffff : reserved
> > > > ecbc0000-efffffff : System RAM
> > > > ecbc7000-ecbdffff : reserved
> > > > f0000000-f00fffff : a40000000.pcie config
> > > > f0200000-f0ffffff : pcie@fe150000
> > > > f0200000-f020ffff : 0000:00:00.0
> > > > f0300000-f03fffff : PCI Bus 0000:01
> > > > f0300000-f0303fff : 0000:01:00.0
> > > > f0300000-f0303fff : nvme
> > > > f0304000-f03040ff : 0000:01:00.0
> > > > f0304000-f03040ff : nvme
> > > > f2000000-f20fffff : a40800000.pcie config
> > > > f2200000-f2ffffff : pcie@fe170000
> > > > f2200000-f27fffff : PCI Bus 0002:21
> > > > f2200000-f220ffff : 0002:21:00.0
> > > > f2400000-f27fffff : 0002:21:00.0
> > > > f2800000-f280ffff : 0002:20:00.0
> > > > f3000000-f30fffff : a40c00000.pcie config
> > > > f3200000-f3ffffff : pcie@fe180000
> > > > f3200000-f320ffff : 0003:30:00.0
> > > > f3300000-f33fffff : PCI Bus 0003:31
> > > > f3300000-f3303fff : 0003:31:00.0
> > > > f3304000-f3304fff : 0003:31:00.0
> > > > f3304000-f3304fff : r8169
> > > > fb000000-fb1fffff : fb000000.gpu gpu@fb000000
> > > > fc00c100-fc3fffff : fc000000.usb usb@fc000000
> > > > fc400000-fc407fff : usb@fc400000
> > > > fc400000-fc407fff : xhci-hcd.10.auto usb@fc400000
> > > > fc40c100-fc7fffff : fc400000.usb usb@fc400000
> > > > fc800000-fc83ffff : fc800000.usb usb@fc800000
> > > > fc840000-fc87ffff : fc840000.usb usb@fc840000
> > > > fc880000-fc8bffff : fc880000.usb usb@fc880000
> > > > fc8c0000-fc8fffff : fc8c0000.usb usb@fc8c0000
> > > > fc900000-fc900dff : fc900000.iommu
> > > > fc910000-fc910dff : fc900000.iommu
> > > > fd600000-fd6fffff : fd600000.sram sram@fd600000
> > > > fd8a0000-fd8a00ff : fd8a0000.gpio gpio@fd8a0000
> > > > fdb50000-fdb507ff : fdb50000.video-codec video-codec@fdb50000
> > > > fdb50800-fdb5083f : fdb50800.iommu iommu@fdb50800
> > > > fdb80000-fdb8017f : fdb80000.rga rga@fdb80000
> > > > fdba0000-fdba07ff : fdba0000.video-codec video-codec@fdba0000
> > > > fdba0800-fdba083f : fdba0800.iommu iommu@fdba0800
> > > > fdba4800-fdba483f : fdba4800.iommu iommu@fdba4800
> > > > fdba8800-fdba883f : fdba8800.iommu iommu@fdba8800
> > > > fdbac800-fdbac83f : fdbac800.iommu iommu@fdbac800
> > > > fdc70000-fdc707ff : fdc70000.video-codec video-codec@fdc70000
> > > > fdd90000-fdd941ff : fdd90000.vop vop
> > > > fdd95000-fdd95fff : fdd90000.vop gamma-lut
> > > > fdd97e00-fdd97eff : fdd97e00.iommu iommu@fdd97e00
> > > > fdd97f00-fdd97fff : fdd97e00.iommu iommu@fdd97e00
> > > > fddf0000-fddf0fff : fddf0000.i2s i2s@fddf0000
> > > > fddf4000-fddf4fff : fddf4000.i2s i2s@fddf4000
> > > > fde80000-fde9ffff : fde80000.hdmi hdmi@fde80000
> > > > fdea0000-fdebffff : fdea0000.hdmi hdmi@fdea0000
> > > > fdee0000-fdee5fff : fdee0000.hdmi_receiver hdmi_receiver@fdee0000
> > > > fe060000-fe06ffff : fe060000.dfi dfi@fe060000
> > > > fe150000-fe15ffff : a40000000.pcie apb
> > > > fe170000-fe17ffff : a40800000.pcie apb
> > > > fe180000-fe18ffff : a40c00000.pcie apb
> > > > fe1b0000-fe1bffff : fe1b0000.ethernet ethernet@fe1b0000
> > > > fe210000-fe210fff : fe210000.sata sata@fe210000
> > > > fe2c0000-fe2c3fff : fe2c0000.mmc mmc@fe2c0000
> > > > fe2e0000-fe2effff : fe2e0000.mmc mmc@fe2e0000
> > > > fe470000-fe470fff : fe470000.i2s i2s@fe470000
> > > > fe600000-fe60ffff : GICD
> > > > fe680000-fe77ffff : GICR
> > > > fea10000-fea13fff : dma-controller@fea10000
> > > > fea10000-fea13fff : fea10000.dma-controller dma-controller@fea10000
> > > > fea30000-fea33fff : dma-controller@fea30000
> > > > fea30000-fea33fff : fea30000.dma-controller dma-controller@fea30000
> > > > feaa0000-feaa0fff : feaa0000.i2c i2c@feaa0000
> > > > feaf0000-feaf00ff : feaf0000.watchdog watchdog@feaf0000
> > > > feb20000-feb20fff : feb20000.spi spi@feb20000
> > > > feb50000-feb500ff : serial
> > > > fec00000-fec003ff : fec00000.tsadc tsadc@fec00000
> > > > fec10000-fec1ffff : fec10000.adc adc@fec10000
> > > > fec20000-fec200ff : fec20000.gpio gpio@fec20000
> > > > fec30000-fec300ff : fec30000.gpio gpio@fec30000
> > > > fec40000-fec400ff : fec40000.gpio gpio@fec40000
> > > > fec50000-fec500ff : fec50000.gpio gpio@fec50000
> > > > fec90000-fec90fff : fec90000.i2c i2c@fec90000
> > > > fed10000-fed13fff : dma-controller@fed10000
> > > > fed10000-fed13fff : fed10000.dma-controller dma-controller@fed10000
> > > > fed60000-fed61fff : fed60000.phy phy@fed60000
> > > > fed70000-fed71fff : fed70000.phy phy@fed70000
> > > > fed80000-fed8ffff : fed80000.phy phy@fed80000
> > > > fed90000-fed9ffff : fed90000.phy phy@fed90000
> > > > fee00000-fee000ff : fee00000.phy phy@fee00000
> > > > fee10000-fee100ff : fee10000.phy phy@fee10000
> > > > fee20000-fee200ff : fee20000.phy phy@fee20000
> > > > fee80000-fee9ffff : fee80000.phy phy@fee80000
> > > > ff001000-ff0effff : ff001000.sram sram@ff001000
> > > > 100000000-3fbffffff : System RAM
> > > > 3ec000000-3fbffffff : reserved
> > > > 3fc500000-3ffefffff : System RAM
> > > > 4f0000000-4ffffffff : System RAM
> > > > 4fc611000-4fc6d0fff : reserved
> > > > 4fc6d1000-4fded1fff : reserved
> > > > 4fded2000-4fdf91fff : reserved
> > > > 4fdf93000-4fdf96fff : reserved
> > > > 4fdf97000-4fdfabfff : reserved
> > > > 4fdfac000-4fe051fff : reserved
> > > > 4fe052000-4ffffffff : reserved
> > > > 900000000-93fffffff : pcie@fe150000
> > > > 900000000-93fffffff : 0000:00:00.0
> > > > 980000000-9bfffffff : pcie@fe170000
> > > > 9c0000000-9ffffffff : pcie@fe180000
> > > > a40000000-a403fffff : a40000000.pcie dbi
> > > > a40800000-a40bfffff : a40800000.pcie dbi
> > > > a40c00000-a40ffffff : a40c00000.pcie dbi
> > > >
> > > > Thank you,
> > > > o.
> > > >
> > > > > --
> > > > > i.
> > > >
> >

Next message: Konrad Dybcio: "Re: [PATCH v1 2/2] arm64: dts: qcom: sa8775p: Add support for camss"
Previous message: kernel test robot: "Re: [PATCH v7 3/4] drm/panthor: Label all kernel BO's"
In reply to: Ilpo Järvinen: "Re: [PATCH 24/25] PCI: Perform reset_resource() and build fail list in sync"
Next in thread: Ilpo Järvinen: "Re: [PATCH 24/25] PCI: Perform reset_resource() and build fail list in sync"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]