Re: [BUG] Bisected Problem with LSI PCI FC Adapter

[Dirk Gouders]

Yinghai Lu <yinghai@xxxxxxxxxx> writes:

> On Fri, Sep 12, 2014 at 3:05 PM, Dirk Gouders <dirk@xxxxxxxxxxx> wrote:
>> Yinghai Lu <yinghai@xxxxxxxxxx> writes:
>>
>>> On Fri, Sep 12, 2014 at 1:54 PM, Dirk Gouders <dirk@xxxxxxxxxxx> wrote:
>>>> Yinghai Lu <yinghai@xxxxxxxxxx> writes:
>>>>
>>>>> On Fri, Sep 12, 2014 at 1:05 PM, Dirk Gouders <dirk@xxxxxxxxxxx> wrote:
>>>>>> Dirk Gouders <dirk@xxxxxxxxxxx> writes:
>>>> -[0000:00]-+-01.0-[01-02]--+-0d.0-[02]--
>>>>            |               +-0e.0  Broadcom BCM5785 [HT1000] SATA (PATA/IDE Mode)
>>>>            |               \-0e.1  Broadcom BCM5785 [HT1000] SATA (PATA/IDE Mode)
>>>>            +-02.0  Broadcom BCM5785 [HT1000] Legacy South Bridge
>>>>            +-02.1  Broadcom BCM5785 [HT1000] IDE
>>>>            +-02.2  Broadcom BCM5785 [HT1000] LPC
>>>>            +-03.0  Broadcom BCM5785 [HT1000] USB
>>>>            +-03.1  Broadcom BCM5785 [HT1000] USB
>>>>            +-03.2  Broadcom BCM5785 [HT1000] USB
>>>>            +-06.0-[03]--
>>>>            +-07.0-[04]--+-04.0  Broadcom Corporation NetXtreme BCM5780 Gigabit Ethernet
>>>>            |            \-04.1  Broadcom Corporation NetXtreme BCM5780 Gigabit Ethernet
>>>>            +-0a.0-[05]--
>>>>            +-0b.0-[06]--
>>>
>>> what is "lspci -tv" and "lspci -vvxxx" look like before your patch that cut down
>>> end bus to 07 ...?
>>
>> Rebooted with 3.16 kernel:
>>
>> # lspci -tv
>>
>> -[0000:00]-+-01.0-[01-02]--+-0d.0-[02]--
>>            |               +-0e.0  Broadcom BCM5785 [HT1000] SATA (PATA/IDE Mode)
>>            |               \-0e.1  Broadcom BCM5785 [HT1000] SATA (PATA/IDE Mode)
>>            +-02.0  Broadcom BCM5785 [HT1000] Legacy South Bridge
>>            +-02.1  Broadcom BCM5785 [HT1000] IDE
>>            +-02.2  Broadcom BCM5785 [HT1000] LPC
>>            +-03.0  Broadcom BCM5785 [HT1000] USB
>>            +-03.1  Broadcom BCM5785 [HT1000] USB
>>            +-03.2  Broadcom BCM5785 [HT1000] USB
>>            +-06.0-[03]--
>>            +-07.0-[04]--+-04.0  Broadcom Corporation NetXtreme BCM5780 Gigabit Ethernet
>>            |            \-04.1  Broadcom Corporation NetXtreme BCM5780 Gigabit Ethernet
>>            +-08.0-[05]--
>>            +-09.0-[06]--
>>            +-0a.0-[07]----00.0  LSI Logic / Symbios Logic FC949ES Fibre Channel Adapter
>>            +-0b.0-[08]--
>
> where are 00:08.0 and 00:09.0 going? Did you just skip them in "cut
> end bus patch" ?

Yes, I skip them in pci_scan_device() to get the unused bus number space
that 0a.0 and 0b.0 then can use when reconfiguration is triggered.

> Proper simulation could be:
> 1. remove 07:00.0
>     echo 1 > /sys/bus/pci/devices/0000:07:00.0/remove
> 2. change bus range in 00:0a.0 bus
>     setpci -s 00:0a.0 0x1c.l=0x00101000
> 3. remove 00:0a.0
>     echo 1 > /sys/bus/pci/devices/0000:00:0a.0/remove
> 4. rescan pci
>     echo 1 > /sys/bus/pci/rescan
> 5. reset pcie link from 00:0a.0
>     setpci -s 00:0a.0 0xc0.b=0x18
>     sleep 1s
>     setpci -s 00:0a.0 0xc0.b=0x08
> 6. rescan pci bus
>     echo 1 > /sys/bus/pci/rescan

OK, I did that (with 3.16) but the reconfiguration code doesn't
get triggered.  Here is what dmesg and lspci -tv say:

[  166.775570] pci_bus 0000:07: busn_res: [bus 07] is released
[  166.775744] pci_scan_child_bus: pci_bus 0000:00: scanning bus
[  166.775787] pci 0000:00:0a.0: [1166:0132] type 01 class 0x060400
[  166.775902] pci 0000:00:0a.0: System wakeup disabled by ACPI
[  166.776031] pci_scan_bridge: pci 0000:00:01.0: scanning [bus 01-02] behind bridge, pass 0
[  166.776038] pci_scan_child_bus: pci_bus 0000:01: scanning bus
[  166.776071] pci_scan_bridge: pci 0000:01:0d.0: scanning [bus 02-02] behind bridge, pass 0
[  166.776076] pci_scan_child_bus: pci_bus 0000:02: scanning bus
[  166.776114] pci_scan_child_bus: pci_bus 0000:02: bus scan returning with max=02
[  166.776119] pci_scan_bridge: pci 0000:01:0d.0: scanning [bus 02-02] behind bridge, pass 1
[  166.776124] pci_scan_child_bus: pci_bus 0000:01: bus scan returning with max=02
[  166.776130] pci_scan_bridge: pci 0000:00:06.0: scanning [bus 03-03] behind bridge, pass 0
[  166.776134] pci_scan_child_bus: pci_bus 0000:03: scanning bus
[  166.776163] pci_scan_child_bus: pci_bus 0000:03: bus scan returning with max=03
[  166.776168] pci_scan_bridge: pci 0000:00:07.0: scanning [bus 04-04] behind bridge, pass 0
[  166.776172] pci_scan_child_bus: pci_bus 0000:04: scanning bus
[  166.776197] pci_scan_child_bus: pci_bus 0000:04: bus scan returning with max=04
[  166.776202] pci_scan_bridge: pcieport 0000:00:08.0: scanning [bus 05-05] behind bridge, pass 0
[  166.776206] pci_scan_child_bus: pci_bus 0000:05: scanning bus
[  166.776210] pci_scan_child_bus: pci_bus 0000:05: bus scan returning with max=05
[  166.776215] pci_scan_bridge: pcieport 0000:00:09.0: scanning [bus 06-06] behind bridge, pass 0
[  166.776219] pci_scan_child_bus: pci_bus 0000:06: scanning bus
[  166.776223] pci_scan_child_bus: pci_bus 0000:06: bus scan returning with max=06
[  166.776227] pci_scan_bridge: pcieport 0000:00:0b.0: scanning [bus 08-08] behind bridge, pass 0
[  166.776231] pci_scan_child_bus: pci_bus 0000:08: scanning bus
[  166.776235] pci_scan_child_bus: pci_bus 0000:08: bus scan returning with max=08
[  166.776240] pci_scan_bridge: pci 0000:00:0a.0: scanning [bus 07-07] behind bridge, pass 0
[  166.776284] pci_scan_child_bus: pci_bus 0000:07: scanning bus
[  166.776300] pci 0000:07:00.0: [1000:0646] type 00 class 0x0c0400
[  166.776314] pci 0000:07:00.0: reg 0x10: [io  0xb000-0xb0ff]
[  166.776327] pci 0000:07:00.0: reg 0x14: [mem 0xff5fc000-0xff5fffff 64bit]
[  166.776340] pci 0000:07:00.0: reg 0x1c: [mem 0xff5e0000-0xff5effff 64bit]
[  166.776355] pci 0000:07:00.0: reg 0x30: [mem 0xff400000-0xff4fffff pref]
[  166.776396] pci 0000:07:00.0: supports D1 D2
[  166.778033] pci_scan_child_bus: pci_bus 0000:07: fixups for bus
[  166.778038] pci 0000:00:0a.0: PCI bridge to [bus 07]
[  166.778045] pci 0000:00:0a.0:   bridge window [io  0x0000-0x1fff]
[  166.778048] pci 0000:00:0a.0:   bridge window [mem 0xff300000-0xff5fffff]
[  166.778054] pci 0000:00:0a.0:   bridge window [mem 0xcfe00000-0xcfefffff 64bit pref]
[  166.778057] pci_scan_child_bus: pci_bus 0000:07: bus scan returning with max=07
[  166.778062] pci_scan_bridge: pci 0000:00:01.0: scanning [bus 01-02] behind bridge, pass 1
[  166.778068] pci_scan_bridge: pci 0000:00:06.0: scanning [bus 03-03] behind bridge, pass 1
[  166.778073] pci_scan_bridge: pci 0000:00:07.0: scanning [bus 04-04] behind bridge, pass 1
[  166.778079] pci_scan_bridge: pcieport 0000:00:08.0: scanning [bus 05-05] behind bridge, pass 1
[  166.778084] pci_scan_bridge: pcieport 0000:00:09.0: scanning [bus 06-06] behind bridge, pass 1
[  166.778089] pci_scan_bridge: pcieport 0000:00:0b.0: scanning [bus 08-08] behind bridge, pass 1
[  166.778094] pci_scan_bridge: pci 0000:00:0a.0: scanning [bus 07-07] behind bridge, pass 1
[  166.778098] pci_scan_child_bus: pci_bus 0000:00: bus scan returning with max=08
[  166.778152] pci 0000:00:0a.0: BAR 8: assigned [mem 0x80000000-0x802fffff]
[  166.778159] pci 0000:00:0a.0: BAR 9: assigned [mem 0x200000000-0x2000fffff 64bit pref]
[  166.778164] pci 0000:00:0a.0: BAR 7: assigned [io  0x1000-0x2fff]
[  166.778169] pci 0000:01:0d.0: PCI bridge to [bus 02]
[  166.778179] pci 0000:00:06.0: PCI bridge to [bus 03]
[  166.778188] pci 0000:07:00.0: BAR 6: assigned [mem 0x80000000-0x800fffff pref]
[  166.778193] pci 0000:07:00.0: BAR 3: assigned [mem 0x80100000-0x8010ffff 64bit]
[  166.778204] pci 0000:07:00.0: BAR 1: assigned [mem 0x80110000-0x80113fff 64bit]
[  166.778213] pci 0000:07:00.0: BAR 0: assigned [io  0x1000-0x10ff]
[  166.778218] pci 0000:00:0a.0: PCI bridge to [bus 07]
[  166.778221] pci 0000:00:0a.0:   bridge window [io  0x1000-0x2fff]
[  166.778226] pci 0000:00:0a.0:   bridge window [mem 0x80000000-0x802fffff]
[  166.778229] pci 0000:00:0a.0:   bridge window [mem 0x200000000-0x2000fffff 64bit pref]
[  166.778244] fixup_debug_start: pci 0000:00:0a.0: calling quirk_msi_ht_cap+0x0/0x50
[  166.778251] pci 0000:00:0a.0: Found enabled HT MSI Mapping
[  167.800394] pci_scan_child_bus: pci_bus 0000:00: scanning bus
[  167.800456] pci_scan_bridge: pci 0000:00:01.0: scanning [bus 01-02] behind bridge, pass 0
[  167.800463] pci_scan_child_bus: pci_bus 0000:01: scanning bus
[  167.800495] pci_scan_bridge: pci 0000:01:0d.0: scanning [bus 02-02] behind bridge, pass 0
[  167.800501] pci_scan_child_bus: pci_bus 0000:02: scanning bus
[  167.800536] pci_scan_child_bus: pci_bus 0000:02: bus scan returning with max=02
[  167.800541] pci_scan_bridge: pci 0000:01:0d.0: scanning [bus 02-02] behind bridge, pass 1
[  167.800547] pci_scan_child_bus: pci_bus 0000:01: bus scan returning with max=02
[  167.800552] pci_scan_bridge: pci 0000:00:06.0: scanning [bus 03-03] behind bridge, pass 0
[  167.800556] pci_scan_child_bus: pci_bus 0000:03: scanning bus
[  167.800584] pci_scan_child_bus: pci_bus 0000:03: bus scan returning with max=03
[  167.800589] pci_scan_bridge: pci 0000:00:07.0: scanning [bus 04-04] behind bridge, pass 0
[  167.800593] pci_scan_child_bus: pci_bus 0000:04: scanning bus
[  167.800618] pci_scan_child_bus: pci_bus 0000:04: bus scan returning with max=04
[  167.800623] pci_scan_bridge: pcieport 0000:00:08.0: scanning [bus 05-05] behind bridge, pass 0
[  167.800627] pci_scan_child_bus: pci_bus 0000:05: scanning bus
[  167.800631] pci_scan_child_bus: pci_bus 0000:05: bus scan returning with max=05
[  167.800635] pci_scan_bridge: pcieport 0000:00:09.0: scanning [bus 06-06] behind bridge, pass 0
[  167.800640] pci_scan_child_bus: pci_bus 0000:06: scanning bus
[  167.800643] pci_scan_child_bus: pci_bus 0000:06: bus scan returning with max=06
[  167.800648] pci_scan_bridge: pcieport 0000:00:0b.0: scanning [bus 08-08] behind bridge, pass 0
[  167.800652] pci_scan_child_bus: pci_bus 0000:08: scanning bus
[  167.800656] pci_scan_child_bus: pci_bus 0000:08: bus scan returning with max=08
[  167.800660] pci_scan_bridge: pcieport 0000:00:0a.0: scanning [bus 07-07] behind bridge, pass 0
[  167.800664] pci_scan_child_bus: pci_bus 0000:07: scanning bus
[  167.800668] pci_scan_child_bus: pci_bus 0000:07: bus scan returning with max=07
[  167.800673] pci_scan_bridge: pci 0000:00:01.0: scanning [bus 01-02] behind bridge, pass 1
[  167.800678] pci_scan_bridge: pci 0000:00:06.0: scanning [bus 03-03] behind bridge, pass 1
[  167.800683] pci_scan_bridge: pci 0000:00:07.0: scanning [bus 04-04] behind bridge, pass 1
[  167.800689] pci_scan_bridge: pcieport 0000:00:08.0: scanning [bus 05-05] behind bridge, pass 1
[  167.800693] pci_scan_bridge: pcieport 0000:00:09.0: scanning [bus 06-06] behind bridge, pass 1
[  167.800698] pci_scan_bridge: pcieport 0000:00:0b.0: scanning [bus 08-08] behind bridge, pass 1
[  167.800704] pci_scan_bridge: pcieport 0000:00:0a.0: scanning [bus 07-07] behind bridge, pass 1
[  167.800708] pci_scan_child_bus: pci_bus 0000:00: bus scan returning with max=08
[  167.800752] pci 0000:01:0d.0: PCI bridge to [bus 02]
[  167.800762] pci 0000:00:06.0: PCI bridge to [bus 03]

# lspci -tv
-[0000:00]-+-01.0-[01-02]--+-0d.0-[02]--
           |               +-0e.0  Broadcom BCM5785 [HT1000] SATA (PATA/IDE Mode)
           |               \-0e.1  Broadcom BCM5785 [HT1000] SATA (PATA/IDE Mode)
           +-02.0  Broadcom BCM5785 [HT1000] Legacy South Bridge
           +-02.1  Broadcom BCM5785 [HT1000] IDE
           +-02.2  Broadcom BCM5785 [HT1000] LPC
           +-03.0  Broadcom BCM5785 [HT1000] USB
           +-03.1  Broadcom BCM5785 [HT1000] USB
           +-03.2  Broadcom BCM5785 [HT1000] USB
           +-06.0-[03]--
           +-07.0-[04]--+-04.0  Broadcom Corporation NetXtreme BCM5780 Gigabit Ethernet
           |            \-04.1  Broadcom Corporation NetXtreme BCM5780 Gigabit Ethernet
           +-08.0-[05]--
           +-09.0-[06]--
           +-0a.0-[07]----00.0  LSI Logic / Symbios Logic FC949ES Fibre Channel Adapter
           +-0b.0-[08]--
           +-0c.0  Advanced Micro Devices, Inc. [AMD/ATI] ES1000
           +-18.0  Advanced Micro Devices, Inc. [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
           +-18.1  Advanced Micro Devices, Inc. [AMD] K8 [Athlon64/Opteron] Address Map
           +-18.2  Advanced Micro Devices, Inc. [AMD] K8 [Athlon64/Opteron] DRAM Controller
           +-18.3  Advanced Micro Devices, Inc. [AMD] K8 [Athlon64/Opteron] Miscellaneous Control
           +-19.0  Advanced Micro Devices, Inc. [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
           +-19.1  Advanced Micro Devices, Inc. [AMD] K8 [Athlon64/Opteron] Address Map
           +-19.2  Advanced Micro Devices, Inc. [AMD] K8 [Athlon64/Opteron] DRAM Controller
           \-19.3  Advanced Micro Devices, Inc. [AMD] K8 [Athlon64/Opteron] Miscellaneous Control
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/