Re: PCIe can not rescan for new PCIe device ( FPGA board )

From: Bjorn Helgaas
Date: Wed Oct 12 2011 - 11:58:31 EST

Next message: Vivek Goyal: "Re: [Patch 1/4][kernel][slimdump] Add new elf-note of typeNT_NOCOREDUMP to capture slimdump"
Previous message: Stephen Warren: "RE: [PATCH v9 0/3] Add support for tegra2 based ventana board"
In reply to: Abdelghani Ouchabane: "Re: PCIe can not rescan for new PCIe device ( FPGA board )"
Next in thread: Abdelghani Ouchabane: "Re: PCIe can not rescan for new PCIe device ( FPGA board )"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On Wed, Oct 12, 2011 at 2:03 AM, Abdelghani Ouchabane
<abdelghani@xxxxxxxxx> wrote:
> On 11/10/11 17:22, Bjorn Helgaas wrote:
>>
>> How did you get this read to work? Is this in a different system?
>> Maybe the difference between this working scenario and the broken
>> scenario will have a clue.
>
> Hi,
>
> yes, it was on a different system. I will give you more details:
>
> Our old systems are based on the Congatec COM Express conga-B945 ( based
> on the Intel Mobile 945GME chipset ), and we are using a GPO line to control
> the power switch on the FPGA board that is connected using PCIe line 0. On
> the B945 this line is high by default during the boot ( The FPGA board gets
> powered while the system boots "this is the desired behaviour" ).
>
> But now we are working to use the new Congatec COM Express conga-BAF (
> based on the AMD Embedded G-Series Processors ), The GPO line is zero by
> default during the boot. So, after the system boots, we power the FPGA
> board on, but unfortunately the system can not access the FPGA registers.
>
> We are trying to solve this issue and get the FPGA board works without
> changing the hardware.
>
>> The BIOS left the bridge to bus 02 with all windows disabled, but Linux
>> allocated and enabled the windows as above, and we assigned BARs of device
>> 02:00.0 inside those windows. As far as I can tell, everything leading to
>> the FPGA is set up correctly. Can you try another, known-working (not your
>> FPGA), card in that slot? It still looks to me like a problem with your
>> FPGA. Bjorn
>
> Yes, I did the following test cases with a standard WiFi card ( Network
> controller: Ralink corp. RT2860 --- Belkin Device 817c --- Kernel modules:
> rt2800pci ):
>
> A - I booted the system without the express card.
>
> 1 - Introduce the express card after the system has booted up.
> 2 - lspci -vt doesn't show the express card, express card divers
> did not loaded (lsmod).
> 3 - Rescan the PCIe bus "echo 1 > /sys/bus/pci/rescan"
> 4 - The card gets detected (lspci), drivers get loaded (lsmod),
> But ifconfig doesn't show the interface (wlan0).
> dmesg :
> [ 159.849763] pci 0000:01:00.0: [1814:0781]
> type 0 class 0x000280
> [ 159.849932] pci 0000:01:00.0: reg 10: [mem
> 0x00000000-0x0000ffff]
> [ 159.850220] pci 0000:01:00.0: PME# supported
> from D0 D3hot
> [ 159.850350] pci 0000:01:00.0: PME# disabled
> [ 159.850601] radeon 0000:00:01.0: BAR 6: [???
> 0x00000000 flags 0x2] has bogus alignment
> [ 159.850787] pci 0000:01:00.0: BAR 0: assigned
> [mem 0x40400000-0x4040ffff]
> [ 159.850941] pci 0000:01:00.0: BAR 0: set to
> [mem 0x40400000-0x4040ffff] (PCI address [0x40400000-0x4040ffff])
> [ 159.879110] cfg80211: Calling CRDA to update
> world regulatory domain
> [ 159.920212] rt2800pci 0000:01:00.0: enabling
> device (0000 -> 0002)
> [ 159.920379] rt2800pci 0000:01:00.0: PCI INT A
> -> GSI 16 (level, low) -> IRQ 16
> [ 159.920551] rt2800pci 0000:01:00.0: setting
> latency timer to 64
> [ 159.930927] phy0 -> rt2x00pci_regbusy_read:
> Error - Indirect register access failed: offset=0x00000580, value=0xffffffff
> [ 159.941346] phy0 -> rt2x00pci_regbusy_read:
> Error - Indirect register access failed: offset=0x00000580, value=0xffffffff
> [ 159.951737] phy0 -> rt2x00pci_regbusy_read:
> Error - Indirect register access failed: offset=0x00000580, value=0xffffffff
> [ 159.962130] phy0 -> rt2x00pci_regbusy_read:
> Error - Indirect register access failed: offset=0x00000580, value=0xffffffff
> [ 159.972521] phy0 -> rt2x00pci_regbusy_read:
> Error - Indirect register access failed: offset=0x00000580, value=0xffffffff
> [ 159.982907] phy0 -> rt2x00pci_regbusy_read:
> Error - Indirect register access failed: offset=0x00000580, value=0xffffffff
> [ 159.993299] phy0 -> rt2x00pci_regbusy_read:
> Error - Indirect register access failed: offset=0x00000580, value=0xffffffff
> [ 160.003676] phy0 -> rt2x00pci_regbusy_read:
> Error - Indirect register access failed: offset=0x00000580, value=0xffffffff
> [ 160.014056] phy0 -> rt2x00pci_regbusy_read:
> Error - Indirect register access failed: offset=0x00000580, value=0xffffffff
> [ 160.024447] phy0 -> rt2x00pci_regbusy_read:
> Error - Indirect register access failed: offset=0x00000580, value=0xffffffff
> [ 160.034825] phy0 -> rt2x00pci_regbusy_read:
> Error - Indirect register access failed: offset=0x00000580, value=0xffffffff
> [ 160.045230] phy0 -> rt2x00pci_regbusy_read:
> Error - Indirect register access failed: offset=0x00000580, value=0xffffffff
> [ 160.055622] phy0 -> rt2x00pci_regbusy_read:
> Error - Indirect register access failed: offset=0x00000580, value=0xffffffff
> [ 160.066014] phy0 -> rt2x00pci_regbusy_read:
> Error - Indirect register access failed: offset=0x00000580, value=0xffffffff
> [ 160.079940] phy0 -> rt2x00pci_regbusy_read:
> Error - Indirect register access failed: offset=0x00000580, value=0xffffffff
> [ 160.093825] phy0 -> rt2x00pci_regbusy_read:
> Error - Indirect register access failed: offset=0x00000580, value=0xffffffff
> [ 160.107635] phy0 -> rt2x00pci_regbusy_read:
> Error - Indirect register access failed: offset=0x00000580, value=0xffffffff
> [ 160.111266] phy0 -> rt2800_init_eeprom: Error
> - Invalid RT chipset detected.
> [ 160.114860] phy0 -> rt2x00lib_probe_dev:
> Error - Failed to allocate device.
> [ 160.118496] rt2800pci 0000:01:00.0: PCI INT A
> disabled
>
>
> B - I booted the system with express card.
>
> 1 - Insert the express card when the system is off.
> 2 - Boot the system.
> 3 - The card gets detected (lspci), drivers get loaded (lsmod),
> But ifconfig shows the interface(wlan0).
>
>
> C - I booted the system with express card then removed it then inserted it.
>
> 1 - Insert the express card when the system is off.
> 2 - Boot the system
> 3 - The card gets detected (lspci), drivers get loaded (lsmod),
> But ifconfig shows the interface(wlan0).
> 4 - Remove from pci bus "echo 1 >
> /sys/bus/pci/devices/0000\:0?\:00.0/remove" - NOTE = ? depends on the slot.
> dmesg :
> [ 69.510267] rt2800pci 0000:01:00.0: PCI INT A
> disabled
> 5 - Remove the card.
> 6 - Plug the card back in.
> 7 - Rescan the bus "echo 1 > /sys/bus/pci/rescan".
> 8 - The card gets detected (lspci), drivers get loaded (lsmod),
> But ifconfig shows the interface(wlan1).
> dmesg :
> [ 208.328481] pci 0000:01:00.0: [1814:0781]
> type 0 class 0x000280
> [ 208.328658] pci 0000:01:00.0: reg 10: [mem
> 0x00000000-0x0000ffff]
> [ 208.328910] pci 0000:01:00.0: PME# supported
> from D0 D3hot
> [ 208.329068] pci 0000:01:00.0: PME# disabled
> [ 208.329328] radeon 0000:00:01.0: BAR 6: [???
> 0x00000000 flags 0x2] has bogus alignment
> [ 208.329516] pci 0000:01:00.0: BAR 0: assigned
> [mem 0xfea00000-0xfea0ffff]
> [ 208.329671] pci 0000:01:00.0: BAR 0: set to
> [mem 0xfea00000-0xfea0ffff] (PCI address [0xfea00000-0xfea0ffff])
> [ 208.330511] rt2800pci 0000:01:00.0: enabling
> device (0000 -> 0002)
> [ 208.330665] rt2800pci 0000:01:00.0: PCI INT A
> -> GSI 16 (level, low) -> IRQ 16
> [ 208.330834] rt2800pci 0000:01:00.0: setting
> latency timer to 64
> [ 208.341348] ieee80211 phy1: Selected rate
> control algorithm 'minstrel_ht'
> [ 208.342396] Registered led device:
> rt2800pci-phy1::radio
> [ 208.342575] Registered led device:
> rt2800pci-phy1::assoc
> [ 208.342752] Registered led device:
> rt2800pci-phy1::quality
> [ 208.428306] udev[1339]: renamed network
> interface wlan0 to wlan1
>
>
> As you can see that my system doesn't detect the express card in test case
> "A" and the values of registers are 0xffffffff, the same as in my FPGA
> board.

Thanks for the details.

If I understand correctly:

A - fails (card not present at power-on, added later)
B - works (card always present)
C - works (card present at power-on, later removed and re-added,
requires manual rescan)

I see this note in section 9.4.8 (BIOS setup) of the conga-BAF User's Guide:

Note: Unless the hotplug support for this port is enabled as well,
an unpopulated
port will still be disabled if no PCI Express device is connected.

If you don't have hotplug enabled in the BIOS setup, it sounds like it
might result in the behavior you're seeing. Do you have that enabled?

Bjorn
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Next message: Vivek Goyal: "Re: [Patch 1/4][kernel][slimdump] Add new elf-note of typeNT_NOCOREDUMP to capture slimdump"
Previous message: Stephen Warren: "RE: [PATCH v9 0/3] Add support for tegra2 based ventana board"
In reply to: Abdelghani Ouchabane: "Re: PCIe can not rescan for new PCIe device ( FPGA board )"
Next in thread: Abdelghani Ouchabane: "Re: PCIe can not rescan for new PCIe device ( FPGA board )"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]