RE: Why does BIOS assign memory to 16 byte BAR

[Bharat Kumar Gogada]

> Subject: Re: Why does BIOS assign memory to 16 byte BAR
> 
> On Fri, Jul 22, 2016 at 10:15:46AM -0500, Bjorn Helgaas wrote:
> > Hi Bharat,
> >
> > On Fri, Jul 22, 2016 at 09:24:22AM +0000, Bharat Kumar Gogada wrote:
> > > Hi,
> > >
> > > I'm observing that on x86 BIOS successfully assigns memory if an End
> > > point requests BAR of size 16byte.
> > >
> > > But as per Spec:
> > > The minimum memory address range requested by a BAR is 128 bytes.
> >
> > Can you provide the spec reference for this?  I don't see it in PCI
> > r3.0.
> >
> > PCI r3.0, sec 6.2.5.1, shows bits 4-31 of a memory BAR as writable,
> > which would correspond to a minimum size of 16 bytes.
> 
> The reference above is to the conventional PCI spec.  I happened to trip over
> a note in PCIe r3.0, sec 1.3.2.2, that for a PCI Express endpoint, "the minimum
> memory address range requested by a BAR is 128 bytes."
> 
> I don't think linux currently enforces this minimum.
> 

Hi Bjorn Thanks for the reply.

Here is what the issue we are seeing.

We have total memory for BAR's on our SoC of 256 MB.
When an End Point request individually 16 byte BAR's our root port assigns memory to BAR's successfully.

But if I have an End point which has 4 BAR's each 32 bit and request as following:
When 1st BAR requests 1GB BAR it fails due to lack of memory. (We are running this as part of SIG compliance test case)
2nd BAR requests 1MB and other 2 BAR's request 16byte, but these are not getting BAR's assigned. (Even though BAR space is available, since 1GB failed,
We have 256 MB still)

We have only one End point connected to our root port.

Here is the log:
[    2.319289] nwl-pcie fd0e0000.pcie: Link is UP
[    2.319332] PCI host bridge /amba/pcie@fd0e0000 ranges:
[    2.319349]   No bus range found for /amba/pcie@fd0e0000, using [bus 00-ff]
[    2.319374]    IO 0xe0000000..0xe000ffff -> 0x00000000
[    2.319415]   MEM 0xe0100000..0xefffffff -> 0xe0100000
[    2.319431]   MEM 0x600000000..0x7ffffffff -> 0x600000000
[    2.319539] nwl-pcie fd0e0000.pcie: PCI host bridge to bus 0000:00
[    2.319557] pci_bus 0000:00: root bus resource [bus 00-ff]
[    2.319573] pci_bus 0000:00: root bus resource [io  0x0000-0xffff]
[    2.319589] pci_bus 0000:00: root bus resource [mem 0xe0100000-0xefffffff]
[    2.319606] pci_bus 0000:00: root bus resource [mem 0x600000000-0x7ffffffff pref]
[    2.319845] pci 0000:00:00.0: cannot attach to SMMU, is it on the same bus?
[    2.319861] iommu: Adding device 0000:00:00.0 to group 1
[    2.320243] pci 0000:01:00.0: cannot attach to SMMU, is it on the same bus?
[    2.320258] iommu: Adding device 0000:01:00.0 to group 1
[    2.320313] pci 0000:00:00.0: BAR 8: no space for [mem size 0x60000000]
[    2.320331] pci 0000:00:00.0: BAR 8: failed to assign [mem size 0x60000000]
[    2.320349] pci 0000:00:00.0: BAR 6: assigned [mem 0xe0100000-0xe01007ff pref]
[    2.320374] pci 0000:01:00.0: BAR 0: no space for [mem size 0x40000000]
[    2.320390] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x40000000]
[    2.320407] pci 0000:01:00.0: BAR 4: no space for [mem size 0x00100000 64bit]
[    2.320423] pci 0000:01:00.0: BAR 4: failed to assign [mem size 0x00100000 64bit]
[    2.320446] pci 0000:01:00.0: BAR 2: no space for [mem size 0x00000010]
[    2.320461] pci 0000:01:00.0: BAR 2: failed to assign [mem size 0x00000010]
[    2.320477] pci 0000:01:00.0: BAR 3: no space for [mem size 0x00000010]
[    2.320493] pci 0000:01:00.0: BAR 3: failed to assign [mem size 0x00000010]
[    2.320509] pci 0000:00:00.0: PCI bridge to [bus 01-0c]

Please let me know, what might might be the issue.

Thanks & Regards,
Bharat