Re: [EXT] Re: [PATCH 2/3] arm64: dts: ls1028a: Add PCIe controller DT nodes

[Ard Biesheuvel]

On Fri, 17 May 2019 at 10:59, Arnd Bergmann <arnd@xxxxxxxx> wrote:
>
> On Fri, May 17, 2019 at 5:21 AM Xiaowei Bao <xiaowei.bao@xxxxxxx> wrote:
> > -----Original Message-----
> > From: Arnd Bergmann <arnd@xxxxxxxx>
> > On Wed, May 15, 2019 at 9:36 AM Xiaowei Bao <xiaowei.bao@xxxxxxx> wrote:
> > > Signed-off-by: Xiaowei Bao <xiaowei.bao@xxxxxxx>
> > > ---
> > >  arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi |   52 ++++++++++++++++++++++++
> > >  1 files changed, 52 insertions(+), 0 deletions(-)
> > >
> > > diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi b/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi
> > > index b045812..50b579b 100644
> > > --- a/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi
> > > +++ b/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi
> > > @@ -398,6 +398,58 @@
> > >                         status = "disabled";
> > >                 };
> > >
> > > +               pcie@3400000 {
> > > +                       compatible = "fsl,ls1028a-pcie";
> > > +                       reg = <0x00 0x03400000 0x0 0x00100000   /* controller registers */
> > > +                              0x80 0x00000000 0x0 0x00002000>; /* configuration space */
> > > +                       reg-names = "regs", "config";
> > > +                       interrupts = <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>, /* PME interrupt */
> > > +                                    <GIC_SPI 109 IRQ_TYPE_LEVEL_HIGH>; /* aer interrupt */
> > > +                       interrupt-names = "pme", "aer";
> > > +                       #address-cells = <3>;
> > > +                       #size-cells = <2>;
> > > +                       device_type = "pci";
> > > +                       dma-coherent;
> > > +                       num-lanes = <4>;
> > > +                       bus-range = <0x0 0xff>;
> > > +                       ranges = <0x81000000 0x0 0x00000000 0x80 0x00010000 0x0 0x00010000   /* downstream I/O */
> > > +                                 0x82000000 0x0 0x40000000 0x80 0x40000000 0x0 0x40000000>; /* non-prefetchable memory */
> >
> > Are you sure there is no support for 64-bit BARs or prefetchable memory?
> > [Xiaowei Bao] sorry for late reply, Thought that our Layerscape platform has not added prefetchable memory support in DTS, so this platform has not been added, I will submit a separate patch to add prefetchable memory support for all Layerscape platforms.
>
> Ok, thanks.
>
> > Of course, the prefetchable PCIE device can work in our boards, because the RC will
> > assign non-prefetchable memory for this device. We reserve 1G no-prefetchable
> > memory for PCIE device, it is enough for general devices.
>
> Sure, many devices work just fine, this is mostly a question of supporting those
> devices that do require multiple gigabytes, or that need prefetchable memory
> semantics to get the expected performance. GPUs are the obvious example,
> but I think there are others (infiniband?).
>

Some implementations of the Synopsys dw PCIe IP contain a 'root port'
(within quotes because it is not actually a root port but an arbitrary
set of MMIO registers that looks like a type 01 config region) that
does not permit the prefetchable bridge window BAR to be written (a
thing which is apparently permitted by the PCIe spec). So while the
host bridge is capable of supporting more than one MMIO BAR window,
the OS visible software interface does not expose this functionality

In practice, it probably doesn't matter, since the driver uses the
same iATU attributes for prefetchable and non-prefetchable windows,
but I guess 1 GB of MMIO BAR space is a bit restrictive for modern
systems.