Re: [PATCH 00/11] of: Fix DMA configuration for non-DT masters

[Robin Murphy]

On 25/09/2019 17:16, Rob Herring wrote:
> On Wed, Sep 25, 2019 at 10:30 AM Nicolas Saenz Julienne
> <nsaenzjulienne@xxxxxxx> wrote:
> > On Wed, 2019-09-25 at 16:09 +0100, Robin Murphy wrote:
> > > On 25/09/2019 15:52, Nicolas Saenz Julienne wrote:
> > > > On Tue, 2019-09-24 at 16:59 -0500, Rob Herring wrote:
> > > > > On Tue, Sep 24, 2019 at 1:12 PM Nicolas Saenz Julienne
> > > > > <nsaenzjulienne@xxxxxxx> wrote:
> > > > > > Hi All,
> > > > > > this series tries to address one of the issues blocking us from
> > > > > > upstreaming Broadcom's STB PCIe controller[1]. Namely, the fact that
> > > > > > devices not represented in DT which sit behind a PCI bus fail to get the
> > > > > > bus' DMA addressing constraints.
> > > > > >
> > > > > > This is due to the fact that of_dma_configure() assumes it's receiving a
> > > > > > DT node representing the device being configured, as opposed to the PCIe
> > > > > > bridge node we currently pass. This causes the code to directly jump
> > > > > > into PCI's parent node when checking for 'dma-ranges' and misses
> > > > > > whatever was set there.
> > > > > >
> > > > > > To address this I create a new API in OF - inspired from Robin Murphys
> > > > > > original proposal[2] - which accepts a bus DT node as it's input in
> > > > > > order to configure a device's DMA constraints. The changes go deep into
> > > > > > of/address.c's implementation, as a device being having a DT node
> > > > > > assumption was pretty strong.
> > > > > >
> > > > > > On top of this work, I also cleaned up of_dma_configure() removing its
> > > > > > redundant arguments and creating an alternative function for the special
> > > > > > cases
> > > > > > not applicable to either the above case or the default usage.
> > > > > >
> > > > > > IMO the resulting functions are more explicit. They will probably
> > > > > > surface some hacky usages that can be properly fixed as I show with the
> > > > > > DT fixes on the Layerscape platform.
> > > > > >
> > > > > > This was also tested on a Raspberry Pi 4 with a custom PCIe driver and
> > > > > > on a Seattle AMD board.
> > > > >
> > > > > Humm, I've been working on this issue too. Looks similar though yours
> > > > > has a lot more churn and there's some other bugs I've found.
> > > >
> > > > That's good news, and yes now that I see it, some stuff on my series is
> > > > overly
> > > > complicated. Specially around of_translate_*().
> > > >
> > > > On top of that, you removed in of_dma_get_range():
> > > >
> > > > -   /*
> > > > -    * At least empty ranges has to be defined for parent node if
> > > > -    * DMA is supported
> > > > -    */
> > > > -   if (!ranges)
> > > > -           break;
> > > >
> > > > Which I assumed was bound to the standard and makes things easier.
> > > >
> > > > > Can you test out this branch[1]. I don't have any h/w needing this,
> > > > > but wrote a unittest and tested with modified QEMU.
> > > >
> > > > I reviewed everything, I did find a minor issue, see the patch attached.
> > >
> > > WRT that patch, the original intent of "force_dma" was purely to
> > > consider a device DMA-capable regardless of the presence of
> > > "dma-ranges". Expecting of_dma_configure() to do anything for a non-OF
> > > device has always been bogus - magic paravirt devices which appear out
> > > of nowhere and expect to be treated as genuine DMA masters are a
> > > separate problem that we haven't really approached yet.
> >
> > I agree it's clearly abusing the function. I have no problem with the behaviour
> > change if it's OK with you.

Thinking about it, you could probably just remove that call from the Xen 
DRM driver now anyway - since the dma-direct rework, we lost the ability 
to set dma_dummy_ops by default, and NULL ops now represent what it 
(presumably) wants.

> > Robin, have you looked into supporting multiple dma-ranges? It's the next thing
> > we need for BCM STB's PCIe. I'll have a go at it myself if nothing is in the
> > works already.
>
> Multiple dma-ranges as far as configuring inbound windows should work
> already other than the bug when there's any parent translation. But if
> you mean supporting multiple DMA offsets and masks per device in the
> DMA API, there's nothing in the works yet.

There's also the in-between step of making of_dma_get_range() return a 
size based on all the dma-ranges entries rather than only the first one 
- otherwise, something like [1] can lead to pretty unworkable default 
masks. We implemented that when doing acpi_dma_get_range(), it's just 
that the OF counterpart never caught up.

Robin.

[1] 
http://linux-arm.org/git?p=linux-rm.git;a=commitdiff;h=a2814af56b3486c2985a95540a88d8f9fa3a699f