Re: [RFT PATCH v3 12/27] of/address: Add infrastructure to declare MMIO as non-posted

[Rob Herring]

On Thu, Mar 11, 2021 at 2:12 AM Arnd Bergmann <arnd@xxxxxxxxxx> wrote:
>
> On Wed, Mar 10, 2021 at 6:01 PM Rob Herring <robh@xxxxxxxxxx> wrote:
> >
> > On Wed, Mar 10, 2021 at 1:27 AM Hector Martin <marcan@xxxxxxxxx> wrote:
> > >
> > > On 10/03/2021 07.06, Rob Herring wrote:
> > > >> My main concern here is that this creates an inconsistency in the device
> > > >> tree representation that only works because PCI drivers happen not to
> > > >> use these code paths. Logically, having "nonposted-mmio" above the PCI
> > > >> controller would imply that it applies to that bus too. Sure, it doesn't
> > > >> matter for Linux since it is ignored, but this creates an implicit
> > > >> exception that PCI buses always use posted modes.
> > > >
> > > > We could be stricter that "nonposted-mmio" must be in the immediate
> > > > parent. That's kind of in line with how addressing already works.
> > > > Every level has to have 'ranges' to be an MMIO address, and the
> > > > address cell size is set by the immediate parent.
> > > >
> > > >> Then if a device comes along that due to some twisted fabric logic needs
> > > >> nonposted nGnRnE mappings for PCIe (even though the actual PCIe ops will
> > > >> end up posted at the bus anyway)... how do we represent that? Declare
> > > >> that another "nonposted-mmio" on the PCIe bus means "no, really, use
> > > >> nonposted mmio for this"?
> > > >
> > > > If we're strict, yes. The PCI host bridge would have to have "nonposted-mmio".
> > >
> > > Works for me; then let's just make it non-recursive.
> > >
> > > Do you think we can get rid of the Apple-only optimization if we do
> > > this? It would mean only looking at the parent during address
> > > resolution, not recursing all the way to the top, so presumably the
> > > performance impact would be quite minimal.
>
> Works for me.
>
> > Yeah, that should be fine. I'd keep an IS_ENABLED() config check
> > though. Then I'll also know if anyone else needs this.
>
> Ok, makes sense.
>
> Conceptually, I'd like to then see a check that verifies that the
> property is only set for nodes whose parent also has it set, since
> that is how AXI defines it: A bus can wait for the ack from its
> child node, or it can acknowledge the write to its parent early.
> However, this breaks down as soon as a bus does the early ack:
> all its children by definition use posted writes (as seen by the
> CPU), even if they wait for stores that come from other masters.
>
> Does this make sense to you?

BTW, I don't think it's clear in this thread, but the current
definition proposed for the spec[1] and schema is 'nonposted-mmio' is
specific to 'simple-bus'. I like this restriction and we can expand
where 'nonposted-mmio' is allowed later if needed.

It's possible to express in json-schema, but I think it wouldn't be
pretty. Json-schema is not great for expressing inter-property
constraints and it gets worse if we're talking inter-node constraints.
I'd like to define a way to do python snippets of code for something
like this, but that's way down on the wish list.

Rob

[1] https://github.com/devicetree-org/devicetree-specification/pull/40