Re: [PATCH v3] amba: Remove deferred device addition

[Saravana Kannan]

On Fri, Aug 27, 2021 at 1:54 AM Greg Kroah-Hartman
<gregkh@xxxxxxxxxxxxxxxxxxx> wrote:
>
> On Tue, Aug 24, 2021 at 12:26:16PM -0700, Saravana Kannan wrote:
> > On Mon, Mar 8, 2021 at 11:15 AM Saravana Kannan <saravanak@xxxxxxxxxx> wrote:
> > >
> > > On Sun, Mar 7, 2021 at 11:28 PM Marek Szyprowski
> > > <m.szyprowski@xxxxxxxxxxx> wrote:
> > > >
> > > > Hi Saravana,
> > > >
> > > > On 05.03.2021 19:02, Saravana Kannan wrote:
> > > > > On Fri, Mar 5, 2021 at 3:45 AM Marek Szyprowski
> > > > > <m.szyprowski@xxxxxxxxxxx> wrote:
> > > > >> On 04.03.2021 20:51, Saravana Kannan wrote:
> > > > >>> The uevents generated for an amba device need PID and CID information
> > > > >>> that's available only when the amba device is powered on, clocked and
> > > > >>> out of reset. So, if those resources aren't available, the information
> > > > >>> can't be read to generate the uevents. To workaround this requirement,
> > > > >>> if the resources weren't available, the device addition was deferred and
> > > > >>> retried periodically.
> > > > >>>
> > > > >>> However, this deferred addition retry isn't based on resources becoming
> > > > >>> available. Instead, it's retried every 5 seconds and causes arbitrary
> > > > >>> probe delays for amba devices and their consumers.
> > > > >>>
> > > > >>> Also, maintaining a separate deferred-probe like mechanism is
> > > > >>> maintenance headache.
> > > > >>>
> > > > >>> With this commit, instead of deferring the device addition, we simply
> > > > >>> defer the generation of uevents for the device and probing of the device
> > > > >>> (because drivers needs PID and CID to match) until the PID and CID
> > > > >>> information can be read. This allows us to delete all the amba specific
> > > > >>> deferring code and also avoid the arbitrary probing delays.
> > > > >>>
> > > > >>> Cc: Rob Herring <robh@xxxxxxxxxx>
> > > > >>> Cc: Ulf Hansson <ulf.hansson@xxxxxxxxxx>
> > > > >>> Cc: John Stultz <john.stultz@xxxxxxxxxx>
> > > > >>> Cc: Saravana Kannan <saravanak@xxxxxxxxxx>
> > > > >>> Cc: Linus Walleij <linus.walleij@xxxxxxxxxx>
> > > > >>> Cc: Sudeep Holla <sudeep.holla@xxxxxxx>
> > > > >>> Cc: Nicolas Saenz Julienne <nsaenzjulienne@xxxxxxx>
> > > > >>> Cc: Geert Uytterhoeven <geert+renesas@xxxxxxxxx>
> > > > >>> Cc: Marek Szyprowski <m.szyprowski@xxxxxxxxxxx>
> > > > >>> Cc: Russell King <linux@xxxxxxxxxxxxxxx>
> > > > >>> Signed-off-by: Saravana Kannan <saravanak@xxxxxxxxxx>
> > > > >>> ---
> > > > >>>
> > > > >>> v1 -> v2:
> > > > >>> - Dropped RFC tag
> > > > >>> - Complete rewrite to not use stub devices.
> > > > >>> v2 -> v3:
> > > > >>> - Flipped the if() condition for hard-coded periphids.
> > > > >>> - Added a stub driver to handle the case where all amba drivers are
> > > > >>>     modules loaded by uevents.
> > > > >>> - Cc Marek after I realized I forgot to add him.
> > > > >>>
> > > > >>> Marek,
> > > > >>>
> > > > >>> Would you mind testing this? It looks okay with my limited testing.
> > > > >> It looks it works fine on my test systems. I've checked current
> > > > >> linux-next and this patch. You can add:
> > > > >>
> > > > >> Tested-by: Marek Szyprowski <m.szyprowski@xxxxxxxxxxx>
> > > > > Hi Marek,
> > > > >
> > > > > Thanks! Does your test set up have amda drivers that are loaded based
> > > > > on uevents? That's the one I couldn't test.
> > > >
> > > > I've checked both, the built-in and all amba drivers compiled as
> > > > modules, loaded by udev. Both works fine here.
> > > >
> > > > >> I've briefly scanned the code and I'm curious how does it work. Does it
> > > > >> depend on the recently introduced "fw_devlink=on" feature? I don't see
> > > > >> other mechanism, which would trigger matching amba device if pm domains,
> > > > >> clocks or resets were not available on time to read pid/cid while adding
> > > > >> a device...
> > > > > No, it does not depend on fw_devlink or device links in any way.
> > > > >
> > > > > When a device is attempted to be probed (when it's added or during
> > > > > deferred probe), it's matched with all the drivers on the bus.
> > > > > When a new driver is registered to a bus, all devices in that bus are
> > > > > matched with the driver to see if they'll work together.
> > > > > That's how match is called. And match() can return -EPROBE_DEFER and
> > > > > that'll cause the device to be put in the deferred probe list by
> > > > > driver core.
> > > > >
> > > > > The tricky part in this patch was the uevent handling and the
> > > > > chicken-and-egg issue I talk about in the comments.
> > > >
> > > > Thanks for the explanation. This EPROBE_DEFER support in match()
> > > > callback must be something added after I crafted that periodic retry
> > > > based workaround.
> > > >
> > >
> > > I think it got in just a few months before your patches, but your
> > > patches worked :) I actually don't like match returning -EPROBE_DEFER,
> > > but I can work around it for some of my fw_devlink optimization plans.
> > >
> > > More context here:
> > > https://lore.kernel.org/lkml/CAGETcx_qO4vxTSyBtBR2k7fd_3rGJF42iBbJH37HPNw=FheDKg@xxxxxxxxxxxxxx/
> >
> > I just noticed that this still hasn't been picked up.
> >
> > Russell/Greg, can we please pick this up. This finally cleans up
> > deferred probing of AMBA devices so that they don't have any special
> > case.
>
> Now picked up, thanks.

This patch is apparently causing issues in a specific platform.
https://lore.kernel.org/lkml/df8e7756-8b0d-d7de-a9ff-3f6eb0ffa8a5@xxxxxxxxxx/

Even though it worked fine for Marek:
https://lore.kernel.org/lkml/077fcc5b-cd09-d587-6906-d10bcc991eaf@xxxxxxxxxxx/#t

Here's my current analysis:
https://lore.kernel.org/lkml/CAGETcx-N4+u0iw9n5ncx_9MNnTa3ViyesxsDD7xN3jtEPT-uBw@xxxxxxxxxxxxxx/

I'll leave it up to you on how to proceed -- revert or wait for another fix.

-Saravana