Re: [PATCH v16 2/5] iommu/arm-smmu: Invoke pm_runtime during probe, add/remove device

[Robin Murphy]

Hi Vivek,

On 2018-09-25 6:56 AM, Vivek Gautam wrote:
> Hi Robin, Will,
>
> On Tue, Sep 18, 2018 at 8:41 AM Vivek Gautam
> <vivek.gautam@xxxxxxxxxxxxxx> wrote:
> > Hi Robin,
> >
> > On Fri, Sep 7, 2018 at 3:52 PM Vivek Gautam <vivek.gautam@xxxxxxxxxxxxxx> wrote:
> > > On Fri, Sep 7, 2018 at 3:22 PM Tomasz Figa <tfiga@xxxxxxxxxxxx> wrote:
> > > > On Fri, Sep 7, 2018 at 6:38 PM Vivek Gautam <vivek.gautam@xxxxxxxxxxxxxx> wrote:
> > > > > Hi Tomasz,
> > > > >
> > > > >
> > > > > On 9/7/2018 2:46 PM, Tomasz Figa wrote:
> > > > > > Hi Vivek,
> > > > > >
> > > > > > On Thu, Aug 30, 2018 at 11:46 PM Vivek Gautam
> > > > > > <vivek.gautam@xxxxxxxxxxxxxx> wrote:
> > > > > > > From: Sricharan R <sricharan@xxxxxxxxxxxxxx>
> > > > > > >
> > > > > > > The smmu device probe/remove and add/remove master device callbacks
> > > > > > > gets called when the smmu is not linked to its master, that is without
> > > > > > > the context of the master device. So calling runtime apis in those places
> > > > > > > separately.
> > > > > > > Global locks are also initialized before enabling runtime pm as the
> > > > > > > runtime_resume() calls device_reset() which does tlb_sync_global()
> > > > > > > that ultimately requires locks to be initialized.
> > > > > > >
> > > > > > > Signed-off-by: Sricharan R <sricharan@xxxxxxxxxxxxxx>
> > > > > > > [vivek: Cleanup pm runtime calls]
> > > > > > > Signed-off-by: Vivek Gautam <vivek.gautam@xxxxxxxxxxxxxx>
> > > > > > > Reviewed-by: Tomasz Figa <tfiga@xxxxxxxxxxxx>
> > > > > > > Tested-by: Srinivas Kandagatla <srinivas.kandagatla@xxxxxxxxxx>
> > > > > > > ---
> > > > > > >    drivers/iommu/arm-smmu.c | 89 +++++++++++++++++++++++++++++++++++++++++++-----
> > > > > > >    1 file changed, 81 insertions(+), 8 deletions(-)
> > > > > > [snip]
> > > > > > > @@ -2215,10 +2281,17 @@ static int arm_smmu_device_remove(struct platform_device *pdev)
> > > > > > >           if (!bitmap_empty(smmu->context_map, ARM_SMMU_MAX_CBS))
> > > > > > >                   dev_err(&pdev->dev, "removing device with active domains!\n");
> > > > > > >
> > > > > > > +       arm_smmu_rpm_get(smmu);
> > > > > > >           /* Turn the thing off */
> > > > > > >           writel(sCR0_CLIENTPD, ARM_SMMU_GR0_NS(smmu) + ARM_SMMU_GR0_sCR0);
> > > > > > > +       arm_smmu_rpm_put(smmu);
> > > > > > > +
> > > > > > > +       if (pm_runtime_enabled(smmu->dev))
> > > > > > > +               pm_runtime_force_suspend(smmu->dev);
> > > > > > > +       else
> > > > > > > +               clk_bulk_disable(smmu->num_clks, smmu->clks);
> > > > > > >
> > > > > > > -       clk_bulk_disable_unprepare(smmu->num_clks, smmu->clks);
> > > > > > > +       clk_bulk_unprepare(smmu->num_clks, smmu->clks);
> > > > > > Aren't we missing pm_runtime_disable() here? We'll have the enable
> > > > > > count unbalanced if the driver is removed and probed again.
> > > > >
> > > > > pm_runtime_force_suspend() does a pm_runtime_disable() also if i am not
> > > > > wrong.
> > > > > And, as mentioned in a previous thread [1], we were seeing a warning
> > > > > which we avoided
> > > > > by keeping force_suspend().
> > > > >
> > > > > [1] https://lkml.org/lkml/2018/7/8/124
> > > >
> > > > I see, thanks. I didn't realize that pm_runtime_force_suspend()
> > > > already disables runtime PM indeed. Sorry for the noise.
> > >
> > > Hi Tomasz,
> > > No problem. Thanks for looking back at it.
> > >
> > > Hi Robin,
> > > If you are fine with this series, then can you please consider giving
> > > Reviewed-by, so that we are certain that this series will go in the next merge
> > > window.
> > > Thanks
> >
> > Gentle ping.
> > You ack will be very helpful in letting Will pull this series for 4.20.
> > Thanks.
>
> I would really appreciate if you could provide your ack for this series.
> Or if there are any concerns, I am willing to address them.

Apologies, I thought I'd replied to say I'd be getting to this shortly, 
but apparently not :(

FWIW, "shortly" is now tomorrow - I don't *think* there's anything 
outstanding, but given the number of subtleties we've turned up so far I 
do just want one last thorough double-check to make sure.

Thanks,
Robin.