Re: [PATCH 2/2] PM: sleep: Fix runtime PM based cpuidle support

[Rafael J. Wysocki]

On Fri, Oct 22, 2021 at 2:57 PM Ulf Hansson <ulf.hansson@xxxxxxxxxx> wrote:
>
> On Fri, 22 Oct 2021 at 14:02, Rafael J. Wysocki <rafael@xxxxxxxxxx> wrote:
> >
> > On Fri, Oct 22, 2021 at 12:18 PM Ulf Hansson <ulf.hansson@xxxxxxxxxx> wrote:
> > >
> > > On Thu, 21 Oct 2021 at 21:56, Ulf Hansson <ulf.hansson@xxxxxxxxxx> wrote:
> > > >
> > > > On Thu, 21 Oct 2021 at 21:02, Rafael J. Wysocki <rafael@xxxxxxxxxx> wrote:
> > > > >
> > > > > On Thu, Oct 21, 2021 at 8:12 PM Ulf Hansson <ulf.hansson@xxxxxxxxxx> wrote:
> > > > > >
> > > > > > On Thu, 21 Oct 2021 at 18:33, Rafael J. Wysocki <rafael@xxxxxxxxxx> wrote:
> > > > > > >
> > > > > > > On Thu, Oct 21, 2021 at 6:17 PM Ulf Hansson <ulf.hansson@xxxxxxxxxx> wrote:
> > > > >
> > > > > [cut]
> > > > >
> > > > > > > So in theory you could check the pm_runtime_put_sync_suspend() return
> > > > > > > value and fall back to something like WFI if that's an error code.
> > > > > >
> > > > > > I have already tried that, but it simply got too complicated. The main
> > > > > > issue was that runtime PM could become disabled for the device in the
> > > > > > middle of executing the ->enter() callback.
> > > > >
> > > > > So IIUC the problem is that you cannot resume after suspending in that case.
> > > > >
> > > > > IOW, you need to guarantee that if the suspend is successful, the
> > > > > resume also will take place, but if the suspend fails, you basically
> > > > > don't care.
> > > >
> > > > Exactly.
> > > >
> > > > >
> > > > > > For example, if pm_runtime_get_sync() fails, I still need to make sure
> > > > > > the reference counting in genpd becomes correct - and I can't do that
> > > > > > using dev_pm_genpd_resume(). That's because it's not designed to be
> > > > > > called in this "unknown" suspend phase, but should be called after the
> > > > > > noirq phase and be properly balanced with dev_pm_genpd_suspend().
> > > > > >
> > > > > > In other words, the error path didn't work out for me.
> > > > >
> > > > > It should be sufficient to call wake_up_all_idle_cpus() in the suspend
> > > > > path before dpm_suspend_late(), because system suspend acquires a
> > > > > PM-runtime reference on every device.  IOW, it won't let any devices
> > > > > runtime-suspend, so if your power domain devices are resumed in that
> > > > > path, they will never suspend again in it and the
> > > > > pm_runtime_put_sync_suspend() in __psci_enter_domain_idle_state()
> > > > > becomes a reference counter management call which works regardless of
> > > > > whether or not PM runtime is disabled.
> > > >
> > > > That sounds like a great idea, this should work too! Then the question
> > > > is, how to make that call to wake_up_all_idle_cpus() to become
> > > > optional - or only invoked for the cpuidle drivers that need it.
> >
> > It need not be optional.
> >
> > For suspend-to-idle it doesn't matter, because all CPUs will be woken
> > up from idle shortly anyway.
> >
> > For other suspend variants this doesn't matter, because all secondary
> > CPUs will be taken offline shortly and the primary CPU will call into
> > the platform "sleep" handler.
> >
> > > >
> > > > In any case, I will try this out, thanks for the suggestion!
> > >
> > > I now recall that I have already tried this, but unfortunately it doesn't work.
> > >
> > > The problem is that the dev->power.syscore flag is set for the device,
> > > which makes device_prepare() to bail out early and skip calling
> > > pm_runtime_get_noresume().
> >
> > This needs to be fixed, then.
>
> So bumping the usage count even if the dev->power.syscore is set,
> should be fine? (And of course dropping it in the complete phase).

Yes, please see
https://patchwork.kernel.org/project/linux-pm/patch/5773062.lOV4Wx5bFT@kreacher/

It should have been done this way from the outset, but I messed up a
merge (and said that it was "trivial" :-/).

> I can work with that, let me try!