Re: Regression in v5.0-rc1 with autosuspend hrtimers

From: Vincent Guittot
Date: Wed Jan 09 2019 - 09:12:40 EST

Please keep all thread list when replying :-)

On Wed, 9 Jan 2019 at 14:33, Ladislav Michl <ladis@xxxxxxxxxxxxxx> wrote:
>
> On Wed, Jan 09, 2019 at 02:24:37PM +0100, Vincent Guittot wrote:
> > On Wed, 9 Jan 2019 at 12:58, Ladislav Michl <ladis@xxxxxxxxxxxxxx> wrote:
> > >
> > > On Wed, Jan 09, 2019 at 12:27:57PM +0100, Vincent Guittot wrote:
> > > > On Wed, 9 Jan 2019 at 12:17, Ladislav Michl <ladis@xxxxxxxxxxxxxx> wrote:
> > > > >
> > > > > On Wed, Jan 09, 2019 at 02:42:18AM +0100, Vincent Guittot wrote:
> > > > > > Le Tuesday 08 Jan 2019 Ã 13:37:43 (-0800), Tony Lindgren a Ãcrit :
> > > > > > > * Vincent Guittot <vincent.guittot@xxxxxxxxxx> [190108 16:42]:
> > > > > > > > On Tue, 8 Jan 2019 at 16:53, Tony Lindgren <tony@xxxxxxxxxxx> wrote:
> > > > > > > > > Hmm so could it be that we now rely on timers that that may
> > > > > > > > > not be capable of waking up the system from idle states with
> > > > > > > > > hrtimer?
> > > > > > > >
> > > > > > > > With nohz and hrtimer enabled, timer relies on hrtimer to generate
> > > > > > > > the tick so you should use the same interrupt.
> > > > > > >
> > > > > > > OK yeah looks like that part is working just fine.
> > > > > > >
> > > > > > > Adding some printks and debugging over ssh, looks like
> > > > > > > omap8250_runtime_resume() gets called just fine based on a wakeirq,
> > > > > > > but then omap8250_runtime_suspend() runs immediately instead of
> > > > > > > waiting for the three second timeout.
> > > > > > >
> > > > > > > Lowering the autosuspend_delay_ms to 2100 ms makes things work again.
> > > > > > > Anything higher than 2200 ms seems to somehow time out immediately
> > > > > > > now :)
> > > > > >
> > > > > > This is quite close to the max ns of an int on arm 32bits
> > > > > >
> > > > > > Could you try the patch below ?
> > > > > >
> > > > > > ---
> > > > > > drivers/base/power/runtime.c | 2 +-
> > > > > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > > > >
> > > > > > diff --git a/drivers/base/power/runtime.c b/drivers/base/power/runtime.c
> > > > > > index 7062469..44c5c76 100644
> > > > > > --- a/drivers/base/power/runtime.c
> > > > > > +++ b/drivers/base/power/runtime.c
> > > > > > @@ -141,7 +141,7 @@ u64 pm_runtime_autosuspend_expiration(struct device *dev)
> > > > > >
> > > > > > last_busy = READ_ONCE(dev->power.last_busy);
> > > > > >
> > > > > > - expires = last_busy + autosuspend_delay * NSEC_PER_MSEC;
> > > > > > + expires = last_busy + (u64)(autosuspend_delay) * NSEC_PER_MSEC;
> > > > > > if (expires <= now)
> > > > > > expires = 0; /* Already expired. */
> > > > >
> > > > > Hmm, comment above function states it returns "the expiration time in jiffies
> > > > > (adjusted to be nonzero)", so there's probably more to fix...
> > > >
> > > > The comment is wrong and should be updated as commit 8234f6734c5d has
> > > > moved on hrtimer and expires is now in raw ns unit
> > >
> > > Yup. Who'll send a patch? Is it worth optimizing as bellow? I'm fine with doing
> >
> > You can send a patch for updating the comment.
> >
> > > both.
> >
> > Regarding proposal below:
> > - pm_runtime_autosuspend_expiration() returns u64 and not ktime_t
>
> Well, that's matter of adding ktime_to_ns (which is noop).
>
> > - use helper ktime_before/after to compare ktime_t value
> >
> > Using ktime helper function makes the code less readable and the
>
> That why I avoided it.

But you must use helper function if you use ktime_t
That's one main reason for using u64 instead of ktime_t

>
> > proposal make the function more difficult to read IMHO when you
> > compare
> > expires = last_busy + autosuspend_delay * NSEC_PER_MSEC;
> > with
> > expires = ktime_add_ns(ms_to_ktime(autosuspend_delay),
> > READ_ONCE(dev->power.last_busy));
>
> I agree, but it doea all the magic correctly, so you won't need
> to touch that code is ktime_t changes (I know, unlikely..)

But the current implementation doesn't care of any changes in ktime_t
as it uses raw ns

>
> > or when you compare
> > if (expires <= now)
> > with
> > if (ktime_after(now, expires))
> >
> > And I'm not sure that this will optimize the code at the end
>
> Checked generated code on ARM and x86 and gcc does pretty good job here.
>
> > Only the replacement of the "goto out" by return 0 would make sense IMO
>
> Well, main concern was not to call ktime_get at the beginning of function
> as it is not too cheap.

Doesn't the compiler optimize that to just before the test ?

>
> > Regards,
> > Vincent
> >
> > >
> > > > > You can also consider change like this (still does not return jiffies):
> > > > > diff --git a/drivers/base/power/runtime.c b/drivers/base/power/runtime.c
> > > > > index 70624695b6d5..c72eaf21a61c 100644
> > > > > --- a/drivers/base/power/runtime.c
> > > > > +++ b/drivers/base/power/runtime.c
> > > > > @@ -129,23 +129,20 @@ static void pm_runtime_cancel_pending(struct device *dev)
> > > > > u64 pm_runtime_autosuspend_expiration(struct device *dev)
> > > > > {
> > > > > int autosuspend_delay;
> > > > > - u64 last_busy, expires = 0;
> > > > > - u64 now = ktime_to_ns(ktime_get());
> > > > > + ktime_t expires;
> > > > >
> > > > > if (!dev->power.use_autosuspend)
> > > > > - goto out;
> > > > > + return 0;
> > > > >
> > > > > autosuspend_delay = READ_ONCE(dev->power.autosuspend_delay);
> > > > > if (autosuspend_delay < 0)
> > > > > - goto out;
> > > > > -
> > > > > - last_busy = READ_ONCE(dev->power.last_busy);
> > > > > + return 0;
> > > > >
> > > > > - expires = last_busy + autosuspend_delay * NSEC_PER_MSEC;
> > > > > - if (expires <= now)
> > > > > - expires = 0; /* Already expired. */
> > > > > + expires = ktime_add_ns(ms_to_ktime(autosuspend_delay),
> > > > > + READ_ONCE(dev->power.last_busy));
> > > > > + if (expires <= ktime_get())
> > > > > + return 0; /* Already expired. */
> > > > >
> > > > > - out:
> > > > > return expires;
> > > > > }
> > > > > EXPORT_SYMBOL_GPL(pm_runtime_autosuspend_expiration);
> > > > >
> > > > > Regards,
> > > > > ladis