Re: timers: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected

[Paul E. McKenney]

On Tue, Jan 12, 2016 at 09:18:56PM +0100, Peter Zijlstra wrote:
> On Tue, Jan 12, 2016 at 03:03:27PM -0500, Sasha Levin wrote:
> > [ 3408.703754] Call Trace:
> 
> > [ 3408.733192] rcu_read_unlock_special (kernel/rcu/tree_plugin.h:503)
> > [ 3408.735155] __rcu_read_unlock (kernel/rcu/update.c:223)
> > [ 3408.736090] __lock_timer (include/linux/rcupdate.h:495 include/linux/rcupdate.h:930 kernel/time/posix-timers.c:709)
> 
> I'm thinking this is one of those magic preemptible RCU bits..

Hmmm...  Looking back at Sasha's original email, RCU doesn't have much
choice about making ->wait_lock HARDIRQ-irq-unsafe, since it acquires
it via a call to rt_mutex_lock(), which cannot be invoked with irqs
disabled.  In fact, it seems a bit odd to acquire something named
->wait_lock with irqs disabled.

That said...

> ---
>  kernel/time/posix-timers.c | 12 ++++++++++--
>  1 file changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/time/posix-timers.c b/kernel/time/posix-timers.c
> index 31d11ac9fa47..09e28733e725 100644
> --- a/kernel/time/posix-timers.c
> +++ b/kernel/time/posix-timers.c
> @@ -701,17 +701,25 @@ static struct k_itimer *__lock_timer(timer_t timer_id, unsigned long *flags)
>  	if ((unsigned long long)timer_id > INT_MAX)
>  		return NULL;
> 
> +	/*
> +	 * One of the few rules of preemptible RCU is that one cannot do
> +	 * rcu_read_unlock() while holding a scheduler (or nested) lock when
> +	 * part of the read side critical section was irqs-enabled -- see
> +	 * rcu_read_unlock_special().
> +	 */
> +	local_irq_safe(*flags);
>  	rcu_read_lock();
>  	timr = posix_timer_by_id(timer_id);
>  	if (timr) {
> -		spin_lock_irqsave(&timr->it_lock, *flags);
> +		spin_lock(&timr->it_lock);
>  		if (timr->it_signal == current->signal) {
>  			rcu_read_unlock();

If ->it_lock is ever acquired while one of the rq or pi locks was held,
Peter's patch is needed.

It is just that I am not seeing what I would expect to see in Sasha's
lockdep splat if that were the case.

							Thanx, Paul

>  			return timr;
>  		}
> -		spin_unlock_irqrestore(&timr->it_lock, *flags);
> +		spin_unlock(&timr->it_lock);
>  	}
>  	rcu_read_unlock();
> +	local_irq_restore(*flags);
> 
>  	return NULL;
>  }
>