Re: BUG: using __this_cpu_read() in preemptible code in trace_hardirqs_on

[Steven Rostedt]

On Wed, 21 Oct 2020 15:17:33 +0200
Peter Zijlstra <peterz@xxxxxxxxxxxxx> wrote:

> > And I'm also guessing that we can call this with interrupts enabled (based
> > on the comment).
> > 
> > And we have this:
> > 
> >    local_irq_enable()
> >       trace_hardirqs_on()
> >          lockdep_hardirqs_on()
> >              __this_cpu_read()  
> 
> Moo, two threads..
> 
> 20201019183355.GS2611@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

But this one's much older ;-)

> 
> ---
> 
> On Tue, Oct 20, 2020 at 12:55:46AM +0800, kernel test robot wrote:
> > [   92.898145] BUG: using __this_cpu_read() in preemptible [00000000] code: trinity-c6/526  
> 
> > [   92.903305] Call Trace:
> > [   92.905182]  __this_cpu_preempt_check+0xf/0x11
> > [   92.905968]  lockdep_hardirqs_on_prepare+0x2c/0x18f
> > [   92.906853]  trace_hardirqs_on+0x49/0x53
> > [   92.907578]  __bad_area_nosemaphore+0x3a/0x134  
> 
> Hurph, that's a spurious local_irq_enable(). I suppose this'll fix it.
> 
> ---
> diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c
> index 3e99dfef8408..9f818145ef7d 100644
> --- a/kernel/locking/lockdep.c
> +++ b/kernel/locking/lockdep.c
> @@ -4057,9 +4057,6 @@ void lockdep_hardirqs_on_prepare(unsigned long ip)
>  	if (unlikely(in_nmi()))
>  		return;
>  
> -	if (unlikely(__this_cpu_read(lockdep_recursion)))
> -		return;
> -
>  	if (unlikely(lockdep_hardirqs_enabled())) {

Hmm, would moving the recursion check below the check of the
lockdep_hardirqs_enable() cause a large skew in the spurious enable stats?
May not be an issue, but something we should check to make sure that
there's not a path that constantly hits this.

-- Steve

>  		/*
>  		 * Neither irq nor preemption are disabled here
> @@ -4070,6 +4067,9 @@ void lockdep_hardirqs_on_prepare(unsigned long ip)
>  		return;
>  	}
>  
> +	if (unlikely(__this_cpu_read(lockdep_recursion)))
> +		return;
> +