Re: [rcu:dev.2019.09.23a 62/77] kernel/rcu/tree.c:2882 kfree_call_rcu() warn: inconsistent returns 'irqsave:flags'.

[Paul E. McKenney]

On Thu, Sep 26, 2019 at 02:19:54PM -0400, Joel Fernandes wrote:
> On Tue, Sep 24, 2019 at 09:03:48AM -0700, Paul E. McKenney wrote:
> > On Tue, Sep 24, 2019 at 08:15:12AM -0400, Joel Fernandes wrote:
> > > On Tue, Sep 24, 2019 at 4:05 AM Dan Carpenter <dan.carpenter@xxxxxxxxxx> wrote:
> > > >
> > > > tree:   https://kernel.googlesource.com/pub/scm/linux/kernel/git/paulmck/linux-rcu.git dev.2019.09.23a
> > > > head:   97de53b94582c208ee239178b208b8e8b9472585
> > > > commit: 39676bb72323718a9e7bab1ba9c4a68319a96f62 [62/77] rcu: Add support for debug_objects debugging for kfree_rcu()
> > > >
> > > > If you fix the issue, kindly add following tag
> > > > Reported-by: kbuild test robot <lkp@xxxxxxxxx>
> > > > Reported-by: Dan Carpenter <dan.carpenter@xxxxxxxxxx>
> > > >
> > > > smatch warnings:
> > > > kernel/rcu/tree.c:2882 kfree_call_rcu() warn: inconsistent returns 'irqsave:flags'.
> > > >   Locked on:   line 2867
> > > >   Unlocked on: line 2882
> > > >
> > > > git remote add rcu https://kernel.googlesource.com/pub/scm/linux/kernel/git/paulmck/linux-rcu.git
> > > > git remote update rcu
> > > > git checkout 39676bb72323718a9e7bab1ba9c4a68319a96f62
> > > > vim +2882 kernel/rcu/tree.c
> > > >
> > > > 5f5046cc15d514 Joel Fernandes (Google  2019-08-05  2850) void kfree_call_rcu(struct rcu_head *head, rcu_callback_t func)
> > > > 5f5046cc15d514 Joel Fernandes (Google  2019-08-05  2851) {
> > > > 5f5046cc15d514 Joel Fernandes (Google  2019-08-05  2852)        unsigned long flags;
> > > > 5f5046cc15d514 Joel Fernandes (Google  2019-08-05  2853)        struct kfree_rcu_cpu *krcp;
> > > > 5f5046cc15d514 Joel Fernandes (Google  2019-08-05  2854)
> > > > 5f5046cc15d514 Joel Fernandes (Google  2019-08-05  2855)        head->func = func;
> > > > 5f5046cc15d514 Joel Fernandes (Google  2019-08-05  2856)
> > > > 87970ccf5a9125 Paul E. McKenney        2019-09-19  2857         local_irq_save(flags);  // For safely calling this_cpu_ptr().
> > > >                                                                 ^^^^^^^^^^^^^^^^^^^^^
> > > > IRQs disabled.
> > > >
> > > > 5f5046cc15d514 Joel Fernandes (Google  2019-08-05  2858)        krcp = this_cpu_ptr(&krc);
> > > > ff8db005e371cb Paul E. McKenney        2019-09-19  2859         if (krcp->initialized)
> > > > 5f5046cc15d514 Joel Fernandes (Google  2019-08-05  2860)                spin_lock(&krcp->lock);
> > > > 5f5046cc15d514 Joel Fernandes (Google  2019-08-05  2861)
> > > > 87970ccf5a9125 Paul E. McKenney        2019-09-19  2862         // Queue the object but don't yet schedule the batch.
> > > > 39676bb7232371 Joel Fernandes (Google  2019-09-22  2863)        if (debug_rcu_head_queue(head)) {
> > > > 39676bb7232371 Joel Fernandes (Google  2019-09-22  2864)                // Probable double kfree_rcu(), just leak.
> > > > 39676bb7232371 Joel Fernandes (Google  2019-09-22  2865)                WARN_ONCE(1, "%s(): Double-freed call. rcu_head %p\n",
> > > > 39676bb7232371 Joel Fernandes (Google  2019-09-22  2866)                          __func__, head);
> > > > 39676bb7232371 Joel Fernandes (Google  2019-09-22  2867)                return;
> > > >
> > > > Need to re-enable before returning.
> > > 
> > > Right. At this point the system has a bug anyway since it is a
> > > double-free. But yes, no reason to introduce more bugs. Will fix.
> > > Thank you, Dan!
> > 
> > What Joel said!
> > 
> > As long as I am doing several other reports anyway...
> > 
> > How does the to-be-squashed patch below look?
> 
> Hi Paul,
> 
> Sorry for the late reply due to the conference..
> 
> This patch looks Ok, the code in your tree though is slightly different mine
> I think because you reworked the kfree_rcu() code for working during early
> boot.

Yes, I did pull some of the changes earlier.

> After my talk tomorrow, I will go through your rcu/dev tree in detail to
> review the reworks you did in detail.

Sounds good, and best of everything with the talk!

> Let me know if you have the kfree_rcu() changes in a separate topic branch,
> as well. So it will make reviewing easier.

They will be, shortly after -rc1 comes out.

							Thanx, Paul

> thanks,
> 
>  - Joel
> 
> 
> > 
> > 							Thanx, Paul
> > 
> > ------------------------------------------------------------------------
> > 
> > commit 39af4c08038a8a6a7b1c9804fdd2c1921d583222
> > Author: Paul E. McKenney <paulmck@xxxxxxxxxx>
> > Date:   Tue Sep 24 09:02:00 2019 -0700
> > 
> >     squash! rcu: Add support for debug_objects debugging for kfree_rcu()
> >     
> >     [ paulmck: Fix IRQ per kbuild test robot <lkp@xxxxxxxxx> feedback. ]
> >     Signed-off-by: Paul E. McKenney <paulmck@xxxxxxxxxx>
> > 
> > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> > index 921daa7..edb7539 100644
> > --- a/kernel/rcu/tree.c
> > +++ b/kernel/rcu/tree.c
> > @@ -2870,7 +2870,7 @@ void kfree_call_rcu(struct rcu_head *head, rcu_callback_t func)
> >  		// Probable double kfree_rcu(), just leak.
> >  		WARN_ONCE(1, "%s(): Double-freed call. rcu_head %p\n",
> >  			  __func__, head);
> > -		return;
> > +		goto unlock_return;
> >  	}
> >  	head->func = func;
> >  	head->next = krcp->head;
> > @@ -2883,6 +2883,7 @@ void kfree_call_rcu(struct rcu_head *head, rcu_callback_t func)
> >  		schedule_delayed_work(&krcp->monitor_work, KFREE_DRAIN_JIFFIES);
> >  	}
> >  
> > +unlock_return:
> >  	if (krcp->initialized)
> >  		spin_unlock(&krcp->lock);
> >  	local_irq_restore(flags);