Re: [PATCH -tip] kprobes: Lock rcu_read_lock() while searching kprobe

[Joel Fernandes]

On Tue, Dec 03, 2019 at 11:09:59PM -0500, Joel Fernandes wrote:
> On Tue, Dec 03, 2019 at 08:13:29AM +0100, Ingo Molnar wrote:
> > 
> > * Joel Fernandes <joel@xxxxxxxxxxxxxxxxx> wrote:
> > 
> > > On Mon, Dec 02, 2019 at 04:32:13PM +0900, Masami Hiramatsu wrote:
> > > > Anders reported that the lockdep warns that suspicious
> > > > RCU list usage in register_kprobe() (detected by
> > > > CONFIG_PROVE_RCU_LIST.) This is because get_kprobe()
> > > > access kprobe_table[] by hlist_for_each_entry_rcu()
> > > > without rcu_read_lock.
> > > > 
> > > > If we call get_kprobe() from the breakpoint handler context,
> > > > it is run with preempt disabled, so this is not a problem.
> > > > But in other cases, instead of rcu_read_lock(), we locks
> > > > kprobe_mutex so that the kprobe_table[] is not updated.
> > > > So, current code is safe, but still not good from the view
> > > > point of RCU.
> > > > 
> > > > Let's lock the rcu_read_lock() around get_kprobe() and
> > > > ensure kprobe_mutex is locked at those points.
> > > > 
> > > > Note that we can safely unlock rcu_read_lock() soon after
> > > > accessing the list, because we are sure the found kprobe has
> > > > never gone before unlocking kprobe_mutex. Unless locking
> > > > kprobe_mutex, caller must hold rcu_read_lock() until it
> > > > finished operations on that kprobe.
> > > > 
> > > > Reported-by: Anders Roxell <anders.roxell@xxxxxxxxxx>
> > > > Signed-off-by: Masami Hiramatsu <mhiramat@xxxxxxxxxx>
> > > 
> > > Instead of this, can you not just pass the lockdep_is_held() expression as
> > > the last argument to list_for_each_entry_rcu() to silence the warning? Then
> > > it will be a simpler patch.
> > 
> > Come on, we do not silence warnings!
> 
> By silence, I mean remove a false-positive warning. In this case since lock
> is held, it is not a valid warning.
> 
> > If it's safely inside the lock then why not change it from 
> > hlist_for_each_entry_rcu() to hlist_for_each_entry()?
> > 
> > I do think that 'lockdep flag' inside hlist_for_each_entry_rcu():
> > 
> > /**
> >  * hlist_for_each_entry_rcu - iterate over rcu list of given type
> >  * @pos:        the type * to use as a loop cursor.
> >  * @head:       the head for your list.
> >  * @member:     the name of the hlist_node within the struct.
> >  * @cond:       optional lockdep expression if called from non-RCU protection.
> >  *
> >  * This list-traversal primitive may safely run concurrently with
> >  * the _rcu list-mutation primitives such as hlist_add_head_rcu()
> >  * as long as the traversal is guarded by rcu_read_lock().
> >  */
> > #define hlist_for_each_entry_rcu(pos, head, member, cond...)            \
> > 
> > is actively harmful. Why is it there?
> 
> Because as Paul also said, the code can be common between regular lock
> holders and RCU lock holders. I am not sure if this is the case with the
> kprobe code though.

Here are some more details on the kprobe side of things:

get_kprobe() can be called wither from preempt disabled section, or under
kprobe_mutex lock as evident from also the code comments on this function [1]

If called from a preempt disable section, then it is in an RCU reader section
and no warning will be emitted by use of hlist_for_each_entry_rcu(). This is
because hlist_for_each_entry_rcu() will internally check if preempt is
disabled.  However, if it is called under kprobe_mutex lock, then we have no
way of knowing in hlist_for_each_entry_rcu() if lock was held. So we must
pass the lockdep expression (which tests if lock is held) to the macro so
that the false-positive warning is silenced.

thanks,

 - Joel

[1]
/*
 * This routine is called either:
 *      - under the kprobe_mutex - during kprobe_[un]register()
 *                              OR
 *      - with preemption disabled - from arch/xxx/kernel/kprobes.c
 */
struct kprobe *get_kprobe(void *addr)