Re: [PATCH 2/2] IB/mad: Use IDR for agent IDs

From: jackm
Date: Tue Jun 12 2018 - 04:51:02 EST


On Fri, 8 Jun 2018 10:42:18 -0700
Matthew Wilcox <willy@xxxxxxxxxxxxx> wrote:

> + rcu_read_lock();
> + mad_agent = idr_find(&ib_mad_clients, hi_tid);
> + if (mad_agent
> && !atomic_inc_not_zero(&mad_agent->refcount))
> + mad_agent = NULL;
> + rcu_read_unlock();

Hi Matthew,

I don't see the flow which can explain using atomic_inc_not_zero() here.

The refcount will go to zero only when unregister_mad_agent() is
called (code below, see asterisks):
idr_lock(&ib_mad_clients);
*** idr_remove(&ib_mad_clients, mad_agent_priv->agent.hi_tid);
idr_unlock(&ib_mad_clients);

flush_workqueue(port_priv->wq);
ib_cancel_rmpp_recvs(mad_agent_priv);

*** deref_mad_agent(mad_agent_priv);
[JPM] The call to idr_find in the interrupt context
would need to occur here for the refcount to have a
possibility of being zero.
Shouldn't idr_find in the interrupt context fail, since
idr_remove has already been invoked?
wait_for_completion(&mad_agent_priv->comp);

The refcount will be able to go to zero only after deref_mad_agent is
called above. Before this, however, idr_remove() has been called --
so, if my understanding is correct, the idr_find call in
find_mad_agent() should not succeed since the refcount can get to zero
only AFTER the idr_remove call.

Could you please explain the flow which can result in idr_find
succeeding (in the interrupt context) after idr_remove has been invoked
(in the process context)? Will idr_find succeed even after
idr_remove, and only fail after kfree_rcu is invoked as well? (or,
maybe after some garbage-collection delay?)

Thx!

-Jack