Re: [GIT PULL] io_uring fixes for 5.6-rc

[Paul E. McKenney]

On Fri, Mar 13, 2020 at 01:18:30PM -0700, Linus Torvalds wrote:
> On Fri, Mar 13, 2020 at 10:50 AM Jens Axboe <axboe@xxxxxx> wrote:
> >
> > Just a single fix here, improving the RCU callback ordering from last
> > week. After a bit more perusing by Paul, he poked a hole in the
> > original.
> 
> Ouch.
> 
> If I read this patch correctly, you're now adding a rcu_barrier() onto
> the system workqueue for each io_uring context freeing op.
> 
> This makes me worry:
> 
>  - I think system_wq is unordered, so does it even guarantee that the
> rcu_barrier happens after whatever work you're expecting it to be
> after?
> 
> Or is it using a workqueue not because it wants to serialize with any
> other work, but because it needs to use rcu_barrier in a context where
> it can't sleep?
> 
> But the commit message does seem to imply that ordering is important..
> 
>  - doesn't this have the potential to flood the system_wq be full of
> flushing things that all could take a while..
> 
> I've pulled it, and it may all be correct, just chalk this message up
> to "Linus got nervous looking at it".
> 
> Added Paul and Tejun to the participants explicitly.

The idea is that rcu_barrier() waits for callbacks from all earlier
call_rcu()s to be invoked.  So as long as you know that the call_rcu()
happened earlier than the rcu_barrier(), the rcu_barrier() is guaranteed
to wait for that call_rcu()'s callback.

In this case (and Jens will correct me in the sadly likely event that
I get the story confused), we have a call_rcu() followed by scheduling
work on that same task.  The work has to start executing after it was
scheduled, so if that work does an rcu_barrier(), then that rcu_barrier()
will wait on the call_rcu()'s callback to be invoked.

Jens could invoke the rcu_barrier() just before scheduling the work,
but the synchronous delay from the rcu_barrier() is a problem.

Jens, what did I mess up in the above story?  ;-)

I defer to Jens and Tejun on the possibility of ending up with all
workqueue kthreads waiting on rcu_barrier().  If that is a problem,
there are some ways of dealing with it, though none that I can think of
that come for free.

							Thanx, Paul