Re: [PATCH v2] blk-cgroup: remove entries in blkg_tree before queue release

[Bart Van Assche]

On Wed, 2018-04-11 at 10:00 -0700, tj@xxxxxxxxxx wrote:
> On Wed, Apr 11, 2018 at 04:42:55PM +0000, Bart Van Assche wrote:
> > On Wed, 2018-04-11 at 07:56 -0700, Tejun Heo wrote:
> > > And looking at the change, it looks like the right thing we should
> > > have done is caching @lock on the print_blkg side and when switching
> > > locks make sure both locks are held.  IOW, do the following in
> > > blk_cleanup_queue()
> > > 
> > > 	spin_lock_irq(lock);
> > > 	if (q->queue_lock != &q->__queue_lock) {
> > > 		spin_lock(&q->__queue_lock);
> > > 		q->queue_lock = &q->__queue_lock;
> > > 		spin_unlock(&q->__queue_lock);
> > > 	}
> > > 	spin_unlock_irq(lock);
> > > 
> > > Otherwise, there can be two lock holders thinking they have exclusive
> > > access to the request_queue.
> > 
> > I think that's a bad idea. A block driver is allowed to destroy the
> > spinlock it associated with the request queue as soon as blk_cleanup_queue()
> > has finished. If the block cgroup controller would cache a pointer to the
> > block driver spinlock then that could cause the cgroup code to attempt to
> > lock a spinlock after it has been destroyed. I don't think we need that kind
> > of race conditions.
> 
> I see, but that problem is there with or without caching as long as we
> have queu_lock usage which reach beyond cleanup_queue, right?  Whether
> that user caches the lock for matching unlocking or not doesn't really
> change the situation.
> 
> Short of adding protection around queue_lock switching, I can't think
> of a solution tho.  Probably the right thing to do is adding queue
> lock/unlock helpers which are safe to use beyond cleanup_queue.

Hello Tejun,

A simple and effective solution is to dissociate a request queue from the
block cgroup controller before blk_cleanup_queue() returns. This is why commit
a063057d7c73 ("block: Fix a race between request queue removal and the block
cgroup controller") moved the blkcg_exit_queue() call from __blk_release_queue()
into blk_cleanup_queue().

Thanks,

Bart.