Re: [RFC PATCH v2 2/2] blk-mq: Lockout tagset iter when freeing rqs

[John Garry]

On 22/12/2020 16:16, Bart Van Assche wrote:
> On 12/22/20 3:15 AM, John Garry wrote:
> > So then we could have something like this:
> >
> > ---8<---
> >
> >   -435,9 +444,13 @@ void blk_mq_queue_tag_busy_iter(struct 
> > request_queue *q, busy_iter_fn *fn,
> >      if (!blk_mq_hw_queue_mapped(hctx))
> >              continue;
> >
> > +    while (!atomic_inc_not_zero(&tags->iter_usage_counter));
> > +
> >      if (tags->nr_reserved_tags)
> >          bt_for_each(hctx, tags->breserved_tags, fn, priv, true);
> >      bt_for_each(hctx, tags->bitmap_tags, fn, priv, false);
> >
> > +    atomic_dec(&tags->iter_usage_counter);
> > }
> >
> > blk_queue_exit(q);
> >
> > --->8---
> >
> > And similar for blk_mq_tagset_busy_iter(). How about it?

Hi Bart,

> Are there any blk_mq_tagset_busy_iter() calls that happen from a context 
> where the tag set can disappear while that function is in progress?

So isn't the blk_mq_tag_set always a member of the host driver data for 
those cases, and, since blk_mq_tagset_busy_iter() is for iter'ing block 
driver tags and called from block driver or hctx_busy_show(), it would 
exist for the lifetime of the host device.

> Some blk_mq_tagset_busy_iter() calls happen from a context where it is 
> not allowed to sleep but also where it is guaranteed that the tag set 
> won't disappear, e.g. the call from inside sdk_mq_queue_rq().

You're talking about skd_mq_queue_rq() -> skd_in_flight() -> 
blk_mq_tagset_busy_iter(), right?

So I would expect any .queue_rq calls to complete before the associated 
request queue and tagset may be unregistered.

> How about using a mutex inside blk_mq_queue_tag_busy_iter() instead? As 
> far as I can see all blk_mq_queue_tag_busy_iter() happen from a context 
> where it is allowed to sleep.

Well then it seems sensible to add might_sleep() also.

And we still have the blk_mq_queue_tag_busy_iter() problem. As Ming 
mentioned yesterday, we know contexts where from where it is called 
which may not sleep.

Thanks,
John