Re: [RFC PATCH v2 2/2] blk-mq: Lockout tagset iter when freeing rqs

[John Garry]

Resend without ppvk@xxxxxxxxxxxxxx, which bounces for me

On 22/12/2020 02:13, Bart Van Assche wrote:
> On 12/21/20 10:47 AM, John Garry wrote:
>> Yes, I agree, and I'm not sure what I wrote to give that impression.
>>
>> About "root partition", above, I'm just saying that / is mounted on a
>> sda partition:
>>
>> root@ubuntu:/home/john# mount | grep sda
>> /dev/sda2 on / type ext4 (rw,relatime,errors=remount-ro,stripe=32)
>> /dev/sda1 on /boot/efi type vfat
>> 
(rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)
> Hi John,
>

Hi Bart, Ming,

> Thanks for the clarification. I want to take back my suggestion about
> adding rcu_read_lock() / rcu_read_unlock() in blk_mq_tagset_busy_iter()
> since it is not allowed to sleep inside an RCU read-side critical
> section, since blk_mq_tagset_busy_iter() is used in request timeout
> handling and since there may be blk_mq_ops.timeout implementations that
> sleep.

Yes, that's why I was going with atomic, rather than some 
synchronization primitive which may sleep.

>
> Ming's suggestion to serialize blk_mq_tagset_busy_iter() and
> blk_mq_free_rqs() looks interesting to me.
>

So then we could have something like this:

---8<---

 -435,9 +444,13 @@ void blk_mq_queue_tag_busy_iter(struct request_queue 
*q, busy_iter_fn *fn,
    if (!blk_mq_hw_queue_mapped(hctx))
            continue;

+    while (!atomic_inc_not_zero(&tags->iter_usage_counter));
+
    if (tags->nr_reserved_tags)
        bt_for_each(hctx, tags->breserved_tags, fn, priv, true);
    bt_for_each(hctx, tags->bitmap_tags, fn, priv, false);

+    atomic_dec(&tags->iter_usage_counter);
}

blk_queue_exit(q);

--->8---

And similar for blk_mq_tagset_busy_iter(). How about it?

Thanks,
John