Re: [PATCH 1/2] blk-mq: share one tags_srcu instance across all tag sets

[Ming Lei]

On Thu, Oct 01, 2026 at 05:17:50PM +0800, Qiliang Yuan wrote:
> Freed request pages and flush queues are handed to call_srcu() so that
> concurrent tag iterators never see them disappear. Each tag set embeds
> its own srcu_struct for this, which blk_mq_free_tag_set() has to drain
> with srcu_barrier() before cleanup_srcu_struct(). The barrier waits for
> a full SRCU grace period even when the callbacks are long done, adding
> 1-3 ms to every tag set teardown. Drivers that free tag sets under a
> lock make it worse: ublk does it under the global ublk_ctl_mutex, so
> deleting devices from many threads serializes on that grace period.
> 
> The callbacks free memory that only blk-mq owns. They touch neither
> the tag set nor driver code, so they don't need a per-set domain.
> Replace the per-set srcu_struct with a single global one. Freeing a tag
> set no longer waits for a grace period, and each tag set no longer
> allocates per-CPU SRCU state. The read side is unchanged. The only cost
> is that a long-running iterator can now delay these frees for other tag
> sets too, which only postpones freeing memory.
> 
> On 7.3-rc5 in a 16-vCPU VM, with 1 queue and depth 16 per ublk device,
> median of 3 runs (5 for the 16-thread null_blk case):
> 
>                                        before     after
>   ublk DEL_DEV p50, serial             3.45 ms    0.17 ms
>   ublk delete 1000 devices, serial     19.5 s     14.9 s
>   ublk delete 4000 devices, 16 thr     19.0 s     2.9 s
>   null_blk destroy 500 devs, serial    9.97 s     9.25 s
>   null_blk destroy 256 devs, 16 thr    5.15 s     4.30 s
> 
> Fixes: ad0d05dbddc1 ("blk-mq: Defer freeing of tags page_list to SRCU callback")
> Signed-off-by: Qiliang Yuan <odys.yuan@xxxxxxxxx>

Reviewed-by: Ming Lei <tom.leiming@xxxxxxxxx>


Thanks,
Ming