[PATCH v9 0/1] block/blk-mq: use atomic_t for quiesce_depth to avoid lock contention on RT
From: Ionut Nechita (Wind River)
Date: Thu Nov 19 2026 - 12:41:55 EST
From: Ionut Nechita <ionut.nechita@xxxxxxxxxxxxx>
Hi Jens,
This is v9 of the fix for the PREEMPT_RT performance regression caused by
commit 6bda857bcbb86 ("block: fix ordering between checking
QUEUE_FLAG_QUIESCED request adding").
Changes since v8 (Jul 9):
- No code changes. Rebased on linux-next (next-20261118); the patch
applies cleanly and builds without new warnings (W=1) with and without
PREEMPT_RT.
- Re-checked the users of the quiesce state in next-20261118. Nothing
outside the patch references QUEUE_FLAG_QUIESCED or q->quiesce_depth
any more. blk_queue_quiesced() has eight callers, all lockless reads:
block/blk-mq-sched.c blk_mq_sched_dispatch_requests()
block/blk-mq.c blk_mq_rq_inflight(), blk_mq_hw_queue_need_run(),
blk_mq_try_issue_directly(),
blk_mq_request_issue_directly(),
__blk_mq_flush_list()
drivers/block/ublk_drv.c ublk_wait_tagset_rqs_idle() (WARN_ON_ONCE)
drivers/mmc/core/queue.c mmc_cleanup_queue()
They keep the same meaning (at least one quiesce in effect), now as an
atomic_read() of quiesce_depth instead of test_bit() on the removed
flag. All of them pass a non-const struct request_queue *, so the
switch from a macro to a static inline is fine; the affected objects
(blk-mq, blk-mq-sched, blk-mq-debugfs, blk-core, ublk_drv, mmc queue)
were build-tested.
- Kept the Assisted-by: trailer from v8 (Claude:claude-opus-4-8,
checkpatch), in the AGENT:MODEL [TOOLS] form used in mainline. Claude
also helped with this rebase: re-checking the quiesce state users above
and the wording of this cover letter. checkpatch in next-20261118
reports the trailer as a non-standard signature; that seems to be a
checkpatch gap, since the same form is already in mainline. I reviewed
and tested everything and I'm responsible for the submission.
- Kept the Reviewed-by tags from Sebastian and Bart.
The problem: on PREEMPT_RT, the spinlock_t q->queue_lock that commit
6bda857bcbb86 added to blk_mq_run_hw_queue() converts to a sleeping
rt_mutex. blk_mq_run_hw_queue() runs from every MSI-X IRQ thread and hits
that lock on the common "nothing pending" path, so all IRQ threads
serialise and go to D-state. On a Broadcom/LSI MegaRAID 12GSAS/PCIe
Secure SAS39xx (megaraid_sas, 128 MSI-X vectors, 120 hw queues),
throughput drops from 640 MB/s to 153 MB/s.
The fix takes the memory-barrier alternative and folds the quiesce
indicator into quiesce_depth itself: quiesce_depth becomes atomic_t,
QUEUE_FLAG_QUIESCED goes away, and no lock is left on the dispatch hot
path.
checkpatch reports "memory barrier without comment" for the smp_mb() in
blk_mq_run_hw_queue(). In the resulting code the block comment is directly
above it; checkpatch only misses it because the removed
spin_lock_irqsave() line sits between the comment and the barrier in the
diff.
Is there anything else needed for this to go in? Happy to rework it if
you'd prefer a different approach.
v8: https://lore.kernel.org/lkml/20260709063803.23538-1-ionut.nechita@xxxxxxxxxxxxx/
v7: https://lore.kernel.org/linux-block/20260512062815.10815-1-ionut.nechita@xxxxxxxxxxxxx/
Thanks,
Ionut
Ionut Nechita (1):
block/blk-mq: use atomic_t for quiesce_depth to avoid lock contention
on RT
block/blk-core.c | 1 +
block/blk-mq-debugfs.c | 1 -
block/blk-mq.c | 69 ++++++++++++++++++++++++++----------------
include/linux/blkdev.h | 9 ++++--
4 files changed, 50 insertions(+), 30 deletions(-)
base-commit: e8017d29067fd1310a65c3494ce1a1b492adbac6
--
2.55.0