[PATCH v8 0/1] block/blk-mq: use atomic_t for quiesce_depth to avoid lock contention on RT

[Ionut Nechita (Wind River)]

From: Ionut Nechita <ionut.nechita@xxxxxxxxxxxxx>

Hi Jens,

This is v8 of the fix for the PREEMPT_RT performance regression caused by
commit 6bda857bcbb86 ("block: fix ordering between checking
QUEUE_FLAG_QUIESCED request adding").

Changes since v7 (May 12):
- No code or commit-message-body changes; this is a tags-only respin.
  Rebased on linux-next (next-20260708); the patch applies cleanly with
  no context changes.
- Picked up Reviewed-by: Bart Van Assche <bvanassche@xxxxxxx> on 1/1.
- Added an "Assisted-by:" trailer per the new AI contribution guidance in
  Documentation/process/coding-assistants.rst, to record that Claude
  assisted with this patch and that checkpatch was the analysis tool run.
  As that document requires, the Signed-off-by / DCO certification is
  and remains mine as the human submitter; no Signed-off-by was added by
  the AI.

Changes since v6 (May 6):
- Reader-side barrier in blk_mq_run_hw_queue() changed from smp_rmb() to
  smp_mb().  The race closed by commit 6bda857bcbb86 is a store-buffer
  pattern: one CPU inserts a request and then reads the quiesce state,
  another CPU unquiesces and then reads "has pending work".  A full
  barrier is needed on *both* sides, not just a read barrier on the
  reader, so smp_mb() now pairs with the existing writer-side
  smp_mb__after_atomic().  Thanks to Bart Van Assche for pointing out
  that smp_rmb() was insufficient.
- Rewrote the in-code comments and the commit message to spell out which
  ordering the removed q->queue_lock acquisitions provided and how it is
  preserved.

The problem: on PREEMPT_RT, the spinlock_t q->queue_lock that commit
6bda857bcbb86 added to blk_mq_run_hw_queue() converts to a sleeping
rt_mutex.  blk_mq_run_hw_queue() runs from every MSI-X IRQ thread and
hits that lock on the common "nothing pending" path, so all IRQ threads
serialise and go to D-state.  On a Broadcom/LSI MegaRAID 12GSAS/PCIe
Secure SAS39xx (megaraid_sas, 128 MSI-X vectors, 120 hw queues),
throughput drops from 640 MB/s to 153 MB/s.

The fix takes the memory-barrier alternative and folds the quiesce
indicator into quiesce_depth itself: quiesce_depth becomes atomic_t,
QUEUE_FLAG_QUIESCED goes away, and no lock is left on the dispatch hot
path.

v7: https://lore.kernel.org/linux-block/20260512062815.10815-1-ionut.nechita@xxxxxxxxxxxxx/

Ionut Nechita (1):
  block/blk-mq: use atomic_t for quiesce_depth to avoid lock contention
    on RT

 block/blk-core.c       |  1 +
 block/blk-mq-debugfs.c |  1 -
 block/blk-mq.c         | 69 ++++++++++++++++++++++++++----------------
 include/linux/blkdev.h |  9 ++++--
 4 files changed, 50 insertions(+), 30 deletions(-)


base-commit: b9810cd75b9fb56a3425d391cba3f608502bd474
--
2.54.0