[PATCH] blk-mq: dispatch kworker to housekeeping CPU when hctx cpumask is empty

[Aaron Tomlin]

Commit a46c27026da1 ("blk-mq: don't schedule block kworker on isolated
CPUs") excluded isolated CPUs from hctx->cpumask to prevent worker
threads from executing block work on isolated CPUs. When all CPUs mapped
to a hardware queue are isolated (i.e. blk_mq_hctx_empty_cpumask(hctx)
evaluates to true), blk_mq_hctx_next_cpu() returns WORK_CPU_UNBOUND with
the intention of offloading execution.

However, kblockd_workqueue is allocated as a per-CPU workqueue
(WQ_PERCPU). When WORK_CPU_UNBOUND is queued on a per-CPU workqueue, the
workqueue core resolves the target CPU via raw_smp_processor_id().
Consequently, if a task executing on an isolated CPU submits I/O and
triggers an asynchronous queue dispatch with zero delay, &hctx->run_work
is queued directly onto the local isolated CPU's worker pool. This
causes the local per-CPU high-priority kworker
(e.g. "kworker/6:1H-kblockd") to be scheduled on the isolated CPU,
introducing unwanted latency jitter and execution delays, thereby
defeating the original objective of commit a46c27026da1.

Rectify this by selecting an online housekeeping CPU via
housekeeping_any_cpu(HK_TYPE_TIMER) when blk_mq_hctx_empty_cpumask(hctx)
is true. This ensures that:
    1.  The delayed work is cleanly offloaded to a housekeeping CPU.

    2.  The per-CPU high-priority kworker on the isolated CPU is not
        scheduled.

    3.  Hardware queues with valid mapped CPUs preserve their per-CPU
        locality and batch round-robin behaviour.

Fixes: a46c27026da1 ("blk-mq: don't schedule block kworker on isolated CPUs")
Cc: Ming Lei <ming.lei@xxxxxxxxxx>
Cc: Jens Axboe <axboe@xxxxxxxxx>
Cc: stable@xxxxxxxxxxxxxxx
Signed-off-by: Aaron Tomlin <atomlin@xxxxxxxxxxx>
---
 block/blk-mq.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index 025a799f35b9..9480c3aac898 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -2252,8 +2252,11 @@ static int blk_mq_hctx_next_cpu(struct blk_mq_hw_ctx *hctx)
 	bool tried = false;
 	int next_cpu = hctx->next_cpu;
 
-	/* Switch to unbound if no allowable CPUs in this hctx */
-	if (hctx->queue->nr_hw_queues == 1 || blk_mq_hctx_empty_cpumask(hctx))
+	/* Switch to housekeeping CPU if no allowable CPUs in this hctx */
+	if (blk_mq_hctx_empty_cpumask(hctx))
+		return housekeeping_any_cpu(HK_TYPE_TIMER);
+
+	if (hctx->queue->nr_hw_queues == 1)
 		return WORK_CPU_UNBOUND;
 
 	if (--hctx->next_cpu_batch <= 0) {
-- 
2.55.0