[PATCH 5.10 022/103] scsi: core: Fix hang of freezing queue between blocking and running device

From: Greg Kroah-Hartman
Date: Wed Sep 01 2021 - 08:37:19 EST


From: Li Jinlin <lijinlin3@xxxxxxxxxx>

commit 02c6dcd543f8f051973ee18bfbc4dc3bd595c558 upstream.

We found a hang, the steps to reproduce are as follows:

1. blocking device via scsi_device_set_state()

2. dd if=/dev/sda of=/mnt/t.log bs=1M count=10

3. echo none > /sys/block/sda/queue/scheduler

4. echo "running" >/sys/block/sda/device/state

Step 3 and 4 should complete after step 4, but they hang.

CPU#0 CPU#1 CPU#2
--------------- ---------------- ----------------
Step 1: blocking device

Step 2: dd xxxx
^^^^^^ get request
q_usage_counter++

Step 3: switching scheculer
elv_iosched_store
elevator_switch
blk_mq_freeze_queue
blk_freeze_queue
> blk_freeze_queue_start
^^^^^^ mq_freeze_depth++

> blk_mq_run_hw_queues
^^^^^^ can't run queue when dev blocked

> blk_mq_freeze_queue_wait
^^^^^^ Hang here!!!
wait q_usage_counter==0

Step 4: running device
store_state_field
scsi_rescan_device
scsi_attach_vpd
scsi_vpd_inquiry
__scsi_execute
blk_get_request
blk_mq_alloc_request
blk_queue_enter
^^^^^^ Hang here!!!
wait mq_freeze_depth==0

blk_mq_run_hw_queues
^^^^^^ dispatch IO, q_usage_counter will reduce to zero

blk_mq_unfreeze_queue
^^^^^ mq_freeze_depth--

To fix this, we need to run queue before rescanning device when the device
state changes to SDEV_RUNNING.

Link: https://lore.kernel.org/r/20210824025921.3277629-1-lijinlin3@xxxxxxxxxx
Fixes: f0f82e2476f6 ("scsi: core: Fix capacity set to zero after offlinining device")
Reviewed-by: Bart Van Assche <bvanassche@xxxxxxx>
Signed-off-by: Li Jinlin <lijinlin3@xxxxxxxxxx>
Signed-off-by: Qiu Laibin <qiulaibin@xxxxxxxxxx>
Signed-off-by: Martin K. Petersen <martin.petersen@xxxxxxxxxx>
Signed-off-by: Greg Kroah-Hartman <gregkh@xxxxxxxxxxxxxxxxxxx>
---
drivers/scsi/scsi_sysfs.c | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)

--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -808,12 +808,15 @@ store_state_field(struct device *dev, st
ret = scsi_device_set_state(sdev, state);
/*
* If the device state changes to SDEV_RUNNING, we need to
- * rescan the device to revalidate it, and run the queue to
- * avoid I/O hang.
+ * run the queue to avoid I/O hang, and rescan the device
+ * to revalidate it. Running the queue first is necessary
+ * because another thread may be waiting inside
+ * blk_mq_freeze_queue_wait() and because that call may be
+ * waiting for pending I/O to finish.
*/
if (ret == 0 && state == SDEV_RUNNING) {
- scsi_rescan_device(dev);
blk_mq_run_hw_queues(sdev->request_queue, true);
+ scsi_rescan_device(dev);
}
mutex_unlock(&sdev->state_mutex);