[PATCH] md: allow growing logical_block_size on a running array

[Jack Wang]

From: Jack Wang <jinpu.wang@xxxxxxxxxxxxxxx>

Growing logical_block_size on a running array required stopping it
first. Handle it directly in lbs_store(), similar to
mddev_set_closing_and_sync_blockdev(): under open_mutex, refuse with
-EBUSY if the array is closing or has openers, otherwise apply the
bigger LBS.

mddev_lock() is not taken on this path, to avoid holding reconfig_mutex
while blocking on the queue limits lock.

md_release() drops openers without open_mutex, so a bio of a just-closed
opener may not yet be accounted in active_io. Commit the limits with
queue_limits_commit_update_frozen() to drain it. Also take open_mutex
around the existing "enable LBS" branch so it cannot race with the grow
path on mddev->logical_block_size.

Suggested-by: Florian-Ewald Mueller <florian-ewald.mueller@xxxxxxxxx>
Signed-off-by: Jack Wang <jinpu.wang@xxxxxxxxxxxxxxx>
---
 drivers/md/md.c | 40 +++++++++++++++++++++++++++++++++++-----
 1 file changed, 35 insertions(+), 5 deletions(-)

diff --git a/drivers/md/md.c b/drivers/md/md.c
index 03e35386a802..cd9c73f46786 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -6013,12 +6013,12 @@ static int mddev_set_logical_block_size(struct mddev *mddev,
 	lim.logical_block_size = lbs;
 	pr_info("%s: logical_block_size is changed, data may be lost\n",
 		mdname(mddev));
-	err = queue_limits_commit_update(mddev->gendisk->queue, &lim);
+	err = queue_limits_commit_update_frozen(mddev->gendisk->queue, &lim);
 	if (err)
 		return err;
 
 	mddev->logical_block_size = lbs;
-	/* New lbs will be written to superblock after array is running */
+	/* New lbs will be written to superblock on the next update */
 	set_bit(MD_SB_CHANGE_DEVS, &mddev->sb_flags);
 	return 0;
 }
@@ -6046,8 +6046,36 @@ lbs_store(struct mddev *mddev, const char *buf, size_t len)
 	if (mddev->pers) {
 		unsigned int curr_lbs;
 
-		if (mddev->logical_block_size)
+		/*
+		 * open_mutex serialises both paths below on
+		 * mddev->logical_block_size and keeps new openers out.
+		 * MD_CLOSING means a stop is under way. mddev_lock() is not
+		 * taken, to avoid holding reconfig_mutex while blocking on
+		 * the queue limits lock.
+		 */
+		mutex_lock(&mddev->open_mutex);
+		curr_lbs = queue_logical_block_size(mddev->gendisk->queue);
+		if (lbs > curr_lbs) {
+			/*
+			 * A bigger LBS can be applied to a running array if
+			 * nobody has it open, as there is no I/O through the
+			 * MD gendisk queue without openers. The queue is
+			 * still frozen for the limits update to drain a bio
+			 * of an opener that has just closed.
+			 */
+			if (test_bit(MD_CLOSING, &mddev->flags) ||
+			    atomic_read(&mddev->openers) > 0)
+				err = -EBUSY;
+			else
+				err = mddev_set_logical_block_size(mddev, lbs);
+			mutex_unlock(&mddev->open_mutex);
+			return err ?: len;
+		}
+
+		if (mddev->logical_block_size) {
+			mutex_unlock(&mddev->open_mutex);
 			return -EBUSY;
+		}
 		/*
 		 * To fix forward compatibility issues, LBS is not
 		 * configured for arrays from old kernels (<=6.18) by default.
@@ -6055,12 +6083,14 @@ lbs_store(struct mddev *mddev, const char *buf, size_t len)
 		 * enable LBS by writing current LBS — to prevent data
 		 * loss from LBS changes.
 		 */
-		curr_lbs = queue_logical_block_size(mddev->gendisk->queue);
-		if (lbs != curr_lbs)
+		if (lbs != curr_lbs) {
+			mutex_unlock(&mddev->open_mutex);
 			return -EINVAL;
+		}
 
 		mddev->logical_block_size = curr_lbs;
 		set_bit(MD_SB_CHANGE_DEVS, &mddev->sb_flags);
+		mutex_unlock(&mddev->open_mutex);
 		pr_info("%s: logical block size configured successfully, array will not be assembled in old kernels (<= 6.18)\n",
 			mdname(mddev));
 		return len;
-- 
2.43.0