[PATCH] block: avoid long sync_blockdev() under disk->open_mutex during parallel close

[Tetsuo Handa]

Commit b849dd84b6cc ("bdev: Reduce time holding bd_mutex in sync in
blkdev_close()") introduced a sync_blockdev() call immediately before
holding disk->open_mutex. But this check was racy and best-effort basis.

If more than one process concurrently called bdev_release() and saw
atomic_read(&bdev->bd_openers) > 1, the process which actually hit
the atomic_dec_and_test(&bdev->bd_openers) condition will call
sync_blockdev() from blkdev_flush_mapping() (which could take several
minutes) with disk->open_mutex held.

Kill this race window and make sync_blockdev() always be called before
holding disk->open_mutex, by using a marker that checks whether lockless
sync_blockdev() has raced.

Signed-off-by: Tetsuo Handa <penguin-kernel@xxxxxxxxxxxxxxxxxxx>
---
 block/bdev.c              | 41 +++++++++++++++++++++++++++++++++++++--
 include/linux/blk_types.h |  1 +
 2 files changed, 40 insertions(+), 2 deletions(-)

diff --git a/block/bdev.c b/block/bdev.c
index cd8323083740..07e4cc0e5c2a 100644
--- a/block/bdev.c
+++ b/block/bdev.c
@@ -1167,10 +1167,47 @@ void bdev_release(struct file *bdev_file)
 	 * of the world and we want to avoid long (could be several minute)
 	 * syncs while holding the mutex.
 	 */
-	if (atomic_read(&bdev->bd_openers) == 1)
+ retry:
+	mutex_lock(&disk->open_mutex);
+	if (bdev_is_partition(bdev) &&
+	    atomic_read(&bdev_whole(bdev)->bd_openers) == 1) {
+		/*
+		 * This "bdev" is a partition, and blkdev_put_whole() from
+		 * blkdev_put_part() will trigger blkdev_flush_mapping() on
+		 * the whole disk. Therefore, we need to sync the whole disk.
+		 */
+		bdev->bdev_syncer = current;
+		mutex_unlock(&disk->open_mutex);
+		sync_blockdev(bdev_whole(bdev));
+		mutex_lock(&disk->open_mutex);
+		/* Need to retry if we raced. */
+		if (bdev->bdev_syncer != current)
+			goto retry;
+	} else if (atomic_read(&bdev->bd_openers) == 1) {
+		/*
+		 * If "bdev" is a whole disk, blkdev_put_whole() will trigger
+		 * blkdev_flush_mapping() on this disk. Therefore, sync this
+		 * "bdev".
+		 *
+		 * If "bdev" is a partition, blkdev_put_part() will trigger
+		 * blkdev_flush_mapping() on this partition, but
+		 * blkdev_put_whole() from blkdev_put_part() won't trigger
+		 * blkdev_flush_mapping() on the whole disk because we chedked
+		 * bdev_whole(bdev)->bd_openers != 1. Therefore, we need to
+		 * sync only this "bdev".
+		 */
+		bdev->bdev_syncer = current;
+		mutex_unlock(&disk->open_mutex);
 		sync_blockdev(bdev);
+		mutex_lock(&disk->open_mutex);
+		/* Need to retry if we raced. */
+		if (bdev->bdev_syncer != current)
+			goto retry;
+	} else {
+		/* Mark sync_blockdev() being possibly raced. */
+		bdev->bdev_syncer = NULL;
+	}
 
-	mutex_lock(&disk->open_mutex);
 	bdev_yield_write_access(bdev_file);
 
 	if (holder)
diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h
index 98e21b4cbf32..d79bb4b6100f 100644
--- a/include/linux/blk_types.h
+++ b/include/linux/blk_types.h
@@ -74,6 +74,7 @@ struct block_device {
 #ifdef CONFIG_SECURITY
 	void			*bd_security;
 #endif
+	struct task_struct	*bdev_syncer; /* Not refcounted. */
 	/*
 	 * keep this out-of-line as it's both big and not needed in the fast
 	 * path
-- 
2.56.0