Re: [PATCH] md: avoid block autoload while importing rdev under reconfig_mutex
From: 王继晨
Date: Fri Nov 13 2026 - 06:47:58 EST
> From: "yu kuai"<yukuai@xxxxxxx>
> Date: Wed, Nov 11, 2026, 18:38
> Subject: Re: [PATCH] md: avoid block autoload while importing rdev under reconfig_mutex
> To: "王继晨"<wangjichen@xxxxxxxxxxx>, "yu kuai"<yukuai@xxxxxxx>
> Cc: "Jens Axboe"<axboe@xxxxxxxxx>, "Song Liu"<song@xxxxxxxxxx>, "Li Nan"<magiclinan@xxxxxxxxxxxxxx>, "Xiao Ni"<xiao@xxxxxxxxxx>, <linux-block@xxxxxxxxxxxxxxx>, <linux-kernel@xxxxxxxxxxxxxxx>, <linux-raid@xxxxxxxxxxxxxxx>, <syzbot+cd254889291ef8666036@xxxxxxxxxxxxxxxxxxxxxxxxx>
> Hi,
>
> 在 2026/11/11 14:32, 王继晨 写道:
> >> From: "yu kuai"<yukuai@xxxxxxx>
> >> Date: Sun, Nov 8, 2026, 11:18
> >> Subject: Re: [PATCH] md: avoid block autoload while importing rdev under reconfig_mutex
> >> To: "Wang Jichen"<wangjichen@xxxxxxxxxxx>, "Jens Axboe"<axboe@xxxxxxxxx>, "Song Liu"<song@xxxxxxxxxx>, "Li Nan"<magiclinan@xxxxxxxxxxxxxx>, "Xiao Ni"<xiao@xxxxxxxxxx>, <linux-block@xxxxxxxxxxxxxxx>, <linux-kernel@xxxxxxxxxxxxxxx>, <linux-raid@xxxxxxxxxxxxxxx>, "yu kuai"<yukuai@xxxxxxx>
> >> Cc: <syzbot+cd254889291ef8666036@xxxxxxxxxxxxxxxxxxxxxxxxx>
> >> Hi,
> >>
> >> 在 2026/11/3 19:55, Wang Jichen 写道:
> >>> From: wangjichen <wangjichen@xxxxxxxxxxx>
> >>>
> >>> md_import_device() runs with mddev->reconfig_mutex held from md_ioctl().
> >>> Opening a member disk via bdev_file_open_by_dev() triggers legacy block
> >>> autoload (blk_probe_dev), which takes major_names_lock. md_probe() can
> >>> run under that lock and call md_alloc(), which takes disks_mutex and
> >>> reconfig_mutex — a lockdep ABBA with the ioctl path.
> >>
> >> Is this a real deadlock? mddev is already existed from md_import_device(),
> >> while md_alloc() just allocate a new mddev, so the reconfig_mtuex is not
> >> the same.
> > Hi Kuai,
> >
> > Thanks for the question.
> >
> > You're right that md_ioctl() holds reconfig_mutex on the existing array
> > mddev, while md_alloc() takes reconfig_mutex on a newly allocated mddev —
> > they are different per-device mutex instances, not the same lock.
> >
> > The syzbot/lockdep report is about a lock *class* cycle, not a
> > single-mddev self-deadlock. Lockdep treats all &mddev->reconfig_mutex as
> > one class, so the dependency it reports is:
> >
> > reconfig_mutex -> major_names_lock -> disks_mutex -> reconfig_mutex
> >
> > Concrete race (syzbot extid cd254889291ef8666036, C repro on QEMU):
> >
> > - Thread doing ADD_NEW_DISK on array A holds A's reconfig_mutex, then
> > md_import_device() opens a member via bdev_file_open_by_dev(), which
> > with legacy autoload calls blk_probe_dev() and takes major_names_lock.
> >
> > - Concurrent open of an unregistered md node takes major_names_lock in
> > blk_probe_dev(), runs md_probe() -> md_alloc() for device B, takes
> > disks_mutex, then B's reconfig_mutex.
> >
> > Without lockdep, the same ABBA ordering between threads can still hang;
> > the syzbot report includes the standard CPU0/CPU1 "Possible unsafe
> > locking scenario" text for this chain.
>
> I still don't understand how is the ABBA deadlock possible, md_alloc() hold
> reconfig_mutex before add_disk() and kobject_add() for mddev->kobj, there is no
> concurrent context that can hold reconfig_mutex before md_alloc() hold it.
>
> >
> > For context on when reconfig_mutex entered the md_alloc() path: the cycle
> > became visible after the fix for the half-initialised kobj window:
> >
> > ca39f7502425 ("md: fix mddev->kobj lifetime")
> > d1472ca2ca5d ("md: don't hand out the array before md_alloc() has added
> > mddev->kobj")
> >
> > The latter holds reconfig_mutex across add_disk() and kobject_add() so
> > ioctls wait instead of touching an uninitialised mddev->kobj (Fixes:
> > ca39f7502425). That closes the kobject race but completes this lockdep
> > ring with ioctl + legacy autoload on the other side.
>
> For this test case, you can just use mddev private lockdep class to initialize
> reconfig_mutex, or if there is still other lockdep wanings please attach the
> report.
>
Hi Kuai,
Thanks for the follow-up and for the suggestion to use a per-mddev lockdep
class for reconfig_mutex.
I implemented that on top of current mainline (same repro setup as before:
syzbot C repro cd254889 on QEMU, 6628e7d4-era config, CONFIG_LOCKDEP,
CONFIG_BLOCK_LEGACY_AUTOLOAD). In mddev_init() I use lockdep_register_key()
and mutex_init_with_key(&mddev->reconfig_mutex, &mddev->reconfig_lockdep_key),
with lockdep_unregister_key() in mddev_destroy(), following the pattern in
drivers/phy/phy-core.c.
With this change, the repro still reports a circular locking dependency (full
report from guest dmesg below). The chain is still:
disks_mutex --> &mddev->reconfig_mutex --> major_names_lock
The ioctl side still shows md_ioctl -> md_add_new_disk -> md_import_device ->
bdev_file_open_by_dev -> blk_probe_dev under major_names_lock while ioctl
holds &mddev->reconfig_mutex. The concurrent side is md_probe -> md_alloc
trying to take disks_mutex while already under major_names_lock.
So in my testing, per-mddev reconfig lock classes alone do not silence this
particular syzbot repro; the reconfig -> major_names edge via legacy autoload
in md_import_device() remains. For comparison, my earlier patch that used
bdev_file_open_by_dev_autoload(..., false) from md_import_device() did not
trigger this WARNING on the same repro (QEMU, before/after on 6628e7d4 bzImage).
Full lockdep report (dmesg excerpt from the repro run on the lockdep-class
tree):
======================================================
WARNING: possible circular locking dependency detected
7.4.0-rc1-00001-g2308e050e32d-dirty #2 Not tainted
------------------------------------------------------
repro/4976 is trying to acquire lock:
(disks_mutex){+.+.}-{4:4}, at: md_alloc+0x3e/0x1080
but task is already holding lock:
(major_names_lock){+.+.}-{4:4}, at: blk_probe_dev+0x25/0x190
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #2 (major_names_lock){+.+.}-{4:4}:
lock_acquire+0x1bf/0x370
__mutex_lock+0x173/0x1dd0
blk_probe_dev+0x25/0x190
blk_request_module+0x16/0xb0
blkdev_get_no_open+0xa0/0x100
bdev_file_open_by_dev+0x72/0x210
md_import_device+0x120/0x490
md_add_new_disk+0xa90/0x1ac0
md_ioctl+0x2a4e/0x3ba0
blkdev_ioctl+0x366/0x6c0
__x64_sys_ioctl+0x18f/0x210
do_syscall_64+0x132/0x7e0
entry_SYSCALL_64_after_hwframe+0x77/0x7f
-> #1 (&mddev->reconfig_mutex){+.+.}-{4:4}:
lock_acquire+0x1bf/0x370
__mutex_lock+0x173/0x1dd0
md_alloc+0x3cf/0x1080
md_alloc_and_put+0x17/0xe0
add_named_array+0x23d/0x380
param_attr_store+0x197/0x300
module_attr_store+0x57/0x80
sysfs_kf_write+0xef/0x150
kernfs_fop_write_iter+0x397/0x580
vfs_write+0xc1a/0x1150
ksys_write+0x121/0x240
do_syscall_64+0x132/0x7e0
entry_SYSCALL_64_after_hwframe+0x77
-> #0 (disks_mutex){+.+.}-{4:4}:
check_prev_add+0xeb/0xe60
__lock_acquire+0x1661/0x2110
lock_acquire+0x1bf/0x370
__mutex_lock+0x173/0x1dd0
md_alloc+0x3e/0x1080
md_probe+0x6d/0x100
blk_probe_dev+0x116/0x190
blk_request_module+0x16/0xb0
blkdev_get_no_open+0xa0/0x100
blkdev_open+0x143/0x500
do_dentry_open+0x68e/0x14f0
vfs_open_consume+0xcc/0x460
path_openat+0x1c16/0x2240
do_file_open+0x216/0x470
do_sys_openat2+0x139/0x1d0
__x64_sys_open+0x11c/0x1c0
do_syscall_64+0x132/0x7e0
entry_SYSCALL_64_after_hwframe+0x77/0x7f
other info that might help us debug this:
Chain exists of:
disks_mutex --> &mddev->reconfig_mutex --> major_names_lock
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(major_names_lock);
lock(&mddev->reconfig_mutex);
lock(major_names_lock);
lock(disks_mutex);
*** DEADLOCK ***
locks held by repro/4976: 1, last CPU#0:
#0: (major_names_lock){+.+.}-{4:4}, at: blk_probe_dev+0x25/0x190
stack backtrace:
CPU: 0 UID: 0 PID: 4976 Comm: repro Not tainted 7.4.0-rc1-00001-g2308e050e32d-dirty #2 PREEMPT(full)
Call Trace:
dump_stack_lvl+0x116/0x1b0
print_circular_bug+0x2e8/0x420
check_noncircular+0x12d/0x150
check_prev_add+0xeb/0xe60
__lock_acquire+0x1661/0x2110
lock_acquire+0x1bf/0x370
__mutex_lock+0x173/0x1dd0
md_alloc+0x3e/0x1080
md_probe+0x6d/0x100
blk_probe_dev+0x116/0x190
blk_request_module+0x16/0xb0
blkdev_get_no_open+0xa0/0x100
blkdev_open+0x143/0x500
do_dentry_open+0x68e/0x14f0
vfs_open_consume+0xcc/0x460
path_openat+0x1c16/0x2240
do_file_open+0x216/0x470
do_sys_openat2+0x139/0x1d0
__x64_sys_open+0x11c/0x1c0
do_syscall_64+0x132/0x7e0
entry_SYSCALL_64_after_hwframe+0x77/0x7f
======================================================
How would you like to proceed?
- lockdep class only (accept remaining ordering warnings for this path), or
- also break the ioctl-side reconfig -> major_names link (e.g. no legacy
autoload under reconfig_mutex, as in my earlier [PATCH]), or
- a block-layer change under major_names_lock, or something else?
I can send a revised patch on the list once you confirm which direction you
prefer. I am happy to drop or combine pieces of my earlier patch accordingly.
Thanks,
Jichen
> >
> > This patch does not revert that behaviour. It only breaks the ioctl-side
> > edge by not triggering legacy block autoload while reconfig_mutex is
> > held: md_import_device() uses bdev_file_open_by_dev_autoload(..., false).
> > Member disks are expected to already exist for normal mdadmin usage; the
> > syz repro still gets -ENXIO for bogus minors, but on a fixed kernel there
> > is no lockdep warning (Tested with syzbot's C repro on QEMU, 6628e7d4
> > bzImage, before/after).
> >
> > If maintainers prefer a block-layer approach instead (e.g. not calling
> > md probe callbacks under major_names_lock in blk_probe_dev()), I'm happy
> > to respin.
> >
> > Thanks,
> > Jichen
> >
> >>
> >>> Add bdev_file_open_by_dev_autoload() and use autoload=false from
> >>> md_import_device(); member devices must already be present in the
> >>> block layer (normal mdadmin usage).
> >>>
> >>> Link: https://syzbot.org/bug?extid=cd254889291ef8666036
> >>> Reported-by: syzbot+cd254889291ef8666036@xxxxxxxxxxxxxxxxxxxxxxxxx
> >>> Tested-by: Wang Jichen <wangjichen@xxxxxxxxxxx>
> >>> Signed-off-by: Wang Jichen <wangjichen@xxxxxxxxxxx>
> >>> ---
> >>> block/bdev.c | 13 ++++++++++---
> >>> drivers/md/md.c | 8 ++++++--
> >>> include/linux/blkdev.h | 2 ++
> >>> 3 files changed, 18 insertions(+), 5 deletions(-)
> >>>
> >>> diff --git a/block/bdev.c b/block/bdev.c
> >>> index f20f91b31..e78936556 100644
> >>> --- a/block/bdev.c
> >>> +++ b/block/bdev.c
> >>> @@ -1076,8 +1076,8 @@ static unsigned blk_to_file_flags(blk_mode_t mode)
> >>> return flags;
> >>> }
> >>>
> >>> -struct file *bdev_file_open_by_dev(dev_t dev, blk_mode_t mode, void *holder,
> >>> - const struct blk_holder_ops *hops)
> >>> +struct file *bdev_file_open_by_dev_autoload(dev_t dev, blk_mode_t mode,
> >>> + void *holder, const struct blk_holder_ops *hops, bool autoload)
> >>> {
> >>> struct file *bdev_file;
> >>> struct block_device *bdev;
> >>> @@ -1088,7 +1088,7 @@ struct file *bdev_file_open_by_dev(dev_t dev, blk_mode_t mode, void *holder,
> >>> if (ret)
> >>> return ERR_PTR(ret);
> >>>
> >>> - bdev = blkdev_get_no_open(dev, true);
> >>> + bdev = blkdev_get_no_open(dev, autoload);
> >>> if (!bdev)
> >>> return ERR_PTR(-ENXIO);
> >>>
> >>> @@ -1110,6 +1110,13 @@ struct file *bdev_file_open_by_dev(dev_t dev, blk_mode_t mode, void *holder,
> >>> }
> >>> return bdev_file;
> >>> }
> >>> +EXPORT_SYMBOL(bdev_file_open_by_dev_autoload);
> >>> +
> >>> +struct file *bdev_file_open_by_dev(dev_t dev, blk_mode_t mode, void *holder,
> >>> + const struct blk_holder_ops *hops)
> >>> +{
> >>> + return bdev_file_open_by_dev_autoload(dev, mode, holder, hops, true);
> >>> +}
> >>> EXPORT_SYMBOL(bdev_file_open_by_dev);
> >>>
> >>> struct file *bdev_file_open_by_path(const char *path, blk_mode_t mode,
> >>> diff --git a/drivers/md/md.c b/drivers/md/md.c
> >>> index 03e35386a..d3717e2e8 100644
> >>> --- a/drivers/md/md.c
> >>> +++ b/drivers/md/md.c
> >>> @@ -3858,9 +3858,13 @@ static struct md_rdev *md_import_device(dev_t newdev, int super_format, int supe
> >>> if (err)
> >>> goto out_clear_rdev;
> >>>
> >>> - rdev->bdev_file = bdev_file_open_by_dev(newdev,
> >>> + /*
> >>> + * md_ioctl() holds reconfig_mutex; legacy autoload in bdev open takes
> >>> + * major_names_lock via blk_probe_dev() and can deadlock with md_probe().
> >>> + */
> >>> + rdev->bdev_file = bdev_file_open_by_dev_autoload(newdev,
> >>> BLK_OPEN_READ | BLK_OPEN_WRITE,
> >>> - super_format == -2 ? &claim_rdev : rdev, NULL);
> >>> + super_format == -2 ? &claim_rdev : rdev, NULL, false);
> >>> if (IS_ERR(rdev->bdev_file)) {
> >>> pr_warn("md: could not open device unknown-block(%u,%u).\n",
> >>> MAJOR(newdev), MINOR(newdev));
> >>> diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
> >>> index 0c2010230..e5121101a 100644
> >>> --- a/include/linux/blkdev.h
> >>> +++ b/include/linux/blkdev.h
> >>> @@ -1699,6 +1699,8 @@ struct blk_holder_ops {
> >>>
> >>> struct file *bdev_file_open_by_dev(dev_t dev, blk_mode_t mode, void *holder,
> >>> const struct blk_holder_ops *hops);
> >>> +struct file *bdev_file_open_by_dev_autoload(dev_t dev, blk_mode_t mode,
> >>> + void *holder, const struct blk_holder_ops *hops, bool autoload);
> >>> struct file *bdev_file_open_by_path(const char *path, blk_mode_t mode,
> >>> void *holder, const struct blk_holder_ops *hops);
> >>> int bd_prepare_to_claim(struct block_device *bdev, void *holder,
> >>
> >> --
> >> Thanks,
> >> Kuai
> >>
>
> --
> Thanks,
> Kuai
>