Re: md deadlock (2.6.31-rc5-git2)

[Mike Snitzer]

On Mon, Aug 17, 2009 at 4:17 PM, Dave Jones <davej@xxxxxxxxxx> wrote:
>
> This kernel is a bit old (it's what we froze on for Fedora 12 alpha,
> and we haven't started building install images with anything newer yet),
> but I don't recall seeing anything similar posted recently..
>
> While creating a series of md arrays, I got the mdadm process to just lock up.
> Looking in dmesg showed that it had warned about it too ..
...
> INFO: task mdadm:2249 blocked for more than 120 seconds.
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> mdadm         D 0000000000000003  4664  2249    333 0x00000080
>  ffff880023c81938 0000000000000086 0000000000000000 0000000000000001
>  ffff88002bb124a0 0000000000000007 0000000000000006 ffff88003f417028
>  ffff88002bb12890 000000000000fa20 ffff88002bb12890 00000000001d5bc0
> Call Trace:
>  [<ffffffff81096546>] ? trace_hardirqs_on_caller+0x139/0x175
>  [<ffffffff8116ed21>] ? revalidate_disk+0x5e/0x9d
>  [<ffffffff814fb166>] __mutex_lock_common+0x21e/0x3bf
>  [<ffffffff8116ed21>] ? revalidate_disk+0x5e/0x9d
>  [<ffffffff814fb42a>] mutex_lock_nested+0x4f/0x6b
>  [<ffffffff8116ed21>] revalidate_disk+0x5e/0x9d
>  [<ffffffff813f35ce>] do_md_run+0x886/0x92f
>  [<ffffffff814fb356>] ? mutex_lock_interruptible_nested+0x4f/0x6a
>  [<ffffffff813f5f30>] md_ioctl+0x11b6/0x142b
>  [<ffffffff81095fab>] ? mark_lock+0x3c/0x253
>  [<ffffffff81095fab>] ? mark_lock+0x3c/0x253
>  [<ffffffff81263f12>] __blkdev_driver_ioctl+0x36/0x95
>  [<ffffffff81264895>] blkdev_ioctl+0x8d6/0x925
>  [<ffffffff8101aa23>] ? native_sched_clock+0x2d/0x62
>  [<ffffffff8122bae6>] ? __rcu_read_unlock+0x34/0x4a
>  [<ffffffff8122ca90>] ? avc_has_perm_noaudit+0x3c9/0x3ef
>  [<ffffffff8122cb21>] ? avc_has_perm+0x6b/0x91
>  [<ffffffff81096546>] ? trace_hardirqs_on_caller+0x139/0x175
>  [<ffffffff8116e5b4>] block_ioctl+0x4a/0x62
>  [<ffffffff81150e03>] vfs_ioctl+0x31/0xaa
>  [<ffffffff811513c5>] do_vfs_ioctl+0x4aa/0x506
>  [<ffffffff81151486>] sys_ioctl+0x65/0x9c
>  [<ffffffff81012f42>] system_call_fastpath+0x16/0x1b
> 2 locks held by mdadm/2249:
>  #0:  (&new->reconfig_mutex#2){+.+.+.}, at: [<ffffffff813edcab>] mddev_lock+0x2a/0x40
>  #1:  (&bdev->bd_mutex){+.+.+.}, at: [<ffffffff8116ed21>] revalidate_disk+0x5e/0x9d
> INFO: task mdadm:2249 blocked for more than 120 seconds.
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> mdadm         D 0000000000000003  4664  2249    333 0x00000080
>  ffff880023c81938 0000000000000086 0000000000000000 0000000000000001
>  ffff88002bb124a0 0000000000000007 0000000000000006 ffff88003f417028
>  ffff88002bb12890 000000000000fa20 ffff88002bb12890 00000000001d5bc0
> Call Trace:
>  [<ffffffff81096546>] ? trace_hardirqs_on_caller+0x139/0x175
>  [<ffffffff8116ed21>] ? revalidate_disk+0x5e/0x9d
>  [<ffffffff814fb166>] __mutex_lock_common+0x21e/0x3bf
>  [<ffffffff8116ed21>] ? revalidate_disk+0x5e/0x9d
>  [<ffffffff814fb42a>] mutex_lock_nested+0x4f/0x6b
>  [<ffffffff8116ed21>] revalidate_disk+0x5e/0x9d
>  [<ffffffff813f35ce>] do_md_run+0x886/0x92f
>  [<ffffffff814fb356>] ? mutex_lock_interruptible_nested+0x4f/0x6a
>  [<ffffffff813f5f30>] md_ioctl+0x11b6/0x142b
>  [<ffffffff81095fab>] ? mark_lock+0x3c/0x253
>  [<ffffffff81095fab>] ? mark_lock+0x3c/0x253
>  [<ffffffff81263f12>] __blkdev_driver_ioctl+0x36/0x95
>  [<ffffffff81264895>] blkdev_ioctl+0x8d6/0x925
>  [<ffffffff8101aa23>] ? native_sched_clock+0x2d/0x62
>  [<ffffffff8122bae6>] ? __rcu_read_unlock+0x34/0x4a
>  [<ffffffff8122ca90>] ? avc_has_perm_noaudit+0x3c9/0x3ef
>  [<ffffffff8122cb21>] ? avc_has_perm+0x6b/0x91
>  [<ffffffff81096546>] ? trace_hardirqs_on_caller+0x139/0x175
>  [<ffffffff8116e5b4>] block_ioctl+0x4a/0x62
>  [<ffffffff81150e03>] vfs_ioctl+0x31/0xaa
>  [<ffffffff811513c5>] do_vfs_ioctl+0x4aa/0x506
>  [<ffffffff81151486>] sys_ioctl+0x65/0x9c
>  [<ffffffff81012f42>] system_call_fastpath+0x16/0x1b
> 2 locks held by mdadm/2249:
>  #0:  (&new->reconfig_mutex#2){+.+.+.}, at: [<ffffffff813edcab>] mddev_lock+0x2a/0x40
>  #1:  (&bdev->bd_mutex){+.+.+.}, at: [<ffffffff8116ed21>] revalidate_disk+0x5e/0x9d

This was fixed with commit: c8c00a6915a2e3d10416e8bdd3138429beb96210
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/