Re: INFO: task hung in io_sq_thread_park

[Jens Axboe]

On 2/28/21 2:58 PM, syzbot wrote:
> Hello,
> 
> syzbot found the following issue on:
> 
> HEAD commit:    d01f2f7e Add linux-next specific files for 20210226
> git tree:       linux-next
> console output: https://syzkaller.appspot.com/x/log.txt?x=1265dcead00000
> kernel config:  https://syzkaller.appspot.com/x/.config?x=a1746d2802a82a05
> dashboard link: https://syzkaller.appspot.com/bug?extid=fb5458330b4442f2090d
> syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=175427f2d00000
> C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=11109782d00000
> 
> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> Reported-by: syzbot+fb5458330b4442f2090d@xxxxxxxxxxxxxxxxxxxxxxxxx
> 
> INFO: task syz-executor458:8401 blocked for more than 143 seconds.
>       Not tainted 5.11.0-next-20210226-syzkaller #0
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> task:syz-executor458 state:D stack:27536 pid: 8401 ppid:  8400 flags:0x00004004
> Call Trace:
>  context_switch kernel/sched/core.c:4324 [inline]
>  __schedule+0x90c/0x21a0 kernel/sched/core.c:5075
>  schedule+0xcf/0x270 kernel/sched/core.c:5154
>  schedule_timeout+0x1db/0x250 kernel/time/timer.c:1868
>  do_wait_for_common kernel/sched/completion.c:85 [inline]
>  __wait_for_common kernel/sched/completion.c:106 [inline]
>  wait_for_common kernel/sched/completion.c:117 [inline]
>  wait_for_completion+0x168/0x270 kernel/sched/completion.c:138
>  io_sq_thread_park fs/io_uring.c:7115 [inline]
>  io_sq_thread_park+0xd5/0x130 fs/io_uring.c:7103
>  io_uring_cancel_task_requests+0x24c/0xd90 fs/io_uring.c:8745
>  __io_uring_files_cancel+0x110/0x230 fs/io_uring.c:8840
>  io_uring_files_cancel include/linux/io_uring.h:47 [inline]
>  do_exit+0x299/0x2a60 kernel/exit.c:780
>  do_group_exit+0x125/0x310 kernel/exit.c:922
>  __do_sys_exit_group kernel/exit.c:933 [inline]
>  __se_sys_exit_group kernel/exit.c:931 [inline]
>  __x64_sys_exit_group+0x3a/0x50 kernel/exit.c:931
>  do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46
>  entry_SYSCALL_64_after_hwframe+0x44/0xae
> RIP: 0033:0x43e899
> RSP: 002b:00007ffe89376d48 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
> RAX: ffffffffffffffda RBX: 00000000004af2f0 RCX: 000000000043e899
> RDX: 000000000000003c RSI: 00000000000000e7 RDI: 0000000000000000
> RBP: 0000000000000000 R08: ffffffffffffffc0 R09: 0000000010000000
> R10: 0000000000008011 R11: 0000000000000246 R12: 00000000004af2f0
> R13: 0000000000000001 R14: 0000000000000000 R15: 0000000000000001
> INFO: task iou-sqp-8401:8402 can't die for more than 143 seconds.
> task:iou-sqp-8401    state:D stack:30272 pid: 8402 ppid:  8400 flags:0x00004004
> Call Trace:
>  context_switch kernel/sched/core.c:4324 [inline]
>  __schedule+0x90c/0x21a0 kernel/sched/core.c:5075
>  schedule+0xcf/0x270 kernel/sched/core.c:5154
>  schedule_timeout+0x1db/0x250 kernel/time/timer.c:1868
>  do_wait_for_common kernel/sched/completion.c:85 [inline]
>  __wait_for_common kernel/sched/completion.c:106 [inline]
>  wait_for_common kernel/sched/completion.c:117 [inline]
>  wait_for_completion+0x168/0x270 kernel/sched/completion.c:138
>  io_sq_thread+0x27d/0x1ae0 fs/io_uring.c:6717
>  ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:294
> INFO: task iou-sqp-8401:8402 blocked for more than 143 seconds.
>       Not tainted 5.11.0-next-20210226-syzkaller #0
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> task:iou-sqp-8401    state:D stack:30272 pid: 8402 ppid:  8400 flags:0x00004004
> Call Trace:
>  context_switch kernel/sched/core.c:4324 [inline]
>  __schedule+0x90c/0x21a0 kernel/sched/core.c:5075
>  schedule+0xcf/0x270 kernel/sched/core.c:5154
>  schedule_timeout+0x1db/0x250 kernel/time/timer.c:1868
>  do_wait_for_common kernel/sched/completion.c:85 [inline]
>  __wait_for_common kernel/sched/completion.c:106 [inline]
>  wait_for_common kernel/sched/completion.c:117 [inline]
>  wait_for_completion+0x168/0x270 kernel/sched/completion.c:138
>  io_sq_thread+0x27d/0x1ae0 fs/io_uring.c:6717
>  ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:294
> 
> Showing all locks held in the system:
> 1 lock held by khungtaskd/1666:
>  #0: ffffffff8bf741e0 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x53/0x260 kernel/locking/lockdep.c:6327
> 1 lock held by syz-executor458/8401:
>  #0: ffff88801cafe870 (&sqd->lock){+.+.}-{3:3}, at: io_sq_thread_park+0x5d/0x130 fs/io_uring.c:7108
> 
> =============================================
> 
> NMI backtrace for cpu 0
> CPU: 0 PID: 1666 Comm: khungtaskd Not tainted 5.11.0-next-20210226-syzkaller #0
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:79 [inline]
>  dump_stack+0xfa/0x151 lib/dump_stack.c:120
>  nmi_cpu_backtrace.cold+0x44/0xd7 lib/nmi_backtrace.c:105
>  nmi_trigger_cpumask_backtrace+0x1b3/0x230 lib/nmi_backtrace.c:62
>  trigger_all_cpu_backtrace include/linux/nmi.h:147 [inline]
>  check_hung_uninterruptible_tasks kernel/hung_task.c:253 [inline]
>  watchdog+0xd8e/0xf40 kernel/hung_task.c:338
>  kthread+0x3b1/0x4a0 kernel/kthread.c:292
>  ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:294
> Sending NMI from CPU 0 to CPUs 1:
> NMI backtrace for cpu 1 skipped: idling at native_safe_halt arch/x86/include/asm/irqflags.h:51 [inline]
> NMI backtrace for cpu 1 skipped: idling at arch_safe_halt arch/x86/include/asm/irqflags.h:89 [inline]
> NMI backtrace for cpu 1 skipped: idling at acpi_safe_halt drivers/acpi/processor_idle.c:110 [inline]
> NMI backtrace for cpu 1 skipped: idling at acpi_idle_do_entry+0x1c9/0x250 drivers/acpi/processor_idle.c:516

Once for-next gets to something more recent, then this one (and others) will
be gone.

-- 
Jens Axboe