BUG: workqueue leaked lock or atomic in smc_tx_work

From: syzbot
Date: Mon Jun 24 2019 - 04:21:13 EST

Hello,

syzbot found the following crash on:

HEAD commit: abf02e29 Merge tag 'pm-5.2-rc6' of git://git.kernel.org/pu..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=16ef68aaa00000
kernel config: https://syzkaller.appspot.com/x/.config?x=e5c77f8090a3b96b
dashboard link: https://syzkaller.appspot.com/bug?extid=8759e3927fd85a7c520a
compiler: gcc (GCC) 9.0.0 20181231 (experimental)

Unfortunately, I don't have any reproducer for this crash yet.

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+8759e3927fd85a7c520a@xxxxxxxxxxxxxxxxxxxxxxxxx

BUG: workqueue leaked lock or atomic: kworker/1:2/0x00000000/22354
last function: smc_tx_work
1 lock held by kworker/1:2/22354:
#0: 0000000093e90241 (sk_lock-AF_SMC){+.+.}, at: lock_sock include/net/sock.h:1522 [inline]
#0: 0000000093e90241 (sk_lock-AF_SMC){+.+.}, at: smc_tx_work+0x22/0x1d0 net/smc/smc_tx.c:577
CPU: 1 PID: 22354 Comm: kworker/1:2 Not tainted 5.2.0-rc5+ #57
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: events smc_tx_work
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x172/0x1f0 lib/dump_stack.c:113
process_one_work+0x108f/0x1790 kernel/workqueue.c:2284
worker_thread+0x98/0xe40 kernel/workqueue.c:2415
kthread+0x354/0x420 kernel/kthread.c:255
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352

======================================================
WARNING: possible circular locking dependency detected
5.2.0-rc5+ #57 Not tainted
------------------------------------------------------
kworker/1:2/22354 is trying to acquire lock:
0000000007a836d5 ((wq_completion)events){+.+.}, at: __write_once_size include/linux/compiler.h:221 [inline]
0000000007a836d5 ((wq_completion)events){+.+.}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
0000000007a836d5 ((wq_completion)events){+.+.}, at: atomic64_set include/asm-generic/atomic-instrumented.h:855 [inline]
0000000007a836d5 ((wq_completion)events){+.+.}, at: atomic_long_set include/asm-generic/atomic-long.h:40 [inline]
0000000007a836d5 ((wq_completion)events){+.+.}, at: set_work_data kernel/workqueue.c:620 [inline]
0000000007a836d5 ((wq_completion)events){+.+.}, at: set_work_pool_and_clear_pending kernel/workqueue.c:647 [inline]
0000000007a836d5 ((wq_completion)events){+.+.}, at: process_one_work+0x87e/0x1790 kernel/workqueue.c:2240

but task is already holding lock:
0000000093e90241 (sk_lock-AF_SMC){+.+.}, at: lock_sock include/net/sock.h:1522 [inline]
0000000093e90241 (sk_lock-AF_SMC){+.+.}, at: smc_tx_work+0x22/0x1d0 net/smc/smc_tx.c:577

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #2 (sk_lock-AF_SMC){+.+.}:
lock_sock_nested+0xcb/0x120 net/core/sock.c:2924
lock_sock include/net/sock.h:1522 [inline]
smc_tcp_listen_work+0x8a/0xf60 net/smc/af_smc.c:1366
process_one_work+0x989/0x1790 kernel/workqueue.c:2269
worker_thread+0x98/0xe40 kernel/workqueue.c:2415
kthread+0x354/0x420 kernel/kthread.c:255
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352

-> #1 ((work_completion)(&smc->tcp_listen_work)){+.+.}:
process_one_work+0x90f/0x1790 kernel/workqueue.c:2245
worker_thread+0x98/0xe40 kernel/workqueue.c:2415
kthread+0x354/0x420 kernel/kthread.c:255
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352

-> #0 ((wq_completion)events){+.+.}:
lock_acquire+0x16f/0x3f0 kernel/locking/lockdep.c:4303
process_one_work+0x8df/0x1790 kernel/workqueue.c:2244
worker_thread+0x98/0xe40 kernel/workqueue.c:2415
kthread+0x354/0x420 kernel/kthread.c:255
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352

other info that might help us debug this:

Chain exists of:
(wq_completion)events --> (work_completion)(&smc->tcp_listen_work) --> sk_lock-AF_SMC

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock(sk_lock-AF_SMC);
lock((work_completion)(&smc->tcp_listen_work));
lock(sk_lock-AF_SMC);
lock((wq_completion)events);

*** DEADLOCK ***

1 lock held by kworker/1:2/22354:
#0: 0000000093e90241 (sk_lock-AF_SMC){+.+.}, at: lock_sock include/net/sock.h:1522 [inline]
#0: 0000000093e90241 (sk_lock-AF_SMC){+.+.}, at: smc_tx_work+0x22/0x1d0 net/smc/smc_tx.c:577

stack backtrace:
CPU: 1 PID: 22354 Comm: kworker/1:2 Not tainted 5.2.0-rc5+ #57
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: events vmpressure_work_fn
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x172/0x1f0 lib/dump_stack.c:113
print_circular_bug.cold+0x1cc/0x28f kernel/locking/lockdep.c:1565
check_prev_add kernel/locking/lockdep.c:2310 [inline]
check_prevs_add kernel/locking/lockdep.c:2418 [inline]
validate_chain kernel/locking/lockdep.c:2800 [inline]
__lock_acquire+0x3755/0x5490 kernel/locking/lockdep.c:3793
lock_acquire+0x16f/0x3f0 kernel/locking/lockdep.c:4303
process_one_work+0x8df/0x1790 kernel/workqueue.c:2244
worker_thread+0x98/0xe40 kernel/workqueue.c:2415
kthread+0x354/0x420 kernel/kthread.c:255
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@xxxxxxxxxxxxxxxxx

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.