Re: BUG: sleeping function called from invalid context in lock_sock_nested

From: Eric Dumazet
Date: Wed Dec 04 2019 - 23:59:18 EST

Next message: Andrew Jeffery: "Re: [PATCH 05/14] ARM: dts: aspeed-g5: Fix aspeed,external-nodes description"
Previous message: Eric Dumazet: "[PATCH] time: tick-sched: Annotate lockless access to last_jiffies_update"
In reply to: syzbot: "BUG: sleeping function called from invalid context in lock_sock_nested"
Next in thread: Herbert Xu: "[PATCH] crypto: af_alg - Use bh_lock_sock in sk_destruct"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On 12/4/19 8:35 PM, syzbot wrote:
> Hello,
>
> syzbot found the following crash on:
>
> HEAD commit:ÂÂÂ 63de3747 Merge tag 'tag-chrome-platform-for-v5.5' of git:/..
> git tree:ÂÂÂÂÂÂ upstream
> console output: https://syzkaller.appspot.com/x/log.txt?x=1727d59ce00000
> kernel config:Â https://syzkaller.appspot.com/x/.config?x=1d189d07c6717979
> dashboard link: https://syzkaller.appspot.com/bug?extid=c2f1558d49e25cc36e5e
> compiler:ÂÂÂÂÂÂ gcc (GCC) 9.0.0 20181231 (experimental)
> syz repro:ÂÂÂÂÂ https://syzkaller.appspot.com/x/repro.syz?x=16fcf97ee00000
>
> IMPORTANT: if you fix the bug, please add the following tag to the commit:
> Reported-by: syzbot+c2f1558d49e25cc36e5e@xxxxxxxxxxxxxxxxxxxxxxxxx
>
> BUG: sleeping function called from invalid context at net/core/sock.c:2935
> in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 9008, name: udevd
> 4 locks held by udevd/9008:
> Â#0: ffff888098598428 (sb_writers#5){.+.+}, at: sb_start_write include/linux/fs.h:1650 [inline]
> Â#0: ffff888098598428 (sb_writers#5){.+.+}, at: mnt_want_write+0x3f/0xc0 fs/namespace.c:354
> Â#1: ffff8880a02bb248 (&type->i_mutex_dir_key#4/1){+.+.}, at: inode_lock_nested include/linux/fs.h:826 [inline]
> Â#1: ffff8880a02bb248 (&type->i_mutex_dir_key#4/1){+.+.}, at: filename_create+0x17c/0x4f0 fs/namei.c:3630
> Â#2: ffffffff89bb27a8 (tomoyo_ss){....}, at: tomoyo_path_perm+0x1cb/0x430 security/tomoyo/file.c:847
> Â#3: ffffffff897a3fc0 (rcu_callback){....}, at: __rcu_reclaim kernel/rcu/rcu.h:210 [inline]
> Â#3: ffffffff897a3fc0 (rcu_callback){....}, at: rcu_do_batch kernel/rcu/tree.c:2183 [inline]
> Â#3: ffffffff897a3fc0 (rcu_callback){....}, at: rcu_core+0x5f8/0x1540 kernel/rcu/tree.c:2408
> Preemption disabled at:
> [<ffffffff880000f3>] __do_softirq+0xf3/0x98c kernel/softirq.c:269
> CPU: 1 PID: 9008 Comm: udevd Not tainted 5.4.0-syzkaller #0
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
> Call Trace:
> Â<IRQ>
> Â__dump_stack lib/dump_stack.c:77 [inline]
> Âdump_stack+0x197/0x210 lib/dump_stack.c:118
> Â___might_sleep.cold+0x1fb/0x23e kernel/sched/core.c:6800
> Â__might_sleep+0x95/0x190 kernel/sched/core.c:6753
> Âlock_sock_nested+0x39/0x120 net/core/sock.c:2935
> Âlock_sock include/net/sock.h:1526 [inline]
> Âaf_alg_release_parent+0x1a6/0x290 crypto/af_alg.c:137
> Âhash_sock_destruct+0x164/0x1c0 crypto/algif_hash.c:423
> Â__sk_destruct+0x53/0x7f0 net/core/sock.c:1695
> Â__rcu_reclaim kernel/rcu/rcu.h:222 [inline]
> Ârcu_do_batch kernel/rcu/tree.c:2183 [inline]
> Ârcu_core+0x570/0x1540 kernel/rcu/tree.c:2408
> Ârcu_core_si+0x9/0x10 kernel/rcu/tree.c:2417
> Â__do_softirq+0x262/0x98c kernel/softirq.c:292
> Âinvoke_softirq kernel/softirq.c:373 [inline]
> Âirq_exit+0x19b/0x1e0 kernel/softirq.c:413
> Âexiting_irq arch/x86/include/asm/apic.h:536 [inline]
> Âsmp_apic_timer_interrupt+0x1a3/0x610 arch/x86/kernel/apic/apic.c:1137
> Âapic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:829
> Â</IRQ>
> RIP: 0010:__read_once_size include/linux/compiler.h:199 [inline]
> RIP: 0010:check_kcov_mode kernel/kcov.c:70 [inline]
> RIP: 0010:__sanitizer_cov_trace_pc+0x20/0x50 kernel/kcov.c:102
> Code: ff cc cc cc cc cc cc cc cc cc 55 48 89 e5 65 48 8b 04 25 c0 1e 02 00 65 8b 15 34 23 8d 7e 81 e2 00 01 1f 00 48 8b 75 08 75 2b <8b> 90 80 13 00 00 83 fa 02 75 20 48 8b 88 88 13 00 00 8b 80 84 13
> RSP: 0018:ffffc90001d67a58 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13
> RAX: ffff88809ac86680 RBX: 0000000000000011 RCX: ffffffff835559d0
> RDX: 0000000000000000 RSI: ffffffff8355597e RDI: 0000000000000003
> RBP: ffffc90001d67a58 R08: ffff88809ac86680 R09: ffffed1014154557
> R10: ffffed1014154556 R11: ffff8880a0aa2ab7 R12: dffffc0000000000
> R13: ffff8880a0aa2ab0 R14: ffffc90001d67ba0 R15: 0000000000000011
> Âtomoyo_check_acl+0x28e/0x3e0 security/tomoyo/domain.c:181
> Âtomoyo_path_permission security/tomoyo/file.c:586 [inline]
> Âtomoyo_path_permission+0x1fb/0x360 security/tomoyo/file.c:573
> Âtomoyo_path_perm+0x374/0x430 security/tomoyo/file.c:838
> Âtomoyo_path_symlink+0xaa/0xf0 security/tomoyo/tomoyo.c:206
> Âsecurity_path_symlink+0x10a/0x170 security/security.c:1053
> Âdo_symlinkat+0x137/0x290 fs/namei.c:4148
> Â__do_sys_symlink fs/namei.c:4169 [inline]
> Â__se_sys_symlink fs/namei.c:4167 [inline]
> Â__x64_sys_symlink+0x59/0x80 fs/namei.c:4167
> Âdo_syscall_64+0xfa/0x790 arch/x86/entry/common.c:294
> Âentry_SYSCALL_64_after_hwframe+0x49/0xbe
> RIP: 0033:0x7f68d8603517
> Code: 09 01 00 00 0f 05 48 3d 00 f0 ff ff 77 02 f3 c3 48 8b 15 14 39 2b 00 f7 d8 64 89 02 83 c8 ff c3 90 90 90 b8 58 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d f1 38 2b 00 31 d2 48 29 c2 64
> RSP: 002b:00007ffc322deae8 EFLAGS: 00000206 ORIG_RAX: 0000000000000058
> RAX: ffffffffffffffda RBX: 00000000025b6250 RCX: 00007f68d8603517
> RDX: 0000000000000002 RSI: 00007ffc322deb10 RDI: 00000000025cf730
> RBP: 00000000025b62d0 R08: 00007ffc322de6c0 R09: 00007f68d8657de0
> R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000032
> R13: 00000000025c7970 R14: 00000000025b6250 R15: 000000000000000b
>
> ================================
> WARNING: inconsistent lock state
> 5.4.0-syzkaller #0 Tainted: GÂÂÂÂÂÂÂ W
> --------------------------------
> inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
> udevd/9008 [HC0[0]:SC1[3]:HE1:SE0] takes:
> ffff88809f5a9110 (sk_lock-AF_ALG){+.?.}, at: lock_sock include/net/sock.h:1526 [inline]
> ffff88809f5a9110 (sk_lock-AF_ALG){+.?.}, at: af_alg_release_parent+0x1a6/0x290 crypto/af_alg.c:137
> {SOFTIRQ-ON-W} state was registered at:
> Â __trace_hardirqs_on_caller kernel/locking/lockdep.c:3389 [inline]
> Â lockdep_hardirqs_on+0x421/0x5e0 kernel/locking/lockdep.c:3434
> Â trace_hardirqs_on+0x67/0x240 kernel/trace/trace_preemptirq.c:31
> Â __local_bh_enable_ip+0x15a/0x270 kernel/softirq.c:194
> Â local_bh_enable include/linux/bottom_half.h:32 [inline]
> Â lock_sock_nested+0xe2/0x120 net/core/sock.c:2945
> Â lock_sock include/net/sock.h:1526 [inline]
> Â alg_bind+0x288/0x570 crypto/af_alg.c:187
> Â __sys_bind+0x239/0x290 net/socket.c:1649
> Â __do_sys_bind net/socket.c:1660 [inline]
> Â __se_sys_bind net/socket.c:1658 [inline]
> Â __x64_sys_bind+0x73/0xb0 net/socket.c:1658
> Â do_syscall_64+0xfa/0x790 arch/x86/entry/common.c:294
> Â entry_SYSCALL_64_after_hwframe+0x49/0xbe
> irq event stamp: 55316
> hardirqs lastÂ enabled at (55316): [<ffffffff81006743>] trace_hardirqs_on_thunk+0x1a/0x1c arch/x86/entry/thunk_64.S:41
> hardirqs last disabled at (55315): [<ffffffff8100675f>] trace_hardirqs_off_thunk+0x1a/0x1c arch/x86/entry/thunk_64.S:42
> softirqs lastÂ enabled at (53682): [<ffffffff812aa92e>] memcpy include/linux/string.h:380 [inline]
> softirqs lastÂ enabled at (53682): [<ffffffff812aa92e>] fpu__copy+0x17e/0x8c0 arch/x86/kernel/fpu/core.c:195
> softirqs last disabled at (55075): [<ffffffff81475c8b>] invoke_softirq kernel/softirq.c:373 [inline]
> softirqs last disabled at (55075): [<ffffffff81475c8b>] irq_exit+0x19b/0x1e0 kernel/softirq.c:413
>
> other info that might help us debug this:
> ÂPossible unsafe locking scenario:
>
> ÂÂÂÂÂÂ CPU0
> ÂÂÂÂÂÂ ----
> Â lock(sk_lock-AF_ALG);
> Â <Interrupt>
> ÂÂÂ lock(sk_lock-AF_ALG);
>
> Â*** DEADLOCK ***
>
> 4 locks held by udevd/9008:
> Â#0: ffff888098598428 (sb_writers#5){.+.+}, at: sb_start_write include/linux/fs.h:1650 [inline]
> Â#0: ffff888098598428 (sb_writers#5){.+.+}, at: mnt_want_write+0x3f/0xc0 fs/namespace.c:354
> Â#1: ffff8880a02bb248 (&type->i_mutex_dir_key#4/1){+.+.}, at: inode_lock_nested include/linux/fs.h:826 [inline]
> Â#1: ffff8880a02bb248 (&type->i_mutex_dir_key#4/1){+.+.}, at: filename_create+0x17c/0x4f0 fs/namei.c:3630
> Â#2: ffffffff89bb27a8 (tomoyo_ss){....}, at: tomoyo_path_perm+0x1cb/0x430 security/tomoyo/file.c:847
> Â#3: ffffffff897a3fc0 (rcu_callback){....}, at: __rcu_reclaim kernel/rcu/rcu.h:210 [inline]
> Â#3: ffffffff897a3fc0 (rcu_callback){....}, at: rcu_do_batch kernel/rcu/tree.c:2183 [inline]
> Â#3: ffffffff897a3fc0 (rcu_callback){....}, at: rcu_core+0x5f8/0x1540 kernel/rcu/tree.c:2408
>
> stack backtrace:
> CPU: 1 PID: 9008 Comm: udevd Tainted: GÂÂÂÂÂÂÂ WÂÂÂÂÂÂÂÂ 5.4.0-syzkaller #0
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
> Call Trace:
> Â<IRQ>
> Â__dump_stack lib/dump_stack.c:77 [inline]
> Âdump_stack+0x197/0x210 lib/dump_stack.c:118
> Âprint_usage_bug.cold+0x327/0x378 kernel/locking/lockdep.c:3101
> Âvalid_state kernel/locking/lockdep.c:3112 [inline]
> Âmark_lock_irq kernel/locking/lockdep.c:3309 [inline]
> Âmark_lock+0xbb4/0x1220 kernel/locking/lockdep.c:3666
> Âmark_usage kernel/locking/lockdep.c:3566 [inline]
> Â__lock_acquire+0x1e8e/0x4a00 kernel/locking/lockdep.c:3909
> Âlock_acquire+0x190/0x410 kernel/locking/lockdep.c:4485
> Âlock_sock_nested+0xcb/0x120 net/core/sock.c:2944
> Âlock_sock include/net/sock.h:1526 [inline]
> Âaf_alg_release_parent+0x1a6/0x290 crypto/af_alg.c:137
> Âhash_sock_destruct+0x164/0x1c0 crypto/algif_hash.c:423
> Â__sk_destruct+0x53/0x7f0 net/core/sock.c:1695
> Â__rcu_reclaim kernel/rcu/rcu.h:222 [inline]
> Ârcu_do_batch kernel/rcu/tree.c:2183 [inline]
> Ârcu_core+0x570/0x1540 kernel/rcu/tree.c:2408
> Ârcu_core_si+0x9/0x10 kernel/rcu/tree.c:2417
> Â__do_softirq+0x262/0x98c kernel/softirq.c:292
> Âinvoke_softirq kernel/softirq.c:373 [inline]
> Âirq_exit+0x19b/0x1e0 kernel/softirq.c:413
> Âexiting_irq arch/x86/include/asm/apic.h:536 [inline]
> Âsmp_apic_timer_interrupt+0x1a3/0x610 arch/x86/kernel/apic/apic.c:1137
> Âapic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:829
> Â</IRQ>
> RIP: 0010:__read_once_size include/linux/compiler.h:199 [inline]
> RIP: 0010:check_kcov_mode kernel/kcov.c:70 [inline]
> RIP: 0010:__sanitizer_cov_trace_pc+0x20/0x50 kernel/kcov.c:102
> Code: ff cc cc cc cc cc cc cc cc cc 55 48 89 e5 65 48 8b 04 25 c0 1e 02 00 65 8b 15 34 23 8d 7e 81 e2 00 01 1f 00 48 8b 75 08 75 2b <8b> 90 80 13 00 00 83 fa 02 75 20 48 8b 88 88 13 00 00 8b 80 84 13
> RSP: 0018:ffffc90001d67a58 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13
> RAX: ffff88809ac86680 RBX: 0000000000000011 RCX: ffffffff835559d0
> RDX: 0000000000000000 RSI: ffffffff8355597e RDI: 0000000000000003
> RBP: ffffc90001d67a58 R08: ffff88809ac86680 R09: ffffed1014154557
> R10: ffffed1014154556 R11: ffff8880a0aa2ab7 R12: dffffc0000000000
> R13: ffff8880a0aa2ab0 R14: ffffc90001d67ba0 R15: 0000000000000011
> Âtomoyo_check_acl+0x28e/0x3e0 security/tomoyo/domain.c:181
> Âtomoyo_path_permission security/tomoyo/file.c:586 [inline]
> Âtomoyo_path_permission+0x1fb/0x360 security/tomoyo/file.c:573
> Âtomoyo_path_perm+0x374/0x430 security/tomoyo/file.c:838
> Âtomoyo_path_symlink+0xaa/0xf0 security/tomoyo/tomoyo.c:206
> Âsecurity_path_symlink+0x10a/0x170 security/security.c:1053
> Âdo_symlinkat+0x137/0x290 fs/namei.c:4148
> Â__do_sys_symlink fs/namei.c:4169 [inline]
> Â__se_sys_symlink fs/namei.c:4167 [inline]
> Â__x64_sys_symlink+0x59/0x80 fs/namei.c:4167
> Âdo_syscall_64+0xfa/0x790 arch/x86/entry/common.c:294
> Âentry_SYSCALL_64_after_hwframe+0x49/0xbe
> RIP: 0033:0x7f68d8603517
> Code: 09 01 00 00 0f 05 48 3d 00 f0 ff ff 77 02 f3 c3 48 8b 15 14 39 2b 00 f7 d8 64 89 02 83 c8 ff c3 90 90 90 b8 58 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d f1 38 2b 00 31 d2 48 29 c2 64
> RSP: 002b:00007ffc322deae8 EFLAGS: 00000206 ORIG_RAX: 0000000000000058
> RAX: ffffffffffffffda RBX: 00000000025b6250 RCX: 00007f68d8603517
> RDX: 0000000000000002 RSI: 00007ffc322deb10 RDI: 00000000025cf730
> RBP: 00000000025b62d0 R08: 00007ffc322de6c0 R09: 00007f68d8657de0
> R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000032
> R13: 00000000025c7970 R14: 00000000025b6250 R15: 000000000000000b
> BUG: sleeping function called from invalid context at net/core/sock.c:2935
> in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 16, name: ksoftirqd/1
> INFO: lockdep is turned off.
> Preemption disabled at:
> [<ffffffff880000f3>] __do_softirq+0xf3/0x98c kernel/softirq.c:269
> CPU: 1 PID: 16 Comm: ksoftirqd/1 Tainted: GÂÂÂÂÂÂÂ WÂÂÂÂÂÂÂÂ 5.4.0-syzkaller #0
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
> Call Trace:
> Â__dump_stack lib/dump_stack.c:77 [inline]
> Âdump_stack+0x197/0x210 lib/dump_stack.c:118
> Â___might_sleep.cold+0x1fb/0x23e kernel/sched/core.c:6800
> Â__might_sleep+0x95/0x190 kernel/sched/core.c:6753
> Âlock_sock_nested+0x39/0x120 net/core/sock.c:2935
> Âlock_sock include/net/sock.h:1526 [inline]
> Âaf_alg_release_parent+0x1a6/0x290 crypto/af_alg.c:137
> Âhash_sock_destruct+0x164/0x1c0 crypto/algif_hash.c:423
> Â__sk_destruct+0x53/0x7f0 net/core/sock.c:1695
> Â__rcu_reclaim kernel/rcu/rcu.h:222 [inline]
> Ârcu_do_batch kernel/rcu/tree.c:2183 [inline]
> Ârcu_core+0x570/0x1540 kernel/rcu/tree.c:2408
> Ârcu_core_si+0x9/0x10 kernel/rcu/tree.c:2417
> Â__do_softirq+0x262/0x98c kernel/softirq.c:292
> Ârun_ksoftirqd kernel/softirq.c:603 [inline]
> Ârun_ksoftirqd+0x8e/0x110 kernel/softirq.c:595
> Âsmpboot_thread_fn+0x6a3/0xa40 kernel/smpboot.c:165
> Âkthread+0x361/0x430 kernel/kthread.c:255
> Âret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352
> BUG: sleeping function called from invalid context at net/core/sock.c:2935
> in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 16, name: ksoftirqd/1
> INFO: lockdep is turned off.
> Preemption disabled at:
> [<ffffffff880000f3>] __do_softirq+0xf3/0x98c kernel/softirq.c:269
> CPU: 1 PID: 16 Comm: ksoftirqd/1 Tainted: GÂÂÂÂÂÂÂ WÂÂÂÂÂÂÂÂ 5.4.0-syzkaller #0
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
> Call Trace:
> Â__dump_stack lib/dump_stack.c:77 [inline]
> Âdump_stack+0x197/0x210 lib/dump_stack.c:118
> Â___might_sleep.cold+0x1fb/0x23e kernel/sched/core.c:6800
> Â__might_sleep+0x95/0x190 kernel/sched/core.c:6753
> Âlock_sock_nested+0x39/0x120 net/core/sock.c:2935
> Âlock_sock include/net/sock.h:1526 [inline]
> Âaf_alg_release_parent+0x1a6/0x290 crypto/af_alg.c:137
> Âhash_sock_destruct+0x164/0x1c0 crypto/algif_hash.c:423
> Â__sk_destruct+0x53/0x7f0 net/core/sock.c:1695
> Â__rcu_reclaim kernel/rcu/rcu.h:222 [inline]
> Ârcu_do_batch kernel/rcu/tree.c:2183 [inline]
> Ârcu_core+0x570/0x1540 kernel/rcu/tree.c:2408
> Ârcu_core_si+0x9/0x10 kernel/rcu/tree.c:2417
> Â__do_softirq+0x262/0x98c kernel/softirq.c:292
> Ârun_ksoftirqd kernel/softirq.c:603 [inline]
> Ârun_ksoftirqd+0x8e/0x110 kernel/softirq.c:595
> Âsmpboot_thread_fn+0x6a3/0xa40 kernel/smpboot.c:165
> Âkthread+0x361/0x430 kernel/kthread.c:255
> Âret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352
>
>
> ---
> This bug is generated by a bot. It may contain errors.
> See https://goo.gl/tpsmEJ for more information about syzbot.
> syzbot engineers can be reached at syzkaller@xxxxxxxxxxxxxxxxx
>
> syzbot will keep track of this bug report. See:
> https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
> syzbot can test patches for this bug, for details see:
> https://goo.gl/tpsmEJ#testing-patches

crypto layer (hash_sock_destruct()) is called from rcu callback (this in BH context) but tries to grab a socket lock.

A socket lock can schedule, which is illegal in BH context.

Next message: Andrew Jeffery: "Re: [PATCH 05/14] ARM: dts: aspeed-g5: Fix aspeed,external-nodes description"
Previous message: Eric Dumazet: "[PATCH] time: tick-sched: Annotate lockless access to last_jiffies_update"
In reply to: syzbot: "BUG: sleeping function called from invalid context in lock_sock_nested"
Next in thread: Herbert Xu: "[PATCH] crypto: af_alg - Use bh_lock_sock in sk_destruct"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]