[Kernel Bug] possible deadlock in crypto_alg_lookup

From: 李龙兴

Date: Mon Feb 02 2026 - 01:39:08 EST


Dear Linux kernel developers and maintainers,

We would like to report a new kernel bug found by our tool. possible
deadlock in crypto_alg_lookup. Details are as follows.

Kernel commit: v6.18.2
Kernel config: see attachment
report: see attachment

We are currently analyzing the root cause and working on a
reproducible PoC. We will provide further updates in this thread as
soon as we have more information.

Best regards,
Longxing Li

======================================================
WARNING: possible circular locking dependency detected
6.18.2 #1 Not tainted
------------------------------------------------------
syz.2.389/14495 is trying to acquire lock:
ffffffff8ebbd970 (crypto_alg_sem){++++}-{4:4}, at:
crypto_alg_lookup+0x44/0x1e0 crypto/api.c:262

but task is already holding lock:
ffff888037928fd8 (sk_lock-AF_INET6){+.+.}-{0:0}, at: lock_sock
include/net/sock.h:1679 [inline]
ffff888037928fd8 (sk_lock-AF_INET6){+.+.}-{0:0}, at: do_tls_setsockopt
net/tls/tls_main.c:820 [inline]
ffff888037928fd8 (sk_lock-AF_INET6){+.+.}-{0:0}, at:
tls_setsockopt+0x1d7/0x1970 net/tls/tls_main.c:849

which lock already depends on the new lock.

...

other info that might help us debug this:

Chain exists of:
crypto_alg_sem --> &nsock->tx_lock --> sk_lock-AF_INET6

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock(sk_lock-AF_INET6);
lock(&nsock->tx_lock);
lock(sk_lock-AF_INET6);
rlock(crypto_alg_sem);

*** DEADLOCK ***

1 lock held by syz.2.389/14495:
#0: ffff888037928fd8 (sk_lock-AF_INET6){+.+.}-{0:0}, at: lock_sock
include/net/sock.h:1679 [inline]
#0: ffff888037928fd8 (sk_lock-AF_INET6){+.+.}-{0:0}, at:
do_tls_setsockopt net/tls/tls_main.c:820 [inline]
#0: ffff888037928fd8 (sk_lock-AF_INET6){+.+.}-{0:0}, at:
tls_setsockopt+0x1d7/0x1970 net/tls/tls_main.c:849

stack backtrace:
CPU: 0 UID: 0 PID: 14495 Comm: syz.2.389 Not tainted 6.18.2 #1 PREEMPT(full)
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:94 [inline]
dump_stack_lvl+0x116/0x1f0 lib/dump_stack.c:120
print_circular_bug+0x275/0x350 kernel/locking/lockdep.c:2043
check_noncircular+0x14c/0x170 kernel/locking/lockdep.c:2175
check_prev_add kernel/locking/lockdep.c:3165 [inline]
check_prevs_add kernel/locking/lockdep.c:3284 [inline]
validate_chain kernel/locking/lockdep.c:3908 [inline]
__lock_acquire+0x126f/0x1c90 kernel/locking/lockdep.c:5237
lock_acquire kernel/locking/lockdep.c:5868 [inline]
lock_acquire+0x179/0x350 kernel/locking/lockdep.c:5825
down_read+0x9b/0x480 kernel/locking/rwsem.c:1537
crypto_alg_lookup+0x44/0x1e0 crypto/api.c:262
crypto_larval_lookup crypto/api.c:300 [inline]
crypto_alg_mod_lookup+0x71/0x520 crypto/api.c:353
crypto_find_alg crypto/api.c:599 [inline]
crypto_alloc_tfm_node+0xd3/0x260 crypto/api.c:636
tls_set_sw_offload+0xd73/0x16e0 net/tls/tls_sw.c:2839
do_tls_setsockopt_conf net/tls/tls_main.c:695 [inline]
do_tls_setsockopt net/tls/tls_main.c:821 [inline]
tls_setsockopt+0x11ea/0x1970 net/tls/tls_main.c:849
do_sock_setsockopt+0xf3/0x1d0 net/socket.c:2360
__sys_setsockopt+0x1a0/0x230 net/socket.c:2385
__do_sys_setsockopt net/socket.c:2391 [inline]
__se_sys_setsockopt net/socket.c:2388 [inline]
__x64_sys_setsockopt+0xbd/0x160 net/socket.c:2388
do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
do_syscall_64+0xcd/0xfa0 arch/x86/entry/syscall_64.c:94
entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x5656ed
Code: 02 b8 ff ff ff ff c3 66 0f 1f 44 00 00 f3 0f 1e fa 48 89 f8 48
89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d
01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f6ce7edafc8 EFLAGS: 00000246 ORIG_RAX: 0000000000000036
RAX: ffffffffffffffda RBX: 0000000000715f80 RCX: 00000000005656ed
RDX: 0000000000000001 RSI: 000000000000011a RDI: 0000000000000004
RBP: 0000000000000000 R08: 0000000000000038 R09: 0000000000000000
R10: 0000000020000200 R11: 0000000000000246 R12: 0000000000715f8c
R13: 0000000000000000 R14: 0000000000715f80 R15: 00007f6ce7ebb000
</TASK>

https://drive.google.com/file/d/17HbDTI_iPjA72SkV5MnO-_w7IQZ9HIHW/view?usp=drive_link

https://drive.google.com/file/d/1egT1-2GQukoY_pwOr66U1XvvpKJLChpp/view?usp=drive_link