Re: [RFC] Deadlock via recursive wakeup via RCU with threadirqs

[Joel Fernandes]

On Thu, Jun 27, 2019 at 11:40 AM Sebastian Andrzej Siewior
<bigeasy@xxxxxxxxxxxxx> wrote:
>
> On 2019-06-27 11:37:10 [-0400], Joel Fernandes wrote:
> > Sebastian it would be nice if possible to trace where the
> > t->rcu_read_unlock_special is set for this scenario of calling
> > rcu_read_unlock_special, to give a clear idea about whether it was
> > really because of an IPI. I guess we could also add additional RCU
> > debug fields to task_struct (just for debugging) to see where there
> > unlock_special is set.
> >
> > Is there a test to reproduce this, or do I just boot an intel x86_64
> > machine with "threadirqs" and run into it?
>
> Do you want to send me a patch or should I send you my kvm image which
> triggers the bug on boot?

I could reproduce this as well just booting Linus tree with threadirqs
command line and running rcutorture. In 15 seconds or so it locks
up... gdb backtrace shows the recursive lock:

(gdb) bt
#0  queued_spin_lock_slowpath (lock=0xffff88813af16b44, val=0) at
kernel/locking/qspinlock.c:381
#1  0xffffffff81a75a0f in queued_spin_lock (lock=<optimized out>) at
./include/asm-generic/qspinlock.h:81
#2  do_raw_spin_lock_flags (flags=<optimized out>, lock=<optimized
out>) at ./include/linux/spinlock.h:193
#3  __raw_spin_lock_irqsave (lock=<optimized out>) at
./include/linux/spinlock_api_smp.h:119
#4  _raw_spin_lock_irqsave (lock=0xffff88813af16b44) at
kernel/locking/spinlock.c:159
#5  0xffffffff81092158 in try_to_wake_up (p=0xffff88813af16400,
state=3, wake_flags=0) at kernel/sched/core.c:2000
#6  0xffffffff8109265c in wake_up_process (p=<optimized out>) at
kernel/sched/core.c:2114
#7  0xffffffff8106b019 in wakeup_softirqd () at kernel/softirq.c:76
#8  0xffffffff8106b8ef in raise_softirq_irqoff (nr=<optimized out>) at
kernel/softirq.c:437
#9  0xffffffff810d080d in rcu_read_unlock_special (t=<optimized out>)
at kernel/rcu/tree_plugin.h:632
#10 0xffffffff810d0868 in __rcu_read_unlock () at kernel/rcu/tree_plugin.h:414
#11 0xffffffff810aa58e in rcu_read_unlock () at ./include/linux/rcupdate.h:646
#12 cpuacct_charge (tsk=<optimized out>, cputime=<optimized out>) at
kernel/sched/cpuacct.c:352
#13 0xffffffff81097994 in cgroup_account_cputime
(delta_exec=<optimized out>, task=<optimized out>) at
./include/linux/cgroup.h:764
#14 update_curr (cfs_rq=<optimized out>) at kernel/sched/fair.c:843
#15 0xffffffff8109a882 in enqueue_entity (flags=<optimized out>,
se=<optimized out>, cfs_rq=<optimized out>) at
kernel/sched/fair.c:3901
#16 enqueue_task_fair (rq=<optimized out>, p=<optimized out>, flags=9)
at kernel/sched/fair.c:5194
#17 0xffffffff81091699 in enqueue_task (flags=<optimized out>,
p=<optimized out>, rq=<optimized out>) at kernel/sched/core.c:774
#18 activate_task (rq=0xffff88813af16b44, p=0x0 <fixed_percpu_data>,
flags=<optimized out>) at kernel/sched/core.c:795
#19 0xffffffff81091a1a in ttwu_do_activate (rq=0xffff88813af16b44,
p=0x0 <fixed_percpu_data>, wake_flags=0, rf=<optimized out>) at
kernel/sched/core.c:1737
#20 0xffffffff810924b3 in ttwu_queue (wake_flags=<optimized out>,
cpu=<optimized out>, p=<optimized out>) at kernel/sched/core.c:1882
#21 try_to_wake_up (p=0xffff88813af16400, state=<optimized out>,
wake_flags=0) at kernel/sched/core.c:2092
#22 0xffffffff8109265c in wake_up_process (p=<optimized out>) at
kernel/sched/core.c:2114
#23 0xffffffff8106b019 in wakeup_softirqd () at kernel/softirq.c:76
#24 0xffffffff8106b843 in invoke_softirq () at kernel/softirq.c:383
#25 irq_exit () at kernel/softirq.c:413
#26 0xffffffff81c01f8e in exiting_irq () at ./arch/x86/include/asm/apic.h:536


>
> Sebastian