[PATCH] softirq: WARN_ON !preemptible() not check softirq cnt in bh disable on RT

[Xin Zhao]

In RT-Linux, when enabling CONFIG_PREEMPT_RT_NEEDS_BH_LOCK, calling
__local_bh_disable_ip() when !preemptible() is illegal because it uses
local_lock, which might sleep. The only exception is during the
cgroup_init() logic in start_kernel() while preemption is disabled,
cgroup_init() calls cgroup_idr_alloc(), which calls spin_lock_bh().
It is sufficient to only exclude the system startup phase in macro
DEBUG_LOCKS_WARN_ON.

Although the original check of this_cpu_read(softirq_ctrl.cnt) can also
prevent the WARN_ON print during the boot process, it may hide some issues
that should be exposed immediately. Because softirq_ctrl.cnt maybe 0 when
__local_bh_disabled_ip() is called in !preemptible() context.

In RT-Linux, __local_bh_disable_ip() will be used by numerous _bh variants
locks and local_bh_disable(). Since locks call __might_resched() check, we
analyze the scenario of using local_bh_disable() in !preemptible() context.

If CONFIG_PREEMPT_RT_NEEDS_BH_LOCK is not enabled, __local_bh_disable_ip()
does not enter the local_lock lock and thus using local_bh_disable() in
!preemptible() context does not lead to might sleep problem, but using
local_bh_disable() in !preemptible() state is not meaningful in RT-Linux.

In non RT-Linux, we use local_irq_save() followed by local_bh_disable() to
keep soft interrupts disabled after restoring interrupts. However, in
RT-Linux, when CONFIG_PREEMPT_RT_NEEDS_BH_LOCK is not enabled,
local_bh_disable() merely increments the softirq_ctrl.cnt counter without
actually disabling the soft interrupt behavior, because other tasks on the
CPU can preempt the task that wants to disable soft interrupts and execute
soft interrupt-related logic.
Consider the sequence diagram below:
Task A                        Task B
                              __local_bh_enable_ip()
                                __do_softirq()
                                  handle_softirqs()
                                    ...
                                    local_irq_enable();
                                    ...
local_irq_save()
local_bh_disable()
local_irq_restore()
                                    h->action(); -- it is serving softirq
local_bh_enable()

Signed-off-by: Xin Zhao <jackzxcui1989@xxxxxxx>
---
 kernel/softirq.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/softirq.c b/kernel/softirq.c
index 77198911b..320a52583 100644
--- a/kernel/softirq.c
+++ b/kernel/softirq.c
@@ -173,7 +173,7 @@ void __local_bh_disable_ip(unsigned long ip, unsigned int cnt)
 			/* Required to meet the RCU bottomhalf requirements. */
 			rcu_read_lock();
 		} else {
-			DEBUG_LOCKS_WARN_ON(this_cpu_read(softirq_ctrl.cnt));
+			DEBUG_LOCKS_WARN_ON(system_state != SYSTEM_BOOTING);
 		}
 	}
 
-- 
2.34.1