[PATCH v2] stop_machine: Defer legacy console flushes while a CPU runs a stopper callback

[Aditya Chillara]

The cpu stopper thread runs above every other scheduling class, so while a
stopper callback executes, nothing else on that CPU is scheduled. If such a
callback emits a normal-priority printk(), the legacy console path can
synchronously flush the pending console backlog. On systems with a slow
UART and a large backlog, this holds the CPU long enough to starve RT
kthreads such as the watchdog pet, and for multi_cpu_stop() prevents the
CPU from advancing the state machine while the other CPUs wait. The
resulting delay can prevent watchdog servicing long enough to trigger a
watchdog bark or bite.

The same can happen from an interrupt taken during the callback:
multi_cpu_stop() keeps interrupts enabled during MULTI_STOP_PREPARE, and a
printk() may be emitted from a softirq run on irq exit.

Run CPU stopper callbacks in printk-deferred context to prevent legacy
console flushes while they execute.

Signed-off-by: Aditya Chillara <aditya.chillara@xxxxxxxxxxxxxxxx>
---
A device using a legacy UART console (console=ttyMSM0,115200n8) hit a
watchdog bark/bite about 40 seconds after boot.

stop_machine() (used here for kprobe text patching) stops every CPU by
running multi_cpu_stop() on each of them, through the per-CPU
"migration/%u" threads. These threads run at a higher priority than the
msm_watchdog thread. At bite time, all eight CPUs were still spinning in
multi_cpu_stop()'s MULTI_STOP_PREPARE state, where interrupts are left
enabled.

Heavy SELinux denial logging had built up a large backlog on the
console. One CPU took an interrupt while spinning in MULTI_STOP_PREPARE.
Handling it eventually led to a printk(), and because the console was a
legacy console, that printk() synchronously drained the whole backlog
over the slow UART. While the drain was still running, the watchdog bark
interrupt hit the same CPU, found no recent pet, and escalated to a
bite.

The captured stack for that CPU, innermost frame first:

  qcom_soc_set_wdt_bite
  qcom_wdt_bark_handler
  __handle_irq_event_percpu
  handle_irq_event
  handle_fasteoi_irq
  generic_handle_domain_irq
  gic_handle_irq
  do_interrupt_handler
  el1_interrupt
  el1h_64_irq_handler
  el1h_64_irq
  console_flush_all
  console_unlock
  vprintk_emit
  dev_vprintk_emit
  dev_printk_emit
  __dev_printk
  _dev_err
  btspi_sleep_timeout_handler
  call_timer_fn
  __run_timer_base
  run_timer_softirq
  handle_softirqs
  __do_softirq
  ____do_softirq
  call_on_irq_stack
  do_softirq_own_stack
  __irq_exit_rcu
  irq_exit_rcu
  el1_interrupt
  el1h_64_irq_handler
  el1h_64_irq
  multi_cpu_stop
  cpu_stopper_thread
  smpboot_thread_fn
  kthread
  ret_from_fork

Every other CPU stayed parked in the rendezvous the whole time, since
their stopper threads outrank msm_watchdog. Nothing could pet the
watchdog until the drain finished.

This was observed through multi_cpu_stop(), but the hazard is not
specific to it. Every cpu stopper callback runs in stop_sched_class,
above msm_watchdog and every other thread on the CPU, so a slow flush
from any of them (including single-CPU callbacks such as the migration
and task-migration stoppers) can starve the watchdog just as well. The
fix therefore covers all stopper callbacks, not only multi_cpu_stop().

Fix this by running the CPU stopper callbacks in printk-deferred context.

Reproduced and verified with an out-of-tree test module that triggers
stop_machine() with a queued console backlog and a printk() inside the
rendezvous, paired with a kprobe-based script that flags any console
flush happening while a CPU is inside a stopper callback.
---
Changes in v2:
- Use printk_deferred_enter/exit() to defer legacy console flushes instead
  of in_cpu_stop().
- Link to v1: https://patch.msgid.link/20260827-defer-legacy-console-write-on-multi_cpu_stop-v1-0-3b9f6bb4679f@xxxxxxxxxxxxxxxx
---
 kernel/stop_machine.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/kernel/stop_machine.c b/kernel/stop_machine.c
index d085ba1f4b44..31f7af41249f 100644
--- a/kernel/stop_machine.c
+++ b/kernel/stop_machine.c
@@ -507,7 +507,9 @@ static void cpu_stopper_thread(unsigned int cpu)
 		stopper->caller = work->caller;
 		stopper->fn = fn;
 		preempt_count_inc();
+		printk_deferred_enter();
 		ret = fn(arg);
+		printk_deferred_exit();
 		if (done) {
 			if (ret)
 				done->ret = ret;

---
base-commit: 77ae27fd98f3b548797c9f22c10ab5cf1c4ada53
change-id: 20260824-defer-legacy-console-write-on-multi_cpu_stop-d3ef6f6b150b

Best regards,
--  
Aditya Chillara <aditya.chillara@xxxxxxxxxxxxxxxx>