[PATCH 2/3] irq_work: Flush lazy work CPU down on PREEMPT_RT

[Sebastian Andrzej Siewior]

PREEMPT_RT invokes IRQ_WORK_LAZY callbacks and callbacks which are not
explicitly marked IRQ_WORK_HARD_IRQ from thread context.  If a CPU gets
shutdown, the irq_work is flushed during smpcfd_dying_cpu() which is
invoked on the target CPU with disabled interrupts.

The callbacks enqueued on lazy_list on PREEMPT_RT are not flushed because
they require thread context. The callbacks remain and get processed once
the CPU gets back online.

per-CPU irq_work items continue to work for enqueues on their local-CPU,
only the work on the "offline" CPU get stuck. "Single" irq_work are
worse because they remain "claimed" and can not be used again. One
"symptom" is the late printk() in CPU down path which enqueues the
irq_work for printing but it never gets scheduled. Further prints don't
print to the console because the irq_work is "pending" of the offline
CPU.
Another problem is the canceling/ flushing of the irq_work which is
stuck. Since the irq_work will not continue, the task waiting for its
completion will block waiting.

While looking through the users of irq_work, it does not matter if the
callback is enqueued on another CPU. This makes it possible to invoke
the callbacks from a remote CPU by smpcfd_dead_cpu(). This is invoked
from the control CPU with enabled interrupts shortly after the
hotplugged CPU is shutdown so the llist can be accessed safely.

Add irq_work_run_cpu() which invokes the irq_work callbacks of the
specified dead CPU. Invoke it from smpcfd_dead_cpu().

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@xxxxxxxxxxxxx>
---
 include/linux/irq_work.h |  2 ++
 kernel/irq_work.c        | 11 +++++++++++
 kernel/smp.c             |  1 +
 3 files changed, 14 insertions(+)

diff --git a/include/linux/irq_work.h b/include/linux/irq_work.h
index c5afd053ae32c..7184e12739653 100644
--- a/include/linux/irq_work.h
+++ b/include/linux/irq_work.h
@@ -59,6 +59,7 @@ void irq_work_sync(struct irq_work *work);
 #include <asm/irq_work.h>
 
 void irq_work_run(void);
+void irq_work_run_cpu(unsigned int cpu);
 bool irq_work_needs_cpu(void);
 void irq_work_single(void *arg);
 
@@ -67,6 +68,7 @@ void arch_irq_work_raise(void);
 #else
 static inline bool irq_work_needs_cpu(void) { return false; }
 static inline void irq_work_run(void) { }
+static inline void irq_work_run_cpu(unsigned int cpu) { }
 static inline void irq_work_single(void *arg) { }
 #endif
 
diff --git a/kernel/irq_work.c b/kernel/irq_work.c
index 9f19c05c9962c..73eabcbdcd50c 100644
--- a/kernel/irq_work.c
+++ b/kernel/irq_work.c
@@ -263,6 +263,17 @@ void irq_work_run(void)
 }
 EXPORT_SYMBOL_GPL(irq_work_run);
 
+void irq_work_run_cpu(unsigned int cpu)
+{
+	if (WARN_ON_ONCE(!cpumask_test_cpu(cpu, cpu_dying_mask)))
+		return;
+
+	if (!IS_ENABLED(CONFIG_PREEMPT_RT))
+		return;
+
+	irq_work_run_list(per_cpu_ptr(&lazy_list, cpu));
+}
+
 void irq_work_tick(void)
 {
 	struct llist_head *raised = this_cpu_ptr(&raised_list);
diff --git a/kernel/smp.c b/kernel/smp.c
index b696bcc60c08f..8f530a094c304 100644
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -87,6 +87,7 @@ int smpcfd_dead_cpu(unsigned int cpu)
 
 	free_cpumask_var(cfd->cpumask);
 	free_cpumask_var(cfd->cpumask_ipi);
+	irq_work_run_cpu(cpu);
 	return 0;
 }
 
-- 
2.55.0