[PATCH v2] rseq: fix hard lockup on granted time slice extension

[Niels Pressel]

In __exit_to_user_mode_loop(), TSE eligibility is checked while
IRQs are enabled. Granting a TSE might involve rearming the
hrtimers. However, hrtimer_rearm_deferred_tif() is expected to be
called with IRQs disabled (see include/linux/hrtimer_rearm.h:17).

Calling the function with IRQs enabled can lead to a hard lockup
because __hrtimer_rearm_deferred() acquires a raw spinlock (without
disabling IRQs) that is also acquired in hard IRQ context within
hrtimer_run_queues().

Lockdep flags the issue when running the rseq selftests on the
7.2-rc5 release:

    WARNING: ./include/linux/hrtimer_rearm.h:17 at irqentry_exit, CPU#1: slice_test

    ================================
    WARNING: inconsistent lock state
    7.2.0-rc5 #1 Tainted: G        W
    --------------------------------
    inconsistent {IN-HARDIRQ-W} -> {HARDIRQ-ON-W} usage.
    slice_test [HC0[0]:SC0[0]:HE1:SE1] takes:
    ffff95b82ec5c698 (hrtimer_bases.lock){?.-.}-{2:2}, at: __hrtimer_rearm_deferred
    {IN-HARDIRQ-W} state was registered at:
    lock_acquire
    _raw_spin_lock_irqsave
    hrtimer_run_queues
    update_process_times
    tick_periodic
    tick_handle_periodic
    timer_interrupt
    __handle_irq_event_percpu
    handle_irq_event_percpu
    handle_irq_event
    handle_level_irq
    __common_interrupt
    common_interrupt
    asm_common_interrupt
    _raw_spin_unlock_irqrestore
    __setup_irq
    request_threaded_irq
    hpet_time_init
    x86_late_time_init
    start_kernel
    x86_64_start_reservations
    x86_64_start_kernel
    common_startup_64

    Possible unsafe locking scenario:

        CPU0
        ----
    lock(hrtimer_bases.lock);
    <Interrupt>
        lock(hrtimer_bases.lock);

                    *** DEADLOCK ***

    Call Trace:
    <TASK>
    dump_stack_lvl
    print_usage_bug
    mark_lock.part.0
    __lock_acquire
    lock_acquire
    _raw_spin_lock
    __hrtimer_rearm_deferred
    irqentry_exit
    asm_sysvec_apic_timer_interrupt
    </TASK>

Originally, the issue was discovered because of intermittent lockups
when heavily using rseq TSEs.

Following the suggestion from Peter Zijlstra, fix this potential lockup
by reflowing __exit_to_user_mode_loop() to only enable IRQs after the
TSE check.

Fixes: 15dd3a948855 ("hrtimer: Push reprogramming timers into the interrupt return path")
Signed-off-by: Niels Pressel <npressel@xxxxxxx>
---
Changes in v2:
- Move to enabling IRQS in __exit_to_user_mode_loop() after the rseq 
  TSE check
- Remove IRQ disable in __rseq_grant_slice_extension()
- Rebased on core/entry
- Link to v1: https://lore.kernel.org/all/20260802124423.51616-1-npressel@xxxxxxx/

 include/linux/rseq_entry.h |  8 ++++----
 kernel/entry/common.c      | 10 +++++-----
 2 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/include/linux/rseq_entry.h b/include/linux/rseq_entry.h
index ed9da6e41a2a..cb41b30c9cb9 100644
--- a/include/linux/rseq_entry.h
+++ b/include/linux/rseq_entry.h
@@ -127,6 +127,8 @@ do {									\
 
 static __always_inline bool __rseq_grant_slice_extension(bool work_pending)
 {
+	lockdep_assert_irqs_disabled();
+
 	struct task_struct *curr = current;
 	struct rseq_slice_ctrl usr_ctrl;
 	union rseq_slice_state state;
@@ -219,10 +221,8 @@ static __always_inline bool __rseq_grant_slice_extension(bool work_pending)
 	 *
 	 * which would be inconsistent state.
 	 */
-	scoped_guard(irq) {
-		clear_tsk_need_resched(curr);
-		clear_preempt_need_resched();
-	}
+	clear_tsk_need_resched(curr);
+	clear_preempt_need_resched();
 	return true;
 
 efault:
diff --git a/kernel/entry/common.c b/kernel/entry/common.c
index e3d381fd3d25..9cefb37f4dd4 100644
--- a/kernel/entry/common.c
+++ b/kernel/entry/common.c
@@ -47,13 +47,13 @@ static __always_inline unsigned long __exit_to_user_mode_loop(struct pt_regs *re
 	 * items have been completed.
 	 */
 	while (ti_work & EXIT_TO_USER_MODE_WORK_LOOP) {
-
+		/* Check rseq slice extensions with IRQs disabled */
+		bool sched = (ti_work & (_TIF_NEED_RESCHED | _TIF_NEED_RESCHED_LAZY)) &&
+			      !rseq_grant_slice_extension(ti_work, TIF_SLICE_EXT_DENY);
 		local_irq_enable();
 
-		if (ti_work & (_TIF_NEED_RESCHED | _TIF_NEED_RESCHED_LAZY)) {
-			if (!rseq_grant_slice_extension(ti_work, TIF_SLICE_EXT_DENY))
-				schedule();
-		}
+		if (sched)
+			schedule();
 
 		if (ti_work & _TIF_UPROBE)
 			uprobe_notify_resume(regs);

base-commit: 05c033db7e9ad3c34f6968ec568cb6ee01051c57
-- 
2.50.1 (Apple Git-155)