[PATCH] rseq: defer time slice extension yield for sys_futex_wake

[Alice Ryhl]

When a task is granted an rseq scheduler time slice extension, it is
expected to finish its critical section and relinquish the CPU via
rseq_slice_yield(2). If the task issues any other system call while a
grant is active, rseq_syscall_enter_work() forces an immediate
reschedule on syscall entry via cond_resched(). This may cause
significant latency penalty for userspace lock implementations that use
rseq time slice extensions when unlocking the futex.

In a userspace mutex unlock sequence:
1. The lock is released in userspace.
2. If there are waiters, the unlocking thread calls sys_futex_wake()
   to wake a sleeping waiter.

Because sys_futex_wake() is currently treated as an arbitrary syscall,
rseq_syscall_enter_work() schedules out the unlocking thread upon
syscall entry, which is before it has executed the wakeup. Consequently,
the lock is free in userspace, but the waiter remains blocked in the
kernel while the CPU switches to an unrelated task. The waiter is only
woken when the unlocking thread is eventually scheduled back in to
finish the syscall, causing lock handoff delays.

Thus, update rseq_syscall_enter_work() for sys_futex_wake() so that it
does not reschedule during syscall entry. The thread will yield the CPU
on the syscall exit path instead.

There is no need to apply this optimization to the multiplexed futex()
syscall since any userspace code that can invoke rseq_slice_yield() can
also invoke futex_wake().

This patch was verified via ftrace that when sys_futex_wake() is invoked
during an active slice grant, the wakeup (sched_waking) occurs before
any context switch:
1. sys_enter_futex_wake
2. sched_waking (wakes waiting thread)
3. sys_exit_futex_wake -> 0x1
4. sched_switch (on syscall exit due to TIF_NEED_RESCHED)

Assisted-by: LLM
Signed-off-by: Alice Ryhl <aliceryhl@xxxxxxxxxx>
---
 Documentation/userspace-api/rseq.rst | 17 +++++++++++++----
 kernel/rseq.c                        | 16 +++++++++++++---
 2 files changed, 26 insertions(+), 7 deletions(-)

diff --git a/Documentation/userspace-api/rseq.rst b/Documentation/userspace-api/rseq.rst
index 8549a6c61531..30bdb1885f67 100644
--- a/Documentation/userspace-api/rseq.rst
+++ b/Documentation/userspace-api/rseq.rst
@@ -217,10 +217,10 @@ operation.
 
 If the thread issues a syscall other than rseq_slice_yield(2) within the
 granted timeslice extension, the grant is also revoked and the CPU is
-relinquished immediately when entering the kernel. This is required as
-syscalls might consume arbitrary CPU time until they reach a scheduling
-point when the preemption model is either NONE or VOLUNTARY and therefore
-might exceed the grant by far.
+relinquished. For most syscalls, this occurs immediately when entering the
+kernel. This is required as syscalls might consume arbitrary CPU time until
+they reach a scheduling point when the preemption model is either NONE or
+VOLUNTARY and therefore might exceed the grant by far.
 
 The preferred solution for user space is to use rseq_slice_yield(2) which
 is side effect free. The support for arbitrary syscalls is required to
@@ -228,5 +228,14 @@ support onion layer architectured applications, where the code handling the
 critical section and requesting the time slice extension has no control
 over the code within the critical section.
 
+For futex_wake(2), the CPU is instead relinquished when returning to userspace,
+so that it gets a chance to wake any waiting tasks before yielding the CPU.
+This makes it possible to terminate the critical region of a userspace mutex
+using rseq and futexes with futex_wake(2) instead of rseq_slice_yield(2).
+Currently, this is the only syscall that does not relinquish the CPU
+immediately on syscall entry. Note in particular that this applies only to the
+dedicated futex_wake(2) syscall, and not to the futex(2) syscall even when
+using op=FUTEX_WAKE.
+
 The kernel enforces flag consistency and terminates the thread with SIGSEGV
 if it detects a violation.
diff --git a/kernel/rseq.c b/kernel/rseq.c
index e75e3a5e312c..81bc2ed997f0 100644
--- a/kernel/rseq.c
+++ b/kernel/rseq.c
@@ -723,9 +723,18 @@ void rseq_syscall_enter_work(long syscall)
 		 * the task was already rescheduled before arriving here.
 		 */
 		if (!curr->rseq.event.sched_switch) {
-			rseq_slice_set_need_resched(curr);
+			if (syscall == __NR_futex_wake) {
+				/*
+				 * For this syscall, reschedule on syscall exit
+				 * instead of syscall entry to avoid delaying
+				 * the wakeup.
+				 */
+				set_tsk_need_resched(curr);
+			} else {
+				rseq_slice_set_need_resched(curr);
+			}
 
-			if (syscall == __NR_rseq_slice_yield) {
+			if (syscall == __NR_rseq_slice_yield || syscall == __NR_futex_wake) {
 				rseq_stat_inc(rseq_stats.s_yielded);
 				/* Update the yielded state for syscall return */
 				curr->rseq.slice.yielded = 1;
@@ -735,7 +744,8 @@ void rseq_syscall_enter_work(long syscall)
 		}
 	}
 	/* Reschedule on NONE/VOLUNTARY preemption models */
-	cond_resched();
+	if (syscall != __NR_futex_wake)
+		cond_resched();
 
 	/* Clear the grant in kernel state and user space */
 	curr->rseq.slice.state.granted = false;

---
base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
change-id: 20260831-sys-futex-wake-time-slice-c36738bf7b94

Best regards,
-- 
Alice Ryhl <aliceryhl@xxxxxxxxxx>