Re: [PATCH 1/5] locking/rtmutex: Stop spinning on the owner when the waiter is woken

[Mathieu Desnoyers]

On 2026-11-07 05:43, Peter Zijlstra wrote:
> On Sat, Nov 07, 2026 at 12:29:27AM -0500, Steven Rostedt wrote:
> > On Fri,  6 Nov 2026 18:59:27 -0500
> > Mathieu Desnoyers <mathieu.desnoyers@xxxxxxxxxxxx> wrote:
> >
> > > The top waiter of an rt_mutex spins while the owner runs,
> > > rtmutex_spin_on_owner(), and does not stop when it is woken: its timeout
> > > and a signal find it running, set it TASK_RUNNING, and the spin goes on
> > > until the owner leaves its CPU or the waiter's own CPU has to
> > > reschedule. A PI futex owner is a user task that may run as long as it
> > > likes. On v7.4-rc1, in 8-CPU VMs, behind an owner spinning in userspace,
> > > a 1ms pthread_mutex_timedlock() returns ETIMEDOUT 126ms to 631ms late on
> > > average, 1.4s at most, a SIGUSR1 is handled up to 1.9s late, and a
> > > waiter killed with SIGKILL is reaped 0.5s late on average. With
> > > PREEMPT_RT the wound of a ww_mutex is missed the same way: two contexts
> > > in an ABBA on two CPUs at SCHED_FIFO spin on each other, and the older
> > > one gets its second lock after 96ms on average, 0.95s at most.
> >
> > The above is very verbose. Are you starting to use AI? ;-)

I definitely am, as stated in the cover letter. :-)

> He definitely is. I can't read the above, its word soup. Part of using
> AI is making sure the Changelog is actually readable. This is not.

Feedback on the commit message wording is welcome, I'll try to improve
things for next iterations. Is the following easier to parse ? I fed
Peter's and my own criticism to Claude and got it to rewrite the entire
thing (and keep those learnings for next time):

locking/rtmutex: Stop spinning on the owner when the waiter is woken

The top waiter of an rt_mutex sets its sleep state. Then it spins in
rtmutex_spin_on_owner() as long as the owner is running. A wakeup sets
the waiter back to TASK_RUNNING. The spin loop does not check for this.
So a timeout or a signal is only handled when the owner blocks, or when
the waiter has to reschedule:

  CPU 0 (waiter)                            CPU 1 (owner)

                                            pthread_mutex_lock()
                                              takes the PI futex
                                            spins in userspace
  pthread_mutex_timedlock(1ms)
    futex_lock_pi()
      rt_mutex_wait_proxy_lock()
        set_current_state(TASK_INTERRUPTIBLE)
        rt_mutex_slowlock_block()
          rtmutex_spin_on_owner()
            <timeout> hrtimer_wakeup()
              wake_up_process(waiter)
                __state = TASK_RUNNING
            owner_on_cpu(owner): true
            need_resched(): false
            keeps spinning                  still spins in userspace

The owner of a PI futex is a user task. It can run for a long time.
With an owner spinning in userspace, a 1ms pthread_mutex_timedlock()
returns up to 1.4s late. A signal to the waiter is delayed in the same
way.

With PREEMPT_RT, a ww_mutex wound is missed too. Two contexts take two
ww_mutexes in opposite order, on two CPUs. They spin on each other for
up to a second.

Fix this by also stopping the spin when the waiter is TASK_RUNNING. The
slowpath then handles the timeout, the signal or the wound, as it does
after schedule().

rtlock waiters are not affected. Only the lock handoff wakes them from
TASK_RTLOCK_WAIT.

With this patch, the 1ms timed lock returns after about 1ms. The
ww_mutex ABBA resolves in microseconds. PI mutex throughput is
unchanged. The futex and ww_mutex selftests added later in this series
cover both cases.

Fixes: 992caf7f1724 ("locking/rtmutex: Add adaptive spinwait mechanism")
Cc: stable@xxxxxxxxxxxxxxx
Reviewed-by: Steven Rostedt <rostedt@xxxxxxxxxxx>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@xxxxxxxxxxxx>

--
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com