Re: [PATCH] futex: Avoid hash-bucket locking for mismatched waits

[Dmitry Ilvokhin]

On Fri, Aug 14, 2026 at 06:02:45PM +0200, Thomas Gleixner wrote:
> On Fri, Aug 14 2026 at 18:01, Thomas Gleixner wrote:
> > On Mon, Aug 10 2026 at 13:17, Usama Arif wrote:
> >> On 07/08/2026 16:42, Thomas Gleixner wrote:
> >>> On Wed, Aug 05 2026 at 06:28, Usama Arif wrote:
> >>>> On Tue, 4 Aug 2026 17:07:59 +0000 Dmitry Ilvokhin <d@xxxxxxxxxxxx> wrote:
> >>>> The above data shows the significance of the patch.
> >>>> It provides a very meaningful improvement (22.4% of time spent in futex_q_lock()
> >>>> will be significantly optimized and will also deliver second-order effects)
> >>>> and has no measurable impact on latency in the matching path.
> >>>> IMHO, this patch is a free lunch.
> >>> 
> >>> Not really free. The user space access is not exactly cheap either
> >>> because CLAC/STAC are memory fencing to meet the SMAP guarantees.
> >>
> >> My understanding from 86e6b1547b3d is that STAC/CLAC “end up serializing
> >> execution on older Zen,” while Zen 5’s AC renaming “improves performance
> >> of STAC/CLAC a lot a lot.”  Architecturally, they only change the AC bit.
> >> They are not memory-ordering instructions like LFENCE.
> >
> > It's not a memory ordering instruction, but it has to guarantee that the
> > AC change is effective when the subsequent permission check
> > happens. That's true for both STAC and CLAC.
> >
> > So it _cannot_ be free by definition and the penalty depends on the
> > micro architecture.
> >
> >> I am currently testing on Zen5 which could be why I didn't see any
> >> wall-time regression in futex_wait_timeout.c from [1].
> >
> > It's not relevant whether your ZEN5 works fine or not. We are not
> > optimizing for a particular machine.
> >
> > A trivial futex bouncing test case with two threads degrades on a ZEN3
> > by ~20% and when looking at it with perf top clearly the extra user
> > access stands out very prominently.
> >
> > The below variant does not expose that behavior and actually improves
> > the same test case by ~5% on that machine.
> 
> Bah. Included the broken version. Fixed one is below.
> 
> Thanks,
> 
>         tglx
> ---
>  kernel/futex/waitwake.c |   16 +++++++++++++++-
>  1 file changed, 15 insertions(+), 1 deletion(-)
> 
> --- a/kernel/futex/waitwake.c
> +++ b/kernel/futex/waitwake.c
> @@ -857,7 +857,21 @@ int futex_wait_setup(u32 __user *uaddr,
>  		CLASS(hbr, hbr)(&q->key);
>  		auto hb = hbr.hb;
>  
> -		futex_q_lock(q, hb);
> +		futex_hb_waiters_inc(hb);
> +		q->lock_ptr = &hb->lock;
> +
> +		if (!spin_trylock(&hb->lock)) {
> +			ret = get_user_inline(uval, uaddr);
> +			if (ret) {
> +				futex_hb_waiters_dec(hb);
> +				return ret;
> +			}
> +			if (uval != val) {
> +				futex_hb_waiters_dec(hb);
> +				return -EWOULDBLOCK;
> +			}
> +			spin_lock(&hb->lock);
> +		}
>  
>  		ret = futex_get_value_locked(&uval, uaddr);
>

I looked at production data to understand better where
futex_wait_setup() calls are coming from. Majority of the cost is
contended userspace mutex hammering the same futex word from different
threads, where amount of threads differ from usecase to usecase.

The userspace mutex implementation is pthread_mutex_t from glibc, which
has only three states: free (0), locked (1) and locked with waiters (2).
When critcal section is short, mutex releaser and winner acquirer switch
the state from 2 to 0 and 1 and this produces high futex value missmatch
rate. The higher contention is, the higher futex value missmatch rate
is.

I've built a benchmark to simulate this behaviour on the smaller scale
and run it with two threads and `nproc` threads to simulate low and high
contention cases. The benchmark is a bit on a extreme side, but I think
it approximates the real world case quite well.

The numbers below are averaged across 10 runs.

SKYLAKE (2 NUMA NODES, 80 CPUS, Intel(R) Xeon(R) Gold 6138 CPU)

Threads   Baseline (ops/sec)    Patched            95% CI        Diff
-----------------------------------------------------------------------
2           5,917,777          6,441,061     [ -1.98%, +19.67%]   +8.84%
80            122,926             77,801     [-41.17%, -32.25%]  -36.71%


BERGAMO (1 NUMA NODE, 176 CPUS, AMD EPYC 9D64)

Threads   Baseline (ops/sec)    Patched            95% CI        Diff
-----------------------------------------------------------------------
2            14,952,962        14,482,494     [ -4.62%, -1.68%]   -3.15%
176             161,294           127,563     [-21.72%, -20.11%] -20.91%


Skylake two thread case seems in line with ~5% improvement you measured,
but numbers are quite noisy. Other runs do not look encouraging.

It looks like early bail out amplifies contention problem even more.
Instead of letting thread spin on the hb->lock, early check returns just
for userspace to retry again moment later. Ideally, it would be better
to park spinning thread as fast as possible and with early bail out we
do opposite of that.

Anyway, I would be curious to know what do you think about it. Maybe
there is a better way to measure this optimization.