Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

From: Waiman Long
Date: Thu Sep 22 2016 - 17:15:47 EST

On 09/22/2016 04:26 PM, Thomas Gleixner wrote:
On Thu, 22 Sep 2016, Waiman Long wrote:
On 09/22/2016 09:34 AM, Thomas Gleixner wrote:
On Thu, 22 Sep 2016, Peter Zijlstra wrote:
I'd leave out the TO part entirely (or only mention it in changelogs).

That is, I'd call the futex ops: FUTEX_LOCK and FUTEX_UNLOCK.
That brings me to a different question:

How is user space going to support this, i.e. is this some extra magic for
code which implements its own locking primitives or is there going to be a
wide use via e.g. glibc.
There are some applications that use futex(2) directly to implement their
synchronization primitives. For those applications, they will need to modify
their code to detect the presence of the new futexes. They can then use the
new futexes if available and use wait-wake futexes if not.
That's what I suspected. Did you talk to the other folks who complain about
futex performance (database, JVM, etc.) and play their own games with user
space spinlocks and whatever?

I am also part of the team that help large application vendors to tune their application performance on our large SMP systems. Those application vendors tend to use futex directly instead of relying on glibc. We had seen spinlock contention in the futex could sometimes be a significant portion of the CPU cycles consumed depending on the workloads that were being run. We had been providing suggestions on the best practice of how to use futexes. But there is only so much you can do with tuning their locking code implementation. That is why I am also looking for way to improve the performance of the futex code in the kernel.

I am also planning to take a look at the pthread_mutex* APIs to see if they
can be modified to use the new futexes later on when the patch becomes more
Please involve glibc people who are interested in the futex stuff early and
discuss the concept before it's set in stone for your particular usecase.

Sure, I will start to do some prototyping and performance testing with glibc and then engage those folks about that.

Also what's the reason that we can't do probabilistic spinning for
FUTEX_WAIT and have to add yet another specialized variant of futexes?

The main reason is that a FUTEX_WAIT waiter has no idea who the owner of the
futex is. We usually do spinning when the lock owner is running and abort when
it goes to sleep. We can't do that for FUTEX_WAIT.
Fair enough. This wants to be spelled out in the changelog and explained a
bit more detailed.

I will.