[RFC patch 7/7] [PATCH] glibc: nptl: Add support for attached pthread_mutexes

From: Thomas Gleixner
Date: Sat Apr 02 2016 - 07:11:54 EST

Next message: Thomas Gleixner: "[RFC patch 2/7] futex: Add some more function commentry"
Previous message: Thomas Gleixner: "[RFC patch 4/7] futex: Add support for attached futexes"
In reply to: Carlos O'Donell: "Re: [RFC patch 4/7] futex: Add support for attached futexes"
Next in thread: Peter Zijlstra: "Re: [RFC patch 7/7] [PATCH] glibc: nptl: Add support for attached pthread_mutexes"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

pthread_mutexes on Linux are based on the futex mechanism. The standard futex
mechanism in the Linux kernel uses a global hash to store transient
state. Collisions on that hash can lead to performance degradation and on
real-time enabled kernels even to priority inversions.

To guarantee futexes without collisions on the global kernel hash, the kernel
provides a mechanism to attach to a futex. This creates futex private state
which avoids hash collisions and on NUMA systems also cross node memory
access.

To utilize this mechanism each thread has to attach to the futex before any
other operations on that futex which involve kernel interaction.

At pthread_mutex_init() the pthread_mutex attribute needs to be initialized
for attached mode via:

pthread_mutexattr_setattached_np(&attr, 1);

All threads which are using the mutex - including the one which called
pthread_mutex_init() - must invoke

pthread_mutex_attach_np(&mutex);

before any other pthread_mutex related operations.

Example:
pthread_mutexattr_t attr;
pthread_mutex_t lock;

pthread_mutexattr_init(&attr);
pthread_mutexattr_setattached_np(&attr, 1);
pthread_mutex_init(&lock, &attr);

pthread_mutex_attach_np(&lock);

pthread_mutex_lock(&lock);

In ptrace this should look like this:

futex(<addr>, 0x280 /* FUTEX_??? */, 1, NULL <unfinished ...>

0x280: FUTEX_ATTACHED | FUTEX_PRIVATE | FUTEX_WAIT

To undo the attachment each involved thread needs to call

pthread_mutex_detach_np(&mutex);

When the last user detaches the kernel state is destroyed.

Attached mode works with all futex operations. For operations which involve
two futexes, i.e. FUTEX_REQUEUE_* both futexes have to be either attached or
detached (similar to FUTEX_PRIVATE). This excludes pthread_condvars for now,
but it will be implemented once the details of this new mechanism are agreed
on.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@xxxxxxxxxxxxx>
Signed-off-by: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
---
nptl/Makefile | 3 ++
nptl/Versions | 7 +++++
nptl/pthreadP.h | 15 ++++++++++
nptl/pthread_cond_destroy.c | 2 +-
nptl/pthread_mutex_attach_np.c | 35 ++++++++++++++++++++++
nptl/pthread_mutex_cond_lock.c | 6 ++--
nptl/pthread_mutex_destroy.c | 3 ++
nptl/pthread_mutex_detach_np.c | 35 ++++++++++++++++++++++
nptl/pthread_mutex_init.c | 3 ++
nptl/pthread_mutex_lock.c | 18 +++++------
nptl/pthread_mutex_setprioceiling.c | 4 +--
nptl/pthread_mutex_timedlock.c | 20 ++++++-------
nptl/pthread_mutex_trylock.c | 8 ++---
nptl/pthread_mutex_unlock.c | 12 ++++----
nptl/pthread_mutexattr_getattached_np.c | 15 ++++++++++
nptl/pthread_mutexattr_setattached_np.c | 17 +++++++++++
sysdeps/nptl/pthread.h | 19 ++++++++++++
sysdeps/unix/sysv/linux/i386/libpthread.abilist | 5 ++++
sysdeps/unix/sysv/linux/lowlevellock-futex.h | 9 ++++--
.../unix/sysv/linux/x86_64/64/libpthread.abilist | 5 ++++
.../unix/sysv/linux/x86_64/x32/libpthread.abilist | 5 ++++
21 files changed, 208 insertions(+), 38 deletions(-)
create mode 100644 nptl/pthread_mutex_attach_np.c
create mode 100644 nptl/pthread_mutex_detach_np.c
create mode 100644 nptl/pthread_mutexattr_getattached_np.c
create mode 100644 nptl/pthread_mutexattr_setattached_np.c

diff --git a/nptl/Makefile b/nptl/Makefile
index dc3ccab991ad..d4944f905c5b 100644
--- a/nptl/Makefile
+++ b/nptl/Makefile
@@ -59,6 +59,9 @@ libpthread-routines = nptl-init vars events version pt-interp \
pthread_mutexattr_init pthread_mutexattr_destroy \
pthread_mutexattr_getpshared \
pthread_mutexattr_setpshared \
+ pthread_mutexattr_setattached_np \
+ pthread_mutexattr_getattached_np \
+ pthread_mutex_attach_np pthread_mutex_detach_np \
pthread_mutexattr_gettype pthread_mutexattr_settype \
pthread_rwlock_init pthread_rwlock_destroy \
pthread_rwlock_rdlock pthread_rwlock_timedrdlock \
diff --git a/nptl/Versions b/nptl/Versions
index 0ae5def464b9..06981b7ee6ca 100644
--- a/nptl/Versions
+++ b/nptl/Versions
@@ -265,6 +265,13 @@ libpthread {
GLIBC_2.22 {
}

+ GLIBC_2.23 {
+ pthread_mutexattr_getattached_np;
+ pthread_mutexattr_setattached_np;
+ pthread_mutex_attach_np;
+ pthread_mutex_detach_np;
+ }
+
GLIBC_PRIVATE {
__pthread_initialize_minimal;
__pthread_clock_gettime; __pthread_clock_settime;
diff --git a/nptl/pthreadP.h b/nptl/pthreadP.h
index 4edc74b1ef54..b7656c292ecb 100644
--- a/nptl/pthreadP.h
+++ b/nptl/pthreadP.h
@@ -109,6 +109,7 @@ enum
PTHREAD_MUTEX_TIMED_NP | PTHREAD_MUTEX_NO_ELISION_NP,
};
#define PTHREAD_MUTEX_PSHARED_BIT 128
+#define PTHREAD_MUTEX_ATTACHED_BIT 1024

#define PTHREAD_MUTEX_TYPE(m) \
((m)->__data.__kind & 127)
@@ -125,10 +126,19 @@ enum
(((m)->__data.__kind & 128) ? LLL_SHARED : LLL_PRIVATE)
#endif

+#define PTHREAD_MUTEX_PATTACHED(m) \
+ (((m)->__data.__kind & PTHREAD_MUTEX_ATTACHED_BIT) ? FUTEX_ATTACHED : 0)
+
+#define PTHREAD_MUTEX_PSHARED_PATTACHED(m) \
+ (PTHREAD_MUTEX_PSHARED(m) | PTHREAD_MUTEX_PATTACHED(m))
+
/* The kernel when waking robust mutexes on exit never uses
FUTEX_PRIVATE_FLAG FUTEX_WAKE. */
#define PTHREAD_ROBUST_MUTEX_PSHARED(m) LLL_SHARED

+#define PTHREAD_ROBUST_MUTEX_PSHARED_PATTACHED(m) \
+ (PTHREAD_MUTEX_PATTACHED(m) | LLL_SHARED)
+
/* Ceiling in __data.__lock. __data.__lock is signed, so don't
use the MSB bit in there, but in the mask also include that bit,
so that the compiler can optimize & PTHREAD_MUTEX_PRIO_CEILING_MASK
@@ -139,6 +149,7 @@ enum

/* Flags in mutex attr. */
+#define PTHREAD_MUTEXATTR_FLAG_ATTACHED 0x08000000
#define PTHREAD_MUTEXATTR_PROTOCOL_SHIFT 28
#define PTHREAD_MUTEXATTR_PROTOCOL_MASK 0x30000000
#define PTHREAD_MUTEXATTR_PRIO_CEILING_SHIFT 12
@@ -422,6 +433,8 @@ extern int __pthread_mutex_unlock (pthread_mutex_t *__mutex);
extern int __pthread_mutex_unlock_usercnt (pthread_mutex_t *__mutex,
int __decr)
attribute_hidden internal_function;
+extern int __pthread_mutex_attach_np (pthread_mutex_t *__mutex);
+extern int __pthread_mutex_detach_np (pthread_mutex_t *__mutex);
extern int __pthread_mutexattr_init (pthread_mutexattr_t *attr);
extern int __pthread_mutexattr_destroy (pthread_mutexattr_t *attr);
extern int __pthread_mutexattr_settype (pthread_mutexattr_t *attr, int kind);
@@ -497,6 +510,8 @@ hidden_proto (__pthread_mutex_init)
hidden_proto (__pthread_mutex_destroy)
hidden_proto (__pthread_mutex_lock)
hidden_proto (__pthread_mutex_unlock)
+hidden_proto (__pthread_mutex_attach_np)
+hidden_proto (__pthread_mutex_detach_np)
hidden_proto (__pthread_rwlock_rdlock)
hidden_proto (__pthread_rwlock_wrlock)
hidden_proto (__pthread_rwlock_unlock)
diff --git a/nptl/pthread_cond_destroy.c b/nptl/pthread_cond_destroy.c
index 1acd8042d841..3f7ae9fb36db 100644
--- a/nptl/pthread_cond_destroy.c
+++ b/nptl/pthread_cond_destroy.c
@@ -64,7 +64,7 @@ __pthread_cond_destroy (pthread_cond_t *cond)
{
pthread_mutex_t *mut = (pthread_mutex_t *) cond->__data.__mutex;
lll_futex_wake (&mut->__data.__lock, INT_MAX,
- PTHREAD_MUTEX_PSHARED (mut));
+ PTHREAD_MUTEX_PSHARED_PATTACHED (mut));
}

do
diff --git a/nptl/pthread_mutex_attach_np.c b/nptl/pthread_mutex_attach_np.c
new file mode 100644
index 000000000000..1038f898ac37
--- /dev/null
+++ b/nptl/pthread_mutex_attach_np.c
@@ -0,0 +1,35 @@
+#include <errno.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <sys/param.h>
+#include "pthreadP.h"
+#include <lowlevellock.h>
+#include <stap-probe.h>
+
+int __pthread_mutex_attach_np (pthread_mutex_t *mutex)
+{
+ int ret;
+ int private = FUTEX_ATTACHED;
+ INTERNAL_SYSCALL_DECL (__err);
+
+ if (!(mutex->__data.__kind & PTHREAD_MUTEX_ATTACHED_BIT))
+ return -EINVAL;
+
+ if (mutex->__data.__kind & PTHREAD_MUTEX_ROBUST_NORMAL_NP)
+ private |= LLL_SHARED;
+ else if (mutex->__data.__kind & PTHREAD_MUTEX_PSHARED_BIT)
+ private |= LLL_SHARED;
+
+ ret = INTERNAL_SYSCALL (futex, __err, 4, &mutex->__data.__lock,
+ __lll_private_flag(FUTEX_ATTACH,
+ private), 0, 0);
+ if (INTERNAL_SYSCALL_ERROR_P (ret, __err))
+ return INTERNAL_SYSCALL_ERRNO (ret, __err);
+
+ return ret;
+}
+
+#ifndef __pthread_mutex_lock
+strong_alias (__pthread_mutex_attach_np, pthread_mutex_attach_np)
+hidden_def (__pthread_mutex_attach_np)
+#endif
diff --git a/nptl/pthread_mutex_cond_lock.c b/nptl/pthread_mutex_cond_lock.c
index 2ac421fd63af..fd1932088996 100644
--- a/nptl/pthread_mutex_cond_lock.c
+++ b/nptl/pthread_mutex_cond_lock.c
@@ -1,11 +1,11 @@
#include <pthreadP.h>

#define LLL_MUTEX_LOCK(mutex) \
- lll_cond_lock ((mutex)->__data.__lock, PTHREAD_MUTEX_PSHARED (mutex))
+ lll_cond_lock ((mutex)->__data.__lock, PTHREAD_MUTEX_PSHARED_PATTACHED (mutex))

/* Not actually elided so far. Needed? */
#define LLL_MUTEX_LOCK_ELISION(mutex) \
- ({ lll_cond_lock ((mutex)->__data.__lock, PTHREAD_MUTEX_PSHARED (mutex)); 0; })
+ ({ lll_cond_lock ((mutex)->__data.__lock, PTHREAD_MUTEX_PSHARED_PATTACHED (mutex)); 0; })

#define LLL_MUTEX_TRYLOCK(mutex) \
lll_cond_trylock ((mutex)->__data.__lock)
@@ -13,7 +13,7 @@

#define LLL_ROBUST_MUTEX_LOCK(mutex, id) \
lll_robust_cond_lock ((mutex)->__data.__lock, id, \
- PTHREAD_ROBUST_MUTEX_PSHARED (mutex))
+ PTHREAD_ROBUST_MUTEX_PSHARED_PATTACHED (mutex))
#define __pthread_mutex_lock internal_function __pthread_mutex_cond_lock
#define __pthread_mutex_lock_full __pthread_mutex_cond_lock_full
#define NO_INCR
diff --git a/nptl/pthread_mutex_destroy.c b/nptl/pthread_mutex_destroy.c
index 8a57d49272c1..99550852c2f2 100644
--- a/nptl/pthread_mutex_destroy.c
+++ b/nptl/pthread_mutex_destroy.c
@@ -31,6 +31,9 @@ __pthread_mutex_destroy (pthread_mutex_t *mutex)
&& mutex->__data.__nusers != 0)
return EBUSY;

+ if (mutex->__data.__kind & PTHREAD_MUTEX_ATTACHED_BIT)
+ return __pthread_mutex_detach_np(mutex);
+
/* Set to an invalid value. */
mutex->__data.__kind = -1;

diff --git a/nptl/pthread_mutex_detach_np.c b/nptl/pthread_mutex_detach_np.c
new file mode 100644
index 000000000000..4c4f8fd1c9af
--- /dev/null
+++ b/nptl/pthread_mutex_detach_np.c
@@ -0,0 +1,35 @@
+#include <errno.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <sys/param.h>
+#include "pthreadP.h"
+#include <lowlevellock.h>
+#include <stap-probe.h>
+
+int __pthread_mutex_detach_np (pthread_mutex_t *mutex)
+{
+ int ret;
+ int private = FUTEX_ATTACHED;
+ INTERNAL_SYSCALL_DECL (__err);
+
+ if (!(mutex->__data.__kind & PTHREAD_MUTEX_ATTACHED_BIT))
+ return -EINVAL;
+
+ if (mutex->__data.__kind & PTHREAD_MUTEX_ROBUST_NORMAL_NP)
+ private |= LLL_SHARED;
+ else if (mutex->__data.__kind & PTHREAD_MUTEX_PSHARED_BIT)
+ private |= LLL_SHARED;
+
+ ret = INTERNAL_SYSCALL (futex, __err, 4, &mutex->__data.__lock,
+ __lll_private_flag(FUTEX_DETACH,
+ private), 0, 0);
+ if (INTERNAL_SYSCALL_ERROR_P (ret, __err))
+ return INTERNAL_SYSCALL_ERRNO (ret, __err);
+
+ return ret;
+}
+
+#ifndef __pthread_mutex_lock
+strong_alias (__pthread_mutex_detach_np, pthread_mutex_detach_np)
+hidden_def (__pthread_mutex_detach_np)
+#endif
diff --git a/nptl/pthread_mutex_init.c b/nptl/pthread_mutex_init.c
index 71ac7bc7f3df..6f6344fcc805 100644
--- a/nptl/pthread_mutex_init.c
+++ b/nptl/pthread_mutex_init.c
@@ -136,6 +136,9 @@ __pthread_mutex_init (pthread_mutex_t *mutex,
| PTHREAD_MUTEXATTR_FLAG_ROBUST)) != 0)
mutex->__data.__kind |= PTHREAD_MUTEX_PSHARED_BIT;

+ if (imutexattr->mutexkind & PTHREAD_MUTEXATTR_FLAG_ATTACHED)
+ mutex->__data.__kind |= PTHREAD_MUTEX_ATTACHED_BIT;
+
/* Default values: mutex not used yet. */
// mutex->__count = 0; already done by memset
// mutex->__owner = 0; already done by memset
diff --git a/nptl/pthread_mutex_lock.c b/nptl/pthread_mutex_lock.c
index bdfa529f639b..41f18caf5ba8 100644
--- a/nptl/pthread_mutex_lock.c
+++ b/nptl/pthread_mutex_lock.c
@@ -38,18 +38,18 @@

#ifndef LLL_MUTEX_LOCK
# define LLL_MUTEX_LOCK(mutex) \
- lll_lock ((mutex)->__data.__lock, PTHREAD_MUTEX_PSHARED (mutex))
+ lll_lock ((mutex)->__data.__lock, PTHREAD_MUTEX_PSHARED_PATTACHED (mutex))
# define LLL_MUTEX_TRYLOCK(mutex) \
lll_trylock ((mutex)->__data.__lock)
# define LLL_ROBUST_MUTEX_LOCK(mutex, id) \
lll_robust_lock ((mutex)->__data.__lock, id, \
- PTHREAD_ROBUST_MUTEX_PSHARED (mutex))
+ PTHREAD_ROBUST_MUTEX_PSHARED_PATTACHED (mutex))
# define LLL_MUTEX_LOCK_ELISION(mutex) \
lll_lock_elision ((mutex)->__data.__lock, (mutex)->__data.__elision, \
- PTHREAD_MUTEX_PSHARED (mutex))
+ PTHREAD_MUTEX_PSHARED_PATTACHED (mutex))
# define LLL_MUTEX_TRYLOCK_ELISION(mutex) \
lll_trylock_elision((mutex)->__data.__lock, (mutex)->__data.__elision, \
- PTHREAD_MUTEX_PSHARED (mutex))
+ PTHREAD_MUTEX_PSHARED_PATTACHED (mutex))
#endif

#ifndef FORCE_ELISION
@@ -261,7 +261,7 @@ __pthread_mutex_lock_full (pthread_mutex_t *mutex)
/* This mutex is now not recoverable. */
mutex->__data.__count = 0;
lll_unlock (mutex->__data.__lock,
- PTHREAD_ROBUST_MUTEX_PSHARED (mutex));
+ PTHREAD_ROBUST_MUTEX_PSHARED_PATTACHED (mutex));
THREAD_SETMEM (THREAD_SELF, robust_head.list_op_pending, NULL);
return ENOTRECOVERABLE;
}
@@ -333,8 +333,8 @@ __pthread_mutex_lock_full (pthread_mutex_t *mutex)
/* The mutex is locked. The kernel will now take care of
everything. */
int private = (robust
- ? PTHREAD_ROBUST_MUTEX_PSHARED (mutex)
- : PTHREAD_MUTEX_PSHARED (mutex));
+ ? PTHREAD_ROBUST_MUTEX_PSHARED_PATTACHED (mutex)
+ : PTHREAD_MUTEX_PSHARED_PATTACHED (mutex));
INTERNAL_SYSCALL_DECL (__err);
int e = INTERNAL_SYSCALL (futex, __err, 4, &mutex->__data.__lock,
__lll_private_flag (FUTEX_LOCK_PI,
@@ -395,7 +395,7 @@ __pthread_mutex_lock_full (pthread_mutex_t *mutex)
INTERNAL_SYSCALL_DECL (__err);
INTERNAL_SYSCALL (futex, __err, 4, &mutex->__data.__lock,
__lll_private_flag (FUTEX_UNLOCK_PI,
- PTHREAD_ROBUST_MUTEX_PSHARED (mutex)),
+ PTHREAD_ROBUST_MUTEX_PSHARED_PATTACHED (mutex)),
0, 0);

THREAD_SETMEM (THREAD_SELF, robust_head.list_op_pending, NULL);
@@ -484,7 +484,7 @@ __pthread_mutex_lock_full (pthread_mutex_t *mutex)

if (oldval != ceilval)
lll_futex_wait (&mutex->__data.__lock, ceilval | 2,
- PTHREAD_MUTEX_PSHARED (mutex));
+ PTHREAD_MUTEX_PSHARED_PATTACHED (mutex));
}
while (atomic_compare_and_exchange_val_acq (&mutex->__data.__lock,
ceilval | 2, ceilval)
diff --git a/nptl/pthread_mutex_setprioceiling.c b/nptl/pthread_mutex_setprioceiling.c
index 6b98b5d046c9..55d4d2276f56 100644
--- a/nptl/pthread_mutex_setprioceiling.c
+++ b/nptl/pthread_mutex_setprioceiling.c
@@ -84,7 +84,7 @@ pthread_mutex_setprioceiling (pthread_mutex_t *mutex, int prioceiling,

if (oldval != ceilval)
lll_futex_wait (&mutex->__data.__lock, ceilval | 2,
- PTHREAD_MUTEX_PSHARED (mutex));
+ PTHREAD_MUTEX_PSHARED_PATTACHED (mutex));
}
while (atomic_compare_and_exchange_val_acq (&mutex->__data.__lock,
ceilval | 2, ceilval)
@@ -115,7 +115,7 @@ pthread_mutex_setprioceiling (pthread_mutex_t *mutex, int prioceiling,
atomic_full_barrier ();

lll_futex_wake (&mutex->__data.__lock, INT_MAX,
- PTHREAD_MUTEX_PSHARED (mutex));
+ PTHREAD_MUTEX_PSHARED_PATTACHED (mutex));

return 0;
}
diff --git a/nptl/pthread_mutex_timedlock.c b/nptl/pthread_mutex_timedlock.c
index 07f0901e5266..06f466ef5dde 100644
--- a/nptl/pthread_mutex_timedlock.c
+++ b/nptl/pthread_mutex_timedlock.c
@@ -74,7 +74,7 @@ pthread_mutex_timedlock (pthread_mutex_t *mutex,

/* We have to get the mutex. */
result = lll_timedlock (mutex->__data.__lock, abstime,
- PTHREAD_MUTEX_PSHARED (mutex));
+ PTHREAD_MUTEX_PSHARED_PATTACHED (mutex));

if (result != 0)
goto out;
@@ -97,7 +97,7 @@ pthread_mutex_timedlock (pthread_mutex_t *mutex,
simple:
/* Normal mutex. */
result = lll_timedlock (mutex->__data.__lock, abstime,
- PTHREAD_MUTEX_PSHARED (mutex));
+ PTHREAD_MUTEX_PSHARED_PATTACHED (mutex));
break;

case PTHREAD_MUTEX_TIMED_ELISION_NP:
@@ -106,7 +106,7 @@ pthread_mutex_timedlock (pthread_mutex_t *mutex,
return lll_timedlock_elision (mutex->__data.__lock,
mutex->__data.__spins,
abstime,
- PTHREAD_MUTEX_PSHARED (mutex));
+ PTHREAD_MUTEX_PSHARED_PATTACHED (mutex));

case PTHREAD_MUTEX_ADAPTIVE_NP:
@@ -123,7 +123,7 @@ pthread_mutex_timedlock (pthread_mutex_t *mutex,
if (cnt++ >= max_cnt)
{
result = lll_timedlock (mutex->__data.__lock, abstime,
- PTHREAD_MUTEX_PSHARED (mutex));
+ PTHREAD_MUTEX_PSHARED_PATTACHED (mutex));
break;
}
atomic_spin_nop ();
@@ -204,7 +204,7 @@ pthread_mutex_timedlock (pthread_mutex_t *mutex,
}

result = lll_robust_timedlock (mutex->__data.__lock, abstime, id,
- PTHREAD_ROBUST_MUTEX_PSHARED (mutex));
+ PTHREAD_ROBUST_MUTEX_PSHARED_PATTACHED (mutex));

if (__builtin_expect (mutex->__data.__owner
== PTHREAD_MUTEX_NOTRECOVERABLE, 0))
@@ -212,7 +212,7 @@ pthread_mutex_timedlock (pthread_mutex_t *mutex,
/* This mutex is now not recoverable. */
mutex->__data.__count = 0;
lll_unlock (mutex->__data.__lock,
- PTHREAD_ROBUST_MUTEX_PSHARED (mutex));
+ PTHREAD_ROBUST_MUTEX_PSHARED_PATTACHED (mutex));
THREAD_SETMEM (THREAD_SELF, robust_head.list_op_pending, NULL);
return ENOTRECOVERABLE;
}
@@ -288,8 +288,8 @@ pthread_mutex_timedlock (pthread_mutex_t *mutex,
everything. The timeout value must be a relative value.
Convert it. */
int private = (robust
- ? PTHREAD_ROBUST_MUTEX_PSHARED (mutex)
- : PTHREAD_MUTEX_PSHARED (mutex));
+ ? PTHREAD_ROBUST_MUTEX_PSHARED_PATTACHED (mutex)
+ : PTHREAD_MUTEX_PSHARED_PATTACHED (mutex));
INTERNAL_SYSCALL_DECL (__err);

int e = INTERNAL_SYSCALL (futex, __err, 4, &mutex->__data.__lock,
@@ -370,7 +370,7 @@ pthread_mutex_timedlock (pthread_mutex_t *mutex,
INTERNAL_SYSCALL_DECL (__err);
INTERNAL_SYSCALL (futex, __err, 4, &mutex->__data.__lock,
__lll_private_flag (FUTEX_UNLOCK_PI,
- PTHREAD_ROBUST_MUTEX_PSHARED (mutex)),
+ PTHREAD_ROBUST_MUTEX_PSHARED_PATTACHED (mutex)),
0, 0);

THREAD_SETMEM (THREAD_SELF, robust_head.list_op_pending, NULL);
@@ -489,7 +489,7 @@ pthread_mutex_timedlock (pthread_mutex_t *mutex,

lll_futex_timed_wait (&mutex->__data.__lock,
ceilval | 2, &rt,
- PTHREAD_MUTEX_PSHARED (mutex));
+ PTHREAD_MUTEX_PSHARED_PATTACHED (mutex));
}
}
while (atomic_compare_and_exchange_val_acq (&mutex->__data.__lock,
diff --git a/nptl/pthread_mutex_trylock.c b/nptl/pthread_mutex_trylock.c
index 48c7865702ba..c0221cc86c02 100644
--- a/nptl/pthread_mutex_trylock.c
+++ b/nptl/pthread_mutex_trylock.c
@@ -170,7 +170,7 @@ __pthread_mutex_trylock (pthread_mutex_t *mutex)
mutex->__data.__count = 0;
if (oldval == id)
lll_unlock (mutex->__data.__lock,
- PTHREAD_ROBUST_MUTEX_PSHARED (mutex));
+ PTHREAD_ROBUST_MUTEX_PSHARED_PATTACHED (mutex));
THREAD_SETMEM (THREAD_SELF, robust_head.list_op_pending, NULL);
return ENOTRECOVERABLE;
}
@@ -252,8 +252,8 @@ __pthread_mutex_trylock (pthread_mutex_t *mutex)
/* The mutex owner died. The kernel will now take care of
everything. */
int private = (robust
- ? PTHREAD_ROBUST_MUTEX_PSHARED (mutex)
- : PTHREAD_MUTEX_PSHARED (mutex));
+ ? PTHREAD_ROBUST_MUTEX_PSHARED_PATTACHED (mutex)
+ : PTHREAD_MUTEX_PSHARED_PATTACHED (mutex));
INTERNAL_SYSCALL_DECL (__err);
int e = INTERNAL_SYSCALL (futex, __err, 4, &mutex->__data.__lock,
__lll_private_flag (FUTEX_TRYLOCK_PI,
@@ -299,7 +299,7 @@ __pthread_mutex_trylock (pthread_mutex_t *mutex)
INTERNAL_SYSCALL_DECL (__err);
INTERNAL_SYSCALL (futex, __err, 4, &mutex->__data.__lock,
__lll_private_flag (FUTEX_UNLOCK_PI,
- PTHREAD_ROBUST_MUTEX_PSHARED (mutex)),
+ PTHREAD_ROBUST_MUTEX_PSHARED_PATTACHED (mutex)),
0, 0);

THREAD_SETMEM (THREAD_SELF, robust_head.list_op_pending, NULL);
diff --git a/nptl/pthread_mutex_unlock.c b/nptl/pthread_mutex_unlock.c
index 334ce383420e..f2c276881f03 100644
--- a/nptl/pthread_mutex_unlock.c
+++ b/nptl/pthread_mutex_unlock.c
@@ -52,7 +52,7 @@ __pthread_mutex_unlock_usercnt (pthread_mutex_t *mutex, int decr)
--mutex->__data.__nusers;

/* Unlock. */
- lll_unlock (mutex->__data.__lock, PTHREAD_MUTEX_PSHARED (mutex));
+ lll_unlock (mutex->__data.__lock, PTHREAD_MUTEX_PSHARED_PATTACHED (mutex));

LIBC_PROBE (mutex_release, 1, mutex);

@@ -62,7 +62,7 @@ __pthread_mutex_unlock_usercnt (pthread_mutex_t *mutex, int decr)
{
/* Don't reset the owner/users fields for elision. */
return lll_unlock_elision (mutex->__data.__lock, mutex->__data.__elision,
- PTHREAD_MUTEX_PSHARED (mutex));
+ PTHREAD_MUTEX_PSHARED_PATTACHED (mutex));
}
else if (__builtin_expect (PTHREAD_MUTEX_TYPE (mutex)
== PTHREAD_MUTEX_RECURSIVE_NP, 1))
@@ -151,7 +151,7 @@ __pthread_mutex_unlock_full (pthread_mutex_t *mutex, int decr)

/* Unlock. */
lll_robust_unlock (mutex->__data.__lock,
- PTHREAD_ROBUST_MUTEX_PSHARED (mutex));
+ PTHREAD_ROBUST_MUTEX_PSHARED_PATTACHED (mutex));

THREAD_SETMEM (THREAD_SELF, robust_head.list_op_pending, NULL);
break;
@@ -235,8 +235,8 @@ __pthread_mutex_unlock_full (pthread_mutex_t *mutex, int decr)
lll_unlock). */
int robust = mutex->__data.__kind & PTHREAD_MUTEX_ROBUST_NORMAL_NP;
int private = (robust
- ? PTHREAD_ROBUST_MUTEX_PSHARED (mutex)
- : PTHREAD_MUTEX_PSHARED (mutex));
+ ? PTHREAD_ROBUST_MUTEX_PSHARED_PATTACHED (mutex)
+ : PTHREAD_MUTEX_PSHARED_PATTACHED (mutex));
if ((mutex->__data.__lock & FUTEX_WAITERS) != 0
|| atomic_compare_and_exchange_bool_rel (&mutex->__data.__lock, 0,
THREAD_GETMEM (THREAD_SELF,
@@ -290,7 +290,7 @@ __pthread_mutex_unlock_full (pthread_mutex_t *mutex, int decr)

if ((oldval & ~PTHREAD_MUTEX_PRIO_CEILING_MASK) > 1)
lll_futex_wake (&mutex->__data.__lock, 1,
- PTHREAD_MUTEX_PSHARED (mutex));
+ PTHREAD_MUTEX_PSHARED_PATTACHED (mutex));

int oldprio = newval >> PTHREAD_MUTEX_PRIO_CEILING_SHIFT;

diff --git a/nptl/pthread_mutexattr_getattached_np.c b/nptl/pthread_mutexattr_getattached_np.c
new file mode 100644
index 000000000000..033cfdb141a8
--- /dev/null
+++ b/nptl/pthread_mutexattr_getattached_np.c
@@ -0,0 +1,15 @@
+#include <pthreadP.h>
+
+int pthread_mutexattr_getattached_np (const pthread_mutexattr_t *attr,
+ int *attached)
+{
+ const struct pthread_mutexattr *iattr;
+
+ iattr = (const struct pthread_mutexattr *) attr;
+
+ if (iattr->mutexkind & PTHREAD_MUTEXATTR_FLAG_ATTACHED)
+ *attached = 1;
+ else
+ *attached = 0;
+ return 0;
+}
diff --git a/nptl/pthread_mutexattr_setattached_np.c b/nptl/pthread_mutexattr_setattached_np.c
new file mode 100644
index 000000000000..aadbdc7478a8
--- /dev/null
+++ b/nptl/pthread_mutexattr_setattached_np.c
@@ -0,0 +1,17 @@
+#include <errno.h>
+#include <pthreadP.h>
+
+int pthread_mutexattr_setattached_np (pthread_mutexattr_t *attr,
+ int attached)
+{
+ struct pthread_mutexattr *iattr = (struct pthread_mutexattr *) attr;
+
+ if (attached < 0 || attached > 1)
+ return EINVAL;
+
+ if (!attached)
+ iattr->mutexkind &= ~PTHREAD_MUTEXATTR_FLAG_ATTACHED;
+ else
+ iattr->mutexkind |= PTHREAD_MUTEXATTR_FLAG_ATTACHED;
+ return 0;
+}
diff --git a/sysdeps/nptl/pthread.h b/sysdeps/nptl/pthread.h
index fd0894efd210..75c1c9c2112e 100644
--- a/sysdeps/nptl/pthread.h
+++ b/sysdeps/nptl/pthread.h
@@ -762,6 +762,12 @@ extern int pthread_mutex_trylock (pthread_mutex_t *__mutex)
extern int pthread_mutex_lock (pthread_mutex_t *__mutex)
__THROWNL __nonnull ((1));

+extern int pthread_mutex_attach_np (pthread_mutex_t *__mutex)
+ __THROWNL __nonnull ((1));
+
+extern int pthread_mutex_detach_np (pthread_mutex_t *__mutex)
+ __THROWNL __nonnull ((1));
+
#ifdef __USE_XOPEN2K
/* Wait until lock becomes available, or specified time passes. */
extern int pthread_mutex_timedlock (pthread_mutex_t *__restrict __mutex,
@@ -846,6 +852,19 @@ extern int pthread_mutexattr_setprotocol (pthread_mutexattr_t *__attr,
int __protocol)
__THROW __nonnull ((1));

+/* Return in *ATTACHED the mutex attached attribute in *ATTR. */
+ extern int pthread_mutexattr_getattached_np (const pthread_mutexattr_t *
+ __restrict __attr,
+ int *__restrict __attached)
+ __THROW __nonnull ((1, 2));
+
+/* Set the mutex attached attribute in *ATTR to ATTACHED (either
+ 0, or 1). */
+extern int pthread_mutexattr_setattached_np (pthread_mutexattr_t *__attr,
+ int __attached)
+ __THROW __nonnull ((1));
+
+
/* Return in *PRIOCEILING the mutex prioceiling attribute in *ATTR. */
extern int pthread_mutexattr_getprioceiling (const pthread_mutexattr_t *
__restrict __attr,
diff --git a/sysdeps/unix/sysv/linux/i386/libpthread.abilist b/sysdeps/unix/sysv/linux/i386/libpthread.abilist
index 8f9c3254be88..c0e711985fc4 100644
--- a/sysdeps/unix/sysv/linux/i386/libpthread.abilist
+++ b/sysdeps/unix/sysv/linux/i386/libpthread.abilist
@@ -216,6 +216,11 @@ GLIBC_2.2.3 GLIBC_2.2.3 A
GLIBC_2.2.3 pthread_getattr_np F
GLIBC_2.2.6 GLIBC_2.2.6 A
GLIBC_2.2.6 __nanosleep F
+GLIBC_2.23 GLIBC_2.23 A
+GLIBC_2.23 pthread_mutexattr_getattached_np F
+GLIBC_2.23 pthread_mutexattr_setattached_np F
+GLIBC_2.23 pthread_mutex_attach_np F
+GLIBC_2.23 pthread_mutex_detach_np F
GLIBC_2.3.2 GLIBC_2.3.2 A
GLIBC_2.3.2 pthread_cond_broadcast F
GLIBC_2.3.2 pthread_cond_destroy F
diff --git a/sysdeps/unix/sysv/linux/lowlevellock-futex.h b/sysdeps/unix/sysv/linux/lowlevellock-futex.h
index 7111bac9431e..be03683e2e08 100644
--- a/sysdeps/unix/sysv/linux/lowlevellock-futex.h
+++ b/sysdeps/unix/sysv/linux/lowlevellock-futex.h
@@ -38,8 +38,11 @@
#define FUTEX_WAKE_BITSET 10
#define FUTEX_WAIT_REQUEUE_PI 11
#define FUTEX_CMP_REQUEUE_PI 12
+#define FUTEX_ATTACH 13
+#define FUTEX_DETACH 14
#define FUTEX_PRIVATE_FLAG 128
#define FUTEX_CLOCK_REALTIME 256
+#define FUTEX_ATTACHED 512

#define FUTEX_BITSET_MATCH_ANY 0xffffffff

@@ -72,9 +75,9 @@
# else
# define __lll_private_flag(fl, private) \
(__builtin_constant_p (private) \
- ? ((private) == 0 \
- ? ((fl) | THREAD_GETMEM (THREAD_SELF, header.private_futex)) \
- : (fl)) \
+ ? ((private & LLL_SHARED) == 0 \
+ ? ((fl) | THREAD_GETMEM (THREAD_SELF, header.private_futex) | (private & FUTEX_ATTACHED)) \
+ : (fl) | (private & FUTEX_ATTACHED)) \
: ((fl) | (((private) ^ FUTEX_PRIVATE_FLAG) \
& THREAD_GETMEM (THREAD_SELF, header.private_futex))))
# endif
diff --git a/sysdeps/unix/sysv/linux/x86_64/64/libpthread.abilist b/sysdeps/unix/sysv/linux/x86_64/64/libpthread.abilist
index 85365c057cd7..daa0ac8f2320 100644
--- a/sysdeps/unix/sysv/linux/x86_64/64/libpthread.abilist
+++ b/sysdeps/unix/sysv/linux/x86_64/64/libpthread.abilist
@@ -203,6 +203,11 @@ GLIBC_2.2.5 waitpid F
GLIBC_2.2.5 write F
GLIBC_2.2.6 GLIBC_2.2.6 A
GLIBC_2.2.6 __nanosleep F
+GLIBC_2.23 GLIBC_2.23 A
+GLIBC_2.23 pthread_mutexattr_getattached_np F
+GLIBC_2.23 pthread_mutexattr_setattached_np F
+GLIBC_2.23 pthread_mutex_attach_np F
+GLIBC_2.23 pthread_mutex_detach_np F
GLIBC_2.3.2 GLIBC_2.3.2 A
GLIBC_2.3.2 pthread_cond_broadcast F
GLIBC_2.3.2 pthread_cond_destroy F
diff --git a/sysdeps/unix/sysv/linux/x86_64/x32/libpthread.abilist b/sysdeps/unix/sysv/linux/x86_64/x32/libpthread.abilist
index 6cd0fc348708..de17ebf46818 100644
--- a/sysdeps/unix/sysv/linux/x86_64/x32/libpthread.abilist
+++ b/sysdeps/unix/sysv/linux/x86_64/x32/libpthread.abilist
@@ -224,3 +224,8 @@ GLIBC_2.16 write F
GLIBC_2.18 GLIBC_2.18 A
GLIBC_2.18 pthread_getattr_default_np F
GLIBC_2.18 pthread_setattr_default_np F
+GLIBC_2.23 GLIBC_2.23 A
+GLIBC_2.23 pthread_mutexattr_getattached_np F
+GLIBC_2.23 pthread_mutexattr_setattached_np F
+GLIBC_2.23 pthread_mutex_attach_np F
+GLIBC_2.23 pthread_mutex_detach_np F
--
2.8.0.rc3

Next message: Thomas Gleixner: "[RFC patch 2/7] futex: Add some more function commentry"
Previous message: Thomas Gleixner: "[RFC patch 4/7] futex: Add support for attached futexes"
In reply to: Carlos O'Donell: "Re: [RFC patch 4/7] futex: Add support for attached futexes"
Next in thread: Peter Zijlstra: "Re: [RFC patch 7/7] [PATCH] glibc: nptl: Add support for attached pthread_mutexes"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]