Re: [PATCH 1/3] ipc/sem.c: Chance memory barrier in sem_lock() to smp_rmb()

From: Davidlohr Bueso
Date: Thu Oct 09 2014 - 19:07:58 EST


On Mon, 2014-10-06 at 20:32 +0200, Manfred Spraul wrote:
> When I fixed bugs in the sem_lock() logic, I was more conservative than
> necessary.
> Therefore it is safe to replace the smp_mb() with smp_rmb().
> And: With smp_rmb(), semop() syscalls are up to 10% faster.
>
> The race we must protect against is:
>
> sem->lock is free
> sma->complex_count = 0
> sma->sem_perm.lock held by thread B
>
> thread A:
>
> A: spin_lock(&sem->lock)
>
> B: sma->complex_count++; (now 1)
> B: spin_unlock(&sma->sem_perm.lock);
>
> A: spin_is_locked(&sma->sem_perm.lock);
> A: XXXXX memory barrier
> A: if (sma->complex_count == 0)
>
> Thread A must read the increased complex_count value, i.e. the read must
> not be reordered with the read of sem_perm.lock done by spin_is_locked().
>
> Since it's about ordering of reads, smp_rmb() is sufficient.
>
> Signed-off-by: Manfred Spraul <manfred@xxxxxxxxxxxxxxxx>

Reviewed-by: Davidlohr Bueso <dave@xxxxxxxxxxxx>

With a suggestion below.

> ---
> ipc/sem.c | 12 +++++++++---
> 1 file changed, 9 insertions(+), 3 deletions(-)
>
> diff --git a/ipc/sem.c b/ipc/sem.c
> index 454f6c6..ffc71de 100644
> --- a/ipc/sem.c
> +++ b/ipc/sem.c
> @@ -326,10 +326,16 @@ static inline int sem_lock(struct sem_array *sma, struct sembuf *sops,
>
> /* Then check that the global lock is free */
> if (!spin_is_locked(&sma->sem_perm.lock)) {
> - /* spin_is_locked() is not a memory barrier */
> - smp_mb();
> + /*
> + * The next test must happen after the test for
> + * sem_perm.lock, otherwise we can race with another
> + * thread that does
> + * complex_count++;spin_unlock(sem_perm.lock);
> + */

How about this comment instead:

/*
* The ipc object lock check must be visible on all cores before
* rechecking the complex count. Otherwise we can race with
* another thread that does:
* complex_count++++;
* spin_unlock(sem_perm.lock);
*/

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/