[tip:locking/core] locking/qspinlock: Avoid redundant read of next pointer

From: tip-bot for Waiman Long
Date: Mon Nov 23 2015 - 11:28:41 EST

Commit-ID: aa68744f80bfb6f26fbe7f10e42876066f7dac1b
Gitweb: http://git.kernel.org/tip/aa68744f80bfb6f26fbe7f10e42876066f7dac1b
Author: Waiman Long <Waiman.Long@xxxxxxx>
AuthorDate: Mon, 9 Nov 2015 19:09:23 -0500
Committer: Ingo Molnar <mingo@xxxxxxxxxx>
CommitDate: Mon, 23 Nov 2015 10:02:00 +0100

locking/qspinlock: Avoid redundant read of next pointer

With optimistic prefetch of the next node cacheline, the next pointer
may have been properly inititalized. As a result, the reading
of node->next in the contended path may be redundant. This patch
eliminates the redundant read if the next pointer value is not NULL.

Signed-off-by: Waiman Long <Waiman.Long@xxxxxxx>
Signed-off-by: Peter Zijlstra (Intel) <peterz@xxxxxxxxxxxxx>
Cc: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
Cc: Davidlohr Bueso <dave@xxxxxxxxxxxx>
Cc: Douglas Hatch <doug.hatch@xxxxxxx>
Cc: H. Peter Anvin <hpa@xxxxxxxxx>
Cc: Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx>
Cc: Paul E. McKenney <paulmck@xxxxxxxxxxxxxxxxxx>
Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
Cc: Scott J Norton <scott.norton@xxxxxxx>
Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
Link: http://lkml.kernel.org/r/1447114167-47185-4-git-send-email-Waiman.Long@xxxxxxx
Signed-off-by: Ingo Molnar <mingo@xxxxxxxxxx>
kernel/locking/qspinlock.c | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
index 365b203..9862078 100644
--- a/kernel/locking/qspinlock.c
+++ b/kernel/locking/qspinlock.c
@@ -396,6 +396,7 @@ queue:
* p,*,* -> n,*,*
old = xchg_tail(lock, tail);
+ next = NULL;

* if there was a previous node; link it and wait until reaching the
@@ -463,10 +464,12 @@ queue:

- * contended path; wait for next, release.
+ * contended path; wait for next if not observed yet, release.
- while (!(next = READ_ONCE(node->next)))
- cpu_relax();
+ if (!next) {
+ while (!(next = READ_ONCE(node->next)))
+ cpu_relax();
+ }

pv_kick_node(lock, next);
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/