[PATCH] futex: Improve get_inode_sequence_number()
From: Uros Bizjak
Date: Thu Oct 03 2024 - 08:19:35 EST
Rewrite FOR loop to a DO-WHILE loop where returns are moved out of
the loop. Use atomic64_inc_return() instead of atomic64_add_return().
Use !atomic64_try_cmpxchg_relaxed(*ptr, &old, new) instead of
atomic64_cmpxchg_relaxed (*ptr, old, new) != old. x86 CMPXCHG
instruction returns success in ZF flag, so this change saves
a compare after CMPXCHG..
Note that due to early return, "old" equals to 0 before
atomic64_cmpxchg_relaxed(), so initialization of variable to 0
is not needed.
Signed-off-by: Uros Bizjak <ubizjak@xxxxxxxxx>
Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
Cc: Ingo Molnar <mingo@xxxxxxxxxx>
Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
Cc: Darren Hart <dvhart@xxxxxxxxxxxxx>
Cc: Davidlohr Bueso <dave@xxxxxxxxxxxx>
Cc: "André Almeida" <andrealmeid@xxxxxxxxxx>
---
kernel/futex/core.c | 17 +++++++----------
1 file changed, 7 insertions(+), 10 deletions(-)
diff --git a/kernel/futex/core.c b/kernel/futex/core.c
index 136768ae2637..665501c885d0 100644
--- a/kernel/futex/core.c
+++ b/kernel/futex/core.c
@@ -173,23 +173,20 @@ futex_setup_timer(ktime_t *time, struct hrtimer_sleeper *timeout,
static u64 get_inode_sequence_number(struct inode *inode)
{
static atomic64_t i_seq;
- u64 old;
+ u64 old, new;
/* Does the inode already have a sequence number? */
old = atomic64_read(&inode->i_sequence);
if (likely(old))
return old;
- for (;;) {
- u64 new = atomic64_add_return(1, &i_seq);
- if (WARN_ON_ONCE(!new))
- continue;
+ do {
+ new = atomic64_inc_return(&i_seq);
+ } while (WARN_ON_ONCE(!new));
- old = atomic64_cmpxchg_relaxed(&inode->i_sequence, 0, new);
- if (old)
- return old;
- return new;
- }
+ if (!atomic64_try_cmpxchg_relaxed(&inode->i_sequence, &old, new))
+ return old;
+ return new;
}
/**
--
2.46.2