[PATCH 2/2] futex: Use atomic64_try_cmpxchg_relaxed() in get_inode_sequence_number()
From: Uros Bizjak
Date: Thu Oct 10 2024 - 04:16:47 EST
Optimize get_inode_sequence_number() to use simpler and faster:
!atomic64_try_cmpxchg_relaxed(*ptr, &old, new)
instead of:
atomic64_cmpxchg relaxed(*ptr, old, new) != old
The x86 CMPXCHG instruction returns success in ZF flag, so
this change saves a compare after cmpxchg. The generated
code improves from:
3da: 31 c0 xor %eax,%eax
3dc: f0 48 0f b1 8a 38 01 lock cmpxchg %rcx,0x138(%rdx)
3e3: 00 00
3e5: 48 85 c0 test %rax,%rax
3e8: 48 0f 44 c1 cmove %rcx,%rax
to:
3da: 31 c0 xor %eax,%eax
3dc: f0 48 0f b1 8a 38 01 lock cmpxchg %rcx,0x138(%rdx)
3e3: 00 00
3e5: 48 0f 44 c1 cmove %rcx,%rax
Signed-off-by: Uros Bizjak <ubizjak@xxxxxxxxx>
Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
Cc: Ingo Molnar <mingo@xxxxxxxxxx>
Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
Cc: Darren Hart <dvhart@xxxxxxxxxxxxx>
Cc: Davidlohr Bueso <dave@xxxxxxxxxxxx>
Cc: "André Almeida" <andrealmeid@xxxxxxxxxx>
---
kernel/futex/core.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/kernel/futex/core.c b/kernel/futex/core.c
index 3146730e55f7..11795439efb7 100644
--- a/kernel/futex/core.c
+++ b/kernel/futex/core.c
@@ -185,8 +185,9 @@ static u64 get_inode_sequence_number(struct inode *inode)
if (WARN_ON_ONCE(!new))
continue;
- old = atomic64_cmpxchg_relaxed(&inode->i_sequence, 0, new);
- if (old)
+ old = 0;
+ if (!atomic64_try_cmpxchg_relaxed(&inode->i_sequence,
+ &old, new))
return old;
return new;
}
--
2.46.2