[PATCH 1/3] arm64: Always inline raw preempt count updates
From: Karl Mehltretter
Date: Mon Nov 16 2026 - 02:54:50 EST
From: Hongyan Xia <hongyan.xia@xxxxxxxxxxxxx>
Raw preempt count updates can be emitted out of line by the compiler.
For a noinstr caller, that puts the update in ordinary instrumentable
text. Force __preempt_count_add() and __preempt_count_sub() inline so
their code stays in the caller's section.
Link: https://lore.kernel.org/all/c3a90368dd6c096af9d918e036a27728696fb8a4.1786603168.git.hongyan.xia@xxxxxxxxxxxxx/
Signed-off-by: Hongyan Xia <hongyan.xia@xxxxxxxxxxxxx>
[kmehltretter: split out the arm64 preempt.h hunk and rewrite the changelog]
Signed-off-by: Karl Mehltretter <kmehltretter@xxxxxxxxx>
---
arch/arm64/include/asm/preempt.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/preempt.h b/arch/arm64/include/asm/preempt.h
index 75a592b7f47f7..f3a5c2cd1936d 100644
--- a/arch/arm64/include/asm/preempt.h
+++ b/arch/arm64/include/asm/preempt.h
@@ -41,14 +41,14 @@ static inline bool test_preempt_need_resched(void)
return !current_thread_info()->preempt.need_resched;
}
-static inline void __preempt_count_add(int val)
+static __always_inline void __preempt_count_add(int val)
{
u32 pc = READ_ONCE(current_thread_info()->preempt.count);
pc += val;
WRITE_ONCE(current_thread_info()->preempt.count, pc);
}
-static inline void __preempt_count_sub(int val)
+static __always_inline void __preempt_count_sub(int val)
{
u32 pc = READ_ONCE(current_thread_info()->preempt.count);
pc -= val;