[PATCH v21 14/15] arm64: Inline el0_svc_common()

[Jinjie Ruan]

After converting arm64 to Generic Entry framework, the compiler no longer
inlines el0_svc_common() into its caller do_el0_svc(). This introduces
a small but measurable overhead in the critical system call path.

Manually forcing el0_svc_common() to be inlined restores the
performance. Benchmarking with perf bench syscall basic on a
Kunpeng 920 platform (based on v6.19-rc1) shows a ~1% performance
uplift.

Inlining this function reduces function prologue/epilogue overhead
and allows for better compiler optimization in the hot system call
dispatch path.

| Metric     | W/O this patch | With this patch | Change    |
| ---------- | -------------- | --------------- | --------- |
| Total time | 2.195 [sec]    | 2.171 [sec]     |  ↓1.1%   |
| usecs/op   | 0.219575       | 0.217192        |  ↓1.1%   |
| ops/sec    | 4,554,260      | 4,604,225       |  ↑1.1%    |

Two notes on the inlining:

- Syscall-path stack usage grows slightly.  do_el0_svc() now has a
  single 112-byte frame and 13 calls, where previously a 16-byte frame
  called a 48-byte el0_svc_common(); that is +48 bytes of stack on the
  syscall path.  lkdtm's KSTACK_ERASE confirms the 48-byte delta at
  runtime.

- do_el0_svc() is not noinstr and stays in .text, so the inlined code
  is instrumentable, as el0_svc_common() was before. This is fine:
  el0_svc() only calls do_el0_svc() after
  arm64_syscall_enter_from_user_mode() has set up the context tracking,
  lockdep and RCU state, and do_el0_svc() is in a different translation
  unit, so it is not inlined into the noinstr el0_svc().

Cc: Mark Rutland <mark.rutland@xxxxxxx>
Cc: Will Deacon <will@xxxxxxxxxx>
Cc: Catalin Marinas <catalin.marinas@xxxxxxx>
Reviewed-by: Ada Couprie Diaz <ada.coupriediaz@xxxxxxx>
Reviewed-by: Linus Walleij <linusw@xxxxxxxxxx>
Reviewed-by: Yeoreum Yun <yeoreum.yun@xxxxxxx>
Reviewed-by: Kevin Brodsky <kevin.brodsky@xxxxxxx>
Signed-off-by: Jinjie Ruan <ruanjinjie@xxxxxxxxxx>
---
 arch/arm64/kernel/syscall.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kernel/syscall.c b/arch/arm64/kernel/syscall.c
index 4a13108defb0..2535cae9413d 100644
--- a/arch/arm64/kernel/syscall.c
+++ b/arch/arm64/kernel/syscall.c
@@ -55,8 +55,8 @@ static void invoke_syscall(struct pt_regs *regs, unsigned int scno,
 	syscall_set_return_value(current, regs, 0, ret);
 }
 
-static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,
-			   const syscall_fn_t syscall_table[])
+static __always_inline void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,
+					   const syscall_fn_t syscall_table[])
 {
 	unsigned long flags = read_thread_flags();
 	unsigned long work;
-- 
2.34.1