[tip: x86/core] x86,bpf: Fix bpf_arch_text_poke()

[tip-bot2 for Peter Zijlstra]

The following commit has been merged into the x86/core branch of tip:

Commit-ID:     9e1db76f44de4d9439e48c9ef61e5d457395202b
Gitweb:        https://git.kernel.org/tip/9e1db76f44de4d9439e48c9ef61e5d457395202b
Author:        Peter Zijlstra <peterz@xxxxxxxxxxxxx>
AuthorDate:    Fri, 11 Mar 2022 10:44:37 +01:00
Committer:     Peter Zijlstra <peterz@xxxxxxxxxxxxx>
CommitterDate: Fri, 11 Mar 2022 13:05:08 +01:00

x86,bpf: Fix bpf_arch_text_poke()

Alexei reported that BPF direct trampolines are no longer working with
IBT=y builds.

Ensure that bpf_arch_text_poke() can find the NOP callsite emitted by
emit_prologue().

Fixes: 2f6f7bf13ab6 ("x86/ibt,bpf: Add ENDBR instructions to prologue and trampoline")
Reported-by: Alexei Starovoitov <alexei.starovoitov@xxxxxxxxx>
Signed-off-by: Peter Zijlstra (Intel) <peterz@xxxxxxxxxxxxx>
Link: https://lore.kernel.org/r/YionV0%2bv%2fcUBiOh0@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
---
 arch/x86/net/bpf_jit_comp.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 159b79f..f071c03 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -395,6 +395,13 @@ int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type t,
 		/* BPF poking in modules is not supported */
 		return -EINVAL;
 
+	/*
+	 * See emit_prologue(), for IBT builds the trampoline hook is preceded
+	 * with an ENDBR instruction.
+	 */
+	if (is_endbr(*(u32 *)ip))
+		ip += ENDBR_INSN_SIZE;
+
 	return __bpf_arch_text_poke(ip, t, old_addr, new_addr, true);
 }