[PATCH bpf-next v10 0/5] emit ENDBR/BTI instructions for indirect jump targets

[Xu Kuohai]

On architectures with CFI protection enabled that require landing pad
instructions at indirect jump targets, such as x86 with CET/IBT enabled
and arm64 with BTI enabled, kernel panics when an indirect jump lands on
a target without landing pad. Therefore, the JIT must emit landing pad
instructions for indirect jump targets.

The verifier already recognizes which instructions are indirect jump
targets during the verification phase. So we can store this information
in env->insn_aux_data and pass it to the JIT as new parameter, so the JIT
knows which instructions are indirect jump targets.

During JIT, constants blinding is performed. It rewrites the private copy
of instructions for the JITed program, but it does not adjust the global
env->insn_aux_data array. As a result, after constants blinding, the
instruction indexes used by JIT may no longer match the indexes in
env->insn_aux_data, so the JIT can not lookup env->insn_aux_data directly.

To avoid this mismatch, and considering that all existing arch-specific JITs
already implement constants blinding with largely duplicated code, move
constants blinding from JIT to generic code.

v10:
- Fix the incorrect call_imm restore in jit_subprogs 
- Define a dummy void version of bpf_jit_prog_release_other and
  bpf_patch_insn_data when the corresponding config is not set
- Remove the unnecessary #ifdef in x86_64 JIT (Leon Hwang)

v9: https://lore.kernel.org/bpf/20260312170255.3427799-1-xukuohai@xxxxxxxxxxxxxxx/
- Make constant blinding available for classic bpf (Eduard)
- Clear prog->bpf_func, prog->jited ... on the error path of extra pass (Eduard)
- Fix spelling errors and remove unused parameter (Anton Protopopov)

v8: https://lore.kernel.org/bpf/20260309140044.2652538-1-xukuohai@xxxxxxxxxxxxxxx/
- Define void bpf_jit_blind_constants() function when CONFIG_BPF_JIT is not set 
- Move indirect_target fixup for insn patching from bpf_jit_blind_constants()
  to adjust_insn_aux_data()

v7: https://lore.kernel.org/bpf/20260307103949.2340104-1-xukuohai@xxxxxxxxxxxxxxx
- Move constants blinding logic back to bpf/core.c
- Compute ip address before switch statement in x86 JIT
- Clear JIT state from error path on arm64 and loongarch 

v6: https://lore.kernel.org/bpf/20260306102329.2056216-1-xukuohai@xxxxxxxxxxxxxxx/
- Move constants blinding from JIT to verifier
- Move call to bpf_prog_select_runtime from bpf_prog_load to verifier

v5: https://lore.kernel.org/bpf/20260302102726.1126019-1-xukuohai@xxxxxxxxxxxxxxx/
- Switch to pass env to JIT directly to get rid of copying private insn_aux_data for
  each prog

v4: https://lore.kernel.org/all/20260114093914.2403982-1-xukuohai@xxxxxxxxxxxxxxx/
- Switch to the approach proposed by Eduard, using insn_aux_data to identify indirect
  jump targets, and emit ENDBR on x86

v3: https://lore.kernel.org/bpf/20251227081033.240336-1-xukuohai@xxxxxxxxxxxxxxx/
- Get rid of unnecessary enum definition (Yonghong Song, Anton Protopopov)

v2: https://lore.kernel.org/bpf/20251223085447.139301-1-xukuohai@xxxxxxxxxxxxxxx/
- Exclude instruction arrays not used for indirect jumps (Anton Protopopov)

v1: https://lore.kernel.org/bpf/20251127140318.3944249-1-xukuohai@xxxxxxxxxxxxxxx/

Xu Kuohai (5):
  bpf: Move constants blinding out of arch-specific JITs
  bpf: Pass bpf_verifier_env to JIT
  bpf: Add helper to detect indirect jump targets
  bpf, x86: Emit ENDBR for indirect jump targets
  bpf, arm64: Emit BTI for indirect jump target

 arch/arc/net/bpf_jit_core.c      |  41 ++++-------
 arch/arm/net/bpf_jit_32.c        |  43 +++--------
 arch/arm64/net/bpf_jit_comp.c    |  87 ++++++++---------------
 arch/loongarch/net/bpf_jit.c     |  61 +++++-----------
 arch/mips/net/bpf_jit_comp.c     |  22 +-----
 arch/parisc/net/bpf_jit_core.c   |  75 ++++++++------------
 arch/powerpc/net/bpf_jit_comp.c  |  70 +++++++-----------
 arch/riscv/net/bpf_jit_core.c    |  63 ++++++-----------
 arch/s390/net/bpf_jit_comp.c     |  61 ++++++----------
 arch/sparc/net/bpf_jit_comp_64.c |  63 ++++++-----------
 arch/x86/net/bpf_jit_comp.c      |  73 ++++++-------------
 arch/x86/net/bpf_jit_comp32.c    |  35 ++-------
 include/linux/bpf.h              |   2 +
 include/linux/bpf_verifier.h     |   9 +--
 include/linux/filter.h           |  27 ++++++-
 kernel/bpf/core.c                | 118 +++++++++++++++++++++----------
 kernel/bpf/syscall.c             |   4 --
 kernel/bpf/verifier.c            |  79 +++++++++++++++------
 18 files changed, 392 insertions(+), 541 deletions(-)

-- 
2.47.3