[PATCH bpf-next v2 0/6] bpf: Introduce 64-bit bitops kfuncs

[Leon Hwang]

This series adds generic 64-bit bitops kfuncs and JIT inlining support
on x86_64 and arm64.

The new kfuncs are:

* bpf_clz64(): Count leading zeros.
* bpf_ctz64(): Count trailing zeros.
* bpf_ffs64(): Find first set bit, 1-based index, returns 0 when input is 0.
* bpf_fls64(): Find last set bit, 1-based index.
* bpf_bitrev64(): Reverse bits.
* bpf_popcnt64(): Population count.
* bpf_rol64(): Rotate left.
* bpf_ror64(): Rotate right.

Defined zero-input behavior:

* bpf_clz64(0) = 64
* bpf_ctz64(0) = 64
* bpf_ffs64(0) = 0
* bpf_fls64(0) = 0

bpf_ffs64() was previously discussed in
"bpf: Add generic kfunc bpf_ffs64()" [1].

The main concern in that discussion was ABI overhead: a regular kfunc call
follows the BPF calling convention and can introduce extra spill/fill compared
to dedicated instructions.

This series keeps the user-facing API as kfuncs while avoiding that overhead
on hot paths. When the JIT/backend and CPU support it, calls are inlined into
native instructions; otherwise they fall back to regular function calls.

Links:
[1] https://lore.kernel.org/bpf/20240131155607.51157-1-hffilwlqm@xxxxxxxxx/

Changes:
v1 -> v2:
* Drop RFC.
* Add __cpu_feature annotation for CPU-feature-gated tests.
* Add JIT disassembly tests for 64-bit bitops kfuncs
* Address comments from Alexei:
  * Drop KF_MUST_INLINE.
  * Drop internal BPF_ALU64 opcode BPF_BITOPS.
  * Mark all of the kfuncs as fastcall and do push/pop in JIT when necessary.
* v1: https://lore.kernel.org/bpf/20260209155919.19015-1-leon.hwang@xxxxxxxxx/

Leon Hwang (6):
  bpf: Introduce 64-bit bitops kfuncs
  bpf, x86: Add 64-bit bitops kfuncs support for x86_64
  bpf, arm64: Add 64-bit bitops kfuncs support
  selftests/bpf: Add tests for 64-bit bitops kfuncs
  selftests/bpf: Add __cpu_feature annotation for CPU-feature-gated
    tests
  selftests/bpf: Add JIT disassembly tests for 64-bit bitops kfuncs

 arch/arm64/net/bpf_jit_comp.c                 | 123 ++++++++++++
 arch/x86/net/bpf_jit_comp.c                   | 141 +++++++++++++
 include/linux/filter.h                        |  10 +
 kernel/bpf/core.c                             |   6 +
 kernel/bpf/helpers.c                          |  50 +++++
 kernel/bpf/verifier.c                         |  53 ++++-
 .../testing/selftests/bpf/bpf_experimental.h  |   9 +
 .../testing/selftests/bpf/prog_tests/bitops.c | 188 ++++++++++++++++++
 tools/testing/selftests/bpf/progs/bitops.c    |  68 +++++++
 .../testing/selftests/bpf/progs/bitops_jit.c  | 153 ++++++++++++++
 tools/testing/selftests/bpf/progs/bpf_misc.h  |   7 +
 tools/testing/selftests/bpf/test_loader.c     | 150 ++++++++++++++
 12 files changed, 957 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/bitops.c
 create mode 100644 tools/testing/selftests/bpf/progs/bitops.c
 create mode 100644 tools/testing/selftests/bpf/progs/bitops_jit.c

-- 
2.52.0