Re: [PATCH 0/2] arm64: ftrace: support DIRECT_CALLS without CALL_OPS

[Mark Rutland]

On Tue, Jun 09, 2026 at 05:19:25AM +0000, Jose Fernandez (Anthropic) wrote:
> On arm64, HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS is currently selected
> only when DYNAMIC_FTRACE_WITH_CALL_OPS is available. CALL_OPS, in
> turn, is mutually exclusive with kCFI: the pre-function NOPs it needs
> would change the offset of the pre-function type hash (see
> baaf553d3bc3 ("arm64: Implement HAVE_DYNAMIC_FTRACE_WITH_CALL_OPS")),
> and the compiler support needed to reconcile the two does not exist
> yet.
> 
> The result is that a CONFIG_CFI=y arm64 kernel has no
> ftrace direct calls at all, so register_fentry() fails with -ENOTSUPP
> and no BPF trampoline can attach: fentry/fexit, fmod_ret and BPF LSM
> programs are all unavailable. Deployments that want both kCFI
> hardening and BPF-based security monitoring currently have to give
> one of them up. systemd's bpf-restrict-fs feature hits this today:
> https://lore.kernel.org/all/20250610232418.GA3544567@ax162/
> 
> CALL_OPS is an optimization for direct calls, not a dependency.
> In-BL-range trampolines are reached by a direct branch without
> consulting the ops pointer, and out-of-range trampolines already
> fall back to ftrace_caller, where the DIRECT_CALLS machinery
> (call_direct_funcs() storing the trampoline in ftrace_regs, the
> ftrace_caller tail-call) is gated on DIRECT_CALLS alone. 

It's true that this is not a strict functional dependency, but there is
a substantial performance cliff here.

With CALL_OPS we only fall back to the ftrace_ops_list_func() when there
are multiple ops associated with a given callsite. Where we have a
single ops for a callsite, if the direct call is out of range, the
ftrace caller will call trace_ops::direct_call.

Without CALL_OPS, if there are unrelated ops registered (attached to
difrerent callsites), the ftrace_caller trampoline will call
ftrace_ops_list_func(), and we'll iterate over all of the registered ops
to find the relevant ops and trampoline, every time an out-of-range
direct call is used.

So the big questions are:

(1) How often is a direct call being made?
(2) Where does BPF allocate its direct call trampolines?

>From a skim of arch/arm64/mm/init.c, EXECMEM_BPF covers
VMALLOC_START..VMALLOC_END, the vast majority of which is out-of-range
for a BL from the core kernel or a module. So if EXECMEM_BPF is used,
that's likely to be problematic.  Hopefully I'm missing something such
that BPF trampolines are almost always allocated in-range of a BL?

When we originally implemented CALL_OPS and DIRECT_CALLS for arm64, the
understanding was that we'd have many ops with direct calls. We believed
that the ftrace_ops_list_func() penalty was so substantial that it
wasn't worthwhile supporting direct calls in a configuration where
bouncing through that would be common.

If we need to support this configuration, then I strongly suspect we
need to make other changes too in order for it to be worthwhile.

> s390 and loongarch ship HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS this
> way, without having CALL_OPS at all.

Following from the comment regarding EXECMEM_BPF above, it looks like
both s390 and loongarch don't define EXECMEM_BPF, and only define
EXECMEM_DEFAULT with a subsstantially limited range, which I suspect
keeps their direct calls within range.

> Patch 1 prepares ftrace_modify_call() to build without CALL_OPS by
> widening its #ifdef and using the existing ftrace_rec_update_ops()
> wrapper (no functional change for current configurations). Patch 2
> drops the CALL_OPS requirement from the DIRECT_CALLS select.
> 
> Configurations that keep CALL_OPS (clang !CFI, and GCC without
> CC_OPTIMIZE_FOR_SIZE) are unchanged. We verified this: in an arm64
> clang build, every object file is byte-identical before and after
> the series except ftrace.o itself, and its disassembly is identical.
> CFI builds (and GCC -Os builds) gain working direct calls, with
> out-of-range attachments taking the ftrace_caller dispatch path
> instead of the per-callsite fast path.

Can you comment on how you've tested in-range vs out-of-range
attachements here?

> We tested on a 6.18.y-based kernel and on this base with clang
> kCFI builds (CONFIG_CFI=y, enforcing) under qemu (TCG, and KVM on an
> arm64 host) and on GB200-based arm64 hardware: fentry/fexit, fmod_ret
> and BPF LSM programs load, attach and execute; the ftrace-direct
> sample modules (including both modify samples, exercising
> ftrace_modify_call()) run cleanly; no CFI violations observed. The
> fentry_test, fexit_test, fentry_fexit, fexit_sleep, fexit_stress,
> modify_return, tracing_struct, lsm and trampoline_count selftests and
> the ftrace direct-call selftests (test.d/direct) pass on the new
> configuration with results identical to a CALL_OPS kernel built from
> the same tree, and a broader test_progs sweep showed no differences
> attributable to this series. Without the series, all of the above
> fail at attach time with -ENOTSUPP.

Have you made any performance measurements, or was this just functional
enablement?

Mark.

> riscv has the same gap (its DIRECT_CALLS select also requires
> CALL_OPS, and its CALL_OPS is likewise !CFI); if this approach is
> acceptable for arm64 we can follow up there.
> 
> ---
> Jose Fernandez (Anthropic) (2):
>       arm64: ftrace: prepare ftrace_modify_call() for use without CALL_OPS
>       arm64: ftrace: allow DIRECT_CALLS without CALL_OPS
> 
>  arch/arm64/Kconfig         | 2 +-
>  arch/arm64/kernel/ftrace.c | 5 +++--
>  2 files changed, 4 insertions(+), 3 deletions(-)
> ---
> base-commit: 254f49634ee16a731174d2ae34bc50bd5f45e731
> change-id: 20260607-arm64-ftrace-direct-calls-152230ef7077
> 
> Best regards,
> --  
> Jose Fernandez (Anthropic) <jose.fernandez@xxxxxxxxx>
>