[PATCH bpf-next v3 0/3] bpf: Use an on-stack pt_regs in the raw tracepoint helpers
From: Mike Bazov via B4 Relay
Date: Fri Nov 20 2026 - 05:59:53 EST
bpf_perf_event_output(), bpf_get_stackid() and bpf_get_stack() of raw
tracepoint and tracing programs share three per-CPU pt_regs slots,
picked by a nest counter. These programs can be preempted, and tasks
then leave the slots out of order, so a sample or stack trace of one
task gets the registers of another. Patch 3 fills in a pt_regs on the
stack of each helper instead and removes the slots.
The slots also limited how deep these helpers nest. Two things need a
limit of their own without them, and both can already be hit today
without raw tracepoint programs:
- bpf_perf_event_output() WARNs when entered a fourth time on a CPU.
- Build ID collection in bpf_get_stack()/bpf_get_stackid() and
bpf_find_vma() take mmap_read_trylock(), which fires the mmap_lock
tracepoints. Programs on them that call these helpers nest into each
other as deep as there are programs. Patch 2 limits it to three
levels.
Results on bpf-next 0fb18ce8a2e9, x86-64, debug config, CONFIG_PREEMPT=y,
preempt=full. "base" is that tree, "v2" adds only patch 3, "v3" all
three.
The slot race, 5 runs of 10s each. Two raw tracepoint programs on
sys_enter call the helper, two threads make syscalls on CPU 0 and a
SCHED_FIFO thread preempts them every 20us. For bpf_perf_event_output()
the reproducer reads the return address of each program's helper call
from its JIT image and checks the IP of every sample; bpf_get_stack()
and bpf_get_stackid() get an empty kernel stack instead, because the
unwinder looks for the other task's sp on the current stack:
base v3
bpf_perf_event_output, wrong IP 7-18 0 per program
bpf_get_stack, empty stacks 362-535 0
bpf_get_stackid, empty stacks 168-364 0
The bpf_get_stackid() case with bpftrace, whose kstack calls it; @a[]
and @b[] count the empty stacks:
bpftrace -e '
rawtracepoint:sys_enter /comm == "dd"/ { @a[kstack(1)] = count(); }
rawtracepoint:sys_enter /comm == "dd"/ { @b[kstack(1)] = count(); }' &
taskset -c 0 dd if=/dev/zero of=/dev/null bs=1 &
taskset -c 0 dd if=/dev/zero of=/dev/null bs=1 &
taskset -c 0 chrt -f 1 perl -e 'select(undef, undef, undef, 2e-5) while 1' &
Nesting. For the WARN, a raw tracepoint or kprobe program calls
bpf_perf_event_output(), and three fentry programs on
perf_event_output() call it too. For mmap_read_trylock(), six programs
on mmap_lock_acquire_returned call bpf_get_stack() with build IDs, or
bpf_find_vma(), while the process does 1000 mmap()/munmap():
base v2 v3
WARN, started from raw tracepoint no yes no
WARN, started from kprobe yes - no
bpf_get_stack, build IDs: depth 4 6 4
calls 1.5M 5.9M 1.5M
bpf_find_vma: depth 6 6 4
calls 5.9M 5.9M 1.5M
Depth counts the programs on the stack; at 4, the last one gets -EBUSY
or the fallback to IPs. The kprobe case wasn't run on v2, which doesn't
change that path.
---
Changes in v3:
- New patch 1: don't WARN when bpf_perf_event_output() nests too deep
(reported by the BPF CI bot).
- New patch 2: limit nesting of mmap_read_trylock() in BPF helpers
(per Alexei's suggestion).
- Patch 3: correct the commit message, bpf_perf_event_output() keeps
its own nesting limit.
- Link to v2: https://lore.kernel.org/r/20261119-bpf-raw-tp-regs-preempt-v2-1-e1e1b90857d6@xxxxxxxxx
Changes in v2:
- Use a pt_regs on the stack of each helper and remove the per-CPU
slots, instead of disabling preemption while a slot is in use
(per Andrii's suggestion).
- Link to v1: https://lore.kernel.org/r/20261116-bpf-raw-tp-regs-preempt-v1-1-d01b6dd8573c@xxxxxxxxx
---
Mike Bazov (3):
bpf: Don't WARN when bpf_perf_event_output() nests too deep
bpf: Limit nesting of mmap_read_trylock() in BPF helpers
bpf: Use an on-stack pt_regs in the raw tracepoint helpers
kernel/bpf/mmap_unlock_work.h | 16 +++++++--
kernel/trace/bpf_trace.c | 80 +++++++------------------------------------
2 files changed, 26 insertions(+), 70 deletions(-)
---
base-commit: 8835113107b2cf75df33a71e265cdeec5d5d13e0
change-id: 20261116-bpf-raw-tp-regs-preempt-1f7019989ab9
Best regards,
--
Mike Bazov <mike.bazov@xxxxxxxxx>