Re: [PATCH bpf-next] bpf: Disable preemption while a raw tracepoint regs slot is in use

From: Andrii Nakryiko

Date: Mon Nov 16 2026 - 13:21:44 EST


On Mon, Nov 16, 2026 at 9:18 AM Mike Bazov via B4 Relay
<devnull+mike.bazov.gmail.com@xxxxxxxxxx> wrote:
>
> From: Mike Bazov <mike.bazov@xxxxxxxxx>
>
> bpf_perf_event_output(), bpf_get_stackid() and bpf_get_stack() of raw
> tracepoint and tracing programs have no pt_regs, so they fill in one of
> three per-CPU slots, picked by a per-CPU nest counter:
>
> get_bpf_raw_tp_regs() level++, use regs[level - 1]
> put_bpf_raw_tp_regs() level--
>
> That only works if the users of a CPU leave in the reverse order they
> came, as a task, an interrupt and an NMI do. These programs run with
> migration disabled but preemption enabled, so tasks can leave in any
> order:
>
> task A task B task C
> get: regs[0]
> <preempted>
> get: regs[1]
> <preempted>
> put: level 1
> get: regs[1]
> fills in its regs
> uses regs[1]
>
> B sends a sample or a stack trace with the registers of C. Profilers and
> tracers then attribute it to the wrong task, and nothing reports an
> error. The per-CPU recursion counter of a program doesn't prevent it:
> the slots are shared by all programs, so it takes two of them.
>
> Disable preemption from get to put, as the two other users of this
> scheme do since commit f2c67a3e60d1 ("bpf: Disable preemption in
> bpf_perf_event_output") and commit d62cc390c2e9 ("bpf: Disable
> preemption in bpf_event_output"). Nothing in between sleeps: the
> sleepable bpf_get_stack() of these programs doesn't use the slots.
>
> The cost is small: the helpers already disable preemption around most
> of their work, the unwind in bpf_get_stack() and bpf_get_stackid() and
> all of bpf_perf_event_output(); this adds the few instructions around
> it.
>
> Signed-off-by: Mike Bazov <mike.bazov@xxxxxxxxx>
> ---
> A reproducer, x86-64: two raw tracepoint programs on sys_enter call
> bpf_perf_event_output() once each, with the program's number as data.
> Two threads make syscalls on CPU 0, and a SCHED_FIFO thread preempts
> them every 20us. The IP of a sample is regs->ip of the slot, the return
> address of the helper call. The reproducer reads that address for both
> programs from their JIT images and checks every sample.
>
> Without this patch, 7 to 18 samples per program and 10s run have the IP
> of the other program, out of 1.4M. With it, none. bpf_get_stack() and
> bpf_get_stackid() in the same setup return an empty kernel stack
> instead: the unwinder looks for the other task's sp on the current
> stack and doesn't find it.
>
> 5 runs of 10s, debug config, CONFIG_PREEMPT=y, preempt=full, on
> bpf-next 0fb18ce8a2e9:
>
> without with
> bpf_perf_event_output, wrong IP 7-18 0 per program
> bpf_get_stack, empty stacks 362-535 0
> bpf_get_stackid, empty stacks 168-364 0
>
> The bpf_get_stackid() case with bpftrace, whose kstack calls it; @a[]
> and @b[] count the empty stacks:
>
> bpftrace -e '
> rawtracepoint:sys_enter /comm == "dd"/ { @a[kstack(1)] = count(); }
> rawtracepoint:sys_enter /comm == "dd"/ { @b[kstack(1)] = count(); }' &
> taskset -c 0 dd if=/dev/zero of=/dev/null bs=1 &
> taskset -c 0 dd if=/dev/zero of=/dev/null bs=1 &
> taskset -c 0 chrt -f 1 perl -e 'select(undef, undef, undef, 2e-5) while 1' &
> ---
> kernel/trace/bpf_trace.c | 11 +++++++++--
> 1 file changed, 9 insertions(+), 2 deletions(-)
>

this looks fine per se, but I'm just questioning why do we need all
those shenanigans with per-cpu stack of pt_regs and why we don't just
have local bpf_raw_tp_regs where we need them? seems like pt_regs are
around 200-300 bytes, that doesn't seem ridiculous to have them on the
stack

anyone would object if we just has a simple and straightforward local
variable and get rid of per-cpu hacks?


| Arch | pt_regs | BPF ctx |
|-------------|-----------------|-----------------------|
| x86_64 | 168 | pt_regs |
| i386 | 68 | pt_regs |
| arm64 | 336/320 | user_pt_regs, 272 |
| arm | 72 | pt_regs |
| riscv64 | 288 | user_regs_struct, 256 |
| riscv32 | 144 | user_regs_struct, 128 |
| powerpc64 | 384 | user_pt_regs, 352 |
| powerpc32 | 176/192/224/240 | user_pt_regs, 176 |
| s390x | 200 | user_pt_regs, 152 |
| loongarch64 | 320 | user_pt_regs, 360 |
| mips64 | 304/400 | pt_regs |
| mips32 | 184 | pt_regs |
| sparc64 | 160 | pt_regs |
| sparc32 | 80 | pt_regs |



> diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
> index b4ff9d4e8..eb3daf2bf 100644
> --- a/kernel/trace/bpf_trace.c
> +++ b/kernel/trace/bpf_trace.c
> @@ -1639,11 +1639,17 @@ static DEFINE_PER_CPU(struct bpf_raw_tp_regs, bpf_raw_tp_regs);
> static DEFINE_PER_CPU(int, bpf_raw_tp_nest_level);
> static struct pt_regs *get_bpf_raw_tp_regs(void)
> {
> - struct bpf_raw_tp_regs *tp_regs = this_cpu_ptr(&bpf_raw_tp_regs);
> - int nest_level = this_cpu_inc_return(bpf_raw_tp_nest_level);
> + struct bpf_raw_tp_regs *tp_regs;
> + int nest_level;
> +
> + /* The slots nest: a task must not be preempted while it holds one */
> + preempt_disable();
> + tp_regs = this_cpu_ptr(&bpf_raw_tp_regs);
> + nest_level = this_cpu_inc_return(bpf_raw_tp_nest_level);
>
> if (nest_level > ARRAY_SIZE(tp_regs->regs)) {
> this_cpu_dec(bpf_raw_tp_nest_level);
> + preempt_enable();
> return ERR_PTR(-EBUSY);
> }
>
> @@ -1653,6 +1659,7 @@ static struct pt_regs *get_bpf_raw_tp_regs(void)
> static void put_bpf_raw_tp_regs(void)
> {
> this_cpu_dec(bpf_raw_tp_nest_level);
> + preempt_enable();
> }
>
> BPF_CALL_5(bpf_perf_event_output_raw_tp, struct bpf_raw_tracepoint_args *, args,
>
> ---
> base-commit: 8835113107b2cf75df33a71e265cdeec5d5d13e0
> change-id: 20261116-bpf-raw-tp-regs-preempt-1f7019989ab9
>
> Best regards,
> --
> Mike Bazov <mike.bazov@xxxxxxxxx>
>
>