[PATCH v3 01/19] perf/core: Protect per-CPU callchain buffer across bpf_overflow_handler()
From: Ian Rogers
Date: Mon Nov 16 2026 - 02:45:20 EST
In bpf_overflow_handler(), perf_prepare_sample() runs before bpf_prog_run()
and saves data->callchain pointing into the current CPU's callchain buffer
for the active interrupt_context_level(), after get_perf_callchain() has
already released its callchain_recursion slot.
If the attached BPF_PROG_TYPE_PERF_EVENT program invokes a helper at the
same interrupt_context_level() that calls get_callchain_entry() (such as
bpf_perf_event_output() on a PERF_COUNT_SW_BPF_OUTPUT event configured with
PERF_SAMPLE_CALLCHAIN and a larger stack, or bpf_get_task_stack()), it
re-acquires the same per-CPU callchain buffer and overwrites
data->callchain->nr in place. When bpf_prog_run() returns non-zero and
__perf_event_output() emits the outer event's sample, perf_output_begin()
reserves only the smaller dyn_size computed earlier while
perf_output_sample() copies the overwritten larger data->callchain->nr
entries, overflowing the ring-buffer reservation.
Hold the callchain recursion slot via get_callchain_entry() across
bpf_prog_run() when PERF_SAMPLE_CALLCHAIN has been prepared so nested
callers at the same context level cannot overwrite the active per-CPU
callchain buffer.
Fixes: 0eed28220598 ("perf/core: Call perf_prepare_sample() before running BPF")
Signed-off-by: Ian Rogers <irogers@xxxxxxxxxx>
---
kernel/events/core.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 614f9f0e6500..fd9f654de269 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -10818,8 +10818,15 @@ static int bpf_overflow_handler(struct perf_event *event,
rcu_read_lock();
prog = READ_ONCE(event->prog);
if (prog) {
+ struct perf_callchain_entry *callchain = NULL;
+ int callchain_rctx;
+
perf_prepare_sample(data, event, regs);
+ if (data->sample_flags & PERF_SAMPLE_CALLCHAIN)
+ callchain = get_callchain_entry(&callchain_rctx);
ret = bpf_prog_run(prog, &ctx);
+ if (callchain)
+ put_callchain_entry(callchain_rctx);
}
rcu_read_unlock();
out:
--
2.56.1.703.gf848ed807c-goog