Re: [PATCH bpf-next v3 2/4] bpf: Mark programs whose runs must not overlap on a CPU
From: JP Kobryn (CrowdStrike)
Date: Thu Nov 19 2026 - 00:05:54 EST
On 11/17/26 3:29 AM, Mike Bazov via B4 Relay wrote:
> From: Mike Bazov <mikeba@xxxxxxxxxxxxxxx>
>
> The per-CPU recursion counter of tracing and raw tracepoint programs
> also keeps a second task out of a program while another task is
> preempted inside it on the same CPU. Programs rely on that:
>
> - A per-CPU map value that a program looked up stays its own until the
> run ends, which is how programs use these maps as scratch space.
> - The budget of timed may_goto is per-CPU memory that the first insns
> of a run reset, see commit ec274bb4665c ("bpf: One may_goto time
> slice per prog invocation"). A second run on the CPU gives the first
> one more time.
> - A private stack is per CPU. After the previous patch, struct_ops
> programs that request one are the only programs with this check that
> still have it.
>
> The next patch checks recursion per task, which lets a second task in.
> For these programs that would turn a counted, dropped run into silently
> corrupted data, or into a may_goto loop that another task keeps
> renewing. Mark them at load time, so that they keep the per-CPU counter.
>
> Set prog->recur_per_cpu when the verifier sees:
>
> - a helper called with a per-CPU map, also through an inner map,
> - the address of a per-CPU array value loaded directly,
> - bpf_this_cpu_ptr() or bpf_per_cpu_ptr(),
> - a tail call: the target runs without its own recursion check,
> - timed may_goto,
> - a private stack, in a struct_ops program that requests one.
This feels brittle. As code evolves, it may be hard to keep track of these.
Would explicitly opting in avoid unexpected regressions?
If the marking approach is accepted, would it make sense to make it visible
via bpftool?
>
> Only verified code counts: a per-CPU map referenced in a branch that a
> constant in .rodata disables doesn't mark the program.
>
> No functional change: nothing reads the flag yet.
>
> Trade-offs, once the next patch reads it:
>
> - A program that keeps per-CPU slots of its own, e.g. in a regular
> array indexed by bpf_get_smp_processor_id(), is not marked.
> - Marked programs keep losing runs to preemption.
>
> Signed-off-by: Mike Bazov <mikeba@xxxxxxxxxxxxxxx>
> ---
> include/linux/bpf.h | 3 ++-
> kernel/bpf/fixups.c | 1 +
> kernel/bpf/verifier.c | 11 +++++++++++
> 3 files changed, 14 insertions(+), 1 deletion(-)
>
> diff --git a/include/linux/bpf.h b/include/linux/bpf.h
> index a5ad08b7937c..7873f0ef2817 100644
> --- a/include/linux/bpf.h
> +++ b/include/linux/bpf.h
> @@ -2063,7 +2063,8 @@ struct bpf_prog {
> call_get_func_ip:1, /* Do we call get_func_ip() */
> call_session_cookie:1, /* Do we call bpf_session_cookie() */
> tstamp_type_access:1, /* Accessed __sk_buff->tstamp_type */
> - sleepable:1; /* BPF program is sleepable */
> + sleepable:1, /* BPF program is sleepable */
> + recur_per_cpu:1; /* Runs must not overlap on a CPU */
> enum bpf_prog_type type; /* Type of BPF program */
> enum bpf_attach_type expected_attach_type; /* For some prog types */
> u32 len; /* Number of filter blocks */
> diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
> index 4d9b67981520..82ad0d3bf465 100644
> --- a/kernel/bpf/fixups.c
> +++ b/kernel/bpf/fixups.c
> @@ -2591,6 +2591,7 @@ int bpf_do_misc_fixups(struct bpf_verifier_env *env)
> if (!tmg)
> return -ENOMEM;
> prog->aux->timed_may_goto = tmg;
> + prog->recur_per_cpu = true;
> }
> prog->jit_required = 1;
> cnt = may_goto_expand_timed(insn_buf, &jmp, tmg, reg, save_off);
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 2ca1469d8dee..3d54520df3e3 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -6050,6 +6050,7 @@ static int check_max_stack_depth(struct bpf_verifier_env *env)
> if (round_up_stack_depth(env, si[s].stack_depth) >= BPF_PRIV_STACK_MIN_SIZE) {
> si[s].priv_stack_mode = PRIV_STACK_ADAPTIVE;
> env->prog->aux->jits_use_priv_stack = true;
> + env->prog->recur_per_cpu = true;
> }
> }
>
> @@ -12916,6 +12917,10 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
> if (err)
> return err;
>
> + /* A per-CPU value stays the run's own only while no other run starts on the CPU */
> + if (meta.map.ptr && bpf_map_is_percpu_map(meta.map.ptr->map_type))
> + env->prog->recur_per_cpu = true;
> +
> err = record_func_map(env, &meta, func_id, insn_idx);
> if (err)
> return err;
> @@ -12942,6 +12947,8 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
>
> switch (func_id) {
> case BPF_FUNC_tail_call:
> + /* The target runs without a recursion check of its own */
> + env->prog->recur_per_cpu = true;
> err = check_resource_leak(env, false, true, "tail_call");
> if (err)
> return err;
> @@ -13048,6 +13055,8 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
> struct bpf_reg_state *reg = ®s[BPF_REG_1];
> const struct btf_type *type;
>
> + env->prog->recur_per_cpu = true;
> +
> if (reg->type & MEM_RCU) {
> type = btf_type_by_id(reg->btf, reg->btf_id);
> if (!type || !btf_type_is_struct(type)) {
> @@ -19690,6 +19699,8 @@ static int check_ld_imm(struct bpf_verifier_env *env, struct bpf_insn *insn)
> __mark_reg_known(dst_reg, aux->map_off);
> dst_reg->type = PTR_TO_MAP_VALUE;
> dst_reg->map_ptr = map;
> + if (bpf_map_is_percpu_map(map->map_type))
> + env->prog->recur_per_cpu = true;
> WARN_ON_ONCE(map->map_type != BPF_MAP_TYPE_INSN_ARRAY &&
> map->max_entries != 1);
> /* We want reg->id to be same (0) as map_value is not distinct */
>