Re: [PATCH bpf-next 00/16] Add percpu variable support for bpf arena

[Xu Kuohai]

On 10/20/2026 10:47 PM, Alexei Starovoitov wrote:
> On Tue, Oct 20, 2026 at 06:22 AM Xu Kuohai <xukuohai@xxxxxxxxxxxxxxx> wrote:
> > To use percpu data in a bpf arena today, we need to define an array
> > sized to the maximum possible cpu number, like the cpu_ctxs array
> > defined in tools/sched_ext/scx_qmap.h. This is not a convenient
> > pattern. It lacks performance and scalability, and wastes memory.
>
> It depends on how you use it.
> Patches 10 and 11 emit the same thing as
> &arr[bpf_get_smp_processor_id()]: load the cpu id, multiply, add.
> The x86 verifier and the arm64 JIT inline that helper already.
>
> __u64 hits takes a page per possible cpu with this series and 8 bytes
> per cpu in an array.
>
> cpu_ctxs[] is indexed by cid and qmap_dump_cid() reads other cpus'
> slots from bpf, so scx_qmap cannot use this.
> > Keep it in the bpf prog.
>
> Any arena offset is safe to access, so the kernel doesn't need to know
> about percpu chunks. bpf_arena_alloc.h does it already with
> page_frag_cur_page[NR_CPUS]: a table of pointers and a page that the
> cpu allocates on the first use. NUMA_NO_NODE gives the local node and
> user space reads the table through its mmap.
> Then no need for BPF_PSEUDO_ARENA_PERCPU, the map_create attrs, verifier
> and JIT changes, new libbpf APIs and libbpf changing thread affinity
> during load. What is lost is plain 'hits++', but kernel C doesn't have
> that either. Initial values are left to the prog below.

Another thing I initially counted as a loss is isolation, but since
bpf prog is allowed to read or write any address in the arena, the
isolation is fake.

> Something like this:
>
> +/*
> + * One copy of 'type' per cpu. A cpu allocates its copy on the first use,
> + * so it comes from the local NUMA node and cpus that never run the prog
> + * cost nothing. User space reads any cpu's copy through name[cpu].
> + */
> +#define DEFINE_ARENA_PER_CPU(type, name)				\
> +type __arena * __arena name[NR_CPUS];					\
> +									\
> +static inline type __arena *this_cpu_##name(void)			\
> +{									\
> +	__u32 cpu = bpf_get_smp_processor_id();				\
> +	type __arena *p = name[cpu];					\
> +									\
> +	if (!p) {							\
> +		p = bpf_arena_alloc_pages(&arena, NULL,			\
> +					  round_up(sizeof(type), PAGE_SIZE) / PAGE_SIZE, \
> +					  NUMA_NO_NODE, 0);		\
> +		name[cpu] = p;						\
> +	}								\
> +	cast_kern(p);							\
> +	return p;							\
> +}

Thanks for the sketch. The macro makes senses to me, except two concerns:

1. The table wastes a pointer (8 bytes) for each non-exist CPUs, as the
   global data is populated at load time by libbpf. On my debian machine,
   the CONFIG_NR_CPUS is 8192, while the machine has just a dozen CPUs, so
   the 64 KB bytes per variable are almost all wasted. An alternative way
   is to allocate the pointers at load time.

2. For small variables, the page rounding is a big cost as this series. A
   better way is to make percpu copies on the same NUMA node share the same
   page.

> Orthogonal to that people are working on slab-quality allocator
> for arena. It's also going to be done completely on bpf side.

I noticed the slab allocator work just sent by Emil. I will find out what
percpu support will look like with that.

> pw-bot: cr