Re: [PATCH bpf-next v5 1/3] bpf, x86: Support fetching AND/OR/XOR atomics in arena

[Tejun Heo]

On Fri, Oct 16, 2026 at 10:13:12AM -0700, Puranjay Mohan wrote:
> x86-64 has no single instruction for a fetching AND/OR/XOR, so the JIT
> lowers them to a CMPXCHG loop. That loop could not be used against arena
> memory: it contains two memory accesses, the load of the old value and
> the CMPXCHG itself, either of which can fault when the arena page goes
> away, while the verifier reserves only one exception table entry per
> instruction. bpf_jit_supports_insn() therefore rejected the three opcodes
> and such programs failed to load with
>
>   BPF_ATOMIC stores into R1 arena is not allowed
>
> Emit the same loop with R12-indexed addressing and give each of the two
> accesses its own exception table entry. The verifier still reserves one
> per instruction, so the JIT accounts for the extra entry itself by
> rescanning the instruction stream in bpf_int_jit_compile() before the
> extable is sized, as s390 does in bpf_jit_alloc() for its BPF_XCHG
> lowering. The rescan sits after the extra-pass early exit, so a program
> is counted exactly once. Drop the bpf_jit_supports_insn() rejection.
>
> A loop without the load would need only one entry, since a CMPXCHG that
> loses the comparison reloads RAX, but its first CMPXCHG would then almost
> always lose. Every operation would pay for a second locked CMPXCHG, which
> makes the fetching insn more expensive than the load and CMPXCHG loop
> programs write by hand today. The second entry is paid for once, in the
> JIT.
>
> Both entries resume past the whole loop rather than past the faulting
> instruction alone, with the fetch destination cleared, so a fault cannot
> re-enter the loop, and both are reported as writes since the BPF insn is
> a read-modify-write. This repurposes the INSN_LEN field of the fixup as a
> resume distance rather than an instruction length; a sequence long enough
> to overflow its 8 bits would first have to exceed BPF_MAX_INSN_SIZE, which
> do_jit() rejects before the image is used.
>
> The loop needs RAX for CMPXCHG and saves R0 in BPF_REG_AX, which it also
> substitutes for R0 when either operand is R0. ex_handler_bpf() may then
> have to name r10 as the register holding the arena address or the one to
> clear on fault; reg2pt_regs[] already has that entry.
>
> An access through a number, made by a program loaded with
> BPF_F_ARENA_SCALAR, keeps its address in BPF_REG_AX itself (see
> arena_scalar_access()), which is where the loop would save R0. Save R0 on
> the stack in that case, as the div and mod lowering does. An R0 operand
> stays in that stack slot: the new value is built from it, RAX ends up
> holding the old value that R0 is to return, and the slot is dropped
> instead of popped. Both entries resume before that pop or drop, so a
> fault leaves the stack balanced.
>
> Signed-off-by: Puranjay Mohan <puranjay@xxxxxxxxxx>

Acked-by: Tejun Heo <tj@xxxxxxxxxx>

Thanks.

-- 
tejun