Re: [PATCH bpf-next v4 0/6] Mixing bpf2bpf and tailcalls for RV64

[Pu Lehui]

Hi Björn,

Sorry for late reply.

On 2026/6/29 22:21, Björn Töpel wrote:
> Hey!
>
> On Mon, 29 Jun 2026 at 15:56, Pu Lehui <pulehui@xxxxxxxxxxxxxxx> wrote:
> > From: Pu Lehui <pulehui@xxxxxxxxxx>
> >
> > In the current RV64 JIT, if we just don't initialize the TCC in subprog,
> > the TCC can be propagated from the parent process to the subprocess, but
> > the updated TCC of the parent process cannot be restored when the
> > subprocess exits. Since the RV64 TCC is initialized before saving the
> > callee saved registers into the stack, we cannot use the callee saved
> > register to pass the TCC, otherwise the original value of the callee
> > saved register will be destroyed. So we implemented mixing bpf2bpf and
> > tailcalls similar to x86_64, i.e. using a non-callee saved register to
> > transfer the TCC between functions, and saving that register to the
> > stack to protect the TCC value. As for the tailcall hierarchy issue,
> > inspired by the s390's low-overhead approach, we store TCC from
> > RV_REG_TCC back to stack after calling bpf2bpf call or calling orig bpf
> > func in bpf trampoline.
> >
> > In addition, some code cleans are also attached to this patchset.
> >
> > Tests test_bpf.ko and test_verifier have passed, as well as the relative
> > testcases of test_progs*.
> >
> > v4:
> > - Fix tailcall hierarchy issue.
> > - use is_struct_ops_tramp helper in bpf trampoline
>
> Alexei outlined some (major) concerns on the whole mixing side of
> things [0]. I haven't been following that space -- did Leon have a

Yes, Leon addressed this issue years ago [0]. He solved it by passing 
the TCC PTR to the subprog, ensuring that the updated TCC is correctly 
propagated back upon the subprog's return. It's time for riscv64 to 
implement this feature.

[0] 
https://lore.kernel.org/all/CAADnVQ+TOFOA0Ya4qzrLDFV331uxZy8ay0svBMt-2Sh5E0tr2A@xxxxxxxxxxxxxx/

> workaround/fix? Can you explain more how you fixed it, and why it
> would make sense to add it to RISC-V now?

When dealing with the tailcall hierarchy issue, x64 and arm64 check if a 
register is a TCC PTR right in the function prologue, which may adds 
some overhead. To get around this, riscv64 plan to pass the updated TCC 
value to the stack using the TCC register after calling a subprog or a 
trampoline's orig_call, which something similar to s390.

Because of this, we have to use a caller-saved (non-callee-saved) 
register for the TCC. If we used a callee-saved one, the old TCC value 
would just get restored by mistake when the function returns.

As for the overall plan for mixing tailcalls and bpf2bpf, the main idea 
is to keep the TCC value on the stack and pass it between function calls 
using that caller-saved TCC register. Since we need to grab the latest 
updated TCC value after calling a subprog or a tailcall leaf node, we 
have to make sure the prologue always pushes the TCC to the stack, and 
the epilogue always restores it from the stack back into the TCC register.

> Thanks,
> Björn
>
>
> [0] https://lore.kernel.org/bpf/CAADnVQ+rLneO4t=YYmLYtc945Fz0=ucNTWZBxgvs8toFY-onRg@xxxxxxxxxxxxxx/