[PATCH bpf 2/2] LoongArch: BPF: Don't charge an empty prog_array slot to the tail call count

[George Guo]

From: George Guo <guodongtai@xxxxxxxxxx>

emit_bpf_tail_call() bumped the tail call count and stored it back to
*tcc_ptr before loading array->ptrs[index] and testing it for NULL.  A
tail call that targets an empty slot therefore consumed one unit of the
tail call budget even though control never transferred.

The interpreter increments tail_call_cnt only after the prog pointer is
found to be non-NULL (kernel/bpf/core.c, BPF_TAIL_CALL), so a fall-through
to an empty slot leaves the count untouched.  The JIT must do the same.

This is visible with selftests/bpf tailcalls/tailcall_3, whose entry prog
tail-calls an empty slot before the real target: the observed count is 32
instead of the expected 33.

Defer the store of the bumped count until after the NULL check.  The limit
comparison is unchanged: t3 = *tcc_ptr + 1, and "t3 > MAX_TAIL_CALL_CNT" is
equivalent to "*tcc_ptr >= MAX_TAIL_CALL_CNT".

The check-before-NULL ordering dates back to the original JIT; commit
c0fcc955ff82 ("LoongArch: BPF: Fix the tailcall hierarchy") reworked the
counter into the *tcc_ptr form but preserved the same ordering.

Fixes: 5dc615520c4d ("LoongArch: Add BPF JIT support")
Cc: stable@xxxxxxxxxxxxxxx
Signed-off-by: George Guo <guodongtai@xxxxxxxxxx>
---
 arch/loongarch/net/bpf_jit.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c
index f705de099f23..f2aa0b7f65ad 100644
--- a/arch/loongarch/net/bpf_jit.c
+++ b/arch/loongarch/net/bpf_jit.c
@@ -323,13 +323,18 @@ static int emit_bpf_tail_call(struct jit_ctx *ctx, int insn)
 		goto toofar;
 
 	/*
-	 * if ((*tcc_ptr)++ >= MAX_TAIL_CALL_CNT)
+	 * if (*tcc_ptr + 1 > MAX_TAIL_CALL_CNT)
 	 *      goto out;
+	 *
+	 * Compute the bumped count but do not write it back yet: the
+	 * interpreter increments tail_call_cnt only after the prog pointer is
+	 * found to be non-NULL, so a tail call to an empty slot must not
+	 * consume the tail call budget.  The store is deferred until the call
+	 * is known to be taken (below).
 	 */
 	emit_insn(ctx, ldd, REG_TCC, LOONGARCH_GPR_SP, tcc_ptr_off);
 	emit_insn(ctx, ldd, t3, REG_TCC, 0);
 	emit_insn(ctx, addid, t3, t3, 1);
-	emit_insn(ctx, std, t3, REG_TCC, 0);
 	emit_insn(ctx, addid, t2, LOONGARCH_GPR_ZERO, MAX_TAIL_CALL_CNT);
 	if (emit_tailcall_jmp(ctx, BPF_JSGT, t3, t2, jmp_offset) < 0)
 		goto toofar;
@@ -346,6 +351,9 @@ static int emit_bpf_tail_call(struct jit_ctx *ctx, int insn)
 	if (emit_tailcall_jmp(ctx, BPF_JEQ, t2, LOONGARCH_GPR_ZERO, jmp_offset) < 0)
 		goto toofar;
 
+	/* (*tcc_ptr)++; the tail call is taken, so commit the bumped count */
+	emit_insn(ctx, std, t3, REG_TCC, 0);
+
 	/* goto *(prog->bpf_func + 4); */
 	off = offsetof(struct bpf_prog, bpf_func);
 	emit_insn(ctx, ldd, t3, t2, off);
-- 
2.25.1