[PATCH] bpf: fix dangling jump table entries for verified indirect jump targets

[Sandipan Roy]

Resending with the correct Signed-off-by.  (My earlier send carried a
   placeholder sign-off from a local test setup; please disregard it.)


Hi,

While reviewing the BPF subsystem's recently introduced indirect jump
support (BPF_MAP_TYPE_INSN_ARRAY / `gotox`), I found a verifier soundness
bug: a program that passes verification can, at run time, execute a jump
through a NULL (or stale) code pointer, causing a kernel oops/panic. I
believe this affects all kernels since the feature was added.

The core problem is the interaction between post-verification instruction
removal and the insn_array jump table:

- `bpf_opt_remove_nops()` removes instructions byte-equal to `JA +0` even
  when they are targets of a verified `gotox`, because it does not check
  `insn_aux_data[].indirect_target`.
- When such a target is removed, `bpf_insn_array_adjust_after_remove()`
  marks the map entries `INSN_DELETED`, the JIT skips them and leaves
  `ips[]` NULL (or stale), and `bpf_insn_array_ready()` skips validation
  of `INSN_DELETED` entries.

The result is that a verified `gotox rX` can execute `jmp *rX` with
`rX == NULL`. I reproduced this on the current mainline (v7.3-rc1,
commit bc35965f6940) with a minimal BPF program: the load succeeds and
running it produces "BUG: kernel NULL pointer dereference" with
"#PF: supervisor instruction fetch" at RIP 0x0. A map previously bound
to another program can instead leave a stale pointer into a freed JIT
image (latent use-after-free).

While validating the fix I also noticed a related lifecycle bug:
`bpf_insn_array_release()` is only invoked from verifier error paths, so
after a successful load the map remains permanently "used" (the map can
never be bound to another program, failing with -EBUSY) and `ips[]` keeps
pointing into the freed jitted image of the unloaded program.

The attached patch fixes all of these:

1. Never remove indirect jump targets in `bpf_opt_remove_nops()`, and
   hard-guarantee it in `verifier_remove_insns()` by rejecting any removal
   of an instruction flagged as an indirect jump target.
2. Turn the final safety gate in `bpf_insn_array_ready()` into a hard
   rejection of `INSN_DELETED` entries instead of silently skipping them.
3. Call `bpf_insn_array_release()` from `__bpf_free_used_maps()` when the
   bound program is freed, and clear the stale per-entry state
   (`xlated_off`, `ips[]`) there so a subsequent bind cannot inherit
   dangling pointers.


Thanks,
Sandipan Roy
Senior Product Security Engineer, Red Hat
From 09f9d3fdc383b2b118386824b40337c12acb05c0 Mon Sep 17 00:00:00 2001
From: Sandipan Roy <saroy@xxxxxxxxxx>
Date: Fri, 4 Sep 2026 21:41:42 +0530
Subject: [PATCH] bpf: fix dangling jump table entries for verified indirect
 jump targets

BPF_MAP_TYPE_INSN_ARRAY lets a program perform indirect jumps (`gotox
rX`, BPF_JMP|BPF_JA|BPF_X).  For every map index reachable by rX the
verifier verifies the target instruction, and the JIT later records the
jitted target address in insn_array->ips[].

After verification the privileged pass bpf_opt_remove_nops() removes
instructions byte-equal to JA +0 (NOP) without checking whether they are
targets of a verified indirect jump.  If such a target is removed,
bpf_insn_array_adjust_after_remove() marks the corresponding map entries
INSN_DELETED, the JIT then skips them and leaves ips[] NULL (or stale
from a previously bound program), while bpf_insn_array_ready() skips
validation of INSN_DELETED entries.  The result is a program that passes
verification and, at run time, executes `jmp *rX` with rX == NULL or a
dangling pointer, causing a kernel oops or panic.

Fix it by:
- never removing instructions marked insn_aux_data[].indirect_target in
  bpf_opt_remove_nops();
- rejecting, in verifier_remove_insns(), any removal of an instruction
  flagged as an indirect jump target as a hard guarantee for all removal
  passes;
- turning the final safety gate in bpf_insn_array_ready() into a hard
  rejection of INSN_DELETED entries instead of silently skipping them.

Additionally, bpf_insn_array_release() was only invoked from verifier
error paths, so the insn_array map stayed bound forever after a
successful load: the map's used flag was never cleared, making the map
unusable for subsequent programs, and ips[] kept pointing into the freed
jitted image of the unloaded program.  Call bpf_insn_array_release()
from __bpf_free_used_maps() when the program is freed, and clear the
stale per-entry state (xlated_off and ips[]) there.

Fixes: 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
Signed-off-by: Sandipan Roy <saroy@xxxxxxxxxx>
---
 kernel/bpf/bpf_insn_array.c | 20 +++++++++++++++++++-
 kernel/bpf/core.c           |  8 ++++++++
 kernel/bpf/fixups.c         | 25 ++++++++++++++++++++++++-
 3 files changed, 51 insertions(+), 2 deletions(-)

diff --git a/kernel/bpf/bpf_insn_array.c b/kernel/bpf/bpf_insn_array.c
index a2f84afe6f7c..84983dec6426 100644
--- a/kernel/bpf/bpf_insn_array.c
+++ b/kernel/bpf/bpf_insn_array.c
@@ -215,8 +215,15 @@ int bpf_insn_array_ready(struct bpf_map *map)
 	int i;
 
 	for (i = 0; i < map->max_entries; i++) {
+		/*
+		 * A tracked instruction that was deleted is a broken jump
+		 * table entry: the JIT never fills insn_array->ips[i] for it,
+		 * so a verified `gotox` covering this index would jump through
+		 * a NULL or stale code pointer.  Reject such programs instead
+		 * of silently skipping the entry.
+		 */
 		if (insn_array->values[i].xlated_off == INSN_DELETED)
-			continue;
+			return -EFAULT;
 		if (!insn_array->ips[i])
 			return -EFAULT;
 	}
@@ -227,8 +234,19 @@ int bpf_insn_array_ready(struct bpf_map *map)
 void bpf_insn_array_release(struct bpf_map *map)
 {
 	struct bpf_insn_array *insn_array = cast_insn_array(map);
+	int i;
 
 	atomic_set(&insn_array->used, 0);
+
+	/*
+	 * Reset the per-entry state so a subsequent bind cannot observe a
+	 * stale xlated offset or inherit pointers into a freed jitted image
+	 * of the previously bound program.
+	 */
+	for (i = 0; i < map->max_entries; i++) {
+		insn_array->values[i].xlated_off = insn_array->values[i].orig_off;
+		insn_array->ips[i] = 0;
+	}
 }
 
 void bpf_insn_array_adjust(struct bpf_map *map, u32 off, u32 len)
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index d55e737ed75a..c90fcf2c9592 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -3028,6 +3028,14 @@ void __bpf_free_used_maps(struct bpf_prog_aux *aux,
 			map->ops->map_poke_untrack(map, aux);
 		if (sleepable)
 			atomic64_dec(&map->sleepable_refcnt);
+		if (map->map_type == BPF_MAP_TYPE_INSN_ARRAY)
+			/*
+			 * Drop the bind to the program once it is gone so the
+			 * map can be reused, and clear the stale jitted
+			 * pointers / xlated offsets so they do not outlive
+			 * the bound program.
+			 */
+			bpf_insn_array_release(map);
 		bpf_map_put(map);
 	}
 }
diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index 65b441e4a351..c01b9c084b57 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -516,8 +516,25 @@ static int verifier_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt)
 {
 	struct bpf_insn_aux_data *aux_data = env->insn_aux_data;
 	unsigned int orig_prog_len = env->prog->len;
+	u32 i;
 	int err;
 
+	/*
+	 * Instructions that are targets of verified indirect jumps
+	 * (insn_array / gotox) must never be removed: the jump table is
+	 * verified against these very instructions and the JIT fills the
+	 * jitted pointers into the table.  Removing such an instruction
+	 * would leave the corresponding table entry dangling and turn a
+	 * verifier-approved `gotox rX` into a jump through a NULL or stale
+	 * code pointer at run time.
+	 */
+	for (i = off; i < off + cnt; i++) {
+		if (aux_data[i].indirect_target) {
+			verbose(env, "cannot remove instruction %u: indirect jump target\n", i);
+			return -EFAULT;
+		}
+	}
+
 	if (bpf_prog_is_offloaded(env->prog->aux))
 		bpf_prog_offload_remove_insns(env, off, cnt);
 
@@ -613,6 +630,7 @@ int bpf_opt_remove_dead_code(struct bpf_verifier_env *env)
 
 int bpf_opt_remove_nops(struct bpf_verifier_env *env)
 {
+	struct bpf_insn_aux_data *aux_data = env->insn_aux_data;
 	struct bpf_insn *insn = env->prog->insnsi;
 	int insn_cnt = env->prog->len;
 	bool is_may_goto_0, is_ja;
@@ -622,7 +640,12 @@ int bpf_opt_remove_nops(struct bpf_verifier_env *env)
 		is_may_goto_0 = !memcmp(&insn[i], &MAY_GOTO_0, sizeof(MAY_GOTO_0));
 		is_ja = !memcmp(&insn[i], &NOP, sizeof(NOP));
 
-		if (!is_may_goto_0 && !is_ja)
+		/* Do not remove indirect jump targets: the jump table entries
+		 * for them were verified against these instructions, and the
+		 * removal would leave the table entries dangling (see
+		 * verifier_remove_insns()).
+		 */
+		if ((!is_may_goto_0 && !is_ja) || aux_data[i].indirect_target)
 			continue;
 
 		err = verifier_remove_insns(env, i, 1);
-- 
2.55.0