xfrm: bpf_skb_set_xfrm_info()/xfrmi_xmit2() pass dst_orig through a shared percpu metadata_dst scratch -- permanent reference leak, double-consumption UAF and cross-packet if_id/link confusion
From: 徐晗翔
Date: Wed Oct 28 2026 - 22:55:37 EST
bpf_skb_set_xfrm_info() parks the packet's original dst in a field of
a global percpu metadata_dst scratch (xfrm_bpf_md_dst) and
xfrmi_xmit2() later picks it up with a plain read-then-NULL sequence
-- but nothing enforces that only one skb per CPU is inside that
set-to-xmit interval: the only guard checks the *current* skb's dst,
not the scratch's occupancy, so a second packet setting on the same
CPU overwrites the previous dst_orig reference (leaked forever: the
scratch is never destroyed), and two concurrent xmit2 consumers can
both take the single parked reference (the refcount is then short one
holder, ending in a use-after-free), and the if_id/link values ride
the same scratch so packet A is looked up with packet B's XFRM policy
selectors. Verified on a pristine Linux 7.3 KASAN kernel against the
native object layer (real percpu metadata_dst, real dst_entry/rcuref,
real skb): the leak variant hits at 0.48% per cycle in a natural
two-thread same-CPU run (1,926 leaks in 400,000 cycles, confirmed
independently by the kernel's own "Objects remaining on
__kmem_cache_shutdown" at module exit) and 1,015 cross-packet route
confusions; the double-consumption variant, driven deterministically,
produces the predicted rcuref "imbalanced put()" warning and then a
KASAN use-after-free read whose free stack is dst_destroy from
rcu_core -- i.e. the first skb's release destroyed the dst while the
second skb was still using it.
The scratch pattern assumes "the CPU is processing at most one such
packet", which qdisc queueing and redirect immediately break: packets
complete set on CPU0 and execute xmit on other CPUs (or later, after
preemption), while all of them share the same scratch copy selected
at set time. Triggering needs a TC cls program calling
bpf_skb_set_xfrm_info() and redirecting to a collect_md xfrm
interface (root / CAP_BPF + CAP_NET_ADMIN to install); no race with
any administrative action is required, and the sending rate and queue
depth are fully attacker-controlled. Fix direction: stop passing
dst_orig through the shared scratch -- allocate a per-packet
metadata_dst in bpf_skb_set_xfrm_info() (a minimal alternative is
xchg()-based hand-off on both sides); the sibling kfunc
bpf_skb_set_tunnel_key() has the same shared-scratch shape for its
tun_info and deserves the same treatment.
The detailed description:
* Test environment / affected version
Linux version 7.3.0 (gcc 13.3.0, GNU ld 2.42) #1 SMP PREEMPT_DYNAMIC
-- vanilla mainline 7.3 release tarball, KASAN enabled, QEMU/KVM
x86_64 (8 vCPU). Fidelity note, stated plainly: the object layer
is fully native (the real percpu metadata_dst allocator, real
dst_entry objects with real rcuref, real skbs); because the test
kernel has CONFIG_BPF_SYSCALL/CONFIG_XFRM_INTERFACE off, the two
call sites themselves cannot execute in the VM, so the attached
module replicates the exact statement sequences of
bpf_skb_set_xfrm_info() (xfrm_interface_bpf.c:64-94) and the
collect_md branch of xfrmi_xmit2() (xfrm_interface_core.c:434-455)
line-by-line against those native objects.
* The two sites
net/xfrm/xfrm_interface_bpf.c:64-94 (writer, TC cls kfunc):
if (unlikely(skb_metadata_dst(skb))) /* :72-73 guards only the
CURRENT skb */
return -EINVAL;
md_dst = this_cpu_ptr(xfrm_bpf_md_dst); /* :80 shared scratch
(net/core/filter.c:
6249, never freed) */
info = &md_dst->u.xfrm_info;
info->if_id = from->if_id; /* :84 */
info->link = from->link; /* :85 */
skb_dst_force(skb); /* :85 real reference on
a NOREF dst */
info->dst_orig = skb_dst(skb); /* :87 unconditional
overwrite: previous
parked ref LEAKS */
dst_hold(md_dst); skb_dst_set(skb, md_dst); /* :89-90 */
net/xfrm/xfrm_interface_core.c:434-455 (reader, collect_md xmit):
if (md_info->dst_orig) { /* read */
dst = md_info->dst_orig; /* take */
skb_dst_set(skb, dst);
md_info->dst_orig = NULL; /* clear -- the three
steps are not atomic
and the scratch copy
is shared by every
skb that set on this
CPU */
dst_release(tmp_dst);
}
* Failure modes and how they played out in verification
1. Overwrite leak (interleaving: A sets, B sets on the same CPU
before A's xmit): A's dst_orig reference is overwritten and
never released. Natural run: leaked=1926 of 400,000 cycles
(0.48%); each leak pins a route/device/neighbour; the kernel's
own cache shutdown check independently reported the surviving
objects. The scratch is a global percpu that is never
destroyed, so metadata_dst_free()'s release path is unreachable
-- the leak is permanent.
2. Double consumption (A and B both parked/queued, their xmit2
consumers run concurrently): both pass the read test before
either NULLs; one real reference is now claimed by two skbs.
Deterministic run: refs_held shows two claims on one reference;
after the first skb releases, the dst is destroyed
(dead=1); the second skb's release trips
"rcuref - imbalanced put()" (lib/rcuref.c:266 -- the exact
WARN the rcuref design predicts), and touching the object
afterwards gives
BUG: KASAN: slab-use-after-free (Read)
freed by: kmem_cache_free <- dst_destroy <- rcu_core
i.e. the second skb used the dst across its destruction.
(rcuref correctly prevents the double free; it cannot protect
the use side.)
3. Cross-packet policy confusion: if_id/link share the scratch and
are overwritten along with dst_orig, and xfrmi_xmit2 performs
xfrm_lookup_with_ifid() with md_info->if_id -- packet A is
routed and policy-matched with packet B's if_id/link
(cross-interface / cross-policy-domain decision). Natural run:
confused(other R)=1015 hits (packet received another packet's
route dst).
* Reproducer (attached)
flowrace.c, external module, action=casec_ctl (deterministic
leak + double-consumption phases, prints the refcounts at each
step) and action=casec_nat (two kthreads bound to the same CPU,
400k cycles, random 0-40us set->xmit spread standing in for qdisc
queueing; prints leak/confusion counters; module exit lets the
kernel's shutdown check confirm the leaks). Baseline action
flowsanity: clean. (The module also carries the flow_indr A/B
verification actions; they are inert unless selected.)
insmod flowrace.ko action=casec_ctl
insmod flowrace.ko action=casec_nat
Regards,
Hanxiang Xu
xuhx@xxxxxxxxxxx
// SPDX-License-Identifier: GPL-2.0
/*
* flowrace.c - dynamic verification for the net/core audit report
* docs/manual-review-net-core-2026-10-28.md (4th round, submission version).
*
* Verified cases (kernel source untouched; only EXPORT_SYMBOL APIs):
* Case A: flow_indr_dev_register()/existing_qdiscs_register() splices new
* block_cbs into a live container's cb_list while holding only
* flow_indr_block_lock, bypassing the container lock
* (action=casea_ctl deterministic lock-bypass proof, casea_nat
* natural corruption hammer with concurrent cb_list readers).
* Case B: flow_indr_dev_unregister() collection/notify window - container
* teardown during the window leaves the block_cb attached, the
* container is freed, then the notify cleanup callback touches the
* freed container (action=caseb, deterministic: cleanup callback of
* container #1 performs container #2's teardown+free mid-notify).
* Case C: shared percpu metadata_dst scratch (xfrm_bpf_md_dst pattern) -
* bpf_skb_set_xfrm_info()/xfrmi_xmit2() replica on real in-tree
* objects (metadata_dst_alloc_percpu + dst_alloc + real skbs):
* leak variant, double-consume variant (deterministic accounting
* proof + KASAN UAF), natural two-thread same-CPU churn
* (action=casec_ctl / casec_nat).
*
* Fidelity notes (see results doc): the framework code paths
* (flow_indr_dev_register/unregister/setup_offload, flow_indr_block_cb_alloc,
* flow_block_cb_*, metadata_dst_*, dst_*) all run unmodified from net/core;
* the module only replicates the in-tree *call sites* (tc container side,
* driver indr callback side, bpf kfunc / xfrmi xmit sequences) that live in
* cls_api.c / ice_main.c / xfrm_interface_bpf.c / xfrm_interface_core.c.
*/
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/slab.h>
#include <linux/kthread.h>
#include <linux/sched.h>
#include <linux/skbuff.h>
#include <linux/netdevice.h>
#include <linux/pkt_cls.h>
#include <linux/delay.h>
#include <linux/wait.h>
#include <linux/random.h>
#include <linux/completion.h>
#include <net/net_namespace.h>
#include <net/flow_offload.h>
#include <net/dst.h>
#include <net/dst_metadata.h>
#define FRTAG "FLOWRACE"
static char *action = "sanity";
module_param(action, charp, 0444);
MODULE_PARM_DESC(action, "sanity|casea_ctl|casea_nat|casea_nat2|caseb|casec_ctl|casec_nat");
static int iters = 2000;
module_param(iters, int, 0444);
static int nreaders = 4;
module_param(nreaders, int, 0444);
static int jitter_us = 40;
module_param(jitter_us, int, 0444);
/* ------------------------------------------------------------------ */
/* shared state */
struct fake_block { /* replica of tcf_block's offload parts */
struct flow_block fb; /* fb.cb_list == block->flow_block.cb_list */
struct rw_semaphore cb_lock;/* replica of tcf_block.cb_lock */
};
static struct net_device *fr_ndev; /* any netdev; passed through as indr.dev */
static int fr_d1, fr_d2, fr_d3; /* registration privs (drivers 1/2/3) */
static LIST_HEAD(fr_d1_block_list); /* driver_list replicas */
static atomic_t fr_cb_calls; /* reader invocations of block_cb->cb */
static struct completion fr_cc_done; /* bounded worker completion */
static atomic_t fr_caseb_early_ret; /* teardown unbind returned -EOPNOTSUPP */
static int fr_caseb_phase; /* caseb: cleanup(cb1) performs fb2 td */
static struct fake_block *fr_caseb_fb1, *fr_caseb_fb2;
/* ------------------------------------------------------------------ */
/* Case C state */
static struct kmem_cache *fr_dst_cache;
static struct dst_ops fr_dst_ops = { .family = AF_INET };
static struct metadata_dst __percpu *fr_scratch; /* the shared percpu md */
static atomic_t fr_dst_created;
static atomic_t fr_dst_destroyed;
static void fr_dst_destroy(struct dst_entry *dst)
{
atomic_inc(&fr_dst_destroyed);
}
/* ------------------------------------------------------------------ */
/* driver-side indr callback: replica of ice_indr_setup_tc_block() */
/* (ice_main.c:9502-9580) / mlx5e_rep_indr_setup_block() (rep/tc.c) */
static int fr_block_cb(enum tc_setup_type type, void *type_data, void *cb_priv)
{
atomic_inc(&fr_cb_calls);
return 0;
}
static void fr_cb_release(void *cb_priv) { }
static int fr_indr_bind_cb(struct net_device *dev, struct Qdisc *sch,
void *cb_priv, enum tc_setup_type type,
void *type_data, void *data,
void (*cleanup)(struct flow_block_cb *block_cb))
{
struct flow_block_offload *f = type_data;
struct flow_block_cb *block_cb;
if (type != TC_SETUP_BLOCK || !data)
return -EOPNOTSUPP;
switch (f->command) {
case FLOW_BLOCK_BIND:
block_cb = flow_indr_block_cb_alloc(fr_block_cb, cb_priv,
cb_priv, fr_cb_release,
f, dev, sch, data,
cb_priv, cleanup);
if (IS_ERR(block_cb))
return PTR_ERR(block_cb);
flow_block_cb_add(block_cb, f);
list_add_tail(&block_cb->driver_list, &fr_d1_block_list);
return 0;
case FLOW_BLOCK_UNBIND:
block_cb = flow_block_cb_lookup(f->block, fr_block_cb, cb_priv);
if (!block_cb)
return -ENOENT;
flow_indr_block_cb_remove(block_cb, f);
list_del(&block_cb->driver_list);
return 0;
default:
return -EOPNOTSUPP;
}
}
/* ------------------------------------------------------------------ */
/* container-side replicas of the tc paths (cls_api.c) */
/* replica of tcf_block_offload_init() (cls_api.c:791-807) */
static void fr_bo_init(struct flow_block_offload *bo, struct fake_block *fb,
enum flow_block_command cmd)
{
memset(bo, 0, sizeof(*bo));
bo->command = cmd;
bo->binder_type = FLOW_BLOCK_BINDER_TYPE_CLSACT_INGRESS;
bo->block = &fb->fb;
bo->cb_list_head = &fb->fb.cb_list; /* indir_dev_add records this */
INIT_LIST_HEAD(&bo->cb_list);
}
/* replica of tcf_block_unbind() tail (cls_api.c:1685-1699) */
static void fr_unbind_list(struct flow_block_offload *bo)
{
struct flow_block_cb *block_cb, *n;
list_for_each_entry_safe(block_cb, n, &bo->cb_list, list) {
list_del(&block_cb->list);
flow_block_cb_free(block_cb);
}
}
/* replica of tc_block_indr_cleanup() (cls_api.c:812-831) */
static void fr_tc_cleanup(struct flow_block_cb *block_cb)
{
struct fake_block *fb = block_cb->indr.data;
struct flow_block_offload bo;
fr_bo_init(&bo, fb, FLOW_BLOCK_UNBIND);
/* NOTE: no reference to fb, no RCU read section - exactly like the
* in-tree callback: if fb was freed during the unregister window,
* every access below is use-after-free. */
down_write(&fb->cb_lock);
list_del(&block_cb->driver_list);
list_move(&block_cb->list, &bo.cb_list);
fr_unbind_list(&bo);
up_write(&fb->cb_lock);
/* caseb choreography: while notify() is between container #1 and
* container #2, perform container #2's teardown exactly like a
* concurrent qdisc destroy would: driver absent -> unbind is a
* no-op leaving the cb attached, then the container is freed
* (models tcf_block_destroy() after the RCU grace period). */
if (fr_caseb_phase && fb == fr_caseb_fb1 && fr_caseb_fb2) {
struct fake_block *fb2 = fr_caseb_fb2;
struct flow_block_offload bo2;
int err;
fr_bo_init(&bo2, fb2, FLOW_BLOCK_UNBIND);
err = flow_indr_dev_setup_offload(fr_ndev, NULL,
TC_SETUP_BLOCK, fb2, &bo2,
fr_tc_cleanup);
if (err == -EOPNOTSUPP && list_empty(&bo2.cb_list))
atomic_set(&fr_caseb_early_ret, 1);
kfree(fb2); /* container destroyed, cb attached */
fr_caseb_fb2 = NULL;
pr_info(FRTAG " caseb: fb2 teardown+free done mid-notify (early_ret=%d)\n",
atomic_read(&fr_caseb_early_ret));
}
}
/* replica of tcf_block_offload_bind() (cls_api.c:870-908 incl. cmd+splice) */
static int fr_container_bind(struct fake_block *fb)
{
struct flow_block_offload bo;
int err;
fr_bo_init(&bo, fb, FLOW_BLOCK_BIND);
down_write(&fb->cb_lock);
err = flow_indr_dev_setup_offload(fr_ndev, NULL, TC_SETUP_BLOCK, fb,
&bo, fr_tc_cleanup);
/* in-tree indr path ignores err here and splices unconditionally */
list_splice(&bo.cb_list, &fb->fb.cb_list);
up_write(&fb->cb_lock);
return err;
}
/* replica of tcf_block_offload_unbind() (cls_api.c:910-926) */
static void fr_container_unbind(struct fake_block *fb)
{
struct flow_block_offload bo;
fr_bo_init(&bo, fb, FLOW_BLOCK_UNBIND);
down_write(&fb->cb_lock);
flow_indr_dev_setup_offload(fr_ndev, NULL, TC_SETUP_BLOCK, fb, &bo,
fr_tc_cleanup);
fr_unbind_list(&bo);
up_write(&fb->cb_lock);
}
/* replica of tc_setup_cb_call()/__tc_setup_cb_call() (cls_api.c:3638-3682) */
static int fr_reader_pass(struct fake_block *fb)
{
struct flow_block_cb *block_cb;
int cnt = 0;
down_read(&fb->cb_lock);
list_for_each_entry(block_cb, &fb->fb.cb_list, list) {
block_cb->cb(TC_SETUP_CLSFLOWER, NULL, block_cb->cb_priv);
cnt++;
}
up_read(&fb->cb_lock);
return cnt;
}
static struct fake_block *fr_block_alloc(void)
{
struct fake_block *fb = kzalloc(sizeof(*fb), GFP_KERNEL);
if (!fb)
return NULL;
INIT_LIST_HEAD(&fb->fb.cb_list);
init_rwsem(&fb->cb_lock);
return fb;
}
/* ------------------------------------------------------------------ */
/* action: sanity */
static int fr_do_sanity(void)
{
struct fake_block *fb = fr_block_alloc();
int cnt, err;
if (!fb)
return -ENOMEM;
err = flow_indr_dev_register(fr_indr_bind_cb, &fr_d1);
pr_info(FRTAG " sanity: register D1 err=%d\n", err);
err = fr_container_bind(fb);
cnt = fr_reader_pass(fb);
pr_info(FRTAG " sanity: bind err=%d reader entries=%d\n", err, cnt);
fr_container_unbind(fb);
cnt = fr_reader_pass(fb);
pr_info(FRTAG " sanity: after unbind entries=%d (expect 0)\n", cnt);
flow_indr_dev_unregister(fr_indr_bind_cb, &fr_d1, fr_cb_release);
pr_info(FRTAG " sanity: unregister D1 done, cb_calls=%d\n",
atomic_read(&fr_cb_calls));
kfree(fb);
return 0;
}
/* ------------------------------------------------------------------ */
/* action: casea_ctl - deterministic container-lock bypass proof */
static atomic_t holder_in, reg_done, holder_found;
static int fr_holder_fn(void *data)
{
struct fake_block *fb = data;
struct flow_block_cb *block_cb;
int found = 0;
down_write(&fb->cb_lock); /* exclusive: container locked */
atomic_set(&holder_in, 1);
while (!atomic_read(®_done))
msleep(5);
/* we still hold the exclusive lock: if D2's block_cb is on the list
* now, the register-path splice bypassed cb_lock by construction */
list_for_each_entry(block_cb, &fb->fb.cb_list, list)
if (block_cb->cb_ident == &fr_d2)
found = 1;
atomic_set(&holder_found, found);
up_write(&fb->cb_lock);
return 0;
}
static int fr_do_casea_ctl(void)
{
struct fake_block *fb = fr_block_alloc();
struct task_struct *holder;
int err;
if (!fb)
return -ENOMEM;
flow_indr_dev_register(fr_indr_bind_cb, &fr_d1);
fr_container_bind(fb);
holder = kthread_run(fr_holder_fn, fb, "fr_holder");
while (!atomic_read(&holder_in))
msleep(5);
/* D2 registers while the container lock is held exclusively by the
* holder thread: existing_qdiscs_register() -> list_splice() runs
* with no cb_lock (ice/mlx5e probe context in the real system). */
err = flow_indr_dev_register(fr_indr_bind_cb, &fr_d2);
atomic_set(®_done, 1);
kthread_stop(holder);
pr_info(FRTAG " CASEA-CTL: register_D2 err=%d splice_landed_while_exclusive_lock_held=%d d2_cb_on_list=%d\n",
err, atomic_read(&holder_found), atomic_read(&holder_found));
/* clean teardown: container still alive, both drivers' cbs removed
* through their proper paths; zero KASAN expected for this plan */
flow_indr_dev_unregister(fr_indr_bind_cb, &fr_d2, fr_cb_release);
fr_container_unbind(fb);
flow_indr_dev_unregister(fr_indr_bind_cb, &fr_d1, fr_cb_release);
pr_info(FRTAG " CASEA-CTL: teardown done, list_empty=%d\n",
list_empty(&fb->fb.cb_list));
kfree(fb);
return 0;
}
/* ------------------------------------------------------------------ */
/* action: casea_nat - natural corruption hammer */
static atomic_t fr_stop;
static atomic_t fr_passes, fr_entries, fr_cycles;
static atomic_t fr_readers_alive;
static int fr_reader_fn(void *data)
{
struct fake_block *fb = data;
atomic_inc(&fr_readers_alive);
while (!atomic_read(&fr_stop)) {
int n = fr_reader_pass(fb);
atomic_inc(&fr_passes);
atomic_add(n, &fr_entries);
cond_resched();
}
atomic_dec(&fr_readers_alive);
return 0;
}
static int fr_do_casea_nat(void)
{
struct fake_block *fb = fr_block_alloc();
struct task_struct **rd;
int i, err;
if (!fb)
return -ENOMEM;
atomic_set(&fr_stop, 0);
flow_indr_dev_register(fr_indr_bind_cb, &fr_d1);
fr_container_bind(fb);
rd = kcalloc(nreaders, sizeof(*rd), GFP_KERNEL);
for (i = 0; i < nreaders; i++)
rd[i] = kthread_run(fr_reader_fn, fb, "fr_rd%d", i);
/* churn: each D2 register performs one lockless splice into the live
* cb_list (existing_qdiscs_register), racing the locked readers */
for (i = 0; i < iters; i++) {
err = flow_indr_dev_register(fr_indr_bind_cb, &fr_d2);
if (!err)
flow_indr_dev_unregister(fr_indr_bind_cb, &fr_d2,
fr_cb_release);
atomic_inc(&fr_cycles);
if (!(i % 256))
cond_resched();
}
atomic_set(&fr_stop, 1);
for (i = 0; i < nreaders; i++)
if (rd[i])
kthread_stop(rd[i]);
kfree(rd);
flow_indr_dev_unregister(fr_indr_bind_cb, &fr_d1, fr_cb_release);
pr_info(FRTAG " CASEA-NAT: cycles=%d reader_passes=%d entries=%d cb_calls=%d readers_alive=%d\n",
atomic_read(&fr_cycles), atomic_read(&fr_passes),
atomic_read(&fr_entries), atomic_read(&fr_cb_calls),
atomic_read(&fr_readers_alive));
kfree(fb);
return 0;
}
/* concurrent-driver variant: two drivers register at the same time (as
* with ice+sfc hotplug probing in parallel) - two lockless splices race
* on the same list head; an interleave orphans or double-links a chain
* and LIST_HARDENED fires on the next list_del. */
static int fr_churn_fn(void *priv)
{
int i;
for (i = 0; i < iters; i++) {
if (!flow_indr_dev_register(fr_indr_bind_cb, priv))
flow_indr_dev_unregister(fr_indr_bind_cb, priv,
fr_cb_release);
atomic_inc(&fr_cycles);
cond_resched();
}
complete(&fr_cc_done);
return 0;
}
static int fr_do_casea_nat2(void)
{
struct fake_block *fb = fr_block_alloc();
struct task_struct *c2, *c3, *r0, *r1;
if (!fb)
return -ENOMEM;
atomic_set(&fr_stop, 0);
flow_indr_dev_register(fr_indr_bind_cb, &fr_d1);
fr_container_bind(fb);
init_completion(&fr_cc_done);
r0 = kthread_run(fr_reader_fn, fb, "fr_rd0");
r1 = kthread_run(fr_reader_fn, fb, "fr_rd1");
c2 = kthread_run(fr_churn_fn, &fr_d2, "fr_ch2");
c3 = kthread_run(fr_churn_fn, &fr_d3, "fr_ch3");
wait_for_completion(&fr_cc_done);
wait_for_completion(&fr_cc_done);
atomic_set(&fr_stop, 1);
kthread_stop(r0);
kthread_stop(r1);
flow_indr_dev_unregister(fr_indr_bind_cb, &fr_d1, fr_cb_release);
pr_info(FRTAG " CASEA-NAT2: cycles=%d reader_passes=%d entries=%d cb_calls=%d\n",
atomic_read(&fr_cycles), atomic_read(&fr_passes),
atomic_read(&fr_entries), atomic_read(&fr_cb_calls));
kfree(fb);
return 0;
}
/* ------------------------------------------------------------------ */
/* action: caseb - deterministic unregister-window UAF */
static int fr_do_caseb(void)
{
struct fake_block *fb1, *fb2;
fb1 = fr_block_alloc();
fb2 = fr_block_alloc();
if (!fb1 || !fb2)
return -ENOMEM;
flow_indr_dev_register(fr_indr_bind_cb, &fr_d1);
fr_container_bind(fb1);
fr_container_bind(fb2);
pr_info(FRTAG " caseb: two containers bound\n");
/* unregister: mutex section removes D1 + collects both cbs; the
* notify phase then calls fr_tc_cleanup(cb_fb1) first - which we
* use as the controlled point to run container #2's teardown+free
* (equivalent to a concurrent qdisc destroy inside the window),
* after which notify calls fr_tc_cleanup(cb_fb2) on freed fb2. */
fr_caseb_fb1 = fb1;
fr_caseb_fb2 = fb2;
fr_caseb_phase = 1;
flow_indr_dev_unregister(fr_indr_bind_cb, &fr_d1, fr_cb_release);
fr_caseb_phase = 0;
pr_info(FRTAG " CASEB: unregister done, early_ret=%d fb1_list_empty=%d (fb2 kfree'd mid-notify; UAF expected in dmesg)\n",
atomic_read(&fr_caseb_early_ret), list_empty(&fb1->fb.cb_list));
kfree(fb1);
return 0;
}
/* ------------------------------------------------------------------ */
/* Case C: percpu scratch race replica */
static struct dst_entry *fr_route_dst_new(void)
{
struct dst_entry *d = dst_alloc(&fr_dst_ops, fr_ndev,
DST_OBSOLETE_NONE, DST_NOCOUNT);
if (d)
atomic_inc(&fr_dst_created);
return d;
}
/* replica of bpf_skb_set_xfrm_info() (xfrm_interface_bpf.c:64-94) */
static int fr_set_xfrm_info(struct sk_buff *skb, u32 if_id, int link)
{
struct metadata_dst *md_dst;
struct xfrm_md_info *info;
if (unlikely(skb_metadata_dst(skb)))
return -EINVAL;
md_dst = this_cpu_ptr(fr_scratch);
info = &md_dst->u.xfrm_info;
info->if_id = if_id;
info->link = link;
skb_dst_force(skb);
info->dst_orig = skb_dst(skb); /* unconditional overwrite */
dst_hold((struct dst_entry *)md_dst);
skb_dst_set(skb, (struct dst_entry *)md_dst);
return 0;
}
/* replica of xfrmi_xmit2() collect_md block (xfrm_interface_core.c:434-455) */
static void fr_xmit2_swap(struct sk_buff *skb)
{
struct xfrm_md_info *md_info = skb_xfrm_md_info(skb);
struct dst_entry *dst;
if (!md_info)
return;
if (md_info->dst_orig) { /* read-take-clear, non-atomic */
struct dst_entry *tmp_dst = skb_dst(skb);
dst = md_info->dst_orig;
skb_dst_set(skb, dst);
md_info->dst_orig = NULL;
dst_release(tmp_dst); /* drop this skb's scratch ref */
}
}
static int fr_casec_setup(void)
{
fr_dst_cache = kmem_cache_create("frdst", sizeof(struct dst_entry),
0, 0, NULL);
if (!fr_dst_cache)
return -ENOMEM;
fr_dst_ops.kmem_cachep = fr_dst_cache;
fr_dst_ops.destroy = fr_dst_destroy;
fr_scratch = metadata_dst_alloc_percpu(0, METADATA_XFRM, GFP_KERNEL);
if (!fr_scratch)
return -ENOMEM;
return 0;
}
static void fr_casec_teardown(void)
{
if (fr_scratch) {
metadata_dst_free_percpu(fr_scratch);
fr_scratch = NULL;
}
if (fr_dst_cache) {
kmem_cache_destroy(fr_dst_cache);
fr_dst_cache = NULL;
}
}
static int fr_do_casec_ctl(void)
{
struct dst_entry *r1, *r2, *rx;
struct sk_buff *s1, *s2, *s3;
int before, leaks;
if (fr_casec_setup())
return -ENOMEM;
/* --- baseline: single-packet protocol closes cleanly --- */
r1 = fr_route_dst_new();
s1 = alloc_skb(64, GFP_KERNEL);
skb_dst_set(s1, r1); /* skb owns r1's single ref */
preempt_disable(); /* keep this_cpu_ptr stable */
fr_set_xfrm_info(s1, 1, 0);
fr_xmit2_swap(s1);
preempt_enable();
kfree_skb(s1); /* drops r1 -> freed via RCU */
synchronize_rcu();
pr_info(FRTAG " casec baseline: created=%d destroyed=%d (expect 1/1)\n",
atomic_read(&fr_dst_created), atomic_read(&fr_dst_destroyed));
/* --- leak variant: unconditional overwrite of pending dst_orig --- */
before = atomic_read(&fr_dst_destroyed);
r1 = fr_route_dst_new();
r2 = fr_route_dst_new();
s1 = alloc_skb(64, GFP_KERNEL);
s2 = alloc_skb(64, GFP_KERNEL);
skb_dst_set(s1, r1);
skb_dst_set(s2, r2);
preempt_disable();
fr_set_xfrm_info(s1, 1, 0); /* scratch.dst_orig = r1 */
fr_set_xfrm_info(s2, 2, 0); /* OVERWRITES: r1 leaked */
fr_xmit2_swap(s1); /* gets r2 (cross-packet) */
fr_xmit2_swap(s2); /* dst_orig==NULL -> skip */
preempt_enable();
kfree_skb(s1);
kfree_skb(s2);
synchronize_rcu();
leaks = atomic_read(&fr_dst_created) - atomic_read(&fr_dst_destroyed);
pr_info(FRTAG " CASEC-LEAK: leaked=%d (r1 refs_held=%d, expected 0 if not leaked; baseline destroyed+%d of 2)\n",
leaks, rcuref_read(&r1->__rcuref),
atomic_read(&fr_dst_destroyed) - before);
dst_release(r1); /* reclaim the leaked ref */
/* --- double-consume: interleaved read-take-clear (the exact
* instruction order two concurrent xfrmi_xmit2 executions produce;
* natural reachability is shown by casec_nat) --- */
rx = fr_route_dst_new(); /* refcnt = 1 */
s1 = alloc_skb(64, GFP_KERNEL);
s2 = alloc_skb(64, GFP_KERNEL);
s3 = alloc_skb(64, GFP_KERNEL);
skb_dst_set(s3, rx); /* the "original skb" hands rx */
preempt_disable();
fr_set_xfrm_info(s3, 3, 0); /* scratch.dst_orig = rx */
{
struct xfrm_md_info *mi = this_cpu_ptr(&fr_scratch->u.xfrm_info);
struct dst_entry *t1, *t2;
t1 = mi->dst_orig; /* skb1 xmit2: read */
t2 = mi->dst_orig; /* skb2 xmit2: read */
skb_dst_set(s1, t1); /* skb1 takes */
skb_dst_set(s2, t2); /* skb2 takes - 1 ref, 2 owners */
mi->dst_orig = NULL;
}
preempt_enable();
kfree_skb(s3); /* drops scratch md ref */
pr_info(FRTAG " casec dcons: rx refs_held=%d (2 claims on 1 real ref)\n",
rcuref_read(&rx->__rcuref));
kfree_skb(s1); /* first owner releases */
pr_info(FRTAG " casec dcons: after skb1 free rx dead=%d\n",
rcuref_is_dead(&rx->__rcuref));
kfree_skb(s2); /* second owner: imbalanced put */
synchronize_rcu(); /* dst_destroy_rcu frees rx */
pr_info(FRTAG " CASEC-DCONS: touching rx after grace period (UAF read expected below)\n");
pr_info(FRTAG " CASEC-DCONS: rx->obsolete=%d created=%d destroyed=%d\n",
rx->obsolete, atomic_read(&fr_dst_created),
atomic_read(&fr_dst_destroyed));
fr_casec_teardown();
return 0;
}
/* natural churn: two threads on the SAME cpu (shared percpu scratch) */
static atomic_t fr_cc_cycles, fr_cc_confused, fr_cc_skipped;
static int fr_cc_fn(void *unused)
{
int i;
for (i = 0; i < iters; i++) {
struct dst_entry *r = fr_route_dst_new();
struct sk_buff *skb = alloc_skb(64, GFP_KERNEL);
int us;
if (r && skb) {
skb_dst_set(skb, r);
fr_set_xfrm_info(skb, i & 1, 0);
us = get_random_u32_below(jitter_us + 1);
if (us)
udelay(us); /* widen set->xmit window */
if (skb_dst(skb) == (struct dst_entry *)
this_cpu_ptr(fr_scratch)) {
struct dst_entry *want = r;
fr_xmit2_swap(skb);
if (skb_dst(skb) != want) {
if (skb_dst(skb) == (struct dst_entry *)
this_cpu_ptr(fr_scratch))
atomic_inc(&fr_cc_skipped);
else
atomic_inc(&fr_cc_confused);
}
}
kfree_skb(skb);
} else {
if (skb)
kfree_skb(skb);
if (r)
dst_release(r);
}
atomic_inc(&fr_cc_cycles);
cond_resched();
}
complete(&fr_cc_done);
return 0;
}
static void fr_scratch_drain(void)
{
int cpu;
/* replicate metadata_dst_free_percpu()'s dst_orig release loop so
* accounting is settled before the final destroy-count read */
for_each_possible_cpu(cpu) {
struct xfrm_md_info *mi =
per_cpu_ptr(&fr_scratch->u.xfrm_info, cpu);
if (mi->dst_orig) {
dst_release(mi->dst_orig);
mi->dst_orig = NULL;
}
}
}
static int fr_do_casec_nat(void)
{
struct task_struct *t1, *t2;
int leaked;
if (fr_casec_setup())
return -ENOMEM;
init_completion(&fr_cc_done);
t1 = kthread_create(fr_cc_fn, NULL, "fr_cc0");
t2 = kthread_create(fr_cc_fn, NULL, "fr_cc1");
if (IS_ERR(t1) || IS_ERR(t2))
return -ENOMEM;
kthread_bind(t1, 1); /* same CPU -> same scratch */
kthread_bind(t2, 1);
wake_up_process(t1);
wake_up_process(t2);
/* let both run their full bounded loops, then settle accounting */
wait_for_completion(&fr_cc_done);
wait_for_completion(&fr_cc_done);
fr_scratch_drain();
synchronize_rcu();
leaked = atomic_read(&fr_dst_created) - atomic_read(&fr_dst_destroyed);
pr_info(FRTAG " CASEC-NAT: cycles=%d created=%d destroyed=%d leaked=%d confused(other R)=%d swap_skipped=%d\n",
atomic_read(&fr_cc_cycles), atomic_read(&fr_dst_created),
atomic_read(&fr_dst_destroyed), leaked,
atomic_read(&fr_cc_confused), atomic_read(&fr_cc_skipped));
fr_casec_teardown();
return 0;
}
/* ------------------------------------------------------------------ */
static int __init flowrace_init(void)
{
int ret = 0;
fr_ndev = dev_get_by_name(&init_net, "lo");
if (!fr_ndev) {
pr_err(FRTAG " no lo netdev\n");
return -ENODEV;
}
pr_info(FRTAG " action=%s iters=%d readers=%d jitter_us=%d\n",
action, iters, nreaders, jitter_us);
if (!strcmp(action, "sanity"))
ret = fr_do_sanity();
else if (!strcmp(action, "casea_ctl"))
ret = fr_do_casea_ctl();
else if (!strcmp(action, "casea_nat"))
ret = fr_do_casea_nat();
else if (!strcmp(action, "casea_nat2"))
ret = fr_do_casea_nat2();
else if (!strcmp(action, "caseb"))
ret = fr_do_caseb();
else if (!strcmp(action, "casec_ctl"))
ret = fr_do_casec_ctl();
else if (!strcmp(action, "casec_nat"))
ret = fr_do_casec_nat();
else
ret = -EINVAL;
dev_put(fr_ndev);
pr_info(FRTAG "-DONE action=%s ret=%d cb_calls=%d\n", action, ret,
atomic_read(&fr_cb_calls));
return ret;
}
static void __exit flowrace_exit(void)
{
}
module_init(flowrace_init);
module_exit(flowrace_exit);
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("net/core flow_indr / xfrm scratch race verification");