Re: [PATCH net-next v3 2/3] ipv6: add ip6_dst_lookup_flow_cookie
From: Ralf Lici
Date: Wed Nov 18 2026 - 09:39:39 EST
Hi Xia,
On Wed, 18 Nov 2026 16:48:34 +0800, Xia Yan <yanxia@xxxxxxxxxxxxxxxxxxxx> wrote:
> I traced the race and the fix against current net-next.
>
> On the writer side, xfrm_policy_insert() links the policy and then calls
> rt_genid_bump_ipv6(), which resolves to fib6_flush_trees(). A fresh serial
> is allocated from the namespace counter first, then every node is
> restamped under tb6_lock. A late rt6_get_cookie() can pick up that
> post-insertion serial for a dst whose lookup ran before the insertion,
> and the pair keeps passing dst_check until the next flush. Sampling the
> cookie between two generation reads closes the in-lookup window. A
> policy inserted after the helper returns is still caught later, because
> the restamp changes fn_sernum itself and the stored pre-insertion cookie
> stops matching. fib6_new_sernum() never returns 0, so the fallback
> cookie always fails fib6_check().
>
Yes, that's a pretty good summary of the change with the small
correction that a newly linked node holds fn_sernum 0 until fib6_add
stamps it. So a cookie-0 dst can pass a check in that window, and fails
as soon as the stamp lands.
> 1/3 checks out as well. The old code kept writing -1 into the node, so a
> second failure changed nothing and the namespace generation never moved.
> The fresh allocation under tb6_lock matches the discipline the
> fib6_ifup()/fib6_ifdown() paths already get from __fib6_clean_all().
> The conversion boundary in 3/3 also matches the tree: rawv6 does not
> cache the dst, and the accepted-socket and refill stores in tcp_ipv6.c
> are listed as follow-up.
>
In fact there are many users of cached IPv6 dsts that will need
conversion and this series is mainly here to introduce the new API.
> One thing I could not fully work out is the ordering behind the
> generation checks. When the lookup misses a change, the reader relies on
> the two generation reads to notice it. My understanding is that this
> needs the change visible before the counter advance, and the counter
> advance visible before the node restamps. I could not establish either
> hop, though I may well be missing something.
>
> In xfrm_policy_insert(), the policy is linked into the hash and
> rt_genid_bump_ipv6() runs within the same critical section. As far as I
> can tell, the linking stores and the cmpxchg on fib6_sernum are all
> relaxed, with no barrier between them. Could a reader read the advanced
> generation in gen0, miss the policy during the lookup, sample a
> restamped fn_sernum, and still read the same generation in gen1?
>
Both hops are covered by fib6_new_sernum. It allocates the serial with
atomic_try_cmpxchg, which is fully ordered when it succeeds, so it acts
as an smp_mb before and after the update of fib6_sernum.
For the first hop, the policy (or route) is linked before
fib6_new_sernum runs, which gives message passing against the smp_rmb
after gen0: a reader that reads the advanced generation in gen0 also
finds the policy.
> In fib6_flush_trees(), the serial is allocated before __fib6_clean_all()
> takes tb6_lock, and fib6_clean_node() stamps the nodes with a plain
> WRITE_ONCE(). spin_lock() is an ACQUIRE, and my reading of
> memory-barriers.txt is that operations issued before an ACQUIRE may
> complete after it. So a reader could sample the restamped fn_sernum
For the second hop, every fn_sernum write follows a successful
fib6_new_sernum on the same CPU: fib6_update_sernum,
__fib6_update_sernum_upto_root, and fib6_clean_node in the flush case.
The full barrier orders the counter update before the tb6_lock
acquisition and the stamps, so the ACQUIRE semantics of spin_lock don't
matter here. A reader that samples a restamped fn_sernum reads the
advanced generation in gen1 after the smp_rmb, and the helper returns
cookie 0.
> between two reads that both still return the old generation. The only
> pairing I found written down here is fib6_get_cookie_safe()'s smp_rmb()
> against the smp_wmb() in __fib6_update_sernum_upto_root().
>
Right, this isn't documented. The barrier comments in
ip6_dst_lookup_flow_cookie should name fib6_new_sernum as the pairing
side, and fib6_new_sernum should state that its full ordering is relied
upon.
I also checked both hops with an LKMM litmus test: the outcome you
describe is forbidden with atomic_cmpxchg and becomes reachable with
atomic_cmpxchg_relaxed.
While going through the writers I also noticed that xfrm_policy_insert
unlinks a replaced policy only after rt_genid_bump_ipv6, so a lookup
after the bump can still match the old policy. That is a writer-side
ordering issue independent of this series, and it affects the current
late cookie sampling in the same way. I'll send a separate fix.
Does this explanation address your concern? If so, I'll carry your
Reviewed-by. I'm waiting for the Sashiko review of v3: if it reports
anything that needs a respin, I'll add the barrier comments in v4,
otherwise I'll send them as a follow-up.
Thanks for the careful review!
--
Ralf Lici
Mandelbit Srl