[PATCH net-next v4 3/4] nexthop: release cached routes when a nexthop leaves a group

From: Qiliang Yuan

Date: Tue Nov 17 2026 - 21:29:27 EST


When a nexthop is removed while other members keep its group alive, only
the group array changes: the routes through the group stay in the FIB,
and the dsts they cached through the removed nexthop are released when
that nexthop is freed, from an RCU callback.

Until then those dsts stay valid. IPv6 route exceptions, such as the
ones created on Packet Too Big, are checked against the cookie of their
FIB node, which does not change, so a socket keeps sending through the
removed nexthop. When the nexthop goes away with its device, removing
the device's IPv6 addresses usually flushes the routing tree first, but
not when the device carries no IPv6 address.

Once all readers see the new group arrays, release the per-cpu routes and
exceptions of a removed IPv6 nexthop, as replacing a group already does
for its old members, and flush the IPv4 route cache once if any of the
groups carries IPv4 routes. The routes using the nexthop directly are
already gone at that point.

A connected UDP socket in netns A sends to a route through group {nh 1,
nh 2}, on an IPv6 exception of nh 1 created by Packet Too Big, while nh 1
is deleted. Datagrams still sent through nh 1 after the new group is
published and before nh 1 is freed, 10 trials, median and range, the
ones left after the change being sent before the dsts are released:

before (datagrams) after (datagrams) change
per trial 1140 (367-3788) 1 (1-3) -99.9%

Signed-off-by: Qiliang Yuan <qiliang.yuan@xxxxxxxxx>
---
net/ipv4/nexthop.c | 15 +++++++++++++++
1 file changed, 15 insertions(+)

diff --git a/net/ipv4/nexthop.c b/net/ipv4/nexthop.c
index 42e55b5a755ee..3fccd4f2f916b 100644
--- a/net/ipv4/nexthop.c
+++ b/net/ipv4/nexthop.c
@@ -2116,6 +2116,7 @@ remove_nexthop_from_groups(struct net *net, struct nexthop *nh,
struct nh_grp_entry *nhge, *tmp;
LIST_HEAD(deferred_free);
bool need_flush = false;
+ struct nh_info *nhi;

/* If there is nothing to do, let's avoid the costly call to
* synchronize_net()
@@ -2130,6 +2131,20 @@ remove_nexthop_from_groups(struct net *net, struct nexthop *nh,
/* make sure all see the newly published array before releasing rtnl */
synchronize_net();

+ /* Routes through the groups can still hold dsts cached through @nh,
+ * which are only released when @nh is freed. Release them now that
+ * no reader can pick @nh from the old groups any more.
+ */
+ nhi = rtnl_dereference(nh->nh_info);
+ if (nhi->family == AF_INET6)
+ fib6_nh_release_dsts(&nhi->fib6_nh);
+ list_for_each_entry(nhge, &deferred_free, nh_list) {
+ if (!list_empty(&nhge->nh_parent->fi_list)) {
+ rt_cache_flush(net);
+ break;
+ }
+ }
+
/* Now safe to free percpu stats — all RCU readers have finished */
list_for_each_entry_safe(nhge, tmp, &deferred_free, nh_list) {
list_del(&nhge->nh_list);

--
2.43.0