[PATCH net-next] veth: set the real number of queues before registering the devices

[Qiliang Yuan]

Since commit 9d3684c24a52 ("veth: create by default nr_possible_cpus
queues"), each veth device is allocated with one rx and one tx queue
per possible CPU, and veth_newlink() reduces the real number of queues
to one unless the user asked for a number.

Both devices are already registered at that point, so
netif_set_real_num_tx_queues() and netif_set_real_num_rx_queues()
remove the sysfs objects of every extra queue that register_netdevice()
has just created, each one emitting an add and a remove uevent, and the
tx side also waits for an expedited RCU grace period. This happens with
RTNL held, twice per pair, and grows with the number of possible CPUs:
with 240 of them a veth pair emits about 1900 uevents, and udev
processing them slows down later sysfs work such as deleting devices.

Set the number of queues before registering each device. For a device
that is not registered yet this only updates the counters, so the extra
queues never appear in sysfs, and the error paths that undid a
registration are no longer needed.

Creating and then deleting 500 veth pairs, one ip -batch each, in a 16
vCPU guest, median of 3 runs, per pair:

                                   before      after
  16 possible CPUs    create      1644 us     520 us
                      delete     11822 us    9750 us
  240 possible CPUs   create     17253 us    1339 us
                      delete     17082 us    9941 us
                      uevents        1918          6

Signed-off-by: Qiliang Yuan <odys.yuan@xxxxxxxxx>
---
 drivers/net/veth.c | 20 ++++++++------------
 1 file changed, 8 insertions(+), 12 deletions(-)

diff --git a/drivers/net/veth.c b/drivers/net/veth.c
index 38f263d4f06b4..92ad064567327 100644
--- a/drivers/net/veth.c
+++ b/drivers/net/veth.c
@@ -1875,6 +1875,10 @@ static int veth_newlink(struct net_device *dev,
 
 	netif_inherit_tso_max(peer, dev);
 
+	err = veth_init_queues(peer, tb);
+	if (err)
+		goto err_register_peer;
+
 	err = register_netdevice(peer);
 	if (err < 0)
 		goto err_register_peer;
@@ -1904,6 +1908,10 @@ static int veth_newlink(struct net_device *dev,
 	else
 		snprintf(dev->name, IFNAMSIZ, DRV_NAME "%%d");
 
+	err = veth_init_queues(dev, tb);
+	if (err)
+		goto err_register_dev;
+
 	err = register_netdevice(dev);
 	if (err < 0)
 		goto err_register_dev;
@@ -1917,16 +1925,10 @@ static int veth_newlink(struct net_device *dev,
 	priv = netdev_priv(dev);
 	rcu_assign_pointer(priv->peer, peer);
 	netdev_hold(peer, &priv->peer_tracker, GFP_KERNEL);
-	err = veth_init_queues(dev, tb);
-	if (err)
-		goto err_queues;
 
 	priv = netdev_priv(peer);
 	rcu_assign_pointer(priv->peer, dev);
 	netdev_hold(dev, &priv->peer_tracker, GFP_KERNEL);
-	err = veth_init_queues(peer, tb);
-	if (err)
-		goto err_peer_queues;
 
 	veth_disable_gro(dev);
 	/* update XDP supported features */
@@ -1935,12 +1937,6 @@ static int veth_newlink(struct net_device *dev,
 
 	return 0;
 
-err_peer_queues:
-	netdev_put(dev, &priv->peer_tracker);
-	priv = netdev_priv(dev);
-err_queues:
-	netdev_put(peer, &priv->peer_tracker);
-	unregister_netdevice(dev);
 err_register_dev:
 	/* nothing to do */
 err_configure_peer:

---
base-commit: 6b3e73774815910a39d8ca75b9ad7df963f61d50
change-id: 20261016-bug-veth-init-queues-before-register-eebe763a01a7

Best regards,
-- 
Qiliang Yuan <odys.yuan@xxxxxxxxx>