[PATCH net V2 0/4] net/mlx5: fixes for LAG max_tx_speed update flow

[Tariq Toukan]

Hi,

Commit 50f1d188c580 ("net/mlx5: Propagate LAG effective max_tx_speed to
vports") added a mechanism that computes the aggregated LAG TX speed and
programs it into each vport's max_tx_speed via MODIFY_VPORT_STATE. This
series fixes bugs found in that flow.

The bugs:

1. The aggregate was not limited by the NIC's PCIe bandwidth. It was
   taken from the bond speed, which is calculated from PTYS, and PTYS
   is not bounded by the PCI bandwidth of the NIC. FW initializes
   max_tx_speed to the minimum of the port speed and the PCIe
   bandwidth, so modifying it with a value computed by different rules
   changes the meaning of the field rather than updating it.

2. RoCE LAG never told the RDMA layer about speed changes. The NIC is
   considered to be in a bare metal configuration, without a valid or
   enabled eswitch, so there are no VF, SF or PF rep vports to modify
   and no event of speed change was raised.

3. LAG teardown did not reset the vport speeds correctly. It wrote back
   the oper speed of the link, which reads 0 for a down link, and FW
   ignores max_tx_speed=0 as "not provided", so the aggregate stayed on
   the vports. With the link up, it replaced the max_tx_speed of a
   vport with the speed of the link instead of the value the vport had
   before the LAG, for example when FW limits it by the PCIe bandwidth.

Patch #1 caches the TX-enabled aggregate speed and programs that into
        the vports, instead of reading it back from the bond netdev.

Patch #2 limits the aggregated TX speed by the NIC's PCIe bandwidth.

Patch #3 notifies the RDMA layer of LAG aggregate speed changes via a
        new driver event.

Patch #4 saves the max_tx_speed of each vport when it is enabled and
        restores it at LAG teardown, with a link-derived fallback when
        no value was saved or the link is down. It fixes bug 3.

Regards,
Tariq

V2:
- Patch "cache and use TX-enabled aggregate speed for vports": only
  TX-enabled, link-up ports from the LAG tracker are counted in
  non-MPESW modes, which was part of the split patch in v1 and moved
  here, mlx5_lag_update_agg_speed() now returns int and
  propagates query errors, the cached speed starts as unknown and a
  failed measurement invalidates it, so that
  mlx5_lag_query_aggregated_speed() returns -ENODATA instead of a zero
  speed.
- Patch "limit aggregated TX speed by PCIe bandwidth": the aggregate is
  clamped as a whole rather than clamping each port's speed to its own
  PCIe bandwidth before summing, which could let the total reach a
  multiple of a shared link's real bandwidth. Only ports that
  contribute to the aggregate are counted. In Socket Direct the
  bandwidths of the mdevs of a port are summed, and a bandwidth that
  cannot be read fails the calculation instead of skipping the clamp.
- Patch "notify IB uplink on LAG aggregate speed change": the event is
  raised whenever the cached aggregate changes, in any LAG mode, and
  the commit message describes the gap it closes.
- New patch: "Lag, save and restore vport max_tx_speed across LAG".

V1:
https://lore.kernel.org/all/20260910102432.3845360-1-tariqt@xxxxxxxxxx/

Or Har-Toov (4):
  {RDMA,net}/mlx5: cache and use TX-enabled aggregate speed for vports
  net/mlx5: Lag, limit aggregated TX speed by PCIe bandwidth
  {RDMA,net}/mlx5: notify IB uplink on LAG aggregate speed change
  net/mlx5: Lag, save and restore vport max_tx_speed across LAG

 drivers/infiniband/hw/mlx5/main.c             |  12 +-
 .../net/ethernet/mellanox/mlx5/core/esw/qos.c |   1 +
 .../net/ethernet/mellanox/mlx5/core/eswitch.c |  22 ++
 .../net/ethernet/mellanox/mlx5/core/eswitch.h |   1 +
 .../net/ethernet/mellanox/mlx5/core/lag/lag.c | 251 +++++++++++++++---
 .../net/ethernet/mellanox/mlx5/core/lag/lag.h |  17 +-
 drivers/pci/pci.c                             |  16 ++
 include/linux/mlx5/device.h                   |   1 +
 include/linux/mlx5/driver.h                   |   2 +-
 include/linux/pci.h                           |   2 +
 10 files changed, 291 insertions(+), 34 deletions(-)


base-commit: f4377bf85887f57b59abb5479f7660a81ffe4449
-- 
2.44.0