[PATCH net-next V2 00/13] net/mlx5e: Add netdev support for data direct

[Tariq Toukan]

Hi,

This patch series by Dragos adds data direct support for netdev devices,
allowing a mlx5 netdev to issue DMA commands through multiple paths.
This feature is critical for improving performance and reaching line
rate in certain environments where issuing PCI transactions over one
path may be significantly faster than over another. These differences
can arise from various PCI generations in the system or the specific
system topology.

Data direct for netdev will work with devmem by returning the data
direct DMA device instead of the DMA device of the NIC. Each PF netdev
registers its own data direct device and support for RX and TX datapath
is added.

The feature is enabled though a ethtool private flag. When this flag
is enabled, the driver will return the data direct device in the
ndo_queue_get_dma_dev op.

When the data direct device is unbound, the netdev is closed, the
memory providers and TX dmabuf bindings mapped against the device are
unmapped through the new netdev_unbind_dma_dev() core helper, the flag
is disabled and the netdev is reopened. The helper does not care which
memory provider is installed: devmem and io_uring zcrx are both
uninstalled through their mp_ops->uninstall callback, like on netdev
unregister.

Regards,
Tariq

V2:
- Patch 2 (new): Detach auxiliary devices on attach failure during load
  in mlx5_load_one_devl_locked().
- Patch 3: Move the data direct notifier unregister to after
  mlx5e_nic_set_rx_mode() in mlx5e_nic_disable(), alongside the other
  NIC-disable cleanups.
- Patch 4: Reword the commit message.
- Patch 6: Size the ICOSQ for the UMR mode of the RQ, since KSM UMR WQEs
  map half the pages of MTT ones.
- Patch 6: Initialize umr_mkey to avoid a maybe-uninitialized warning.
- Patch 7 (new): Use the page_pool DMA sync helper for SHAMPO SKB frags,
  as unreadable netmem is mapped against the data direct device rather
  than rq->pdev.
- Patch 8: Document why the data_direct private flag is not propagated
  to PTP SQs.
- Patch 9: Return the data direct device from ndo_queue_get_dma_dev and
  use it for the page_pool of data direct RQs. Fail with -ENODEV if the
  device is gone. Reject leased RX queues in data direct mode.
  Bound-check queue_index.
- Patch 10: Walk the bindings xarray and report only bindings that are
  still mapped, instead of tracking them on a separate list.
- Patch 11 (new): Add the netdev_unbind_dma_dev() core helper, which
  uninstalls the memory providers (devmem and io_uring zcrx) and unmaps
  TX bindings of the departing DMA device.
- Patch 12: Replace the channel re-creation on unbind with closing the
  netdev, calling netdev_unbind_dma_dev(), disabling the flag and
  reopening. Also release the providers on suspend. This addresses the
  dmabuf binding limitation noted in V1.
- Patch 13: Reject enabling data direct on uplink representors and in
  socket direct mode. Block flag changes while a memory provider is
  installed on an RX queue or RX queues are leased.
- Add Reviewed-by tags from Cosmin where applicable.
- Drop the "SHAMPO required" gap: only enable data direct on an RX queue
  with an unreadable memory provider when SHAMPO is active.

V1:
https://lore.kernel.org/all/20261008132815.654147-1-tariqt@xxxxxxxxxx/

Dragos Tatulea (13):
  net/mlx5: Log the data direct to PF device mapping
  net/mlx5: Detach auxiliary devices on attach failure during load
  net/mlx5e: Register supported netdevs as data direct users
  net/mlx5e: Pre-calculate UMR padding and entry size
  net/mlx5e: Add data direct ethtool private flag
  net/mlx5e: Add data direct RX infrastructure
  net/mlx5e: SHAMPO, Use page_pool DMA sync helper for SKB frags
  net/mlx5e: Add data direct TX infrastructure
  net/mlx5e: Use the correct DMA dev when data_direct is enabled
  net: devmem: add netdev_has_dmabuf_binding() helper
  net: add netdev_unbind_dma_dev() helper
  net/mlx5e: Disable data direct on data direct device unbind
  net/mlx5e: Enable the data direct netdev feature

 .../ethernet/mellanox/mlx5/core/data_direct.c |   9 +-
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |  14 +
 .../ethernet/mellanox/mlx5/core/en/params.c   |  18 +-
 .../ethernet/mellanox/mlx5/core/en/params.h   |   2 +
 .../mellanox/mlx5/core/en/reporter_rx.c       |   1 +
 .../mellanox/mlx5/core/en/reporter_tx.c       |   1 +
 .../ethernet/mellanox/mlx5/core/en_common.c   |  38 ++-
 .../ethernet/mellanox/mlx5/core/en_ethtool.c  |  67 +++++
 .../net/ethernet/mellanox/mlx5/core/en_main.c | 253 +++++++++++++++++-
 .../net/ethernet/mellanox/mlx5/core/en_rx.c   |  33 +--
 .../net/ethernet/mellanox/mlx5/core/en_tx.c   |   4 +-
 .../net/ethernet/mellanox/mlx5/core/main.c    |   1 +
 include/linux/mlx5/driver.h                   |   1 +
 include/net/netdev_queues.h                   |   1 +
 include/net/netmem.h                          |  24 ++
 net/core/devmem.c                             |  85 ++++++
 net/core/devmem.h                             |   6 +
 net/core/netdev_queues.c                      |  51 ++++
 18 files changed, 573 insertions(+), 36 deletions(-)


base-commit: fa5196392691051963726787944db872420dfa75
-- 
2.44.0