[PATCH net-next 0/5] net/mlx5: Introduce mlx5_nodnic, a driver for DRAM-less NVIDIA Mellanox NIC functions

[Tariq Toukan]

Hi,

This series by Shahar introduces mlx5_nodnic, a Linux driver for the
NODNIC (No-DRAM NIC) hardware interface present in Mellanox/NVIDIA
ConnectX-family ASICs since 2015.

NODNIC is a first-class hardware capability.
It defines a lightweight NIC function intended for contexts where local
DRAM is unavailable or too costly: embedded controllers, management
planes, pre-boot environments, or any resource-constrained instantiation
of a ConnectX NIC. The hardware sends and receives ordinary Ethernet
frames via SQs, RQs and CQs. There is no encapsulation, no private
protocol, and no FW interposition: packets are forwarded as standard
Ethernet between whatever peers are connected to the function. The
interface is documented in the HW spec and has been in production in
non-Linux OS drivers for over a decade.

Where it is used today:
- Embedded management controllers on ConnectX-class ASICs that need a
  network path to the host without consuming a data-path PF.
- Resource-constrained SoC integrations (no local DRAM) where a
  minimal-footprint driver is required.
- Pre-boot / early-OS environments (PXE, iSCSI boot) that need a NIC
  without a full mlx5e stack.
- BF4 management PFs (one concrete deployment): on BlueField-4 the
  firmware exposes a NODNIC PF on the host side and a peer PF on the ARM
  side, using standard vport forwarding to connect them. This provides
  in-band Ethernet connectivity between the host and the DPU OS - SSH,
  PXE, and any standard networking application work over it without any
  special handling.

Design notes:
- The driver is separate from mlx5e rather than a mode within it.
  NODNIC uses PCI Vendor Specific Capability (VSC) for command
  operations - reads and writes over PCI config space - instead of the
  DMA-based command interface used by mlx5_core. This is what makes
  NODNIC usable in environments without reliable DMA access, and it
  makes the two command paths fundamentally incompatible. Folding this
  into mlx5e would mean carrying a second command transport and a second
  resource model through code that is already dense.
- A single SQ, RQ and shared CQ are exposed per function, with two MSI-X
  vectors: the data vector fires on CQEs and drives both TX and RX
  completions, and the event vector notifies on port state changes.
  This is the hardware's model, not a driver simplification.
- The driver exposes a plain netdev with no private uAPI, no ioctls and
  no vendor-specific configuration surface. Everything is driven through
  standard netdev, ethtool and devlink interfaces.
- The code lives under drivers/net/ethernet/mellanox/mlx5/nodnic/ and
  shares no paths with mlx5e, keeping the footprint small and the blast
  radius of changes contained.

Regards,
Tariq

Shahar Shitrit (5):
  net/mlx5: nodnic: Introduce the mlx5_nodnic driver
  net/mlx5: nodnic: Implement data path
  net/mlx5: nodnic: Add TX/RX statistics
  net/mlx5: nodnic: Add ethtool driver info
  net/mlx5: nodnic: Add health monitoring

 drivers/net/ethernet/mellanox/Kconfig         |    1 +
 drivers/net/ethernet/mellanox/Makefile        |    1 +
 .../net/ethernet/mellanox/mlx5/nodnic/Kconfig |   18 +
 .../ethernet/mellanox/mlx5/nodnic/Makefile    |    8 +
 .../net/ethernet/mellanox/mlx5/nodnic/en.c    |  263 ++++
 .../net/ethernet/mellanox/mlx5/nodnic/en.h    |  158 +++
 .../ethernet/mellanox/mlx5/nodnic/en_cfg.c    | 1107 +++++++++++++++++
 .../ethernet/mellanox/mlx5/nodnic/health.c    |  315 +++++
 .../net/ethernet/mellanox/mlx5/nodnic/main.c  |  876 +++++++++++++
 .../ethernet/mellanox/mlx5/nodnic/nodnic.h    |  105 ++
 .../mellanox/mlx5/nodnic/nodnic_ifc.h         |  318 +++++
 .../mellanox/mlx5/nodnic/nodnic_pci_vsc.c     |  418 +++++++
 .../mellanox/mlx5/nodnic/nodnic_pci_vsc.h     |   45 +
 .../net/ethernet/mellanox/mlx5/nodnic/rx.c    |  172 +++
 .../net/ethernet/mellanox/mlx5/nodnic/rx.h    |   25 +
 .../net/ethernet/mellanox/mlx5/nodnic/tx.c    |  230 ++++
 .../net/ethernet/mellanox/mlx5/nodnic/tx.h    |   61 +
 17 files changed, 4121 insertions(+)
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/Kconfig
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/en.c
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/health.c
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/main.c
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic.h
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_ifc.h
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.c
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.h
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/rx.h
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/tx.h


base-commit: 47a1446725732cd3996edf607e8739334bbf4d78
-- 
2.44.0