[PATCH 00/11] Subject: [PATCH for-next 00/11] RDMA/cxi: add the HPE Cassini verbs driver

[Doug Ledford]

This series adds cxiverbs, an RDMA driver for the HPE Slingshot Cassini
NIC. It exposes the NIC through the verbs API using the UET reliable
unconnected (RU) queue pair type from the UET RDMA core series.

cxiverbs does not drive the hardware itself. It is an auxiliary bus
client of the cxi-ss1 Cassini core driver and does everything through
that driver's exported CXI interface.

Patches 1-3 extend the RDMA core:

 - read selected counters, with per-item timestamps, through the
   COUNTERS object, so that a monitor can read a few of the NIC's
   thousands of telemetry values
 - allow UET jobs without an address table. Cassini transmit commands
   carry the destination fabric address directly and the hardware has
   no programmable peer-address table equivalent to the UET job table.
   This relaxes a rule in the UET core series and needs agreement from
   its authors; the commit message gives the hardware reasons.
 - add the Slingshot node type, transport, port capability and a
   driver-owned GID type that carries the 20-bit NID of the port

Patches 4-11 add the driver:

 - the user space ABI is built around explicit resource ownership:
   the CXI service, LNI (logical network interface) and PID are
   allocated explicitly through private uverbs methods instead of as a
   side effect of opening the device
 - VNI-isolated RU queue pairs, native memory registration, services,
   jobs, communication profiles, command and event queues, counting
   triggers, and NIC telemetry read through the counters extension

Dependencies
------------

This series depends on:

 - the Cassini NIC driver series for net-next,
   net: hpe: add HPE Slingshot NIC drivers
   https://lore.kernel.org/netdev/20261030-cxi-netdev-submission-v1-0-8c2c391cc283@xxxxxxx/
 - the UET RDMA core series
   RDMA/core: Ultra Ethernet transport verbs
   https://lore.kernel.org/linux-rdma/20261019-uet-ib-core-v2-0-8bd654f25370@xxxxxxxxxxxx/T/#t

The upcoming cxiverbs rdma-core provider pull request will depend on
this series.

Known open issues
-----------------

Every patch with a known unresolved issue says so in its commit
message. In summary:

 - SR-IOV: patches 9 and 10 carry virtual-function code from an older,
   incomplete version of the Cassini SR-IOV support. V2 will include the
   updated SR-IOV implementation, so please ignore issues in the VF
   paths for now.
 - Patch 9: on a physical function cxi_eq_free() cannot fail. On a
   virtual function it fails if the PF mailbox is unreachable. If that
   happens while the file is closed or the device is removed, the RDMA
   core drops the EQ object without native cleanup, so the VF EQ and its
   references leak. Explicit EQ destroys are retried. This will be fixed
   with the SR-IOV update in V2.

checkpatch
----------

Patches 1-3, 10 and 11 are checkpatch --strict clean. Patches 4-9 still
report CHECK-level items only (157 in total, mostly
PARENTHESIS_ALIGNMENT and OPEN_ENDED_LINE); each patch lists them. This
code is also maintained in internal HPE repositories, so these fixes
need coordination with those repositories and will be made in V2.

Review and testing
------------------

Before posting, the series was reviewed with the Sashiko AI code
review tool and checkpatch. All valid findings are fixed, except the
open items listed above.

Tool use: GitHub Copilot, an AI coding assistant, was used under our
direction to write many of the review fixes, to fold them into these
patches, to write commit messages and to run the build and hardware
tests. Every change was reviewed by the submitter, who takes
responsibility for it as for any other patch.

 - every patch builds incrementally, without compiler warnings, on top
   of the two series it depends on, with an x86_64 configuration that
   enables the driver
 - with this series, the netdev series and the rdma-core provider,
   booted on two Cassini 2 x86_64 nodes, the complete stack
   passed a full test matrix on both nodes: the rdma-core Python suite
   (434 tests; tests that need features this device lacks are
   skipped), resource and VNI isolation tests in default and indexed
   addressing modes, including wrong-VNI rejection, privileged and
   unprivileged CSR access tests, and 36 two-node RU ping-pong and RMA
   traffic cases (polling and event modes) in each direction, 72 in
   total. Native NIC resource accounting was identical before and
   after the traffic, no verbs references remained, and the kernel
   logs had no WARN, BUG or oops reports and no module load errors.
   The only later change is in the netdev series (cxi-eth hardware
   timestamp configuration), which was tested separately on hardware.
 - targeted user space models of the functions changed by the review
   fixes pass under AddressSanitizer and UndefinedBehaviorSanitizer.

Co-developed by Mamila Santosh and Iraimani P, as shown by the trailers
on each patch.

Doug Ledford (11):
  RDMA/core: read selected counters with timestamps
  RDMA/core: allow jobs without an address table
  RDMA/core: add Slingshot identifiers and driver-owned GIDs
  RDMA/cxi: add the native ABI and resource definitions
  RDMA/cxi: add the auxiliary device and driver ops
  RDMA/cxi: add explicit context LNI and PID lifetimes
  RDMA/cxi: add VNI-isolated queue pairs
  RDMA/cxi: add native memory registration
  RDMA/cxi: add services jobs profiles queues and counting triggers
  RDMA/cxi: read the NIC telemetry through counters
  RDMA/cxi: wire the driver into the build

 25 files changed, 5696 insertions(+), 7 deletions(-)

Signed-off-by: Doug Ledford <doug.ledford@xxxxxxx>
---
Doug Ledford (11):
      RDMA/core: read selected counters with timestamps
      RDMA/core: allow jobs without an address table
      RDMA/core: add Slingshot identifiers and driver-owned GIDs
      RDMA/cxi: add the native ABI and resource definitions
      RDMA/cxi: add the auxiliary device and driver ops
      RDMA/cxi: add explicit context LNI and PID lifetimes
      RDMA/cxi: add VNI-isolated queue pairs
      RDMA/cxi: add native memory registration
      RDMA/cxi: add services jobs profiles queues and counting triggers
      RDMA/cxi: read the NIC telemetry through counters
      RDMA/cxi: wire the driver into the build

 MAINTAINERS                                        |    9 +-
 drivers/infiniband/Kconfig                         |    1 +
 drivers/infiniband/core/cache.c                    |    3 +
 drivers/infiniband/core/device.c                   |    1 +
 drivers/infiniband/core/nldev.c                    |    3 +
 drivers/infiniband/core/roce_gid_mgmt.c            |    3 +
 drivers/infiniband/core/sysfs.c                    |    2 +
 .../infiniband/core/uverbs_std_types_counters.c    |   68 +-
 drivers/infiniband/core/uverbs_std_types_job.c     |    3 +-
 drivers/infiniband/core/verbs.c                    |    2 +
 drivers/infiniband/hw/Makefile                     |    1 +
 drivers/infiniband/hw/cxi/Kconfig                  |   11 +
 drivers/infiniband/hw/cxi/Makefile                 |    5 +
 drivers/infiniband/hw/cxi/cxiverbs.h               |  443 +++++
 drivers/infiniband/hw/cxi/cxiverbs_counters.c      |   38 +
 drivers/infiniband/hw/cxi/cxiverbs_ct.c            | 1906 ++++++++++++++++++++
 drivers/infiniband/hw/cxi/cxiverbs_drv.c           |  270 +++
 drivers/infiniband/hw/cxi/cxiverbs_main.c          | 1105 ++++++++++++
 drivers/infiniband/hw/cxi/cxiverbs_mr.c            |  192 ++
 drivers/infiniband/hw/cxi/cxiverbs_qp.c            |  949 ++++++++++
 include/rdma/ib_verbs.h                            |   25 +
 include/uapi/rdma/cxiverbs-abi.h                   |  658 +++++++
 include/uapi/rdma/ib_user_ioctl_cmds.h             |    2 +
 include/uapi/rdma/ib_user_ioctl_verbs.h            |    2 +
 include/uapi/rdma/rdma_netlink.h                   |    1 +
 25 files changed, 5696 insertions(+), 7 deletions(-)
---
base-commit: 306a2f8432f516069b2e673de7a07043132c034f
change-id: 20261030-cxi-for-next-8f50ba2bd59f

Best regards,
--  
Doug Ledford <doug.ledford@xxxxxxx>