[PATCH v7 00/10] PCI: endpoint: Remote DMA support via vNTB

[Koichiro Den]

Hi,

This series lets host Linux use EPC-local DMA read (RC-to-EP) channels
through vNTB. I also tried a separate PCI DMA EPF, which reached v7 [1],
but came back to this vNTB-embedded approach after discussion.

An EPC-local DMA engine can use host DMA addresses (IOVAs when an IOMMU
is enabled) as PCI addresses to reach any host buffer mapped for the
endpoint. Letting the RC side drive its read channels also gives the RC
a direct path into endpoint-local buffers.

A major motivation is direct TX in ntb_transport. Both sides could use
the same dmaengine-based TX path (ie. symmetric), sending directly to RX
buffer DMA addresses supplied by the peer. This would avoid MW bounce
buffers and per-transfer MW remapping for payloads in both directions.

This assumes both sides trust each other. The RC side still programs DMA
over PCIe MMIO, so this is expected to pay off for large data transfers.

DesignWare eDMA is the only supported dmaengine backend for now. Other
EPC-local DMA engines can be added if there is a need and they have a
suitable programming interface.

This series does not include a consumer for the dmaengine channels exposed
to RC Linux. Native HDMA support is also left out to keep the series from
growing too large.

Setting pci-epf-vntb's 'dma_bar' configfs attribute enables DMA export.
The default is -1, which leaves it disabled. pci-epf-ntb only switches to
the shared control layout header and is functionally unchanged.


Dependency
==========

(Re)based on next-20261020, which includes the following dmaengine series
needed to build these patches:

  [PATCH v4 0/3] dmaengine: dw-edma: Prepare channels for remote use
  https://lore.kernel.org/r/20260912174023.1227397-1-den@xxxxxxxxxxxxx/


Proposed merge plan
===================

Patches 1-7 are for the PCI tree, and patches 8-10 for the NTB tree.
The NTB patches need the shared header as updated by patches 4-6.


Compatibility matrix
====================

Below, "Old" means next-20261020 without this series, and "new" means with it
applied. The table describes expected (and tested) behavior.

  EP                              | Old host     | New host
  --------------------------------+--------------+----------
  Old                             | OK*          | OK*
  New (dma_bar = -1)              | OK*          | OK*
  New (dedicated DMA BAR)         | OK*          | OK
  New (shared MW/DMA BAR)         | Rejected**   | OK

  *  NTB works without DMA.
  ** LINK_UP is rejected with an error message. In v6, an "Old" host
     could mistake the entire shared BAR for an MW and overwrite DMA
     registers or descriptor memory.


Testing
=======

The existing MW-based ntb_netdev path worked with DMA export disabled
(dma_bar = -1) and enabled with the following EP-side BAR settings:

* Dedicated DMA BAR configuration on RK3588 Rock 5B:

  ctrl_bar = 0
  mw1_bar  = 2
  db_bar   = 4
  dma_bar  = 5

* MW-shared DMA BAR configuration on R-Car S4 Spider:

  ctrl_bar = 0
  mw1_bar  = 2
  dma_bar  = 2
  db_bar   = 4

I also tested a v7 EP with a shared MW/DMA BAR against a next-20261020
host. LINK_UP was rejected with "Host does not support MW/DMA BAR sharing".


Best regards,
Koichiro
---
Changes in v7:
  - Split and reorder the patches for the PCI and NTB trees. (Frank)
  - Reject LINK_UP from hosts without MW/DMA boundary support when
    a BAR is shared. (Frank)
  - Rebase on next-20261020.
  - Link to v6: https://lore.kernel.org/r/20261020054948.1218370-1-den@xxxxxxxxxxxxx/

Changes in v6:
  - Add a prep patch that moves the control region layout to
    include/linux/pci-epf-ntb.h, shared by pci-epf-ntb, pci-epf-vntb and
    ntb_hw_epf. (Frank)

For earlier changes, see v5:
https://lore.kernel.org/r/20261009045949.1109871-1-den@xxxxxxxxxxxxx/

[1] Superseded approach, PCI DMA EPF, v7:
    https://lore.kernel.org/r/20260813063757.3131865-1-den@xxxxxxxxxxxxx/


Koichiro Den (10):
  PCI: endpoint: Add DMA auxiliary resource metadata
  PCI: dwc: Expose endpoint DMA resources
  PCI: endpoint: pci-epf-vntb: Move epf_ntb_is_bar_used() up
  PCI: endpoint: Add pci-epf-ntb.h for the shared NTB control layout
  PCI: endpoint: pci-epf-vntb: Export endpoint DMA channels
  PCI: endpoint: pci-epf-vntb: Allow DMA and MW to share a BAR
  Documentation: PCI: endpoint: Document vNTB DMA export
  NTB: ntb_hw_epf: Use the shared NTB control layout
  NTB: ntb_hw_epf: Discover vNTB-embedded DMA
  NTB: ntb_hw_epf: Support DMA and MW sharing a BAR

 Documentation/PCI/endpoint/pci-vntb-howto.rst |  31 +-
 MAINTAINERS                                   |   1 +
 drivers/ntb/hw/epf/ntb_hw_epf.c               | 410 +++++++--
 .../pci/controller/dwc/pcie-designware-ep.c   | 107 ++-
 drivers/pci/endpoint/functions/pci-epf-ntb.c  |  85 +-
 drivers/pci/endpoint/functions/pci-epf-vntb.c | 790 +++++++++++++++---
 include/linux/pci-epc.h                       |  34 +-
 include/linux/pci-epf-ntb.h                   |  90 ++
 8 files changed, 1272 insertions(+), 276 deletions(-)
 create mode 100644 include/linux/pci-epf-ntb.h


base-commit: 8f7df005a7564bd7ffa75da9c3085d5b996f1e16
-- 
2.54.0