[RFC PATCH 00/21] Implement NTB Controller using multiple PCI
From: Kishon Vijay Abraham I
Date: Thu Sep 26 2019 - 07:30:46 EST
This series is sent as RFC since this series is dependent on
[1] (cannot be merged before that series) and to get early review
comments.
I'll also split this series into smaller chunk when I post the
next revision.
This series is about implementing SW defined NTB using
multiple endpoint instances. This series has been tested using
2 endpoint instances in J7 connected to two DRA7 boards.
This was presented in Linux Plumbers Conference. The presentation
can be found @ [2]
This series:
*) Add support to define endpoint function using device tree
*) Add a specification for implementing NTB controller using
multiple endpoint instances.
*) Add a NTB endpoint function driver and a NTB host side PCI
driver that follows the specification.
*) Add support in PCIe endpoint core to support secondary
interface.
*) Add a device tree overlay file to configure J7 as NTB
The test setup is something like below
+-------------+ +-------------+
| | | |
| DRA72 | | DRA76 |
| | | |
+------^------+ +------^------+
| |
| |
+---------|-------------------------------------------------|---------+
| +------v------+ +------v------+ |
| | | | | |
| | EP | | EP | |
| | CONTROLLER1 | | CONTROLLER2 | |
| | <-----------------------------------> | |
| | | | | |
| | | | | |
| | | J7 | | |
| | | (Configured using NTB Function) | | |
| +-------------+ +-------------+ |
+---------------------------------------------------------------------+
Here DRA72 and DRA76 could be replaced with *any* PCI host.
EP side (J7):
=============
In the kernel:
cd /sys/kernel/config/pci_ep/
echo 1 > controllers/d800000.pcie-ep/start
echo 1 > controllers/d000000.pcie-ep/start
RC side (DRA7):
===============
echo 0000:01:00.0 > /sys/bus/pci/devices/0000\:01\:00.0/driver/unbind
echo 0000:01:00.0 > /sys/bus/pci/drivers/ntb_hw_epf/bind
modprobe ntb_transport
modprobe ntb_netdev
On each of the hosts Ethernet Interface will be created.
Provide an IP address to each of the hosts:
HOST1 (dra72):
ifconfig eth2 192.168.1.2 up
HOST2 (dra76):
ifconfig eth2 192.168.1.1 up
Once this is done standard network utilities like ping or iperf can be
used.
root@dra7xx-evm:~# iperf -c 192.168.1.2
------------------------------------------------------------
Client connecting to 192.168.1.2, TCP port 5001
TCP window size: 2.50 MByte (default)
------------------------------------------------------------
[ 3] local 192.168.1.1 port 60814 connected with 192.168.1.2 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 705 MBytes 591 Mbits/sec
[1] -> http://lore.kernel.org/r/20190604131516.13596-1-kishon@xxxxxx
[2] -> https://www.linuxplumbersconf.org/event/4/contributions/395/attachments/284/481/Implementing_NTB_Controller_Using_PCIe_Endpoint_-_final.pdf
Kishon Vijay Abraham I (21):
dt-bindings: PCI: Endpoint: Add DT bindings for PCI EPF Bus
dt-bindings: PCI: Endpoint: Add DT bindings for PCI EPF Device
dt-bindings: PCI: Endpoint: Add DT bindings for PCI EPF NTB Device
Documentation: PCI: Add specification for the *PCI NTB* function
device
PCI: endpoint: Add API to get reference to EPC from device-tree
PCI: endpoint: Add API to create EPF device from device tree
PCI: endpoint: Add "pci-epf-bus" driver
PCI: endpoint: Make *_get_first_free_bar() take into account 64 bit
BAR
PCI: endpoint: Add helper API to get the 'next' unreserved BAR
PCI: endpoint: Make pci_epf_driver ops optional
PCI: endpoint: Add helper API to populate header with values from DT
PCI: endpoint: Add support to associate secondary EPC with EPF
PCI: endpoint: Add pci_epc_ops to map MSI irq
PCI: cadence: Implement ->msi_map_irq() ops
PCI: endpoint: Remove unused pci_epf_match_device()
PCI: endpoint: Fix missing mutex_unlock in error case
PCI: endpoint: *_free_bar() to return error codes on failure
PCI: endpoint: Add EP function driver to provide NTB functionality
PCI: Add TI J721E device to pci ids
NTB: Add support for EPF PCI-Express Non-Transparent Bridge
NTB: tool: Enable the NTB/PCIe link on the local or remote side of
bridge
Documentation/PCI/endpoint/pci-test-ntb.txt | 315 +++++
.../bindings/pci/endpoint/pci-epf-bus.txt | 27 +
.../bindings/pci/endpoint/pci-epf-ntb.txt | 31 +
.../bindings/pci/endpoint/pci-epf.txt | 28 +
drivers/ntb/hw/Kconfig | 1 +
drivers/ntb/hw/Makefile | 1 +
drivers/ntb/hw/epf/Kconfig | 5 +
drivers/ntb/hw/epf/Makefile | 1 +
drivers/ntb/hw/epf/ntb_hw_epf.c | 648 ++++++++++
drivers/ntb/test/ntb_tool.c | 1 +
drivers/pci/controller/pcie-cadence-ep.c | 59 +
drivers/pci/endpoint/Makefile | 3 +-
drivers/pci/endpoint/functions/Kconfig | 12 +
drivers/pci/endpoint/functions/Makefile | 1 +
drivers/pci/endpoint/functions/pci-epf-ntb.c | 1143 +++++++++++++++++
drivers/pci/endpoint/functions/pci-epf-test.c | 12 +-
drivers/pci/endpoint/pci-ep-cfs.c | 6 +-
drivers/pci/endpoint/pci-epc-core.c | 221 +++-
drivers/pci/endpoint/pci-epf-bus.c | 54 +
drivers/pci/endpoint/pci-epf-core.c | 133 +-
include/linux/pci-epc.h | 42 +-
include/linux/pci-epf.h | 35 +-
include/linux/pci_ids.h | 1 +
23 files changed, 2715 insertions(+), 65 deletions(-)
create mode 100644 Documentation/PCI/endpoint/pci-test-ntb.txt
create mode 100644 Documentation/devicetree/bindings/pci/endpoint/pci-epf-bus.txt
create mode 100644 Documentation/devicetree/bindings/pci/endpoint/pci-epf-ntb.txt
create mode 100644 Documentation/devicetree/bindings/pci/endpoint/pci-epf.txt
create mode 100644 drivers/ntb/hw/epf/Kconfig
create mode 100644 drivers/ntb/hw/epf/Makefile
create mode 100644 drivers/ntb/hw/epf/ntb_hw_epf.c
create mode 100644 drivers/pci/endpoint/functions/pci-epf-ntb.c
create mode 100644 drivers/pci/endpoint/pci-epf-bus.c
--
2.17.1