RE: Enabling peer to peer device transactions for PCIe devices

[David Laight]

From: Petrosyan Ludwig
> Sent: 22 October 2017 07:14
> Could be I have done is stupid...
> But at first sight it has to be simple:
> The PCIe Write transactions are address routed, so if in the packet header the other endpoint address
> is written the TLP has to be routed (by PCIe Switch to the endpoint), the DMA reading from the end
> point is really write transactions from the endpoint, usually (Xilinx core) to start DMA one has to
> write to the DMA control register of the endpoint the destination address. So I have change the device
> driver to set in this register the physical address of the other endpoint (get_resource start called
> to other endpoint, and it is the same address which I could see in lspci -vvvv -s bus-address of the
> switch port, memories behind bridge), so now the endpoint has to start send writes TLP with the other
> endpoint address in the TLP header.
> But this is not working (I want to understand why ...), but I could see the first address of the
> destination endpoint is changed (with the wrong value 0xFF),
> now I want to try prepare in the driver of one endpoint the DMA buffer , but using physical address of
> the other endpoint,
> Could be it will never work, but I want to understand why, there is my error ...

It is also worth checking that the hardware actually supports p2p transfers.
Writes are more likely to be supported then reads.
ISTR that some intel cpus support some p2p writes, but there could easily
be errata against them.

I'd certainly test a single word write to read/write memory location.
First verify against kernel memory, then against a 'slave' board.

I don't know about Xilinx fpga, but we've had 'fun' getting Altera fpga
to do sensible PCIe cycles (I ended up writing a simple dma controller 
that would generate long TLP).
We also found a bug in the Altera logic that processed interleaved
read completions.

	David