Re: Enabling peer to peer device transactions for PCIe devices

From: Serguei Sagalovitch
Date: Wed Nov 23 2016 - 14:59:09 EST




On 2016-11-23 02:12 PM, Jason Gunthorpe wrote:
On Wed, Nov 23, 2016 at 10:40:47AM -0800, Dan Williams wrote:

I don't think that was designed for the case where the backing memory
is a special/static physical address range rather than anonymous
"System RAM", right?
The hardware doesn't care where the memory is. ODP is just a generic
mechanism to provide demand-fault behavior for a mirrored page table.

ODP has the same issue as everything else, it needs to translate a
page table entry into a DMA address, and we have no API to do that
when the page table points to peer-peer memory.

Jason
I would like to note that for graphics applications (especially for VR support) we
should avoid ODP case at any cost during graphics commands execution due
to requirement to have smooth and predictable playback. We want to load / "pin"
all required resources before graphics processor begin to touch them. This is not
so critical for compute applications. Because only graphics / compute stack
knows which resource will be in used as well as all statistics accordingly only graphics
stack is capable to make the correct decision when and _where_ evict as well
as when and _where_ to put memory back.