Re: [PATCH v4 04/14] PCI/P2PDMA: Clear ACS P2P flags for all devices behind switches

From: Alex Williamson
Date: Thu May 10 2018 - 15:10:23 EST


On Thu, 10 May 2018 18:41:09 +0000
"Stephen Bates" <sbates@xxxxxxxxxxxx> wrote:
> > Reasons is that GPU are giving up on PCIe (see all specialize link like
> > NVlink that are popping up in GPU space). So for fast GPU inter-connect
> > we have this new links.
>
> I look forward to Nvidia open-licensing NVLink to anyone who wants to use it ;-).

No doubt, the marketing for it is quick to point out the mesh topology
of NVLink, but I haven't seen any technical documents that describe the
isolation capabilities or IOMMU interaction. Whether this is included
or an afterthought, I have no idea.

> > Also the IOMMU isolation do matter a lot to us. Think someone using this
> > peer to peer to gain control of a server in the cloud.

>From that perspective, do we have any idea what NVLink means for
topology and IOMMU provided isolation and translation? I've seen a
device assignment user report that seems to suggest it might pretend to
be PCIe compatible, but the assigned GPU ultimately doesn't work
correctly in a VM, so perhaps the software compatibility is only so
deep. Thanks,

Alex