Re: Summary of LPC guest MSI discussion in Santa Fe

From: Don Dutile
Date: Wed Nov 09 2016 - 13:59:15 EST


On 11/09/2016 12:03 PM, Will Deacon wrote:
On Tue, Nov 08, 2016 at 09:52:33PM -0500, Don Dutile wrote:
On 11/08/2016 06:35 PM, Alex Williamson wrote:
On Tue, 8 Nov 2016 21:29:22 +0100
Christoffer Dall <christoffer.dall@xxxxxxxxxx> wrote:
Is my understanding correct, that you need to tell userspace about the
location of the doorbell (in the IOVA space) in case (2), because even
though the configuration of the device is handled by the (host) kernel
through trapping of the BARs, we have to avoid the VFIO user programming
the device to create other DMA transactions to this particular address,
since that will obviously conflict and either not produce the desired
DMA transactions or result in unintended weird interrupts?

Yes, that's the crux of the issue.

Correct, if the MSI doorbell IOVA range overlaps RAM in the VM, then
it's potentially a DMA target and we'll get bogus data on DMA read from
the device, and lose data and potentially trigger spurious interrupts on
DMA write from the device. Thanks,

That's b/c the MSI doorbells are not positioned *above* the SMMU, i.e.,
they address match before the SMMU checks are done. if
all DMA addrs had to go through SMMU first, then the DMA access could
be ignored/rejected.

That's actually not true :( The SMMU can't generally distinguish between MSI
writes and DMA writes, so it would just see a write transaction to the
doorbell address, regardless of how it was generated by the endpoint.

Will

So, we have real systems where MSI doorbells are placed at the same IOVA
that could have memory for a guest, but not at the same IOVA as memory on real hw ?
How are memory holes passed to SMMU so it doesn't have this issue for bare-metal
(assign an IOVA that overlaps an MSI doorbell address)?