Re: [PATCH V7 1/5] swiotlb: Add swiotlb bounce buffer remap function for HV IVM

[Tianyu Lan]

On 12/15/2021 6:40 AM, Dave Hansen wrote:
> On 12/14/21 2:23 PM, Tom Lendacky wrote:
> > > I don't really understand how this can be more general any *not* get
> > > utilized by the existing SEV support.
> >
> > The Virtual Top-of-Memory (VTOM) support is an SEV-SNP feature that is
> > meant to be used with a (relatively) un-enlightened guest. The idea is
> > that the C-bit in the guest page tables must be 0 for all accesses. It
> > is only the physical address relative to VTOM that determines if the
> > access is encrypted or not. So setting sme_me_mask will actually cause
> > issues when running with this feature. Since all DMA for an SEV-SNP
> > guest must still be to shared (unencrypted) memory, some enlightenment
> > is needed. In this case, memory mapped above VTOM will provide that via
> > the SWIOTLB update. For SEV-SNP guests running with VTOM, they are
> > likely to also be running with the Reflect #VC feature, allowing a
> > "paravisor" to handle any #VCs generated by the guest.
> >
> > See sections 15.36.8 "Virtual Top-of-Memory" and 15.36.9 "Reflect #VC"
> > in volume 2 of the AMD APM [1].
>
> Thanks, Tom, that's pretty much what I was looking for.
>
> The C-bit normally comes from the page tables.  But, the hardware also
> provides an alternative way to effectively get C-bit behavior without
> actually setting the bit in the page tables: Virtual Top-of-Memory
> (VTOM).  Right?
>
> It sounds like Hyper-V has chosen to use VTOM instead of requiring the
> guest to do the C-bit in its page tables.
>
> But, the thing that confuses me is when you said: "it (VTOM) is meant to
> be used with a (relatively) un-enlightened guest".  We don't have an
> unenlightened guest here.  We have Linux, which is quite enlightened.
>
> > Is VTOM being used because there's something that completely rules out
> > using the C-bit in the page tables?  What's that "something"?


For "un-enlightened" guest, there is an another system running insider
the VM to emulate some functions(tsc, timer, interrupt and so on) and
this helps not to modify OS(Linux/Windows) a lot. In Hyper-V Isolation
VM, we called the new system as HCL/paravisor. HCL runs in the VMPL0 and 
Linux runs in VMPL2. This is similar with nested virtualization. HCL
plays similar role as L1 hypervisor to emulate some general functions
(e.g, rdmsr/wrmsr accessing and interrupt injection) which needs to be
enlightened in the enlightened guest. Linux kernel needs to handle
#vc/#ve exception directly in the enlightened guest. HCL handles such
exception in un-enlightened guest and emulate interrupt injection which
helps not to modify OS core part code. Using vTOM also is same purpose.
Hyper-V uses vTOM avoid changing page table related code in OS(include
Windows and Linux)and just needs to map memory into decrypted address
space above vTOM in the driver code.

Linux has generic swiotlb bounce buffer implementation and so introduce
swiotlb_unencrypted_base here to set shared memory boundary or vTOM.
Hyper-V Isolation VM is un-enlightened guest. Hyper-V doesn't expose 
sev/sme capability to guest and so SEV code actually doesn't work.
So we also can't interact current existing SEV code and these code is
for enlightened guest support without HCL/paravisor. If other platforms
or SEV want to use similar vTOM feature, swiotlb_unencrypted_base can
be reused. So swiotlb_unencrypted_base is a general solution for all
platforms besides SEV and Hyper-V.