Re: [PATCH 2/4] swiotlb: Add a new cc-swiotlb implementation for Confidential VMs

[Andi Kleen]

On 1/28/2023 12:32 AM, GuoRui.Yu wrote:
> Under COnfidential COmputing (CoCo) scenarios, the VMM cannot access
> guest memory directly but requires the guest to explicitly mark the
> memory as shared (decrypted). To make the streaming DMA mappings work,
> the current implementation relays on legacy SWIOTLB to bounce the DMA
> buffer between private (encrypted) and shared (decrypted) memory.
>
> However, the legacy swiotlb is designed for compatibility rather than
> efficiency and CoCo purpose, which will inevitably introduce some
> unnecessary restrictions.
> 1. Fixed immutable swiotlb size cannot accommodate to requirements of
>     multiple devices. And 1GiB (current maximum size) of swiotlb in our
>     testbed cannot afford multiple disks reads/writes simultaneously.
> 2. Fixed immutable IO_TLB_SIZE (2KiB) cannot satisfy various kinds of
>     devices. At the moment, the minimal size of a swiotlb buffer is 2KiB,
>     which will waste memory on small network packets (under 512 bytes)
>     and decrease efficiency on a large block (up to 256KiB) size
>     reads/writes of disks. And it is hard to have a trade-off on legacy
>     swiotlb to rule them all.
> 3. The legacy swiotlb cannot efficiently support larger swiotlb buffers.
>     In the worst case, the current implementation requires a full scan of
>     the entire swiotlb buffer, which can cause severe performance hits.
>
> Instead of keeping "infecting" the legacy swiotlb code with CoCo logic,
> this patch tries to introduce a new cc-swiotlb for Confidential VMs.
>
> Confidential VMs usually have reasonable modern devices (virtio devices,
> NVME, etc.), which can access memory above 4GiB, cc-swiotlb could
> allocate TLB buffers dynamically on-demand, and this design solves
> problem 1.

When you say solving you mean support for growing the size dynamically 
without pre-allocation?

The IOMMU is traditionally called in non preemptible regions in drivers, 
and also allocating memory in IO paths is still not considered fully 
safe due to potential deadlocks. Both makes it difficult to allocate 
large memory regions dynamically.

It's not clear how you would solve that?

-Andi