Re: [RFC PATCH v3 0/5] KVM: guest_memfd: Add KVM_LINK_GUEST_MEMFD ioctl

[Ackerley Tng]

Tao Gong <gongtao@xxxxxxxxxx> writes:

Adding a few more people to Cc after today's discussion at guest_memfd
biweekly.

Sean mentioned the link ioctl as the line of sight to sharing memory,
would like people's thoughts on this. Captured some of today's
discussion below.

> [...snip...]
>
> Use cases
> ---------
>
> 1. Intra-host migration of VMs with private memory, where the
>    destination VM takes over the source VM's guest memory in place:
>
>    a. Userspace passes the source VM's guest_memfd fd to the destination
>       VM, e.g. over a Unix socket.
>    b. The destination VM calls KVM_LINK_GUEST_MEMFD to get a new fd for
>       the same inode, bound to the destination VM.
>    c. The destination VM binds the new fd to its memslots with
>       KVM_SET_USER_MEMORY_REGION2, at the same offsets as the source.
>    d. The remaining VM state is moved with KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM
>       (SEV only today; generalized by the follow-up series) and the
>       destination VM is run as normal.
>
>    Linking does not revoke the source VM's access; the source simply
>    stops using the memory, so every private page keeps a single owner.
>
> 2. Sharing guest memory between non-confidential VMs. Both VMs bind the
>    same inode at the same time, each through its own file, and the
>    memory keeps its guest_memfd properties: neither VMM needs a
>    userspace mapping of it, and inode-level operations such as a hole
>    punch apply to every linked VM.
>
>    A related use raised in discussion is Nitro Enclaves-style memory
>    donation, where a guest gives part of its memory to a sibling VM on
>    the same host ("nesting without real nesting").
>

We discussed Fred's "aliased" attribute that was going to be passed to
guest_memfd through .get_pfn(). "aliased" was going to indicate that
this memory is shared between two users of the same memory range.

> Alternatives considered
> -----------------------
>
> [...snip...]
>
> * Keeping the memory and its attributes in a "memory provider" and
>   creating two independent guest_memfd instances on top of it. The
>   guest_memfd inode already plays that role today (page cache,
>   attributes and the list of files), so linking gets the same result
>   without a new abstraction. If guest_memfd grows a provider layer,
>   linking could be expressed on top of it.
>

Sean pointed out that we need to take into account that at some point
there CoCo protected memory might need sharing, where protected
indicates protected for a given guest (and let encryption be orthogonal
to protection).

Then, we'd need to know which struct kvm the memory belongs to, to
determine whether to share or not, and for that, guest_memfd would be
the natural choke point that understands memory, which kvm owns it, and
shared/private state, and hence memory sharing should be handled via
guest_memfd rather than directly through the provider.

>
>
> [...snip...]
>