Re: [PATCH v10 17/26] docs: gunyah: Document Gunyah VM Manager

[Alex Elder]

On 2/14/23 3:25 PM, Elliot Berman wrote:
> Document the ioctls and usage of Gunyah VM Manager driver.
>
> Signed-off-by: Elliot Berman <quic_eberman@xxxxxxxxxxx>
> ---
>   Documentation/virt/gunyah/index.rst      |   1 +
>   Documentation/virt/gunyah/vm-manager.rst | 106 +++++++++++++++++++++++
>   2 files changed, 107 insertions(+)
>   create mode 100644 Documentation/virt/gunyah/vm-manager.rst
>
> diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst
> index 45adbbc311db..b204b85e86db 100644
> --- a/Documentation/virt/gunyah/index.rst
> +++ b/Documentation/virt/gunyah/index.rst
> @@ -7,6 +7,7 @@ Gunyah Hypervisor
>   .. toctree::
>      :maxdepth: 1
>   
> +   vm-manager
>      message-queue
>   
>   Gunyah is a Type-1 hypervisor which is independent of any OS kernel, and runs in
> diff --git a/Documentation/virt/gunyah/vm-manager.rst b/Documentation/virt/gunyah/vm-manager.rst
> new file mode 100644
> index 000000000000..c0126cfeadc7
> --- /dev/null
> +++ b/Documentation/virt/gunyah/vm-manager.rst
> @@ -0,0 +1,106 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +=======================
> +Virtual Machine Manager
> +=======================
> +
> +The Gunyah Virtual Machine Manager is a Linux driver to support launching
> +virtual machines using Gunyah. It presently supports launching non-proxy
> +scheduled Linux-like virtual machines.
> +
> +Except for some basic information about the location of initial binaries,
> +most of the configuration about a Gunyah virtual machine is described in the
> +VM's devicetree. The devicetree is generated by userspace. Interacting with the
> +virtual machine is still done via the kernel and VM configuration requires some
> +of the corresponding functionality to be set up in the kernel. For instance,
> +sharing userspace memory with a VM is done via the GH_VM_SET_USER_MEM_REGION
> +ioctl. The VM itself is configured to use the memory region via the
> +devicetree.
> +
> +Sample Userspace VMM
> +====================
> +
> +A sample userspace VMM is included in samples/gunyah/ along with a minimal
> +devicetree that can be used to launch a VM. To build this sample, enable
> +CONFIG_SAMPLE_GUNYAH.
> +
> +IOCTLs and userspace VMM flows
> +==============================
> +
> +The kernel exposes a char device interface at /dev/gunyah.
> +
> +To create a VM, use the GH_CREATE_VM ioctl. A successful call will return a
> +"Gunyah VM" file descriptor.
> +
> +/dev/gunyah API Descriptions
> +----------------------------
> +
> +GH_CREATE_VM
> +~~~~~~~~~~~~
> +
> +Creates a Gunyah VM. The argument is reserved for future use and must be 0.

I wouldn't say it "must be zero".  Instead maybe say it is
is currently ignored.

> +
> +Gunyah VM API Descriptions
> +--------------------------
> +
> +GH_VM_SET_USER_MEM_REGION
> +~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +::
> +
> +  struct gh_userspace_memory_region {
> +	__u32 label;
> +	__u32 flags;
> +	__u64 guest_phys_addr;
> +	__u64 memory_size;
> +	__u64 userspace_addr;
> +  };
> +
> +This ioctl allows the user to create or delete a memory parcel for a guest
> +virtual machine. Each memory region is uniquely identified by a label;
> +attempting to create two regions with the same label is not allowed.

Must the label be unique across a single instance of Gunyah (on
a single physical machine)?  Or is it unique within a VM?  Or
something else?  (It's not universally unique, right?)

> +
> +While VMM is guest-agnostic and allows runtime addition of memory regions,
> +Linux guest virtual machines do not support accepting memory regions at runtime.
> +Thus, memory regions should be provided before starting the VM and the VM must
> +be configured to accept these at boot-up.
> +
> +The guest physical address is used by Linux kernel to check that the requested
> +user regions do not overlap and to help find the corresponding memory region
> +for calls like GH_VM_SET_DTB_CONFIG. It should be page aligned.

The physical address must be page aligned.  Can a page be anything
other than 4096 bytes?

> +
> +memory_size and userspace_addr should be page-aligned.

Not should, must, right?  (The address isn't rounded down to a
page boundary, for example.)

> +
> +The flags field of gh_userspace_memory_region accepts the following bits. All
> +other bits must be 0 and are reserved for future use. The ioctl will return
> +-EINVAL if an unsupported bit is detected.
> +
> +  - GH_MEM_ALLOW_READ/GH_MEM_ALLOW_WRITE/GH_MEM_ALLOW_EXEC sets read/write/exec
> +    permissions for the guest, respectively.
> +  - GH_MEM_LENT means that the memory will be unmapped from the host and be
> +    unaccessible by the host while the guest has the region.
> +
> +To add a memory region, call GH_VM_SET_USER_MEM_REGION with fields set as
> +described above.
> +
> +To delete a memory region, call GH_VM_SET_USER_MEM_REGION with label set to the
> +desired region and memory_size set to 0.
> +
> +GH_VM_SET_DTB_CONFIG
> +~~~~~~~~~~~~~~~~~~~~
> +
> +::
> +
> +  struct gh_vm_dtb_config {
> +	__u64 gpa;

What is "gpa"?  Guest physical address?  Gunyah pseudo address?
Can this have a longer and more descriptive name please?

> +	__u64 size;
> +  };
> +
> +This ioctl sets the location of the VM's devicetree blob and is used by Gunyah
> +Resource Manager to allocate resources. The guest physical memory should be part
> +of the primary memory parcel provided to the VM prior to GH_VM_START.

Any alignment constraints?  (If not, you could say "there are no
alignment constraints on the address or size.")

> +
> +GH_VM_START
> +~~~~~~~~~~~
> +
> +This ioctl starts the VM.

Is there anything you can say about what gets returned for
these (at least for significant cases, like permission
problems or something)?

Are IOCTLs the normal way for virtual machine mechanisms
to set up things like this?  (Noob question.)

					-Alex