Re: [RFC PATCH v1 0/6] KVM/VFIO: guest_memfd support for device MMIO resources

[Ackerley Tng]

"Aneesh Kumar K.V (Arm)" <aneesh.kumar@xxxxxxxxxx> writes:

> This series adds a guest_memfd device-resource framework for exposing PCI
> BAR memory to guests. The intended use is confidential device assignment,
> where KVM needs to coordinate private device mappings with shared access
> through VFIO. The Arm CCA backend is provided separately; these patches
> contain the guest_memfd, VFIO and IOMMUFD interfaces it uses.
>

Thank you Aneesh!

I definitely like this since it builds upon [1], the model where
guest_memfd coordinates zapping during conversions for the
provider. Please see comparison with the alternative at [2].

>
> [...snip...]
>

I have questions:

> Userspace creates a device-backed guest_memfd using the ordinary VFIO
> device fd as resource_fd, with GUEST_MEMFD_FLAG_USE_RESOURCE,
> GUEST_MEMFD_FLAG_DEVICE

With the model described at [1][2][3], I was thinking that the VFIO fd
would be the resource fd. I'm hoping that there wouldn't be a need for
GUEST_MEMFD_FLAG_DEVICE.

The plan [3] was for resource fd to represent just the resource but not
provide any way to access the underlying resource, so that after
.attach(), guest_memfd's fd should be the way userspace would access the
underlying resource.

For tmpfs, the example in [1], fsmount() returns an fd that has no
.mmap/.read/.write ops, which fits the plan above perfectly. It tells
guest_memfd about the resource, but does not give userspace access to
the resource.

In VFIO's case, vfio_device_fops does implement .mmap()/.read/.write,
and I don't think there's a better existing fd that represents a single
device.

> and GUEST_MEMFD_FLAG_INIT_SHARED. Eligible BAR
> intervals are registered as guest_memfd memslots. File offsets use the
> existing VFIO region-offset namespace, allowing one guest_memfd to cover
> multiple BAR intervals. Device-backed guest_memfd rejects mmap and
> fallocate; shared guest faults obtain bare BAR PFNs from the provider
> instead of resolving the memslot HVA or allocating folios.
>

The model I have in mind is kind of opposite from above. Instead of the
guest_memfd rejecting mmap(), I'm thinking that the VFIO fd should be
the one that rejects mmap(), read() and write() once guest_memfd
attaches to it.

If userspace needed to mmap() the VFIO fd, it should instead mmap() the
guest_memfd fd. guest_memfd doesn't support read() and write() now, but
support can be added. The read/write for guest_memfd just forwards to
vfio if the offset range is shared and blocks if the range is
private.

For mmap(), for regular memory, mmap() is always allowed, but faulting
dependsd on shared/private status. It would be nice if the same logic
could apply to guest_memfds that wrap VFIO.

I prefer the userspace accesses to go through guest_memfd to VFIO so
that the shared/private-ness can be enforced from guest_memfd.


Is GUEST_MEMFD_FLAG_DEVICE there because GUEST_MEMFD_FLAG_USE_RESOURCE
tells guest_memfd to look up
resource_inode->i_sb->s_op->gmem_provider_ops but for VFIO,
gmem_provider_ops can't be stored in super_operations? If that's the
only reason, that's a very good reason to store gmem_provider_ops in
file_operations for gmem_provider_ops to serve different kinds resources
and include VFIO.

I also noticed the need to guest_memfd_register_device_fops, so that
guest_memfd knows the device supports being attached to?

What do you think of this: we have a common attach function for all VFIO
fds in file_operations that would look up the struct vfio_device to
determine if guest_memfd can attach to the vfio device? Actually, which
devices would not support attaching? Wouldn't all devices support
attaching? I'm thinking that using a device that is all-shared via
guest_memfd should be no different from using the device directly. All
operations should flow through to the device.


Regarding conversions, I would like to check: with Confidential IO, are
conversions at runtime actually necessary?

Is the only "conversion" the one when the guest accepts a device? If so,
maybe that "conversion" can be implemented by replacing memslots.

Pre-accept, mmap() the vfio fd directly, set it up in memslots.

Post-accept, remove the old memslot, wrap the vfio fd with guest_memfd,
set up the guest_memfd statically with shared/private ranges as the TDI
report requested and never allow conversions, replace the pre-accept
memslot.

> IOMMUFD resolves and pins the device's existing vDEVICE for the VM and
> serializes MMIO conversion through its MMIO mutex. VFIO supplies BAR PFNs,
> checks eligible ranges and excludes independent dma-buf exports while a
> guest_memfd provider is attached. VFIO host mmap faults, read/write and
> ioeventfd accesses check the requested range's guest_memfd attributes.
> Conversion revokes cached host mappings and shared guest mappings before
> installing private mappings. Other shared intervals remain accessible.
>
>
> [...snip...]
>

[1] https://lore.kernel.org/all/20260925-gmem-tmpfs-backend-v1-0-d36159822d18@xxxxxxxxxx/
[2] https://lore.kernel.org/all/CAEvNRgEseLO+LbkwXrEit0H2LH5Wx-srSvHEMpt0=upPL3tF=g@xxxxxxxxxxxxxx/
[3] https://lpc.events/event/20/contributions/2551/attachments/2047/4693/lpc-2026-gmem-other-backends.pdf