Re: [PATCH v1 1/2] vfio:iommu: Use capabilities do report IOMMU informations

[Pierre Morel]

On 09/01/2019 20:43, Alex Williamson wrote:
> On Wed, 9 Jan 2019 18:07:19 +0100
> Pierre Morel <pmorel@xxxxxxxxxxxxx> wrote:
>
> > On 09/01/2019 16:37, Alex Williamson wrote:
> > > On Wed,  9 Jan 2019 13:41:53 +0100
> > > Pierre Morel <pmorel@xxxxxxxxxxxxx> wrote:
> > >    
> > > > We add a new flag, VFIO_IOMMU_INFO_CAPABILITIES, inside the
> > > > vfio_iommu_type1_info to specify the support for capabilities.
> > > >
> > > > We add a new capability, with id VFIO_IOMMU_INFO_CAP_DMA
> > > > in the capability list of the VFIO_IOMMU_GET_INFO ioctl.
> > > >
> > > > Signed-off-by: Pierre Morel <pmorel@xxxxxxxxxxxxx>
> > > > ---
> > > >    include/uapi/linux/vfio.h | 9 +++++++++
> > > >    1 file changed, 9 insertions(+)
> > > >
> > > > diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
> > > > index 8131028..54c4fcb 100644
> > > > --- a/include/uapi/linux/vfio.h
> > > > +++ b/include/uapi/linux/vfio.h
> > > > @@ -669,6 +669,15 @@ struct vfio_iommu_type1_info {
> > > >    	__u32	flags;
> > > >    #define VFIO_IOMMU_INFO_PGSIZES (1 << 0)	/* supported page sizes info */
> > > >    	__u64	iova_pgsizes;		/* Bitmap of supported page sizes */
> > > > +#define VFIO_IOMMU_INFO_CAPABILITIES (1 << 1)  /* support capabilities info */
> > > > +	__u64   cap_offset;     /* Offset within info struct of first cap */
> > > > +};
> > > > +
> > > > +#define VFIO_IOMMU_INFO_CAP_DMA 1
> > > > +struct vfio_iommu_cap_dma {
> > > > +	struct vfio_info_cap_header header;
> > > > +	__u64   dma_start;
> > > > +	__u64   dma_end;
> > > >    };
> > > >    
> > > >    #define VFIO_IOMMU_GET_INFO _IO(VFIO_TYPE, VFIO_BASE + 12)
> > >
> > > Unfortunately for most systems, a simple start and end is not really
> > > sufficient to describe the available IOVA space, there are often
> > > reserved regions intermixed, so this is not really a complete
> > > solution.  Shameer tried to solve this last year[1] but we ran into a
> > > road block that Intel IGD devices impose a reserved range of IOVA
> > > spaces reported to the user that conflict with existing assignment of
> > > this device and we haven't figured out yet how to be more selective of
> > > the enforcement of those reserved ranges.  Thanks,
> > >
> > > Alex
> > >
> > > [1] https://lkml.org/lkml/2018/4/18/293
> > >    
> >
> > I understand that some architecture may be more complex and have special
> > needs.
> > However the IOMMU geometry is a constant for all IOMMU devices and
> > is reported by the geometry in the iommu operations.
> >
> > This makes the IOMMU geometry a special case.
>
> I'm not so sure that the geometry is a constant for all IOMMU devices,

Sorry, I did not express myself correctly, what I mean is that the IOMMU 
geometry is independent of the system memory map but is a constant of 
the IOMMU device.
May be different for another IOMMU device, as we can have several 
different IOMMU device, typically one per PCI device, in the Z architecture.

> nor am I sure how if that were true and it's part of an in-kernel
> interface that it automatically qualifies it as the right way to expose
> it to userspace.  The fact that we have a reserved region interface to
> augment a basic contiguous range suggests it's known to be insufficient
> even for in-kernel use.
>
> > It is also a special case because it is an inclusive description of
> > available memory, to oppose to the exclusive description given by the
> > windows.
>
> Geometry doesn't really have anything to do with available memory, it's
> the minimum and maximum IOVA aperture.  Shameer's proposal gave us an
> IOVA list, which is based on the IOMMU geometry, from which it excludes
> various reserved ranges.  So if you have a less complex architecture,
> you might only have one entry in the list, which gives you the start
> and end of the base geometry.  Move complex architectures might have
> more entries, but the geometry can still be deduced from the absolute
> highest and lowest addresses within the list.  Therefore a basic
> geometry capability is automatically redundant to the interface that's
> already been proposed.
>
> > Isn't it possible to separate the IOMMU geometry, which is really
> > related to the IOMMU chip, from other windows exclusion related to the
> > system memory mapping?
>
> Why would we ever have both given the description above?

My idea for this is based on that restrictions are comming from two 
different address spaces:
- One address space is the device view before IOMMU (aperture)
- The other is the physical system view (reserved windows)

which, I thought, makes things difficult to merge.
However reading you and Shameer and the pointers you both sent, I 
understand that there are much more for me to learn, especially about 
other architectures (Intel / RMRR)

> > Retrieving the IOMMU geometry is very important for us because the
> > driver inside the guest must get it and program the IOMMU based on these
> > values.
>
> So you have motivation to help move the IOVA list proposal forward,
> or some equally inclusive proposal that isn't just a stop-gap ;)
> Thanks,
>
> Alex

I will be happy to help.
I will need some time to catch up however.
Thanks for your comments.

Regards,
Pierre

--
Pierre Morel
Linux/KVM/QEMU in BÃblingen - Germany