Re: [PATCH v4 3/4] vfio: zpci: defining the VFIO headers

[Alex Williamson]

On Thu, 19 Sep 2019 16:27:08 -0600
Alex Williamson <alex.williamson@xxxxxxxxxx> wrote:

> On Thu, 19 Sep 2019 16:55:57 -0400
> Matthew Rosato <mjrosato@xxxxxxxxxxxxx> wrote:
> 
> > On 9/19/19 11:20 AM, Cornelia Huck wrote:  
> > > On Fri,  6 Sep 2019 20:13:50 -0400
> > > Matthew Rosato <mjrosato@xxxxxxxxxxxxx> wrote:
> > >     
> > >> From: Pierre Morel <pmorel@xxxxxxxxxxxxx>
> > >>
> > >> We define a new device region in vfio.h to be able to
> > >> get the ZPCI CLP information by reading this region from
> > >> userland.
> > >>
> > >> We create a new file, vfio_zdev.h to define the structure
> > >> of the new region we defined in vfio.h
> > >>
> > >> Signed-off-by: Pierre Morel <pmorel@xxxxxxxxxxxxx>
> > >> Signed-off-by: Matthew Rosato <mjrosato@xxxxxxxxxxxxx>
> > >> ---
> > >>  include/uapi/linux/vfio.h      |  1 +
> > >>  include/uapi/linux/vfio_zdev.h | 35 +++++++++++++++++++++++++++++++++++
> > >>  2 files changed, 36 insertions(+)
> > >>  create mode 100644 include/uapi/linux/vfio_zdev.h
> > >>
> > >> diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
> > >> index 8f10748..8328c87 100644
> > >> --- a/include/uapi/linux/vfio.h
> > >> +++ b/include/uapi/linux/vfio.h
> > >> @@ -371,6 +371,7 @@ struct vfio_region_gfx_edid {
> > >>   * to do TLB invalidation on a GPU.
> > >>   */
> > >>  #define VFIO_REGION_SUBTYPE_IBM_NVLINK2_ATSD	(1)
> > >> +#define VFIO_REGION_SUBTYPE_ZDEV_CLP		(2)    
> > > 
> > > Using a subtype is fine, but maybe add a comment what this is for?
> > >     
> > 
> > Fair point.  Maybe something like "IBM ZDEV CLP is used to pass zPCI
> > device features to guest"  
> 
> And if you're going to use a PCI vendor ID subtype, maintain consistent
> naming, VFIO_REGION_SUBTYPE_IBM_ZPCI_CLP or something.  Ideally there'd
> also be a reference to the struct provided through this region
> otherwise it's rather obscure to find by looking for the call to
> vfio_pci_register_dev_region() and ops defined for the region.  I
> wouldn't be opposed to defining the region structure here too rather
> than a separate file, but I guess you're following the example set by
> ccw.
> 
> > >>  
> > >>  /*
> > >>   * The MSIX mappable capability informs that MSIX data of a BAR can be mmapped
> > >> diff --git a/include/uapi/linux/vfio_zdev.h b/include/uapi/linux/vfio_zdev.h
> > >> new file mode 100644
> > >> index 0000000..55e0d6d
> > >> --- /dev/null
> > >> +++ b/include/uapi/linux/vfio_zdev.h
> > >> @@ -0,0 +1,35 @@
> > >> +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
> > >> +/*
> > >> + * Region definition for ZPCI devices
> > >> + *
> > >> + * Copyright IBM Corp. 2019
> > >> + *
> > >> + * Author(s): Pierre Morel <pmorel@xxxxxxxxxxxxx>
> > >> + */
> > >> +
> > >> +#ifndef _VFIO_ZDEV_H_
> > >> +#define _VFIO_ZDEV_H_
> > >> +
> > >> +#include <linux/types.h>
> > >> +
> > >> +/**
> > >> + * struct vfio_region_zpci_info - ZPCI information.    
> > > 
> > > Hm... probably should also get some more explanation. E.g. is that
> > > derived from a hardware structure?
> > >     
> > 
> > The structure itself is not mapped 1:1 to a hardware structure, but it
> > does serve as a collection of information that was derived from other
> > hardware structures.
> > 
> > "Used for passing hardware feature information about a zpci device
> > between the host and guest" ?
> >   
> > >> + *
> > >> + */
> > >> +struct vfio_region_zpci_info {
> > >> +	__u64 dasm;
> > >> +	__u64 start_dma;
> > >> +	__u64 end_dma;
> > >> +	__u64 msi_addr;
> > >> +	__u64 flags;
> > >> +	__u16 pchid;
> > >> +	__u16 mui;
> > >> +	__u16 noi;
> > >> +	__u16 maxstbl;
> > >> +	__u8 version;
> > >> +	__u8 gid;
> > >> +#define VFIO_PCI_ZDEV_FLAGS_REFRESH 1

Why is this defined so far away from the flags field?  I thought it was
lost at first.  I also wonder what it means... brief descriptions?
Thanks,

Alex

> > >> +	__u8 util_str[];
> > >> +} __packed;
> > >> +
> > >> +#endif    
> 
> I'm half tempted to suggest that this struct could be exposed directly
> through an info capability, the trouble is where.  It would be somewhat
> awkward to pick an arbitrary BAR or config space region to expose this
> info.  The VFIO_DEVICE_GET_INFO ioctl could include it, but we don't
> support capabilities on that return structure and I'm not sure it's
> worth implementing versus the solution here.  Just a thought.  Thanks,
> 
> Alex