Re: [PATCH v4 3/4] vfio: zpci: defining the VFIO headers

[Alex Williamson]

On Thu, 19 Sep 2019 16:55:57 -0400
Matthew Rosato <mjrosato@xxxxxxxxxxxxx> wrote:

> On 9/19/19 11:20 AM, Cornelia Huck wrote:
> > On Fri,  6 Sep 2019 20:13:50 -0400
> > Matthew Rosato <mjrosato@xxxxxxxxxxxxx> wrote:
> >   
> >> From: Pierre Morel <pmorel@xxxxxxxxxxxxx>
> >>
> >> We define a new device region in vfio.h to be able to
> >> get the ZPCI CLP information by reading this region from
> >> userland.
> >>
> >> We create a new file, vfio_zdev.h to define the structure
> >> of the new region we defined in vfio.h
> >>
> >> Signed-off-by: Pierre Morel <pmorel@xxxxxxxxxxxxx>
> >> Signed-off-by: Matthew Rosato <mjrosato@xxxxxxxxxxxxx>
> >> ---
> >>  include/uapi/linux/vfio.h      |  1 +
> >>  include/uapi/linux/vfio_zdev.h | 35 +++++++++++++++++++++++++++++++++++
> >>  2 files changed, 36 insertions(+)
> >>  create mode 100644 include/uapi/linux/vfio_zdev.h
> >>
> >> diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
> >> index 8f10748..8328c87 100644
> >> --- a/include/uapi/linux/vfio.h
> >> +++ b/include/uapi/linux/vfio.h
> >> @@ -371,6 +371,7 @@ struct vfio_region_gfx_edid {
> >>   * to do TLB invalidation on a GPU.
> >>   */
> >>  #define VFIO_REGION_SUBTYPE_IBM_NVLINK2_ATSD	(1)
> >> +#define VFIO_REGION_SUBTYPE_ZDEV_CLP		(2)  
> > 
> > Using a subtype is fine, but maybe add a comment what this is for?
> >   
> 
> Fair point.  Maybe something like "IBM ZDEV CLP is used to pass zPCI
> device features to guest"

And if you're going to use a PCI vendor ID subtype, maintain consistent
naming, VFIO_REGION_SUBTYPE_IBM_ZPCI_CLP or something.  Ideally there'd
also be a reference to the struct provided through this region
otherwise it's rather obscure to find by looking for the call to
vfio_pci_register_dev_region() and ops defined for the region.  I
wouldn't be opposed to defining the region structure here too rather
than a separate file, but I guess you're following the example set by
ccw.

> >>  
> >>  /*
> >>   * The MSIX mappable capability informs that MSIX data of a BAR can be mmapped
> >> diff --git a/include/uapi/linux/vfio_zdev.h b/include/uapi/linux/vfio_zdev.h
> >> new file mode 100644
> >> index 0000000..55e0d6d
> >> --- /dev/null
> >> +++ b/include/uapi/linux/vfio_zdev.h
> >> @@ -0,0 +1,35 @@
> >> +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
> >> +/*
> >> + * Region definition for ZPCI devices
> >> + *
> >> + * Copyright IBM Corp. 2019
> >> + *
> >> + * Author(s): Pierre Morel <pmorel@xxxxxxxxxxxxx>
> >> + */
> >> +
> >> +#ifndef _VFIO_ZDEV_H_
> >> +#define _VFIO_ZDEV_H_
> >> +
> >> +#include <linux/types.h>
> >> +
> >> +/**
> >> + * struct vfio_region_zpci_info - ZPCI information.  
> > 
> > Hm... probably should also get some more explanation. E.g. is that
> > derived from a hardware structure?
> >   
> 
> The structure itself is not mapped 1:1 to a hardware structure, but it
> does serve as a collection of information that was derived from other
> hardware structures.
> 
> "Used for passing hardware feature information about a zpci device
> between the host and guest" ?
> 
> >> + *
> >> + */
> >> +struct vfio_region_zpci_info {
> >> +	__u64 dasm;
> >> +	__u64 start_dma;
> >> +	__u64 end_dma;
> >> +	__u64 msi_addr;
> >> +	__u64 flags;
> >> +	__u16 pchid;
> >> +	__u16 mui;
> >> +	__u16 noi;
> >> +	__u16 maxstbl;
> >> +	__u8 version;
> >> +	__u8 gid;
> >> +#define VFIO_PCI_ZDEV_FLAGS_REFRESH 1
> >> +	__u8 util_str[];
> >> +} __packed;
> >> +
> >> +#endif  

I'm half tempted to suggest that this struct could be exposed directly
through an info capability, the trouble is where.  It would be somewhat
awkward to pick an arbitrary BAR or config space region to expose this
info.  The VFIO_DEVICE_GET_INFO ioctl could include it, but we don't
support capabilities on that return structure and I'm not sure it's
worth implementing versus the solution here.  Just a thought.  Thanks,

Alex