Re: [PATCH v4 3/4] vfio: zpci: defining the VFIO headers

[Matthew Rosato]

On 9/19/19 6:49 PM, Alex Williamson wrote:
> On Thu, 19 Sep 2019 16:27:08 -0600
> Alex Williamson <alex.williamson@xxxxxxxxxx> wrote:
> 
>> On Thu, 19 Sep 2019 16:55:57 -0400
>> Matthew Rosato <mjrosato@xxxxxxxxxxxxx> wrote:
>>
>>> On 9/19/19 11:20 AM, Cornelia Huck wrote:  
>>>> On Fri,  6 Sep 2019 20:13:50 -0400
>>>> Matthew Rosato <mjrosato@xxxxxxxxxxxxx> wrote:
>>>>     
>>>>> From: Pierre Morel <pmorel@xxxxxxxxxxxxx>
>>>>>
>>>>> We define a new device region in vfio.h to be able to
>>>>> get the ZPCI CLP information by reading this region from
>>>>> userland.
>>>>>
>>>>> We create a new file, vfio_zdev.h to define the structure
>>>>> of the new region we defined in vfio.h
>>>>>
>>>>> Signed-off-by: Pierre Morel <pmorel@xxxxxxxxxxxxx>
>>>>> Signed-off-by: Matthew Rosato <mjrosato@xxxxxxxxxxxxx>
>>>>> ---
>>>>>  include/uapi/linux/vfio.h      |  1 +
>>>>>  include/uapi/linux/vfio_zdev.h | 35 +++++++++++++++++++++++++++++++++++
>>>>>  2 files changed, 36 insertions(+)
>>>>>  create mode 100644 include/uapi/linux/vfio_zdev.h
>>>>>
>>>>> diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
>>>>> index 8f10748..8328c87 100644
>>>>> --- a/include/uapi/linux/vfio.h
>>>>> +++ b/include/uapi/linux/vfio.h
>>>>> @@ -371,6 +371,7 @@ struct vfio_region_gfx_edid {
>>>>>   * to do TLB invalidation on a GPU.
>>>>>   */
>>>>>  #define VFIO_REGION_SUBTYPE_IBM_NVLINK2_ATSD	(1)
>>>>> +#define VFIO_REGION_SUBTYPE_ZDEV_CLP		(2)    
>>>>
>>>> Using a subtype is fine, but maybe add a comment what this is for?
>>>>     
>>>
>>> Fair point.  Maybe something like "IBM ZDEV CLP is used to pass zPCI
>>> device features to guest"  
>>
>> And if you're going to use a PCI vendor ID subtype, maintain consistent
>> naming, VFIO_REGION_SUBTYPE_IBM_ZPCI_CLP or something.  Ideally there'd
>> also be a reference to the struct provided through this region
>> otherwise it's rather obscure to find by looking for the call to
>> vfio_pci_register_dev_region() and ops defined for the region.  I

Sure, will rename and add reference

>> wouldn't be opposed to defining the region structure here too rather
>> than a separate file, but I guess you're following the example set by
>> ccw.
>>

Indeed.

>>>>>  
>>>>>  /*
>>>>>   * The MSIX mappable capability informs that MSIX data of a BAR can be mmapped
>>>>> diff --git a/include/uapi/linux/vfio_zdev.h b/include/uapi/linux/vfio_zdev.h
>>>>> new file mode 100644
>>>>> index 0000000..55e0d6d
>>>>> --- /dev/null
>>>>> +++ b/include/uapi/linux/vfio_zdev.h
>>>>> @@ -0,0 +1,35 @@
>>>>> +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
>>>>> +/*
>>>>> + * Region definition for ZPCI devices
>>>>> + *
>>>>> + * Copyright IBM Corp. 2019
>>>>> + *
>>>>> + * Author(s): Pierre Morel <pmorel@xxxxxxxxxxxxx>
>>>>> + */
>>>>> +
>>>>> +#ifndef _VFIO_ZDEV_H_
>>>>> +#define _VFIO_ZDEV_H_
>>>>> +
>>>>> +#include <linux/types.h>
>>>>> +
>>>>> +/**
>>>>> + * struct vfio_region_zpci_info - ZPCI information.    
>>>>
>>>> Hm... probably should also get some more explanation. E.g. is that
>>>> derived from a hardware structure?
>>>>     
>>>
>>> The structure itself is not mapped 1:1 to a hardware structure, but it
>>> does serve as a collection of information that was derived from other
>>> hardware structures.
>>>
>>> "Used for passing hardware feature information about a zpci device
>>> between the host and guest" ?
>>>   
>>>>> + *
>>>>> + */
>>>>> +struct vfio_region_zpci_info {
>>>>> +	__u64 dasm;
>>>>> +	__u64 start_dma;
>>>>> +	__u64 end_dma;
>>>>> +	__u64 msi_addr;
>>>>> +	__u64 flags;
>>>>> +	__u16 pchid;
>>>>> +	__u16 mui;
>>>>> +	__u16 noi;
>>>>> +	__u16 maxstbl;
>>>>> +	__u8 version;
>>>>> +	__u8 gid;
>>>>> +#define VFIO_PCI_ZDEV_FLAGS_REFRESH 1
> 
> Why is this defined so far away from the flags field?  I thought it was
> lost at first.  I also wonder what it means... brief descriptions?
> Thanks,
> 

Not sure why Pierre chose to put it here, but I have no issues moving it
up beneath flags.

Otherwise, I am getting the general gist of the feedback:  more comments
to explain what this is doing.

> Alex
> 
>>>>> +	__u8 util_str[];
>>>>> +} __packed;
>>>>> +
>>>>> +#endif    
>>
>> I'm half tempted to suggest that this struct could be exposed directly
>> through an info capability, the trouble is where.  It would be somewhat
>> awkward to pick an arbitrary BAR or config space region to expose this
>> info.  The VFIO_DEVICE_GET_INFO ioctl could include it, but we don't
>> support capabilities on that return structure and I'm not sure it's
>> worth implementing versus the solution here.  Just a thought.  Thanks,
>>
>> Alex
> 
>