Re: [PATCH v4] PCI/DOE: Expose the DOE protocols via sysfs

[Chaitanya Kulkarni]

On 8/10/2023 9:33 AM, Alistair Francis wrote:
> The PCIe 6 specification added support for the Data Object Exchange (DOE).
> When DOE is supported the Discovery Data Object Protocol must be
> implemented. The protocol allows a requester to obtain information about
> the other DOE protocols supported by the device.
> 
> The kernel is already querying the DOE protocols supported and cacheing
> the values. This patch exposes the values via sysfs. This will allow
> userspace to determine which DOE protocols are supported by the PCIe
> device.
> 
> By exposing the information to userspace tools like lspci can relay the
> information to users. By listing all of the supported protocols we can
> allow userspace to parse and support the list, which might include
> vendor specific protocols as well as yet to be supported protocols.
> 
> Each DOE feature is exposed as a single file. The files are empty and
> the information is contained in the file name.
> 
> This uses pci_sysfs_init() instead of the ->is_visible() function as
> is_visible only applies to the attributes under the group. Which
> means that every PCIe device will see a `doe_protos` directory, no
> matter if DOE is supported at all on the device.
> 
> On top of that ->is_visible() is only called
> (fs/sysfs/group.c:create_files()) if there are sub attrs, which we
> don't necessary have. There are no static attrs, instead they are
> all generated dynamically.
> 
> Signed-off-by: Alistair Francis <alistair.francis@xxxxxxx>
> ---
> v4:
>   - Fixup typos in the documentation
>   - Make it clear that the file names contain the information
>   - Small code cleanups
>   - Remove most #ifdefs
>   - Remove extra NULL assignment
> v3:
>   - Expose each DOE feature as a separate file
> v2:
>   - Add documentation
>   - Code cleanups
> 
> We did talk about exposing DOE types under DOE vendor IDs, but I couldn't
> figure out a simple way to do that
> 
>   Documentation/ABI/testing/sysfs-bus-pci |  10 +++
>   drivers/pci/doe.c                       | 104 ++++++++++++++++++++++++
>   drivers/pci/pci-sysfs.c                 |   7 ++
>   include/linux/pci-doe.h                 |   1 +
>   4 files changed, 122 insertions(+)
> 
> diff --git a/Documentation/ABI/testing/sysfs-bus-pci b/Documentation/ABI/testing/sysfs-bus-pci
> index ecf47559f495..e09c51449284 100644
> --- a/Documentation/ABI/testing/sysfs-bus-pci
> +++ b/Documentation/ABI/testing/sysfs-bus-pci
> @@ -500,3 +500,13 @@ Description:
>   		console drivers from the device.  Raw users of pci-sysfs
>   		resourceN attributes must be terminated prior to resizing.
>   		Success of the resizing operation is not guaranteed.
> +
> +What:		/sys/bus/pci/devices/.../doe_protos
> +Date:		August 2023
> +Contact:	Linux PCI developers <linux-pci@xxxxxxxxxxxxxxx>
> +Description:
> +		This directory contains a list of the supported Data Object Exchange (DOE)
> +		features. The feature values are in the file name; the files have no contents.
> +		The value comes from the device and specifies the vendor and
> +		data object type supported. The lower byte is the data object type and the next
> +		two bytes are the vendor ID.

overly long lines above than what is present in the file, please wrap it
to 80 char limit ..

> diff --git a/drivers/pci/doe.c b/drivers/pci/doe.c
> index 1b97a5ab71a9..918872152fb6 100644
> --- a/drivers/pci/doe.c
> +++ b/drivers/pci/doe.c
> @@ -56,6 +56,8 @@ struct pci_doe_mb {
>   	wait_queue_head_t wq;
>   	struct workqueue_struct *work_queue;
>   	unsigned long flags;
> +
> +	struct device_attribute *sysfs_attrs;
>   };
>   
>   struct pci_doe_protocol {
> @@ -92,6 +94,108 @@ struct pci_doe_task {
>   	struct pci_doe_mb *doe_mb;
>   };
>   
> +#ifdef CONFIG_SYSFS
> +static struct attribute *pci_dev_doe_proto_attrs[] = {
> +	NULL,
> +};
> +
> +static const struct attribute_group pci_dev_doe_proto_group = {
> +	.name	= "doe_protos",
> +	.attrs	= pci_dev_doe_proto_attrs,
> +};
> +
> +static void pci_doe_sysfs_remove_desc(struct pci_doe_mb *doe_mb)
> +{
> +	struct device_attribute *attrs = doe_mb->sysfs_attrs;
> +	unsigned long i;
> +	void *entry;
> +
> +	if (!doe_mb->sysfs_attrs)
> +		return;
> +
> +	doe_mb->sysfs_attrs = NULL;
> +	xa_for_each(&doe_mb->prots, i, entry)
> +		kfree(attrs[i].attr.name);
> +
> +	kfree(attrs);
> +}
> +

I didn't understand the need for the above function. It only has one
caller and isn't complicated enough to be open-coded. In fact, it adds 3
local variables simply due to the new function definition, which could
be avoided by open coding the above function, but for whatever reason if
this has already been discussed and decision has been made to keep this
helper, please ignore this comment...

> +static int pci_doe_sysfs_proto_supports(struct pci_dev *pdev, struct pci_doe_mb *doe_mb)

overly long line as compare to existing code ?

> +{
> +	struct device *dev = &pdev->dev;
> +	struct device_attribute *attrs;
> +	unsigned long num_protos = 0;
> +	unsigned long vid, type;
> +	unsigned long i;
> +	void *entry;
> +	int ret;
> +
> +	xa_for_each(&doe_mb->prots, i, entry)
> +		num_protos++;
> +
> +	attrs = kcalloc(num_protos, sizeof(*attrs), GFP_KERNEL);
> +	if (!attrs)
> +		return -ENOMEM;
> +
> +	doe_mb->sysfs_attrs = attrs;
> +	xa_for_each(&doe_mb->prots, i, entry) {
> +		sysfs_attr_init(&attrs[i].attr);
> +		vid = xa_to_value(entry) >> 8;
> +		type = xa_to_value(entry) & 0xFF;
> +		attrs[i].attr.name = kasprintf(GFP_KERNEL, "0x%04lX:%02lX", vid, type);

overly long line as compare to existing code ?

> +		if (!attrs[i].attr.name) {
> +			ret = -ENOMEM;
> +			goto fail;
> +		}
> +
> +		attrs[i].attr.mode = 0444;
> +
> +		ret = sysfs_add_file_to_group(&dev->kobj, &attrs[i].attr,
> +					      pci_dev_doe_proto_group.name);
> +		if (ret)
> +			goto fail;
> +	}
> +
> +	return 0;
> +
> +fail:
> +	pci_doe_sysfs_remove_desc(doe_mb);
> +	return ret;
> +}
> +
> +int doe_sysfs_init(struct pci_dev *pdev)
> +{
> +	unsigned long total_protos = 0;
> +	struct pci_doe_mb *doe_mb;
> +	unsigned long index, j;
> +	void *entry;
> +	int ret;
> +
> +	xa_for_each(&pdev->doe_mbs, index, doe_mb) {
> +		xa_for_each(&doe_mb->prots, j, entry)
> +			total_protos++;
> +	}
> +

value calculated for total_protos is only used to check non-zero case
and the calculated value is not used anywhere apart from following zero
check right ?

> +	if (total_protos == 0)
> +		return 0;
> +

perhaps exit the the loop to avoid unnecessary iterations after
finding first proto with bool to make it clear that we don't care about
the total number, unless there is a bug in the following code consider
something like this (totally untested) :-

diff --git a/drivers/pci/doe.c b/drivers/pci/doe.c 
                                                     index 
918872152fb6..a72efb32453c 100644 
                                               --- a/drivers/pci/doe.c 
 
                            +++ b/drivers/pci/doe.c 
 
         @@ -165,20 +165,23 @@ static int 
pci_doe_sysfs_proto_supports(struct pci_dev *pdev, struct pci_doe_mb 
 
 
  int doe_sysfs_init(struct pci_dev *pdev) 
                                                       { 
 
                                   -       unsigned long total_protos = 0;
+       bool add_doe_group = false;
	struct pci_doe_mb *doe_mb;
	unsigned long index, j;
	void *entry;
	int ret;

	xa_for_each(&pdev->doe_mbs, index, doe_mb) {
-               xa_for_each(&doe_mb->prots, j, entry) 
                                                     - 
     total_protos++;
+               xa_for_each(&doe_mb->prots, j, entry) {
+                       add_doe_group = true;
+                       goto add_doe_group;
+               }
	}
-       if (total_protos == 0)
+       if (!add_doe_group)
		return 0;

+add_doe_group:
	ret = devm_device_add_group(&pdev->dev, &pci_dev_doe_proto_group);
	if (ret) { 

		pci_err(pdev, "can't create DOE goup: %d\n", ret);


-ck