Re: [PATCH V3 2/2] Xen/PCIback: Implement PCI flr/slot/bus reset with 'reset' SysFS attribute

[Govinda Tatti]

Thanks Jan for your review comments. Please see below for my comments.

On 12/8/2017 3:34 AM, Jan Beulich wrote:
> > > > On 07.12.17 at 23:21,<Govinda.Tatti@xxxxxxxxxx>  wrote:
> > Due to the complexity with the PCI lock we cannot do the reset when a
> > device is bound ('echo $BDF > bind') or when unbound ('echo $BDF > unbind')
> > as the pci_[slot|bus]_reset also takes the same lock resulting in a
> > dead-lock.
> It took me a moment to figure that here you're referring to the
> process of (un)binding, not the state. To avoid that ambiguity in
> wording, how about "... we cannot do the reset while a device is
> being bound (...) or while it is being unbound ..."?
Sure, I will fix it.
> > --- a/Documentation/ABI/testing/sysfs-driver-pciback
> > +++ b/Documentation/ABI/testing/sysfs-driver-pciback
> > @@ -11,3 +11,18 @@ Description:
> >                   #echo 00:19.0-E0:2:FF > /sys/bus/pci/drivers/pciback/quirks
> >                   will allow the guest to read and write to the configuration
> >                   register 0x0E.
> > +
> > +What:           /sys/bus/pci/drivers/pciback/reset
> > +Date:           Dec 2017
> > +KernelVersion:  4.15
> > +Contact:xen-devel@xxxxxxxxxxxxxxxxxxxx  
> > +Description:
> > +                An option to perform a flr/slot/bus reset when a PCI device
> > +		is owned by Xen PCI backend. Writing a string of DDDD:BB:DD.F
> SSSS:BB:DD.F (or else the D-s are ambiguous, the more that "domain"
> in Xen code is ambiguous anyway - I continue to be mislead by struct
> pcistub_device_id's domain field)
Thanks for catching this issue. I will fix it.
> Also I assume the SSSS part is optional (default zero), which
> probably can and should be expressed in some way.
SSSS can be 0 or non-zero, subject to system configuration.
> > --- a/drivers/xen/xen-pciback/pci_stub.c
> > +++ b/drivers/xen/xen-pciback/pci_stub.c
> > @@ -313,6 +313,102 @@ void pcistub_put_pci_dev(struct pci_dev *dev)
> >   	up_write(&pcistub_sem);
> >   }
> >   
> > +struct pcistub_args {
> > +	const struct pci_dev *dev;
> > +	unsigned int dcount;
> The sole use of this field is for a debug message. Why not drop it
> and make "dev" the "data" argument without further indirection?
I prefer to keep this data structure since it will be helpful to debug 
any issues
orfor future enhancements.
> > +static int pcistub_device_search(struct pci_dev *dev, void *data)
> > +{
> > +	struct pcistub_device *psdev;
> > +	struct pcistub_args *arg = data;
> > +	bool found = false;
> > +	unsigned long flags;
> > +
> > +	spin_lock_irqsave(&pcistub_devices_lock, flags);
> > +
> > +	list_for_each_entry(psdev, &pcistub_devices, dev_list) {
> > +		if (psdev->dev == dev) {
> > +			found = true;
> > +			arg->dcount++;
> > +			break;
> Neither here nor in the caller I can see a check whether the device
> is currently assigned to a guest. Ownership by pciback alone imo is
> not sufficient to allow a reset to be performed.
I can add the following check

if ((psdev->dev == dev) && (pci_is_dev_assigned(dev)))
> > +static int pcistub_device_reset(struct pci_dev *dev)
> > +{
> > +	struct xen_pcibk_dev_data *dev_data;
> > +	bool slot = false, bus = false;
> > +	struct pcistub_args arg = {};
> > +
> > +	if (!dev)
> > +		return -EINVAL;
> > +
> > +	dev_dbg(&dev->dev, "[%s]\n", __func__);
> > +
> > +	/* First check and try FLR */
> > +	if (pcie_has_flr(dev)) {
> > +		dev_dbg(&dev->dev, "resetting %s device using FLR\n",
> > +			pci_name(dev));
> > +		pcie_flr(dev);
> The lack of error check here puzzled me, but I see the function
> indeed returns void right now. I think the prereq patch should
> change this along with exporting the function - you really don't
> want the device to be handed to a guest when the FLR timed
> out.
We will change pcie_flr() to return error code. I will make this change
in the next version of this patch.
> > +		return 0;
> > +	}
> > +
> > +	if (!pci_probe_reset_slot(dev->slot))
> > +		slot = true;
> > +	else if ((!pci_probe_reset_bus(dev->bus)) &&
> > +		 (!pci_is_root_bus(dev->bus)))
> Too many parentheses for my taste.
I will fix it.
> > +static ssize_t reset_store(struct device_driver *drv, const char *buf,
> > +			   size_t count)
> > +{
> > +	struct pcistub_device *psdev;
> > +	int domain, bus, slot, func;
> > +	int err;
> > +
> > +	err = str_to_slot(buf, &domain, &bus, &slot, &func);
> > +	if (err)
> > +		return err;
> > +
> > +	psdev = pcistub_device_find(domain, bus, slot, func);
> > +	if (psdev) {
> > +		err = pcistub_device_reset(psdev->dev);
> > +		pcistub_device_put(psdev);
> > +	} else {
> > +		err = -ENODEV;
> > +	}
> > +
> > +	if (!err)
> > +		err = count;
> > +
> > +	return err;
> > +}
> > +static DRIVER_ATTR_WO(reset);
> Would it be worth for reads of the file to return whether the device
> can be reset this way (i.e. the result of the checks you do before
> actually doing the reset)?
I don't think so. Plus, it makes this interface and its usage more 
complicated.

Cheers
GOVINDA