RE: [PATCH v3 6/6] hisi_acc_vfio_pci: Add support for VFIO live migration

[Shameerali Kolothum Thodi]

> -----Original Message-----
> From: Jason Gunthorpe [mailto:jgg@xxxxxxxxxx]
> Sent: 27 September 2021 16:01
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@xxxxxxxxxx>;
> Leon Romanovsky <leonro@xxxxxxxxxx>
> Cc: kvm@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx;
> linux-crypto@xxxxxxxxxxxxxxx; alex.williamson@xxxxxxxxxx;
> mgurtovoy@xxxxxxxxxx; liulongfang <liulongfang@xxxxxxxxxx>; Zengtao (B)
> <prime.zeng@xxxxxxxxxxxxx>; Jonathan Cameron
> <jonathan.cameron@xxxxxxxxxx>; Wangzhou (B) <wangzhou1@xxxxxxxxxxxxx>
> Subject: Re: [PATCH v3 6/6] hisi_acc_vfio_pci: Add support for VFIO live
> migration
> 
> On Mon, Sep 27, 2021 at 01:46:31PM +0000, Shameerali Kolothum Thodi
> wrote:
> 
> > > > > Nope, this is locked wrong and has no lifetime management.
> > > >
> > > > Ok. Holding the device_lock() sufficient here?
> > >
> > > You can't hold a hisi_qm pointer with some kind of lifecycle
> > > management of that pointer. device_lock/etc is necessary to call
> > > pci_get_drvdata()
> >
> > Since this migration driver only supports VF devices and the PF
> > driver will not be removed until all the VF devices gets removed,
> > is the locking necessary here?
> 
> Oh.. That is really busted up. pci_sriov_disable() is called under the
> device_lock(pf) and obtains the device_lock(vf).
> 
> This means a VF driver can never use the device_lock(pf), otherwise it
> can deadlock itself if PF removal triggers VF removal.

Exactly. I can easily simulate that in this driver.

> 
> But you can't access these members without using the device_lock(), as
> there really are no safety guarentees..

Hmm.. I was hoping that we can avoid holding the lock since
we are sure of the PF driver behavior. But right, there are no
guarantee here.

> The mlx5 patches have this same sketchy problem.
> 
> We may need a new special function 'pci_get_sriov_pf_devdata()' that
> confirms the vf/pf relationship and explicitly interlocks with the
> pci_sriov_enable/disable instead of using device_lock()
> 
> Leon, what do you think?
> 

Thanks,
Shameer