Re: [PATCH v1 2/8] vfio/type1: Add vfio_iommu_type1 parameter for quota tuning

[Alex Williamson]

On Mon, 30 Mar 2020 11:44:08 +0000
"Tian, Kevin" <kevin.tian@xxxxxxxxx> wrote:

> > From: Liu, Yi L <yi.l.liu@xxxxxxxxx>
> > Sent: Monday, March 30, 2020 5:27 PM
> >   
> > > From: Tian, Kevin <kevin.tian@xxxxxxxxx>
> > > Sent: Monday, March 30, 2020 5:20 PM
> > > To: Liu, Yi L <yi.l.liu@xxxxxxxxx>; alex.williamson@xxxxxxxxxx;
> > > Subject: RE: [PATCH v1 2/8] vfio/type1: Add vfio_iommu_type1 parameter  
> > for quota  
> > > tuning
> > >  
> > > > From: Liu, Yi L <yi.l.liu@xxxxxxxxx>
> > > > Sent: Monday, March 30, 2020 4:53 PM
> > > >  
> > > > > From: Tian, Kevin <kevin.tian@xxxxxxxxx>
> > > > > Sent: Monday, March 30, 2020 4:41 PM
> > > > > To: Liu, Yi L <yi.l.liu@xxxxxxxxx>; alex.williamson@xxxxxxxxxx;
> > > > > Subject: RE: [PATCH v1 2/8] vfio/type1: Add vfio_iommu_type1
> > > > > parameter  
> > > > for quota  
> > > > > tuning
> > > > >  
> > > > > > From: Liu, Yi L <yi.l.liu@xxxxxxxxx>
> > > > > > Sent: Sunday, March 22, 2020 8:32 PM
> > > > > >
> > > > > > From: Liu Yi L <yi.l.liu@xxxxxxxxx>
> > > > > >
> > > > > > This patch adds a module option to make the PASID quota tunable by
> > > > > > administrator.
> > > > > >
> > > > > > TODO: needs to think more on how to  make the tuning to be per-  
> > process.  
> > > > > >
> > > > > > Previous discussions:
> > > > > > https://patchwork.kernel.org/patch/11209429/
> > > > > >
> > > > > > Cc: Kevin Tian <kevin.tian@xxxxxxxxx>
> > > > > > CC: Jacob Pan <jacob.jun.pan@xxxxxxxxxxxxxxx>
> > > > > > Cc: Alex Williamson <alex.williamson@xxxxxxxxxx>
> > > > > > Cc: Eric Auger <eric.auger@xxxxxxxxxx>
> > > > > > Cc: Jean-Philippe Brucker <jean-philippe@xxxxxxxxxx>
> > > > > > Signed-off-by: Liu Yi L <yi.l.liu@xxxxxxxxx>
> > > > > > ---
> > > > > >  drivers/vfio/vfio.c             | 8 +++++++-
> > > > > >  drivers/vfio/vfio_iommu_type1.c | 7 ++++++-
> > > > > >  include/linux/vfio.h            | 3 ++-
> > > > > >  3 files changed, 15 insertions(+), 3 deletions(-)
> > > > > >
> > > > > > diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index
> > > > > > d13b483..020a792 100644
> > > > > > --- a/drivers/vfio/vfio.c
> > > > > > +++ b/drivers/vfio/vfio.c
> > > > > > @@ -2217,13 +2217,19 @@ struct vfio_mm  
> > > > *vfio_mm_get_from_task(struct  
> > > > > > task_struct *task)
> > > > > >  }
> > > > > >  EXPORT_SYMBOL_GPL(vfio_mm_get_from_task);
> > > > > >
> > > > > > -int vfio_mm_pasid_alloc(struct vfio_mm *vmm, int min, int max)
> > > > > > +int vfio_mm_pasid_alloc(struct vfio_mm *vmm, int quota, int min,
> > > > > > +int  
> > > > max)  
> > > > > >  {
> > > > > >  	ioasid_t pasid;
> > > > > >  	int ret = -ENOSPC;
> > > > > >
> > > > > >  	mutex_lock(&vmm->pasid_lock);
> > > > > >
> > > > > > +	/* update quota as it is tunable by admin */
> > > > > > +	if (vmm->pasid_quota != quota) {
> > > > > > +		vmm->pasid_quota = quota;
> > > > > > +		ioasid_adjust_set(vmm->ioasid_sid, quota);
> > > > > > +	}
> > > > > > +  
> > > > >
> > > > > It's a bit weird to have quota adjusted in the alloc path, since the
> > > > > latter  
> > > > might  
> > > > > be initiated by non-privileged users. Why not doing the simple math
> > > > > in  
> > > > vfio_  
> > > > > create_mm to set the quota when the ioasid set is created? even in
> > > > > the  
> > > > future  
> > > > > you may allow per-process quota setting, that should come from
> > > > > separate privileged path instead of thru alloc..  
> > > >
> > > > The reason is the kernel parameter modification has no event which can
> > > > be used to adjust the quota. So I chose to adjust it in pasid_alloc
> > > > path. If it's not good, how about adding one more IOCTL to let user-
> > > > space trigger a quota adjustment event? Then even non-privileged user
> > > > could trigger quota adjustment, the quota is actually controlled by
> > > > privileged user. How about your opinion?
> > > >  
> > >
> > > why do you need an event to adjust? As I said, you can set the quota when  
> > the set is  
> > > created in vfio_create_mm...  
> > 
> > oh, it's to support runtime adjustments. I guess it may be helpful to let
> > per-VM quota tunable even the VM is running. If just set the quota in
> > vfio_create_mm(), it is not able to adjust at runtime.
> >   
> 
> ok, I didn't note the module parameter was granted with a write permission.
> However there is a further problem. We cannot support PASID reclaim now.
> What about the admin sets a quota smaller than previous value while some
> IOASID sets already exceed the new quota? I'm not sure how to fail a runtime
> module parameter change due to that situation. possibly a normal sysfs 
> node better suites the runtime change requirement...

Yep, making this runtime adjustable seems a bit unpredictable and racy,
and it's not clear to me how a user is going to jump in at just the
right time for a user and adjust the limit.  I'd probably go for a
simple non-runtime adjustable module option.  It's a safety net at this
point anyway afaict.  Thanks,

Alex