Re: [PATCH v2 2/5] mm/migrate: add a direction parameter to migrate_vma

[Ralph Campbell]

On 7/20/20 12:59 PM, Jason Gunthorpe wrote:
> On Mon, Jul 20, 2020 at 12:54:53PM -0700, Ralph Campbell wrote:
> > > > diff --git a/include/linux/migrate.h b/include/linux/migrate.h
> > > > index 3e546cbf03dd..620f2235d7d4 100644
> > > > +++ b/include/linux/migrate.h
> > > > @@ -180,6 +180,11 @@ static inline unsigned long migrate_pfn(unsigned long pfn)
> > > >    	return (pfn << MIGRATE_PFN_SHIFT) | MIGRATE_PFN_VALID;
> > > >    }
> > > > +enum migrate_vma_direction {
> > > > +	MIGRATE_VMA_FROM_SYSTEM,
> > > > +	MIGRATE_VMA_FROM_DEVICE_PRIVATE,
> > > > +};
> > >
> > > I would have guessed this is more natural as _FROM_DEVICE_ and
> > > TO_DEVICE_ ?
> >
> > The caller controls where the destination memory is allocated so it isn't
> > necessarily device private memory, it could be from system to system.
> > The use case for system to system memory migration is for hardware
> > like ARM SMMU or PCIe ATS where a single set of page tables is shared by
> > the device and a CPU process over a coherent system memory bus.
> > Also many integrated GPUs in SOCs fall into this category too.
>
> Maybe just TO/FROM_DEIVCE then? Even though the memory is not
> DEVICE_PRIVATE it is still device owned pages right?
>
> > So to me, it makes more sense to specify the direction based on the
> > source location.
>
> It feels strange because the driver doesn't always know or control the
> source?
>
> Jason

The driver can't really know where the source is currently located because the
API is designed to not initially hold the page locks, migrate_vma_setup() only knows
the source once it holds the page table locks and isolates/locks the pages being
migrated. The direction and pgmap_owner are supposed to filter which pages
the caller is interested in migrating.
Perhaps the direction should instead be a flags field with separate bits for
system memory and device private memory selecting source candidates for
migration. I can imagine use cases for all 4 combinations of
d->d, d->s, s->d, and s->s being valid.

I didn't really think a direction was needed, this was something that
Christoph Hellwig seemed to think made the API safer.