Re: [RFC 0/8] Define coherent device memory node

[Jerome Glisse]

On Fri, Oct 28, 2016 at 10:59:52AM +0530, Aneesh Kumar K.V wrote:
> Jerome Glisse <j.glisse@xxxxxxxxx> writes:
> 
> > On Wed, Oct 26, 2016 at 04:39:19PM +0530, Aneesh Kumar K.V wrote:
> >> Jerome Glisse <j.glisse@xxxxxxxxx> writes:
> >> 
> >> > On Tue, Oct 25, 2016 at 09:56:35AM +0530, Aneesh Kumar K.V wrote:
> >> >> Jerome Glisse <j.glisse@xxxxxxxxx> writes:
> >> >> 
> >> >> > On Mon, Oct 24, 2016 at 10:01:49AM +0530, Anshuman Khandual wrote:
> >> >> >
> >> >> I looked at the hmm-v13 w.r.t migration and I guess some form of device
> >> >> callback/acceleration during migration is something we should definitely
> >> >> have. I still haven't figured out how non addressable and coherent device
> >> >> memory can fit together there. I was waiting for the page cache
> >> >> migration support to be pushed to the repository before I start looking
> >> >> at this closely.
> >> >> 
> >> >
> >> > The page cache migration does not touch the migrate code path. My issue with
> >> > page cache is writeback. The only difference with existing migrate code is
> >> > refcount check for ZONE_DEVICE page. Everything else is the same.
> >> 
> >> What about the radix tree ? does file system migrate_page callback handle
> >> replacing normal page with ZONE_DEVICE page/exceptional entries ?
> >> 
> >
> > It use the exact same existing code (from mm/migrate.c) so yes the radix tree
> > is updated and buffer_head are migrated.
> >
> 
> I looked at the the page cache migration patches shared and I find that
> you are not using exceptional entries when we migrate a page cache page to
> device memory. But I am now not sure how a read from page cache will
> work with that.
> 
> ie, a file system read will now find the page in page cache. But we
> cannot do a copy_to_user of that page because that is now backed by an
> unaddressable memory right ?
> 
> do_generic_file_read() does
>       page = find_get_page(mapping, index);
>       ....
>       ret = copy_page_to_iter(page, offset, nr, iter);
> 
> which does
> 	void *kaddr = kmap_atomic(page);
> 	size_t wanted = copy_to_iter(kaddr + offset, bytes, i);
> 	kunmap_atomic(kaddr);

Like i said right now for un-addressable memory my patches are mostly broken.
For read and write. I am focusing on page write back for now as it seemed to
be the more problematic case. For read/write the intention is to trigger a
migration back to system memory inside read/write of filesystem. This is also
why i will need a flag to indicate if a filesystem support migration to
un-addressable memory.

But in your case where the device memory is accessible then it should just work,
or do you need to do special thing when kmaping  device page ?

Cheers,
Jérôme