Re: [PATCH 1/5] mm/persistent-memory: match IORES_DESC name and enum memory_type one

[Jerome Glisse]

On Tue, Jul 11, 2017 at 12:31:22AM -0700, Dan Williams wrote:
> On Wed, Jul 5, 2017 at 11:49 AM, Jerome Glisse <jglisse@xxxxxxxxxx> wrote:
> > On Wed, Jul 05, 2017 at 09:15:35AM -0700, Dan Williams wrote:
> >> On Wed, Jul 5, 2017 at 7:25 AM, Jerome Glisse <jglisse@xxxxxxxxxx> wrote:
> >> > On Mon, Jul 03, 2017 at 04:49:18PM -0700, Dan Williams wrote:
> >> >> On Mon, Jul 3, 2017 at 2:14 PM, Jérôme Glisse <jglisse@xxxxxxxxxx> wrote:
> >> >> > Use consistent name between IORES_DESC and enum memory_type, rename
> >> >> > MEMORY_DEVICE_PUBLIC to MEMORY_DEVICE_PERSISTENT. This is to free up
> >> >> > the public name for CDM (cache coherent device memory) for which the
> >> >> > term public is a better match.
> >> >> >
> >> >> > Signed-off-by: Jérôme Glisse <jglisse@xxxxxxxxxx>
> >> >> > Cc: Dan Williams <dan.j.williams@xxxxxxxxx>
> >> >> > Cc: Ross Zwisler <ross.zwisler@xxxxxxxxxxxxxxx>
> >> >> > ---
> >> >> >  include/linux/memremap.h | 4 ++--
> >> >> >  kernel/memremap.c        | 2 +-
> >> >> >  2 files changed, 3 insertions(+), 3 deletions(-)
> >> >> >
> >> >> > diff --git a/include/linux/memremap.h b/include/linux/memremap.h
> >> >> > index 57546a07a558..2299cc2d387d 100644
> >> >> > --- a/include/linux/memremap.h
> >> >> > +++ b/include/linux/memremap.h
> >> >> > @@ -41,7 +41,7 @@ static inline struct vmem_altmap *to_vmem_altmap(unsigned long memmap_start)
> >> >> >   * Specialize ZONE_DEVICE memory into multiple types each having differents
> >> >> >   * usage.
> >> >> >   *
> >> >> > - * MEMORY_DEVICE_PUBLIC:
> >> >> > + * MEMORY_DEVICE_PERSISTENT:
> >> >> >   * Persistent device memory (pmem): struct page might be allocated in different
> >> >> >   * memory and architecture might want to perform special actions. It is similar
> >> >> >   * to regular memory, in that the CPU can access it transparently. However,
> >> >> > @@ -59,7 +59,7 @@ static inline struct vmem_altmap *to_vmem_altmap(unsigned long memmap_start)
> >> >> >   * include/linux/hmm.h and Documentation/vm/hmm.txt.
> >> >> >   */
> >> >> >  enum memory_type {
> >> >> > -       MEMORY_DEVICE_PUBLIC = 0,
> >> >> > +       MEMORY_DEVICE_PERSISTENT = 0,
> >> >> >         MEMORY_DEVICE_PRIVATE,
> >> >> >  };
> >> >> >
> >> >> > diff --git a/kernel/memremap.c b/kernel/memremap.c
> >> >> > index b9baa6c07918..e82456c39a6a 100644
> >> >> > --- a/kernel/memremap.c
> >> >> > +++ b/kernel/memremap.c
> >> >> > @@ -350,7 +350,7 @@ void *devm_memremap_pages(struct device *dev, struct resource *res,
> >> >> >         }
> >> >> >         pgmap->ref = ref;
> >> >> >         pgmap->res = &page_map->res;
> >> >> > -       pgmap->type = MEMORY_DEVICE_PUBLIC;
> >> >> > +       pgmap->type = MEMORY_DEVICE_PERSISTENT;
> >> >> >         pgmap->page_fault = NULL;
> >> >> >         pgmap->page_free = NULL;
> >> >> >         pgmap->data = NULL;
> >> >>
> >> >> I think we need a different name. There's nothing "persistent" about
> >> >> the devm_memremap_pages() path. Why can't they share name, is the only
> >> >> difference coherence? I'm thinking something like:
> >> >>
> >> >> MEMORY_DEVICE_PRIVATE
> >> >> MEMORY_DEVICE_COHERENT /* persistent memory and coherent devices */
> >> >> MEMORY_DEVICE_IO /* "public", but not coherent */
> >> >
> >> > No that would not work. Device public (in the context of this patchset)
> >> > is like device private ie device public page can be anywhere inside a
> >> > process address space either as anonymous memory page or as file back
> >> > page of regular filesystem (ie vma->ops is not pointing to anything
> >> > specific to the device memory).
> >> >
> >> > As such device public is different from how persistent memory is use
> >> > and those the cache coherency being the same between the two kind of
> >> > memory is not a discerning factor. So i need to distinguish between
> >> > persistent memory and device public memory.
> >> >
> >> > I believe keeping enum memory_type close to IORES_DESC naming is the
> >> > cleanest way to do that but i am open to other name suggestion.
> >> >
> >>
> >> The IORES_DESC has nothing to do with how the memory range is handled
> >> by the core mm. It sounds like the distinction this is trying to make
> >> is between MEMORY_DEVICE_{PUBLIC,PRIVATE} and MEMORY_DEVICE_HOST.
> >> Where a "host" memory range is one that does not need coordination
> >> with a specific device.
> >
> > I want to distinguish between:
> >   - device memory that is not accessible by the CPU
> >   - device memory that is accessible by the CPU just like regular
> >     memory
> >   - existing user of devm_memremap_pages() which is persistent memory
> >     (only pmem seems to call devm_memremap_pages()) that is use like a
> >     filesystem or block device and thus isn't use like generic page in
> >     a process address space
> >
> > So if existing user of devm_memremap_pages() are only persistent memory
> > then it made sense to match the IORES_DESC we are expecting to see on
> > see such memory.
> >
> > For public device memory (in the sense introduced by this patchset) i
> > do not know how it will be described by IORES_DESC. i think first folks
> > with it are IBM with CAPI and i am not sure they defined something for
> > that already.
> >
> > I am open to any name beside public (well any reasonable name :)) but
> > i do need to be able to distinguish persistent memory as use today from
> > this device memory.
> 
> Right, so that's why I suggested MEMORY_DEVICE_HOST for memory that is
> just normal host memory and does not have any device-entanglements
> outside of the base ZONE_DEVICE registration.

Well the memory considered for DEVICE_PUBLIC is device memory so it is
very much entangled with a device. It is memory that is physically on
the device. It is just that new system bus like CAPI or CCIX allows
CPU to access such memory with same cache coherency as if they were
accessing regular system DDR memory. It is expect that this memory
will be manage by the device driver and not core memory management.

But i am ok with MEMORY_DEVICE_HOST after all this just a name. But
what you put behind that name is not the reality of the memory. I just
want to be clear on that.

Cheers,
Jérôme