Re: Interacting with coherent memory on external devices

From: Jerome Glisse
Date: Mon Apr 27 2015 - 12:43:39 EST


On Mon, Apr 27, 2015 at 11:17:43AM -0500, Christoph Lameter wrote:
> On Mon, 27 Apr 2015, Jerome Glisse wrote:
>
> > > Improvements to the general code would be preferred instead of
> > > having specialized solutions for a particular hardware alone. If the
> > > general code can then handle the special coprocessor situation then we
> > > avoid a lot of code development.
> >
> > I think Paul only big change would be the memory ZONE changes. Having a
> > way to add the device memory as struct page while blocking the kernel
> > allocation from using this memory. Beside that i think the autonuma changes
> > he would need would really be specific to his usecase but would still
> > reuse all of the low level logic.
>
> Well lets avoid that. Access to device memory comparable to what the
> drivers do today by establishing page table mappings or a generalization
> of DAX approaches would be the most straightforward way of implementing it
> and would build based on existing functionality. Page migration currently
> does not work with driver mappings or DAX because there is no struct page
> that would allow the lockdown of the page. That may require either
> continued work on the DAX with page structs approach or new developments
> in the page migration logic comparable to the get_user_page() alternative
> of simply creating a scatter gather table to just submit a couple of
> memory ranges to the I/O subsystem thereby avoiding page structs.

What you refuse to see is that DAX is geared toward filesystem and as such
rely on special mapping. There is a reason why dax.c is in fs/ and not mm/
and i keep pointing out we do not want our mecanism to be perceive as fs
from userspace point of view. We want to be below the fs, at the mm level
where we could really do thing transparently no matter what kind of memory
we are talking about (anonymous, file mapped, share).

The fact is that DAX is about persistant storage but the people that
develop the persitant storage think it would be nice to expose it as some
kind of special memory. I am all for the direct mapping of this kind of
memory but still it is use as a backing store for a filesystem.

While in our case we are talking about "usual" _volatile_ memory that
should be use or expose as a filesystem.

I can't understand why you are so hellbent on the DAX paradigm, but it is
not what suit us in no way. We are not filesystem, we are regular memory,
our realm is mm/ not fs/

Cheers,
Jérôme
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/