Re: [PATCH V3 0/4] Define coherent device memory node

[Michal Hocko]

On Fri 17-02-17 17:11:57, Anshuman Khandual wrote:
[...]
> * User space using mbind() to get CDM memory is an additional benefit
>   we get by making the CDM plug in as a node and be part of the buddy
>   allocator. But the over all idea from the user space point of view
>   is that the application can allocate any generic buffer and try to
>   use the buffer either from the CPU side or from the device without
>   knowing about where the buffer is really mapped physically. That
>   gives a seamless and transparent view to the user space where CPU
>   compute and possible device based compute can work together. This
>   is not possible through a driver allocated buffer.

But how are you going to define any policy around that. Who is allowed
to allocate and how much of this "special memory". Is it possible that
we will eventually need some access control mechanism? If yes then mbind
is really not suitable interface to (ab)use. Also what should happen if
the mbind mentions only CDM memory and that is depleted?

Could you also explain why the transparent view is really better than
using a device specific mmap (aka CDM awareness)?
 
> * The placement of the memory on the buffer can happen on system memory
>   when the CPU faults while accessing it. But a driver can manage the
>   migration between system RAM and CDM memory once the buffer is being
>   used from CPU and the device interchangeably. As you have mentioned
>   driver will have more information about where which part of the buffer
>   should be placed at any point of time and it can make it happen with
>   migration. So both allocation and placement are decided by the driver
>   during runtime. CDM provides the framework for this can kind device
>   assisted compute and driver managed memory placements.
> 
> * If any application is not using CDM memory for along time placed on
>   its buffer and another application is forced to fallback on system
>   RAM when it really wanted is CDM, the driver can detect these kind
>   of situations through memory access patterns on the device HW and
>   take necessary migration decisions.

Is this implemented or at least designed?

Btw. I believe that sending new versions of the patchset with minor
changes is not really helping the review process. I believe the
highlevel concerns about the API are not resolved yet and that is the
number 1 thing to deal with currently.
-- 
Michal Hocko
SUSE Labs