Re: [RFC PATCH 3/3] mm/map_contig: Add mmap(MAP_CONTIG) support

[Vlastimil Babka]

On 10/17/2017 08:23 PM, Mike Kravetz wrote:
> On 10/17/2017 07:20 AM, Guy Shattah wrote:
>> 1. CMA has to preconfigured. We're suggesting mechanism that works 'out of the box'
>> 2. Due to the pre-allocation techniques CMA imposes limitation on maximum 
>>    allocated memory. RDMA users often require 1Gb or more, sometimes more.
>> 3. CMA reserves memory in advance, our suggestion is using existing kernel memory
>>      mechanisms (THP for example) to allocate memory. 
> 
> I would not totally rule out the use of CMA.  I like the way that it reserves
> memory, but does not prohibit use by others.  In addition, there can be
> device (or purpose) specific reservations.

I think the use case are devices that *cannot* function without
contiguous memory, typical examples IIRC are smartphone cameras on with
Android where only single app is working with the device at given time,
so it's ok to reserve single area for the device, and allocation is done
by the driver. Here we are talking about allocations done by potentially
multiple userspace applications, so how do we reconcile that with the
reservations? How does a single flag identify which device's area to
use? How do we prevent one process depleting the area for other
processes? IMHO it's another indication that a generic interface is
infeasible and it should be driver-specific.

BTW, does RDMA need a specific NUMA node to work optimally? (one closest
to the device I presume?) Will it be the job of userspace to discover
and bind itself to that node, in addition to using MAP_CONTIG? Or would
that be another thing best handled by the driver?

> However, since reservations need to happen quite early it is often done on
> the kernel command line.  IMO, this should be avoided if possible.  There
> are interfaces for arch specific code to make reservations.  I do not know
> the system initialization sequence well enough to know if it would be
> possible for driver code to make CMA reservations.  But, it looks doubtful.
>