Re: [RFC 5/8] mm: Add new flag VM_CDM for coherent device memory

[Aneesh Kumar K.V]

Dave Hansen <dave.hansen@xxxxxxxxx> writes:

> On 10/23/2016 09:31 PM, Anshuman Khandual wrote:
>> VMAs containing coherent device memory should be marked with VM_CDM. These
>> VMAs need to be identified in various core kernel paths and this new flag
>> will help in this regard.
>
> ... and it's sticky?  So if a VMA *ever* has one of these funky pages in
> it, it's stuck being VM_CDM forever?  Never to be merged with other
> VMAs?  Never to see the light of autonuma ever again?
>
> What if a 100TB VMA has one page of fancy pants device memory, and the
> rest normal vanilla memory?  Do we really want to consider the whole
> thing fancy?

This definitely needs fine tuning. I guess we should look at this as
possibly stating that, coherent device would like to not participate in
auto numa balancing, because it is difficult to update the core kernel
about access patters within the coherent device. This can result in core
kernel always trying to migrate pages from coherent device to system ram
even though we have large number of access within coherent device.


One possible option is to use a software pte bit (may be steal
_PAGE_DEVMAP) and prevent a numa pte setup from change_prot_numa().
ie, if the pfn backing the pte is from coherent device we don't allow
that to be converted to a prot none pte for numa faults ?


-aneesh