Re: [HMM v13 00/18] HMM (Heterogeneous Memory Management) v13

[Aneesh Kumar K.V]

John Hubbard <jhubbard@xxxxxxxxxx> writes:

> On Fri, 18 Nov 2016, JÃrÃme Glisse wrote:
>
>> Cliff note: HMM offers 2 things (each standing on its own). First
>> it allows to use device memory transparently inside any process
>> without any modifications to process program code. Second it allows
>> to mirror process address space on a device.
>> 
>> Change since v12 is the use of struct page for device memory even if
>> the device memory is not accessible by the CPU (because of limitation
>> impose by the bus between the CPU and the device).
>> 
>> Using struct page means that their are minimal changes to core mm
>> code. HMM build on top of ZONE_DEVICE to provide struct page, it
>> adds new features to ZONE_DEVICE. The first 7 patches implement
>> those changes.
>> 
>> Rest of patchset is divided into 3 features that can each be use
>> independently from one another. First is the process address space
>> mirroring (patch 9 to 13), this allow to snapshot CPU page table
>> and to keep the device page table synchronize with the CPU one.
>> 
>> Second is a new memory migration helper which allow migration of
>> a range of virtual address of a process. This memory migration
>> also allow device to use their own DMA engine to perform the copy
>> between the source memory and destination memory. This can be
>> usefull even outside HMM context in many usecase.
>> 
>> Third part of the patchset (patch 17-18) is a set of helper to
>> register a ZONE_DEVICE node and manage it. It is meant as a
>> convenient helper so that device drivers do not each have to
>> reimplement over and over the same boiler plate code.
>> 
>> 
>> I am hoping that this can now be consider for inclusion upstream.
>> Bottom line is that without HMM we can not support some of the new
>> hardware features on x86 PCIE. I do believe we need some solution
>> to support those features or we won't be able to use such hardware
>> in standard like C++17, OpenCL 3.0 and others.
>> 
>> I have been working with NVidia to bring up this feature on their
>> Pascal GPU. There are real hardware that you can buy today that
>> could benefit from HMM. We also intend to leverage this inside the
>> open source nouveau driver.
>> 
>
> Hi,
>
> We (NVIDIA engineering) have been working closely with Jerome on this for 
> several years now, and I wanted to mention that NVIDIA is committed to 
> using HMM. We've done initial testing of this patchset on Pascal GPUs (a 
> bit more detail below) and it is looking good.
>   

This can also be used on IBM platforms like Minsky (
http://www.tomshardware.com/news/ibm-power8-nvidia-tesla-p100-minsky,32661.html
)

There is also discussion around using this for device accelerated page
migration. That can help with coherent device memory node work.
(https://lkml.kernel.org/r/1477283517-2504-1-git-send-email-khandual@xxxxxxxxxxxxxxxxxx)

-aneesh