Re: [HMM 07/15] mm/ZONE_DEVICE: new type of ZONE_DEVICE for unaddressable memory v2
From: Dan Williams
Date: Mon May 22 2017 - 17:18:05 EST
On Mon, May 22, 2017 at 9:51 AM, JÃrÃme Glisse <jglisse@xxxxxxxxxx> wrote:
> HMM (heterogeneous memory management) need struct page to support migration
> from system main memory to device memory. Reasons for HMM and migration to
> device memory is explained with HMM core patch.
> This patch deals with device memory that is un-addressable memory (ie CPU
> can not access it). Hence we do not want those struct page to be manage
> like regular memory. That is why we extend ZONE_DEVICE to support different
> types of memory.
> A persistent memory type is define for existing user of ZONE_DEVICE and a
> new device un-addressable type is added for the un-addressable memory type.
> There is a clear separation between what is expected from each memory type
> and existing user of ZONE_DEVICE are un-affected by new requirement and new
> use of the un-addressable type. All specific code path are protect with
> test against the memory type.
> Because memory is un-addressable we use a new special swap type for when
> a page is migrated to device memory (this reduces the number of maximum
> swap file).
> The main two additions beside memory type to ZONE_DEVICE is two callbacks.
> First one, page_free() is call whenever page refcount reach 1 (which means
> the page is free as ZONE_DEVICE page never reach a refcount of 0). This
> allow device driver to manage its memory and associated struct page.
> The second callback page_fault() happens when there is a CPU access to
> an address that is back by a device page (which are un-addressable by the
> CPU). This callback is responsible to migrate the page back to system
> main memory. Device driver can not block migration back to system memory,
> HMM make sure that such page can not be pin into device memory.
> If device is in some error condition and can not migrate memory back then
> a CPU page fault to device memory should end with SIGBUS.
> Changed since v1:
> - rename to device private memory (from device unaddressable)
> Signed-off-by: JÃrÃme Glisse <jglisse@xxxxxxxxxx>
> Cc: Dan Williams <dan.j.williams@xxxxxxxxx>
> Cc: Ross Zwisler <ross.zwisler@xxxxxxxxxxxxxxx>
> diff --git a/mm/Kconfig b/mm/Kconfig
> index 229afe3..d49d816 100644
> --- a/mm/Kconfig
> +++ b/mm/Kconfig
> @@ -737,6 +737,19 @@ config ZONE_DEVICE
> If FS_DAX is enabled, then say Y.
> +config DEVICE_UNADDRESSABLE
> + bool "Unaddressable device memory (GPU memory, ...)"
> + depends on X86_64
> + depends on ZONE_DEVICE
> + depends on MEMORY_HOTPLUG
> + depends on MEMORY_HOTREMOVE
> + depends on SPARSEMEM_VMEMMAP
> + help
> + Allows creation of struct pages to represent unaddressable device
> + memory; i.e., memory that is only accessible from the device (or
> + group of devices).
Lets change config symbol naming from "device un-addressable memory"
to "device private memory" the same way we did for the code symbols.
With that change you can add:
Acked-by: Dan Williams <dan.j.williams@xxxxxxxxx>