Re: [PATCH v2 1/7] mm/memory_hotplug: introduce mhp_flag MHP_OFFLINE_INACCESSIBLE

[David Hildenbrand]

On 23.11.23 10:23, Sumanth Korikkar wrote:
> Introduce MHP_OFFLINE_INACCESSIBLE mhp_flag to mark the hotplugged
> memory block as inaccessible during the memory hotplug addition phase.
> With support for "memmap on memory", the altmap is prepared at this
> stage. Architectures like s390 anticipate that memmap should not be
> accessed until memory is physically accessible and is accessible only
> when it enters the memory hotplug onlining phase using the memory
> notifier.  Introduce the flag to inform the memory hotplug
> infrastructure that the memory remains inaccessible until the memory
> hotplug onlining phase begins.
>
> Implementation considerations:
> mhp inaccessible flag is initially set in altmap. This is useful in
> arch_add_memory(). When the memory block device is added, the mhp
> inaccessible information is passed to memory_block. The flag is used in
> subsequent patch to avoid accessing memmap during memory hotplug
> addition phase.
>
> Signed-off-by: Sumanth Korikkar <sumanthk@xxxxxxxxxxxxx>
> ---
>   drivers/base/memory.c          |  2 ++
>   include/linux/memory.h         |  1 +
>   include/linux/memory_hotplug.h | 10 ++++++++++
>   include/linux/memremap.h       |  1 +
>   mm/memory_hotplug.c            |  3 ++-
>   5 files changed, 16 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/base/memory.c b/drivers/base/memory.c
> index 8a13babd826c..51915d5c3f88 100644
> --- a/drivers/base/memory.c
> +++ b/drivers/base/memory.c
> @@ -774,6 +774,8 @@ static int add_memory_block(unsigned long block_id, unsigned long state,
>   	mem->state = state;
>   	mem->nid = NUMA_NO_NODE;
>   	mem->altmap = altmap;
> +	if (altmap)
> +		mem->inaccessible = altmap->inaccessible;
>   	INIT_LIST_HEAD(&mem->group_next);
>   
>   #ifndef CONFIG_NUMA
> diff --git a/include/linux/memory.h b/include/linux/memory.h
> index f53cfdaaaa41..655714d4e65a 100644
> --- a/include/linux/memory.h
> +++ b/include/linux/memory.h
> @@ -67,6 +67,7 @@ struct memory_group {
>   struct memory_block {
>   	unsigned long start_section_nr;
>   	unsigned long state;		/* serialized by the dev->lock */
> +	bool inaccessible;		/* during memory addition phase */

Is that really required? After all, the altmap is stored in the memory 
block and accessible there.

>   	int online_type;		/* for passing data to online routine */
>   	int nid;			/* NID for this memory block */
>   	/*
> diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h
> index 7d2076583494..8988cd5ad55d 100644
> --- a/include/linux/memory_hotplug.h
> +++ b/include/linux/memory_hotplug.h
> @@ -106,6 +106,16 @@ typedef int __bitwise mhp_t;
>    * implies the node id (nid).
>    */
>   #define MHP_NID_IS_MGID		((__force mhp_t)BIT(2))
> +/*
> + * Mark the hotplugged memory block as inaccessible during the memory hotplug
> + * addition phase. With support for "memmap on memory," the altmap is prepared
> + * at this stage. Architectures like s390 anticipate that memmap should not be
> + * accessed until memory is physically accessible and is accessible only when
> + * it enters the memory hotplug onlining phase using the memory notifier.
> + * Utilize this flag to inform the memory hotplug infrastructure that the
> + * memory remains inaccessible until the memory hotplug onlining phase begins.
> + */
> +#define MHP_OFFLINE_INACCESSIBLE	((__force mhp_t)BIT(3))

I'd suggest to squash all 3 patches. Then we can properly document here:

/*
 * The hotplugged memory is completely inaccessible while the memory is
 * offline. The memory provider will handle MEM_PREPARE_ONLINE /
 * MEM_FINISH_OFFLINE notifications and make the memory accessible.
 *
 * This flag is only relevant when used along with MHP_MEMMAP_ON_MEMORY,
 * because the altmap cannot be written (e.g., poisoned) when adding
 * memory -- before it is set online.
 *
 * This allows for adding memory with an altmap that is not currently
 * made available by a hypervisor. When onlining that memory, the
 * hypervisor can be instructed to make that memory available, and
 * the onlining phase will not require any memory allocations, which is
 * helpful in low-memory situations.
 */


Cheers,

David / dhildenb