Re: [PATCH v1 3/3] memory-hotplug.rst: document the "auto-movable" online policy

[David Hildenbrand]

On 06.10.21 02:35, Mike Rapoport wrote:
> On Thu, Sep 30, 2021 at 04:41:17PM +0200, David Hildenbrand wrote:
> > In commit e83a437faa62 ("mm/memory_hotplug: introduce "auto-movable" online
> > policy") we introduced a new memory online policy to automatically
> > select a zone for memory blocks to be onlined. We added a way to
> > set the active online policy and tunables for the auto-movable online
> > policy. In follow-up commits we tweaked the "auto-movable" policy to also
> > consider memory device details when selecting zones for memory blocks to
> > be onlined.
> >
> > Let's document the new toggles and how the two online policies we have
> > work.
> >
> > Signed-off-by: David Hildenbrand <david@xxxxxxxxxx>
> > ---
> >   .../admin-guide/mm/memory-hotplug.rst         | 128 +++++++++++++++---
> >   1 file changed, 108 insertions(+), 20 deletions(-)
> >
> > diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst b/Documentation/admin-guide/mm/memory-hotplug.rst
> > index ee00b70dedde..c20a2c0031cf 100644
> > --- a/Documentation/admin-guide/mm/memory-hotplug.rst
> > +++ b/Documentation/admin-guide/mm/memory-hotplug.rst
> > @@ -165,9 +165,8 @@ Or alternatively::
> >   
> >   	% echo 1 > /sys/devices/system/memory/memoryXXX/online
> >   
> > -The kernel will select the target zone automatically, usually defaulting to
> > -``ZONE_NORMAL`` unless ``movable_node`` has been specified on the kernel
> > -command line or if the memory block would intersect the ZONE_MOVABLE already.
> > +The kernel will select the target zone automatically, depending on the
> > +configured ``online_policy``.
> >   
> >   One can explicitly request to associate an offline memory block with
> >   ZONE_MOVABLE by::
> > @@ -198,6 +197,9 @@ Auto-onlining can be enabled by writing ``online``, ``online_kernel`` or
> >   
> >   	% echo online > /sys/devices/system/memory/auto_online_blocks
> >   
> > +Similarly to manual onlining, with ``online`` the kernel will select the
> > +target zone automatically, depending on the configured ``online_policy``.
> > +
> >   Modifying the auto-online behavior will only affect all subsequently added
> >   memory blocks only.
> >   
> > @@ -393,9 +395,11 @@ command line parameters are relevant:
> >   ======================== =======================================================
> >   ``memhp_default_state``	 configure auto-onlining by essentially setting
> >                            ``/sys/devices/system/memory/auto_online_blocks``.
> > -``movable_node``	 configure automatic zone selection in the kernel. When
> > -			 set, the kernel will default to ZONE_MOVABLE, unless
> > -			 other zones can be kept contiguous.
> > +``movable_node``	 configure automatic zone selection in the kernel when
> > +			 using the ``contig-zones`` online policy. When
> > +			 set, the kernel will default to ZONE_MOVABLE when
> > +			 onlining a memory block, unless other zones can be kept
> > +			 contiguous.
>
> The movable_node main purpose is to allow unplugging an entire node. Zone
> selection is a consequence of this. You may want to cite the description of
> movable_node in kernel-paramenters.txt here.

Right, I only document the effects of these parameters on memory 
hot(un)plug.

What about:

diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst 
b/Documentation/admin-guide/mm/memory-hotplug.rst
index c20a2c0031cf..f8976ded0863 100644
--- a/Documentation/admin-guide/mm/memory-hotplug.rst
+++ b/Documentation/admin-guide/mm/memory-hotplug.rst
@@ -402,6 +402,9 @@ command line parameters are relevant:
                         contiguous.
 ======================== 
=======================================================

+See Documentation/admin-guide/kernel-parameters.txt for a more generic
+description of these command line parameters.
+
 Module Parameters
 ------------------


> And, pardon my ignorance, how movable_node will play with auto-movable
> policy?

It's essentially ignored with the auto-movable policy for memory 
hotplugged after boot (!MEMBLOCK_HOTPLUG). That's why only the 
description of "contig-zones" below describes how it interacts with the 
``movable_node``, and we make it clear here that it's restricted to the 
"contig-zones" policy as well.

<details>
Bare metal, where we care about reliably unplugging hotplugged memory 
usually configures auto-onlining to "online_movable": for example, 
that's the case on RHEL. auto-movable doesn't make too much sense for 
bare metal: the nature of "movable_node" is to essentially online 
anything that might get hotunplugged MOVABLE, especially after 
hotplugging memory and rebooting: that is highly dangerous especially in 
virtualized environments.

"auto-movable" is valuable in virtualized environments, where we add 
memory via:
* add_memory_driver_managed() like virtio-mem, whereby such memory is
  never part of the firmware provided memory-map, so the policy is
  always in control even after a reboot.
* Hotplugged virtual DIMMs, such as provided by x86-64/arm64, whereby we
  don't include these DIMMs in the firmware-provided memory map, but
  ACPI code adds them after early boot, making it behave similar to
  add_memory_driver_managed() -- the policy is always in control even
  after a reboot.
</details>


Thanks!

--
Thanks,

David / dhildenb