Re: [PATCH] DAX: warn when kmem regions are truncated for memory block alignment.

[David Hildenbrand]

On 31.03.25 20:27, Gregory Price wrote:
> On Fri, Mar 21, 2025 at 02:07:31PM -0400, Gregory Price wrote:
> > Device capacity intended for use as system ram should be aligned to the
> > architecture-defined memory block size or that capacity will be silently
> > truncated and capacity stranded.
> >
> > As hotplug dax memory becomes more prevelant, the memory block size
> > alignment becomes more important for platform and device vendors to
> > pay attention to - so this truncation should not be silent.
> >
> > This issue is particularly relevant for CXL Dynamic Capacity devices,
> > whose capacity may arrive in spec-aligned but block-misaligned chunks.
> >
> > Example:
> >   [...] kmem dax0.0: dax region truncated 2684354560 bytes - alignment
> >   [...] kmem dax1.0: dax region truncated 1610612736 bytes - alignment
> >
> > Signed-off-by: Gregory Price <gourry@xxxxxxxxxx>
>
> Gentle pokes.  There were a couple questions last week whether we should
> warn here or actually fix something in memory-hotplug.
>
> Notes from CXL Boot to Bash session discussions:
>
>
> We discussed [1] how this auto-sizing can cause 1GB huge page
> allocation failures (assuming you online as ZONE_NORMAL). That means
> ACPI-informed sizing by default would potentially be harmful to existing
> systems and adding yet-another-boot-option just seems nasty.
>
> I've since dropped acpi-informed block size patch[2].  If there are opinions
> otherwise, I can continue pushing it.

Oh, I thought we would be going forward with that. What's the reason we 
would not want to do that?

> We also discussed[3] variable-sized blocks having some nasty corner cases.
> Not unsolvable, but doesn't help users in the short term.
>
>
> There was some brief discussion about whether a hotplug memblock with a
> portion as offline pages would be possible.  This seems hacky?  There
> was another patch set discussing this, but I can't seem to find it.

Yeah, I proposed something like that as well when I started working on 
virtio-mem and did not really understand the whole hot(un)plug model in 
Linux properly. Someone else proposed it again a couple of years ago, 
but it's just wrong and should not be done that way.

One could implement something like virtio-mem, whereby parts of a Linux 
memory block can be added/removed independently ("fake offlined"). But 
the whole idea of virtio-mem is that all memory in the Linux memory 
block range belongs to it. So it doesn't quite apply to DAX where parts 
of a Linux memory block might be from something completely different 
(e.g., boot memory etc).

--
Cheers,

David / dhildenb