Re: [External] Re: [PATCH] arm64: mm: hugetlb: add support for free vmemmap pages of HugeTLB

[Muchun Song]

On Wed, May 19, 2021 at 7:44 PM Anshuman Khandual
<anshuman.khandual@xxxxxxx> wrote:
> On 5/18/21 2:48 PM, Muchun Song wrote:
> > The preparation of supporting freeing vmemmap associated with each
> > HugeTLB page is ready, so we can support this feature for arm64.
> >
> > Signed-off-by: Muchun Song <songmuchun@xxxxxxxxxxxxx>
> > ---
> >  arch/arm64/mm/mmu.c | 5 +++++
> >  fs/Kconfig          | 2 +-
> >  2 files changed, 6 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
> > index 5d37e461c41f..967b01ce468d 100644
> > --- a/arch/arm64/mm/mmu.c
> > +++ b/arch/arm64/mm/mmu.c
> > @@ -23,6 +23,7 @@
> >  #include <linux/mm.h>
> >  #include <linux/vmalloc.h>
> >  #include <linux/set_memory.h>
> > +#include <linux/hugetlb.h>
> >
> >  #include <asm/barrier.h>
> >  #include <asm/cputype.h>
> > @@ -1134,6 +1135,10 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node,
> >       pmd_t *pmdp;
> >
> >       WARN_ON((start < VMEMMAP_START) || (end > VMEMMAP_END));
> > +
> > +     if (is_hugetlb_free_vmemmap_enabled() && !altmap)
> > +             return vmemmap_populate_basepages(start, end, node, altmap);
>
> Not considering the fact that this will force the kernel to have only
> base page size mapping for vmemmap (unless altmap is also requested)
> which might reduce the performance, it also enables vmemmap mapping to
> be teared down or build up at runtime which could potentially collide
> with other kernel page table walkers like ptdump or memory hotremove
> operation ! How those possible collisions are protected right now ?

For the ptdump, there seems no problem.  The change of pte seems
not to affect the ptdump unless I miss something.

>
> Does not this vmemmap operation increase latency for HugeTLB usage ?
> Should not this runtime enablement also take into account some other
> qualifying information apart from potential memory save from struct
> page areas. Just wondering.

The disadvantage is we add a PTE level mapping for vmemmap
pages, from this point of view, the latency will be increased.

But There's an additional benefit which is that page (un)pinners will
see an improvement, because there are fewer vmemmap pages
and thus the tail/head pages are staying in cache more often.
>From this point of view, the latency will be decreased.

So if the user cares about the memory usage of the struct page, he
can enable this feature via cmdline when boot. As David said "That's
one of the reasons why it explicitly has to be enabled by an admin".

>
> > +
> >       do {
> >               next = pmd_addr_end(addr, end);
> >
> > diff --git a/fs/Kconfig b/fs/Kconfig
> > index 6ce6fdac00a3..02c2d3bf1cb8 100644
> > --- a/fs/Kconfig
> > +++ b/fs/Kconfig
> > @@ -242,7 +242,7 @@ config HUGETLB_PAGE
> >
> >  config HUGETLB_PAGE_FREE_VMEMMAP
> >       def_bool HUGETLB_PAGE
> > -     depends on X86_64
> > +     depends on X86_64 || ARM64
> >       depends on SPARSEMEM_VMEMMAP
> >
> >  config MEMFD_CREATE
> >