Re: [External] Re: [PATCH v19 7/8] mm: hugetlb: add a kernel parameter hugetlb_free_vmemmap

[David Hildenbrand]

On 19.03.21 13:36, David Hildenbrand wrote:
> On 19.03.21 13:15, Muchun Song wrote:
> > On Fri, Mar 19, 2021 at 4:59 PM Oscar Salvador <osalvador@xxxxxxx> wrote:
> > > On Mon, Mar 15, 2021 at 05:20:14PM +0800, Muchun Song wrote:
> > > > --- a/arch/x86/mm/init_64.c
> > > > +++ b/arch/x86/mm/init_64.c
> > > > @@ -34,6 +34,7 @@
> > > >    #include <linux/gfp.h>
> > > >    #include <linux/kcore.h>
> > > >    #include <linux/bootmem_info.h>
> > > > +#include <linux/hugetlb.h>
> > > >
> > > >    #include <asm/processor.h>
> > > >    #include <asm/bios_ebda.h>
> > > > @@ -1557,7 +1558,8 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node,
> > > >    {
> > > >         int err;
> > > >
> > > > -     if (end - start < PAGES_PER_SECTION * sizeof(struct page))
> > > > +     if ((is_hugetlb_free_vmemmap_enabled()  && !altmap) ||
> > > > +         end - start < PAGES_PER_SECTION * sizeof(struct page))
> > > >                 err = vmemmap_populate_basepages(start, end, node, NULL);
> > > >         else if (boot_cpu_has(X86_FEATURE_PSE))
> > > >                 err = vmemmap_populate_hugepages(start, end, node, altmap);
> > >
> > > I've been thinking about this some more.
> > >
> > > Assume you opt-in the hugetlb-vmemmap feature, and assume you pass a valid altmap
> > > to vmemmap_populate.
> > > This will lead to use populating the vmemmap array with hugepages.
> >
> > Right.
> >
> > > What if then, a HugeTLB gets allocated and falls within that memory range (backed
> > > by hugetpages)?
> >
> > I am not sure whether we can allocate the HugeTLB pages from there.
> > Will only device memory pass a valid altmap parameter to
> > vmemmap_populate()? If yes, can we allocate HugeTLB pages from
> > device memory? Sorry, I am not an expert on this.
>
> I think, right now, yes. System RAM that's applicable for HugePages
> never uses an altmap. But Oscar's patch will change that, maybe before
> your series might get included from what I've been reading. [1]
>
> [1] https://lkml.kernel.org/r/20210319092635.6214-1-osalvador@xxxxxxx
>
> > > AFAIK, this will get us in trouble as currently the code can only operate on memory
> > > backed by PAGE_SIZE pages, right?
> > >
> > > I cannot remember, but I do not think nothing prevents that from happening?
> > > Am I missing anything?
> >
> > Maybe David H is more familiar with this.
> >
> > Hi David,
> >
> > Do you have some suggestions on this?
>
> There has to be some way to identify whether we can optimize specific
> vmemmap pages or should just leave them alone. altmap vs. !altmap.
>
> Unfortunately, there is no easy way to detect that - e.g.,
> PageReserved() applies also to boot memory.
>
> We could go back to setting a special PageType for these vmemmap pages,
> indicating "this is a page allocated from an altmap, don't touch it".

With SPARSEMEM we can use

PageReserved(page) && early_section(): vmemmap from bootmem

PageReserved(page) && !early_section(): vmemmap from altmap

!PageReserved(page): vmemmap from buddy

But it's a bit shaky :)

--
Thanks,

David / dhildenb