Re: [PATCH 3/4] mm: Defer ZONE_DEVICE page initialization to the point where we init pgmap

[Alexander Duyck]

On Tue, Sep 11, 2018 at 3:35 PM Dan Williams <dan.j.williams@xxxxxxxxx> wrote:
>
> On Mon, Sep 10, 2018 at 4:43 PM, Alexander Duyck
> <alexander.duyck@xxxxxxxxx> wrote:
> >
> > From: Alexander Duyck <alexander.h.duyck@xxxxxxxxx>
> >
> > The ZONE_DEVICE pages were being initialized in two locations. One was with
> > the memory_hotplug lock held and another was outside of that lock. The
> > problem with this is that it was nearly doubling the memory initialization
> > time. Instead of doing this twice, once while holding a global lock and
> > once without, I am opting to defer the initialization to the one outside of
> > the lock. This allows us to avoid serializing the overhead for memory init
> > and we can instead focus on per-node init times.
> >
> > One issue I encountered is that devm_memremap_pages and
> > hmm_devmmem_pages_create were initializing only the pgmap field the same
> > way. One wasn't initializing hmm_data, and the other was initializing it to
> > a poison value. Since this is something that is exposed to the driver in
> > the case of hmm I am opting for a third option and just initializing
> > hmm_data to 0 since this is going to be exposed to unknown third party
> > drivers.
> >
> > Signed-off-by: Alexander Duyck <alexander.h.duyck@xxxxxxxxx>
> > ---
> >  include/linux/mm.h |    2 +
> >  kernel/memremap.c  |   24 +++++---------
> >  mm/hmm.c           |   12 ++++---
> >  mm/page_alloc.c    |   89 +++++++++++++++++++++++++++++++++++++++++++++++++++-
>
> Hmm, why mm/page_alloc.c and not kernel/memremap.c for this new
> helper? I think that would address the kbuild reports and keeps all
> the devm_memremap_pages / ZONE_DEVICE special casing centralized. I
> also think it makes sense to move memremap.c to mm/ rather than
> kernel/ especially since commit 5981690ddb8f "memremap: split
> devm_memremap_pages() and memremap() infrastructure". Arguably, that
> commit should have went ahead with the directory move.

The issue ends up being the fact that I would then have to start
exporting infrastructure such as __init_single_page from page_alloc. I
have some follow-up patches I am working on that will generate some
other shared functions that can be used by both memmap_init_zone and
memmap_init_zone_device, as well as pulling in some of the code from
the deferred memory init.