Re: [PATCH v5 4/8] mm: add a template-based fast path for zone-device page init

[Alistair Popple]

On 2026-07-06 at 17:45 +1000, Li Zhe <lizhe.67@xxxxxxxxxxxxx> wrote...
> On Fri, 3 Jul 2026 17:06:33 +0300, rppt@xxxxxxxxxx wrote:
> 
> > On Wed, Jul 01, 2026 at 05:05:49PM +0800, Li Zhe wrote:
> > > memmap_init_zone_device() repeats nearly identical head-page
> > > initialization for each PFN. Prepare one reusable ZONE_DEVICE head-page
> > > template through the existing slow path, refresh the PFN-dependent
> > > fields in that template before each copy, and memcpy it into each
> > > destination page.
> > >
> > > This reduces the average rebind time from 244.28 ms to 217.19 ms, or
> > > about 11%.
> > >
> > > Signed-off-by: Li Zhe <lizhe.67@xxxxxxxxxxxxx>
> > > ---
> > >  mm/mm_init.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++--
> > >  1 file changed, 74 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/mm/mm_init.c b/mm/mm_init.c
> > > index 4c7fad440c2a..cc8417951467 100644
> > > --- a/mm/mm_init.c
> > > +++ b/mm/mm_init.c
> > > @@ -1066,6 +1066,50 @@ static void __ref zone_device_page_init_slow(struct page *page,
> > >  		set_page_count(page, 0);
> > >  }
> > >
> > > +static inline bool zone_device_page_init_optimization_enabled(void)
> > > +{
> > > +	/*
> > > +	 * The template fast path copies a preinitialized struct page image.
> > > +	 * Skip it when the page_ref_set tracepoint is enabled.
> > > +	 */
> > > +	return !page_ref_tracepoint_active(page_ref_set);
> > > +}
> > > +
> > > +static inline void zone_device_template_page_init(struct page *template,
> > > +						  struct page *src)
> > > +{
> > > +	memcpy(template, src, sizeof(*template));
> > > +}
> > > +
> > > +/*
> > > + * 'template' is a reusable page prototype rather than a strictly immutable
> > > + * object. Most ZONE_DEVICE fields stay constant across the pages covered by
> > > + * the current template, but section bits and page->virtual may still depend
> > > + * on the PFN. Refresh those PFN-dependent fields in the template before
> > > + * copying it into @page.
> > > + */
> > > +static inline void zone_device_page_update_template(struct page *template,
> > > +		unsigned long pfn)
> > > +{
> > > +	set_page_section_from_pfn(template, pfn);
> > > +#ifdef WANT_PAGE_VIRTUAL
> > > +	if (!is_highmem_idx(ZONE_DEVICE))
> > > +		set_page_address(template, __va(pfn << PAGE_SHIFT));
> > > +#endif
> > > +}
> > > +
> > > +static void zone_device_page_init_from_template(struct page *page,
> > > +		unsigned long pfn, struct page *template)
> > > +{
> > > +	/*
> > > +	 * 'template' carries the invariant portion of a ZONE_DEVICE struct
> > > +	 * page. Update the PFN-dependent fields in place before copying it
> > > +	 * to the destination page.
> > > +	 */
> > > +	zone_device_page_update_template(template, pfn);
> > > +	memcpy(page, template, sizeof(*page));
> > > +}
> > > +
> > 
> > The whole bunch of template functions look like it could be useful for
> > initialization of the non-zone-device struct pages as well.
> > 
> > As I mentioned previously, it's interesting to see if this approach speeds
> > up normal memory map initialization as well. If if does could have a single
> > set of the template functions.
> 
> Thanks for the suggestion. I agree this is worth exploring.
> 
> For this series, I would prefer to keep the scope limited to the
> ZONE_DEVICE initialization path and get that part settled first.

I think that makes sense, especially as my initial perf testing with other types
of ZONE_DEVICE pages without altmap didn't show much of a perf impact. Meaning
it might not be so useful for normal page init.

> After this series is settled, I plan to look into whether a similar
> template-based approach can also help normal page initialization. If it
> shows a real benefit there as well, I can follow up with a separate
> series.

I agree it would be interesting though, please let us know what you find one way
or the other.

 - Alistair

> Thanks,
> Zhe