Re: [RFC PATCH] mm: support CONFIG_ZONE_DEVICE + CONFIG_ZONE_DMA

[Joonsoo Kim]

On Tue, Jan 26, 2016 at 05:37:38PM -0800, Dan Williams wrote:
> On Tue, Jan 26, 2016 at 5:18 PM, Joonsoo Kim <iamjoonsoo.kim@xxxxxxx> wrote:
> > Hello,
> >
> > On Tue, Jan 26, 2016 at 03:11:36PM -0800, Dan Williams wrote:
> >> On Tue, Jan 26, 2016 at 2:51 PM, Andrew Morton
> >> <akpm@xxxxxxxxxxxxxxxxxxxx> wrote:
> >> > On Tue, 26 Jan 2016 14:33:48 -0800 Dan Williams <dan.j.williams@xxxxxxxxx> wrote:
> >> >
> >> >> >> Towards this end, alias ZONE_DMA and ZONE_DEVICE to work around needing
> >> >> >> to maintain a unique zone number for ZONE_DEVICE.  Record the geometry
> >> >> >> of ZONE_DMA at init (->init_spanned_pages) and use that information in
> >> >> >> is_zone_device_page() to differentiate pages allocated via
> >> >> >> devm_memremap_pages() vs true ZONE_DMA pages.  Otherwise, use the
> >> >> >> simpler definition of is_zone_device_page() when ZONE_DMA is turned off.
> >> >> >>
> >> >> >> Note that this also teaches the memory hot remove path that the zone may
> >> >> >> not have sections for all pfn spans (->zone_dyn_start_pfn).
> >> >> >>
> >> >> >> A user visible implication of this change is potentially an unexpectedly
> >> >> >> high "spanned" value in /proc/zoneinfo for the DMA zone.
> >> >> >
> >> >> > Well, all these icky tricks are to avoid increasing ZONES_SHIFT, yes?
> >> >> > Is it possible to just use ZONES_SHIFT=3?
> >> >>
> >> >> Last I tried I hit this warning in mm/memory.c
> >> >>
> >> >> #warning Unfortunate NUMA and NUMA Balancing config, growing
> >> >> page-frame for last_cpupid.
> >> >
> >> > Well yes, it may take a bit of work - perhaps salvaging a bit from
> >> > somewhere else if poss.  But that might provide a better overall
> >> > solution so could you please have a think?
> >> >
> >>
> >> Will do, especially since other efforts are feeling the pinch on the
> >> MAX_NR_ZONES limitation.
> >
> > Please refer my previous attempt to add a new zone, ZONE_CMA.
> >
> > https://lkml.org/lkml/2015/2/12/84
> >
> > It salvages a bit from SECTION_WIDTH by increasing section size.
> > Similarly, I guess we can reduce NODE_WIDTH if needed although
> > it could cause to reduce maximum node size.
> 
> Dave pointed out to me that LAST__PID_SHIFT might be a better
> candidate to reduce to 7 bits.  That field is for storing pids which
> are already bigger than 8 bits.  If it is relying on the fact that
> pids don't rollover very often then likely the impact of 7-bits
> instead of 8 will be minimal.

Hmm... I'm not sure it's possible or not, but, it looks not a general
solution. It will solve your problem because you are using 64 bit arch
but other 32 bit archs can't get the benefit.

Thanks.