Re: [v2 RFC PATCH 0/9] Another Approach to Use PMEM as NUMA Node

[Fengguang Wu]

On Wed, Apr 17, 2019 at 11:17:48AM +0200, Michal Hocko wrote:
> On Tue 16-04-19 12:19:21, Yang Shi wrote:
> > On 4/16/19 12:47 AM, Michal Hocko wrote:
> [...]
> > > Why cannot we simply demote in the proximity order? Why do you make
> > > cpuless nodes so special? If other close nodes are vacant then just use
> > > them.
> >
> > We could. But, this raises another question, would we prefer to just demote
> > to the next fallback node (just try once), if it is contended, then just
> > swap (i.e. DRAM0 -> PMEM0 -> Swap); or would we prefer to try all the nodes
> > in the fallback order to find the first less contended one (i.e. DRAM0 ->
> > PMEM0 -> DRAM1 -> PMEM1 -> Swap)?
>
> I would go with the later. Why, because it is more natural. Because that
> is the natural allocation path so I do not see why this shouldn't be the
> natural demotion path.

"Demotion" should be more performance wise by "demoting to the
next-level (cheaper/slower) memory". Otherwise something like this
may happen.

DRAM0 pressured => demote cold pages to DRAM1 
DRAM1 pressured => demote cold pages to DRAM0

Kind of DRAM0/DRAM1 exchanged a fraction of the demoted cold pages,
which looks not helpful for overall system performance.

Over time, it's even possible some cold pages get "demoted" in path
DRAM0=>DRAM1=>DRAM0=>DRAM1=>...

Thanks,
Fengguang