RE: [RFC PATCH v2 07/23] sched/cache: Prioritize preferred NUMA node selection over LLC selection

[Jianyong Wu]

Hi Peter,

> -----Original Message-----
> From: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
> Sent: Monday, August 31, 2026 9:22 PM
> To: Jianyong Wu <wujianyong@xxxxxxxx>
> Cc: Ingo Molnar <mingo@xxxxxxxxxx>; Juri Lelli <juri.lelli@xxxxxxxxxx>;
> Vincent Guittot <vincent.guittot@xxxxxxxxxx>; Chen Yu
> <yu.c.chen@xxxxxxxxx>; Tim Chen <tim.c.chen@xxxxxxxxxxxxxxx>; Dietmar
> Eggemann <dietmar.eggemann@xxxxxxx>; Steven Rostedt
> <rostedt@xxxxxxxxxxx>; Ben Segall <bsegall@xxxxxxxxxx>; Mel Gorman
> <mgorman@xxxxxxx>; Valentin Schneider <vschneid@xxxxxxxxxx>; K
> Prateek Nayak <kprateek.nayak@xxxxxxx>; Shrikanth Hegde
> <sshegde@xxxxxxxxxxxxx>; Phil Auld <pauld@xxxxxxxxxx>; Andrew
> Morton <akpm@xxxxxxxxxxxxxxxxxxxx>; David Hildenbrand
> <david@xxxxxxxxxx>; linux-kernel@xxxxxxxxxxxxxxx; linux-mm@xxxxxxxxx;
> jianyong.wu@xxxxxxxxxxx; Yuan Zhong <zhongyuan@xxxxxxxx>; Huangsj
> <huangsj@xxxxxxxx>; Fengyu Wang <wangfengyu@xxxxxxxx>; Zhiwei Ying
> <yingzhiwei@xxxxxxxx>; justin.he@xxxxxxx
> Subject: Re: [RFC PATCH v2 07/23] sched/cache: Prioritize preferred NUMA
> node selection over LLC selection
> 
> On Thu, Aug 27, 2026 at 08:28:00PM +0800, Jianyong Wu wrote:
> > In the current implementation, the preferred LLC is selected based on the
> > LLC with the largest running time of the thread group. However, the
> > preferred LLC may be prone to frequent migration when the workload
> > spreads across the entire system, especially when the number of CPUs
> > sharing an LLC is small. A better approach is to first select a preferred
> > NUMA node in the same way, and then select the preferred LLC within
> that
> > preferred NUMA node.
> 
> Hmm, if we're going to look at nodes, should we not also consider the
> numa balancing preferred node, and perhaps priorize an llc inside that
> nore, rather than the occupancy wise busiest node?
> 

The NUMA balancing preferred node is already covered by the scan:
get_scan_cpumasks() folds p->numa_preferred_nid (plus the node of the
current preferred LLC and the current running node) into the scan mask,
so it is not ignored - it is one of the candidates. And it filters out other no
related nodes.

The reason I don't hard-prioritize it is that numa_preferred_nid is
per-task while the preferred LLC is per-process. Different threads of the
same process can hold different preferred nodes, so prioritizing any one
of them would make the process-wide preferred LLC bounce between nodes.
So instead I include the NUMA balancing preferred node as a candidate and
let occupancy decide among them.

> 
> > -				a_occ += occ;
> > -				if (occ > m_occ) {
> > -					m_occ = occ;
> > -					m_cpu = i;
> > +					a_occ += occ;
> > +					if (occ > m_occ) {
> > +						m_occ = occ;
> > +						m_cpu = i;
> > +					}
> > +
> > +					cur = rcu_dereference_all(cpu_rq(i)->curr);
> > +					if (cur && !(cur->flags & (PF_EXITING |
> PF_KTHREAD)) &&
> > +					    cur->mm == mm)
> > +						nr_running++;
> >  				}
> 
> This level of indenting seems to suggest perhaps breaking things out
> into a helper function?

Agreed, the nesting got too deep. I'll factor the inner LLC occupancy
scan out into a helper function in the next revision.

Thanks
Jianyong