RE: [RFC PATCH v2 08/23] sched/topology: Introduce a per-CPU tasks NUMA preferred counter

[Jianyong Wu]

> -----Original Message-----
> From: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
> Sent: Tuesday, September 1, 2026 6:21 PM
> To: Jianyong Wu <wujianyong@xxxxxxxx>
> Cc: Ingo Molnar <mingo@xxxxxxxxxx>; Juri Lelli <juri.lelli@xxxxxxxxxx>;
> Vincent Guittot <vincent.guittot@xxxxxxxxxx>; Chen Yu
> <yu.c.chen@xxxxxxxxx>; Tim Chen <tim.c.chen@xxxxxxxxxxxxxxx>; Dietmar
> Eggemann <dietmar.eggemann@xxxxxxx>; Steven Rostedt
> <rostedt@xxxxxxxxxxx>; Ben Segall <bsegall@xxxxxxxxxx>; Mel Gorman
> <mgorman@xxxxxxx>; Valentin Schneider <vschneid@xxxxxxxxxx>; K
> Prateek Nayak <kprateek.nayak@xxxxxxx>; Shrikanth Hegde
> <sshegde@xxxxxxxxxxxxx>; Phil Auld <pauld@xxxxxxxxxx>; Andrew
> Morton <akpm@xxxxxxxxxxxxxxxxxxxx>; David Hildenbrand
> <david@xxxxxxxxxx>; linux-kernel@xxxxxxxxxxxxxxx; linux-mm@xxxxxxxxx;
> jianyong.wu@xxxxxxxxxxx; Yuan Zhong <zhongyuan@xxxxxxxx>; Huangsj
> <huangsj@xxxxxxxx>; Fengyu Wang <wangfengyu@xxxxxxxx>; Zhiwei Ying
> <yingzhiwei@xxxxxxxx>; justin.he@xxxxxxx
> Subject: Re: [RFC PATCH v2 08/23] sched/topology: Introduce a per-CPU
> tasks NUMA preferred counter
> 
> On Tue, Sep 01, 2026 at 08:31:35AM +0000, Jianyong Wu wrote:
> 
> > > On Thu, Aug 27, 2026 at 08:28:01PM +0800, Jianyong Wu wrote:
> > > > Like the existed task LLC preferred counter, sd->llc_counts, introduce
> > > > sd->numa_counts to denotes the task number that prefer each
> NUMA node in
> > > > a certain rq.
> > > >
> > >
> > > To what purpose? Is not the node occupancy the direct sum of its
> > > constituent llc occupancy?
> >
> > numa_counts is used by the affinity score in patch 12. It holds, for each
> > NUMA node, the number of tasks on this rq whose preferred LLC falls in
> > that node.
> >
> > You're right that numerically it equals the per-node sum of llc_counts.
> > The reason to keep it is granularity: NUMA-level load balance scores at
> > node granularity (numa_counts[node] * distance_delta), while
> NODE-level
> > balancing scores at LLC granularity (llc_counts[llc] * ...). Maintaining
> > the node aggregate costs only a single llc_to_node() lookup (an O(1)
> > array read) plus one increment in enqueue/dequeue, whereas
> recomputing
> > the per-node sum would have to walk every LLC of a node each time the
> > NUMA-level score is evaluated.
> 
> Consider the trade-off. Adding the accounting adds a cache-miss to every
> enqueue/dequeue, while re-computing the value on-demand adds some
> little
> cost to the balancing (slow) path.
> 
> If the llc_counts[] array is dense in node order, then it is likely all
> relevant numbers are in a single line and computing the sum is in fact
> fairly cheap.

This is a good point. I was considering the number of operations, but not the fact that the additional accounting is performed on every enqueue/dequeue and may touch another cache line, while the summation is only needed on the load balancing path.

Since numa_counts[] contains no information that cannot be derived from llc_counts[], I will rework the NUMA-level score to sum the constituent LLC counters on demand and remove the additional numa_counts[] accounting.

Thanks
Jianyong