Re: [Patch v4 17/22] sched/cache: Avoid cache-aware scheduling for memory-heavy processes

[Jirka Hladky]

Hi Peter, Chen Yu, Tim,

We have a workload where cache-aware scheduling (CAS) causes a
large regression: NAS MG class D [3] - a memory-bandwidth-bound
multigrid, 112 OpenMP threads, 26.4 GiB working set - runs 65%
slower with the llc_balancing switch on (25.5s -> 42.2s median,
n=5), because CAS aggregates all 112 threads onto one LLC. On
this system each LLC is a whole socket, so the process is left
with one socket's memory bandwidth for a working set 250x the
LLC size.

The check that should keep such a process out of aggregation is
this patch, merged as 808915f982c2 via the May enhancements series,
with the RSS comparison replaced by a footprint estimate from NUMA
balancing fault statistics following the discussion here. On this
Sierra Forest system that estimate stays at 0 for MG even with
NUMA balancing active, and the NUMA balancing off case Chen Yu
left open here costs the same.

On this platform the footprint check is the only safeguard that can
catch such a process. Of the safety valves listed in the new
sched-cache.rst [1], the thread limit is ~115 threads (80% of 144
CPUs) and the busy threshold is 75% utilization (no SMT): the
112-thread run above sits just under the limit and passes both,
and so does the 64-thread one (44% utilization). And unlike
platforms with several LLCs per node, where aggregation trades a
node's bandwidth for one LLC's, here it simply halves the
bandwidth available to the process.

PLATFORM
  2x Intel Xeon 6780E (Sierra Forest), 144 cores each, SMT off,
  288 CPUs. One 108 MiB LLC per socket; each LLC coincides with one
  NUMA node. Kernel: v7.4-rc2-based, kernel.numa_balancing=1,
  aggr_tolerance=1. n=5 independent runs per configuration, CAS
  master switch off (baseline) vs on.

RESULTS (runtime, lower is better)
  workload                          CAS OFF      CAS ON      delta
  NAS MG class D, 1 proc x 112thr  25.53s med   42.23s med  +65%
  NAS MG class D, 1 proc x 64thr   26.45s med   41.78s med  +58%
  NAS MG class D, 1 proc x 32thr   35.06s med   51.04s med  +46%
  NAS MG class C, 1 proc x 4thr     9.34s med   11.99s med  +28%

THE FOOTPRINT ESTIMATE NEVER LEAVES ZERO (MG class D, 64 threads)
  Traced live on the stock kernel with bpftrace (comm-filtered),
  across three CAS-on runs:

    - grp->footprint: 0 in 44,670 and 40,638 task_cache_work()
      samples over two full runs (min = max = 0), while ~7M pages
      (26.4 GiB) were resident. NUMA balancing was active: 8k-20k
      hint faults and up to 3.3M pages migrated per run.
    - In those two runs the footprint write never executed: all
      ~8k-13k task_numa_placement() calls per run (one per hint
      fault, via task_numa_fault()) returned at the scan_seq gate.

  WHY THE SIGNAL NEVER ACCUMULATES
  1. grp->footprint is only written in task_numa_placement(),
     which runs only when mm->numa_scan_seq changes. That counter
     increments only in reset_ptenuma_scan(): when the NUMA scanner
     has wrapped the whole address space.
  2. The scanner walks the 26.4 GiB of mappings at ~0.3-1 GiB/s
     (traced via mm->numa_scan_offset and change_prot_numa()
     return values), so one sweep takes 30-40s or more. The 43s
     benchmark completed 0 sweeps in two runs, 1 sweep (at t=33s)
     in a third. The accounting runs at most once per task per
     run.
  3. The signal is also small: a run accumulates only 8k-20k
     hint-fault events, 0.1-0.3% of the working set, below the
     108 MiB threshold even when a sweep does complete.
  4. And it is a rate signal, not a size: the increment is
     diff = numa_faults[MEMBUF] - numa_faults[MEM] / 2, which is
     zero at any steady fault rate.

  Cadence, coverage and formula each independently hold the
  footprint orders of magnitude below the resident set.

  exceed_llc_capacity() is evaluated on every tick and every scan
  epoch, but with footprint at 0 the comparison

      llc_bytes * scale  <  footprint * PAGE_SIZE

  is never true, so grp->cpu is never reset. After a 1.8s warm-up it
  pointed at the same LLC for all 44,661 remaining samples.

  Without NUMA balancing the check never triggers, as sched-cache.rst
  documents. With CAS on and numa_balancing=0 the same run takes
  42.2s, the same as with NUMA balancing on (42.0s).

WHAT THE AGGREGATION COSTS (MG class D, 64 threads)
  - With CAS on, 97% of thread-samples are on one node. The 26.4 GiB
    working set starts ~50/50 across the nodes and NUMA balancing
    then pulls it to the busy node (numa_maps 3.3M/2.7M pages ->
    6.7M/0.2M; 1.42M pages migrated during the run).
  - The cost is not on the CPU side: average CPU utilization is
    essentially unchanged (6164% vs 6029% - about 61 of the 64
    threads on-CPU in both configurations), and wakeup latency is
    49us on average, 0.73ms max (perf sched). What grows is
    UserTime, +48%: the threads keep running, each operation just
    stalls longer on memory.
  - The cost is the placement itself, not migration or remote
    accesses:
      a) 32 threads: memory fully converges to the busy node well
         before the run ends, and the run is still 46% slower;
      b) CAS on with numa_balancing=0: no migrations, pages stay
         65/35, and the runtime is unchanged (42.2s);
      c) CAS off with the process simply taskset-pinned to one
         socket (memory 100% local from first touch, no
         migrations): 40.3s, within the CAS-on range (40.6-42.9s).
    64 memory-streaming threads on one socket get about two thirds
    of the throughput they get spread across both sockets (Mop/s
    -34%), wherever the pages live. So the only effective fix is to
    not aggregate such a process in the first place.

QUESTIONS
  1. Was the sweep-boundary gating of this accounting known or
     intended? For a workload with a large address space it means
     the footprint sees at most one update per run; for a shorter
     one, none at all.
  2. Could the check fail closed when there is no usable estimate
     (warm-up, footprint still 0, NUMA balancing off)? RSS is a poor
     estimate of the working set, as Peter said, but it is an upper
     bound: if RSS fits in the LLC, so does the working set. Using it
     only on the permit side - aggregate without a footprint estimate
     only if RSS fits - would keep the gains for small processes and
     not aggregate the ones we cannot judge. The costs are
     asymmetric: a missed aggregation forfeits a gain, a wrong one
     cost 65% here.
  3. Our data supports Peter's point that LLC hit/miss information
     would be the better signal: the damage is purely memory-side
     (UserTime +59% at flat CPU% in the 112-thread run), and a
     memory-streaming process would show a low hit rate regardless
     of how well its footprint is estimated. Is anyone looking at
     that?
  4. Chen Yu mentioned that you are reworking the footprint
     accounting for the prctl v2 series [2]. Could the detection fix
     be part of that rework?

[1] <50a8055e38958cdc39e07435aaad56e4719b2bd3.1794507151.git.tim.c.chen@xxxxxxxxxxxxxxx>
[2] <5c912bd3-8701-4395-a118-4ee6e09233e2@xxxxxxxxx>
[3] NAS Parallel Benchmarks: https://www.nas.nasa.gov/software/npb.html
    MG is the multigrid kernel: four V-cycles of 27-point stencil
    sweeps over a 3D grid hierarchy (finest grid 512^3 for class C,
    1024^3 for class D) solving a Poisson equation. Each sweep
    streams through a few double-precision arrays - a 3.3 GiB working
    set for class C, 26.4 GiB for class D - far beyond any LLC, with
    little reuse between sweeps: the kernel is memory-bandwidth-bound
    by construction.

The per-run logs, the placement/migration/sweep traces, and the
bpftrace scripts behind them are available if useful.

Regards,
Jirka Hladky
Red Hat


On Fri, Apr 10, 2026 at 11:30 AM Peter Zijlstra <peterz@xxxxxxxxxxxxx> wrote:
>
> On Fri, Apr 10, 2026 at 04:59:19PM +0800, Chen, Yu C wrote:
>
> > > This is pretty terrible. If you want LLC size, add it to the topology
> > > information (and ideally integrate with RDT) and make proportional to
> > > cpumask size, such that if someone cuts the domain in pieces, they get
> > > proportional size etc.
> > >
> >
> > If I understand correctly, do you mean the following:
> >
> > 1.Introduce a generic arch_get_llc_size() as a wrapper
> >   around the existing get_cpu_cacheinfo_level(), which
> >   returns the llc_size. Both the scheduler and RDT can
> >   use arch_get_llc_size().
>
> The tie in with RDT was more to affect the return of
> arch_get_llc_size(). Eg. when RDT takes away some ways for specific
> tasks, then the total effective size gets reduced for generic use.
>
>
> > 2. The sched domain stores llc_size in
> >    sd->res_size = llc_size * sd_span / arch_llc_span,
> >    and the cache_aware_scheduler uses sd->res_size for
> >    the comparison.
>
> Just so.
>
> > We will adjust the code accordingly.
>
> Thanks.
>
> > > Also, if we have NUMA_BALANCING on, that can provide a much better
> > > estimate for the actual size.
> > >
> > > Just using RSS seems like a very bad metric here.
> > >
> >
> > Got it. Currently we lack accurate memory footprint metrics in
> > the kernel. If we support user-provided hints in the future, we
> > can leverage RDT llc_occupancy metrics(Is it legal to use
> > RDT's metrics directly in the kernel? It would switch from
> > MSR-read to MMIO read thus less overhead). For now, let me have
> > a try how to leverage NUMA fault-in stats. If NUMA balancing
> > is off, I need to think more on how to avoid over-aggregation for
> > memory-intensive workloads.
>
> There is also things like this:
>
>   https://lkml.kernel.org/r/20260323095104.238982-1-bharata@xxxxxxx
>
> But yeah, in an ideal world we could be looking at LLC cache hit/miss
> information... streaming workloads would have very low hit rate.
>
> But yes, possible prctl() controls could help, create tools to disable
> things per program etc.
>


-- 
-Jirka