Re: [Patch v4 17/22] sched/cache: Avoid cache-aware scheduling for memory-heavy processes
From: Jirka Hladky
Date: Wed Nov 18 2026 - 10:51:13 EST
Hi Phil, Chen Yu,
Phil, on your question: the VM results were in my mail, but they may
have been cut in the quoting. The short version: the patched kernel
at the default tolerance loses the VM's 20 percent packing win
(22.47 s vs 21.81 s with CAS off), and gets it back at
aggr_tolerance=4 (17.70 s). That is the one regression the patches
introduce, and it was the known cost of the RSS bound. The refusal
is also not permanent: the RSS check only governs until the first
address-space sweep completes (~16 s for this VM), after which the
fault-based estimate takes over and grouping can start again, which
I traced on a long-running variant of the VM test. So the worst
case for a workload that genuinely should aggregate is a bounded
startup delay, not a permanent behavior change.
On safety elsewhere: with CAS off, the patched kernel matches stock
on everything we ran (NAS MG D 26.38 vs 26.45 s, MG C 9.38 vs
9.34 s, VM 21.81 vs 22.12 s, all n=5 medians and within noise), so
the patches are inert when the feature is off. With CAS on, the
behavioral change is confined to processes whose RSS exceeds the
LLC (they no longer aggregate) and to the estimate dynamics near
the threshold; processes with a small RSS aggregate exactly as
before.
Chen Yu, on your questions:
> Actually what I mean was if the RSS (not footprint accumulation
> during numa page fault) is used as the last resort if the
> footprint is 0, will this change fix the issue anyway? I guess
> the answer is yes, because you mentioned that the major fix is
> from the RSS gating.
Yes, confirmed with numbers. That is exactly the configuration I
tested (Tim's second patch falls back to the RSS when the history
is 0). NAS runtimes, n=5 medians:
workload stock OFF stock ON patched OFF patched ON
NAS MG D 1x64 26.45 s 41.78 s 26.38 s 26.47 s
NAS MG C 1x4 9.34 s 11.99 s 9.38 s 9.36 s
The gate refuses within ~0.1 s of the allocation phase, so the
CAS-on runtime matches the CAS-off runtime on the patched kernel.
The RSS fallback alone eliminates the reported regressions; your
sketch's accumulation is what makes the decision correct
afterward, not what stops the damage.
> The test code is a single process with several threads just doing
> a while(1) loop, without waking up each other. First, bind all
> threads to LLC0 (socket0), then release their CPU affinity, and
> watch how they are spread to LLC1.
I replicated your test on our 288-CPU box (64 spinning threads,
bound to LLC0, affinity released, placement sampled at 10 Hz).
With CAS off, the mean interval between threads reaching LLC1 is
1.59 s, against your model's 1.44 s for this CPU count: within 10
percent, so the model holds. It also matches the re-spread rate I
measured on the MG workload (~2 s per thread in the main window),
which supports your reading of my trace: the wake placement is
expected wake_affine behavior, and the load balancer's pace is the
rate limiter. My "wake placement plus conservative balancing"
should really read "conservative balancing, with wake placement
doing what it always does".
For completeness, I also ran it with CAS on: only 5 of 64 threads
reached LLC1 in 240 s. The spinning threads form a cache group
with a zero footprint (they never fault), so the group holds them
on LLC0.
On the numa_group idea: one caution from our two workloads. Both
qemu (which benefits from packing) and NAS MG (which loses 58%)
are single processes whose threads share one mm, so group or mm
membership alone does not separate them; what separates them is
the hot-set-to-LLC ratio. So I agree the sharing signal is the
right thing to look for, but it likely has to come from userspace
via prctl, or from a signal that tracks the actual sharing
fraction, rather than from grouping alone.
Thanks,
Jirka
On Tue, Nov 18, 2026 at 9:31 PM Chen Yu <chen.yu@xxxxxxxxx> wrote: