Re: [PATCH v8 1/2] sched/cache: Reduce the overhead of task_cache_work by only scan the visisted cpus

From: Chen, Yu C

Date: Sun Jul 26 2026 - 21:09:54 EST

On 7/23/2026 12:04 PM, Luo Gengkun wrote:

[ ... ]

+static unsigned long fraction_mm_sched(int cpu,
+ struct mm_struct *mm)
{
+ struct sched_cache_time *pcpu_sched =
+ per_cpu_ptr(mm->sc_stat.pcpu_sched, cpu);
+ struct rq *rq = cpu_rq(cpu);
+
guard(raw_spinlock_irqsave)(&rq->cpu_epoch_lock);
__update_mm_sched(rq, pcpu_sched);
+ /* Skip the rq that has not been hit for a long time */
+ if ((rq->cpu_epoch - pcpu_sched->epoch_last_visit) > llc_epoch_affinity_timeout) {

In v2 there is a check if the cpu has been set before writing:
cpumask_test_cpu(cpu_of(rq), &mm->sc_stat.visited_cpus)
https://lore.kernel.org/all/20260414150745.225416-1-luogengkun2@xxxxxxxxxx/
do we need to bring that back?

+ cpumask_clear_cpu(cpu, mm->sc_stat.visited_cpus);
+ return 0;
+ }
+

[ ... ]

- for_each_cpu(i, sched_domain_span(sd)) {
- occ = fraction_mm_sched(cpu_rq(i),
- per_cpu_ptr(mm->sc_stat.pcpu_sched, i));
+ for_each_cpu_and(i, sched_domain_span(sd), mm->sc_stat.visited_cpus) {

Does using
for_each_cpu_and(i, sched_domain_span(sd), cpus)
reduce the race window to read mm->sc_stat.pcpu_sched?

[ ... ]

I ran hackbench[1] with above changes on Intel Sapphire Rapids with memory interleave enabled,
and on AMD Milan. There is no much difference on Sapphire Rapids, while a slight regress on
Milan AMD EPYC 9554P 64-Core when NUMA balancing is enabled(but it looks to be within
run-to-run variance) Please double check if this is the case on your AMD server.
And you can add my tag in next version:

Tested-by: Chen Yu <yu.c.chen@xxxxxxxxx>

Data on Milan
./launch.sh compare hackbench base_numab visit_numab
=========================================
Hackbench Comparison: base_numab vs visit_numab
=========================================
MODE GROUPS FDS | base_numab(s) | visit_numab(s) | DIFF(%) | VERDICT
---------- ------ -----+--------------------+--------------------+------------+-----------
process 1 10 | 34.564 ±0.53% | 34.103 ±1.48% | 1.33% | IMPROVED
process 1 16 | 60.556 ±0.52% | 62.296 ±2.11% | -2.87% | REGRESSED
process 1 2 | 2.981 ±0.03% | 3.814 ±21.34% | -27.94% | REGRESSED <--run-to-run variance
process 1 20 | 80.683 ±1.51% | 81.392 ±0.28% | -0.88% | REGRESSED
process 1 4 | 8.446 ±12.50% | 7.185 ±6.86% | 14.93% | IMPROVED
process 1 6 | 11.355 ±1.09% | 11.775 ±0.98% | -3.70% | REGRESSED
process 1 8 | 19.282 ±6.61% | 19.976 ±7.10% | -3.60% | REGRESSED
process 2 10 | 36.729 ±2.08% | 36.899 ±1.25% | -0.46% | REGRESSED
process 2 16 | 63.758 ±0.47% | 63.373 ±0.74% | 0.60% | IMPROVED
process 2 2 | 3.240 ±4.01% | 3.289 ±6.17% | -1.51% | REGRESSED
process 2 20 | 85.519 ±1.97% | 83.396 ±0.26% | 2.48% | IMPROVED
process 2 4 | 8.551 ±3.08% | 9.132 ±9.56% | -6.79% | REGRESSED
process 2 6 | 15.256 ±10.90% | 11.869 ±1.21% | 22.20% | IMPROVED
process 2 8 | 26.729 ±2.24% | 25.681 ±11.10% | 3.92% | IMPROVED
process 4 10 | 39.921 ±1.01% | 39.274 ±0.37% | 1.62% | IMPROVED
process 4 16 | 74.032 ±2.08% | 83.334 ±5.79% | -12.56% | REGRESSED
process 4 2 | 4.339 ±11.59% | 4.367 ±10.33% | -0.65% | REGRESSED
process 4 20 | 120.967 ±0.67% | 124.640 ±3.31% | -3.04% | REGRESSED
process 4 4 | 11.088 ±4.61% | 11.250 ±6.90% | -1.46% | REGRESSED
process 4 6 | 21.023 ±0.77% | 20.639 ±1.40% | 1.83% | IMPROVED
process 4 8 | 31.368 ±0.36% | 30.306 ±0.77% | 3.39% | IMPROVED
process 8 10 | 56.356 ±1.47% | 55.669 ±0.98% | 1.22% | IMPROVED
process 8 16 | 144.752 ±0.84% | 139.409 ±0.61% | 3.69% | IMPROVED
process 8 2 | 5.697 ±4.14% | 5.947 ±5.28% | -4.39% | REGRESSED
process 8 20 | 220.395 ±0.32% | 215.589 ±0.66% | 2.18% | IMPROVED
process 8 4 | 14.589 ±7.09% | 14.148 ±2.56% | 3.02% | IMPROVED
process 8 6 | 23.872 ±1.39% | 24.558 ±4.92% | -2.87% | REGRESSED
process 8 8 | 34.434 ±0.60% | 36.422 ±4.85% | -5.77% | REGRESSED
threads 1 10 | 35.582 ±0.62% | 35.697 ±0.94% | -0.32% | REGRESSED
threads 1 16 | 62.996 ±1.36% | 63.255 ±0.65% | -0.41% | REGRESSED
threads 1 2 | 3.216 ±0.16% | 3.253 ±1.20% | -1.15% | REGRESSED
threads 1 20 | 83.611 ±1.53% | 82.771 ±0.23% | 1.00% | IMPROVED
threads 1 4 | 6.952 ±0.50% | 6.957 ±0.06% | -0.07% | REGRESSED
threads 1 6 | 14.028 ±3.49% | 12.997 ±0.41% | 7.35% | IMPROVED
threads 1 8 | 21.511 ±1.71% | 21.395 ±2.73% | 0.54% | IMPROVED
threads 2 10 | 37.655 ±1.42% | 38.516 ±0.54% | -2.29% | REGRESSED
threads 2 16 | 67.366 ±2.88% | 68.230 ±3.80% | -1.28% | REGRESSED
threads 2 2 | 3.301 ±1.21% | 3.344 ±1.50% | -1.30% | REGRESSED
threads 2 20 | 94.511 ±1.12% | 87.065 ±0.73% | 7.88% | IMPROVED
threads 2 4 | 8.793 ±7.80% | 10.709 ±3.75% | -21.79% | REGRESSED
threads 2 6 | 16.934 ±11.50% | 16.483 ±15.58% | 2.66% | IMPROVED
threads 2 8 | 26.700 ±7.11% | 25.430 ±3.37% | 4.76% | IMPROVED
threads 4 10 | 40.121 ±1.36% | 40.582 ±2.06% | -1.15% | REGRESSED
threads 4 16 | 83.463 ±2.77% | 79.598 ±0.41% | 4.63% | IMPROVED
threads 4 2 | 4.855 ±3.95% | 4.753 ±15.72% | 2.10% | IMPROVED
threads 4 20 | 134.076 ±0.44% | 131.447 ±1.24% | 1.96% | IMPROVED
threads 4 4 | 13.193 ±6.68% | 11.804 ±13.90% | 10.53% | IMPROVED
threads 4 6 | 21.001 ±3.50% | 18.046 ±0.96% | 14.07% | IMPROVED
threads 4 8 | 30.247 ±2.34% | 28.422 ±2.57% | 6.03% | IMPROVED
threads 8 10 | 58.323 ±0.46% | 60.638 ±1.94% | -3.97% | REGRESSED
threads 8 16 | 149.357 ±0.83% | 153.554 ±0.57% | -2.81% | REGRESSED
threads 8 2 | 5.157 ±3.26% | 5.542 ±6.69% | -7.47% | REGRESSED
threads 8 20 | 227.619 ±0.40% | 229.307 ±1.23% | -0.74% | REGRESSED
threads 8 4 | 15.586 ±3.65% | 14.631 ±0.11% | 6.13% | IMPROVED
threads 8 6 | 25.401 ±2.47% | 24.989 ±2.39% | 1.62% | IMPROVED
threads 8 8 | 36.215 ±3.22% | 37.484 ±3.35% | -3.50% | REGRESSED

Note: Values shown as mean ±%stddev (seconds). DIFF based on means.
DIFF(%) = (base_numab - visit_numab) / base_numab * 100
Positive = improvement (visit_numab faster), Negative = regression.

[1] https://github.com/chen-yu-surf/bench_tool.git