Re: [PATCH v5 1/2] sched/cache: Reduce the overhead of task_cache_work by only scan the visisted cpus
From: Luo Gengkun
Date: Mon Jul 13 2026 - 22:23:09 EST
On 2026/7/9 23:01, Chen, Yu C wrote:
Hi Gengkun,Thanks, I will fix it.
thanks for the update,
On 7/9/2026 9:00 PM, Luo Gengkun wrote:
+static unsigned long fraction_mm_sched(int cpu,
+ struct mm_struct *mm)
{
+ struct sched_cache_time *pcpu_sched =
+ per_cpu_ptr(mm->sc_stat.pcpu_sched, cpu);
+ struct rq *rq = cpu_rq(cpu);
+
guard(raw_spinlock_irqsave)(&rq->cpu_epoch_lock);
+ /* Skip the rq that has not been hit for a long time */
+ if ((rq->cpu_epoch - pcpu_sched->epoch_timeout) > llc_epoch_affinity_timeout) {
+ cpumask_clear_cpu(cpu, &mm->sc_stat.visited_cpus);
+ return 0;
+ }
+
I was wondering if there is a race condition in the scenario
above: If the rq has just updated its cpu_epoch field, and the
CPU subsequently remains idle, then rq->cpu_epoch stays unchanged.
That is to say, a race could occur as follows: if CPU1 becomes idle,
CPU2's task_cache_work() might compute
rq->cpu_epoch - pcpu_sched->epoch_timeout == 0 and consequently fail
to clear CPU1 from visit_cpus.
CPU_1 CPU_2
------------------------------------ ---------------------------------
account_mm_sched(rq_X)
set CPU1 in visited_cpus
(idle: fair task runs here,
CPU1.cpu_epoch frozen afterwards,
pcpu_sched->epoch_timeout = CPU1.epoch)
fraction_mm_sched()
CPU1->cpu_epoch - pcpu_sched->epoch_timeout = 0
-> bit CPU1 not cleared
__update_mm_sched() ->update cpu_epoch, too late
I wonder if we should let __update_mm_sched() be invoked before
if ((rq->cpu_epoch - pcpu_sched->epoch_timeout) > llc_epoch_affinity_timeout)?
BTW, in your test data:
sched_cache_scan: comm=redis-server pid=24660 scan=384
Is it because NUMA balancing is disabled on your platform that it has
to scan all the CPUs? It would be helpful to know what the result would
be with NUMA balancing enabled.
The following tests were conducted based on the fixed version.
It appears that the --mbind parameter in numactl causes NUMA balancing to fail,
so the following test only includes the --cpunodebind parameter.
When NUMA balancing is disabled:
valkey-benchmark rps | baseline | schedcache | schedcache_visit
| (no cache aware) | (without this series) | (with this series)
----------------------+---------------------+---------------------+-----------+-------------------+--------
| p99 latency(ms) | p99 latency(ms) | DIFF(%) | p99 latency(ms) | DIFF(%)
400000 | 0.436 | 0.554 | -25.68% | 0.441 | -1.14%
The overhead of task_cache_work is as follows:
valkey-benchmark rps | schedcache | schedcache_visit
----------------------+-------------------------------------------+---------------------------------------
400000 | 0.81% [kernel] [k] task_cache_work | 0.02% [kernel] [k] task_cache_work
When NUMA balancing is enabled:
valkey-benchmark rps | baseline | schedcache | schedcache_visit
| (no cache aware) | (without this series) | (with this series)
----------------------+---------------------+---------------------+-----------+-------------------+--------
| p99 latency(ms) | p99 latency(ms) | DIFF(%) | p99 latency(ms) | DIFF(%)
400000 | 0.437 | 0.454 | -3.89% | 0.442 | -1.14%
The overhead of task_cache_work is as follows:
valkey-benchmark rps | schedcache | schedcache_visit
----------------------+-------------------------------------------+---------------------------------------
400000 | 0.13% [kernel] [k] task_cache_work | 0.03% [kernel] [k] task_cache_work
The scan count for schedcache has been reduced to 96.
Additionally, I also conducted a test without restricting CPU affinity (no numactl), and
the results are as follows:
When NUMA balancing is disabled:
valkey-benchmark rps | baseline | schedcache | schedcache_visit
| (no cache aware) | (without this series) | (with this series)
----------------------+---------------------+---------------------+-----------+-------------------+--------
| p99 latency(ms) | p99 latency(ms) | DIFF(%) | p99 latency(ms) | DIFF(%)
400000 | 0.741 | 0.82 | -10.66% | 0.705 | +4.86%
When NUMA balancing is enabled:
valkey-benchmark rps | baseline | schedcache | schedcache_visit
| (no cache aware) | (without this series) | (with this series)
----------------------+---------------------+---------------------+-----------+-------------------+--------
| p99 latency(ms) | p99 latency(ms) | DIFF(%) | p99 latency(ms) | DIFF(%)
400000 | 0.75 | 0.744 | +0.80% | 0.736 | +1.87%
I found some reports from sashiko here that are worth taking a look at:Thank you for the reminder. I will fix these issues in the next version.
https://sashiko.dev/#/patchset/20260709130053.2749834-1-luogengkun2%40huawei.com
thanks,
Gengkun
thanks,
Chenyu