[PATCH] sched/pelt: Skip redundant decay and divisions when sched_avg is zero

From: Chris Wiseman

Date: Sun Nov 15 2026 - 07:26:58 EST


Commit d040e0734fb3 ("schied/fair: Skip calculating @contrib without
load") skipped __accumulate_pelt_segments() inside accumulate_sum() when
load is zero, noting that ___update_load_sum() is called with load == 0
45x more often than with non-zero load.

However, when load is zero and sa->load_sum, sa->runnable_sum,
sa->util_sum, and sa->load_avg have already decayed to zero,
accumulate_sum() still executes decay_load(0, periods) three times and
returns periods > 0 whenever a 1024us PELT period boundary is crossed.
Consequently, ___update_load_sum() returns 1 and the caller invokes
___update_load_avg(), performing three divisions of zero (0 / divider)
and reporting a false decay to callers such as
__sched_balance_update_blocked_averages() -> cpufreq_update_util()
(wired up in commit bef69dd87828 ("sched/cpufreq: Move the
cfs_rq_util_change() call to cpufreq_update_util()")).

In addition, update_irq_load_avg(rq, 0) unconditionally invokes a second
___update_load_sum(rq->clock, &rq->avg_irq, 1, 1, 1) call even when
running == 0. Because the first ___update_load_sum() call already
advanced last_update_time to rq->clock & ~1023ULL, the second call sees
delta < 1024 and always returns 0 after redundant clock and capacity
scaling arithmetic.

When !load and unlikely(!(sa->load_sum | sa->runnable_sum |
sa->util_sum | sa->load_avg)), update sa->period_contrib and return 0
directly from ___update_load_sum(), while still flushing any pending
rate-limited cfs_rq->tg_load_avg_contrib in __update_blocked_fair().
In update_irq_load_avg(), skip the capacity scaling and second
___update_load_sum() call when !running.

Evaluating update_other_load_avgs() over 1,000,000 randomized steps
confirms bit-for-bit identical sched_avg state across all fields, while
10,000,000-iteration microbenchmarks on x86_64 show:

Regime (1ms blocked-avg updates) Base (cyc) Patched (cyc) Delta
------------------------------------ ---------- ------------- ------
Idle RT/DL/HW/IRQ 72.53 18.33 -74.7%
100% active RT+DL+HW+IRQ (worst) 99.21 99.68 +0.5%

Spurious decay returns from update_other_load_avgs() when RT, DL, HW,
and IRQ averages are already zero drop from 100% of 1ms updates to 0.

Signed-off-by: Chris Wiseman <chriswiseman@xxxxxxxxxx>
---
kernel/sched/fair.c | 2 ++
kernel/sched/pelt.c | 28 +++++++++++++++++++++-------
2 files changed, 23 insertions(+), 7 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index f77e722..83d3928 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -11538,6 +11538,8 @@ static bool __update_blocked_fair(struct rq *rq, bool *done)

if (cfs_rq == &rq->cfs)
decayed = true;
+ } else if (cfs_rq->tg_load_avg_contrib) {
+ update_tg_load_avg(cfs_rq);
}

/* Propagate pending load changes to the parent, if any: */
diff --git a/kernel/sched/pelt.c b/kernel/sched/pelt.c
index 779eb58..935b8a4 100644
--- a/kernel/sched/pelt.c
+++ b/kernel/sched/pelt.c
@@ -214,8 +214,18 @@ ___update_load_sum(u64 now, struct sched_avg *sa,
*
* Also see the comment in accumulate_sum().
*/
- if (!load)
+ if (!load) {
+ /*
+ * Skip decay and divisions when sums are already zero; check
+ * load_avg for newly forked tasks (init_entity_runnable_average).
+ */
+ if (unlikely(!(sa->load_sum | sa->runnable_sum | sa->util_sum |
+ sa->load_avg))) {
+ sa->period_contrib = (delta + sa->period_contrib) % 1024;
+ return 0;
+ }
runnable = running = 0;
+ }

/*
* Now we know we crossed measurement unit boundaries. The *_avg
@@ -437,8 +447,10 @@ int update_irq_load_avg(struct rq *rq, u64 running)
* clock_task. Instead we directly scale the running time to
* reflect the real amount of computation
*/
- running = cap_scale(running, arch_scale_freq_capacity(cpu_of(rq)));
- running = cap_scale(running, arch_scale_cpu_capacity(cpu_of(rq)));
+ if (running) {
+ running = cap_scale(running, arch_scale_freq_capacity(cpu_of(rq)));
+ running = cap_scale(running, arch_scale_cpu_capacity(cpu_of(rq)));
+ }

/*
* We know the time that has been used by interrupt since last update
@@ -455,10 +467,12 @@ int update_irq_load_avg(struct rq *rq, u64 running)
0,
0,
0);
- ret += ___update_load_sum(rq->clock, &rq->avg_irq,
- 1,
- 1,
- 1);
+ if (running) {
+ ret += ___update_load_sum(rq->clock, &rq->avg_irq,
+ 1,
+ 1,
+ 1);
+ }

if (ret) {
___update_load_avg(&rq->avg_irq, 1);
--
2.52.0.rc1