Re: [RFT][DONOTMERGE][PATCH 0/6] timers/migration: Prioritize lower capacity CPUs as migrators
From: Liang Hao
Date: Mon Sep 07 2026 - 10:15:35 EST
Hi,
I've been following the recent timer migration discussion and wanted to
share numbers from a classic 4+4 big.LITTLE board I have on hand.
cpuidle residency only; no power meter. Hope this is useful.
On 4+4 (all CPUs online): split raised LITTLE cpu-sleep under little-pin;
idle and Christian-style load were flat across unaware / split / prefer-low.
1) Hardware
Radxa ROCK 4D, RK3576
CPU0-3 Cortex-A53 cluster0 cpu_capacity=442 max 2016 MHz
CPU4-7 Cortex-A72 cluster1 cpu_capacity=1024 max 2208 MHz
Two capacity classes, 4 CPUs each. This is not Christian's Orion O6
(five classes, many with only 2 CPUs). This report is 4+4 with all
CPUs online; 4+1 / 2+2 topologies are planned after a gated-split
patch (section 7).
Idle states (same names on all 8 CPUs):
state0 WFI latency=1 residency=1
state1 cpu-sleep latency=370 residency=900
No LPI-0/1/2. Tables below are cpu-sleep. WFI was only taken on
idle-600 (unaware): ~0.2s A53 / ~1s A72 per cluster. The rest of
the matrix was not collected once that looked negligible.
2) Kernels
Rootfs: Armbian_26.2.5_Radxa-rock-4d_resolute_edge_7.0.1_minimal.
The stock 7.0.1 edge kernel was replaced. All three Images are
next-20260902 built from that image's .config (same dtb / userspace).
unaware unmodified next-20260902
7.3.0-rc1-next-20260902-next-rockpi4d
dmesg: Timer migration: 1 hierarchy levels;
8 children per group; 1 crossnode level
-> one LVL0 group, all 8 CPUs mixed
split 7.3.0-rc1-next-20260902-tmigr-split+
same tree; tmigr_get_capacity() returns real capacity
(vanilla next keeps everyone at SCHED_CAPACITY_SCALE)
boot dmesg line is unchanged (it is group-width, not
topology). little-pin CPU1 residency moving from ~32s
to ~120s is consistent with a 4-CPU LITTLE-only group.
No timer_migration_tree.py dump: board had no
network, so pip3/ete3 could not be installed.
Sorry — no ASCII tree this time; can follow up
if useful.
prefer-low 7.3.0-rc1-next-20260902-tmigr-prefer-low+
Frederic RFT 0/6 rebased onto the same base
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
timers/migration-capacity, HEAD: 28f02a6747ac4343f3596aaa3b749dea95577478
dmesg: Timer migration: 2 hierarchy levels;
8 children per group; 1 crossfamily level
-> two capacity families + a connecting level
Common: NO_HZ_IDLE, HZ=250, schedutil, kernel.timer_migration=1,
ARM_PSCI_CPUIDLE + menu. No NO_HZ_FULL.
3) Workloads (N=3, 600s each unless noted)
idle-600 sleep 600
tbench tbench_srv; tbench -R 20 -t 600 4 127.0.0.1
(Christian's -R 20; dbench 4.00 ~120s warmup; wall ~720s)
schbench schbench -m 2 -t 4 -r 600
little-pin four 4ms sleep loops (sleep 0.004) pinned to CPU0-3 for 600s
(Sehee-class load, tmigr on; not Sehee's tmigr=off case)
Residency is sum of /cpuidle/state1/time over the cluster, seconds.
A53 max = 4 * wall; A72 max = 4 * wall.
idle/schbench/pinned wall=600s => cluster max 2400s
tbench wall~720s => cluster max ~2880s
4) Main table -- per-run raw; mean is arithmetic mean of the three rows
cpu-sleep seconds = (after-before)/1e6 summed over the cluster.
idle-600 A53 / A72
unaware
N=1 2395.7 / 2399.2
N=2 2386.2 / 2400.2
N=3 2367.6 / 2396.2
mean 2383.2 / 2398.5
split
N=1 2396.4 / 2398.4
N=2 2398.1 / 2397.4
N=3 2399.4 / 2372.5
mean 2398.0 / 2389.4
prefer-low
N=1 2394.8 / 2399.5
N=2 2395.2 / 2380.0
N=3 2377.9 / 2399.2
mean 2389.3 / 2392.9
tbench A53 / A72 | throughput | max latency
unaware
N=1 978.8 / 1985.2 14.4431 MB/s 4.444 ms
N=2 982.6 / 1985.1 14.4405 MB/s 4.131 ms
N=3 982.7 / 1983.9 14.4237 MB/s 3.817 ms
mean 981.4 / 1984.7 14.4358 MB/s 4.131 ms
split
N=1 976.7 / 1979.5 14.5214 MB/s 4.136 ms
N=2 982.1 / 1981.2 14.5000 MB/s 4.017 ms
N=3 982.2 / 1979.1 14.5176 MB/s 4.128 ms
mean 980.3 / 1979.9 14.5130 MB/s 4.094 ms
prefer-low
N=1 978.9 / 1984.1 14.4451 MB/s 4.268 ms
N=2 978.7 / 1983.1 14.4437 MB/s 3.719 ms
N=3 983.2 / 1985.0 14.3993 MB/s 3.876 ms
mean 980.3 / 1984.1 14.4294 MB/s 3.954 ms
schbench A53 / A72 | wake p99 | req p99 | rps
unaware
N=1 401.9 / 3.2 3996 us 56000 us 447.54
N=2 396.3 / 3.2 4004 us 58176 us 433.53
N=3 370.7 / 2.9 3996 us 62272 us 411.69
mean 389.6 / 3.1 3999 us 58816 us 430.92
split
N=1 420.5 / 4.0 3980 us 52544 us 469.75
N=2 391.7 / 3.1 3996 us 56896 us 435.84
N=3 367.7 / 3.1 3996 us 58176 us 417.24
mean 393.3 / 3.4 3991 us 55872 us 440.94
prefer-low
N=1 428.9 / 3.8 3980 us 51136 us 481.38
N=2 425.1 / 3.3 3996 us 54208 us 463.08
N=3 412.7 / 3.1 3996 us 56000 us 449.39
mean 422.2 / 3.4 3991 us 53781 us 464.62
little-pin A53 / A72
unaware
N=1 291.2 / 2377.0
N=2 283.4 / 2376.4
N=3 292.8 / 2376.8
mean 289.1 / 2376.7
split
N=1 346.3 / 2372.1
N=2 408.2 / 2371.6
N=3 381.8 / 2372.0
mean 378.8 / 2371.9
prefer-low
N=1 279.6 / 2379.6
N=2 286.8 / 2377.7
N=3 291.4 / 2377.5
mean 285.9 / 2378.3
5) little-pin per-CPU cpu-sleep (s)
unaware
cpu0 cpu1 cpu2 cpu3 cpu4 cpu5 cpu6 cpu7
N=1 12.7 32.4 122.7 123.4 589.3 598.7 591.5 597.5
N=2 11.7 31.7 120.8 119.1 588.3 596.1 595.4 596.6
N=3 12.9 32.4 122.6 124.8 590.3 596.4 598.0 592.0
split
cpu0 cpu1 cpu2 cpu3 cpu4 cpu5 cpu6 cpu7
N=1 15.6 109.2 108.2 113.3 586.5 592.3 596.0 597.3
N=2 16.1 132.4 129.2 130.5 585.2 589.5 598.2 598.7
N=3 16.5 122.9 119.2 123.1 586.6 591.1 596.2 598.1
prefer-low
cpu0 cpu1 cpu2 cpu3 cpu4 cpu5 cpu6 cpu7
N=1 12.8 116.9 31.9 118.0 593.2 591.5 598.0 596.9
N=2 12.4 121.5 31.5 121.3 592.5 595.4 590.7 599.1
N=3 12.9 122.1 32.5 124.0 589.6 595.7 595.5 596.8
split: cpu1 109/132/123 vs unaware 32/32/32. prefer-low keeps two
busy LITTLE CPUs (~13s / ~32s on cpu0+cpu2); split spreads sleep
across cpu1-3 (~110-130s).
6) Reading the three columns on 4+4
little-pin, big (Sehee-class; tmigr on)
A72 2377 / 2372 / 2378 on all three variants (~594 s/CPU, ~5s below
idle). Big deep idle not broken here; not Sehee's tmigr=off case.
No A72 gap for split or prefer-low to recover.
little-pin, LITTLE: split vs O6
Christian's O6 failure mode (split hurts LITTLE) is opposite here:
A53 289 -> 379 on split (cpu1 recovered). prefer-low A53 286,
unaware-like (cpu2 takes the ~32s slot).
last-idle big / migrator
No sign big held the global migrator under little-pin: A72 ~full
cpu-sleep (~594 s/CPU). No migrator traces.
idle and load (prefer-low vs unaware)
Idle already ~100% cpu-sleep on unaware; prefer-low cannot win.
Load: tbench 14.44 / 14.51 / 14.43 MB/s; schbench wake p99 4.0 ms
on all three (schbench keeps A72 busy, so low A72 residency there
is expected). Not worse, not better. Same Christian O6 reading
under load; here no idle prize either.
prefer-low: does it really? (Frederic, July 2026 thread)
Tree is up (2 levels, crossfamily). want_low_migrator not traced.
little-pin shape (cpu0+cpu2 busy) matches a connected tree, not
isolated split.
7) Next steps
Section 6 on this board: split raised LITTLE cpu-sleep on little-pin;
prefer-low matched unaware; big stayed in cpu-sleep. Christian's O6
numbers went the other way on LITTLE when capacity classes are small.
I plan to patch tmigr_get_capacity() for a gated per-capacity split:
- online count in every capacity class >= 4 -> arch_scale_cpu_capacity()
- otherwise -> SCHED_CAPACITY_SCALE (same as unaware today)
- nohz_full still forces unaware
Re-test with the same Images base plus a gated-split build:
idle + little-pin + tbench on 4+4 here, then static 4+1 / 2+2
topologies (CPUs offlined before each run, held for the full 600s).
First try only: cutoff 4 is a starting guess (this board 4+4, O6 often
2). The first patch would count online CPUs at boot/setup only; no
runtime hotplug tree rebuild, no split-big-only policy.
prefer-low did not beat unaware; next step is gated split, not
another prefer-low round.
I would appreciate feedback on cutoff choice, hotplug handling, and
on section 8 gaps worth closing before a patch.
8) Not in this mail
- tmigr=off
- nohz_full
- tmigr_connect / handle_remote traces
- timer_migration_tree.py (no network on board for pip3/ete3;
can follow up)
- 4+1 / 2+2 hotplug (after gated-split patch; not in this report)
- power (mW)
Thanks,
Liang Hao