Re: [RFT][DONOTMERGE][PATCH 0/6] timers/migration: Prioritize lower capacity CPUs as migrators

[Christian Loehle]

On 6/25/26 17:41, Frederic Weisbecker wrote:
> Hi,
> 
> It has been reported that the timer migration tree wasn't too helpful
> on asymetric systems, with timers migrating too often to big capacity
> CPUs, waking them more frequently from deep idle state.
> 
> Splitting the tree into per CPU capacity hierarchies doesn't seem to
> improve the situation.
> 
> So here is another proposal: gather CPUs belonging to the same capacity
> under the same groups and finally join those capacity groups under a
> common root. Exactly in the same way it is done for NUMA systems.

This seems to work for my orion o6, the resulting hierarchy is:

                           root lvl1
                      family=-1, want_low_migrator=1
          ________________|___________|___________|____________
         /                /           /           /            /
    mask 0x1         mask 0x2    mask 0x4    mask 0x8    mask 0x10
    grp279           grp866      grp905      grp984      grp1024
    CPUs 2,3,4,5     CPUs 8,9    CPUs 6,7    CPUs 10,11  CPUs 0,1

> 
> And then always pick up the lowest capacity non-idle CPU group as the
> migrator with the hope that handling timers will keep lower capacity
> CPUs busy enough to increase their chances to become global idle
> migrators.
> 
> Please give it a try but make sure you have fewer than 9 capacities
> on your system. Handling more will require a bit more throughts. Let's
> see first if this is the right way to go.

Honestly I think buckets should be fine here? One for each 128-capacity
step (or even 256), which would also make these awkward Grace capacities all
fall into one, which makes a lot of sense IMO?

> 
> git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
> 	timers/migration-capacity
> 
> HEAD: 28f02a6747ac4343f3596aaa3b749dea95577478
> Thanks,
> 	Frederic
> ---
> 
> Frederic Weisbecker (6):
>       timers/migration: Revert per CPU capacity hierarchy
>       timers/migration: Defer initialization after capacity topology is setup
>       sched/topology: Account asym capacities number
>       timers/migration: Group CPUs per capacity
>       timers/migration: Prefer lower capacity groups as migrators
>       scripts/timer_migration_tree.py: Dump mask of each group
> 
>  include/linux/sched/topology.h         |   3 +
>  include/trace/events/timer_migration.h |  48 ++--
>  kernel/sched/topology.c                |  44 ++++
>  kernel/time/timer_migration.c          | 393 +++++++++++++++------------------
>  kernel/time/timer_migration.h          |  33 +--
>  scripts/timer_migration_tree.py        |  65 +++---
>  6 files changed, 288 insertions(+), 298 deletions(-)

Thanks for giving this a shot, here are the results, I tested this against
tmigr-off (which I've added ontop of series and capacity-unaware dd50a6e260ea "timers/migration: Revert per CPU capacity hierarchy"))
In short: tmigr-off-600, capacity-unaware-600, series-600


idle-600s
----------

Overall idle states:

+------------------------+---------+---------+---------+--------------+
| Variant                | LPI-0   | LPI-1   | LPI-2   | LPI-1 + LPI-2|
+------------------------+---------+---------+---------+--------------+
| tmigr-off-600          | 3771.0s | 650.9s  | 2212.0s | 2862.9s      |
| capacity-unaware-600   | 2461.7s | 1245.3s | 2708.9s | 3954.3s      |
| series-600             | 2429.2s | 1260.7s | 2726.5s | 3987.2s      |
+------------------------+---------+---------+---------+--------------+

Delta vs tmigr-off-600:

+------------------------+----------+---------+---------+--------------+
| Variant                | LPI-0    | LPI-1   | LPI-2   | LPI-1 + LPI-2|
+------------------------+----------+---------+---------+--------------+
| capacity-unaware-600   | -1309.2s | 594.4s  | 497.0s  | 1091.4s      |
| series-600             | -1341.7s | 609.8s  | 514.5s  | 1124.3s      |
+------------------------+----------+---------+---------+--------------+

LPI-2 by capacity group:

+-------+----------------+------------------------+------------+
| Group | tmigr-off-600  | capacity-unaware-600   | series-600 |
+-------+----------------+------------------------+------------+
| 279   | 805.3s         | 1007.3s                | 1003.5s    |
| 866   | 391.7s         | 483.5s                 | 477.3s     |
| 905   | 411.6s         | 466.1s                 | 501.8s     |
| 984   | 361.3s         | 468.3s                 | 458.9s     |
| 1024  | 242.2s         | 283.8s                 | 285.0s     |
+-------+----------------+------------------------+------------+


schbench-600s
-------------

Overall idle states:

+------------------------+--------+--------+--------+--------------+
| Variant                | LPI-0  | LPI-1  | LPI-2  | LPI-1 + LPI-2|
+------------------------+--------+--------+--------+--------------+
| tmigr-off-600          | 167.7s | 34.1s  | 22.3s  | 56.4s        |
| capacity-unaware-600   | 158.8s | 37.2s  | 28.1s  | 65.3s        |
| series-600             | 161.5s | 37.3s  | 26.3s  | 63.6s        |
+------------------------+--------+--------+--------+--------------+

Delta vs tmigr-off-600:

+------------------------+--------+-------+-------+--------------+
| Variant                | LPI-0  | LPI-1 | LPI-2 | LPI-1 + LPI-2|
+------------------------+--------+-------+-------+--------------+
| capacity-unaware-600   | -8.9s  | 3.1s  | 5.8s  | 8.9s         |
| series-600             | -6.2s  | 3.2s  | 4.0s  | 7.3s         |
+------------------------+--------+-------+-------+--------------+

LPI-2 by capacity group:

+-------+----------------+------------------------+------------+
| Group | tmigr-off-600  | capacity-unaware-600   | series-600 |
+-------+----------------+------------------------+------------+
| 279   | 8.9s           | 10.6s                  | 10.3s      |
| 866   | 3.3s           | 4.8s                   | 4.3s       |
| 905   | 3.8s           | 4.6s                   | 4.3s       |
| 984   | 3.3s           | 4.8s                   | 4.7s       |
| 1024  | 3.0s           | 3.2s                   | 2.7s       |
+-------+----------------+------------------------+------------+


tbench-4clients-600s
--------------------

Overall idle states:

+------------------------+---------+---------+--------+--------------+
| Variant                | LPI-0   | LPI-1   | LPI-2  | LPI-1 + LPI-2|
+------------------------+---------+---------+--------+--------------+
| tmigr-off-600          | 3125.7s | 1395.7s | 425.9s | 1821.6s      |
| capacity-unaware-600   | 3009.4s | 1437.7s | 456.3s | 1894.0s      |
| series-600             | 2998.3s | 1436.3s | 453.6s | 1889.9s      |
+------------------------+---------+---------+--------+--------------+

Delta vs tmigr-off-600:

+------------------------+---------+--------+--------+--------------+
| Variant                | LPI-0   | LPI-1  | LPI-2  | LPI-1 + LPI-2|
+------------------------+---------+--------+--------+--------------+
| capacity-unaware-600   | -116.2s | 41.9s  | 30.5s  | 72.4s        |
| series-600             | -127.3s | 40.6s  | 27.7s  | 68.3s        |
+------------------------+---------+--------+--------+--------------+

LPI-2 by capacity group:

+-------+----------------+------------------------+------------+
| Group | tmigr-off-600  | capacity-unaware-600   | series-600 |
+-------+----------------+------------------------+------------+
| 279   | 162.9s         | 179.3s                 | 180.5s     |
| 866   | 68.6s          | 73.4s                  | 71.9s      |
| 905   | 67.7s          | 69.5s                  | 71.8s      |
| 984   | 72.9s          | 75.5s                  | 71.5s      |
| 1024  | 53.7s          | 58.6s                  | 57.9s      |
+-------+----------------+------------------------+------------+

Seems the series does what it claims, I'd love to also hear about Sehee's
results to the original regression (as honestly, capacity-unaware doesn't
seem so bad for me in the first case, so maybe my testing has a blind spot
here?)