[PATCH 0/5] cpuidle: speed up do_idle() by caching the governor latency QoS constraint
From: Yaxiong Tian
Date: Tue Jul 21 2026 - 05:31:44 EST
cpuidle_governor_latency_req() is evaluated on every idle-state
selection. It aggregates the per-CPU resume latency with the global
CPU latency and wakeup latency QoS limits by repeatedly calling
get_cpu_device() and pm_qos_read_value() cpu_latency_qos_limit()
cpu_wakeup_latency_qos_limit().
Use ftrace's function_graph, we can see:
parent: do_idle
parent_total_ns: 36671010505
parent_count: 5994
SYMBOL TIME_NS %ROOT %PARENT COUNT
--------------------------------------------------------------------------------------------------
do_idle 36671010505 100.00% 100.00% 5994
cpuidle_idle_call 35566528731 96.99% 96.99% 8570
call_cpuidle 35476606844 96.74% 99.75% 8561
cpuidle_enter 35472932468 96.73% 99.99% 8526
cpuidle_select 52097031 0.14% 0.15% 8580
menu_select 49555181 0.14% 95.12% 8580
tick_nohz_get_sleep_length 28843887 0.08% 58.21% 8570
cpuidle_governor_latency_req 9567488 0.03% 19.31% 8580
tick_nohz_tick_stopped 2057031 0.01% 4.15% 15695
cpuidle_reflect 11427201 0.03% 0.03% 8561
menu_reflect 6579506 0.02% 57.58% 8526
tick_nohz_idle_got_tick 2231559 0.01% 33.92% 8526
__sysvec_apic_timer_interrupt 105520 0.00% 0.92% 3
tick_nohz_idle_stop_tick 8279641 0.02% 0.02% 1475
---- skip
The majority of the time spent in cpuidle_enter for CPUs entering
idle state has already been charged to the idle path. Among the
remaining contributors, cpuidle_governor_latency_req() accounts
for a non-negligible portion of the overall latency.
Under the menu governor this shows up hot: ftrace data shows,
cpuidle_governor_latency_req() accounts for about 19.9% of
menu_select() time (~1.9 us/call). After caching the aggregated
value per CPU and invalidating via QoS notifiers, that share drops to
about 4.2% (~0.3 us/call), roughly a 6x reduction on this path.
The ftrace data before and after the optimization is shown below:
1) original
parent: menu_select
parent_total_ns: 160492937
parent_count: 16718
SYMBOL TIME_NS %ROOT %PARENT COUNT
--------------------------------------------------------------------------------------------------
menu_select 160492937 100.00% 100.00% 16718
tick_nohz_get_sleep_length 100262940 62.47% 62.47% 16698
tick_nohz_next_event 67891299 42.30% 67.71% 16689
rcu_needs_cpu 2825649 1.76% 4.16% 16689
timekeeping_max_deferment 2380377 1.48% 3.51% 15296
hrtimer_next_event_without 17865162 11.13% 17.82% 15296
hrtimer_bases_next_event_without 2707631 1.69% 15.16% 15296
_raw_spin_lock_irqsave 2461132 1.53% 13.78% 15296
native_queued_spin_lock_slowpath 177 0.00% 0.01% 1
_raw_spin_unlock_irqrestore 2364647 1.47% 13.24% 15296
can_stop_idle_tick 2906072 1.81% 2.90% 16698
cpuidle_governor_latency_req 31988150 19.93% 19.93% 16718
get_cpu_device 4122502 2.57% 12.89% 16718
pm_qos_read_value 3427318 2.14% 10.71% 16718
cpu_latency_qos_limit 3005804 1.87% 9.40% 16718
cpu_wakeup_latency_qos_limit 3005475 1.87% 9.40% 16718
tick_nohz_tick_stopped 4551981 2.84% 2.84% 29496
%ROOT = share of menu_select; %PARENT = share of immediate caller (inclusive)
2) post-optimized
parent: menu_select
parent_total_ns: 55428604
parent_count: 7626
SYMBOL TIME_NS %ROOT %PARENT COUNT
--------------------------------------------------------------------------------------------------
menu_select 55428604 100.00% 100.00% 7626
tick_nohz_get_sleep_length 37464607 67.59% 67.59% 7544
tick_nohz_next_event 24076913 43.44% 64.27% 7522
get_next_timer_interrupt 16332381 29.47% 67.83% 5854
rcu_needs_cpu 1489633 2.69% 6.19% 7522
timekeeping_max_deferment 870851 1.57% 3.62% 5586
hrtimer_next_event_without 6140153 11.08% 16.39% 5586
hrtimer_bases_next_event_without 979000 1.77% 15.94% 5586
_raw_spin_lock_irqsave 808786 1.46% 13.17% 5586
_raw_spin_unlock_irqrestore 785518 1.42% 12.79% 5586
can_stop_idle_tick 1583137 2.86% 4.23% 7544
cpuidle_governor_latency_req 2321119 4.19% 4.19% 7626
tick_nohz_tick_stopped 1863015 3.36% 3.36% 12751
%ROOT = share of menu_select; %PARENT = share of immediate caller (inclusive)
A self-test case is introduced in patch 5 to validate that the existing
functionality remains intact. This can be handled as a standalone task.
This series:
- adds notifier APIs for the global CPU/wakeup latency QoS
- lets cpuidle subscribe and maintain a per-CPU generation
- invalidates only the affected CPU on per-CPU resume latency
changes
- caches the aggregated constraint in cpuidle_governor_latency_req()
- adds a kselftest covering the three QoS input paths
Yaxiong Tian (5):
pm: qos: add notifiers for CPU latency and wakeup latency QoS
cpuidle: subscribe to global latency QoS notifiers
cpuidle: invalidate latency gen on per-CPU resume QoS changes
cpuidle: cache aggregated governor latency QoS constraint
selftests/cpuidle: add latency_req QoS idle-state selection test
drivers/cpuidle/cpuidle.c | 15 +-
drivers/cpuidle/cpuidle.h | 2 +
drivers/cpuidle/governor.c | 141 +++++++-
include/linux/pm_qos.h | 23 ++
kernel/power/qos.c | 50 +++
tools/testing/selftests/Makefile | 1 +
tools/testing/selftests/cpuidle/Makefile | 6 +
tools/testing/selftests/cpuidle/config | 3 +
.../cpuidle/cpuidle_latency_req_qos.py | 331 ++++++++++++++++++
tools/testing/selftests/cpuidle/settings | 2 +
10 files changed, 566 insertions(+), 8 deletions(-)
create mode 100644 tools/testing/selftests/cpuidle/Makefile
create mode 100644 tools/testing/selftests/cpuidle/config
create mode 100755 tools/testing/selftests/cpuidle/cpuidle_latency_req_qos.py
create mode 100644 tools/testing/selftests/cpuidle/settings
--
2.43.0