[PATCH] x86/aperfmperf: Refresh stale sample via IPI for busy NOHZ_FULL CPUs

[Jing Wu]

An isolated CPU covered by nohz_full stops its periodic tick once it
has only one runnable task, since sched_can_stop_tick() only checks
scheduling-class fairness and has no notion of cpufreq reporting
needs. arch_scale_freq_tick() runs only from scheduler_tick(), so
cpu_samples for that CPU is never refreshed again.

arch_freq_get_on_cpu() then permanently hits its staleness check and
falls back to cpufreq_quick_get(), which returns whatever policy->cur
was left at (typically the P-state floor). This happens even though
HWP hardware keeps running the CPU at full turbo autonomously, as
confirmed by turbostat and by directly reading APERF/MPERF.

Reproduce on an isolated, nohz_full CPU with intel_pstate/HWP by
loading it and watching scaling_cur_freq stay pinned at the floor:

    taskset -c $CPU stress --cpu 1 &
    for i in $(seq 10); do
        cat /sys/devices/system/cpu/cpu$CPU/cpufreq/scaling_cur_freq
        sleep 0.5
    done
    turbostat --cpu $CPU --interval 1 --num_iterations 5

scaling_cur_freq stays at the floor for the whole run, while
turbostat's Bzy_MHz confirms the CPU is actually at full turbo.

Refresh the stale sample with one on-demand arch_scale_freq_tick()
via IPI before falling back, but only when the target CPU is online
and not idle. APERF/MPERF both stop advancing during idle (C1+), so
a delta computed over an arbitrarily long stale window still yields
a correct busy-time frequency average.

Fixes: 7d84c1ebf9dd ("x86/aperfmperf: Replace aperfmperf_get_khz()")
Co-developed-by: Qiliang Yuan <yuanql9@xxxxxxxxxxxxxxx>
Signed-off-by: Qiliang Yuan <yuanql9@xxxxxxxxxxxxxxx>
Co-developed-by: Jian Zhang <zhangj332@xxxxxxxxxxxxxxx>
Signed-off-by: Jian Zhang <zhangj332@xxxxxxxxxxxxxxx>
Signed-off-by: Jing Wu <realwujing@xxxxxxxxx>
---
 arch/x86/kernel/cpu/aperfmperf.c | 34 +++++++++++++++++++++++++++++++++-
 1 file changed, 33 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/aperfmperf.c b/arch/x86/kernel/cpu/aperfmperf.c
index 7ffc78d5ebf21..e51544db9f9bb 100644
--- a/arch/x86/kernel/cpu/aperfmperf.c
+++ b/arch/x86/kernel/cpu/aperfmperf.c
@@ -12,6 +12,7 @@
 #include <linux/math64.h>
 #include <linux/percpu.h>
 #include <linux/rcupdate.h>
+#include <linux/sched.h>
 #include <linux/sched/isolation.h>
 #include <linux/sched/topology.h>
 #include <linux/smp.h>
@@ -503,16 +504,41 @@ void arch_scale_freq_tick(void)
  */
 #define MAX_SAMPLE_AGE	((unsigned long)HZ / 50)
 
+static void aperfmperf_snapshot_cpu_ipi(void *info)
+{
+	arch_scale_freq_tick();
+}
+
+/*
+ * A NOHZ_FULL CPU with a single runnable task (e.g. an isolated CPU running
+ * a pinned PMD/busy-poll workload) stops its periodic tick, so nothing ever
+ * calls arch_scale_freq_tick() for it again and cpu_samples goes stale
+ * forever, not just for one MAX_SAMPLE_AGE window. Force one on-demand
+ * sample via IPI so a deliberate frequency read doesn't report the P-state
+ * floor from before isolation took effect. Skip idle CPUs: their frequency
+ * genuinely doesn't matter and there is no point poking them with an IPI.
+ */
+static bool aperfmperf_refresh_stale_sample(int cpu)
+{
+	if (!cpu_online(cpu) || idle_cpu(cpu))
+		return false;
+
+	smp_call_function_single(cpu, aperfmperf_snapshot_cpu_ipi, NULL, 1);
+	return true;
+}
+
 int arch_freq_get_on_cpu(int cpu)
 {
 	struct aperfmperf *s = per_cpu_ptr(&cpu_samples, cpu);
 	unsigned int seq, freq;
 	unsigned long last;
+	bool refreshed = false;
 	u64 acnt, mcnt;
 
 	if (!cpu_feature_enabled(X86_FEATURE_APERFMPERF))
 		goto fallback;
 
+again:
 	do {
 		seq = raw_read_seqcount_begin(&s->seq);
 		last = s->last_update;
@@ -524,8 +550,14 @@ int arch_freq_get_on_cpu(int cpu)
 	 * Bail on invalid count and when the last update was too long ago,
 	 * which covers idle and NOHZ full CPUs.
 	 */
-	if (!mcnt || (jiffies - last) > MAX_SAMPLE_AGE)
+	if (!mcnt || (jiffies - last) > MAX_SAMPLE_AGE) {
+		if (!refreshed) {
+			refreshed = true;
+			if (aperfmperf_refresh_stale_sample(cpu))
+				goto again;
+		}
 		goto fallback;
+	}
 
 	return div64_u64((cpu_khz * acnt), mcnt);
 

---
base-commit: 502d801f0ab03e4f32f9a33d203154ce84887921
change-id: 20260728-bug-isolatecpu-cpufreq-864a5fba85b7

Best regards,
-- 
Jing Wu <realwujing@xxxxxxxxx>