Re: [PATCH v2] sched/fair: Distribute burst-woken tasks when rq depth reaches 3x average
From: Li kunyu
Date: Sun Nov 08 2026 - 21:51:24 EST
On 2026/11/6 14:26, Li kunyu wrote:
> +/*
> + * Check if a CPU's rq depth reaches the 3x system-wide average threshold
> + * cached in root_domain (floored at 3 tasks).
> + */
> +static inline bool rq_burst_overloaded(struct rq *rq)
> +{
> + if (unlikely(!rq->rd))
> + return false;
> +
> + return READ_ONCE(rq->nr_running) >= READ_ONCE(rq->rd->overload_nr_threshold);
> +}
After multiple rounds of testing, certain optimizations in this patch caused
performance regressions under specific workloads. I have fixed these issues in
v3, and after comprehensive testing confirmed satisfactory results, v3 will be
released.
> +static inline void reset_periodic_lb_interval(int cpu)
> +{
> + struct sched_domain *sd;
> + unsigned long next = jiffies + 1;
> + struct rq *rq = cpu_rq(cpu);
> +
> + if (time_after(READ_ONCE(rq->next_balance), next))
> + WRITE_ONCE(rq->next_balance, next);
> +
> + for_each_domain(cpu, sd) {
> + if (READ_ONCE(sd->balance_interval) > sd->min_interval)
> + WRITE_ONCE(sd->balance_interval, sd->min_interval);
> + }
> +}
READ_ONCE and WRITE_ONCE have been removed from the two hotspots function above
to avoid excessive overhead.
> /*
> * select_task_rq_fair: Select target runqueue for the waking task in domains
> * that have the relevant SD flag set. In practice, this is SD_BALANCE_WAKE,
> @@ -9920,6 +9947,7 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags)
> int want_affine = 0;
> /* SD_flags and WF_flags share the first nibble */
> int sd_flag = wake_flags & 0xF;
> + struct rq *target_rq;
>
>
> /*
> * required for stable ->cpus_allowed
> @@ -9942,6 +9970,17 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags)
> want_affine = !wake_wide(p) && cpumask_test_cpu(cpu, p->cpus_ptr);
> }
>
> + target_rq = want_affine ? this_rq() : cpu_rq(prev_cpu);
> + if (unlikely(rq_burst_overloaded(target_rq))) {
> + int dist_cpu = cpumask_any_and_distribute(p->cpus_ptr, cpu_active_mask);
> +
> + if (likely(dist_cpu < nr_cpumask_bits && cpu_rq(dist_cpu)->nr_running <
> + target_rq->nr_running)) {
> + reset_periodic_lb_interval(dist_cpu);
> + return dist_cpu;
> + }
> + }
> +
Move the CPU overload checking code to the end of the function to avoid jitter.
Invoke it only when no more suitable CPU can be selected; doing so yields a
stable improvement, with relatively balanced task distribution and overall CPU
usage.
--
thanks,
kunyu