[PATCH v5 0/4] Fix uclamp race against non atomic static key enable

[Qais Yousef]

The latest version contains additional fixes to another related problem
exposed by this series. Fair and RT throttling were partially dequeuing
tasks when they are throttled without updating uclamp accounting. Patch
3 is marked as RFC as it partially reverts an old patch and rework it so
that RT usage of updating rq->nr_running is more consistent with the
other classes. Patch 4 then simply ensures that uclamp accounting is
updated as we update nr_running which is the source of truth when tasks
has moved in or out of the rq.

The series as a whole is no longer suitable for LTS though and I am not
sure it is worth more cycles given the low severity of the problem.
I kept the fixes tag on patch 1, but we can drop it and defer to folks
to pick it up if they are impacted by the race condition. Or keep it,
knowing that there will be a new (fairly harmless) warning appearing due
to the throttling path dequeueing without updating uclamp accounting.

I don't have a preference, but will err on dropping Fixes tag if no
clear voting comes up.

The series was tested with the test case below and I did check RT
nr_running accounting works correctly. But didn't check group RT or fair
throttling cases yet. I'd appreciate help in testing under these
scenarios.

In patch 3 I'd appreciate more eyes on inc/dec_rt_tasks() and removal of
rt_queued and its implication on some of the helper functions.

Changes since v4:
	- Fix problems in fair/RT throttling logic that were exposed by
	  the original fix.

Changes since v3:
	- Fix Sashiko report about force_idle paths where
	  rq->nr_running is not 0 highlighted by Christian
	- Update patch 1 commit message to reflect jump_label call
	  sites on the same CPU might not be updated atomically too

Changes since v2:
	- Use uclamp_rq_is_idle() instead of scanning buckets to detect
	  imbalance
	- Do some cleanup on accessing the flag as an additional patch

Changes since v1:
	- Fix kernel test robot report due to missing static inline in dummy
	  definition of init_uclamp_rq()

Tested with the patch below to force imbalance via sched_feat. I could see the
warning appearing and frequencies spiking during imbalance. Correct behavior is
restored when sched_feat is disabled where the idle entry reset fixes things
up.

I had a mix of uclamp_max mainly used in the system via schedqosd.

	diff --git a/kernel/sched/core.c b/kernel/sched/core.c
	index a2035d66270b9..14f9f1e419e02 100644
	--- a/kernel/sched/core.c
	+++ b/kernel/sched/core.c
	@@ -1889,6 +1889,11 @@ static inline void uclamp_rq_inc(struct rq *rq, struct task_struct *p, int flags
		for_each_clamp_id(clamp_id)
			uclamp_rq_inc_id(rq, p, clamp_id);

	+       if (sched_feat(UCLAMP_IMBALANCED)) {
	+               for_each_clamp_id(clamp_id)
	+                       uclamp_rq_inc_id(rq, p, clamp_id);
	+       }
	+
		/* Reset clamp idle holding when there is one RUNNABLE task */
		if (uclamp_rq_is_idle(rq))
			uclamp_rq_clear_idle(rq);
	diff --git a/kernel/sched/features.h b/kernel/sched/features.h
	index 8f0dee8fc4755..b59e532500fd9 100644
	--- a/kernel/sched/features.h
	+++ b/kernel/sched/features.h
	@@ -142,3 +142,5 @@ SCHED_FEAT(LATENCY_WARN, false)
	  */
	 SCHED_FEAT(NI_RANDOM, true)
	 SCHED_FEAT(NI_RATE, true)
	+
	+SCHED_FEAT(UCLAMP_IMBALANCED, false)
	diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c
	index 84301fc49e699..dbc3f5e1a7e01 100644
	--- a/kernel/sched/idle.c
	+++ b/kernel/sched/idle.c
	@@ -519,7 +519,7 @@ static void set_next_task_idle(struct rq *rq, struct task_struct *next, enum snt
		 * are still queued.
		 */
		if (uclamp_is_used() &&
	-           !rq->nr_running && unlikely(!uclamp_rq_is_idle(rq)))
	+           !rq->nr_running && unlikely(WARN_ON(!uclamp_rq_is_idle(rq))))
			init_uclamp_rq(rq);
	 }

Qais Yousef (4):
  sched/uclamp: Fix race condition leading to imbalanced uclamp_rq
  sched/uclamp: Use wrappers instead of open coding UCLAMP_FLAG_IDLE
  Revert "sched/rt: Substract number of tasks of throttled queues from
    rq->nr_running"
  sched/uclamp: Move uclamp_rq_{inc, dec}() from core.c

 kernel/sched/core.c    | 49 +++++++++---------------
 kernel/sched/ext/ext.c |  6 +--
 kernel/sched/fair.c    |  6 +--
 kernel/sched/idle.c    | 19 ++++++++++
 kernel/sched/rt.c      | 86 +++++++++---------------------------------
 kernel/sched/sched.h   | 27 +++++++++----
 6 files changed, 76 insertions(+), 117 deletions(-)

-- 
2.53.0