[PATCH] sched/fair: Take slice protection into account when arming HRTICK

[Christian Loehle]

EEVDF can end a task's slice protection before its virtual deadline when
shorter requests compete on the runqueue. update_curr() requests a new
selection once protection expires, but hrtick_start_fair() still arms the
timer for the virtual deadline. Without an intervening scheduling event,
the next opportunity to reconsider the task can therefore arrive much
later than the protection boundary.

For example, with two continuously runnable equal-weight SCHED_OTHER
tasks requesting 100 us and 1 ms on one CPU, the unpatched HZ=250 kernel
shows approximately 1 ms between context switches for both tasks. A
short request can be repicked repeatedly, so its uninterrupted run is
not limited to one request. CPU shares remain approximately equal;
the problem is the scheduling granularity.

Arm the fair hrtick for the earlier of the virtual deadline and live
slice protection. Keep the deadline fallback once protection expires.
A same-task repick restarts the timer without renewing protection, so
newly eligible tasks can compete without waiting for another protected
quantum.

A wakeup can shorten protection without preempting current. The enqueue
path calls hrtick_update() before wakeup_preempt_fair() updates protection,
and hrtick_update() leaves an active timer alone. Recompute the fair
hrtick after update_protect_slice() changes vprot, including when a timer
is already active. If the new protection boundary has already passed,
request lazy rescheduling, as update_curr() does on protection expiry,
instead of falling back to the later deadline.

Reported-by: Vincent Guittot <vincent.guittot@xxxxxxxxxx>
Link: https://lore.kernel.org/r/CAKfTPtD_aJeAh1biW6gKCLK6BnYcNtbz-FJvxUzZ1tD3V=43mw@xxxxxxxxxxxxxx/
Link: https://lore.kernel.org/r/CAKfTPtApFGcYA22Qtg3OyYEt6EJw_FW9_VV=_vgE_HtNY3V9og@xxxxxxxxxxxxxx/
Signed-off-by: Christian Loehle <christian.loehle@xxxxxxx>
---
 kernel/sched/fair.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 1c687c3c70f1..5b2965591ad8 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -7798,9 +7798,13 @@ static void hrtick_start_fair(struct rq *rq, struct task_struct *p)
 		return;
 
 	/*
-	 * Compute time until virtual deadline
+	 * Reconsider the pick when live slice protection expires, which can be
+	 * earlier than the virtual deadline when shorter requests compete.
+	 * Without live protection, use the virtual deadline instead.
 	 */
 	vdelta = se->deadline - se->vruntime;
+	if (protect_slice(se))
+		vdelta = min_vruntime(se->deadline, se->vprot) - se->vruntime;
 	if ((s64)vdelta < 0) {
 		if (task_current_donor(rq, p))
 			resched_curr(rq);
@@ -10023,8 +10027,18 @@ static void wakeup_preempt_fair(struct rq *rq, struct task_struct *p, int wake_f
 	if (preempt_action == PREEMPT_WAKEUP_SHORT && entity_eligible(cfs_rq, pse))
 		goto preempt;
 update:
-	if (sched_feat(RUN_TO_PARITY))
+	if (sched_feat(RUN_TO_PARITY)) {
+		u64 old_vprot = se->vprot;
+
 		update_protect_slice(cfs_rq, se);
+		if (hrtick_enabled_fair(rq) && se->vprot != old_vprot) {
+			/* The enqueue may have armed the timer before this update. */
+			if (protect_slice(se))
+				hrtick_start_fair(rq, donor);
+			else
+				resched_curr_lazy(rq);
+		}
+	}
 
 	return;
 
-- 
2.34.1