[PATCH 3/9] tools/sched_ext: scx_pair: Don't abort when the dispatching cgroup goes away

[Tao Cui]

From: Tao Cui <cuitao@xxxxxxxxxx>

try_dispatch() caches the cgroup id in pairc->cgid, drops pairc->lock
and then looks the cgroup back up in cgrp_q_idx_hash. pair_cgroup_exit()
deletes that hash entry when the cgroup is removed, so a plain rmdir of
the cgroup that is currently being dispatched, a normal event for
short-lived containers or cgroup churn, hits the lookup failure and
scx_bpf_error() takes the whole scheduler down.

The expiry path already knows how to re-pick a cgroup; make the fast
path tolerate the disappearance too. Mark the pair as drained on lookup
failure and return, letting the next dispatch expire the batch and pick
the next cgroup instead of aborting.

The enqueue side is left for a follow-up: a lookup miss is reachable
there in principle (css offline runs under the cgroup mutex alone,
and a sleeping-task migrate-out does not take the rq lock), but
handling it gracefully needs somewhere to put the task, which on the
enqueue path means a fallback queue; that is a larger change than
this fix scopes. The dispatch path, where the miss costs one skipped batch,
is the cheap half.

Fixes: f0262b102c7c ("tools/sched_ext: add scx_pair scheduler")
Signed-off-by: Tao Cui <cuitao@xxxxxxxxxx>
---
 tools/sched_ext/scx_pair.bpf.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/tools/sched_ext/scx_pair.bpf.c b/tools/sched_ext/scx_pair.bpf.c
index 0d61b7b812db..d38d3a078ade 100644
--- a/tools/sched_ext/scx_pair.bpf.c
+++ b/tools/sched_ext/scx_pair.bpf.c
@@ -478,8 +478,16 @@ static int try_dispatch(s32 cpu)
 	/* again, it'd be better to do all these with the lock held, oh well */
 	vptr = bpf_map_lookup_elem(&cgrp_q_idx_hash, &cgid);
 	if (!vptr) {
-		scx_bpf_error("failed to lookup q_idx for cgroup[%llu]", cgid);
-		return -ENOENT;
+		/*
+		 * @cgid can disappear from under us: pair_cgroup_exit()
+		 * drops the hash entry when the cgroup is going away.
+		 * Mark the pair as drained and let the next dispatch
+		 * expire it instead of taking the whole scheduler down.
+		 */
+		bpf_spin_lock(&pairc->lock);
+		pairc->draining = true;
+		bpf_spin_unlock(&pairc->lock);
+		goto out_maybe_kick;
 	}
 	q_idx = *vptr;
 
-- 
2.53.0

base-commit: 5b740a72b55c4267ba4d8010f17dde26b26f52d3