Re: [PATCH] sched/deadline: Make dl-server nohz full aware

[Juri Lelli]

Hi Furkan,

On 12/05/26 13:06, Furkan Çalışkan wrote:
> Hi Juri,
> 
> On 5/12/26 12:02, Juri Lelli wrote:
> > The dl_server_timer() causes spurious IPIs on nohz_full cores, breaking
> > isolation guarantees. The timer executes on a housekeeping core and
> > eventually calls tick_nohz_dep_set_cpu(), sending IPIs to isolated cores
> > even when only a single task is running.
> > 
> > The problem is that dl-servers are not coordinated with nohz_full tick
> > state. Timers can fire and send IPIs to otherwise undisturbed cores.
> > 
> > Fix by managing servers in sched_can_stop_tick():
> > 
> > - When RT tasks run with CFS/SCX tasks, start the appropriate server
> >   and keep the tick running
> > - When only RT tasks remain, stop all servers and allow tick to stop
> >   (except for >1 RR tasks which need the tick for round-robin)
> > - When only CFS/SCX tasks remain, stop all servers before stopping tick
> > 
> > Introduce dl_servers_stop_all() to reduce duplication and abstract
> > server management from core.c. Unify RT handling into one block that
> > handles both RR and FIFO cases.
> > 
> > Fixes: 557a6bfc662c ("sched/fair: Add trivial fair server")
> > Reported-by: David Haufe <dhaufe@xxxxxxxxxxxxxxxxxx>
> > Closes: https://lore.kernel.org/lkml/CAKJHwtOw_G67edzuHVtL1xC5Vyt6StcZzihtDd0yaKudW=rwVw@xxxxxxxxxxxxxx
> > Signed-off-by: Juri Lelli <juri.lelli@xxxxxxxxxx>
> > ---
> > I had to modify my first original attempt at fixing this (please take a
> > look at the linked report/discussion) to also take SCX into
> > consideration.
> > 
> > FYI, I temporarily pushed the script I'm using to repro and verify the
> > fix here
> > 
> > https://github.com/jlelli/sched-deadline-tests/blob/master/test-dlserver-nohz.sh
> > ---
> >  kernel/sched/core.c     | 43 +++++++++++++++++++++++--------------------
> >  kernel/sched/deadline.c | 14 ++++++++++++++
> >  kernel/sched/sched.h    |  1 +
> >  3 files changed, 38 insertions(+), 20 deletions(-)
> > 
> > diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> > index b905805bbcbe4..98759255c306b 100644
> > --- a/kernel/sched/core.c
> > +++ b/kernel/sched/core.c
> > @@ -1414,30 +1414,35 @@ static inline bool __need_bw_check(struct rq *rq, struct task_struct *p)
> >  
> >  bool sched_can_stop_tick(struct rq *rq)
> >  {
> > -	int fifo_nr_running;
> > -
> >  	/* Deadline tasks, even if single, need the tick */
> >  	if (rq->dl.dl_nr_running)
> >  		return false;
> >  
> >  	/*
> > -	 * If there are more than one RR tasks, we need the tick to affect the
> > -	 * actual RR behaviour.
> > +	 * If there are RT tasks, we may need the tick (for >1 RR tasks),
> > +	 * but we must also service lower-priority CFS/SCX tasks via dl-servers.
> >  	 */
> > -	if (rq->rt.rr_nr_running) {
> > -		if (rq->rt.rr_nr_running == 1)
> > -			return true;
> > -		else
> > +	if (rq->rt.rt_nr_running) {
> > +		if (rq->cfs.h_nr_queued) {
> > +			dl_server_start(&rq->fair_server);
> > +			return false;
> > +		}
> > +#ifdef CONFIG_SCHED_CLASS_EXT
> > +		if (rq->scx.nr_running) {
> > +			dl_server_start(&rq->ext_server);
> > +			return false;
> > +		}
> > +#endif
> 
> In the above block, the CFS and SCX server start paths are mutually exclusive. 
> If both cfs.h_nr_queued and scx.nr_running are non-zero at the same time, only 
> fair_server gets started and ext_server remains stopped. Could that leave SCX 
> tasks without server coverage in a mixed CFS+SCX+RT scenario?

Indeed there is the partial switch mode to consider. Can fix in the next
version.

Thanks,
Juri