Re: [PATCH -tip 10/32] sched: Fix priority inversion of cookied task with sibling

[Balbir Singh]

On Thu, Nov 26, 2020 at 09:29:14AM +0100, Peter Zijlstra wrote:
> On Thu, Nov 26, 2020 at 10:05:19AM +1100, Balbir Singh wrote:
> > > @@ -5259,7 +5254,20 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf)
> > >  			 * Optimize the 'normal' case where there aren't any
> > >  			 * cookies and we don't need to sync up.
> > >  			 */
> > > -			if (i == cpu && !need_sync && !p->core_cookie) {
> > > +			if (i == cpu && !need_sync) {
> > > +				if (p->core_cookie) {
> > > +					/*
> > > +					 * This optimization is only valid as
> > > +					 * long as there are no cookies
> > 
> > This is not entirely true, need_sync is a function of core cookies, so I
> > think this needs more clarification, it sounds like we enter this when
> > the core has no cookies, but the task has a core_cookie? The term cookie
> > is quite overloaded when used in the context of core vs task.
> 
> Nah, its the same. So each task gets a cookie to identify the 'group' of
> tasks (possibly just itself) it is allowed to share a core with.
> 
> When we to core task selection, the core gets assigned the cookie of the
> group it will run, same thing.
> 
> > Effectively from what I understand this means that p wants to be
> > coscheduled, but the core itself is not coscheduling anything at the
> > moment, so we need to see if we should do a sync and that sync might
> > cause p to get kicked out and a higher priority class to come in?
> 
> This whole patch is about eliding code-wide task selection when it is
> not required. IOW an optimization.
> 
> When there wasn't a core cookie (IOW, the previous task selection wasn't
> core wide and limited) and the task we just selected for our own CPU
> also didn't have a cookie (IOW it doesn't have to be core-wide) we can
> skip the core wide task selection and schedule just this CPU and call it
> a day.
> 
> The logic was subtly wrong, this patch fixes it. A next patch completely
> rewrites it again to make it far simpler again. Don't spend time trying
> to understand this patch (unless you're _that_ kind of person ;-) but
> instead apply the whole thing and look at the resulting pick_next_task()
> function.

Thanks, I'll look at the git tree and see what the final outcome looks like.

Balbir Singh.