Re: [PATCH v2] sched/fair: Fix enqueue_task_fair warning some more

[Tao Zhou]

Hi Phil,

On Thu, May 07, 2020 at 04:36:12PM -0400, Phil Auld wrote:
> sched/fair: Fix enqueue_task_fair warning some more
> 
> The recent patch, fe61468b2cb (sched/fair: Fix enqueue_task_fair warning)
> did not fully resolve the issues with the rq->tmp_alone_branch !=
> &rq->leaf_cfs_rq_list warning in enqueue_task_fair. There is a case where
> the first for_each_sched_entity loop exits due to on_rq, having incompletely
> updated the list.  In this case the second for_each_sched_entity loop can
> further modify se. The later code to fix up the list management fails to do
> what is needed because se no longer points to the sched_entity which broke
> out of the first loop.
> 

> Address this by calling leaf_add_rq_list if there are throttled parents while
> doing the second for_each_sched_entity loop.

Thanks for your trace imformation and explanation. I
truely have learned from this and that.

s/leaf_add_rq_list/list_add_leaf_cfs_rq/

> 
> Suggested-by: Vincent Guittot <vincent.guittot@xxxxxxxxxx>
> Signed-off-by: Phil Auld <pauld@xxxxxxxxxx>
> Cc: Peter Zijlstra (Intel) <peterz@xxxxxxxxxxxxx>
> Cc: Vincent Guittot <vincent.guittot@xxxxxxxxxx>
> Cc: Ingo Molnar <mingo@xxxxxxxxxx>
> Cc: Juri Lelli <juri.lelli@xxxxxxxxxx>
> ---
>  kernel/sched/fair.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 02f323b85b6d..c6d57c334d51 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -5479,6 +5479,13 @@ enqueue_task_fair(struct rq *rq, struct task_struct *p, int flags)
>  		/* end evaluation on encountering a throttled cfs_rq */
>  		if (cfs_rq_throttled(cfs_rq))
>  			goto enqueue_throttle;
> +
> +               /*
> +                * One parent has been throttled and cfs_rq removed from the
> +                * list. Add it back to not break the leaf list.
> +                */
> +               if (throttled_hierarchy(cfs_rq))
> +                       list_add_leaf_cfs_rq(cfs_rq);
>  	}

I was confused by why the throttled cfs rq can be on list.
It is possible when enqueue a task and thanks to the 'threads'.
But I think the above comment does not truely put the right
intention, right ?
If throttled parent is onlist, the child cfs_rq is ignored
to be added to the leaf cfs_rq list me think.

unthrottle_cfs_rq() follows the same logic if i am not wrong.
Is it necessary to add the above to it ?

Thanks,
Tau

>  
>  enqueue_throttle:
> -- 
> 2.18.0
> 
> V2 rework the fix based on Vincent's suggestion. Thanks Vincent.
> 
> 
> Cheers,
> Phil
> 
> -- 
>