Re: [PATCH v4] sched/fair: optimize and simplify rq leaf_cfs_rq_list

[Peter Zijlstra]

On Wed, Jun 01, 2022 at 03:22:12PM +0200, Vincent Guittot wrote:
> On Wed, 1 Jun 2022 at 04:19, Chengming Zhou <zhouchengming@xxxxxxxxxxxxx> wrote:
> >
> > We notice the rq leaf_cfs_rq_list has two problems when do bugfix
> > backports and some test profiling.
> >
> > 1. cfs_rqs under throttled subtree could be added to the list, and
> >    make their fully decayed ancestors on the list, even though not needed.
> >
> > 2. #1 also make the leaf_cfs_rq_list management complex and error prone,
> >    this is the list of related bugfix so far:
> >
> >    commit 31bc6aeaab1d ("sched/fair: Optimize update_blocked_averages()")
> >    commit fe61468b2cbc ("sched/fair: Fix enqueue_task_fair warning")
> >    commit b34cb07dde7c ("sched/fair: Fix enqueue_task_fair() warning some more")
> >    commit 39f23ce07b93 ("sched/fair: Fix unthrottle_cfs_rq() for leaf_cfs_rq list")
> >    commit 0258bdfaff5b ("sched/fair: Fix unfairness caused by missing load decay")
> >    commit a7b359fc6a37 ("sched/fair: Correctly insert cfs_rq's to list on unthrottle")
> >    commit fdaba61ef8a2 ("sched/fair: Ensure that the CFS parent is added after unthrottling")
> >    commit 2630cde26711 ("sched/fair: Add ancestors of unthrottled undecayed cfs_rq")
> >
> > commit 31bc6aeaab1d ("sched/fair: Optimize update_blocked_averages()")
> > delete every cfs_rq under throttled subtree from rq->leaf_cfs_rq_list,
> > and delete the throttled_hierarchy() test in update_blocked_averages(),
> > which optimized update_blocked_averages().
> >
> > But those later bugfix add cfs_rqs under throttled subtree back to
> > rq->leaf_cfs_rq_list again, with their fully decayed ancestors, for
> > the integrity of rq->leaf_cfs_rq_list.
> >
> > This patch takes another method, skip all cfs_rqs under throttled
> > hierarchy when list_add_leaf_cfs_rq(), to completely make cfs_rqs
> > under throttled subtree off the leaf_cfs_rq_list.
> >
> > So we don't need to consider throttled related things in
> > enqueue_entity(), unthrottle_cfs_rq() and enqueue_task_fair(),
> > which simplify the code a lot. Also optimize update_blocked_averages()
> > since cfs_rqs under throttled hierarchy and their ancestors
> > won't be on the leaf_cfs_rq_list.
> >
> > Signed-off-by: Chengming Zhou <zhouchengming@xxxxxxxxxxxxx>
> 
> Reviewed-by: Vincent Guittot <vincent.guittot@xxxxxxxxxx>

Thanks!