Re: [PATCH] sched/fair: Prevent dead task groups from regaining cfs_rq's

[Mathias Krause]

Am 05.11.21 um 15:25 schrieb Vincent Guittot:
> On Thu, 4 Nov 2021 at 18:37, Mathias Krause <minipli@xxxxxxxxxxxxxx> wrote:
>>
>> Am 04.11.21 um 17:49 schrieb Vincent Guittot:
>>> [snip]
>>>
>>> Ok so we must have 2 GPs:
>>>
>>> list_del_rcu(&tg->siblings);
>>> GP to wait for the end of ongoing walk_tg_tree_from : synchronize_rcu
>>> in your patch
>>> list_del_leaf_cfs_rq(tg->cfs_rq[cpu]); if on_list
>>> remove_entity_load_avg(tg->se[cpu]);
>>> GP to wait for the end of ongoing for_each_leaf_cfs_rq_safe (print_cfs_stats)
>>> kfree everything
>>
>> Basically yes, but with my patch we already have these two, as there's
>> at least one RCU GP between after sched_offline_group() finishes and
>> sched_free_group() / cpu_cgroup_css_free() starts.
>>
>> So we either use my patch as-is or move unregister_fair_sched_group() to
>> free_fair_sched_group() and use kfree_rcu() instead of kfree(). Both
>> approaches have pros and cons.
>>
>> Pro for my version is the early unlinking of cfs_rq's for dead task
>> groups, so no surprises later on. Con is the explicit synchronize_rcu().
> 
> which blocks the caller and could be problematic
> 
> It seems that LKP has reported such issue:
> 20211104145128.GC6499@xsang-OptiPlex-9020

Heh, indeed.

>>
>> Pro for the kfree_rcu() approach is the lack of the explicit
>> synchronize_rcu() call, so no explicit blocking operation. Con is that
>> we have cfs_rq's re-added to dead task groups which feels wrong and need
>> to find a suitable member to overlap with the rcu_head in each involved
>> data type.
>>
>> Which one do you prefer?

Looks like it needs to be the kfree_rcu() one in this case. I'll prepare
a patch.

Thanks,
Mathias