Re: [PATCH v10 18/20] timers: Implement the hierarchical pull model

[Frederic Weisbecker]

Le Thu, Feb 01, 2024 at 05:15:37PM +0100, Anna-Maria Behnsen a écrit :
> Frederic Weisbecker <frederic@xxxxxxxxxx> writes:
> 
> > Le Mon, Jan 15, 2024 at 03:37:41PM +0100, Anna-Maria Behnsen a écrit :
> >> +static void tmigr_connect_child_parent(struct tmigr_group *child,
> >> +				       struct tmigr_group *parent)
> >> +{
> >> +	union tmigr_state childstate;
> >> +
> >> +	raw_spin_lock_irq(&child->lock);
> >> +	raw_spin_lock_nested(&parent->lock, SINGLE_DEPTH_NESTING);
> >> +
> >> +	child->parent = parent;
> >> +	child->childmask = BIT(parent->num_children++);
> >> +
> >> +	raw_spin_unlock(&parent->lock);
> >> +	raw_spin_unlock_irq(&child->lock);
> >> +
> >> +	/*
> >> +	 * To prevent inconsistent states, active children need to be active in
> >> +	 * the new parent as well. Inactive children are already marked inactive
> >> +	 * in the parent group.
> >> +	 */
> >> +	childstate.state = atomic_read(&child->migr_state);
> >> +	if (childstate.migrator != TMIGR_NONE) {
> >
> > Is it possible here to connect a running online child (not one that we just
> > created) to a new parent?
> 
> connect_child_parent() is only executed for the just created ones. So,
> yes in theory this would be possible, but it doesn't happen as
> tmigr_setup_groups() takes care to make it right (hopefully :)). When a
> LVL0 group has some space left, only the connection between tmc and the
> LVL0 group is done in tmigr_setup_groups(). If there is no space left in
> LVL0 group, then a new group is created and depending on the levels
> which has to be created only executed for the new ones.
> 
> > If not, is it possible that a newly created child is
> > not TMIGR_NONE?
> 
> Yes. See tmigr_cpu_online(). When new groups have to be created starting
> from LVL0, then they are not active - so TMIGR_NONE is set. Activating
> the new online CPU is done afterwards.
> 
> But if it is required to add also a new level at the top, then it is
> mandatory to propagate the active state of the already existing child to
> the new parent. The connect_child_parent() is then also executed for the
> formerly top level group (child) to the newly created group (parent).

Ah and this is why we have the "if (childstate.migrator != TMIGR_NONE)"
branch, right?

> > Heh, I was about to say that it's impossible that timer_base_is_idle()
> > at this stage but actually if we run in nohz_full...
> >
> > It happens so that nohz_full is deactivated until rcutree_online_cpu()
> > which calls tick_dep_clear() but it's a pure coincidence that might
> > disappear one day. So yes, let's keep it that way.
> 
> I instrumented the code (with NOHZ FULL and NOHZ_IDLE) to make sure the
> timer migration hierarchy state 'idle' is in sync with the timer base
> 'idle'. And this was one part where it was possible that it runs out of
> sync as I remember correctly. But if I understood you correctly, this
> shouldn't happen at the moment?

Well, it's not supposed to :-)

Thanks.

> 
> Thanks,
> 
> 	Anna-Maria
>