Re: [PATCH v4] time/tick-sched: idle load balancing when nohz_full cpu becomes idle.

[Peter Zijlstra]

On Thu, May 16, 2024 at 01:43:49PM +0100, Yun Levi wrote:
> > > > > As such, I don't think the HK_TYPE_SCHED check in
> > > > > nohz_balance_enter_idle() actually makes sense, the on_null_omain()
> > > > > check a little below that should already take care of things, no?
> > > >
> > > > IIUC,
> > > > currently, whether cpu belongs on domain or null is determined by
> > > > HK_DOMAIN_FLAGS
> > >
> > > No! you can create NULL domains without any of the HK nonsense. Both
> > > isolcpus and cpusets can create single CPU partitions.
> 
> Yes. However what I said, nohz_full cpu isn't on null_domain
> unless it was configured by cpusets.
> even with option "nohz_full="

So if a CPU isn't isolated, either by isolcpus or cpusets, you get to
participate in load-balancing -- end of story.

> > > > However, when "nohz_full=" is used, it still on HK_DOMAIN, so it
> > > > belongs to sched_domain
> > > > so, it couldn't be filtered out by on_null_domain().
> > > >
> > > > unless "isolcpus=domain" or "isolcpus={cpu_list}", it's on null domain.
> > > > with "isolcpus=tick", it participates sched_domain.
> > >
> > > Frederic ?!? You can use nohz_full without isolcpus? That makes no
> > > sense. If you do that you get to keep the pieces.
> >
> > I fear you can yes, even though most users combine it with isolcpus. I
> > know, that interface is terrible but it dates from times when we weren't
> > sure about all the potential usecases of nohz_full. There was a possibility
> > that HPC could just want to reduce ticks without all the hard and costly
> > isolation around. But all the usecases I have witnessed so far in ten years
> > involved wanting 0 noise after all...
> 
> 
> If I make you annoyed I'm sorry in advance but let me clarify please.
> 
> 1. In case of none-HK-TICK-housekeeping cpu (a.k.a nohz_full cpu),
>     It should be on the null_domain. right?
> 
> 2. If (1) is true, when none-HK-TICK is set, should it set none-HK-DOMAIN
>     to prevent on any sched_domain (cpusets filter out none-HK-DOMAIN cpu)?
> 
> 3. If (1) is true, Is HK_SCHED still necessary? There seems to be no use case
>     and the check for this can be replaced by on_null_domain().

I've no idea about all those HK knobs, it's all insane if you ask me.

Frederic, afaict all the HK_ goo in kernel/sched/fair.c is total
nonsense, can you please explain?

If the CPU participates in load-balancing, it gets to fully participate.
If you want to get out of load-balancing, you get single CPU partitions.