Re: [RFC PATCH v2 1/2] tick/nohz: Add a policy dependency for full dynticks

[Peng Wang]

On Wed, Oct 28, 2026 at 04:58:47PM +0100, Frederic Weisbecker wrote:
> Le Sat, Oct 24, 2026 at 12:32:38AM +0800, Peng Wang a écrit :
> > On Fri, Oct 23, 2026 at 05:00:22PM +0200, Frederic Weisbecker wrote:
> > > Le Sat, Oct 17, 2026 at 09:13:41PM +0800, Peng Wang a 殂rit :
> > > > Cloud hosts may pin vCPU threads one-to-one to host CPUs. At full
> > > > allocation this includes the housekeeping CPUs, which always retain
> > > > their non-idle tick. Operators want comparable performance for otherwise
> > > > equivalent vCPUs. Retaining the non-idle tick on nohz_full CPUs as well
> > > > avoids one placement-dependent difference in host tick interference.
> > > > With spare capacity, selected CPUs may stop the tick to reduce host
> > > > scheduler-tick interference with vCPUs.
> > > > 
> > > > Allocation changes as VMs are created or destroyed while other VMs
> > > > remain running. A vCPU thread may be the only runnable task in either
> > > > state, so existing tick dependencies do not express this policy.
> > > 
> > > I'm not sure I understand correctly what you are trying to solve.
> > > Are you saying that you want to keep the tick running when there
> > > is more than one vCPU per CPU? Because the scheduler already takes
> > > care of that.
> > 
> > Hi Frederic,
> > 
> > No. We pin vCPU threads one-to-one to host CPUs, without
> > oversubscription. By "full allocation" I meant that all allocatable
> > host CPUs are in use, including CPUs outside the nohz_full set.
> > 
> > For example, on an eight-CPU host booted with isolcpus=nohz,1-7,
> > CPU0 is outside the nohz_full set:
> > 
> > - With eight pinned vCPUs, CPU0 retains its non-idle tick while
> >   CPUs1-7 may stop theirs. Retaining the tick on CPUs1-7 too avoids
> >   this placement-dependent difference in tick interference between
> >   otherwise equivalent vCPUs.
> > 
> > - With seven or fewer pinned vCPUs, we can leave CPU0 without a
> >   vCPU and keep them on CPUs1-7. We can then allow tick suppression
> >   on CPUs1-7, subject to the existing dependencies. This gives
> >   these vCPUs a common tick policy while reducing host tick
> >   interference.
> > 
> > If the vCPU on CPU0 is released while the other seven remain
> > running, we allow CPUs1-7 to suppress their ticks. If a new vCPU
> > is later assigned to CPU0, we retain the ticks on CPUs1-7 again
> > to avoid the difference in tick interference.
> > 
> > Throughout these transitions, CPU1 can still have its original
> > pinned vCPU thread as its only runnable task. Its runnable task
> > count does not change merely because CPU0 gains or loses a vCPU,
> > so the scheduler's existing tick dependency cannot express this
> > allocation policy.
> 
> Ok now I understand what you want to do, I just don't understand why.
> Why it matters to turn off nohz_full on all CPUs when vCPUs run on housekeeping?
> 
> > > > Add a per-CPU POLICY dependency within the boot-time nohz_full set.
> > > > Permission defaults to enabled. The policy can be changed without
> > > > taking CPUs offline. Revoking permission uses the existing asynchronous
> > > > kick; granting it leaves all other dependencies effective. Preserve
> > > > POLICY across CPU hotplug and report it in tick_stop tracing.
> > > > 
> > > > Idle tick suppression, nohz_full membership, context tracking, RCU
> > > > offloading and housekeeping configuration are unchanged.
> > > 
> > > This still keeps the overhead of nohz_full even if the tick stays on, right?
> > > (RCU nocb, kernel <-> user overhead, etc...). Isn't it a problem for you?
> > 
> > Yes. Our setup uses only the nohz flag of isolcpus, without
> > additional housekeeping affinity tuning. Context tracking and
> > RCU callback offloading remain enabled when the tick is kept
> > running.
> > 
> > We have observed a performance benefit from this nohz_full setup
> > with our database workload running on pinned vCPUs.
> 

Hi Frederic,

> Do you mean that your database runs better when hosts CPUs are in nohz_full?

Yes. In our database tests, suppressing the host's non-idle
periodic tick improved performance, with the rest of the host
configuration unchanged.

> But how does this new interface help? Ie: why is it worse to have all CPUs in

The tick policy itself can therefore affect guest performance.
A large host runs VMs for many customers with different workloads.
Customers create and release VMs independently, so the same host
CPU can be reassigned to a different workload while other VMs
remain running.

We want to choose whether to retain the non-idle tick or allow its
suppression for the workload currently assigned to each CPU, and
revisit that choice when the CPU is reassigned. Per-CPU control
lets us do this without rebooting or taking CPUs offline.

> nohz_full except the housekeeping rather than shutting down nohz_full
> altogether?
> 
> Or to put it another way: why is it better to have either:
> 
> - all vCPUs on nohz_full host CPUs
> or
> - all vCPUs on ticking CPUs
> 
> And why does it hurt to have a single vCPU running on a ticking CPU.
>

If a VM's vCPUs span housekeeping and nohz_full CPUs, some may
benefit from tick suppression while others cannot. For workloads
sensitive to tick interference, this can cause performance
differences within the VM. A common tick policy is one deployment
option in that case.

In summary, the runtime control would let us:

1. Select or withdraw tick suppression as CPUs are assigned to
   different workloads.
2. Apply a common non-idle tick policy across a VM's vCPUs when
   preferred.

Thanks,
Peng
 
> Thanks.
> 
> -- 
> Frederic Weisbecker
> SUSE Labs