Re: [PATCH] timers: Exclude isolated cpus from timer migation
From: Waiman Long
Date: Thu Apr 10 2025 - 11:09:43 EST
On 4/10/25 10:54 AM, Frederic Weisbecker wrote:
Le Thu, Apr 10, 2025 at 04:46:06PM +0200, Thomas Gleixner a écrit :
On Thu, Apr 10 2025 at 16:20, Frederic Weisbecker wrote:
Le Thu, Apr 10, 2025 at 03:56:02PM +0200, Gabriele Monaco a écrit :
On Thu, 2025-04-10 at 15:27 +0200, Frederic Weisbecker wrote:
But how do we handle global timers that have been initialized and
queued from
isolated CPUs?
I need to sketch a bit more the solution but the rough idea is:
1. isolated CPUs don't pull remote timers
That's the "easy" part.
2. isolated CPUs ignore their global timers and let others pull them
perhaps with some more logic to avoid it expiring
This will always involve added overhead because you may need to wake up
a CPU upon enqueueing a global timer to make sure it will be handled.
At least when all other CPUs are idle.
Which is true for the remote enqueue path too. But you inflict the
handling of this muck into the generic enqueue path as you have to turn
a 'global' timer into a remote timer right in the hot path.
Fair point.
When you enqueue it in the regular way on the 'global' list, then you
can delegate the expiry to a remote CPU on return to user, no?
If you're referring to nohz_full, it's not a problem there because
it's already considered as an idle CPU.
But for isolcpus alone that notification is necessary. I'm not sure
if return to user is the best place. I hear that some kernel threads
can spend a lot of time doing things...
But to begin with, is this all really necessary for isolcpus users?
In some of the actual customer use cases that I have seen, both
"nohz_full" and "isolcpus" are set to the same set of CPUs to achieve
maximum isolation. isolcpus have domain and managed_irq that are not
covered by nohz_full. As isolcpus=nohz has been extended to be
equivalent to nohz_full, users can now just use "isolcpus" to achieve
that maximum CPU isolation.
Cheers,
Longman