Re: [PATCH v2] doc: Add CPU Isolation documentation

[Frederic Weisbecker]

Le Thu, Mar 26, 2026 at 03:17:48PM -0400, Waiman Long a écrit :
> On 3/26/26 10:00 AM, Frederic Weisbecker wrote:
> > nohz_full was introduced in v3.10 in 2013, which means this
> > documentation is overdue for 13 years.
> > 
> > Fortunately Paul wrote a part of the needed documentation a while ago,
> > especially concerning nohz_full in Documentation/timers/no_hz.rst and
> > also about per-CPU kthreads in
> > Documentation/admin-guide/kernel-per-CPU-kthreads.rst
> > 
> > Introduce a new page that gives an overview of CPU isolation in general.
> > 
> > Signed-off-by: Frederic Weisbecker <frederic@xxxxxxxxxx>
> > ---
> > v2:
> >     - Fix links and code blocks (Bagas and Sebastian)
> >     - Isolation is not only about userspace, rephrase accordingly (Valentin)
> >     - Paste BIOS issues suggestion from Valentin
> >     - Include the whole rtla suite (Valentin)
> >     - Rephrase a few details (Waiman)
> >     - Talk about RCU induced overhead rather than slower RCU (Sebastian)
> > 
> >   Documentation/admin-guide/cpu-isolation.rst | 357 ++++++++++++++++++++
> >   Documentation/admin-guide/index.rst         |   1 +
> >   2 files changed, 358 insertions(+)
> >   create mode 100644 Documentation/admin-guide/cpu-isolation.rst
> > 
> > diff --git a/Documentation/admin-guide/cpu-isolation.rst b/Documentation/admin-guide/cpu-isolation.rst
> > new file mode 100644
> > index 000000000000..886dec79b056
> > --- /dev/null
> > +++ b/Documentation/admin-guide/cpu-isolation.rst
> > @@ -0,0 +1,357 @@
> > +.. SPDX-License-Identifier: GPL-2.0
> > +
> > +=============
> > +CPU Isolation
> > +=============
> > +
> > +Introduction
> > +============
> > +
> > +"CPU Isolation" means leaving a CPU exclusive to a given workload
> > +without any undesired code interference from the kernel.
> > +
> > +Those interferences, commonly pointed out as "noise", can be triggered
> > +by asynchronous events (interrupts, timers, scheduler preemption by
> > +workqueues and kthreads, ...) or synchronous events (syscalls and page
> > +faults).
> > +
> > +Such noise usually goes unnoticed. After all synchronous events are a
> > +component of the requested kernel service. And asynchronous events are
> > +either sufficiently well distributed by the scheduler when executed
> > +as tasks or reasonably fast when executed as interrupt. The timer
> > +interrupt can even execute 1024 times per seconds without a significant
> > +and measurable impact most of the time.
> > +
> > +However some rare and extreme workloads can be quite sensitive to
> > +those kinds of noise. This is the case, for example, with high
> > +bandwidth network processing that can't afford losing a single packet
> > +or very low latency network processing. Typically those usecases
> > +involve DPDK, bypassing the kernel networking stack and performing
> > +direct access to the networking device from userscace.
> 
> As also pointed by by Sashiko, there is a typo "userscace" -> "userspace".
> There are also typos reported in
> 
> https://sashiko.dev/#/patchset/20260326140055.41555-1-frederic%40kernel.org

Thanks!

What do you think about these lines of Sashiko's review:

"""
Does this script violate the cgroup v2 "no internal process" constraint?
By enabling the cpuset controller on the test directory's
cgroup.subtree_control file, the cgroup cannot also contain processes.
"""

That is confusing me...

> 
> > +
> > +In order to run a CPU without or with limited kernel noise, the
> > +related housekeeping work needs to be either shutdown, migrated or
> > +offloaded.
> > +
> > +Housekeeping
> > +============
> > +
> > +In the CPU isolation terminology, housekeeping is the work, often
> > +asynchronous, that the kernel needs to process in order to maintain
> > +all its services. It matches the noises and disturbances enumerated
> > +above except when at least one CPU is isolated. Then housekeeping may
> > +make use of further coping mechanisms if CPU-tied work must be
> > +offloaded.
> > +
> > +Housekeeping CPUs are the non-isolated CPUs where the kernel noise
> > +is moved away from isolated CPUs.
> > +
> > +The isolation can be implemented in several ways depending on the
> > +nature of the noise:
> > +
> > +- Unbound work, where "unbound" means not tied to any CPU, can be
> > +  simply migrated away from isolated CPUs to housekeeping CPUs.
> > +  This is the case of unbound workqueues, kthreads and timers.
> > +
> > +- Bound work, where "bound" means tied to a specific CPU, usually
> > +  can't be moved away as-is by nature. Either:
> > +
> > +	- The work must switch to a locked implementation. Eg: This is
> > +	  the case of RCU with CONFIG_RCU_NOCB_CPU.
> > +
> > +	- The related feature must be shutdown and considered
> > +	  incompatible with isolated CPUs. Eg: Lockup watchdog,
> > +	  unreliable clocksources, etc...
> > +
> > +	- An elaborated and heavyweight coping mechanism stands as a
> > +	  replacement. Eg: the timer tick is shutdown on nohz_full but
> 
> "shutdown" should be 2 words as "shutdown" isn't a verb. Should we add CPU
> after "nohz_full" to make it more clear?

Right.

> 
> 
> > +	  with the constraint of running a single task on the CPU. A
> > +	  significant cost penalty is added on kernel entry/exit and
> > +	  a residual 1Hz scheduler tick is offloaded to housekeeping
> > +	  CPUs.
> > +
> > --- a/Documentation/admin-guide/index.rst
> > +++ b/Documentation/admin-guide/index.rst
> > @@ -94,6 +94,7 @@ likely to be of interest on almost any system.
> >      cgroup-v2
> >      cgroup-v1/index
> > +   cpu-isolation
> >      cpu-load
> >      mm/index
> >      module-signing
> 
> Other than the minor nits mentioned above,
> 
> Acked-by: Waiman Long <longman@xxxxxxxxxx>

Thanks!

> 

-- 
Frederic Weisbecker
SUSE Labs