Re: [PATCH v2] isolcpus: affine kernel threads to specified cpumask

From: Chris Friesen
Date: Tue Mar 24 2020 - 11:56:35 EST


I hadn't been keeping up with all the changes to the "isolcpus" boot arg. Given how it's been extended, I agree that it seems the logical place to deal with this. Patch seems okay to me, but I've got a couple of nits in the message portion.

If I want to specify both no_kthreads and managed_irq it then something like "isolcpus=managed_irq,no_kthreads,2-16" would work?


On 3/24/2020 9:20 AM, Marcelo Tosatti wrote:

This is a kernel enhancement to configure the cpu affinity of kernel
threads via kernel boot option isolcpus=no_kthreads,<isolcpus_params>,<cpulist>

https://github.com/torvalds/linux/blob/master/Documentation/admin-guide/kernel-parameters.txt says that "isolcpus" is deprecated. Are we un-deprecating it? Or is it only really deprecated for the "domain" option?

When this option is specified, the cpumask is immediately applied upon
thread launch. This does not affect kernel threads that specify cpu
and node.

This allows CPU isolation (that is not allowing certain threads
to execute on certain CPUs) without using the isolcpus=domain parameter,
making it possible to enable load balancing on such CPUs
during runtime (see

I think you're missing the rest of the sentence here.

Note-1: this is based off on Wind River's patch at
https://github.com/starlingx-staging/stx-integ/blob/master/kernel/kernel-std/centos/patches/affine-compute-kernel-threads.patch

Difference being that this patch is limited to modifying
kernel thread cpumask: Behaviour of other threads can
be controlled via cgroups or sched_setaffinity.

Note-2: MontaVista's patch was based off Christoph Lameter's patch at
https://lwn.net/Articles/565932/ with the only difference being
the kernel parameter changed from kthread to kthread_cpus.

Wind River, not MontaVista. I know all us embedded linux folks look the same...


Signed-off-by: Marcelo Tosatti <mtosatti@xxxxxxxxxx>

---

v2: use isolcpus= subcommand (Thomas Gleixner)

Documentation/admin-guide/kernel-parameters.txt | 8 ++++++++
include/linux/cpumask.h | 5 +++++
include/linux/sched/isolation.h | 1 +
init/main.c | 1 +
kernel/cpu.c | 13 +++++++++++++
kernel/kthread.c | 4 ++--
kernel/sched/isolation.c | 6 ++++++
7 files changed, 36 insertions(+), 2 deletions(-)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index c07815d230bc..7318e3057383 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -1959,6 +1959,14 @@
the CPU affinity syscalls or cpuset.
<cpu number> begins at 0 and the maximum value is
"number of CPUs in system - 1".
+ When using cpusets, use the isolcpus option no_kthreads
+ to avoid creation of kernel threads on isolated CPUs.
+
+ no_kthreads
+ Adjust the CPU affinity mask of unbound kernel threads to
+ not contain CPUs on the isolated list. This complements
+ the isolation provided by the cpusets mechanism described
+ above.

It also complements the "managed_irq" option below. In many cases I'd expect the same set of CPUs to be isolated from both irqs and kernel threads.


Chris