Re: [PATCH 0/2] bind rcu offload (nohz_full/isolation) into cpuset

[Frederic Weisbecker]

On Wed, Oct 27, 2021 at 04:43:17PM -0400, Paul Gortmaker wrote:
> One of the earlier pre-mainline RCU nocb patchsets had a temporary sysfs
> knob in /sys/devices/system/cpu/cpu*/hotplug/nocb for testing[1].
> 
> That not-for-merge commit from Frederic said:
> 
>   This is only intended for those who want to test this patchset. The
>   real interfaces will be cpuset/isolation and rcutorture.
> 
> We've had rcutorture as the one and only mainline user of nocb toggle
> for a while now[2], and so I thought I'd take a crack at what Frederic
> had in mind for cpuset with some code vs. asking 100 random questions.
> 
> Note that I intentionally didn't Cc any cgroup/cpuset people (yet),
> since at this point this is only my guess on what things were to look
> like based on a single sentence fragment.  So this is really early
> "Not-for-Merge", but truly just RFC -- to start a conversation.
> 
> It won't be really useful until we adjust tick/housekeeping in addition
> to nocb, but I think we can develop the interface in parallel to that?
> And maybe use this to expand testing at the same time if it is layered
> on top of those future work/patchsets?  I don't know...
> 
> We'll also have to look at corner cases - like whether we want to treat
> the root cpuset differently; whether we want to sync boot arg values
> with the cpuset's initial isol flag value, whether we un-isolate cores
> when an isolation cpuset is rmdir/removed, etc etc.
> 
> But as a proof of concept, it "works" as can be seen in the 2nd commit.

I'm working on the same thing :o)

With quite a rework of housekeeping core behind (WIP):

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
	isolation/split

It's not yet ready either and I'm glad you posted this, it shows I'm not
the only one interested in it.

One thing about cpuset: I arranged to implement it only on cgroup v2 and
exclusively mutable on root partition (which doesn't mean only _the_ root
partition but also those whose cpuset.cpus.partition == "root". This way
I make sure the set of cpus is exclusive. I didn't want to bother with
intersecting cpusets with different nocb values.

Thanks.

> Paul.
> --
> 
> Cc: Ingo Molnar <mingo@xxxxxxxxxx>
> Cc: Steven Rostedt <rostedt@xxxxxxxxxxx>
> Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
> Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
> Cc: Josh Triplett <josh@xxxxxxxxxxxxxxxx>
> Cc: Paul E. McKenney <paulmck@xxxxxxxxxx>
> Cc: Frederic Weisbecker <frederic@xxxxxxxxxx>
> Cc: Valentin Schneider <valentin.schneider@xxxxxxx>
> 
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git/commit/?h=rcu/nocb&id=6abe8408307e
>     part of https://lwn.net/Articles/820544/
>             https://lwn.net/Articles/832031/   <------ v2
>             https://lwn.net/Articles/835039/   <------ v3
>             https://lwn.net/Articles/837128/   <------ v4
> 
> [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d97b078182406
> 
> 
> Paul Gortmaker (2):
>   sched: isolation: cpu isolation handles for cpuset
>   cpuset: add binding to CPU isolation
> 
>  include/linux/sched/isolation.h |  4 ++++
>  kernel/cgroup/cpuset.c          | 42 +++++++++++++++++++++++++++++++++++++++++
>  kernel/sched/isolation.c        | 22 +++++++++++++++++++++
>  3 files changed, 68 insertions(+)
> 
> -- 
> 2.15.0
>