Re: [PATCH v3 00/12] [PATCH v3 00/12] x86/resctrl: Add kernel-mode (e.g., PLZA) support to the resctrl subsystem

[Babu Moger]

Hi Reinette,

On 6/16/26 23:34, Reinette Chatre wrote:
> Hi Babu,
>
> On 6/12/26 8:37 AM, Moger, Babu wrote:
> > On 6/11/2026 4:53 PM, Reinette Chatre wrote:
> > > Hi Babu,
> > >
> > > On 4/30/26 4:24 PM, Babu Moger wrote:
> > > > Design
> > > > ======
> > > >
> > > > A new sysfs file, info/kernel_mode, holds a single global policy that
> > > > selects what kernel work is steered and which rdtgroup it is steered
> > >
> > > How should "selects *what* kernel work is steered" be interpreted? Do these
> > > modes not all apply to *all* kernel work?
> >
> > How about?
> >
> > A new sysfs file, info/kernel_mode, holds a single global policy for
> > kernel contexts and the rdtgroup associated with the policy.
>
> What does "kernel contexts" mean here?
> Also, since rdtgroup refers more to "struct rdtgroup" that is internal to resctrl
> I would suggest "resource group" be used instead.
> Consider, for example (this is just something to get started, please do not just
> copy&paste):
>
>     A new resctrl file, info/kernel_mode, holds the global policy for
>     resource allocation and monitoring of kernel work and the resource group
>     (when applicable) associated with the policy.

Sounds good.

> > > > to.  Reads describe the supported modes and the currently-active
> > > > binding; writes change the policy or rebind to a different group.
> > > > Look at the thread below for design discussion.
> > > > https://lore.kernel.org/lkml/14a8ad0a-e842-4268-871a-0762f1169e03@xxxxxxxxx/
> > >
> > > ...
> > >
> > > > Examples
> > > > ========
> > > >
> > > > (See Documentation/filesystems/resctrl.rst, "kernel_mode" and
> > > > "kmode_cpus" sections, for the full UAPI.)
> > > >
> > > >     # Mount resctrl
> > > >     # mount -t resctrl resctrl /sys/fs/resctrl
> > > >     # cd /sys/fs/resctrl
> > > >
> > > >     # Read the supported modes.  The active mode is bracketed and reports
> > > >     # the bound "<ctrl>/<mon>/" group; other supported modes report
> > > >     # ":group=none" because nothing is bound to them.
> > > >     # cat info/kernel_mode
> > > >     [inherit_ctrl_and_mon:group=//]
> > >
> > > This is unexpected since associating a group to this mode implies that this
> > > group is used to manage allocations and monitoring of kernel work but this
> > > is not true, right? From what I understand there should be no group associated with
> > > this default "inherit_ctrl_and_mon" mode.
> >
> > The default mode is "inherit_ctrl_and_mon", where both user mode and kernel mode share the same CLOSID and RMID. This is current mode (without this series).
> >
> > I thought we are going to set the default mode with the default group when system boots up. No?
>
> As I have it, the end of our discussion on this topic is at:
> https://lore.kernel.org/lkml/6709398b-269d-47b5-9b41-084f410bb1a6@xxxxxxx/
>
> Based on that discussion there is no resource group associated with the default
> inherit_ctrl_and_mon.
> I find the above output confusing to user space since adding the default group as the only
> group to this mode implies that kernel work inherits resource allocation and monitoring
> from the default group but that is not correct.

Ah, I misunderstood earlier.

The display will look like this when the system boots up.

# cat info/kernel_mode
  inherit_ctrl_and_mon:
  global_assign_ctrl_assign_mon_per_cpu:group=uninitialized
  global_assign_ctrl_assign_mon_per_cpu:group=uninitialized

There will not be any group associated with "inherit_ctrl_and_mon".
It is only used to switch from other two modes.

> Your answer seems to refer to other discussions about what group should be used for
> a mode when switching to a new mode and user space has not set the resource group. If not,
> could you please point me to which discussion you are referring to?
>
> > > >     global_assign_ctrl_inherit_mon_per_cpu:group=none
> > > >     global_assign_ctrl_assign_mon_per_cpu:group=none
> > >
> > > nit: "none" does not reflect state as clearly as "unset"/"uninitialized"/"NA"
> >
> > Lets go with "uninitialized".
>
> ok
>
> ...
> > > >     # echo 0-3 > ctrl1/kmode_cpus_list
> > > >     # cat ctrl1/kmode_cpus
> > > >     f
> > > >     # cat ctrl1/kmode_cpus_list
> > > >     0-3
> > > >
> > > >     # Empty masks are rejected; use info/kernel_mode to reset to
> > > >     # "every online CPU".
> > > >     # echo "" > ctrl1/kmode_cpus_list
> > > >     bash: echo: write error: Invalid argument
> > > >     # cat info/last_cmd_status
> > > >     Empty mask not allowed; use info/kernel_mode to unbind
> > >
> > > Why are empty masks rejected/not allowed?
> >
> > No specific reason.
> >
> > When the mode is switched, we discussed earlier to globally apply the mode to all the online CPUs.
>
> Right. I did not see this being done in this implementation though. As I mentioned in
> my response to patch #8 it appears that it uses old data from the resource
> group's kmode_cpu_mask. I do think that applying it to all the online CPUs
> matches the intention and would make the code much simpler.

Yes. Sure.

> > At this point reading "kmode_cpus_list" will still report empty.
>
> I do not think resctrl should do this. This is not accurate and conflicts with the
> existing cpus resctrl files. It should be simple to just present the actual and
> accurate data to user space, especially after incorporating Qinyun Tan's contributions.

Sure. Will do

> > Users can change it to selectively apply the mode by writing to "kmode_cpus_list".
> >
> > I was not sure what was the action when empty masks are written.
> >
> > Should the empty mask apply the mode to all the online CPUs?
>
> Users are used to being able to use an empty write to remove all CPUs from a
> resource group. It thus seems intuitive that an empty write to the kmode_cpus file

Ok, That seems reasonable. Lets keep the same behavior as we update 
other cpu_lists.

> behave similarly. Sounds like this could mean that if user space sets the
> kmode to global_assign_ctrl_inherit_mon_per_cpu or global_assign_ctrl_assign_mon_per_cpu
> and then writes an empty mask to kmode_cpus then it would essentially be setting
> inherit_ctrl_and_mon mode? This still seems ok since if disabling one of the "global"
> modes on a CPU results in that CPU inheriting from PQR_ASSOC then it seems
> reasonable to extend to when that mode is disabled for all CPUs.

Yes. That is correct.

> > > >     # Disable kernel-mode steering (back to inherit, default group).
> > >
> > > This sounds like kernel work is steered to default group which I
> > > do not think is accurate for the "inherit_ctrl_and_mon" mode.
> >
> > How about ?
> >
> > Drop the kernel-mode binding and restore inherit_ctrl_and_mon on the default group.
>
> No. There is no "inherit_ctrl_and_mon on the default group". There is nothing special
> about the default group when it comes to inherit_ctrl_and_mon mode ... or am I missing
> something?

You are right. There is nothing special about inherit_ctrl_and_mon on 
the default group.

> This could be something like: "Activate inherit_ctrl_and_mon mode to let
> kernel work inherit the resource allocation and monitoring from the user space task."
> (and drop the default group from the output associated with inherit_ctrl_and_mon)

Sure.

Thanks
Babu