Re: [PATCH v3 04/12] x86,fs/resctrl: Program PLZA through kmode arch hooks

[Peter Newman]

Hi Babu,

On Thu, May 21, 2026 at 1:09 AM Moger, Babu <bmoger@xxxxxxx> wrote:
>
> Hi Tony,
>
> On 5/20/2026 5:16 PM, Luck, Tony wrote:
> > On Wed, May 20, 2026 at 12:49:25PM -0500, Babu Moger wrote:
> >> Hi Tony,
> >>
> >>
> >> On 5/19/26 15:59, Luck, Tony wrote:
> >>> On Thu, Apr 30, 2026 at 06:24:49PM -0500, Babu Moger wrote:
> >>>> +void resctrl_arch_configure_kmode(cpumask_var_t cpu_mask, u32 closid, u32 rmid, bool enable)
> >>>> +{
> >>>> +  union msr_pqr_plza_assoc plza = { 0 };
> >>>> +
> >>>> +  plza.split.rmid = rmid;
> >>>> +  plza.split.rmid_en = 1;
> >>>
> >>> Shouldn't there be a parameter for the value of rmid_en?
> >>
> >>
> >> I realized that behavior is not required—it was actually due to a mistake in
> >> my v2 series implementation.

Really? This is in fact the only behavior we wanted:

https://lore.kernel.org/lkml/CABPqkBSq=cgn-am4qorA_VN0vsbpbfDePSi7gubicpROB1=djw@xxxxxxxxxxxxxx/

-Peter


> >>
> >> Below are the relevant definitions:
> >>
> >>
> >> GLOBAL_ASSIGN_CTRL_INHERIT_MON_PER_CPU:
> >> The CLOSID is applied to kernel work, while the RMID used for monitoring is
> >> inherited from the currently running user task.
> >> No separate monitoring group is assigned for kernel work, so kernel
> >> execution naturally inherits the user-space RMID.
> >>
> >>
> >> GLOBAL_ASSIGN_CTRL_ASSIGN_MON_PER_CPU:
> >> Both CLOSID and RMID are explicitly assigned to kernel work.
> >> This allows assigning a dedicated monitoring group for kernel execution and
> >> therefore requires a separate RMID.
> >>
> >> Example: For GLOBAL_ASSIGN_CTRL_INHERIT_MON_PER_CPU:
> >>
> >> # mount -t resctrl resctrl /sys/fs/resctrl
> >>
> >> # cat /sys/fs/resctrl/info/kernel_mode
> >> [inherit_ctrl_and_mon:group=//]
> >> global_assign_ctrl_inherit_mon_per_cpu:group=none
> >> global_assign_ctrl_assign_mon_per_cpu:group=none
> >>
> >> # mkdir /sys/fs/resctrl/ctrl1   (PQR_ASSOC closid=1 rmid=1)
> >>
> >> This configures all the CPU threads to use closid=1 and rmid=1 for both
> >> allocation and monitoring across user and kernel modes.
> >>
> >>
> >> # echo "global_assign_ctrl_inherit_mon_per_cpu:group=ctrl1//" \
> >>    > /sys/fs/resctrl/info/kernel_mode
> >>
> >> # cat /sys/fs/resctrl/info/kernel_mode
> >> inherit_ctrl_and_mon:group=none
> >> [global_assign_ctrl_inherit_mon_per_cpu:group=ctrl1//]
> >> global_assign_ctrl_assign_mon_per_cpu:group=none
> >>
> >> This overrides the previous configuration, and PQR_PLZA_ASSOC is written.
> >>
> >> Possible options:
> >>
> >> 1. (closid=1, rmid_en=0, rmid=1)
> >> Here, hardware uses closid=1 for kernel work, but RMID tracking is disabled
> >> for kernel mode.
> >>
> >> As a result, reading RMID 1 reports only user-mode activity
> >> This contradicts the definition of this mode, since kernel work is expected
> >> to inherit the user RMID for monitoring.
> >>
> >> 2. (closid=1, rmid_en=1, rmid=1)
> >> In this case, RMID tracking is enabled for both user and kernel modes.
> >>
> >> Reading RMID 1 reports combined user + kernel activity
> >> This aligns with the expected inherit_monitoring behavior
> >>
> >>
> >> The preferred approach is to separate kernel monitoring by assigning it a
> >> dedicated monitoring group and updating PQR_PLZA_ASSOC to use a different
> >> RMID (e.g., closid=1, rmid_en=1, rmid=2). This is exactly the behavior
> >> implemented by GLOBAL_ASSIGN_CTRL_ASSIGN_MON_PER_CPU.
> >
> > So maybe I'm just confused by the name "global_assign_ctrl_inherit_mon_per_cpu"
> >
> > That sounds like "Use the CLOSID from PLZA, but keep the RMID from
> > legacy PQR_ASSOC.
>
> Yes. That is correct. We need to work on naming this correctly.
>
> >
> > So:
> >
> > # mkdir ctrl1 # maybe gets CLOSID=1, RMID=1
> > # echo global_assign_ctrl_inherit_mon_per_cpu:group=ctrl1//" > info/kernel_mode
>
> This makes kernel mode run with CLOSID 1 and RMID 1(Use the same RMID as
> the user mode). [1]
>
> > # mkdir ctrl2 # maybe gets CLOSID=2, RMID=2
> > # echo $$ > ctrl2/tasks
> >
> > My shell, and all children run with CLOSID=2 and RMID=2 from ctrl2. But
> > when they do system calls, take page faults or there is an interrupt I'd
> > expect the code in the kernel to run with the CLOSID=1, while inheriting
> > RMID=2.
>
> ctrl2 is not a PLZA group. So, RMID 2 is not connected to PLZA.
> >
> > To make that happen, I thing the PLZA MSR should have rmid_en = 0. But
> > the only code I see that sets this always sets rmid_en=1.
>
> Setting rmid_en = 0 in [1] disables counting of kernel usage for RMID 1
> (from ctrl1).
>
> The key difference between the two modes is:
>
> In one mode, user and kernel usage are counted together.
> In the other mode, kernel usage is counted separately from user usage.
>
> Please feel free to continue the discussion if anything is still unclear.
>
>
> Thanks,
> Babu
>