[PATCH v5 05/16] x86,fs/resctrl: Introduce architecture hooks to program kernel mode

[Babu Moger]

Kernel modes defined by enum resctrl_kernel_mode must be applied on
the CPUs when user space activates, deactivates, or updates a
configuration.

Generic resctrl has no architecture hook to apply these modes across
a CPU mask when the active mode changes.

Add resctrl_arch_configure_kmode() to program kernel mode allocation and
monitoring associations on @cpu_mask. Accept separate assign_ctrl and
assign_mon parameters so CLOSID and RMID can be assigned independently.

Implement the x86 hook to program per-CPU PLZA settings. On x86, PLZA
programs these associations per CPU, with CLOSID and RMID configured
independently.

Provide an MPAM stub so the filesystem layer can call the hook on systems
without PLZA.

Signed-off-by: Babu Moger <babu.moger@xxxxxxx>
---
v5: Fix kernel-doc for resctrl_arch_configure_kmode(), wire assign_ctrl
    through to PLZA closid_en, and update the changelog.
    Add calling context in API doc.

v4: Added assign_mon parameter in resctrl_arch_configure_kmode() to program the RMID
    as discussed in below.
    https://lore.kernel.org/lkml/20260605100642.1103628-1-qinyuntan@xxxxxxxxxxxxxxxxx/
    Reintroduce independent monitoring assignment via assign_mon after removing
    the task-based PLZA approach in v3.
    Changed cpumask type to "const struct cpumask *cpu_mask".
    Added MPAM stub to avoid any linking issues when resctrl_arch_configure_kmode()
    is called from FS layer. Thanks to Qinyun.
    Re-wrote the changelog to be generic.
    Updated code comments.
v3: Removed task based PLZA implementation so related changes are removed.
    Per-task RMID handling is replaced by group type selection in the fs layer.
    Updated the change log with details.
    Removed resctrl_arch_set_kmode() as arch only provides the modes supported.
    It is FS which decided which mode to apply.
v2: Updated the commit message to include the sequence of steps to enable PLZA.
    Added mode code comments for clarity.
    Added kmode to function names to be generic.
---
 arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 38 +++++++++++++++++++++++
 drivers/resctrl/mpam_resctrl.c            |  6 ++++
 include/linux/resctrl.h                   | 33 ++++++++++++++++++++
 3 files changed, 77 insertions(+)

diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
index e74f1ed54b86..40fd5e31c94e 100644
--- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
+++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
@@ -131,3 +131,41 @@ int resctrl_arch_io_alloc_enable(struct rdt_resource *r, bool enable)
 
 	return 0;
 }
+
+static void resctrl_kmode_set_one_amd(void *arg)
+{
+	union msr_pqr_plza_assoc *plza = arg;
+
+	wrmsrq(MSR_IA32_PQR_PLZA_ASSOC, plza->full);
+}
+
+/*
+ * Program Privilege Level Zero Association (PLZA) on @cpu_mask.
+ *
+ * When @enable is true, kernel mode allocation on @cpu_mask uses @closid from
+ * MSR_IA32_PQR_PLZA_ASSOC if @assign_ctrl is true, otherwise the CLOSID from
+ * MSR_IA32_PQR_ASSOC. Kernel mode monitoring uses @rmid from
+ * MSR_IA32_PQR_PLZA_ASSOC if @assign_mon is true, otherwise the RMID of the
+ * current task.
+ *
+ * @cpu_mask:	CPUs whose PLZA MSR should be updated.
+ * @closid:	CLOSID to use for kernel mode allocation when @assign_ctrl is true.
+ * @assign_ctrl: Whether PLZA should provide the kernel mode CLOSID.
+ * @rmid:	RMID to use for kernel mode monitoring when @assign_mon is true.
+ * @assign_mon: Whether PLZA should provide the kernel mode RMID.
+ * @enable:	Whether PLZA should provide the kernel mode association.
+ */
+void resctrl_arch_configure_kmode(const struct cpumask *cpu_mask, u32 closid,
+				  bool assign_ctrl, u32 rmid,
+				  bool assign_mon, bool enable)
+{
+	union msr_pqr_plza_assoc plza = { 0 };
+
+	plza.split.rmid = rmid;
+	plza.split.rmid_en = assign_mon;
+	plza.split.closid = closid;
+	plza.split.closid_en = assign_ctrl;
+	plza.split.plza_en = enable;
+
+	on_each_cpu_mask(cpu_mask, resctrl_kmode_set_one_amd, &plza, 1);
+}
diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
index 9d223057953a..286284ac8423 100644
--- a/drivers/resctrl/mpam_resctrl.c
+++ b/drivers/resctrl/mpam_resctrl.c
@@ -139,6 +139,12 @@ bool resctrl_arch_get_io_alloc_enabled(struct rdt_resource *r)
 	return false;
 }
 
+void resctrl_arch_configure_kmode(const struct cpumask *cpu_mask, u32 closid,
+				  bool assign_ctrl, u32 rmid, bool assign_mon,
+				  bool enable)
+{
+}
+
 void resctrl_arch_pre_mount(void)
 {
 }
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index 4245d1e65ccc..8b30eef835aa 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -729,6 +729,39 @@ enum resctrl_kernel_mode {
 
 #define RESCTRL_NUM_KERNEL_MODES (RESCTRL_KMODE_LAST + 1)
 
+/**
+ * resctrl_arch_configure_kmode() - Program kernel mode association
+ * @cpu_mask:	CPUs to assign the kernel mode on.
+ * @closid:	CLOSID that matches the RMID to program kernel mode. Depending
+ *		on the architecture, the counter may match traffic of both
+ *		@closid and @rmid, or @rmid only.
+ * @assign_ctrl: true to assign @closid for kernel mode; false to inherit
+ *		association from the user-space task.
+ * @rmid:	RMID to program the kernel mode. Some architectures may use
+ *		CLOSID/RMID separately, others will consider them together.
+ * @assign_mon:	true to assign @rmid for kernel mode; false to inherit
+ *		monitoring association from the user-space task.
+ * @enable:	true to enable kernel mode association on CPUs in @cpu_mask;
+ *		false to disable kernel mode.
+ *
+ * The function can be called in the following scenarios:
+ * - If a per-cpu kernel mode is active when user space switches to a new
+ *   per-cpu kernel mode then resctrl_arch_configure_kmode() will first be
+ *   called to de-activate the active kernel mode on all CPUs that the
+ *   kernel mode is active on.
+ * - When user space switches to a new per-cpu kernel mode then
+ *   resctrl_arch_configure_kmode() is called with cpu_online_mask.
+ * - When user space adds a CPU to an active per-cpu kernel mode.
+ * - When user space removes a CPU from an active per-cpu kernel mode.
+ * - resctrl fs will always provide the same closid, assign_ctrl, rmid,
+ *   and assign_mon parameters when activating a kernel mode, all
+ *   interactions (adding/removing CPU) while the kernel mode is active,
+ *   as well as when de-activating the kernel mode.
+ */
+void resctrl_arch_configure_kmode(const struct cpumask *cpu_mask, u32 closid,
+				  bool assign_ctrl, u32 rmid, bool assign_mon,
+				  bool enable);
+
 extern unsigned int resctrl_rmid_realloc_threshold;
 extern unsigned int resctrl_rmid_realloc_limit;
 
-- 
2.43.0