[PATCH v3 04/12] x86,fs/resctrl: Program PLZA through kmode arch hooks

[Babu Moger]

AMD Privilege Level Zero Association (PLZA) exposes kernel CLOSID/RMID
association through MSR_IA32_PQR_PLZA_ASSOC.  Generic resctrl already
tracks supported and effective kernel-mode policy in struct
resctrl_kmode_cfg, but the architecture layer needs a callable entry point
that can push those values into per-CPU hardware on a chosen CPU mask.

Declare resctrl_arch_configure_kmode() in linux/resctrl.h with kernel-doc.
Implement it on x86: add an SMP callback that writes
MSR_IA32_PQR_PLZA_ASSOC on each targeted CPU, and use on_each_cpu_mask()
for the broadcast.

The hook is unused in this patch; later patches in the series wire it into
generic resctrl when an effective kernel-mode policy is selected or a CPU
mask changes.

Signed-off-by: Babu Moger <babu.moger@xxxxxxx>
---
v3: Removed task based PLZA implementation so related changes are removed.
    Removed handling of rmid_en as it is not required. The group type assigned
    will be different so the monitoring part is already taken care.
    Updated the change log with details.
    Removed resctrl_arch_set_kmode() as arch only provides the modes supported.
    It is FS which decided which mode to apply.

v2: Updated the commit message to include the sequence of steps to enable PLZA.
    Added mode code comments for clarity.
    Added kmode to functin names to be generic.
---
 arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 35 +++++++++++++++++++++++
 include/linux/resctrl.h                   | 10 +++++++
 2 files changed, 45 insertions(+)

diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
index b20e705606b8..68f1cf503904 100644
--- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
+++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
@@ -131,3 +131,38 @@ int resctrl_arch_io_alloc_enable(struct rdt_resource *r, bool enable)
 
 	return 0;
 }
+
+/*
+ * SMP call-function callback: each CPU writes its own MSR_IA32_PQR_PLZA_ASSOC
+ * (AMD PLZA).  Invoked via on_each_cpu_mask() with wait=1 so the on-stack
+ * union pointed at by @arg is safe.
+ */
+static void resctrl_kmode_set_one_amd(void *arg)
+{
+	union msr_pqr_plza_assoc *plza = arg;
+
+	wrmsrl(MSR_IA32_PQR_PLZA_ASSOC, plza->full);
+}
+
+/**
+ * resctrl_arch_configure_kmode() - x86/AMD: program PLZA MSR on a CPU subset
+ * @cpu_mask:	CPUs to receive the update (see on_each_cpu_mask() for online subset).
+ * @closid:	CLOSID field written into the MSR with CLOSID_EN set.
+ * @rmid:	RMID field written into the MSR with RMID_EN set.
+ * @enable:	Value for the PLZA_EN split field.
+ *
+ * Context: Do not call with IRQs off or from IRQ context except as allowed for
+ * on_each_cpu_mask(); see kernel/smp.c.
+ */
+void resctrl_arch_configure_kmode(cpumask_var_t cpu_mask, u32 closid, u32 rmid, bool enable)
+{
+	union msr_pqr_plza_assoc plza = { 0 };
+
+	plza.split.rmid = rmid;
+	plza.split.rmid_en = 1;
+	plza.split.closid = closid;
+	plza.split.closid_en = 1;
+	plza.split.plza_en = enable;
+
+	on_each_cpu_mask(cpu_mask, resctrl_kmode_set_one_amd, &plza, 1);
+}
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index ce28418df00f..570918e57e24 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -712,6 +712,16 @@ bool resctrl_arch_get_io_alloc_enabled(struct rdt_resource *r);
  */
 void resctrl_arch_get_kmode_support(struct resctrl_kmode_cfg *kcfg);
 
+/**
+ * resctrl_arch_configure_kmode() - Program MSR_IA32_PQR_PLZA_ASSOC on CPUs in @cpu_mask
+ * @cpu_mask:	Target CPUs; on_each_cpu_mask() applies the callback on the online subset.
+ * @closid:	CLOSID written to the MSR with CLOSID_EN set.
+ * @rmid:	RMID written to the MSR with RMID_EN set.
+ * @enable:	PLZA_EN field value for this update.
+ */
+void resctrl_arch_configure_kmode(cpumask_var_t cpu_mask, u32 closid, u32 rmid,
+				  bool enable);
+
 extern unsigned int resctrl_rmid_realloc_threshold;
 extern unsigned int resctrl_rmid_realloc_limit;
 
-- 
2.43.0