Re: [RESEND PATCH v4 11/15] fs/resctrl: Program kernel-mode binding when CPU comes online
From: Babu Moger
Date: Thu Jul 09 2026 - 16:02:28 EST
On 7/7/26 16:50, Babu Moger wrote:
Kernel-mode resctrl associations are programmed per CPU. When a
global-assign kernel-mode policy is selected, all CPUs that are online at
that time are configured; however, CPUs that come online later are not
automatically updated.
As a result, hot-added CPUs, or CPUs that were offline when the policy was
selected, can run without the active kernel-mode association even though
the resctrl policy is still in effect.
Add resctrl_kmode_online_cpu() and call it from resctrl_online_cpu(). When
a global-assign policy is active, add the newly online CPU to the bound
group's kmode_cpu_mask and program the kernel-mode association on that CPU.
Signed-off-by: Babu Moger <babu.moger@xxxxxxx>
---
v4: New patch in the series. Patch taken from:
https://lore.kernel.org/lkml/20260611111706.1981788-5-qinyuntan@xxxxxxxxxxxxxxxxx/
Updated the code to enable kernel-mode by default whenever a CPU comes online when
one of global-assign mode is enabled.
---
fs/resctrl/rdtgroup.c | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 21659fd75850..413d3ff14546 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -4722,11 +4722,40 @@ int resctrl_online_mon_domain(struct rdt_resource *r, struct rdt_domain_hdr *hdr
return err;
}
+/*
+ * resctrl_kmode_online_cpu() - Configure kernel-mode binding for a CPU that
+ * comes online
+ * @cpu: CPU that has just been brought online.
+ *
+ * Kernel-mode associations are maintained per CPU. When a CPU becomes
+ * online after a global-assign policy has been applied, it is updated
+ * with the current kernel-mode configuration. The CPU is then added to
+ * the bound group's kmode_cpu_mask, and the appropriate association is
+ * programmed for it.
+ */
+static void resctrl_kmode_online_cpu(unsigned int cpu)
+{
+ struct rdtgroup *rdtgrp = resctrl_kcfg.k_rdtgrp;
+ bool assign_mon = false;
+
+ if (resctrl_kcfg.kmode_cur == INHERIT_CTRL_AND_MON || !rdtgrp)
+ return;
+
+ assign_mon = (resctrl_kcfg.kmode_cur == GLOBAL_ASSIGN_CTRL_ASSIGN_MON_PER_CPU);
+
+ cpumask_set_cpu(cpu, &rdtgrp->kmode_cpu_mask);
+
+ resctrl_arch_configure_kmode(cpumask_of(cpu), rdtgrp->closid, rdtgrp->mon.rmid,
+ assign_mon, true);
+}
+
void resctrl_online_cpu(unsigned int cpu)
{
mutex_lock(&rdtgroup_mutex);
/* The CPU is set in default rdtgroup after online. */
cpumask_set_cpu(cpu, &rdtgroup_default.cpu_mask);
+ /* Program any active kernel-mode binding on this CPU. */
+ resctrl_kmode_online_cpu(cpu);
mutex_unlock(&rdtgroup_mutex);
}
Sashiko reported:
https://sashiko.dev/#/patchset/cover.1783461016.git.babu.moger%40amd.com?part=11
--- Patch [11]: [RESEND PATCH v4 11/15] fs/resctrl: Program kernel-mode binding when CPU comes online ---
commit ab44f1490e3dbe9a24a0baa6c7b98b7456c62c03
Author: Babu Moger <babu.moger@xxxxxxx>
fs/resctrl: Program kernel-mode binding when CPU comes online
Kernel-mode resctrl associations are programmed per CPU. When a global-assign
kernel-mode policy is selected, all CPUs that are online at that time are
configured; however, CPUs that come online later are not automatically
updated.
Link: https://lore.kernel.org/lkml/20260611111706.1981788-5-qinyuntan@xxxxxxxxxxxxxxxxx/
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 21659fd75850e..413d3ff14546b 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
[ ... ]
> @@ -4722,11 +4722,40 @@ int resctrl_online_mon_domain(struct rdt_resource *r, struct rdt_domain_hdr *hdr
> return err;
> }
>
> +/*
> + * resctrl_kmode_online_cpu() - Configure kernel-mode binding for a CPU that
> + * comes online
> + * @cpu: CPU that has just been brought online.
> + *
> + * Kernel-mode associations are maintained per CPU. When a CPU becomes
> + * online after a global-assign policy has been applied, it is updated
> + * with the current kernel-mode configuration. The CPU is then added to
> + * the bound group's kmode_cpu_mask, and the appropriate association is
> + * programmed for it.
> + */
> +static void resctrl_kmode_online_cpu(unsigned int cpu)
> +{
> + struct rdtgroup *rdtgrp = resctrl_kcfg.k_rdtgrp;
> + bool assign_mon = false;
> +
> + if (resctrl_kcfg.kmode_cur == INHERIT_CTRL_AND_MON || !rdtgrp)
> + return;
If a CPU goes offline while a kernel-mode policy is active, and the policy is subsequently disabled (reverting to INHERIT_CTRL_AND_MON), does returning early here leave the newly onlined CPU with a stale kernel-mode resctrl association?
Looking at clear_closid_rmid(), it does not reset MSR_IA32_PQR_PLZA_ASSOC during CPU online initialization. Could this lead to the CPU executing CPL0 traffic with an invalid or malicious cache allocation if the stale CLOSID/RMID is reallocated?
--------------------------------------------------------------------
Yes. This is an issue.
Need to clear the kmode_cpu_list if the cpu being offlined is part of
the current kmode. Also, need to disable PLZA on the CPU that is offlined.
Will take care of this in resctrl_offline_cpu().
-------------------------------------------------------------------
> +
> + assign_mon = (resctrl_kcfg.kmode_cur == GLOBAL_ASSIGN_CTRL_ASSIGN_MON_PER_CPU);
> +
> + cpumask_set_cpu(cpu, &rdtgrp->kmode_cpu_mask);
Since the CPU is added to kmode_cpu_mask here when it comes online, shouldn't it be removed when it goes offline?
Looking at resctrl_offline_cpu(), it clears the CPU from rdtgrp->cpu_mask:
if (cpumask_test_and_clear_cpu(cpu, &rdtgrp->cpu_mask)) {
clear_childcpus(rdtgrp, cpu);
break;
}
But there is no corresponding logic to clear it from kmode_cpu_mask. Does this cause stale state to be exposed to userspace via the kmode_cpus file?
----------------------------------------------------------------------
Ditto: Will take care of this in resctrl_offline_cpu().
----------------------------------------------------------------------
Is it possible for a MON group that is actively bound to a kernel-mode policy to be reparented?
Looking at rdtgroup_rename(), it allows reparenting if cpu_mask is empty
without checking kmode_cpu_mask:
if (!cpumask_empty(&rdtgrp->cpu_mask)) {
ret = -EINVAL;
goto out;
}
Then mongrp_reparent() updates the closid but fails to reprogram PLZA MSRs for CPUs in kmode_cpu_mask:
rdtgrp->mon.parent = new_prdtgrp;
rdtgrp->closid = new_prdtgrp->closid;
/* Propagate updated closid to all tasks in this group. */
rdt_move_group_tasks(rdtgrp, rdtgrp, cpus);
update_closid_rmid(cpus, NULL);
Does this leave CPUs in kmode_cpu_mask running with the old parent's CLOSID?
-----------------------------------------------------------------------
I don't see a reason to reparent the active kernel-mode group.
I'll add a check for:
if (!cpumask_empty(&rdtgrp->kmode_cpu_mask))
Thanks
Babu