Re: [PATCH v4 1/6] x86/resctrl: Parse ACPI ERDT table and map RMDD domains by L3 cache ID
From: Chen, Yu C
Date: Wed Jun 24 2026 - 00:05:59 EST
On 6/24/2026 12:46 AM, Reinette Chatre wrote:
Hi Chenyu,
On 6/23/26 4:43 AM, Chen, Yu C wrote:
On 6/23/2026 2:29 PM, Chen, Yu C wrote:
From what I can tell it is tmp that is essentially returned by this function and it
is the result of get_cpu_cacheinfo_id() that returns the cache ID initialized from
CPUID leaf 0x4 ... the function name ("get_l3_cache_id_from_cacd()") and function comment
"Resolve L3 cache ID from CACD subtable" implies differently and the way this is done
makes me wonder why CACD needs to be parsed at all.
resctrl already has a hotplug handler and it dynamically determines the domain ID
based on the cache ID returned from get_cpu_cacheinfo_id(). If the domain ID used by the
new ERDT tables can be, as it appears is done here, trusted to what Linux already
treating as domain ID then why is it needed to parse CACD at all? Why not just
pick the domain ID from RMDD directly and use it directly as domain ID?
Later the domain ID from RMDD is just used in error messages ... never actually used
as a domain ID to guide the implementation so the correlation appears to be implicit with
this CACD parsing not intuitive.
You are right that the cache ID ultimately comes from get_cpu_cacheinfo_id(),
not from CACD directly - the function name is misleading and should be improved.
However, my understanding is that CACD parsing acts as the bridge between ACPI's
RMDD domain namespace and the kernel's L3 domain ID: CACD enumerates which CPUs
belong to each RMDD, and from those CPUs we derive the kernel-side L3 cache ID.
The specification defines an RMDD DomainID as an identifier unique within the
ERDT table, yet it does not guarantee that this ID matches the kernel’s cache ID
obtained via CPUID leaf 0x4. For this reason, we omitted the RMDD domain ID in
the current release.
get_l3_cache_id_from_cacd() implies a straightforward query and the function
comments supports this. At the same time the function includes what appears to be
sanity checks but from what you say turns out to be essential enforcement of the
RMDD to CPUID leaf 0x4 mapping. Is this correct? As sanity checks these additional
checks are ok but as RMDD to CPUID leaf 0x4 mapping enforcement I find them inadequate
because its correctness requires all CPUs of all domains to be online which cannot be
guaranteed when this initialization runs.
Consider for example a scenario where RMDD and CPUID do not agree on which CPUs form
part of a domain. If only one CPU of this RMDD domain is online at the time this
initialization is done then the mismatch will never be noticed and resctrl will just
silently use the CPUID mapping, no?
I suppose you are referring to the following scenario:
CACD says CPUs {0,1,2,3} belong to RMDD-A, and CPUID says
CPUs {0,1} share L3-X while CPUs {2,3} share L3-Y (a mismatch).
If only CPU 0 is brought online:
get_l3_cache_id_from_cacd() iterates CACD's X2APIC list: {0,1,2,3}
CPU 1, 2, 3 are offline -> skipped
CPU 0 is online -> cache_id = get_cpu_cacheinfo_id(0) = L3-X,
all passed.
Then later CPU2 is online, resctrl's hotplug creates domain L3-Y.
But the xarray has no entry for L3-Y, so erdt_mon_read() will return
-EIO for that domain.
An error would at least give indication of a problem but I also see a possible
scenario where the data will happily be read from the wrong register. Consider the
following change to the scenario you present:
CACD: RMDD-A = {0, 2}; RMDD-B = {1, 3}
CPUID: L3-X = {0, 1}; L3-Y = {2, 3}
If ACPI enumeration is done with CPUs 1, 2, and 3 offline then they are skipped.
CPU 0 is online -> cache_id = get_cpu_cacheinfo_id(0) = L3-X
Later when CPU 1 comes online resctrl will consider it to be in L3-X and
also find an xarray entry for L3-X which will result in erdt_mon_read() to
read the RMID data from RMDD-A instead of RMDD-B?
Yes, this approach is feasible. However, I am uncertain whether RDT should
be enabled. The underlying issue is that the firmware(BIOS) and silicon(CPUID)
will be out of sync, leaving RDT in an unstable state.
When considering the example scenario above I do not believe that this implements a
If this is the case, I wonder if we could run the sanity check during
CPU hotplug to reparse the CACD table whenever a CPU comes online. My
thought would be to call get_l3_cache_id_from_cacd() within
resctrl_arch_online_cpu().
In fact, parsing the CACD table during CPU hotplug may not be feasible,
as the ACPI tables could have already been released. We could instead
perform the check directly against the erdt_domain_xa xarray. If a
mismatch is detected, disable ERDT and fall back to legacy MSR-based
access:
/* Called from resctrl_arch_online_cpu() */
void erdt_verify_cpu_domain(int cpu)
{
int cache_id;
if (!__erdt_enabled)
return;
cache_id = get_cpu_cacheinfo_id(cpu, RESCTRL_L3_CACHE);
if (cache_id < 0)
return;
if (!xa_load(&erdt_domain_xa, cache_id)) {
pr_warn(FW_BUG "CPU%d L3 domain %d not described by ERDT, disabling ERDT\n",
cpu, cache_id);
__erdt_enabled = false;
}
}
robust transition from ACPI to CPUID.
Additionally, disabling ERDT during a CPU online has broader implications: if ERDT was
enabled earlier then the associated events will be enabled at this point and have
mov_evt::any_cpu set. From what I can tell this would just implicitly expect that the
events could still be read via MSR but not actually route the read to the correct CPU?
I think it would be simpler to separate the ACPI enumeration from the CPUID enumeration
instead of wedging in these snippets at various touch points. The ACPI enumeration could,
for example, just record the ERDT RMDD domain <-> cpu_mask mapping without taking online CPUs
into account. Depending on which tables are found in ACPI, if ERDT is the intended mechanism then
any duplicate CPUID enumeration should be avoided. This means that there will be two unique
flows to enabling resctrl events, one for ACPI and one for CPUID. The x86 online handler can
perform the sanity check when a CPU comes online and since RMDD and CPUID are expected to
correlate the CPU could not be added to resctrl if it is not in the expected domain. To me
this seems simpler and more robust.
Thanks for the detailed feedback. If I understand correctly, separating
ACPI enumeration from CPUID enumeration would look something like the following.
Could you let me know if this approach is feasible?
----------
1. At ACPI parse time , record the full CPU mask from CACD for each
RMDD domain without checking online state or resolving L3 cache IDs.
Store a per-CPU pointer (erdt_cpu_domain[cpu]) for O(1) lookup.
2. At CPU online time (resctrl_arch_online_cpu), validate that the CPU's
CPUID-derived L3 ID is consistent with other CPUs in the same RMDD:
- First online CPU in an RMDD establishes the L3 ID for that domain
and inserts the domain into erdt_domain_xa keyed by L3 cache ID.
- Subsequent CPUs must match. On mismatch, the CPU is refused from resctrl.
3. erdt_mon_read() looks up erdt_domain_xa by hdr->id (L3 cache ID) as before,
but the xarray is now populated lazily at CPU online time.
Pseudocode:
/* ACPI parse time: store full CPU mask, no online check */
cacd_record_cpus(cacd, domain_info):
for each x2apic_id in cacd:
cpu = topo_lookup_cpuid(x2apic_id)
cpumask_set_cpu(cpu, domain_info->cpu_mask)
erdt_cpu_domain[cpu] = domain_info
domain_info->l3_id = -1 /* unknown until first CPU comes online */
/* CPU online time via resctrl_arch_online_cpu():
* validate and establish real L3 mapping
*/
erdt_verify_cpu_domain(cpu):
dom = erdt_cpu_domain[cpu]
if (!dom)
return -EINVAL /* CPU not in any ERDT domain */
cpuid_l3_id = get_cpu_cacheinfo_id(cpu)
if (dom->l3_id == -1) {
dom->l3_id = cpuid_l3_id
xa_store(&erdt_domain_xa, cpuid_l3_id, dom)
} else if (dom->l3_id != cpuid_l3_id) {
return -EINVAL /* mismatch: refuse CPU */
}
return 0
thanks,
Chenyu