Re: [PATCH v4 1/6] x86/resctrl: Parse ACPI ERDT table and map RMDD domains by L3 cache ID

[Reinette Chatre]

Hi Chenyu,

On 6/23/26 4:43 AM, Chen, Yu C wrote:
> On 6/23/2026 2:29 PM, Chen, Yu C wrote:
>>>>>   From what I can tell it is tmp that is essentially returned by this function and it
>>>>> is the result of get_cpu_cacheinfo_id() that returns the cache ID initialized from
>>>>> CPUID leaf 0x4 ... the function name ("get_l3_cache_id_from_cacd()") and function comment
>>>>> "Resolve L3 cache ID from CACD subtable" implies differently and the way this is done
>>>>> makes me wonder why CACD needs to be parsed at all.
>>>>>
>>>>> resctrl already has a hotplug handler and it dynamically determines the domain ID
>>>>> based on the cache ID returned from get_cpu_cacheinfo_id(). If the domain ID used by the
>>>>> new ERDT tables can be, as it appears is done here, trusted to what Linux already
>>>>> treating as domain ID then why is it needed to parse CACD at all? Why not just
>>>>> pick the domain ID from RMDD directly and use it directly as domain ID?
>>>>>
>>>>> Later the domain ID from RMDD is just used in error messages ... never actually used
>>>>> as a domain ID to guide the implementation so the correlation appears to be implicit with
>>>>> this CACD parsing not intuitive.
>>>>>
>>>>
>>>> You are right that the cache ID ultimately comes from get_cpu_cacheinfo_id(),
>>>> not from CACD directly - the function name is misleading and should be improved.
>>>>
>>>> However, my understanding is that CACD parsing acts as the bridge between ACPI's
>>>> RMDD domain namespace and the kernel's L3 domain ID: CACD enumerates which CPUs
>>>> belong to each RMDD, and from those CPUs we derive the kernel-side L3 cache ID.
>>>> The specification defines an RMDD DomainID as an identifier unique within the
>>>> ERDT table, yet it does not guarantee that this ID matches the kernel’s cache ID
>>>> obtained via CPUID leaf 0x4. For this reason, we omitted the RMDD domain ID in
>>>> the current release.
>>>
>>> get_l3_cache_id_from_cacd() implies a straightforward query and the function
>>> comments supports this. At the same time the function includes what appears to be
>>> sanity checks but from what you say turns out to be essential enforcement of the
>>> RMDD to CPUID leaf 0x4 mapping. Is this correct? As sanity checks these additional
>>> checks are ok but as RMDD to CPUID leaf 0x4 mapping enforcement I find them inadequate
>>> because its correctness requires all CPUs of all domains to be online which cannot be
>>> guaranteed when this initialization runs.
>>>
>>> Consider for example a scenario where RMDD and CPUID do not agree on which CPUs form
>>> part of a domain. If only one CPU of this RMDD domain is online at the time this
>>> initialization is done then the mismatch will never be noticed and resctrl will just
>>> silently use the CPUID mapping, no?
>>>
>>
>> I suppose you are referring to the following scenario:
>> CACD says CPUs {0,1,2,3} belong to RMDD-A, and CPUID says
>> CPUs {0,1} share L3-X while CPUs {2,3} share L3-Y (a mismatch).
>> If only CPU 0 is brought online:
>>
>> get_l3_cache_id_from_cacd() iterates CACD's X2APIC list: {0,1,2,3}
>> CPU 1, 2, 3 are offline -> skipped
>> CPU 0 is online -> cache_id = get_cpu_cacheinfo_id(0) = L3-X,
>> all passed.
>>
>> Then later CPU2 is online, resctrl's hotplug creates domain L3-Y.
>> But the xarray has no entry for L3-Y, so erdt_mon_read() will return
>> -EIO for that domain.

An error would at least give indication of a problem but I also see a possible
scenario where the data will happily be read from the wrong register. Consider the
following change to the scenario you present: 
CACD: RMDD-A = {0, 2}; RMDD-B = {1, 3}
CPUID: L3-X = {0, 1}; L3-Y = {2, 3}

If ACPI enumeration is done with CPUs 1, 2, and 3 offline then they are skipped.
CPU 0 is online -> cache_id = get_cpu_cacheinfo_id(0) = L3-X

Later when CPU 1 comes online resctrl will consider it to be in L3-X and
also find an xarray entry for L3-X which will result in erdt_mon_read() to
read the RMID data from RMDD-A instead of RMDD-B?

>>
>> If this is the case, I wonder if we could run the sanity check during
>> CPU hotplug to reparse the CACD table whenever a CPU comes online. My
>> thought would be to call get_l3_cache_id_from_cacd() within
>> resctrl_arch_online_cpu().
>>
> 
> In fact, parsing the CACD table during CPU hotplug may not be feasible,
> as the ACPI tables could have already been released. We could instead
> perform the check directly against the erdt_domain_xa xarray. If a
> mismatch is detected, disable ERDT and fall back to legacy MSR-based
> access:
> 
> /* Called from resctrl_arch_online_cpu() */
> void erdt_verify_cpu_domain(int cpu)
> {
>     int cache_id;
> 
>     if (!__erdt_enabled)
>         return;
> 
>     cache_id = get_cpu_cacheinfo_id(cpu, RESCTRL_L3_CACHE);
>     if (cache_id < 0)
>         return;
> 
>     if (!xa_load(&erdt_domain_xa, cache_id)) {
>         pr_warn(FW_BUG "CPU%d L3 domain %d not described by ERDT, disabling ERDT\n",
>                cpu, cache_id);
>         __erdt_enabled = false;
>     }
> }
When considering the example scenario above I do not believe that this implements a
robust transition from ACPI to CPUID.

Additionally, disabling ERDT during a CPU online has broader implications: if ERDT was
enabled earlier then the associated events will be enabled at this point and have
mov_evt::any_cpu set. From what I can tell this would just implicitly expect that the
events could still be read via MSR but not actually route the read to the correct CPU?

I think it would be simpler to separate the ACPI enumeration from the CPUID enumeration
instead of wedging in these snippets at various touch points. The ACPI enumeration could,
for example, just record the ERDT RMDD domain <-> cpu_mask mapping without taking online CPUs
into account. Depending on which tables are found in ACPI, if ERDT is the intended mechanism then
any duplicate CPUID enumeration should be avoided. This means that there will be two unique
flows to enabling resctrl events, one for ACPI and one for CPUID. The x86 online handler can
perform the sanity check when a CPU comes online and since RMDD and CPUID are expected to
correlate the CPU could not be added to resctrl if it is not in the expected domain. To me
this seems simpler and more robust.

Reinette