Re: [PATCH v9 06/12] arm,x86,fs/resctrl: Handle change in number of RMIDs on each mount

[Reinette Chatre]

Hi Tony,

On 7/14/26 1:28 PM, Luck, Tony wrote:
> Hi Reinette,
> 
> On Tue, Jul 14, 2026 at 11:16:53AM -0700, Reinette Chatre wrote:
>> Hi Tony,
>>
>> On 7/14/26 10:50 AM, Luck, Tony wrote:
>>> On Tue, Jul 14, 2026 at 09:45:04AM -0700, Reinette Chatre wrote:
>>>> Hi Tony,
>>>
>>> ... trimming to open issue ...
>>>
>>>>>> How much to rely on CPUID is not clear to me. The direction seems to
>>>>>> be to move away from CPUID, which makes adding new CPUID dependencies
>>>>>> less ideal? 
>>>
>>> Where is this direction to move away from CPUID coming from?
>>
>> This is the impression I got from the discussions during series that adds support for
>> LLC occupancy monitoring via ERDT (https://lore.kernel.org/lkml/cover.1782866200.git.yu.c.chen@xxxxxxxxx/).
>> If ERDT is the future direction then I do not think resctrl should assume that the
>> same data will always be available via CPUID also. With ERDT there is a separate
>> per-domain maximum RMID that should be discovered via ACPI.
> 
> My view of system design today is that architects have a box full of Lego(TM)
> bricks for features that they clip together. The bricks often have different
> parameters - sized based on the maximum values across all systems where they
> might be used.
> 
> In this case you end up with different components supporting different
> numbers of RMIDs.

ack.

> 
> For RMID the absolute maximum usable value is defined by CPUID(0xF).EBX.

Would this be the case even if CPUID(0xF).EDX == 0?

> A WRMSR to IA32_PQR_ASSOC.RMID of any value greater than that will #GP
> fault. So, while you might have an ERDT table saying that it supports 500
> RMIDs, if that Lego brick is plugged into an SoC that has CPUID(0xF).EBX =
> 399, then you can only use 400 RMIDs, and the extra 100 counters on the ERDT
> device will sit unused as there is no way for them to be accessed.
> 
>>>>> Here's my current work-in-progress version:
>>>>>
>>>>> u32 resctrl_arch_system_max_rmid_idx(void)
>>>>> {
>>>>> 	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
>>>>> 	u32 ret;
>>>>>
>>>>> 	/* CPUID provides maximum possible RMID value */
>>>>> 	ret = cpuid_ebx(0xf) + 1;
>>>>
>>>> There is also boot_cpu_data.x86_cache_max_rmid initialized in resctrl_cpu_detect() that
>>>> could be used directly? Looks like resctrl_cpu_detect() already scales the number of
>>>> RMID down if needed when L3 monitoring is supported, but it does not take SNC into account.
>>>
>>> I looked at that, but rejected it because it isn't adjusted for SNC.
>>
>> Right. Neither is cpuid_ebx(0xf). Is it necessary for resctrl_arch_system_max_rmid_idx()
>> to call CPUID every time or could it use the cached value in boot_cpu_data.x86_cache_max_rmid?
> 
> Yes. Necessary. See how boot_cpu_data.x86_cache_max_rmid is derived:
> 
> /* Runs once on the BSP during boot. */
> void resctrl_cpu_detect(struct cpuinfo_x86 *c)
> {
> 	if (!cpu_has(c, X86_FEATURE_CQM_LLC) && !cpu_has(c, X86_FEATURE_ABMC)) {
> 		c->x86_cache_max_rmid  = -1;
> 		c->x86_cache_occ_scale = -1;
> 		c->x86_cache_mbm_width_offset = -1;
> 		return;
> 	}
> 
> On a theoretical platform that doesn't support X86_FEATURE_CQM_LLC or X86_FEATURE_ABMC
> x86_cache_max_rmid will be -1.
> 
> In fact you can get this with boot argument of "clearcpuid=cqm_occup_llc"
> 
> I wonder if that test should really be for X86_FEATURE_CQM and thus
> check CPUID(0x7).EBX[12]?

It seems that doing so would match the definition of IA32_PQR_ASSOC found below that
you highlighted earlier and should result in a valid x86_cache_max_rmid as intended?
I am not familiar with the history of the original enabling
(commit cbc82b172638 ("x86: Add support for Intel Cache QoS Monitoring (CQM) detection"))
that resulted in X86_FEATURE_CQM not used.


> 
>>>>> 	/*
>>>>> 	 * if system is capable of L3 monitoring the maximum RMID value may
>>>>> 	 * be lower that system maximum. Either because the L3 monitoring
>>>>> 	 * feature supports fewer RMIDs (CPUID(0xF, 0x1).ECX), or because SNC
>>>>> 	 * (Sub-NUMA Cluster) is enabled and divides RMIDs per cluster.
>>>>> 	 */
>>>>> 	if (r->mon_capable)
>>>>> 		ret = r->mon.num_rmid;
>>>>>
>>>>> 	return ret;
>>>>> }
>>>>>
>>>>> CPUID seems unavoidable in the case that the platform doesn't support
>>>>> (or has disabled the various L3 monitoring events). In that case using
>>>>
>>>> Since L3 monitoring is the only CPUID supported monitoring resource I do not
>>>> think cpuid_ebx(0xf) can be used if the platform does not support L3 monitoring.
>>>> I expect that would mean that leaf 0x7 would indicate that monitoring is
>>>> not supported which would make leaf 0xf invalid? Interestingly resctrl does
>>>> not seem to consider X86_FEATURE_CQM from leaf 0x7 at all and just goes straight
>>>> to leaf 0xf.
>>>
>>> I disagree. The definition of IA32_PQR_ASSOC says:
>>>
>>> 1) The MSR exists if either of CPUID.07H.00H:EBX[12] or CPUID.07H.00H:EBX[15] is set
>>>    (in this case bit 12 for monitoring, It does appear to be a bug that resctrl is
>>>    not checking bit 12 before looking at leaf 0xF).
>>>
>>> 2) The supported width of the RMID field is Ceil(Log2 (CPUID.0FH.00H:EBX[31:0] +1))
>>>
>>> So I believe that a theoretical system that supported AET (or some other
>>> monitoring) but didn't support L3 monitoring, would have to set CPUID.07H.00H:EBX[12]
>>> and provide the max RMID value in CPUID.0FH.00H:EBX[31:0].
>>
>> I see. That would mean that CPUID would report a max RMID value while also reporting
>> that there are no resources being monitored. I assume ERDT is an example of "some other
>> monitoring" and it has its own per-domain max RMID that is separate from the different
>> monitoring features. How do these different max RMID values relate? Should this
>> architectural "get the max RMID" be expanded to also consider the per-RMDD maximum when
>> ERDT support lands?
> 
> Yes, ERDT is an example of a monitoring resource enumerated by ACPI.
> 
> AET is enumerated via a combination of PCIe configuration space VSEC
> entries and an XML file.
> 
> See above "Lego" explanation for why they don't factor into the "max RMID"
> calculation.
> 
>>>
>>> For resctrl we have the practical case of booting with "rdt=!cmt,!mbmtotal,!mbmlocal"
>>> to software disable all L3 monitoring.
>> ack.
>>
>> Reinette
>>
> 
> -Tony

Reinette