Re: [PATCH 1/2] x86/resctrl, Documentation: Keep mbm_assign_mode "default" on boot

[Reinette Chatre]

Hi Boris,

On 7/31/26 6:22 PM, Borislav Petkov wrote:
> On Fri, Jul 31, 2026 at 07:59:46AM -0700, Reinette Chatre wrote:
>> The underlying implementations appear to be different between AMD and Intel.
>> On Intel side only the RMIDs (not separate hardware counters) need to be
>> considered.  resctrl supports this RMID limit by placing a limit on how many
>> monitoring groups the user can create.
> 
> Sounds to me like the AMD side should do this too, no, if it doesn't already?

AMD PQoS (whether it supports ABMC or not) also exposes an RMID limit that is used
as the upper limit for the number of monitor groups that can be tracked by the hardware. 

> I mean, it is kinda natural behavior to track counter occupancy and deny the
> allocation of new ones if there are none free...

On AMD the RMID limit is separate from the hardware counters that feed event count
values into the RMID event counts. On AMD the two limits are significantly different.
Babu shared [1] that the hardware of concern exposes 4096 RMID while it supports 32
(exposed to software) hardware counters when ABMC is enabled and 64 (not exposed to
software) RMID when AMBC is disabled.

resctrl could theoretically limit the number of monitor groups based on the number of
RMID that could be accurately counted (with the caveat that resctrl cannot always 
determine this number) but based on the numbers supported by hardware that would severely
limit the number of monitor groups users can create on this hardware.

Users want to create a lot of monitor groups. Each monitor group contains a group of
tasks that need to be monitored together. Creating a monitor group and assigning tasks
to it is expensive since it involves the actual resctrl monitor group file hierarchy
creation followed by interacting with these files to move the tasks into the monitoring
group that internally involves changes to the various task_struct's.

While resctrl could thus limit the number of monitor groups it would push the
burden of monitoring many different task groups up to user space at significant cost.
Instead, resctrl exposes a per-monitor group interface on AMD systems that support
ABMC by enabling user space to more quickly switch which of the supported monitor groups
are counted at a particular time.

> 
>> The Intel/RDT spec does leave room for this to occur but a return of
>> "Unavailable" when reading a monitoring event is not encountered on current
>> Intel hardware.
> 
> Why not? Because nothing's reaching that limit or?

On Intel resctrl only considers the number of RMID. Hardware will return failure if there
is an attempt to read an unsupported RMID that exceeds the limit it exposes, but resctrl
does not do that. On Intel there is not a separate "hardware counter" limit to consider,
only the number of RMID need to be considered and they are always counting.

> 
>> The user can always expect a value when reading a monitoring event (on
>> current hardware).
> 
> How does that work when you exhaust the counters? Or it does switch between
> them quickly so that there's always *something* that counts?

On Intel the RMIDs are always counting. Please keep in mind that these RMID do not
actually return raw counts that are verbatim passed on to user space but instead
the returned value is "upscaled" using a hardware provided scaling factor to obtain the
value that is returned to user space. This is done on AMD (with and without ABMC)
and Intel.

Reinette

[1] https://lore.kernel.org/lkml/78996219-a8de-4dc3-90ee-db4c19e0d66a@xxxxxxx/