Re: [RFC PATCH 01/19] x86,fs/resctrl: Add support for Global Bandwidth Enforcement (GLBE)

[Reinette Chatre]

Hi Babu,

On 2/13/26 3:14 PM, Moger, Babu wrote:
> Hi Reinette,
> 
> 
> On 2/13/2026 10:17 AM, Reinette Chatre wrote:
>> Hi Babu,
>>
>> On 2/12/26 5:51 PM, Moger, Babu wrote:
>>> On 2/12/2026 6:05 PM, Reinette Chatre wrote:
>>>> On 2/12/26 11:09 AM, Babu Moger wrote:
>>>>> On 2/11/26 21:51, Reinette Chatre wrote:
>>>>>> On 2/11/26 1:18 PM, Babu Moger wrote:
>>>>>>> On 2/11/26 10:54, Reinette Chatre wrote:
>>>>>>>> On 2/10/26 5:07 PM, Moger, Babu wrote:
>>>>>>>>> On 2/9/2026 12:44 PM, Reinette Chatre wrote:
>>>>>>>>>> On 1/21/26 1:12 PM, Babu Moger wrote:
>>>>
>>>> ...
>>>>
>>>>>>>> Another question, when setting aside possible differences between MB and GMB.
>>>>>>>>
>>>>>>>> I am trying to understand how user may expect to interact with these interfaces ...
>>>>>>>>
>>>>>>>> Consider the starting state example as below where the MB and GMB ceilings are the
>>>>>>>> same:
>>>>>>>>
>>>>>>>>       # cat schemata
>>>>>>>>       GMB:0=2048;1=2048;2=2048;3=2048
>>>>>>>>       MB:0=2048;1=2048;2=2048;3=2048
>>>>>>>>
>>>>>>>> Would something like below be accurate? Specifically, showing how the GMB limit impacts the
>>>>>>>> MB limit:
>>>>>>>>          # echo"GMB:0=8;2=8" > schemata
>>>>>>>>       # cat schemata
>>>>>>>>       GMB:0=8;1=2048;2=8;3=2048
>>>>>>>>       MB:0=8;1=2048;2=8;3=2048
>>>>>>> Yes. That is correct.  It will cap the MB setting to  8.   Note that we are talking about unit differences to make it simple.
>>>>>> Thank you for confirming.
>>>>>>
>>>>>>>> ... and then when user space resets GMB the MB can reset like ...
>>>>>>>>
>>>>>>>>       # echo"GMB:0=2048;2=2048" > schemata
>>>>>>>>       # cat schemata
>>>>>>>>       GMB:0=2048;1=2048;2=2048;3=2048
>>>>>>>>       MB:0=2048;1=2048;2=2048;3=2048
>>>>>>>>
>>>>>>>> if I understand correctly this will only apply if the MB limit was never set so
>>>>>>>> another scenario may be to keep a previous MB setting after a GMB change:
>>>>>>>>
>>>>>>>>       # cat schemata
>>>>>>>>       GMB:0=2048;1=2048;2=2048;3=2048
>>>>>>>>       MB:0=8;1=2048;2=8;3=2048
>>>>>>>>
>>>>>>>>       # echo"GMB:0=8;2=8" > schemata
>>>>>>>>       # cat schemata
>>>>>>>>       GMB:0=8;1=2048;2=8;3=2048
>>>>>>>>       MB:0=8;1=2048;2=8;3=2048
>>>>>>>>
>>>>>>>>       # echo"GMB:0=2048;2=2048" > schemata
>>>>>>>>       # cat schemata
>>>>>>>>       GMB:0=2048;1=2048;2=2048;3=2048
>>>>>>>>       MB:0=8;1=2048;2=8;3=2048
>>>>>>>>
>>>>>>>> What would be most intuitive way for user to interact with the interfaces?
>>>>>>> I see that you are trying to display the effective behaviors above.
>>>>>> Indeed. My goal is to get an idea how user space may interact with the new interfaces and
>>>>>> what would be a reasonable expectation from resctrl be during these interactions.
>>>>>>
>>>>>>> Please keep in mind that MB and GMB units differ. I recommend showing only the values the user has explicitly configured, rather than the effective settings, as displaying both may cause confusion.
>>>>>> hmmm ... this may be subjective. Could you please elaborate how presenting the effective
>>>>>> settings may cause confusion?
>>>>>
>>>>> I mean in many cases, we cannot determine the effective settings correctly. It depends on benchmarks or applications running on the system.
>>>>>
>>>>> Even with MB (without GMB support), even though we set the limit to 10GB, it may not use the whole 10GB.  Memory is shared resource. So, the effective bandwidth usage depends on other applications running on the system.
>>>>
>>>> Sounds like we interpret "effective limits" differently. To me the limits(*) are deterministic.
>>>> If I understand correctly, if the GMB limit for domains A and B is set to x GB then that places
>>>> an x GB limit on MB for domains A and B also. Displaying any MB limit in the schemata that is
>>>> larger than x GB for domain A or domain B would be inaccurate, no?
>>>
>>> Yea. But, I was thinking not to mess with values written at registers.
>>
>> This is not about what is written to the registers but how the combined values
>> written to registers control system behavior and how to accurately reflect the
>> resulting system behavior to user space.
>>
>>>> When considering your example where the MB limit is 10GB.
>>>>
>>>> Consider an example where there are two domains in this example with a configuration like below.
>>>> (I am using a different syntax from schemata file that will hopefully make it easier to exchange
>>>> ideas when not having to interpret the different GMB and MB units):
>>>>
>>>>      MB:0=10GB;1=10GB
>>>>
>>>> If user space can create a GMB domain that limits shared bandwidth to 10GB that can be displayed
>>>> as below and will be accurate:
>>>>
>>>>      MB:0=10GB;1=10GB
>>>>      GMB:0=10GB;1=10GB
>>>>
>>>> If user space then reduces the combined bandwidth to 2GB then the MB limit is wrong since it
>>>> is actually capped by the GMB limit:
>>>>
>>>>      MB:0=10GB;1=10GB <==== Does reflect possible per-domain memory bandwidth which is now capped by GMB
>>>>      GMB:0=2GB;1=2GB
>>>>
>>>> Would something like below not be more accurate that reflects that the maximum average bandwidth
>>>> each domain could achieve is 2GB?
>>>>
>>>>      MB:0=2GB;1=2GB <==== Reflects accurate possible per-domain memory bandwidth
>>>>      GMB:0=2GB;1=2GB
>>>
>>> That is reasonable. Will check how we can accommodate that.
>>
>> Right, this is not about the values in the L3BE registers but instead how those values
>> are impacted by GLBE registers and how to most accurately present the resulting system
>> configuration to user space. Thank you for considering.
> 
> 
> I responded too quickly earlier—an internal discussion surfaced several concerns with this approach.
> 
> schemata represents what user space explicitly configured and what the hardware registers contain, not a derived “effective” value that depends on runtime conditions.
> Combining configured limits (MB/GMB) with effective bandwidth—which is inherently workload‑dependent—blurs semantics, breaks existing assumptions, and makes debugging more difficult.
> 
> MB and GMB use different units and encodings, so auto‑deriving values can introduce rounding issues and loss of precision.
> 
> I’ll revisit this and come back with a refined proposal.

Are we still talking about below copied from https://lore.kernel.org/lkml/f0f2e3eb-0fdb-4498-9eb8-73111b1c5a84@xxxxxxx/ ?

	The MBA ceiling is applied at the QoS domain level.
	The GLBE ceiling is applied at the GLBE control  domain level.
	If the MBA ceiling exceeds the GLBE ceiling, the effective MBA limit will be capped by the GLBE ceiling. 

Reinette