Re: [RFC PATCH 04/19] fs/resctrl: Add the documentation for Global Memory Bandwidth Allocation

[Reinette Chatre]

Hi Babu and Tony,

On 2/3/26 8:38 AM, Babu Moger wrote:
> Hi Tony,
> 
> On 2/2/26 18:00, Luck, Tony wrote:
>> On Wed, Jan 21, 2026 at 03:12:42PM -0600, Babu Moger wrote:
>>> +Global Memory bandwidth Allocation
>>> +-----------------------------------
>>> +
>>> +AMD hardware supports Global Memory Bandwidth Allocation (GMBA) provides
>>> +a mechanism for software to specify bandwidth limits for groups of threads
>>> +that span across multiple QoS domains. This collection of QOS domains is
>>> +referred to as GMBA control domain. The GMBA control domain is created by
>>> +setting the same GMBA limits in one or more QoS domains. Setting the default
>>> +max_bandwidth excludes the QoS domain from being part of GMBA control domain.
>> I don't see any checks that the user sets the *SAME* GMBA limits.
>>
>> What happens if the user ignores the dosumentation and sets different
>> limits?
> 
> Good point. Adding checks could be challenging when users update each schema individually with different values. We don't know which one value is the one he is intending to keep.
> 
>> ... snip ...
>>
>> +  # cat schemata
>> +    GMB:0=2048;1=2048;2=2048;3=2048
>> +     MB:0=4096;1=4096;2=4096;3=4096
>> +     L3:0=ffff;1=ffff;2=ffff;3=ffff
>> +
>> +  # echo "GMB:0=8;2=8" > schemata
>> +  # cat schemata
>> +    GMB:0=   8;1=2048;2=   8;3=2048
>> +     MB:0=4096;1=4096;2=4096;3=4096
>> +     L3:0=ffff;1=ffff;2=ffff;3=ffff
>>
>> Can the user go on to set:
>>
>>     # echo "GMB:1=10;3=10" > schemata
>>
>> and have domains 0 & 2 with a combined 8GB limit,
>> while domains 1 & 3 run with a combined 10GB limit?
>> Or is there a single "GMBA domain"?
> 
> In that case, it  is still treated as a single GMBA domain, but the behavior becomes unpredictable. The hardware expert mentioned that it will default to the lowest value among all inputs in this case, 8GB.
> 
> 
>> Will using "2048" as the "this domain isn't limited
>> by GMBA" value come back to haunt you when some
>> system has much more than 2TB bandwidth to divide up?
> 
> It is actually 4096 (4TB). I made a mistake in the example.  I am assuming it may not an issue in the current generation.
> 
> It is expected to go up in next generation.
> 
> GMB:0=4096;1=4096;2=4096;3=4096;
>    MB:0=8192;1=8192;2=8192;3=8192;
>     L3:0=ffff;1=ffff;2=ffff;3=ffff
> 
> 
>>
>> Should resctrl have a non-numeric "unlimited" value
>> in the schemata file for this?
> 
> The value 4096 corresponds to 12th bit set.  It is called U-bit. If the U bit is set then that domain is not part of the GMBA domain.
> 
> I was thinking of displaying the "U" in those cases.  It may be good idea to do something like this.
> 
> GMB:0=      8;1=      U;2=     8 ;3=      U;
>    MB:0=8192;1=8192;2=8192;3=8192;
>     L3:0=ffff;1=ffff;2=ffff;3=ffff
> 
> 
>>
>> The "mba_MBps" feature used U32_MAX as the unlimited
>> value. But it looks somewhat ugly in the schemata
>> file:
> Yes, I agree. Non-numeric would have been better.

How would such a value be described in a generic way as part of the new schema
description format?

Since the proposed format contains a maximum I think just using that
value may be simplest while matching what is currently displayed for
"unlimited" MB, no?

Reinette