Re: [RFC] mpam,x86,fs/resctrl: Generic schema description Proof of Concept

[Reinette Chatre]

Hi Ben,

On 7/23/26 12:58 AM, Ben Horgan wrote:
> On 7/22/26 17:32, Reinette Chatre wrote:
>> On 7/22/26 3:03 AM, Ben Horgan wrote:
>>> On 7/21/26 18:30, Reinette Chatre wrote:
>>>> On 7/21/26 6:23 AM, Ben Horgan wrote:
>>>>> On 7/20/26 23:54, Reinette Chatre wrote:
>>
>> ...
>>>>> I don't think we should introduce more percent based controls and for new controls we can introduce
>>>>> a new format to describe them. Perhaps just the positive integer with a resolution supplied in info/
>>>>
>>>> No, we should not introduce more percentage based controls per se. With the new schema format a
>>>> percentage based control is just a variant of a proportional scalar control.
>>>>
>>>>> as discussed previously. Although, I have been pondering on whether we can do a bit better.
>>>>>
>>>>> We could use hexadecimal point based format for controls which are a proportion of a resource and
>>>>> have a resolution which is a power of 2. The advantage of this is that the meaning of the value is
>>>>> independent of the granularity of the control (number of parts).
>>>>>
>>>>> 0 is represented as 0x0
>>>>> 1 as                0x1
>>>>> 1/2 as              0x0.8
>>>>> 7/256 as            0x0.07
>>>>> 1/2**28             0x0.00000001
>>>>> etc
>>>>>
>>>>> This maps well to the MPAM fixed-point fraction point format without having the weirdness of having
>>>>> values forced to 1 or 0 not being really 0. These MPAM h/w oddities can be hidden just by using the
>>>>> mbw_min mbw_max of a control. In MPAM this could be used in CMIN, CMAX, MB_MAX, MB_MIN and I would
>>>>> hope this would be useful for other architectures too. I am preparing some RFC patches on top of
>>>>> your PoC for consideration of this idea and to explore some of the proposals discussed relating to
>>>>> generic schemata and how they land in practice from the MPAM side.
>>>>
>>>> I'm going to stand with Dave Martin [1] on this point with a preference to avoid floating point in
>>>> resctrl input and output.
>>>
>>> Ok, but I'm not suggesting floating point. Dave's two objections to floating point were that it's
>>> not exact and that it's hard to parse. For the first it is exact as it is using base 16 and for the
>>> parsing it's essentially parsing a prefix, '0x0.', and then a hexadecimal value and shifting for
>>> resolution. 0 and 1 will need to be considered as special cases. What I did wonder if it was too
>>> weird a format for userspace. Anyhow, no point flogging a dead horse, let's just go with using a
>>> number in a proportional schema along with the resolution as Dave summarized in [1]
>>>
>>> [1] https://lore.kernel.org/lkml/aPtfMFfLV1l%2FRB0L@xxxxxxxxxxxxxxx/
>>
>> ok. I do like how the scalar control from that proposal accommodates a variety of proportional and
>> absolute hardware controls without forcing/requiring hardware controls to be a particular shape.
>>
>>>
>>>> Could you please elaborate where values are forced to 1 or 0? The new schema format was intentionally
>>>> created to *avoid* rounding errors (and parsing complexity).
>>>
>>> This a property of the MPAM fixed-point fraction point format in the hardware. It would need to be
>>> hidden whatever resctrl schema format is used, percentage, scalar, fraction ... etc.
>>
>> Just a clarification, all three of the examples you mention (percentage, scalar, fraction) are currently
>> expected to be supported by the single "scalar" control.
> 
> Ack
> 
>>
>> (see later for comment on "need to be hidden")
>>
>>> For an MPAM MBW MAX control with a width of 4 bit the range of values in 0-0xF. In this set up the
>>> resolution is 16 but there are only 15 possible values. For MBW MAX a 0 in hardware the minimum
>>> value which becomes 1/16, ~6%.
>>> See mpam_resctrl.c:mbw_max_to_percent() for how this is currently dealt with.
>>>
>>> For an MPAM MBW MIN control we shift the other way and so 0 is 0 but (with 4 bits) 0xF is 15/16 ~94%.
>>> When the input resolution is a power of 2 these conversions between h/w value and schemata value can
>>> be slightly simplified but there is still some ugly offset of 1.
>>>
>>> (The spec is a little less precise than this and uses weasel words like "Arm recommends..." but this
>>> seems to be the sensible interpretation. See ARM IHI0099B.c "MPAM system component specification",
>>> Section 9.3.)
>>
>> Thank you for the clarification. Sounds related to the weasel words in
>> https://lore.kernel.org/lkml/20260709093111.367851-4-ben.horgan@xxxxxxx/ that took us a while to settle on.
>>
>> You mention above that this would need to be hidden but from what I understand this is accommodated by
>> the "tolerance" property of the "scalar" control?
>>
>> Per https://lore.kernel.org/lkml/aPtfMFfLV1l%2FRB0L@xxxxxxxxxxxxxxx/:
>>
>> 	Note on the "tolerance" parameter:
>>
>> 	This is a new addition.  On the MPAM side, the hardware has a choice
>> 	about how to interpret the control value in some edge-case situations.
>> 	We may not reasonably be able to probe for this, so it may be useful
>> 	to warn software that there is an uncertainty margin.
> 
> Maybe, "tolerance" was the one parameter from that proposal that stuck out as odd to me. I haven't
> managed to understand the semantics. In what situations is this different from the granularity?

I see "tolerance" as an evolution of what can be expressed with the granularity. Granularity as exposed
via the "bandwidth_gran" file is rooted in the original RDT enabling that was percentage based and its
documentation reflects that.

https://lore.kernel.org/lkml/20260709093111.367851-4-ben.horgan@xxxxxxx/ aimed to add weasel words to
support usage by other architectures but could not really accommodate all architectural nuances. This is
because "bandwidth_gran" is, at least claims to be in the documentation, percentage based and always
rounded up.

The "tolerance" parameter enables architecture to provide difference between requested and actual control
value with a finer grained precision than percentages. More in line with the actual properties of the control
value found in the other files introduced as part of this work. Beyond that the "tolerance" parameter also
makes it clear that the effective control value can be larger or smaller than requested.

> Also, isn't the inaccuracy from this h/w interface confusion going to be trumped by inaccuracies and
> inconsistencies in the h/w implementation of the throttling? I'm ok with having the parameter as
> long as we know how to decide what values to have in it and that they provide additional value.
> However personally, I am unclear on what the correct values would be.

resctrl would not do any measurements in an attempt to provide very accurate "tolerance" values, instead
the goal is to provide insight to which values are actually be programmed to the hardware.

I expect the scenario to be that user space writes a control value X to the schemata file and when the
user reads back the schemata file it may actually contain a control value of X ± tolerance. Not seeing
the original written control value may be unexpected to the user and not knowing how much the actual
control value can vary may introduce uncertainty. Having the "tolerance" value exposed to user space
gives insight to the fact that there may be variation while also set expectations on how big a
difference the requested from actual control value can be.

Reinette