Re: [RFC] mpam,x86,fs/resctrl: Generic schema description Proof of Concept
From: Babu Moger
Date: Tue Jun 02 2026 - 16:23:27 EST
Hi Reinette,
For some reason, I couldn’t find your patch on lore.kernel.org:
https://lore.kernel.org/lkml/?q=Reinette+Chatre
I eventually located it here:
https://sashiko.dev/#/message/aab804b9-e8b5-40ad-a85b-af7033391243%40intel.com
Thanks for sharing the patches. I’m still reviewing them.
I was able to build and boot the kernel and can see the MIN and MAX controls. After moving your test code to __rdt_get_mem_config_amd().
On 5/29/26 13:06, Reinette Chatre wrote:
Hi Everybody,
It has been a while since we discussed the resctrl changes required to support
hardware that has controls with fine granularity or hardware that has multiple
controls per resource. For reference, the most recent email discussion can
be found at [1] with a summary of discussions in last year's plumbers slides [2].
I created a PoC that I believe supports what folks have agreed to so far. I
hope this can help us to restart the discussion with the goal that resctrl gains
support for upcoming hardware that require these features.
Request regarding this PoC
==========================
Please consider this PoC as a "direction check" on the schema description and multiple
control discussions held thus far.
Could folks working on enabling new hardware requiring this capability please consider
if this is something you can build on and how it should be improved to support these
upcoming capabilities?
Opens
=====
While the PoC aims to support what folks agreed on some opens remain:
- I attempted to make some MPAM supporting changes but these are all just compile
tested. While MPAM should benefit from the new control properties I did not
initialize them on MPAM and did not attempt refactor to separate out
the architecture specific control properties (more on what this means later).
I did attempt some MPAM refactoring that duplicates the MPAM domain to the
control domain and monitoring domain lists in support of there being multiple
controls each with its own list of control domains but it is definitely not good
design.
- No support for emulated controls (yet). The PoC is quite large already
but I think it can be used as a base for emulated controls for which the software
controller could be a potential first customer. In this PoC mounting with
software controller will still display the original controller's properties.
- One open that needs to be addressed as part of support for emulated controls is
how best to display emulation relationship via resctrl hierarchy.
- No support for "read-modify-write" usage of schemata file. This is where we
discussed (without agreement) on possibly introducing the "#" prefix to schemata
file entries. This PoC does not support this prefix and the current assumption/expectation
is that when user space changes a configuration only the new control values are
written to schemata file. I thus do not have a plan to support this so please
share opinions in this regard if you have some.
- Controls are independent for now. This means that, for example, if a resource
supports a "MIN" and "MAX" control then this implementation would allow user to
set the "maximum" control values to be less than the "minimum" control values.
- PoC supports the "bitmap" control but does not (yet) expose properties of a bitmap
control to the new info/<resource>/resource_schemata directory.
Accessing PoC
=============
Please consider the PoC as a rough draft. It has only been compile tested for Arm
and known to be incomplete in Arm support. To help with experimenting I only
fully adapted the Intel MBA resource to demo two dummy additional MBA controls.
All architectures should immediately benefit from the new schema descriptions
and new info/MB/resource_schemata hierarchy.
I considered the patches self too many for email. Instead, the PoC can be found at:
git://git.kernel.org/pub/scm/linux/kernel/git/reinette/linux.git branch resctrl/controls_rfc_v1
The work is based on v7.1-rc2 that also includes the following series (two of which has
since been queued) included:
"selftests/resctrl: Fixes and improvements focused on Intel platforms"
https://lore.kernel.org/lkml/cover.1775266384.git.reinette.chatre@xxxxxxxxx/
"x86,fs/resctrl: Improve resctrl quality and consistency"
https://lore.kernel.org/lkml/cover.1777419024.git.reinette.chatre@xxxxxxxxx/
"x86,fs/resctrl: Pave the way for MPAM counter assignment"
https://lore.kernel.org/lkml/20260506082855.3694761-1-ben.horgan@xxxxxxx/
Primary resctrl fs data structure changes
=========================================
Introduces a control represented by struct resctrl_ctrl that looks as below. To make
the changes easier to follow I kept some of the original names to help communicate
where familiar data structures land.
What to notice about a control is that it has some common properties required
from all controls (scope, type, etc.) and then depending on the type of control
(RESCTRL_CTRL_BITMAP or RESCTRL_CTRL_SCALAR) there are type specific properties.
/**
* struct resctrl_ctrl - A resource control
* @entry: List entry of rdt_resource::controls
* @scope: Scope of the resource that this control allocates
* @domains: RCU list of all control domains
* @type: The control type that determines the properties of the control,
* format string for displaying control values to user space, and
* parser of control values provided by user space.
* @name: Name of the control. Appended to final resource name
* (rdt_resource_final::name) to create final schema entry.
* Specifically, "rdt_resource_final::name"_"resctrl_ctrl::name".
* For example, with resource name "MB" and control name "MAX" the
* schema entry will be "MB_MAX".
* @cache: Cache allocation control properties.
* @membw: Bandwidth control properties.
*/
struct resctrl_ctrl {
struct list_head entry;
enum resctrl_scope scope;
struct list_head domains;
enum resctrl_ctrl_type type;
enum resctrl_ctrl_name name;
union {
struct resctrl_cache cache;
struct resctrl_membw membw;
};
};
Two members summarize how this new structure fits into the rest of resctrl:
a) resctrl_ctrl::entry
Since a resource can support multiple controls there is a new list
in struct rdt_resource named "controls" that contains the list of all
controls supported by the resource.
b) resctrl_ctrl::domains
Instead of the list of control domains belonging to a resource they
now belong to the control self. By doing so resctrl can support resource
controls at different scope for the same resource. This is intended to
support some upcoming MPAM and RISC-V usages.
I like the idea of supporting multiple controls for each resource.
With these patches, now we have one list containing all the controls.
However, in case of RDT_RESOURCE_L3, we have two lists "mon_domains" and "controls". mon_domains list deals with monitoring and control deals with control parts(multiple).
Have you thought about making the list("control") generic so that the control can be monitoring also. It will just one list containing multiple controls or monitor.
Example architectural data structure changes
============================================
An architecture can use the new control by following a similar pattern to
resource and domain use by architectures. Consider the following for x86
where a new architecture specific struct resctrl_hw_ctrl includes
struct resctrl_ctrl and any architecture private data needed to support
the control:
/*
* struct resctrl_hw_ctrl - Arch private properties of a resource control
* @r_ctrl: Control properties exposed to resctrl file system
* @msr_base: Base MSR address where control values should be programmed
* @msr_update: Function pointer to update control values
*/
struct resctrl_hw_ctrl {
struct resctrl_ctrl r_ctrl;
unsigned int msr_base;
void (*msr_update)(struct msr_param *m);
};
Structure of patch series
=========================
As a PoC the series is not perfectly structured but to help navigate this work
on a high level the changes can be categorized as follows:
Patch 1 to 11:
With a vision of what a "control" is, remove unused/unnecessary
members, make clear what is a *resource* property vs a *control*
property, do some renaming to help with the PoC.
Patch 12:
Introduce struct resctrl_ctrl and re-arrange existing struct rdt_resource
members to form part of new rdt_resource::ctrl
Patch 13 to 44:
A lot of wrangling to introduce struct resctrl_ctrl to all code that needs
to work with a control and/or domain without assuming that the control is
the one and only control embedded in the resource it belongs to. Essentially,
a lot of changes passing the control around in addition to the resource/domain.
Patch 45:
Switch the single struct resctrl_ctrl member of struct rdt_resource to be
a list of struct resctrl_ctrl.
Patch 47 to 49:
Introduce new info/<resource>/resource_schemata hierarchy to first only
consist of properties already known to resctrl fs.
Patch 50 to 52:
Introduce the new control properties per [1], initialize them for x86,
and expose them via info/<resource>/resource_schemata
Patch 53:
Let the new struct resctrl_hw_ctrl contain architecture's control properties.
Patch 54:
Teach resctrl fs about "MIN" and "MAX" controls.
Patch 55:
Sample of "MIN" and "MAX" memory bandwidth controls for x86.
My assumption is that the MIN and MAX controls here are just examples, correct?
You only mentioned patch 55 as "NOT_FOR_INCLUSION". I assume patch 54 should also be marked as "NOT_FOR_INCLUSION"?
Thanks
Babu
Example interactions
====================
This series can be used on an x86 system where it will show two new dummy controls
where it is possible to interact with the new controls.
For example:
# cat schemata
MB_MAX:0=100;1=100
MB_MIN:0=100;1=100
MB:0=100;1=100
L3:0=fff;1=fff
# echo 'MB_MIN:0=50' > schemata
# cat schemata
MB_MAX:0=100;1=100
MB_MIN:0=50;1=100
MB:0=100;1=100
L3:0=fff;1=fff
Writing to the dummy control will call a dummy callback that just prints to the
kernel log:
"resctrl: Updata temporary MIN control on domain 0 with user value 50"
Example output of info/MB/:
/sys/fs/resctrl/info/MB/thread_throttle_mode:max
/sys/fs/resctrl/info/MB/num_closids:15
/sys/fs/resctrl/info/MB/delay_linear:1
/sys/fs/resctrl/info/MB/min_bandwidth:10
/sys/fs/resctrl/info/MB/resource_schemata/MB/resolution:100
/sys/fs/resctrl/info/MB/resource_schemata/MB/tolerance:5
/sys/fs/resctrl/info/MB/resource_schemata/MB/type:scalar
/sys/fs/resctrl/info/MB/resource_schemata/MB/min:10
/sys/fs/resctrl/info/MB/resource_schemata/MB/scale:1
/sys/fs/resctrl/info/MB/resource_schemata/MB/scope:L3
/sys/fs/resctrl/info/MB/resource_schemata/MB/unit:all
/sys/fs/resctrl/info/MB/resource_schemata/MB/max:100
/sys/fs/resctrl/info/MB/resource_schemata/MB_MIN/resolution:100
/sys/fs/resctrl/info/MB/resource_schemata/MB_MIN/tolerance:5
/sys/fs/resctrl/info/MB/resource_schemata/MB_MIN/type:scalar
/sys/fs/resctrl/info/MB/resource_schemata/MB_MIN/min:10
/sys/fs/resctrl/info/MB/resource_schemata/MB_MIN/scale:1
/sys/fs/resctrl/info/MB/resource_schemata/MB_MIN/scope:L3
/sys/fs/resctrl/info/MB/resource_schemata/MB_MIN/unit:all
/sys/fs/resctrl/info/MB/resource_schemata/MB_MIN/max:100
/sys/fs/resctrl/info/MB/resource_schemata/MB_MAX/resolution:100
/sys/fs/resctrl/info/MB/resource_schemata/MB_MAX/tolerance:5
/sys/fs/resctrl/info/MB/resource_schemata/MB_MAX/type:scalar
/sys/fs/resctrl/info/MB/resource_schemata/MB_MAX/min:10
/sys/fs/resctrl/info/MB/resource_schemata/MB_MAX/scale:1
/sys/fs/resctrl/info/MB/resource_schemata/MB_MAX/scope:L3
/sys/fs/resctrl/info/MB/resource_schemata/MB_MAX/unit:all
/sys/fs/resctrl/info/MB/resource_schemata/MB_MAX/max:100
/sys/fs/resctrl/info/MB/bandwidth_gran:10
Any feedback is appreciated.
Reinette
[1] https://lore.kernel.org/lkml/aPtfMFfLV1l%2FRB0L@xxxxxxxxxxxxxxx/
[2] https://lpc.events/event/19/contributions/2093/attachments/1958/4172/resctrl%20Microconference%20LPC%202025%20Tokyo.pdf