Re: [PATCH 1/2] x86/resctrl, Documentation: Keep mbm_assign_mode "default" on boot
From: Babu Moger
Date: Mon Jul 27 2026 - 16:17:57 EST
Hi Reinette,
On 7/27/26 13:12, Reinette Chatre wrote:
Hi Babu,
On 7/27/26 10:24 AM, Babu Moger wrote:
On 7/27/26 10:25, Reinette Chatre wrote:
On 7/27/26 7:05 AM, Borislav Petkov wrote:
On Fri, Jul 24, 2026 at 04:53:26PM -0700, Reinette Chatre wrote:
One clarification here is that as I understand there has not yet been
an actual user complaint. At least not that the pqos utility mentioned in
this thread is aware of. Instead I struggle with the speculation about possible
user complaints with different interpretations on how this change could be
perceived by users.
Sure, but don't you think that it is enough that we know about it?
Only if we are sure that we have the complete picture. I do not believe we do, yet.
We already have an issue open for this, and it is fairly
straightforward to reproduce. Please let me know if there is
anything specific you would like me to try.
Just to be clear, when you refer to issue it means that pqos is returning zero for
unassigned counters?
Yes.
Is this issue public? I am not seeing any related issues in
https://github.com/intel/intel-cmt-cat/issues
Opened one:
[1] https://github.com/intel/intel-cmt-cat/issues/311
>
I connected with the friendly pqos folks on this issue. They subsequently did an audit of
the tool to understand how it may behave under the different scenarios. They found that
the tool currently parses text return values ("Unavailable", "Unassigned") as zero.
This could cause incorrect behavior. (more below)
I mean, the pqos tool shows 0.0 in the MBL/MBR columns now.
And is the
pqos -m all:[0-191]
invocation not something people would usually run?
ABMC enabled (mode claimed to be incompatible with pqos)
--------------------------------------------------------
This example is from Babu's email that highlights that when ABMC is enabled this returns
zero as bandwidth from pqos perspective. This matches the pqos audit that found with text return
states, like the "Unassigned" happening underneath the output above, the return value is zero. When
the counters are not assigned then the events will always return "Unassigned" (until reassigned)
so having it return zero all the time may not cause issues. Once reassigned the event value
will always return a valid value (no text return values).
ABMC disabled (mode claimed to be compatible with pqos)
-------------------------------------------------------
Without ABMC there are scenarios when events may return "Unavailable". These scenarios vary
based on how many monitor groups are created and the workloads run. I am not able to test this
but when I attempt to combine what I know about AMD bandwidth monitoring with the results from
the pqos audit I am concerned that there may be a problem.
Consider a scenario where an event may return "Unavailable", for example:
<return A>, <return "Unavailable">, <return B (larger than A)>, <return "Unavailable">, ...
pqos will see:
A, 0, B, 0, ...
The "pqos -m" usage attempts to determine the bandwidth rate and, for example, when going from
"A" to "0" it will be considered counter wraparound that would appear as a very large and
wrong bandwidth number.
Yes. Agree. This is known issue.
Has this been reported? I do not recognize it in https://github.com/intel/intel-cmt-cat/issues
and the pqos folks I spoke with was not aware of this behavior.
I was not able to recreate this issue. I just opened for the one I have seen [1].
Question to Babu
----------------
Do you perhaps have a test/workloads that, under "default" counter assignment mode, can create
many monitor groups (more than 64) and cause "Unavailable" to be returned frequently? Would it
be possible to put pqos through its paces in this environment?
Yes, I tried to reproduce the "Unavailable" issue with the pqos tool.
So far, I have not been able to observe the issue. I created 64
monitoring groups, ran MLC, and simultaneously executed pqos.
If I understand correctly 64 monitor groups will still have enough underlying hardware counters
and always return event counts, never return "Unavailable". It is when the number of
monitor groups are 65 or more that the hardware counters will start to be re-assigned, no?
Yes. I created 64 monitoring groups and then ran pqos, which creates an additional 15 monitoring groups.
Even so, I do not know if 65 would easily trigger the issue ... sounds like it is possible to create
many more (4096) monitor groups on these systems so there appears to be some room to make it easier to
create scenario where "Unavailable" is returned.
On a higher level: could you please run the test that can expose pqos to environment where it will
encounter "Unavailable" and observe how that is handled?
I was able to reproduce the condition where the hardware reports "Unavailable" for some of the MBM counters.
#cat /sys/fs/resctrl/mon_groups/dir*/mon_data/mon_L3_00/mbm_*
Unavailable
Unavailable
Unavailable
Unavailable
Unavailable
Unavailable
However, the pqos tool continues to operate normally and reports valid bandwidth values:
#sudo pqos -m all:[0-191]
CORE IPC MISSES LLC[KB] MBL[MB/s] MBR[MB/s]
0-191 2.55 55146k 74496.0 20.0 61.4
The issue has not reproduced in any of my tests so far.
I'm confused. It reads like the issue that you mention earlier is a known issue cannot be
reproduced.
I meant that the pqos tool continued to work as expected even after I recreated the condition that caused the hardware to report "Unavailable".
This might require specific test scenario to recreate.
Right.
Of course, "AMBC disabled" scenario also includes AMD systems before ABMC was introduced. I do
not know why the issue with "Unavailable" handling (text returns in general) has not been reported
until now. I assume the pqos team may do most testing on Intel systems that do not return text.
...
The patch notes mention that this change is planned to be reverted at some
point in the future and I have not heard of any changes to this plan.
Where does it say that? I don't see that aspect.
For convenience, copy of patch notes from original commit [1] is below:
There are plans to enable "mbm_event" by default once additional counters
are available. For now, keep the default mode to maintain compatibility
with existing tools.
I added the text below in the comments section (after ---).
On second thought, that was probably my mistake, and I don't think I
should have added it. I also don't see this happening anytime soon.
ok.
The original problem statement is that pqos is not able to handle "Unassigned" return values
Considering all of this, I believe it would make sense to make the
"default" mode the preferred mode.
that will be encountered when "assignable counter mode" is enabled.
The request is to disable "assignable counter mode" by default to address the problem
when handling "Unassigned" counters.
Disabling "assignable counter mode" will under certain circumstances cause "Unavailable" to
be returned on event read.
At this time there seems to be a mismatch in understanding how pqos can handle
"Unavailable" return values. For me to sign off on this patch I would like to fully understand
the risk doing so. Could you please confirm from your side that pqos can handle all scenarios of
the mode that you request to be the default mode?
I created the maximum number of monitoring groups supported by the system and was able to reproduce the condition where the hardware reports "Unavailable". Even under those conditions, the pqos tool appeared to function correctly and continued to report valid results.
It's worth keeping in mind that ABMC mode is not available on the majority of hardware currently accessible to public.
Furthermore, default mode has been the longstanding behavior in deployed systems. Moving back to the default mode is therefore not a new change, but a return to the behavior that users are already familiar with.
Thanks,
Babu