Re: [PATCH 1/2] x86/resctrl, Documentation: Keep mbm_assign_mode "default" on boot
From: Moger, Babu
Date: Mon Jul 27 2026 - 18:44:59 EST
Hi Reinette,
On 7/27/2026 4:42 PM, Reinette Chatre wrote:
Hi Babu,
On 7/27/26 1:17 PM, Babu Moger wrote:
Hi Reinette,
On 7/27/26 13:12, Reinette Chatre wrote:
Hi Babu,
On 7/27/26 10:24 AM, Babu Moger wrote:
On 7/27/26 10:25, Reinette Chatre wrote:
On 7/27/26 7:05 AM, Borislav Petkov wrote:
On Fri, Jul 24, 2026 at 04:53:26PM -0700, Reinette Chatre wrote:
One clarification here is that as I understand there has not yet been
an actual user complaint. At least not that the pqos utility mentioned in
this thread is aware of. Instead I struggle with the speculation about possible
user complaints with different interpretations on how this change could be
perceived by users.
Sure, but don't you think that it is enough that we know about it?
Only if we are sure that we have the complete picture. I do not believe we do, yet.
We already have an issue open for this, and it is fairly
straightforward to reproduce. Please let me know if there is
anything specific you would like me to try.
Just to be clear, when you refer to issue it means that pqos is returning zero for
unassigned counters?
Yes.
Is this issue public? I am not seeing any related issues in
https://github.com/intel/intel-cmt-cat/issues
Opened one:
[1] https://github.com/intel/intel-cmt-cat/issues/311
Oh, now we have a mess, no? You start by stating that you already have an issue for this and when I
What mess?
I mentioned that I have an open issue that was raised against me internally.
I understood that you wanted me to create a public bug to address the issue, so I went ahead and created one.
ask you about it you quickly go create one with the pqos folks for this behavior we are
currently discussing to change! Why not create an issue for "Unavailable" being treated as zero that
I believe all agree is not correct.
What are you really expecting from pqos folks based on your bug report? Or ... are you now just
using that bug report as leverage and planning to close it when this patch of yours is merged?
I’m not sure what the final fix is at this point. We’ll get to that once we have a confirmed solution.
To be honest as AMD representative and the one that enabled ABMC in resctrl I would expect that
a bug report from you would contain guidance how pqos could best support assignable counters.
Again, I don’t know what the final resolution will be at this point. Once we agree on the fix, I’ll update the bug with the relevant details.
Please take a moment and consider your report from their perspective. I think it may be
reasonable to view the current pqos behavior as correct since that is what the kernel exposes.
If you find current behavior incorrect then please provide some guidance to pqos folks. For example,
do you expect pqos to always switch to default mode when it encounters mbm_event mode .... or do
you expect the pqos should do counter assignment, if so, how should it go about doing so, or ...?
What are you expecting from the pqos folks in response to your bug report?
We will discuss that once come to final resolution. They can always say "will not fix" right?
I connected with the friendly pqos folks on this issue. They subsequently did an audit of
the tool to understand how it may behave under the different scenarios. They found that
the tool currently parses text return values ("Unavailable", "Unassigned") as zero.
This could cause incorrect behavior. (more below)
I mean, the pqos tool shows 0.0 in the MBL/MBR columns now.
And is the
pqos -m all:[0-191]
invocation not something people would usually run?
ABMC enabled (mode claimed to be incompatible with pqos)
--------------------------------------------------------
This example is from Babu's email that highlights that when ABMC is enabled this returns
zero as bandwidth from pqos perspective. This matches the pqos audit that found with text return
states, like the "Unassigned" happening underneath the output above, the return value is zero. When
the counters are not assigned then the events will always return "Unassigned" (until reassigned)
so having it return zero all the time may not cause issues. Once reassigned the event value
will always return a valid value (no text return values).
ABMC disabled (mode claimed to be compatible with pqos)
-------------------------------------------------------
Without ABMC there are scenarios when events may return "Unavailable". These scenarios vary
based on how many monitor groups are created and the workloads run. I am not able to test this
but when I attempt to combine what I know about AMD bandwidth monitoring with the results from
the pqos audit I am concerned that there may be a problem.
Consider a scenario where an event may return "Unavailable", for example:
<return A>, <return "Unavailable">, <return B (larger than A)>, <return "Unavailable">, ...
pqos will see:
A, 0, B, 0, ...
The "pqos -m" usage attempts to determine the bandwidth rate and, for example, when going from
"A" to "0" it will be considered counter wraparound that would appear as a very large and
wrong bandwidth number.
Yes. Agree. This is known issue.
Has this been reported? I do not recognize it in https://github.com/intel/intel-cmt-cat/issues
and the pqos folks I spoke with was not aware of this behavior.
I was not able to recreate this issue. I just opened for the one I have seen [1].
So to summarize: it is known that pqos returns zero instead of Unavailable but it is not reproducible?
Question to Babu
----------------
Do you perhaps have a test/workloads that, under "default" counter assignment mode, can create
many monitor groups (more than 64) and cause "Unavailable" to be returned frequently? Would it
be possible to put pqos through its paces in this environment?
Yes, I tried to reproduce the "Unavailable" issue with the pqos tool.
So far, I have not been able to observe the issue. I created 64
monitoring groups, ran MLC, and simultaneously executed pqos.
If I understand correctly 64 monitor groups will still have enough underlying hardware counters
and always return event counts, never return "Unavailable". It is when the number of
monitor groups are 65 or more that the hardware counters will start to be re-assigned, no?
Yes. I created 64 monitoring groups and then ran pqos, which creates an additional 15 monitoring groups.
Even so, I do not know if 65 would easily trigger the issue ... sounds like it is possible to create
many more (4096) monitor groups on these systems so there appears to be some room to make it easier to
create scenario where "Unavailable" is returned.
On a higher level: could you please run the test that can expose pqos to environment where it will
encounter "Unavailable" and observe how that is handled?
I was able to reproduce the condition where the hardware reports "Unavailable" for some of the MBM counters.
#cat /sys/fs/resctrl/mon_groups/dir*/mon_data/mon_L3_00/mbm_*
Unavailable
Unavailable
Unavailable
Unavailable
Unavailable
Unavailable
Thank you for running this.
Do all of these monitor groups have workloads assigned that generate memory bandwidth so
that they do not just keep returning "Unavailable" but often return data also? I just want
to confirm that this is not just a variant of the "Unassigned" test case.
However, the pqos tool continues to operate normally and reports valid bandwidth values:
#sudo pqos -m all:[0-191]
CORE IPC MISSES LLC[KB] MBL[MB/s] MBR[MB/s]
0-191 2.55 55146k 74496.0 20.0 61.4
The issue has not reproduced in any of my tests so far.
I'm confused. It reads like the issue that you mention earlier is a known issue cannot be
reproduced.
I meant that the pqos tool continued to work as expected even after
I recreated the condition that caused the hardware to report
"Unavailable".
ack.
This might require specific test scenario to recreate.
Right.
Of course, "AMBC disabled" scenario also includes AMD systems before ABMC was introduced. I do
not know why the issue with "Unavailable" handling (text returns in general) has not been reported
until now. I assume the pqos team may do most testing on Intel systems that do not return text.
...
The patch notes mention that this change is planned to be reverted at some
point in the future and I have not heard of any changes to this plan.
Where does it say that? I don't see that aspect.
For convenience, copy of patch notes from original commit [1] is below:
There are plans to enable "mbm_event" by default once additional counters
are available. For now, keep the default mode to maintain compatibility
with existing tools.
I added the text below in the comments section (after ---).
On second thought, that was probably my mistake, and I don't think I
should have added it. I also don't see this happening anytime soon.
ok.
The original problem statement is that pqos is not able to handle "Unassigned" return values
Considering all of this, I believe it would make sense to make the
"default" mode the preferred mode.
that will be encountered when "assignable counter mode" is enabled.
The request is to disable "assignable counter mode" by default to address the problem
when handling "Unassigned" counters.
Disabling "assignable counter mode" will under certain circumstances cause "Unavailable" to
be returned on event read.
At this time there seems to be a mismatch in understanding how pqos can handle
"Unavailable" return values. For me to sign off on this patch I would like to fully understand
the risk doing so. Could you please confirm from your side that pqos can handle all scenarios of
the mode that you request to be the default mode?
I created the maximum number of monitoring groups supported by the
system and was able to reproduce the condition where the hardware
reports "Unavailable". Even under those conditions, the pqos tool
appeared to function correctly and continued to report valid results.
Could you please share the test details? Just having the maximum number of
monitoring groups would not trigger the issue by itself, it needs workloads
that generate memory traffic that intermittently do return data.
ack. "We returned wrong data before. We can do better now but we want to
It's worth keeping in mind that ABMC mode is not available on the
majority of hardware currently accessible to public.
Furthermore, default mode has been the longstanding behavior in
deployed systems. Moving back to the default mode is therefore not a
new change, but a return to the behavior that users are already
familiar with.
keep returning wrong data since that is what users are used to".
I think we are getting nowhere here.
Thanks
Babu