[PATCH v3 0/3] x86,fs/resctrl: Keep default MBM mode at boot and fix ABMC

[Babu Moger]

Hi All,

This series restores default MBM assignment mode at boot, assigns counters
when switching to mbm_event, and fixes ABMC programming for more than 32
counters.

Commit 0f1576e43adc ("x86/resctrl: Configure mbm_event mode if supported")
enabled ABMC at boot. That breaks the pqos tool from intel-cmt-cat [1],
which treats a non-numeric event read as zero bandwidth. pqos creates 16
or more groups and uses two counters per group. On a platform with 32
counters per domain, that consumes the pool, so further groups read
"Unassigned" and pqos reports 0 MB/s.

Leaving the mode at "default" does not make pqos correct. pqos still
treats "Unavailable" as zero. Once the default-mode counter pool is
exceeded, hardware can re-allocate a counter between reads and pqos can
report that as a wraparound. That shows up only after the pool is
exceeded. Default mode uses one counter per monitoring group. On existing
AMD platforms that pool is 64 counters and may be larger on newer
hardware, so common deployments stay within it. mbm_event mode uses two
counters per group, so the same 32 counters cover only 16 groups.

mbm_event remains available as an opt-in. Users that need stable
measurements beyond the default-mode pool should enable it and rotate
assignments. The limitation of default mode is documented.

Patch 1: widen cntr_id in L3_QOS_ABMC_CFG from 5 bits to 12 bits, and
limit the enumerated counter count to 4096, which is what that field can
encode. The APM [2] at [3] will be updated for the expanded cntr_id field.

Patch 2: when switching to mbm_event, assign counters to existing
groups, including the default group, matching mkdir auto-assignment.

Patch 3: leave mbm_assign_mode as "default" at boot. This seems to be
the patch that has generated the most discussion. Let me know, if additional
justification in the documentation would be helpful, I'd be happy to add it.

Based on tip/master:

  1aeb52f7869a Merge branch into tip/master: 'x86/tdx'

[1] https://github.com/intel/intel-cmt-cat/issues/311
[2] AMD64 Architecture Programmer's Manual Volume 2: System Programming,
    Publication #24593, Revision 3.41, Section 19.3.3.3 "Assignable
    Bandwidth Monitoring (ABMC)"
[3] https://bugzilla.kernel.org/show_bug.cgi?id=206537

v3:
- Patch 1: drop the 32-bit u64 change and the bw_src widening. Cap the
  enumerated counter count at 4096 instead of truncating the CPUID field.
- Patch 2: clean up comments. Add Reported-by, Closes, and Cc: stable.
- Patch 3: document one counter per group in default mode and two counters
  per group in mbm_event mode. Describe the default-mode pool and the pqos
  wraparound once that pool is exceeded.

v2:
- Added patch 2 to address Sashiko's comment regarding the documentation
  issue. In fact, it exposed a real issue. When switching to mbm_event mode,
  existing monitoring groups should be assigned counters whenever counters
  are available. This provides a smooth transition between modes and aligns
  the behavior with the existing auto-assignment mechanism.
  https://sashiko.dev/#/patchset/8cb66e18e32e4087a9712c1e68ee6da614efe244.1784322818.git.babu.moger%40amd.com

- Combine the two v1 patches and add patch 2 (assign existing groups
  on mbm_event switch; Sashiko review).
- Keep boot-default separate from the encoding/truncation fixes.
- Document the default-mode counter-pool limitation.

v1: https://lore.kernel.org/lkml/980f39d3a0e0d9f73925e362f835aeef070a1bc5.1784322818.git.babu.moger@xxxxxxx/
v2: https://lore.kernel.org/lkml/cover.1788545152.git.babu.moger@xxxxxxx/

Thanks
Babu Moger

Babu Moger (3):
  x86/resctrl: Fix ABMC counter programming
  fs/resctrl: Assign counters to existing groups when enabling mbm_event
  x86,fs/resctrl: Keep mbm_assign_mode in default mode at boot

 Documentation/filesystems/resctrl.rst  | 57 ++++++++++++++++++--------
 arch/x86/kernel/cpu/resctrl/internal.h |  4 +-
 arch/x86/kernel/cpu/resctrl/monitor.c  |  4 +-
 fs/resctrl/monitor.c                   | 35 +++++++++++++---
 4 files changed, 73 insertions(+), 27 deletions(-)

-- 
2.43.0