[RFC PATCH 5/6] fs/resctrl: Reject mba_MBps when the maximum is only a soft limit

[Jinjie Ruan]

From: Jinjie Ruan <ruanjinjie@xxxxxxxxxx>

The MBA software controller lowers the maximum bandwidth control when
the measured bandwidth is above the value the user writes to the
schemata file. It relies on the hardware enforcing that maximum.

A maximum bandwidth control can be implemented as a hard limit, where
requests above the maximum are not serviced, or as a soft limit, where
requests above the maximum are serviced when no other request is
waiting. With only the soft limit, a group that has no competing
traffic keeps using bandwidth above the maximum and the controller
cannot meet the target. MPAM reports this in MPAMF_MBW_IDR.MAX_LIM.

Add resctrl_membw::max_is_soft_limit for architectures whose maximum
control implements only the soft limit, and refuse to mount with the
mba_MBps option when it is set. Architectures that do not set the flag
keep the previous behavior.

Signed-off-by: Jinjie Ruan <ruanjinjie@xxxxxxxxxx>
---
 Documentation/filesystems/resctrl.rst | 6 +++++-
 fs/resctrl/rdtgroup.c                 | 7 ++++++-
 include/linux/resctrl.h               | 5 +++++
 3 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index d70aec6f7096..d2dacb00a848 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -47,7 +47,11 @@ mount options are:
 	Enable code/data prioritization in L2 cache allocations.
 "mba_MBps":
 	Enable the MBA Software Controller(mba_sc) to specify MBA
-	bandwidth in MiBps
+	bandwidth in MiBps. Refused when the maximum bandwidth control is
+	only a soft limit: requests above the maximum are then still
+	serviced when no other request is waiting, so the software
+	controller cannot keep the bandwidth below the value written to
+	the schemata file.
 "debug":
 	Make debug files accessible. Available debug files are annotated with
 	"Available only with debug option".
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 957a3df7af83..572b3d9c79e5 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
@@ -2614,6 +2614,10 @@ static void mba_sc_domain_destroy(struct rdt_resource *r,
  * With assignable counters that is only guaranteed when the architecture
  * uses a fixed assignment mode and assigns counters automatically; keep
  * that assignment enabled while the controller is in use.
+ *
+ * The maximum control must also be enforced by the hardware: a soft limit
+ * still services requests above the maximum when no other request is
+ * waiting, so the controller cannot keep bandwidth below a target.
  */
 static bool supports_mba_mbps(void)
 {
@@ -2623,6 +2627,7 @@ static bool supports_mba_mbps(void)
 	return (resctrl_is_mbm_enabled() &&
 		r->alloc_capable && is_mba_linear() &&
 		r->ctrl_scope == rmbm->mon_scope &&
+		!r->membw.max_is_soft_limit &&
 		(!rmbm->mon.mbm_cntr_assignable ||
 		 rmbm->mon.mbm_cntr_assign_fixed));
 }
@@ -3375,7 +3380,7 @@ static int rdt_parse_param(struct fs_context *fc, struct fs_parameter *param)
 		ctx->enable_cdpl2 = true;
 		return 0;
 	case Opt_mba_mbps:
-		msg = "mba_MBps requires MBM and linear scale MBA at L3 scope";
+		msg = "mba_MBps requires MBM, linear scale MBA at L3 scope and an enforced maximum";
 		if (!supports_mba_mbps())
 			return invalfc(fc, msg);
 		ctx->enable_mba_mbps = true;
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index 10dfdca7f4bf..d1f989554031 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -250,6 +250,10 @@ enum membw_throttle_mode {
  * @bw_gran:		Granularity at which the memory bandwidth is allocated
  * @delay_linear:	True if memory B/W delay is in linear scale
  * @arch_needs_linear:	True if we can't configure non-linear resources
+ * @max_is_soft_limit:	True if the maximum control only marks traffic as low
+ *			priority. Requests above the maximum are then still
+ *			serviced when no other request is waiting, so the software
+ *			controller cannot keep bandwidth below a target.
  * @throttle_mode:	Bandwidth throttling mode when threads request
  *			different memory bandwidths
  * @mba_sc:		True if MBA software controller(mba_sc) is enabled
@@ -261,6 +265,7 @@ struct resctrl_membw {
 	u32				bw_gran;
 	u32				delay_linear;
 	bool				arch_needs_linear;
+	bool				max_is_soft_limit;
 	enum membw_throttle_mode	throttle_mode;
 	bool				mba_sc;
 	u32				*mb_map;
-- 
2.43.0