Re: [PATCH v20 07/18] x86/resctrl: Block use of mba_MBps mount option on Sub-NUMA Cluster (SNC) systems

From: Tony Luck
Date: Fri Jun 21 2024 - 11:25:40 EST


On Thu, Jun 20, 2024 at 06:56:56PM -0700, Reinette Chatre wrote:
> Hi Tony,
>
> On 6/20/24 3:07 PM, Luck, Tony wrote:
> > > > When SNC is enabled there is a mismatch between the MBA control function
> > > > which operates at L3 cache scope and the MBM monitor functions which
> > > > measure memory bandwidth on each SNC node.
> > > >
> > > > Block use of the mba_MBps when scopes for MBA/MBM do not match.
> > > >
> > > > Signed-off-by: Tony Luck <tony.luck@xxxxxxxxx>
> > > > ---
> > > > arch/x86/kernel/cpu/resctrl/rdtgroup.c | 4 +++-
> > > > 1 file changed, 3 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > > > index eb3bbfa96d5a..a0a43dbe011b 100644
> > > > --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > > > +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> > > > @@ -2339,10 +2339,12 @@ static void mba_sc_domain_destroy(struct rdt_resource *r,
> > > > */
> > > > static bool supports_mba_mbps(void)
> > > > {
> > > > + struct rdt_resource *rmbm = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
> > > > struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_MBA].r_resctrl;
> > > >
> > > > return (is_mbm_local_enabled() &&
> > > > - r->alloc_capable && is_mba_linear());
> > > > + r->alloc_capable && is_mba_linear() &&
> > > > + r->ctrl_scope == rmbm->mon_scope);
> > > > }
> > > >
> > > > /*
> > >
> > > The function comments of supports_mba_mbps() needs an update to accompany
> > > this new requirement.
> >
> > Will add comment on extra requirement.
> >
> > > I also think that the "mba_MBps" mount option is now complicated enough to
> > > warrant a clear error to user space when using it fails. invalfc() is
> > > available for this and enables user space to get detailed log message
> > > from a read() on an fd created by fsopen().
> > >
> > > Perhaps something like (please check line length and feel free to improve
> > > since as is it may quite cryptic):
> > > rdt_parse_param(...)
> > > {
> > >
> > >
> > > ...
> > > case Opt_mba_mbps:
> > > if (!supports_mba_mbps())
> > > return invalfc(fc, "mba_MBps requires both MBM and (linear scale) MBA at L3 scope");
> > > ...
> > > }
> >
> > Line length is indeed a problem (108 characters). Usual line split methods barely help as the moving the
> > string to the next line and aligning with the "(" only saves 4 characters.
> >
> > How about this (suggestions for a shorter variable name - line is 97 characters)
> >
> > static char mba_mbps_invalid[] = "mba_MBps requires both MBM and (linear scale) MBA at L3 scope";
> >
> > rdt_parse_param(...)
> > {
> > ...
> > case Opt_mba_mbps:
> > if (!supports_mba_mbps())
> > return invalfc(fc, mba_mbps_invalid);
> > ...
> > }
>
> On 6/20/24 3:12 PM, Luck, Tony wrote:
> > > static char mba_mbps_invalid[] = "mba_MBps requires both MBM and (linear scale) MBA at L3 scope";
> >
> > checkpatch recommends "static const char ..." pushing this over 100 chars :-(
> >
>
> How about something like below that reaches 96:
>
> case Opt_mba_mbps:
> if (!supports_mba_mbps())
> return invalfc(fc,
> "mba_MBps requires both MBM and linear MBA at L3 scope");
>

Reinette,

Alternative option. Move the messaging into supports_mba_mbps() and
split into shorter pieces for each reason. The other callers of
supports_mba_mbps() that are just re-checking status would pass
a NULL argument.

If this looks reasonable I can do it in two patches. First to add
invalfc() for the existing cases. Second to add the SNC change.

-Tony

---

static bool supports_mba_mbps(struct fs_context *fc)
{
return invalfc(fc, mba_mbps_invalid);
struct rdt_resource *rmbm = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_MBA].r_resctrl;

if (!is_mbm_local_enabled()) {
if (fc)
invalfc(fc, "mba_MBps requires local MBM");
return false;
}
if (!r->alloc_capable) {
if (fc)
invalfc(fc, "mba_MBps requires MBA");
return false;
}
if (!is_mba_linear()) {
if (fc)
invalfc(fc, "mba_MBps requires linear MBA");
return false;
}
if (r->ctrl_scope != rmbm->mon_scope) {
if (fc)
invalfc(fc, "mba_MBps requires MBM/MBA at L3 scope");
return false;
}

return true;
}

rdt_parse_param(...)
{
...
case Opt_mba_mbps:
if (!supports_mba_mbps(fc))
return -EINVAL;
...
}