RE: [PATCH v11 01/13] x86/resctrl: Replace smp_call_function_many() with on_each_cpu_mask()

[Moger, Babu]

[AMD Official Use Only - General]

Hi Ashok,

> -----Original Message-----
> From: Ashok Raj <ashok_raj@xxxxxxxxxxxxxxx>
> Sent: Monday, January 9, 2023 5:27 PM
> To: Moger, Babu <Babu.Moger@xxxxxxx>
> Cc: corbet@xxxxxxx; reinette.chatre@xxxxxxxxx; tglx@xxxxxxxxxxxxx;
> mingo@xxxxxxxxxx; bp@xxxxxxxxx; fenghua.yu@xxxxxxxxx;
> dave.hansen@xxxxxxxxxxxxxxx; x86@xxxxxxxxxx; hpa@xxxxxxxxx;
> paulmck@xxxxxxxxxx; akpm@xxxxxxxxxxxxxxxxxxxx; quic_neeraju@xxxxxxxxxxx;
> rdunlap@xxxxxxxxxxxxx; damien.lemoal@xxxxxxxxxxxxxxxxxx;
> songmuchun@xxxxxxxxxxxxx; peterz@xxxxxxxxxxxxx; jpoimboe@xxxxxxxxxx;
> pbonzini@xxxxxxxxxx; chang.seok.bae@xxxxxxxxx;
> pawan.kumar.gupta@xxxxxxxxxxxxxxx; jmattson@xxxxxxxxxx;
> daniel.sneddon@xxxxxxxxxxxxxxx; Das1, Sandipan <Sandipan.Das@xxxxxxx>;
> tony.luck@xxxxxxxxx; james.morse@xxxxxxx; linux-doc@xxxxxxxxxxxxxxx;
> linux-kernel@xxxxxxxxxxxxxxx; bagasdotme@xxxxxxxxx; eranian@xxxxxxxxxx;
> christophe.leroy@xxxxxxxxxx; jarkko@xxxxxxxxxx; adrian.hunter@xxxxxxxxx;
> quic_jiles@xxxxxxxxxxx; peternewman@xxxxxxxxxx; Ashok Raj
> <ashok.raj@xxxxxxxxx>
> Subject: Re: [PATCH v11 01/13] x86/resctrl: Replace smp_call_function_many()
> with on_each_cpu_mask()
> 
> On Mon, Jan 09, 2023 at 10:43:53AM -0600, Babu Moger wrote:
> > on_each_cpu_mask() runs the function on each CPU specified by cpumask,
> > which may include the local processor.
> >
> > Replace smp_call_function_many() with on_each_cpu_mask() to simplify
> > the code.
> >
> > Reviewed-by: Reinette Chatre <reinette.chatre@xxxxxxxxx>
> > Signed-off-by: Babu Moger <babu.moger@xxxxxxx>
> > ---
> >  arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 11 +++------
> >  arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 29 +++++++----------------
> >  2 files changed, 11 insertions(+), 29 deletions(-)
> >
> > diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
> > b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
> > index 1df0e3262bca..7eece3d2d0c3 100644
> > --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
> > +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
> > @@ -310,7 +310,6 @@ int resctrl_arch_update_domains(struct rdt_resource
> *r, u32 closid)
> >  	enum resctrl_conf_type t;
> >  	cpumask_var_t cpu_mask;
> >  	struct rdt_domain *d;
> > -	int cpu;
> >  	u32 idx;
> >
> >  	if (!zalloc_cpumask_var(&cpu_mask, GFP_KERNEL)) @@ -341,13
> +340,9 @@
> > int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)
> >
> >  	if (cpumask_empty(cpu_mask))
> >  		goto done;
> > -	cpu = get_cpu();
> > -	/* Update resource control msr on this CPU if it's in cpu_mask. */
> > -	if (cpumask_test_cpu(cpu, cpu_mask))
> > -		rdt_ctrl_update(&msr_param);
> > -	/* Update resource control msr on other CPUs. */
> > -	smp_call_function_many(cpu_mask, rdt_ctrl_update, &msr_param, 1);
> > -	put_cpu();
> > +
> > +	/* Update resource control msr on all the CPUs. */
> > +	on_each_cpu_mask(cpu_mask, rdt_ctrl_update, &msr_param, 1);
> 
> Do you require these updates to done immediately via an IPI? or can they be
> done bit lazy via schedule_on_each_cpu()?

I have not experimented with lazy schedule.  At least I know the call update_cpu_closid_rmid should be completed immediately. Otherwise, the result might be inconsistent as the tasks(or CPUs)  could be running on two different closed/rmids before it is updated on all CPUs in the domain.
Thanks
Babu