Re: [PATCH] sched: Optimize housekeeping_cpumask in for_each_cpu_and

[Peter Zijlstra]

On Thu, May 27, 2021 at 11:40:42AM +0200, Peter Zijlstra wrote:
> On Sat, Apr 17, 2021 at 11:01:37PM +0800, Yuan ZhaoXiong wrote:
> > On a 128 cores AMD machine, there are 8 cores in nohz_full mode, and
> > the others are used for housekeeping. When many housekeeping cpus are
> > in idle state, we can observe huge time burn in the loop for searching
> > nearest busy housekeeper cpu by ftrace.
> > 
> >    9)               |              get_nohz_timer_target() {
> >    9)               |                housekeeping_test_cpu() {
> >    9)   0.390 us    |                  housekeeping_get_mask.part.1();
> >    9)   0.561 us    |                }
> >    9)   0.090 us    |                __rcu_read_lock();
> >    9)   0.090 us    |                housekeeping_cpumask();
> >    9)   0.521 us    |                housekeeping_cpumask();
> >    9)   0.140 us    |                housekeeping_cpumask();
> > 
> >    ...
> > 
> >    9)   0.500 us    |                housekeeping_cpumask();
> >    9)               |                housekeeping_any_cpu() {
> >    9)   0.090 us    |                  housekeeping_get_mask.part.1();
> >    9)   0.100 us    |                  sched_numa_find_closest();
> >    9)   0.491 us    |                }
> >    9)   0.100 us    |                __rcu_read_unlock();
> >    9) + 76.163 us   |              }
> > 
> > for_each_cpu_and() is a micro function, so in get_nohz_timer_target()
> > function the
> >         for_each_cpu_and(i, sched_domain_span(sd),
> >                 housekeeping_cpumask(HK_FLAG_TIMER))
> > equals to below:
> >         for (i = -1; i = cpumask_next_and(i, sched_domain_span(sd),
> >                 housekeeping_cpumask(HK_FLAG_TIMER)), i < nr_cpu_ids;)
> > That will cause that housekeeping_cpumask() will be invoked many times.
> > The housekeeping_cpumask() function returns a const value, so it is
> > unnecessary to invoke it every time. This patch can minimize the worst
> > searching time from ~76us to ~16us in my testing.
> > 
> > Similarly, the find_new_ilb() function has the same problem.
> > 
> > Signed-off-by: Yuan ZhaoXiong <yuanzhaoxiong@xxxxxxxxx>
> > Signed-off-by: Li RongQing <lirongqing@xxxxxxxxx>
> 
> Just noticed, this SoB chain isn't valid. What do I do with Li's entry?

I'm dropping this patch, please resend with a valid SoB chain.