Re: [PATCH tip/core/rcu 11/17] rcu: Bind grace-period kthreads to non-NO_HZ_FULL CPUs

[Frederic Weisbecker]

On Fri, Jul 11, 2014 at 12:43:14PM -0700, Paul E. McKenney wrote:
> On Fri, Jul 11, 2014 at 09:26:14PM +0200, Frederic Weisbecker wrote:
> > On Fri, Jul 11, 2014 at 12:08:16PM -0700, Paul E. McKenney wrote:
> > > On Fri, Jul 11, 2014 at 08:57:33PM +0200, Frederic Weisbecker wrote:
> > > > On Fri, Jul 11, 2014 at 11:45:28AM -0700, Paul E. McKenney wrote:
> > > > > On Fri, Jul 11, 2014 at 08:25:43PM +0200, Frederic Weisbecker wrote:
> > > > > > On Fri, Jul 11, 2014 at 01:10:41PM -0500, Christoph Lameter wrote:
> > > > > > > On Tue, 8 Jul 2014, Frederic Weisbecker wrote:
> > > > > > > 
> > > > > > > > > I was figuring that a fair number of the kthreads might eventually
> > > > > > > > > be using this, not just for the grace-period kthreads.
> > > > > > > >
> > > > > > > > Ok makes sense. But can we just rename the cpumask to housekeeping_mask?
> > > > > > > 
> > > > > > > That would imply that all no-nohz processors are housekeeping? So all
> > > > > > > processors with a tick are housekeeping?
> > > > > > 
> > > > > > Well, now that I think about it again, I would really like to keep housekeeping
> > > > > > to CPU 0 when nohz_full= is passed.
> > > > > 
> > > > > When CONFIG_NO_HZ_FULL_SYSIDLE=y, then housekeeping kthreads are bound to
> > > > > CPU 0.  However, doing this causes significant slowdowns according to
> > > > > Fengguang's testing, so when CONFIG_NO_HZ_FULL_SYSIDLE=n, I bind the
> > > > > housekeeping kthreads to the set of non-nohz_full CPUs.
> > > > 
> > > > But did he see these slowdowns with nohz_full= parameter passed? I doubt he
> > > > tested that. And I'm not sure that people who need full dynticks will run
> > > > the usecases that trigger slowdowns with grace period kthreads.
> > > > 
> > > > I also doubt that people will often omit other CPUs than CPU 0 nohz_full=
> > > > range.
> > > 
> > > Agreed, this is only a problem when people run workloads for which
> > > NO_HZ_FULL is not well-suited.  Which is why I settled on designating
> > > the non-nohz_full= CPUs as the housekeeping CPUs -- people wanting to
> > > run general workloads not suited to NO_HZ_FULL probably won't specify
> > > nohz_full=.  If they don't, then any CPU can be a housekeeping CPU.
> > 
> > Right. So affining GP kthread to all non-nohz-full CPU works in all case. It's convenient
> > but it requires some plumbing:
> > 
> > * add a housekeeping cpumask and implement housekeeping_affine on top
> > * add kthread_bind_cpumask()
> 
> Yep.
> 
> > So what I propose is to skip these complications and just do:
> > 
> >         if (tick_nohz_full_enabled()) // means that somebody passed nohz_full= kernel parameter
> >             kthread_bind_cpu(GP kthread, 0)
> > 
> > Moreover Thomas didn't like the idea of extending housekeeping duty further CPU 0, arguing that
> > it's too early for that. He meant that for timekeeping but the idea is expandable.
> 
> Although I agree that we can get away with a single timekeeping CPU, I
> don't believe that we get away with having only a single housekeeping CPU.

Ok, well I won't insist too much. As long as the performance issue is fixed, I'm ok :)

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/