Re: [RFC PATCH v2] membarrier: expedited private command

[Nicholas Piggin]

On Sat, 29 Jul 2017 11:23:33 +0200
Peter Zijlstra <peterz@xxxxxxxxxxxxx> wrote:

> On Sat, Jul 29, 2017 at 11:58:40AM +1000, Nicholas Piggin wrote:
> > I haven't had time to read the thread and understand exactly why you need
> > this extra barrier, I'll do it next week. Thanks for cc'ing us on it.  
> 
> Bottom of here:
> 
> https://lkml.kernel.org/r/20170727135610.jwjfvyuacqzj5e4u@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> 
> is probably the fastest way towards understanding the need for a barrier
> after rq->curr assignment.
> 
> Any barrier after that assignment is good for us, but so far it looks
> like PPC doesn't (and PPC only afaict) provide any smp_mb() after that
> point.

Thanks, yeah that's relatively straightforward.

> > A smp_mb is pretty expensive on powerpc CPUs. Removing the sync from
> > switch_to increased thread switch performance by 2-3%. Putting it in
> > switch_mm may be a little less painful, but still we have to weigh it
> > against the benefit of this new functionality. Would that be a net win
> > for the average end-user? Seems unlikely.
> > 
> > But we also don't want to lose sys_membarrier completely. Would it be too
> > painful to make  MEMBARRIER_CMD_PRIVATE_EXPEDITED return error, or make it
> > fall back to a slower case if we decide not to implement it?  
> 
> One ugly thing we've thought of is tagging each mm that has used
> sys_membarrier() and only issue the smp_mb() for those. That way only
> those tasks that actually rely on the syscall get to pay the price.

The biggest hammer that puts everything on the syscall side I think would
be to lock each runqueue to while iterating over them, right?

That could be pretty expensive but it would be interesting to know how
bad that is for real apps... hmm, we might be able to restrict iteration
to mm_cpumask(current->mm), no?