Re: [PATCH] Fix: disable sys_membarrier when nohz_full is enabled

From: Mathieu Desnoyers
Date: Thu Nov 17 2016 - 12:28:17 EST

----- On Nov 17, 2016, at 9:50 AM, rostedt rostedt@xxxxxxxxxxx wrote:

> On Thu, 17 Nov 2016 13:54:27 +0000 (UTC)
> Mathieu Desnoyers <mathieu.desnoyers@xxxxxxxxxxxx> wrote:
>> >> >
>> >> > Acked-by: Lai Jiangshan <jiangshanlai@xxxxxxxxx>
>> >> >
>> >> > But I'm afraid, in the future, tick_nohz_full will become a default y
>> >> > feature. thus it makes sys_membarrier() always disabled. we might
>> >> > need a new MEMBARRIER_CMD_XXX to handle it?
>> >>
>> >> This may require that we send an IPI to nohz_full CPUs, which will
>> >> disturb them real-time wise. Any better ideas ?
>> >
>> > Restrict the IPIs to CPUs running the process executing the
>> > sys_membarrier() system call. This would mean that CPUs only
>> > are interrupted by their own application's request.
>> This would break use-cases of cross-process shared memory. :-(
> Perhaps make this an opt in. That is, all processes that want to be
> affected by this can call this function with some flag that sets a flag
> in tasks struct. And have that process get an IPI even in no-hz-full
> mode if it asked to do it.

That's an interesting approach. I would be tempted to give it a
per-thread (rather than per-process) scope.

E.g., a thread could do the following to ask to be
interrupted by IPIs:


and could unregister with:


We can then keep a per-thread refcount internally.
(not sure the "EXPEDITED" is the right word there...
do we want it to be "NOHZ_FULL" instead ?)

Then in membarrier(MEMBARRIER_CMD_SHARED, 0), for each
nohz_full cpu, we grab the rq lock, and only send an IPI
if the running thread is registered as "expedited".

Thoughts ?



Mathieu Desnoyers
EfficiOS Inc.