Re: [RFC PATCH] membarrier: handle nohz_full with expedited thread registration

[Paul McKenney]

On Wed, Jan 18, 2017 at 3:00 AM, Peter Zijlstra <peterz@xxxxxxxxxxxxx> wrote:
> On Tue, Jan 17, 2017 at 12:53:21PM -0800, Paul E. McKenney wrote:
>> On Tue, Jan 17, 2017 at 04:55:22AM +0100, Frederic Weisbecker wrote:
>>
>> [ . . . ]
>>
>> > In fact due to the complexity involved, I have to ask first if we
>> > really need this feature.  Typically nohz_full workloads don't want to
>> > be disturbed at all, so do we have real and significant usecases of CPU
>> > isolation workloads that want to be concerned by this membarrier so much
>> > that they can tolerate some random IRQ?
>>
>> I believe that we need to explore the options for implementing it and
>> to -at- -least- have a patch ready, even if that patch doesn't go
>> upstream immediately.
>
> I tend to agree with Frederic here in that the design requirements seem
> mutually exclusive.
>
> NOHZ_FULL users do _not_ want interruptions of any sort, in fact some
> want to make that a hard fail of the task.
>
> OTOH sys_membarrier(CMD_SHARED) promises to serialize against anything
> observable.
>
> The only logical solution is to error the sys_membarrier(CMD_SHARED)
> call when a NOHZ_FULL task shares memory with the caller. Now
> determining this is somewhat tricky of course :/
>
>
> I really don't see how there is another possible solution that makes
> sense here. If there is shared memory between a NOHZ_FULL task and
> others, a urcu implementation used by those must not rely on
> sys_membarrier() but instead use a more expensive one, for instance one
> where rcu_read_{,un}lock() do explicit counting and have memory barriers
> in.

Actually, agreed.  After all, if we knew of a solution that made
sense, we would have already implemented it, so some out-of-tree
experimentation makes sense.  One possibility is to have a flag that
user processes could set that aborted (or splatted or whatever) on any
interruption, so that tasks using signal URCU would avoid setting that
flag, and, as you say, tasks setting the flag using URCU would need to
use one of the memory-barrier variants.

                                                       Thanx, Paul