Re: [PATCH v2] membarrier: provide register sync core cmd

[Mathieu Desnoyers]

----- On Aug 30, 2017, at 10:46 AM, Paul E. McKenney paulmck@xxxxxxxxxxxxxxxxxx wrote:

> On Tue, Aug 29, 2017 at 10:01:56PM -0700, Andy Lutomirski wrote:
>> > On Aug 27, 2017, at 8:05 PM, Mathieu Desnoyers <mathieu.desnoyers@xxxxxxxxxxxx>
>> > wrote:
>> >
>> > ----- On Aug 27, 2017, at 3:53 PM, Andy Lutomirski luto@xxxxxxxxxxxxxx wrote:
>> >
>> >>> On Aug 27, 2017, at 1:50 PM, Mathieu Desnoyers <mathieu.desnoyers@xxxxxxxxxxxx>
>> >>> wrote:
>> >>>
>> >>> Add a new MEMBARRIER_CMD_REGISTER_SYNC_CORE command to the membarrier
>> >>> system call. It allows processes to register their intent to have their
>> >>> threads issue core serializing barriers in addition to memory barriers
>> >>> whenever a membarrier command is performed.
>> >>>
>> >>
>> >> Why is this stateful?  That is, why not just have a new membarrier command to
>> >> sync every thread's icache?
>> >
>> > If we'd do it on every CPU icache, it would be as trivial as you say. The
>> > concern here is sending IPIs only to CPUs running threads that belong to the
>> > same process, so we don't disturb unrelated processes.
>> >
>> > If we could just grab each CPU's runqueue lock, it would be fairly simple
>> > to do. But we want to avoid hitting each runqueue with exclusive atomic
>> > access associated with grabbing the lock. (cache-line bouncing)
>> 
>> Hmm.  Are there really arches where there is no clean implementation
>> without this hacker?  It seems rather unfortunate that munmap() can be
>> done efficiently but this barrier can't be.
>> 
>> At the very least, could there be a register command *and* a special
>> sync command?  I dislike the idea that the sync command does something
>> different depending on some other state.  Even better (IMO) would be a
>> design where you ask for an isync and, if the arch can do it
>> efficiently (x86), you get an efficient isync and, if the arch can't
>> (arm64?) you take all the rq locks?
> 
> In some cases I suspect that IPIs might be required.  Regardless of
> that, we might well need to provide a way for architectures to do
> special things.
> 
> But I must defer to Mathieu on this.

Yes, I think you are both correct. It's better to expose a new command
for code sync, so architectures have more freedom in how they choose to
implement it.

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com