Re: Unlock-lock questions and the Linux Kernel Memory Model

From: Paul E. McKenney
Date: Thu Nov 30 2017 - 11:54:48 EST


On Thu, Nov 30, 2017 at 04:41:05PM +0000, Will Deacon wrote:
> On Thu, Nov 30, 2017 at 08:14:01AM -0800, Paul E. McKenney wrote:
> > On Thu, Nov 30, 2017 at 10:20:02AM -0500, Alan Stern wrote:
> > > On Wed, 29 Nov 2017, Daniel Lustig wrote:
> > >
> > > > On 11/29/2017 12:42 PM, Paul E. McKenney wrote:
> > > > > On Wed, Nov 29, 2017 at 02:53:06PM -0500, Alan Stern wrote:
> > > > >> On Wed, 29 Nov 2017, Peter Zijlstra wrote:
> > > > >>
> > > > >>> On Wed, Nov 29, 2017 at 11:04:53AM -0800, Daniel Lustig wrote:
> > > > >>>
> > > > >>>> While we're here, let me ask about another test which isn't directly
> > > > >>>> about unlock/lock but which is still somewhat related to this
> > > > >>>> discussion:
> > > > >>>>
> > > > >>>> "MP+wmb+xchg-acq" (or some such)
> > > > >>>>
> > > > >>>> {}
> > > > >>>>
> > > > >>>> P0(int *x, int *y)
> > > > >>>> {
> > > > >>>> WRITE_ONCE(*x, 1);
> > > > >>>> smp_wmb();
> > > > >>>> WRITE_ONCE(*y, 1);
> > > > >>>> }
> > > > >>>>
> > > > >>>> P1(int *x, int *y)
> > > > >>>> {
> > > > >>>> r1 = atomic_xchg_relaxed(y, 2);
> > > > >>>> r2 = smp_load_acquire(y);
> > > > >>>> r3 = READ_ONCE(*x);
> > > > >>>> }
> > > > >>>>
> > > > >>>> exists (1:r1=1 /\ 1:r2=2 /\ 1:r3=0)
> > > > >>>>
> > > > >>>> C/C++ would call the atomic_xchg_relaxed part of a release sequence
> > > > >>>> and hence would forbid this outcome.
> > > > >>>
> > > > >>> That's just weird. Either its _relaxed, or its _release. Making _relaxed
> > > > >>> mean _release is just daft.
> > > > >>
> > > > >> The C11 memory model specifically allows atomic operations to be
> > > > >> interspersed within a release sequence. But it doesn't say why.
> > > > >
> > > > > The use case put forward within the committee is for atomic quantities
> > > > > with mode bits. The most frequent has the atomic quantity having
> > > > > lock-like properties, in which case you don't want to lose the ordering
> > > > > effects of the lock handoff just because a mode bit got set or cleared.
> > > > > Some claim to actually use something like this, but details have not
> > > > > been forthcoming.
> > > > >
> > > > > I confess to being a bit skeptical. If the mode changes are infrequent,
> > > > > the update could just as well be ordered.
> > > >
> > > > Aren't reference counting implementations which use memory_order_relaxed
> > > > for incrementing the count another important use case? Specifically,
> > > > the synchronization between a memory_order_release decrement and the
> > > > eventual memory_order_acquire/consume free shouldn't be interrupted by
> > > > other (relaxed) increments and (release-only) decrements that happen in
> > > > between. At least that's my understanding of this use case. I wasn't
> > > > there when the C/C++ committee decided this.
> > > >
> > > > > That said, Daniel, the C++ memory model really does require that the
> > > > > above litmus test be forbidden, my denigration of it notwithstanding.
> > > >
> > > > Yes I agree, that's why I'm curious what the Linux memory model has
> > > > in mind here :)
> > >
> > > Bear in mind that the litmus test above uses xchg, not increment or
> > > decrement. This makes a difference as far as the LKMM is concerned,
> > > even if not for C/C++.
> >
> > Finally remembering this discussion... Yes, xchg is special. ;-)
> >
> > Will, are there plans to bring this sort of thing before the standards
> > committee?
>
> We discussed it, but rejected it mainly because of concerns that there could
> be RmW operations that don't necessarily have an order-inducing dependency
> in all scenarios. I think the case that was batted around was a saturating
> add implemented using cmpxchg.

Ah, I do remember now, during the Toronto meeting, correct?

So should we consider making LKMM make xchg act in a manner similar to
the other atomics, or would you prefer that we keep the current special
behavior?

Thanx, Paul