Re: wake_q memory ordering

[Davidlohr Bueso]

On Thu, 10 Oct 2019, Peter Zijlstra wrote:

> On Thu, Oct 10, 2019 at 02:13:47PM +0200, Manfred Spraul wrote:
> > Hi Peter,
> >
> > On 10/10/19 1:42 PM, Peter Zijlstra wrote:
> > > On Thu, Oct 10, 2019 at 12:41:11PM +0200, Manfred Spraul wrote:
> > > > Hi,
> > > >
> > > > Waiman Long noticed that the memory barriers in sem_lock() are not really
> > > > documented, and while adding documentation, I ended up with one case where
> > > > I'm not certain about the wake_q code:
> > > >
> > > > Questions:
> > > > - Does smp_mb__before_atomic() + a (failed) cmpxchg_relaxed provide an
> > > >    ordering guarantee?
> > > Yep. Either the atomic instruction implies ordering (eg. x86 LOCK
> > > prefix) or it doesn't (most RISC LL/SC), if it does,
> > > smp_mb__{before,after}_atomic() are a NO-OP and the ordering is
> > > unconditinoal, if it does not, then smp_mb__{before,after}_atomic() are
> > > unconditional barriers.
> >
> > And _relaxed() differs from "normal" cmpxchg only for LL/SC architectures,
> > correct?
>
> Indeed.
>
> > Therefore smp_mb__{before,after}_atomic() may be combined with
> > cmpxchg_relaxed, to form a full memory barrier, on all archs.
>
> Just so.

We might want something like this?

----8<---------------------------------------------------------

From: Davidlohr Bueso <dave@xxxxxxxxxxxx>
Subject: [PATCH] Documentation/memory-barriers.txt: Mention smp_mb__{before,after}_atomic() and CAS

Explicitly mention possible usages to guarantee serialization even upon
failed cmpxchg (or similar) calls along with smp_mb__{before,after}_atomic().

Signed-off-by: Davidlohr Bueso <dbueso@xxxxxxx>
---
Documentation/memory-barriers.txt | 12 ++++++++++++
1 file changed, 12 insertions(+)

diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt
index 1adbb8a371c7..5d2873d4b442 100644
--- a/Documentation/memory-barriers.txt
+++ b/Documentation/memory-barriers.txt
@@ -1890,6 +1890,18 @@ There are some more advanced barrier functions:
     This makes sure that the death mark on the object is perceived to be set
     *before* the reference counter is decremented.

+     Similarly, these barriers can be used to guarantee serialization for atomic
+     RMW calls on architectures which may not imply memory barriers upon failure.
+
+	obj->next = NULL;
+	smp_mb__before_atomic()
+	if (cmpxchg(&obj->ptr, NULL, val))
+		return;
+
+     This makes sure that the store to the next pointer always has smp_store_mb()
+     semantics. As such, smp_mb__{before,after}_atomic() calls allow optimizing
+     the barrier usage by finer grained serialization.
+
     See Documentation/atomic_{t,bitops}.txt for more information.


--
2.16.4