Re: [PATCH] doc: Update memory-barriers.txt for read-to-write dependencies

[Paul E. McKenney]

On Mon, Jul 03, 2017 at 02:07:03PM +0100, Will Deacon wrote:
> Hi Paul,
> 
> On Fri, Jun 30, 2017 at 04:28:10PM -0700, Paul E. McKenney wrote:
> > The memory-barriers.txt document contains an obsolete passage stating that
> > smp_read_barrier_depends() is required to force ordering for read-to-write
> > dependencies.  We now know that this is not required, even for DEC Alpha.
> > This commit therefore updates this passage to state that read-to-write
> > dependencies are respected even without smp_read_barrier_depends().
> > 
> > Reported-by: Lance Roy <ldr709@xxxxxxxxx>
> > Signed-off-by: Paul E. McKenney <paulmck@xxxxxxxxxxxxxxxxxx>
> > Cc: David Howells <dhowells@xxxxxxxxxx>
> > Cc: Will Deacon <will.deacon@xxxxxxx>
> > Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
> > Cc: Jonathan Corbet <corbet@xxxxxxx>
> > Cc: Alan Stern <stern@xxxxxxxxxxxxxxxxxxx>
> > Cc: Andrea Parri <parri.andrea@xxxxxxxxx>
> > Cc: Jade Alglave <j.alglave@xxxxxxxxx>
> > Cc: Luc Maranget <luc.maranget@xxxxxxxx>
> > 
> > diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt
> > index 9d5e0f853f08..a8a91b9d5a1b 100644
> > --- a/Documentation/memory-barriers.txt
> > +++ b/Documentation/memory-barriers.txt
> > @@ -594,7 +594,10 @@ between the address load and the data load:
> >  This enforces the occurrence of one of the two implications, and prevents the
> >  third possibility from arising.
> >  
> > -A data-dependency barrier must also order against dependent writes:
> > +A data-dependency barrier is not required to order dependent writes
> > +because the CPUs that the Linux kernel supports don't do writes until
> > +they are certain (1) that the write will actually happen, (2) of the
> > +location of the write, and (3) of the value to be written.
> 
> Might be worth mentioning that you have to careful with the compiler here,
> and pointing to the section on "Control dependencies" so that people don't
> just take these three points as guarantees in isolation.
> 
> >  
> >  	CPU 1		      CPU 2
> >  	===============	      ===============
> > @@ -603,19 +606,19 @@ A data-dependency barrier must also order against dependent writes:
> >  	<write barrier>
> >  	WRITE_ONCE(P, &B);
> >  			      Q = READ_ONCE(P);
> > -			      <data dependency barrier>
> >  			      *Q = 5;
> 
> Do we want that write to Q to be a WRITE_ONCE? Again, the control
> dependencies section does call this out.

Both good points!  Like this?

							Thanx, Paul

------------------------------------------------------------------------

commit 00269a0e23dbc50f1c4f101b23c8d74992eace05
Author: Paul E. McKenney <paulmck@xxxxxxxxxxxxxxxxxx>
Date:   Fri Jun 30 16:18:28 2017 -0700

    doc: Update memory-barriers.txt for read-to-write dependencies
    
    The memory-barriers.txt document contains an obsolete passage stating that
    smp_read_barrier_depends() is required to force ordering for read-to-write
    dependencies.  We now know that this is not required, even for DEC Alpha.
    This commit therefore updates this passage to state that read-to-write
    dependencies are respected even without smp_read_barrier_depends().
    
    Reported-by: Lance Roy <ldr709@xxxxxxxxx>
    Signed-off-by: Paul E. McKenney <paulmck@xxxxxxxxxxxxxxxxxx>
    Cc: David Howells <dhowells@xxxxxxxxxx>
    Cc: Will Deacon <will.deacon@xxxxxxx>
    Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
    Cc: Jonathan Corbet <corbet@xxxxxxx>
    Cc: Alan Stern <stern@xxxxxxxxxxxxxxxxxxx>
    Cc: Andrea Parri <parri.andrea@xxxxxxxxx>
    Cc: Jade Alglave <j.alglave@xxxxxxxxx>
    Cc: Luc Maranget <luc.maranget@xxxxxxxx>
    [ paulmck: Reference control-dependencies sections and use WRITE_ONCE()
      per Will Deacon.  Correctly place split-cache paragraph while there. ]

diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt
index 9d5e0f853f08..7be80911e502 100644
--- a/Documentation/memory-barriers.txt
+++ b/Documentation/memory-barriers.txt
@@ -594,7 +594,23 @@ between the address load and the data load:
 This enforces the occurrence of one of the two implications, and prevents the
 third possibility from arising.
 
-A data-dependency barrier must also order against dependent writes:
+
+[!] Note that this extremely counterintuitive situation arises most easily on
+machines with split caches, so that, for example, one cache bank processes
+even-numbered cache lines and the other bank processes odd-numbered cache
+lines.  The pointer P might be stored in an odd-numbered cache line, and the
+variable B might be stored in an even-numbered cache line.  Then, if the
+even-numbered bank of the reading CPU's cache is extremely busy while the
+odd-numbered bank is idle, one can see the new value of the pointer P (&B),
+but the old value of the variable B (2).
+
+
+A data-dependency barrier is not required to order dependent writes
+because the CPUs that the Linux kernel supports don't do writes until
+they are certain (1) that the write will actually happen, (2) of the
+location of the write, and (3) of the value to be written.  But please
+carefully read the "CONTROL DEPENDENCIES" section:  The compiler can
+and does break control dependencies in a great many situations.
 
 	CPU 1		      CPU 2
 	===============	      ===============
@@ -603,29 +619,19 @@ A data-dependency barrier must also order against dependent writes:
 	<write barrier>
 	WRITE_ONCE(P, &B);
 			      Q = READ_ONCE(P);
-			      <data dependency barrier>
-			      *Q = 5;
+			      WRITE_ONCE(*Q, 5);
 
-The data-dependency barrier must order the read into Q with the store
-into *Q.  This prohibits this outcome:
+Therefore, no data-dependency barrier is required to order the read into
+Q with the store into *Q.  In other words, this outcome is prohibited,
+even without a data-dependency barrier:
 
 	(Q == &B) && (B == 4)
 
 Please note that this pattern should be rare.  After all, the whole point
 of dependency ordering is to -prevent- writes to the data structure, along
 with the expensive cache misses associated with those writes.  This pattern
-can be used to record rare error conditions and the like, and the ordering
-prevents such records from being lost.
-
-
-[!] Note that this extremely counterintuitive situation arises most easily on
-machines with split caches, so that, for example, one cache bank processes
-even-numbered cache lines and the other bank processes odd-numbered cache
-lines.  The pointer P might be stored in an odd-numbered cache line, and the
-variable B might be stored in an even-numbered cache line.  Then, if the
-even-numbered bank of the reading CPU's cache is extremely busy while the
-odd-numbered bank is idle, one can see the new value of the pointer P (&B),
-but the old value of the variable B (2).
+can be used to record rare error conditions and the like, and the CPUs'
+naturally occurring ordering prevents such records from being lost.
 
 
 The data dependency barrier is very important to the RCU system,