Re: [PATCH] doc: Update memory-barriers.txt for read-to-write dependencies

[Paul E. McKenney]

On Tue, Jul 04, 2017 at 05:36:36PM +0100, Will Deacon wrote:
> Hello, Paul,
> 
> On Mon, Jul 03, 2017 at 10:41:06AM -0700, Paul E. McKenney wrote:
> > On Mon, Jul 03, 2017 at 02:07:03PM +0100, Will Deacon wrote:
> > > Might be worth mentioning that you have to careful with the compiler here,
> > > and pointing to the section on "Control dependencies" so that people don't
> > > just take these three points as guarantees in isolation.
> > > 
> > > >  
> > > >  	CPU 1		      CPU 2
> > > >  	===============	      ===============
> > > > @@ -603,19 +606,19 @@ A data-dependency barrier must also order against dependent writes:
> > > >  	<write barrier>
> > > >  	WRITE_ONCE(P, &B);
> > > >  			      Q = READ_ONCE(P);
> > > > -			      <data dependency barrier>
> > > >  			      *Q = 5;
> > > 
> > > Do we want that write to Q to be a WRITE_ONCE? Again, the control
> > > dependencies section does call this out.
> > 
> > Both good points!  Like this?
> 
> More or less, yes! Just one wibble below:
> 
> > commit 00269a0e23dbc50f1c4f101b23c8d74992eace05
> > Author: Paul E. McKenney <paulmck@xxxxxxxxxxxxxxxxxx>
> > Date:   Fri Jun 30 16:18:28 2017 -0700
> > 
> >     doc: Update memory-barriers.txt for read-to-write dependencies
> >     
> >     The memory-barriers.txt document contains an obsolete passage stating that
> >     smp_read_barrier_depends() is required to force ordering for read-to-write
> >     dependencies.  We now know that this is not required, even for DEC Alpha.
> >     This commit therefore updates this passage to state that read-to-write
> >     dependencies are respected even without smp_read_barrier_depends().
> >     
> >     Reported-by: Lance Roy <ldr709@xxxxxxxxx>
> >     Signed-off-by: Paul E. McKenney <paulmck@xxxxxxxxxxxxxxxxxx>
> >     Cc: David Howells <dhowells@xxxxxxxxxx>
> >     Cc: Will Deacon <will.deacon@xxxxxxx>
> >     Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
> >     Cc: Jonathan Corbet <corbet@xxxxxxx>
> >     Cc: Alan Stern <stern@xxxxxxxxxxxxxxxxxxx>
> >     Cc: Andrea Parri <parri.andrea@xxxxxxxxx>
> >     Cc: Jade Alglave <j.alglave@xxxxxxxxx>
> >     Cc: Luc Maranget <luc.maranget@xxxxxxxx>
> >     [ paulmck: Reference control-dependencies sections and use WRITE_ONCE()
> >       per Will Deacon.  Correctly place split-cache paragraph while there. ]
> > 
> > diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt
> > index 9d5e0f853f08..7be80911e502 100644
> > --- a/Documentation/memory-barriers.txt
> > +++ b/Documentation/memory-barriers.txt
> > @@ -594,7 +594,23 @@ between the address load and the data load:
> >  This enforces the occurrence of one of the two implications, and prevents the
> >  third possibility from arising.
> >  
> > -A data-dependency barrier must also order against dependent writes:
> > +
> > +[!] Note that this extremely counterintuitive situation arises most easily on
> > +machines with split caches, so that, for example, one cache bank processes
> > +even-numbered cache lines and the other bank processes odd-numbered cache
> > +lines.  The pointer P might be stored in an odd-numbered cache line, and the
> > +variable B might be stored in an even-numbered cache line.  Then, if the
> > +even-numbered bank of the reading CPU's cache is extremely busy while the
> > +odd-numbered bank is idle, one can see the new value of the pointer P (&B),
> > +but the old value of the variable B (2).
> > +
> > +
> > +A data-dependency barrier is not required to order dependent writes
> > +because the CPUs that the Linux kernel supports don't do writes until
> > +they are certain (1) that the write will actually happen, (2) of the
> > +location of the write, and (3) of the value to be written.  But please
> > +carefully read the "CONTROL DEPENDENCIES" section:  The compiler can
> > +and does break control dependencies in a great many situations.
> 
> This makes it sound like only control dependencies are susceptible to
> being broken by compiler optimisations, so I'd drop the "control" and
> just say "The compiler can and does break dependencies in a great many
> situations".
> 
> With that:
> 
> Acked-by: Will Deacon <will.deacon@xxxxxxx>

Done!  I also added a pointer to Documentation/RCU/rcu_dereference.txt,
which lists some of the vandalism^Woptimizations that modern compilers
can commit.  Does the updated patch below capture it?

							Thanx, Paul

------------------------------------------------------------------------

commit 3a2414a7991c579e7391968b382988f0300655e5
Author: Paul E. McKenney <paulmck@xxxxxxxxxxxxxxxxxx>
Date:   Fri Jun 30 16:18:28 2017 -0700

    doc: Update memory-barriers.txt for read-to-write dependencies
    
    The memory-barriers.txt document contains an obsolete passage stating that
    smp_read_barrier_depends() is required to force ordering for read-to-write
    dependencies.  We now know that this is not required, even for DEC Alpha.
    This commit therefore updates this passage to state that read-to-write
    dependencies are respected even without smp_read_barrier_depends().
    
    Reported-by: Lance Roy <ldr709@xxxxxxxxx>
    Signed-off-by: Paul E. McKenney <paulmck@xxxxxxxxxxxxxxxxxx>
    Cc: David Howells <dhowells@xxxxxxxxxx>
    Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
    Cc: Jonathan Corbet <corbet@xxxxxxx>
    Cc: Alan Stern <stern@xxxxxxxxxxxxxxxxxxx>
    Cc: Andrea Parri <parri.andrea@xxxxxxxxx>
    Cc: Jade Alglave <j.alglave@xxxxxxxxx>
    Cc: Luc Maranget <luc.maranget@xxxxxxxx>
    [ paulmck: Reference control-dependencies sections and use WRITE_ONCE()
      per Will Deacon.  Correctly place split-cache paragraph while there. ]
    Acked-by: Will Deacon <will.deacon@xxxxxxx>

diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt
index c239a0cf4b1a..f05aa34fe908 100644
--- a/Documentation/memory-barriers.txt
+++ b/Documentation/memory-barriers.txt
@@ -594,7 +594,24 @@ between the address load and the data load:
 This enforces the occurrence of one of the two implications, and prevents the
 third possibility from arising.
 
-A data-dependency barrier must also order against dependent writes:
+
+[!] Note that this extremely counterintuitive situation arises most easily on
+machines with split caches, so that, for example, one cache bank processes
+even-numbered cache lines and the other bank processes odd-numbered cache
+lines.  The pointer P might be stored in an odd-numbered cache line, and the
+variable B might be stored in an even-numbered cache line.  Then, if the
+even-numbered bank of the reading CPU's cache is extremely busy while the
+odd-numbered bank is idle, one can see the new value of the pointer P (&B),
+but the old value of the variable B (2).
+
+
+A data-dependency barrier is not required to order dependent writes
+because the CPUs that the Linux kernel supports don't do writes
+until they are certain (1) that the write will actually happen, (2)
+of the location of the write, and (3) of the value to be written.
+But please carefully read the "CONTROL DEPENDENCIES" section and the
+Documentation/RCU/rcu_dereference.txt file:  The compiler can and does
+break dependencies in a great many highly creative ways.
 
 	CPU 1		      CPU 2
 	===============	      ===============
@@ -603,29 +620,19 @@ A data-dependency barrier must also order against dependent writes:
 	<write barrier>
 	WRITE_ONCE(P, &B);
 			      Q = READ_ONCE(P);
-			      <data dependency barrier>
-			      *Q = 5;
+			      WRITE_ONCE(*Q, 5);
 
-The data-dependency barrier must order the read into Q with the store
-into *Q.  This prohibits this outcome:
+Therefore, no data-dependency barrier is required to order the read into
+Q with the store into *Q.  In other words, this outcome is prohibited,
+even without a data-dependency barrier:
 
 	(Q == &B) && (B == 4)
 
 Please note that this pattern should be rare.  After all, the whole point
 of dependency ordering is to -prevent- writes to the data structure, along
 with the expensive cache misses associated with those writes.  This pattern
-can be used to record rare error conditions and the like, and the ordering
-prevents such records from being lost.
-
-
-[!] Note that this extremely counterintuitive situation arises most easily on
-machines with split caches, so that, for example, one cache bank processes
-even-numbered cache lines and the other bank processes odd-numbered cache
-lines.  The pointer P might be stored in an odd-numbered cache line, and the
-variable B might be stored in an even-numbered cache line.  Then, if the
-even-numbered bank of the reading CPU's cache is extremely busy while the
-odd-numbered bank is idle, one can see the new value of the pointer P (&B),
-but the old value of the variable B (2).
+can be used to record rare error conditions and the like, and the CPUs'
+naturally occurring ordering prevents such records from being lost.
 
 
 The data dependency barrier is very important to the RCU system,