Re: Documentation/memory-barriers.txt: How can READ_ONCE() and WRITE_ONCE() provide cache coherence?

[Paul E. McKenney]

On Sat, Feb 27, 2016 at 12:14:21AM +0300, Sergey Fedorov wrote:
> Hi,
> 
> I just can't understand how this kind of compiler barrier macros may
> provide any form of cache coherence. Sure, such kind of compiler
> barrier is necessary to "reliably" access a variable from multiple
> CPUs. But why it is stated that these macros *provide* cache
> coherence?

Without READ_ONCE(), common sub-expression elimination optimizations
can cause later reads of a given variable to see older value than
previous reads did.  For a (silly) example:

	a = complicated_pure_function(x);
	b = x;
	c = complicated_pure_function(x);

The compiler is within its rights to transform this into the following:

	a = complicated_pure_function(x);
	b = x;
	c = a(x);

In this case, the assignment to b might see a newer value of x than did
the later assignment to c.  This violates cache coherence, which states
that all reads from a given variable must agree on the order of values
taken on by that variable.

Using READ_ONCE() prevents this violation of cache coherence, albeit
at the price of evaluating complicated_pure_function() twice rather
than once:

	a = complicated_pure_function(READ_ONCE(x));
	b = READ_ONCE(x);
	c = complicated_pure_function(READ_ONCE(x));

Similar examples exist for WRITE_ONCE().

You -want- the compiler to violate cache coherence for normal accesses
to unshared variables, so you have to tell it when cache coherence is
important.

							Thanx, Paul

> From Documentation/memory-barriers.txt:
> >The READ_ONCE() and WRITE_ONCE() functions can prevent any number of
> >optimizations that, while perfectly safe in single-threaded code, can
> >be fatal in concurrent code.  Here are some examples of these sorts
> >of optimizations:
> >
> > (*) The compiler is within its rights to reorder loads and stores
> >     to the same variable, and in some cases, the CPU is within its
> >     rights to reorder loads to the same variable.  This means that
> >     the following code:
> >
> >    a[0] = x;
> >    a[1] = x;
> >
> >     Might result in an older value of x stored in a[1] than in a[0].
> >     Prevent both the compiler and the CPU from doing this as follows:
> >
> >    a[0] = READ_ONCE(x);
> >    a[1] = READ_ONCE(x);
> >
> >     In short, READ_ONCE() and WRITE_ONCE() provide cache coherence for
> >     accesses from multiple CPUs to a single variable.
> 
> Thanks,
> Sergey
>