Re: [PATCH] x86: Add an explicit barrier() to clflushopt()

[Chris Wilson]

On Tue, Jan 12, 2016 at 02:07:35PM -0800, Linus Torvalds wrote:
> On Tue, Jan 12, 2016 at 1:13 PM, Chris Wilson <chris@xxxxxxxxxxxxxxxxxx> wrote:
> > Indeed. So I replaced the post-clflush_cache_range() clflush() with a
> > udelay(10) instead, and the corruption vanished. Putting the udelay(10)
> > before the clflush_cache_range() does not fix the corruption.
> 
> Odd.
> 
> > passes, I'm inclined to point the finger at the mb() following the
> > clflush_cache_range().
> 
> We have an entirely unrelated discussion about the value of "mfence"
> as a memory barrier.
> 
> Mind trying to just make the memory barrier (in
> arch/x86/include/asm/barrier.h) be a locked op instead?

Replacing the following mb() in clflush_cache_range() with
"lock; addl $0,0(%%rsp)" gave no improvement. Neither did replacing all
mb(). Undoubtably the memory stream as seen by the CPU is serialised.
The concern then is back to the visibility of the stream from the external
device. Adding an uncached mmio read after the clflush_cache_range()
also works, but I'm not clear if this achieves anything other than
inserting a delay or if it is flushing some other write buffer in the
chipset. It is odd that is required after the clflush_cache_range() and
ineffective before. It is also odd that such a flush would be needed
multiple times after the GPU write.

The double clflush() remains a mystery.
-Chris

-- 
Chris Wilson, Intel Open Source Technology Centre