Re: [PATCH] sched: Use READ_ONCE()/WRITE_ONCE() in task_cpu()/__set_task_cpu()

[Peter Zijlstra]

On Mon, Jan 21, 2019 at 11:51:21AM +0100, Andrea Parri wrote:
> On Wed, Jan 16, 2019 at 07:42:18PM +0100, Andrea Parri wrote:
> > The smp_wmb() in move_queued_task() (c.f., __set_task_cpu()) pairs with
> > the composition of the dependency and the ACQUIRE in task_rq_lock():
> > 
> > 	move_queued_task()		task_rq_lock()
> > 
> > 	[S] ->on_rq = MIGRATING		[L] rq = task_rq()
> > 	WMB (__set_task_cpu())		ACQUIRE (rq->lock);
> > 	[S] ->cpu = new_cpu		[L] ->on_rq
> > 
> > where "[L] rq = task_rq()" is ordered before "ACQUIRE (rq->lock)" by an
> > address dependency and, in turn, "ACQUIRE (rq->lock)" is ordered before
> > "[L] ->on_rq" by the ACQUIRE itself.
> > 
> > Use READ_ONCE() to load ->cpu in task_rq() (c.f., task_cpu()) to honour
> > this address dependency between loads; also, mark the store to ->cpu in
> > __set_task_cpu() by using WRITE_ONCE() in order to tell the compiler to
> > not mess/tear this (synchronizing) memory access.
> 
> In the light of the recent discussion about the integration of plain
> accesses in the LKMM (c.f., e.g., [1] and discussion thereof), I was
> considering even further changes to this in order to "reinforce" the
> above smp_wmb().  Here's two approaches (one of):
> 
>  1) replace this smp_wmb()+WRITE_ONCE() with an smp_store_release();
> 
>  2) or keep this smp_wmb()+WRITE_ONCE(), but use {WRITE,READ}_ONCE()
>     also for the accesses to ->on_rq.

That should be the least painful I think. Note that we never store a
value larger than a single byte in that word, so tearing shouldn't be a
problem, but yes, that makes it all neat and tidy.

> What do you think?  (maybe I'm just being too paranoid?)
> 
> Adding Will to the Cc:  ((1) should be "painless" for x86, not sure
> about arm64...)

ARM64 should be fine, it is 32bit ARM that will suffer, because it uses
smp_mb() to implement acquire/release.