Re: [PATCH 2/2] [PATCH] sched: Add smp_rmb() in task rq locking cycles

From: Paul E. McKenney
Date: Wed Feb 18 2015 - 13:44:05 EST


On Wed, Feb 18, 2015 at 02:47:06PM +0100, Peter Zijlstra wrote:
> On Tue, Feb 17, 2015 at 01:52:31PM -0800, Paul E. McKenney wrote:
> > I could do a table per communication style. For example, message
> > passing looks like this (give or take likely errors in the table):
> >
> > Side CPU Top CPU
> > -------- -------
> > X = 1; r1 = Y;
> > <some barrier> <some barrier>
> > Y = 1; r2 = X;
> >
> > assert(r1 == 0 || r2 == 1);
> >
> >
> > | mb | wmb | rmb | rbd | acq | rel | ctl |
> > -----+-------+-------+-------+-------+-------+-------+-------+
> > mb | Y | | Y | y | Y | | Y +
> > -----+-------+-------+-------+-------+-------+-------+-------+
> > wmb | Y | | Y | y | Y | | Y +
> > -----+-------+-------+-------+-------+-------+-------+-------+
> > rmb | | | | | | | +
> > -----+-------+-------+-------+-------+-------+-------+-------+
> > rbd | | | | | | | +
> > -----+-------+-------+-------+-------+-------+-------+-------+
> > acq | | | | | | | +
> > -----+-------+-------+-------+-------+-------+-------+-------+
> > rel | Y | | Y | y | Y | | Y +
> > -----+-------+-------+-------+-------+-------+-------+-------+
> > ctl | | | | | | | +
> > -----+-------+-------+-------+-------+-------+-------+-------+
> >
> > Here "Y" says that the barrier pair works, "y" says that it can
> > work but requires an artificial dependency, and " " says that
> > it does not work.
>
> I would maybe do s/artificial/additional/, the pointer deref in RCU is
> not really artificial, is it?

Ah, but RCU is a slightly different pattern because you do have to have
a dependency, so you need a different litmus test:

Initial state: X = 0, Y = &Z, Z = 2

Side CPU Top CPU
-------- -------
X = 1; r1 = READ_ONCE(Y);
<some barrier> <some_barrier>
WRITE_ONCE(Y, &X); r2 = *r1;

assert(r1 == &Z || r2 == 1);

| mb | wmb | rmb | rbd | acq | rel | ctl |
-----+-------+-------+-------+-------+-------+-------+-------+
mb | Y | | Y | Y | Y | | Y +
-----+-------+-------+-------+-------+-------+-------+-------+
wmb | Y | | Y | Y | Y | | Y +
-----+-------+-------+-------+-------+-------+-------+-------+
rmb | | | | | | | +
-----+-------+-------+-------+-------+-------+-------+-------+
rbd | | | | | | | +
-----+-------+-------+-------+-------+-------+-------+-------+
acq | | | | | | | +
-----+-------+-------+-------+-------+-------+-------+-------+
rel | Y | | Y | Y | Y | | Y +
-----+-------+-------+-------+-------+-------+-------+-------+
ctl | | | | | | | +
-----+-------+-------+-------+-------+-------+-------+-------+

But of course you should instead use rcu_assign_pointer() and
rcu_dereference(). And I should also have used READ_ONCE() and
WRITE_ONCE() in my earlier example.

> Also, how many communication styles do you envision to enumerate?

As few as possible, but yes, that is likely to be a bit of a problem.
To get an idea of how big it could get if not contained somehow, take
a look at http://www.cl.cam.ac.uk/~pes20/ppc-supplemental/test6.pdf.
20 of them on the first page alone! ;-)

We most definitely need to stick to the lowest common denominator
if we are taking this approach.

Thanx, Paul

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/