Re: [PATCH RFC] percpu: add data dependency barrier in percpu accessors and operations

[Christoph Lameter]

On Thu, 19 Jun 2014, Tejun Heo wrote:

> On Thu, Jun 19, 2014 at 03:42:07PM -0500, Christoph Lameter wrote:
> > In that case special care needs to be taken to get this right. True.
> >
> > I typically avoid these scenarios by sending an IPI with a pointer to the
> > data structure. The modification is done by the cpu for which the per cpu
> > data is local.
> >
> > Maybe rewrite the code to avoid writing to other processors percpu data
> > would be the right approach?
>
> It depends on the specific use case but in general no.  IPIs would be
> far more expensive than making use of proper barriers in vast majority
> of cases especially when the "hot" side is data dependency barrier,
> IOW, nothing.  Also, we are talking about extremely low frequency
> events like init and recycling after reinit.  Regular per-cpu
> operation isn't really the subject here.

The aim of having percpu data is to have the ability for a processor to
access memory dedicated to that processor in the fastest way possible by
avoiding synchronization. You are beginning to add synchronization
elements into the accesses of a processor to memory dedicated to its sole
use.

Remote write events are contrary to that design and are exceedingly rare.
An IPI is justifiable for such a rare event. At least in my use cases I
have always found that to be sufficient. Well, I designed the data
structures in a way that made this possible because of the design criteria
that did not allow me remote write access to other processors per cpu
data.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/