Re: per-cpu operation madness vs validation

[Christoph Lameter]

On Wed, 27 Jul 2011, Peter Zijlstra wrote:

> On Wed, 2011-07-27 at 09:16 -0500, Christoph Lameter wrote:
>
> > Well yes these are multiple operations that have a sort of atomicity
> > requirement together. You want both variables to be updated together and
> > therefore the individual RMV operations wont do the trick. The algorithm
> > would have to be changed to take advantage of the this_cpu ops or you will
> > have to take the penalty of higher synchronization costs.
>
> You're still not getting it, synchronization cost isn't the point.

It is for me.

> > What you can do is what the slub fastpath does in order to allow updates
> > without caring about preemption or interrupts.
>
> > 1. You get a reference to the current per cpu data.
> > 2. You obtain a transaction id that is cpu specific and operation specifc.
> > 3. Speculatively access an object and work with the information of that
> > object. Do not change anything.
>
> Pretty hard when the whole point if changing stuff.

Essentially you have a pointer to play with. Not unlike a special form of
RCU.

> > 4. Calculate the next transaction id
> > 4. Replace pointer and transaction id with a new object pointer and new
> > tranaction id. If there is no match restart at 1.
>
> Which is utter crap, ease up on the the drugs. That mandates a
> read-copy-update like approach to all per-cpu data.

If one wants performance you can get it that way.

> > So now you have a per cpu access that is fully immunte from interrupt and
> > preemption issues.
>
> And suffers from allocation issues.. allocating a new piece of data,
> copying over the old one, modifying it, installing the new pointer,
> managing references to the old pointer etc.. You're on crack, that's
> often way too much trouble for nearly no gain.

Maybe you just do not know how to handle it? The allocators (even the page
allocator) have been playing tricks with per cpu data for a very long
time.

> > percpu sections are very performance critical components of the kernel.
> > Taking locks could be avoided with serialization through transaction ids
> > f.e.
>
> Not the fucking point! This isn't about making things go faster, this is
> about breaking up the horrid implicit preempt/bh/irq assumptions all
> over the place. It really is no different from killing the BKL.

There is no breaking of anything. These assumptions already exist
throughout the kernel and the issues with access to variables in
different preemption context/interrupt context already exist. Nothing was
changed there.

> You'll need this even if you're going to rewrite the kernel with your
> cmpxchg_double wankery simply because you need to know how stuff is
> supposed to work.
>
> All this is simply about de-obfuscating all per-cpu assumptions. This is
> about verification and traceability/debuggability.

Ok verification  and traceability are good but they should not
be in the way of making core functionality high performance and low
latency.

The key issue is that the -rt kernel has always had grave issues with
performance when it comes to per cpu data access. Solving that by forcing
the kernel to go slow it not the right approach.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/