Re: [RFC] Cancellable MCS spinlock rework

[Peter Zijlstra]

On Wed, Jul 02, 2014 at 09:59:16AM -0700, Jason Low wrote:
> On Wed, 2014-07-02 at 18:27 +0200, Peter Zijlstra wrote:
> > On Wed, Jul 02, 2014 at 09:21:10AM -0700, Jason Low wrote:
> > > The cancellable MCS spinlock is currently used to queue threads that are
> > > doing optimistic spinning. It uses per-cpu nodes, where a thread obtaining
> > > the lock would access and queue the local node corresponding to the CPU that
> > > it's running on. Currently, the cancellable MCS lock is implemented by using
> > > pointers to these nodes.
> > > 
> > > In this RFC patch, instead of operating on pointers to the per-cpu nodes, we
> > > store the CPU numbers in which the per-cpu nodes correspond to in atomic_t.
> > > A similar concept is used with the qspinlock.
> > > 
> > > We add 1 to the CPU number to retrive an "encoded value" representing the node
> > > of that CPU. By doing this, 0 can represent "no CPU", which allows us to
> > > keep the simple "if (CPU)" and "if (!CPU)" checks. In this patch, the next and
> > > prev pointers in each node were also modified to store encoded CPU values.
> > > 
> > > By operating on the CPU # of the nodes using atomic_t instead of pointers
> > > to those nodes, this can reduce the overhead of the cancellable MCS spinlock
> > > by 32 bits (on 64 bit systems).
> > 
> > Still struggling to figure out why you did this.
> 
> Why I converted pointers to atomic_t?
> 
> This would avoid the potentially racy ACCESS_ONCE stores + cmpxchg while
> also using less overhead, since atomic_t is often only 32 bits while
> pointers could be 64 bits.

So no real good reason.. The ACCESS_ONCE stores + cmpxchg stuff is
likely broken all over the place, and 'fixing' this one place doesn't
cure the problem.

Attachment: pgpEG9G4ZoFhJ.pgp
Description: PGP signature