Re: [PATCH v3] locking/rwsem: reduce spinlock contention in wakeup after up_read/up_write

[Waiman Long]

On 04/28/2015 01:17 PM, Peter Zijlstra wrote:
> On Fri, Apr 24, 2015 at 01:54:29PM -0400, Waiman Long wrote:
> > @@ -478,7 +515,40 @@ struct rw_semaphore *rwsem_wake(struct rw_semaphore *sem)
> >   {
> >   	unsigned long flags;
> >
> > -	raw_spin_lock_irqsave(&sem->wait_lock, flags);
> > +	/*
> > +	 * If a spinner is present, it is not necessary to do the wakeup.
> > +	 * Try to do wakeup only if the trylock succeeds to minimize
> > +	 * spinlock contention which may introduce too much delay in the
> > +	 * unlock operation.
> > +	 *
> > +	 *    spinning writer		up_write/up_read caller
> > +	 *    ---------------		-----------------------
> > +	 * [S]   osq_unlock()		[L]   osq
> > +	 *	 MB			      MB
> > +	 * [RmW] rwsem_try_write_lock() [RmW] spin_trylock(wait_lock)
> > +	 *
> > +	 * Here, it is important to make sure that there won't be a missed
> > +	 * wakeup while the rwsem is free and the only spinning writer goes
> > +	 * to sleep without taking the rwsem. In case the spinning writer is
> > +	 * just going to break out of the waiting loop, it will still do a
> > +	 * trylock in rwsem_down_write_failed() before sleeping. IOW, if
> > +	 * rwsem_has_spinner() is true, it will  guarantee at least one
> > +	 * trylock attempt on the rwsem.
> > +	 */
> > +	if (!rwsem_has_spinner(sem)) {
> > +		raw_spin_lock_irqsave(&sem->wait_lock, flags);
> > +	} else {
> > +		/*
> > +		 * rwsem_has_spinner() is an atomic read while spin_trylock
> > +		 * does not guarantee a full memory barrier. Insert a memory
> > +		 * barrier here to make sure that wait_lock isn't read until
> > +		 * after osq.
> > +		 * Note: smp_rmb__after_atomic() should be used if available.
> > +		 */
> > +		smp_mb__after_atomic();
> > +		if (!raw_spin_trylock_irqsave(&sem->wait_lock, flags))
> > +			return sem;
> > +	}
> >
> >   	/* do nothing if list empty */
> >   	if (!list_empty(&sem->wait_list))
> To me it makes more sense to reverse these two branches (identical code
> wise of course) and put the special case first.
>
> Alternatively we could also do something like the below, which to my
> eyes looks a little better still, but I don't care too much.
>
> 	if (rwsem_has_spinner(sem)) {
> 		/*
> 		 * comment ...
> 		 */
> 		 smp_rmb();
> 		 if (!raw_spin_trylock_irqsave(&sem->wait_lock, flags))
> 			return sem;
> 		 goto locked;
> 	}
>
> 	raw_spin_lock_irqsave(&sem->wait_lock, flags);
> locked:

Thanks for the suggested. I have implemented that in the v4 patch. Also 
thanks for correcting my misconception on how to use the 
smp_mb__after_atomic() macro.

Cheers,
Longman
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/