Re: [PATCH v3 0/3] locking/mutex: Enable optimistic spinning of lock waiter

[Waiman Long]

On 07/18/2016 01:59 PM, Jason Low wrote:
> On Tue, 2016-03-22 at 13:46 -0400, Waiman Long wrote:
> > v2->v3:
> >   - Remove patch 4 as it is not useful.
> >   - Allow need_resched() check for waiter&  add more comments about
> >     changes to address issues raised by PeterZ.
> >
> > v1->v2:
> >   - Set task state to running before doing optimistic spinning.
> >   - Add 2 more patches to handle possible missed wakeups and wasteful
> >     spinning in try_to_wake_up() function.
> >
> > This patchset is a variant of PeterZ's "locking/mutex: Avoid spinner
> > vs waiter starvation" patch. The major difference is that the
> > waiter-spinner won't enter into the OSQ used by the spinners. Instead,
> > it will spin directly on the lock in parallel with the queue head
> > of the OSQ. So there will be a bit more cacheline contention on the
> > lock cacheline, but that shouldn't cause noticeable impact on system
> > performance.
> >
> > This patchset tries to address 2 issues with Peter's patch:
> >
> >   1) Ding Tianhong still find that hanging task could happen in some cases.
> >   2) Jason Low found that there was performance regression for some AIM7
> >      workloads.
> >
> > By making the waiter-spinner to spin directly on the mutex, it will
> > increase the chance for the waiter-spinner to get the lock instead
> > of waiting in the OSQ for its turn.
> >
> > Patch 1 modifies the mutex_optimistic_spin() function to enable it
> > to be called by a waiter-spinner that doesn't need to go into the OSQ.
> >
> > Patch 2 modifies the mutex locking slowpath to make the waiter call
> > mutex_optimistic_spin() to do spinning after being waken up.
> Just in case, here was the Waiman's patch that I was referring to for
> addressing some of the starvation issues with mutex by allowing the
> waiter to do optimistic spinning after being woken up.
>
> > Patch 3 reverses the sequence of setting task state and changing
> > mutex count to -1 to prevent the possibility of missed wakeup.
> >
> > Patch 4 modifies the wakeup code to abandon the wakeup operation
> > while spinning on the on_cpu flag if the task has changed back to a
> > non-sleeping state.
> >
> > My own test on a 4-socket E7-4820 v3 system showed a regression of
> > about 4% in the high_systime workload with Peter's patch which this
> > new patch effectively eliminates.
> >
> > Testing on an 8-socket Westmere-EX server, however, has performance
> > change from -9% to than +140% on the fserver workload of AIM7
> > depending on how the system was set up.
> >
> > Waiman Long (3):
> >    locking/mutex: Add waiter parameter to mutex_optimistic_spin()
> >    locking/mutex: Enable optimistic spinning of woken task in wait queue
> >    locking/mutex: Avoid missed wakeup of mutex waiter
> >
> >   kernel/locking/mutex.c |  126 ++++++++++++++++++++++++++++++++++--------------
> >   1 files changed, 90 insertions(+), 36 deletions(-)

I have just sent out an updated patchset to address this issue.

Cheers,
Longman