Re: [RFC] locking/rwsem: Avoid issuing wakeup before setting the reader waiter to nil

[Waiman Long]

On 11/29/2018 01:43 PM, Davidlohr Bueso wrote:
> On Thu, 29 Nov 2018, Peter Zijlstra wrote:
>
>> On Thu, Nov 29, 2018 at 02:12:32PM +0100, Peter Zijlstra wrote:
>>> Yes, I think this is real, and worse, I think we need to go audit all
>>> wake_q_add() users and document this behaviour.
>>>
>>> In the ideal case we'd delay the actual wakeup to the last wake_up_q(),
>>> but I don't think we can easily fix that.
>>
>> See commit: 1d0dcb3ad9d3 ("futex: Implement lockless wakeups"), I think
>> that introduces the exact same bug.
>>
>> Something like the below perhaps, altough this pattern seems to want a
>> wake_a_add() variant that already assumes get_task_struct().
>
> So I was looking at ways to avoid the redundant reference counting,
> but given how wake_q_add() and wake_up_q() are so loose I can't
> see how to avoid it -- we hold reference across the calls to maintain
> valid mem.
>
> For example, wake_q will grab reference iff the cmpxchg succeeds,
> likewise it will enter the wakeup loop in wake_up_q(), and there is no
> awareness of which caller had the failed cmpxchg because another wakup
> was in progress.
>

I think it will be useful to make wake_q_add() to return the status of
wake_q insertion so that we can respond appropriately if it fails.

> And yes, afaict all wake_q users suffer from the same issue, so we have
> to move the wake_q_add() after the condition, while explicitly doing
> the task ref counting.

How about adding two helpers - wake_q_enter(), wake_q_exit() that can
hide all the reference counting?

Thanks,
Longman