BTW, do you have a reproducer that can reproduce the race condition?
so sem->count will be negative after writer is killed
i think if flag RWSEM_FLAG_HANDOFF is not set, we shouldn't clean it
Thanks for reporting this possible race condition.
However, I am still trying to figure how it is possible to set the wstate to WRITER_HANDOFF without actually setting the handoff bit as well. The statement sequence should be as follows:
wstate = WRITER_HANDOFF;
raw_spin_lock_irq(&sem->wait_lock);
if (rwsem_try_write_lock(sem, wstate))
raw_spin_unlock_irq(&sem->wait_lock);
:
if (signal_pending_state(state, current))
goto out_nolock
The rwsem_try_write_lock() function will make sure that we either acquire the lock and clear handoff or set the handoff bit. This should be done before we actually check for signal. I do think that it is probably safer to use atomic_long_andnot to clear the handoff bit instead of using atomic_long_add().