Re: [PATCH] locking: Document that mutex_unlock() is non-atomic

[Jann Horn]

On Fri, Dec 1, 2023 at 1:33 AM Waiman Long <longman@xxxxxxxxxx> wrote:
> On 11/30/23 15:48, Jann Horn wrote:
> > I have seen several cases of attempts to use mutex_unlock() to release an
> > object such that the object can then be freed by another task.
> > My understanding is that this is not safe because mutex_unlock(), in the
> > MUTEX_FLAG_WAITERS && !MUTEX_FLAG_HANDOFF case, accesses the mutex
> > structure after having marked it as unlocked; so mutex_unlock() requires
> > its caller to ensure that the mutex stays alive until mutex_unlock()
> > returns.
> >
> > If MUTEX_FLAG_WAITERS is set and there are real waiters, those waiters
> > have to keep the mutex alive, I think; but we could have a spurious
> > MUTEX_FLAG_WAITERS left if an interruptible/killable waiter bailed
> > between the points where __mutex_unlock_slowpath() did the cmpxchg
> > reading the flags and where it acquired the wait_lock.
> >
> > (With spinlocks, that kind of code pattern is allowed and, from what I
> > remember, used in several places in the kernel.)
> >
> > If my understanding of this is correct, we should probably document this -
> > I think such a semantic difference between mutexes and spinlocks is fairly
> > unintuitive.
>
> Spinlocks are fair. So doing a lock/unlock sequence will make sure that
> all the previously waiting waiters are done with the lock. Para-virtual
> spinlocks, however, can be a bit unfair so doing a lock/unlock sequence
> may not be enough to guarantee there is no waiter. The same is true for
> mutex. Adding a spin_is_locked() or mutex_is_locked() check can make
> sure that all the waiters are gone.

I think this pattern anyway only works when you're only trying to wait
for the current holder of the lock, not tasks that are queued up on
the lock as waiters - so a task initially holds a stable reference to
some object, then acquires the object's lock, then drops the original
reference, and then later drops the lock.
You can see an example of such mutex usage (which is explicitly legal
with userspace POSIX mutexes, but is forbidden with kernel mutexes) at
the bottom of the POSIX manpage for pthread_mutex_destroy() at
<https://pubs.opengroup.org/onlinepubs/007904875/functions/pthread_mutex_destroy.html>,
in the section "Destroying Mutexes".

(I think trying to wait for pending waiters before destroying a mutex
wouldn't make sense because if there can still be pending waiters,
there can almost always also be tasks that are about to _become_
pending waiters but that haven't called mutex_lock() yet.)