[patch V4 00/68] locking, sched: The PREEMPT-RT locking infrastructure

From: Thomas Gleixner
Date: Wed Aug 11 2021 - 08:22:58 EST


Folks,

the following series is an update to V3 which can be found here:

https://lore.kernel.org/r/20210805151300.330412127@xxxxxxxxxxxxx

It contains the bulk of the PREEMPT-RT locking infrastructure. In
PREEMPT-RT enabled kernels the following locking primitives are substituted
by RT-Mutex based variants:

mutex, ww_mutex, rw_semaphore, spinlock, rwlock

semaphores are not substituted because they do not provide strict owner
semantics.

Of course raw_spinlocks are not touched either as they protect low level
operations in the scheduler, timers and hardware access.

The most interesting parts of the series which need a lot of eyeballs
are:

- the scheduler bits which provide the infrastructure for spinlock and
rwlock substitution to ensure that the task state is preserved when
blocking on such a lock and a regular wakeup is handled correctly and
not lost

- the rtmutex core implementation to handle lock contention on spinlocks
and rwlocks correctly vs. the task state

- the rw_semaphore/rwlock substitutions which utilize the same
implementation vs. the reader/writer handling

- The new rtmutex based ww_mutex implementation.

- the PI futex related bits to handle the interaction between blocking
on the underlying rtmutex and contention on the hash bucket lock which
is converted to a 'sleeping spinlock'.

The rest surely needs a thorough review as well, but those parts are pretty
straight forward: quite some code restructuring and the actual wrapper
functions to replace the existing !RT implementations.

The series survived internal testing in RT kernels and is part of the
v5.14-rc5-rt8 release:

https://lore.kernel.org/r/20210810163731.2qvfuhenolq2gdlv@xxxxxxxxxxxxx

For !RT kernels there is no functional change.

The series is also available from git:

git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git rtmutex

and fully integrated into the v5.14-rc5-rt8 release.

Changes vs. V3:

- Add a READ_ONCE() to rt_mutex_base_is_locked() (Davidlohr)

- Fixed the WW mutex prio ordering check (PeterZ)

- Simplified the WW mutex build magic (PeterZ)

- Picked up PeterZ's change for !RT task queueing order

- More cleanups and corrections in the futex code (Davidlohr)

- Rename and final polishing for the spin on owner mechanics (PeterZ, Davidlohr)

Thanks,

tglx
---
b/drivers/staging/media/atomisp/pci/atomisp_ioctl.c | 4
b/include/linux/debug_locks.h | 3
b/include/linux/mutex.h | 93 +
b/include/linux/preempt.h | 4
b/include/linux/rbtree.h | 30
b/include/linux/rbtree_types.h | 34
b/include/linux/rtmutex.h | 56
b/include/linux/rwbase_rt.h | 38
b/include/linux/rwlock_rt.h | 140 ++
b/include/linux/rwlock_types.h | 39
b/include/linux/rwsem.h | 58 +
b/include/linux/sched.h | 119 +-
b/include/linux/sched/wake_q.h | 8
b/include/linux/spinlock.h | 15
b/include/linux/spinlock_api_smp.h | 3
b/include/linux/spinlock_rt.h | 151 ++
b/include/linux/spinlock_types.h | 83 -
b/include/linux/spinlock_types_raw.h | 65 +
b/include/linux/ww_mutex.h | 50
b/kernel/Kconfig.locks | 2
b/kernel/futex.c | 556 ++++++---
b/kernel/locking/Makefile | 3
b/kernel/locking/mutex-debug.c | 5
b/kernel/locking/mutex.c | 431 -------
b/kernel/locking/mutex.h | 33
b/kernel/locking/rtmutex.c | 1134 +++++++++-----------
b/kernel/locking/rtmutex_api.c | 590 ++++++++++
b/kernel/locking/rtmutex_common.h | 122 +-
b/kernel/locking/rwbase_rt.c | 263 ++++
b/kernel/locking/rwsem.c | 109 +
b/kernel/locking/spinlock.c | 7
b/kernel/locking/spinlock_debug.c | 5
b/kernel/locking/spinlock_rt.c | 257 ++++
b/kernel/locking/ww_mutex.h | 569 ++++++++++
b/kernel/locking/ww_rt_mutex.c | 76 +
b/kernel/rcu/tree_plugin.h | 6
b/kernel/sched/core.c | 109 +
b/lib/Kconfig.debug | 11
b/lib/test_lockup.c | 8
kernel/locking/mutex-debug.h | 29
40 files changed, 3818 insertions(+), 1500 deletions(-)