[patch V4 00/68] locking, sched: The PREEMPT-RT locking infrastructure

[Thomas Gleixner]

Folks,

the following series is an update to V3 which can be found here:

  https://lore.kernel.org/r/20210805151300.330412127@xxxxxxxxxxxxx

It contains the bulk of the PREEMPT-RT locking infrastructure. In
PREEMPT-RT enabled kernels the following locking primitives are substituted
by RT-Mutex based variants:

  mutex, ww_mutex, rw_semaphore, spinlock, rwlock

semaphores are not substituted because they do not provide strict owner
semantics.

Of course raw_spinlocks are not touched either as they protect low level
operations in the scheduler, timers and hardware access.

The most interesting parts of the series which need a lot of eyeballs
are:

  - the scheduler bits which provide the infrastructure for spinlock and
    rwlock substitution to ensure that the task state is preserved when
    blocking on such a lock and a regular wakeup is handled correctly and
    not lost

  - the rtmutex core implementation to handle lock contention on spinlocks
    and rwlocks correctly vs. the task state

  - the rw_semaphore/rwlock substitutions which utilize the same
    implementation vs. the reader/writer handling

  - The new rtmutex based ww_mutex implementation.

  - the PI futex related bits to handle the interaction between blocking
    on the underlying rtmutex and contention on the hash bucket lock which
    is converted to a 'sleeping spinlock'.

The rest surely needs a thorough review as well, but those parts are pretty
straight forward: quite some code restructuring and the actual wrapper
functions to replace the existing !RT implementations.

The series survived internal testing in RT kernels and is part of the
v5.14-rc5-rt8 release:

 https://lore.kernel.org/r/20210810163731.2qvfuhenolq2gdlv@xxxxxxxxxxxxx

For !RT kernels there is no functional change.

The series is also available from git:

  git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git rtmutex

and fully integrated into the v5.14-rc5-rt8 release.

Changes vs. V3:

  - Add a READ_ONCE() to rt_mutex_base_is_locked()  (Davidlohr)

  - Fixed the WW mutex prio ordering check (PeterZ)

  - Simplified the WW mutex build magic (PeterZ)

  - Picked up PeterZ's change for !RT task queueing order

  - More cleanups and corrections in the futex code (Davidlohr)

  - Rename and final polishing for the spin on owner mechanics (PeterZ, Davidlohr)

Thanks,

	tglx
---
 b/drivers/staging/media/atomisp/pci/atomisp_ioctl.c |    4 
 b/include/linux/debug_locks.h                       |    3 
 b/include/linux/mutex.h                             |   93 +
 b/include/linux/preempt.h                           |    4 
 b/include/linux/rbtree.h                            |   30 
 b/include/linux/rbtree_types.h                      |   34 
 b/include/linux/rtmutex.h                           |   56 
 b/include/linux/rwbase_rt.h                         |   38 
 b/include/linux/rwlock_rt.h                         |  140 ++
 b/include/linux/rwlock_types.h                      |   39 
 b/include/linux/rwsem.h                             |   58 +
 b/include/linux/sched.h                             |  119 +-
 b/include/linux/sched/wake_q.h                      |    8 
 b/include/linux/spinlock.h                          |   15 
 b/include/linux/spinlock_api_smp.h                  |    3 
 b/include/linux/spinlock_rt.h                       |  151 ++
 b/include/linux/spinlock_types.h                    |   83 -
 b/include/linux/spinlock_types_raw.h                |   65 +
 b/include/linux/ww_mutex.h                          |   50 
 b/kernel/Kconfig.locks                              |    2 
 b/kernel/futex.c                                    |  556 ++++++---
 b/kernel/locking/Makefile                           |    3 
 b/kernel/locking/mutex-debug.c                      |    5 
 b/kernel/locking/mutex.c                            |  431 -------
 b/kernel/locking/mutex.h                            |   33 
 b/kernel/locking/rtmutex.c                          | 1134 +++++++++-----------
 b/kernel/locking/rtmutex_api.c                      |  590 ++++++++++
 b/kernel/locking/rtmutex_common.h                   |  122 +-
 b/kernel/locking/rwbase_rt.c                        |  263 ++++
 b/kernel/locking/rwsem.c                            |  109 +
 b/kernel/locking/spinlock.c                         |    7 
 b/kernel/locking/spinlock_debug.c                   |    5 
 b/kernel/locking/spinlock_rt.c                      |  257 ++++
 b/kernel/locking/ww_mutex.h                         |  569 ++++++++++
 b/kernel/locking/ww_rt_mutex.c                      |   76 +
 b/kernel/rcu/tree_plugin.h                          |    6 
 b/kernel/sched/core.c                               |  109 +
 b/lib/Kconfig.debug                                 |   11 
 b/lib/test_lockup.c                                 |    8 
 kernel/locking/mutex-debug.h                        |   29 
 40 files changed, 3818 insertions(+), 1500 deletions(-)