[RFC PATCH 0/3] RT bandwidth constraints enforced by hierarchical DL scheduling

[Alessio Balsini]

Hi,

This patchset provides hierarchical RT scheduling by nesting the SCHED_RT
fixed-priority scheduler within SCHED_DEADLINE reservations, namely allowing
for groups of tasks to be scheduled within a SCHED_DEADLINE reservation,
choosing tasks within each group according to their RT priorities.

+-- SCHED_DL scheduler
   +-- SCHED_DL task 1 <rt=..., dl=..., period=...>
   +-- SCHED_DL task 2 <rt=..., dl=..., period=...>
   +-- ...
   +-- SCHED_RT group 1 <rt=..., period=...>
   |  +-- T1 <rtprio=...>
   |  +-- T2 <rtprio=...>
   +-- SCHED_RT group 2 <rt=..., period=...>
   |  +-- T3 <rtprio=...>
   |  +-- T4 <rtprio=...>
   ...

Usage Example:
  mkdir /sys/fs/cgroup/cpu/rt1
  echo 100000 > /sys/fs/cgroup/cpu/rt1/cpu.rt_period_us
  echo 10000 > /sys/fs/cgroup/cpu/rt1/cpu.rt_runtime_us
  echo $tid1 > /sys/fs/cgroup/cpu/rt1/tasks
  echo $tid2 > /sys/fs/cgroup/cpu/rt1/tasks
  chrt -r -p $rtprio1 $tid1
  chrt -r -p $rtprio2 $tid2

This is done by:
- ) 1st patch: removing the SCHED_RT-related cgroup mechanisms.
- ) 2nd patch: adding a new hierarchical cgroup throttling mechanism for
      SCHED_RT tasks that exploits SCHED_DL.
- ) 3rd patch: includes the RT cgroups migration of a throttled rq, seeking
      for available bandwidth on other CPUs.

For each group with given <runtime, period> parameters, per-CPU SCHED_DEADLINE
entities with the same parameters are created and independently scheduled on
each CPU. The scheduling entity includes a rt_rq. RT tasks belonging to the
group are free to migrate across the rt_rq of the group, whenever runtime is
available.
For example, if runtime=10ms and period(=deadline)=100ms, and we're on a
quad-core system, then a CPU-hog RT task would exhaust the 10ms on its current
CPU first, then it would migrate to consume the 10ms runtime available on
another CPU, etc., up until running for a total of 40ms, before being throttled
waiting for the next period.

The patchset is based on the preliminary version presented by Andrea Parri at
RTLWS 2014, considering also various issues discussed on LKML:

https://lkml.org/lkml/2016/10/9/164

including recent comments on the need for hierarchical CBS, eg:

https://www.spinics.net/lists/linux-rt-users/msg16003.html

Open issues that deserve attention:
-) The code currently includes independent push/pull functions for RT groups,
     those might be refactored together with the ones available in rt.c.
-) Affinity constraints for current throttling of RT tasks differs from the
     SCHED_DL-based (ie, no arbitrary affinities allowed in SCHED_DL).

The mechanism has been evaluated on several synthetic tests with RT groups,
containing RT tasks with different priorities and the groups bandwidths are
limited as expected. If anyone is interested in the specific tests, we can
share some code.

Alessio Balsini (3):
  sched/rt: Removed CGroup functionalities from RT scheduling class
  sched/deadline: Hierarchical scheduling with DL on top of RT
  sched/rt: DL-RT group migration from throttled rq

 include/linux/sched.h    |   13 +-
 kernel/sched/autogroup.c |    4 +-
 kernel/sched/core.c      |  111 ++--
 kernel/sched/deadline.c  |  259 ++++++---
 kernel/sched/debug.c     |    3 -
 kernel/sched/rt.c        | 1311 +++++++++++++++-------------------------------
 kernel/sched/sched.h     |  172 ++++--
 7 files changed, 837 insertions(+), 1036 deletions(-)

-- 
2.7.4