[PATCH 00/10] sched/fair: A series of load balance patches to improve real-time performance of CFS tasks

[Xin Zhao]

Embedded platforms commonly use CONFIG_HZ_250, and testing has revealed
that there are numerous instances of unreasonable CPU idle events on such
platforms. Unreasonable CPU idle refers to situations where the CPU enters
an idle state for a duration of time (t), while there are tasks that can
run on the idle CPU and are not limited by cgroup constraints, yet these
tasks remain unscheduled for a duration greater than (t), t > 2.5 ms.

Testing has shown that over 95% of these events last less than 4 ms, but
there are still some instances of longer durations between 4-5ms, even
occasionally between 5-10 ms. For a real-time system, scheduling delays
greater than 4 ms can lead to performance spikes.

Enabling this option can effectively reduce the occurrence of unreasonable
CPU idle events on low HZ systems like CONFIG_HZ_250, and completely
eliminate events exceeding 4 ms. Note that the feature only affects fair
tasks.

Note that enabling this feature will increase sys%, as it uses CPU time
that would have been idle to expedite the scheduling of tasks. There will
also be some CPU overhead involved in searching for suitable tasks.

This feature has been split into several smaller patches, which will be
elaborated on one by one later. Below are some test data:

Test one compares the number of unreasonable CPU idle events and their
distribution when this feature is enabled versus when it is not, under
the same fillback scenario. The test duration was 60 seconds.

      LB_PROMOTE(on/off) 2.5-3ms 3-4ms  4ms+
index 0               on       0     0     0
index 1              off       4    13     1
index 2               on       0     0     0
index 3              off       6     3     0
index 4               on       0     0     0
index 5              off       1     1     0

Test two compares the performance of the system with and without the
feature enabled, based on the same fillback scenario. Each test lasts for
25 minutes, and a total of 15 comparative tests were conducted. The
results include a comparison of the maximum and median values of
end-to-end latency and sys%.

LB_PROMOTE(on/off)                    on        off
end-to-end latency(max)              172        180
end-to-end latency(median of avg)    166     167.68
sys%(max)                           9.68       9.35
sys%(median of avg)                 8.81       8.55

All of the patches are derived from analysis through ftrace and process
logs of load balance code flow corresponding to every unreasonable CPU
idle event captured during testing.

In patch 4, we define a new feature called LB_PROMOTE, which means to
promote load balance. Currently, this feature only optimizes the CFS load
balance part. The first three patches are prerequisite patches; patch 1
addresses a common issue unrelated to the LB_PROMOTE feature, while
patches 2 and 3 are two prerequisite patches that can be applied
independently of LB_PROMOTE feature but are primarily intended to support
patches 6 and 7. Patches 5, 7, 8, and 10 are all based on the LB_PROMOTE
feature introduced in patch 4. Patch 5 implements an improved version of
select_task_rq_fair() for embedded platforms, enhancing performance and
real-time capabilities. Patch 7 implements preemptive active balance,
while patch 6 allows active_load_balance_cpu_stop() to be reused in
preemptive active balance scenarios. Patch 8 removes checks related to
avg_idle. Patch 9 is a minor optimization that does not depend on the
LB_PROMOTE feature, but since it modifies sched_balance_rq() as patch 10
does, it serves as a precursor to patch 10.

Xin Zhao (10):
  sched/fair: Do not set_rd_overloaded() if rd->online != env->cpus
  scbed/fair: Remove duplicate check for busiest_cpu in
    active_load_balance_cpu_stop()
  sched/fair: Clear active_balance at the end of
    active_load_balance_cpu_stop()
  sched/fair: Add LB_PROMOTE feature to enhance real-time performance of
    fair tasks
  sched/fair: Introduce select_task_rq_fair_thin() to select rq when
    LB_PROMOTE
  sched/fair: Modify active_load_balance_cpu_stop() to accommodate more
    scenarios
  sched/fair: Trigger active balance if a CFS task is preempted when
    LB_PROMOTE
  sched/fair: Do not check avg_idle to prematurely exit newly idle when
    LB_PROMOTE
  sched/fair: Not goto more_balance if newly idle and has pending task
    when LBF_NEED_BREAK
  sched/fair: Strive to find a task to migrate if newly idle when
    LB_PROMOTE

 kernel/sched/core.c     |   3 +
 kernel/sched/fair.c     | 220 +++++++++++++++++++++++++++++++++++-----
 kernel/sched/features.h |  24 +++++
 kernel/sched/sched.h    |   2 +
 4 files changed, 224 insertions(+), 25 deletions(-)

-- 
2.34.1