[PATCH 0/3] Reduce spurious RT_PUSH_IPI storms

[Mel Gorman]

RT_PUSH_IPI is intended to minimise latency and rq lock contention when
migrating tasks to balance RT task distribution.  For example, if a high
priority task changes to a lower priority then CPUs with off-CPU RT tasks
may be sent an IPI to "push" tasks away to an available CPU. The problem
is that if a CPU is rt_overloaded but has no tasks that can be pushed then
it continues to receive IPIs -- specifically tasks that are migrate_disabled.

An issue was identified where a CPU received a large number of IPIs in a
short space of time but no tasks could move, it was still marked rt
overloaded and continued to receive IPIs.

This problem was examined in commit dae5c0292080 ("sched/rt,dl: Skip
migrate-disabled tasks when picking a push candidate") but the fix is
incomplete as noted in the changelog.

The original bug against an old kernel was due to pgrep running with RT
priority on an isolated CPU with an active RT task. It was acknowledged
that the "proper" fix was to move pgrep, drop its priority or both but
the underlying reason for the storm was pgrep being spinlock-intensive
and excessive IPIs being sent which is still sub-optimal behaviour. A test
case creates a "victim" task with short work/sleep cycles allowed to run
on CPU1 and CPU2 that optionally disables migration (md=1) while running.
A second task mostly does work pinned on CPU1 with short sleeps, a third
task mostly sleeps on CPU2 so CPU2 pulls frequently.  The results are

| Kernel (7.4-rc2)       |     CPU1-md=1 |   CPU2-md=1 |    CPU1-md=0 |    CPU2-md=0 |
|------------------------|---------------|-------------|--------------|--------------|
| vanilla-rt-push-ipi    | 41474.7 ±0.1% |           0 | 4784.4 ±0.2% |   85.0 ±9.9% |
| series-rt-push-ipi     |             0 | 0.3 ±380.6% | 4792.1 ±0.3% |  87.1 ±10.8% |
| series-no-rt-push-ipi  |             0 |           0 |            0 |            0 |
(RT push IPIs per CPU per 5 s interval, mean ±CoV over 30 intervals,
 md=1 means the "victim" with cpumask for CPUs 1-2 disables migration)

The vanilla 7.4-rc2 kernel fired 40k+ IPIs every 5 seconds with low variance
over 30 samples with a migrate-disabled task sitting on CPU1. Separate
testing showed that the IPIs were not resulting in tasks moving and therefore
spurious. The series eliminates the spurious IPIs (second row). Disabling
IPIs also showed no IPIs but it was confirmed that tasks were still moved.

Yousaf (cc'd) is the original author of the first patch, I only decorated it
and provided transport, hence why his Signed-off-by is first on that patch.

Mel Gorman (3):
  sched/rt: Do not account migrate-disabled tasks as pushable
  sched/rt: Check eligibility of pushable task in one place
  sched/rt: Treat migrate-disabled tasks as pushable if RT_PUSH_IPI is
    disabled

 kernel/sched/rt.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

-- 
2.55.0