[RFC PATCH 6/6] sched/docs: Add documentation for SMT Buddy Assist feature

[Li kunyu]

Document the SMT Buddy Assist mechanism, including its motivation,
enabling method (sysctl interface), and implementation overview. This
helps users understand when and why the feature engages, and how it
interacts with existing CFS load balancing.

Tested-by: Li Wang (Hieco) <li.wang@xxxxxxxxx>
Signed-off-by: Li kunyu <likunyu@xxxxxxxxxxxxxx>
---
 Documentation/scheduler/smt-buddy-assist.rst | 126 +++++++++++++++++++
 1 file changed, 126 insertions(+)
 create mode 100644 Documentation/scheduler/smt-buddy-assist.rst

diff --git a/Documentation/scheduler/smt-buddy-assist.rst b/Documentation/scheduler/smt-buddy-assist.rst
new file mode 100644
index 000000000000..c252f4fe955f
--- /dev/null
+++ b/Documentation/scheduler/smt-buddy-assist.rst
@@ -0,0 +1,126 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+================
+SMT Buddy Assist
+================
+
+1. Introduction
+===============
+
+SMT introduces scalability features that constrain the scheduling domain
+within the LLC (L3) cache via the newidle CPU path. Combined with the ability
+to push CFS tasks to SMT sibling CPUs and to quickly distribute tasks to idle
+CPUs using select_task_rq_fair, this improves overall CPU performance under
+varying loads.
+
+2. Design
+=========
+
+The detailed implementation approach is as follows:
+
+Load balancing-related procedures are primarily categorized into two
+major directions:
+
+1. The periodic load balancing path, comprising periodic load balancing
+and NOHZ kick, retains the original logic and continues to maintain
+overall CPU balance.
+
+2. The newidle CPU path logic has been updated solely to quickly obtain
+efficient tasks. By restricting task acquisition to the LLC (L3)
+scheduling domain and avoiding traversal and task migration at levels
+above it, this change avoids jitter in scenarios where the entire
+machine is idle or the CPU load is relatively balanced. This is because
+each traversal of higher-level scheduling domains remains costly, and
+migrating tasks across dies (LLCs) incurs substantial short-term
+overhead, such as TLB flushes/refills and page migration:
+
+  2.1. When overall CPU utilization is relatively low, avoid the
+scenario where the whole CPU is idle yet cross-die (high-latency) tasks
+are being pulled. In such cases, it is better to run only one logical
+CPU per core to avoid cache interference and frequent TLB flushes. This
+applies when:
+    2.1.1. the CPU has multiple sleeping tasks (with delayed dequeue
+disabled).
+    2.1.2. the number of runnable tasks on the CPU is less than 2.
+
+  2.2. When overall CPU load is relatively balanced, refrain from
+pulling cross-die (high-latency) tasks. This avoids task jitter and
+prevents resource exhaustion (similar to the issues in 2.1).
+
+Of course, the change to the newidle CPU path logic has also introduced
+new issues, including:
+1. bursty task creation, where a few CPUs spawn a large number of
+processes or threads in a short period of time and these tasks cannot be
+migrated to low-load CPUs in a timely manner.
+2. uneven system-state load across CPUs. For example, some CPUs show
+0% to 2% user-mode and 30% to 80% kernel-mode utilization, while others show
+0% to 2% user-mode and 10% to 30% kernel-mode utilization. In addition, more
+than 10% of CPUs remain idle.
+
+To address the aforementioned issues, a more lightweight implementation
+was adopted to avoid the interference caused by the original newidle CPU
+path. Subsequently, select_task_rq_fair() is used to quickly select the
+most suitable CPU, as follows:
+
+1. The original task wake-up logic remains unchanged: it first runs the
+current-CPU wake-up, affinity wake-up, and similar paths. when the task
+count on the found CPU exceeds a threshold (default: 4) a burst-task CPU
+allocation process is used.
+
+2. For newly created processes or threads, there is less cause for
+concern. Since the CPU selection in the selected scheduling domain meets
+the burst-task condition during the slow path, applying the burst-task
+CPU allocation process is optimal, as it can quickly utilize idle CPUs
+and thereby achieve rapid overall task distribution.
+
+This series of patches also introduces a notable design: in the periodic
+load balancing path, tasks are pushed to SMT sibling CPUs to quickly
+distribute load within the same core, while also pulling tasks from
+other CPUs (excluding SMT siblings) during the current scheduling domain
+traversal. This provides a new mechanism for evenly distributing tasks
+across the entire machine and strongly supports the change to the
+newidle CPU path.
+
+2.1 SMT_BUDDY_ASSIST
+--------------------
+
+Enable feature:
+    echo "SMT_BUDDY_ASSIST" > /sys/kernel/debug/sched/features
+
+3. System interface
+-------------------
+
+3.1. smt_buddy_imbalance_pct
+----------------------------
+
+When pushing tasks to SMT siblings, the current CPU must exceed a certain
+imbalance ratio before its CFS runqueue is considered heavily loaded.
+
+The imbalance ratio threshold is constrained to a range of 105% to 200%,
+covering both mild deviation and severe imbalance scenarios.
+
+Mild deviation:
+For example, when the pull imbalance ratio within the SMT domain is around
+110%, this feature provides a comparable and configurable trigger condition.
+
+Severe imbalance:
+For example, when the runnable time ratio of the local CPU's cfs_rq is 20%
+while that of its SMT sibling is 40%, load balancing is explicitly required
+to restore fairness.
+
+3.2. smt_buddy_balance_task
+---------------------------
+
+The task-burst detection mechanism employs a configurable threshold derived
+from two primary metrics: the runqueue waiting time of a task on the local
+CPU, and the comparative cost of migrating that task to a remote (cross-die)
+CPU versus allowing it to execute locally after a prolonged wait.
+
+A default threshold of 4 is selected such that migration is triggered when
+the queuing latency approaches the execution duration of an individual task.
+The marginal benefit of migration increases monotonically with the number of
+tasks enqueued on the local CPU.
+
+A lower bound of 2 accounts for scenarios involving long-running tasks, which
+require more aggressive distribution across the CPU topology to avoid localized
+congestion.
-- 
2.47.3