Re: [PATCH v5 0/2] sched: Enable preferred SMT siblings on NVIDIA Olympus

[K Prateek Nayak]

Hello Andrea,

On 9/9/2026 11:56 AM, Andrea Righi wrote:
> NVIDIA Olympus implements SMT with two symmetric processing elements (PEs).
> When only one PE is active, the core operates in single-thread mode and
> that PE can use the full core resources. When both PEs are active, the core
> operates in two-thread mode and the PEs share those resources. This
> behavior is common to SMT implementations, but Olympus is particularly
> sensitive to brief sibling activations because returning from two-thread
> mode to single-thread mode after a sibling becomes idle is not immediate.
> As described by commit 293f9611ae735 ("sched/fair: Prefer fully idle cores
> for NOHZ balancing"):
> 
>   Briefly activating an otherwise idle sibling can reduce the performance
>   available to the other sibling and this effect does not necessarily end
>   once the activated sibling becomes idle: after the ILB finishes and its
>   CPU enters WFI, full single-thread performance is restored only after the
>   sibling has remained idle for a qualification interval (10 Ki cycles on
>   the tested Vera system).
> 
> That change prevents the NOHZ idle load balancer from unnecessarily waking
> a sibling of a busy PE. However, ordinary task placement can still select
> either sibling of an idle core and repeated changes of the active PE can
> keep Olympus cores in two-thread mode despite little or no useful overlap
> between the siblings.
> 
> This series makes PE0 the preferred sibling of an Olympus core using
> SD_ASYM_PACKING and teaches the fair scheduler's idle-selection paths to
> honor asymmetric SMT priority. The scheduler first selects a candidate CPU
> and core according to its existing placement and capacity rules, then
> chooses the highest-priority available sibling within that core. The
> generic scheduler behavior is enabled only when an architecture supplies
> an SD_ASYM_PACKING SMT domain.
> 
> PE0 and PE1 have equal steady-state capacity, the preference does not
> identify a faster PE. PE0 is used only as a canonical choice when both
> siblings are available. Consistently selecting the same sibling avoids
> alternating the active PE across wakeups, lets PE1 remain idle for longer,
> and allows more cores to remain in, or return to, full-resource
> single-thread mode.
> 
> The series was tested on a two-node Vera system using an 88-thread
> single-precision GEMM on the 88 physical cores of NUMA node 0.
> 
> With the workload allowed to choose either sibling of every core, observed
> throughput improved from approximately 9.4 TFLOP/s on the baseline kernel
> to approximately 10.1 TFLOP/s with this series applied. Repeated runs also
> became more predictable because the workload consistently settled on PE0
> while PE1 remained quiet.
> 
> Changes in v5:
>  - Remove the redundant olympus_prefer_pe0 state (K Prateek Nayak)
>  - Link to v4: https://lore.kernel.org/r/20260908082345.103087-1-arighi@xxxxxxxxxx

My replies failed to keep up with your speed on iterations :-)

Since this is same as v4 and I had taken it for a spin on a
x86 and an ARM64 server, feel free to include:

Reviewed-by: K Prateek Nayak <kprateek.nayak@xxxxxxx>
Tested-by: K Prateek Nayak <kprateek.nayak@xxxxxxx>

-- 
Thanks and Regards,
Prateek