Re: [PATCH v4] sched/fair: Preserve wake-affine CPU for non-SMT reciprocal sync wakeups

[K Prateek Nayak]

Hello Shrikanth,

On 8/4/2026 10:10 AM, Shrikanth Hegde wrote:
> As I said in v3, before we add bells/whistles to sync path, i want
> to know what is expected of sync behavior today.
> And that should be documented in Documentation/scheduler/
> 
> Be it,
> - current way of hint only and scheduler can still choose an idle core/idle cpu etc.
> - Should it be enforcing it to waker cpu if waker cpu has only one task.
> - Whatever the policy maybe.
> 
> Current api usage is tricky to use and effect is visible in real life workloads.
> The case I mentioned in v3 of networking code using sync api leads to strange
> results due to sync mechanism.
> - It depends whether waker/wakee are running on same node.
> - Result of wake_wide.
> In other end, user sees inconsistent latency/throughput.
> 
> We can keep on adding minor changes to sync api path,
> but one benchmark will benefit and one will suffer.
> Having the behavior documented is a good start.
> 
> Peter, Ingo, Vincent, Mel, Prateek,
> What do you guys think?

Currently it is very arbitrary and WF_SYNC may, or may not, indicate a
true voluntary blocking behavior. For example, anon_pipe_read() uses a
wake_up_interruptible_sync_poll() to wake up writers once reader has
drained the pipe but if you think about it, why would the reader block
soon after just having the data it needed?

Here are the results on my Zen4 system from running perf bench
sched messaging (threads + pipes) at varying worker counts with
all wake_up_interruptible_sync_poll converted to
wake_up_interruptible_poll:

Test:                   tip                     no_sync
 1-groups:         3.79 (0.00 pct)         3.35 (11.60 pct)
 2-groups:         3.85 (0.00 pct)         3.41 (11.42 pct)
 4-groups:         4.02 (0.00 pct)         3.32 (17.41 pct)
 8-groups:         4.33 (0.00 pct)         4.38 (-1.15 pct)
16-groups:         6.09 (0.00 pct)         6.12 (-0.49 pct)
---

So seems like WF_SYNC hint on this machine with perf bench sched
messaging (thread + pipes) pattern is actually holding it back.
Lemme check processes ...

Test:                   tip                     no_sync
 1-groups:         3.48 (0.00 pct)         3.08 (11.49 pct)
 2-groups:         3.80 (0.00 pct)         3.07 (19.21 pct)
 4-groups:         3.91 (0.00 pct)         3.09 (20.97 pct)
 8-groups:         4.13 (0.00 pct)         4.10 (0.72 pct)
16-groups:         5.81 (0.00 pct)         5.74 (1.20 pct)

Similar stuff. At some point it was pretty bad for Zen3 but
situation might have changed since ¯\_(ツ)_/¯ I'll let you
know once I have a machine.

But ... If I have true 1:1 waiting on pipe as in the case of
"perf bench sched pipe -l 1000000" I go from ~2.5usecs/op on
average to ~4.2usecs/op which is close to a 50% increase in the
benchmark time so that WF_SYNC hint can also help if all we have
is looping over a read waiting for one page worth of write.

The way I look at WF_SYNC nowadays is that it indicates a local LLC
wakeup is beneficial. wake_wide() doesn't even consider WF_SYNc and
simply uses wake-wakee flips and then only at want_affine() do we
actually check the sync hint.

Most benefit come from wake_affine_idle() for the 1 task case where
target is set to current and select_idle_sibling() uses that as the
target from there on.

It could purely be a coincidence that it benefits at all - most of
these microbenchmark we have always hit the same two syscall (mostly
read() and write() in turns) and a most of benefit for those comes
from kernel instructions being primed in cache.

I remember a while back, removing the effect of WF_SYNC on the
networking side hampered a lot of performance - especially for
localhost communications. This one specifically
https://lore.kernel.org/lkml/20220711224704.1672831-1-libo.chen@xxxxxxxxxx/

Let me see if things have miraculously changed there too but for
TCP sockets in real world, with blocking for ACKs, I think
WF_SYNC still makes sense there but I feel most of the benefits
from sync are a second order effect.

-- 
Thanks and Regards,
Prateek