Re: [ANNOUNCE] linux-7.2-ck1, MuQSS CPU scheduler for linux-7.2

[Steven Rostedt]

On Sun, 23 Aug 2026 12:08:19 +1000
Con Kolivas <con@xxxxxxxxxxx> wrote:

> Hi again Steve et. al
> 
> > > [1] https://lwn.net/Articles/1020596/
> > > [2] https://docs.google.com/presentation/d/1dtm0AiiTI30gTFeKj95vmSyirYmk5_QiR_lh17l_Moo/edit?usp=sharing  
> 
> I'm now caught up with your presentation as presented in the youtube
> video linked in the lwn article. Thanks, very informative and
> thoughtful. I was unable to access the google doc but have requested
> read access - though I believe it was all presented on the video.
> 
> Your governor idea is not as dissimilar to plugsched as may appear on
> the surface. Plugsched built in all the schedulers into the kernel and
> allowed you to boot the scheduler of your choice at boot time; it was
> not to just build one scheduler into the kernel. Making it switch on
> the fly was a pipe-dream goal but since it got shot down in
> spectacular fashion I did not pursue it further. Its code is also so
> outdated that literally nothing is of relevance in the current kernel
> tree.

Yeah, the main reason I call it a "governor" and not "plugable" is
because I wanted to stress that it's not for any scheduler. It is for
an environment. Similar to the power management governors. Saying
"plugable" gives a "plug and play" feel that I wanted to avoid. Ingo's
complaint back then was that we would have hundreds of schedulers. Now
with sched_ext, that's exactly what we have. Thus, the governor was an
idea to bring back collaboration between folks that work in the same
environment.

> 
> If you do pursue the governor idea there are a few things worth noting
> about how high up and broad the hooks need to be.
> One overhead problem with plugsched was it added a layer of
> indirection to every single scheduler function call that was shared
> between different schedulers. The cost of this may be considered
> either trivially irrelevant or not remotely worth it depending on your
> viewpoint. A the time I wrote plugsched, Itanic[sic] was still an
> active architecture and the indirection was considered a huge
> downside.

I'm not sure you are aware that the Linux kernel today has something
called a "static_call"[1]. It's a direct function call that can be
switched at runtime with run-time code modification. It is something
that I was planning on using.

> There are four broad aspects to achieving low latency with muqss which
> all need to be adopted to reproduce its behaviour, in order of
> decreasing importance.
> 1. Policy - the simple ordering aspect based on deadline, timeslice
> interval etc. based on a shared monotonically increasing nanosecond
> time counter.
> 2. Shared access to a global queue - BFS did this by having only one
> queue. MuQSS was created as a way to address scalability concerns by
> reintroducing separate runqueues. It became clear very quickly that
> policy alone did not reproduce the behaviour of BFS and that's where
> the idea for having shared runqueues came about. The more the
> runqueues were shared, the closer the latency approximated BFS'. The
> default configuration chooses MC - Multicore. For virtually all
> desktops and mobile devices that means they all end up with one
> runqueue anyway. It is pre-configurable in kconfig, but also boot-time
> selectable.

> 3. Busy and idle load balancing. In MuQSS' case the busy balancing
> happens by proxy through the next task selection, but idle balancing
> is handled separately. Mainline handles both of these separately from
> policy.

For my idea, I would have the governors only affect SCHED_OTHER and
SCHED_IDLE. The RT and DL schedulers would not be affected. But having
the governor take over the idle and other tasks, it would have more
control of what it could do.

> 4. Highres timer based scheduling to effect the nanosecond timers.
> This is to disentangle the scheduler's latency dependency on the
> chosen jiffy Hz which ties all other subsystem components to that
> resolution and/or overhead.

I believe mainline is going in this direction too.

> 
> None of these are insurmountable endpoints with enough LLM tokens, but
> I suspect there will at least be one/some indirection somewhere in the
> implementation.

Again, indirection is solved by the static calls. When spectre and
meltdown mitigations were introduced into the kernel, indirect calls
took a hugh hit. Tracepoints used them quite extensively and it slowed
down hackbench with the sched_switch tracepoint active by 10%!

I started working on a way to replace indirect calls with a direct call
that could be modified. Then Josh Poimbeouf took over and Linus didn't
like the implementation.Finally, Peter Zijlstra got it into the kernel.

They are expensive to change, but for things that do not change often
(like enabling a tracepoint, picking a KVM implementation, or picking a
specific scheduler governor) it is really useful. The hackbench
slowdown disappeared and tracepoints are actually even faster than what
they were with the indirect calls.

-- Steve

[1] https://lwn.net/Articles/815908/