Re: [RFC PATCH v7 00/29] Hierarchical Constant Bandwidth Server
From: Juri Lelli
Date: Thu Nov 19 2026 - 06:24:01 EST
Hello,
On 17/11/26 22:07, Tejun Heo wrote:
> Hello, Yuri.
>
> The summary sounds fine for the most part. Some comments below.
Sorry to interject again, but I had a thought that I would like to
share. Also, the very similar names probably don't help keeping track of
who said what, but anyway. :P
> On Wed, Nov 11, 2026 at 06:56:57PM +0100, Yuri Andriaccio wrote:
> > - `cpu.rt.max` only accepts per-CPU settings, so the global "default" is
> > removed. Removing all overrides means the cgroup is a "max" cgroup, while
> > just a single override means that the cgroup has its own budget.
> > + This new interface disallows zero-runtime overrides, as a zero override
> > is equal to an empty override when there is at least one non-zero override.
> > As an example: [<CPU0 10ms 100ms>, <CPU1 0ms 100ms>] is equal to just
> > having [<CPU0 10ms 100ms>]. This seems a bit odd, how to make this better?
>
> It's odd because a missing entry means "max" when there are no entries
> and zero when there are some. Can a missing entry always mean "max",
> per CPU? On a CPU that a cgroup has no entry for, its tasks would be
> served by the nearest ancestor which has one. Zero then becomes a
> regular value which is shown on read, and the two configs in your
> example are different. The former can't run RT tasks on CPU1. The
> latter runs them on the ancestor's budget there.
>
> This means that a cgroup can have its own budget on some CPUs and be
> "max" on the others. That is a fundamental part of the interface, so I
> think it's best to get it right from the get-go rather than starting
> with a restriction.
>
Agree on making zero a regular value and on a missing entry always
meaning "max". There is one case where "the ancestor's budget" doesn't
exist though, and that is when the walk ends at the root.
The root doesn't have a server. Its cpu.rt.max is the pool of
SCHED_DEADLINE bandwidth that child servers are carved from, and
FIFO/RR tasks served by the root just run in the root runqueue, outside
HCBS (as with !RT_GROUP_SCHED). So take a top level group with
0 runtime=10000 period=100000
no entry for CPU1, and a few while(1) FIFO tasks allowed on CPUs 0-1.
On CPU0 they are throttled by the group's server. Once they get to
CPU1 (wakeup placement, push/pull, affinity changes) they run in the
root runqueue and nothing throttles them, only DL tasks and the
fair/ext servers. The group reads as 10% of CPU0, root reads as
runtime=100000 everywhere, and the admin reasonably thinks the group is
contained. It isn't.
Making a missing entry mean zero at the top would bring back the
ambiguity you pointed out, so what about keeping your semantics and
adding a validity rule instead:
A cgroup can be served by the root either on all of its CPUs or on
none of them.
In practice, a cgroup whose parent is entirely root served (e.g. a
top level group) has entries for all possible CPUs or for none. A
write that would leave some CPUs served by a server (or zero) and
others by the root fails with -EINVAL, like the existing admission
checks. Further down, a missing entry is fine and means the parent's
context on that CPU, which is always a real server or an explicit
zero.
With that, per CPU:
- no entry: served by the parent's context on that CPU
- runtime=0: no RT on that CPU, shown on read
- runtime=N period=P: own server on that CPU
A missing entry never changes meaning; the rule only rejects the
configurations where it would escape HCBS. For the example above, the
group must say "1 runtime=0 ..." (no RT on CPU1) or give CPU1 a
budget. The same config under a parent with servers on CPUs 0-1 is
accepted, and CPU1 runs on the parent's server.
What do you think?
> The root has no ancestor to fall back to, so a missing entry can't mean
> anything there. It has to always carry an entry for every possible CPU,
> which defaults to zero, and reject "max":
>
> 0-127 runtime=0 period=1000000
Also, related to the above: on the root these numbers mean something
else (how much DL bandwidth cgroups may reserve, not a server for the
root's tasks), and showing them in the same file and format is part of
what makes the example misleading. Would you be OK with dropping
cpu.rt.max on the root and exposing the parameters through a separately
named root-only file (or the existing sysctls)? That way, having no
entry all the way up consistently means unmanaged FIFO/RR, and nothing
on the root looks like a budget those tasks share.
Thanks!
Juri