Re: [RFC PATCH v7 00/29] Hierarchical Constant Bandwidth Server
From: Juri Lelli
Date: Thu Nov 19 2026 - 09:36:17 EST
On 19/11/26 15:27, luca abeni wrote:
> Hi Juri,
>
> On Thu, 19 Nov 2026 12:23:48 +0100
> Juri Lelli <juri.lelli@xxxxxxxxxx> wrote:
> [...]
> > > On Wed, Nov 11, 2026 at 06:56:57PM +0100, Yuri Andriaccio wrote:
> > > > - `cpu.rt.max` only accepts per-CPU settings, so the global
> > > > "default" is removed. Removing all overrides means the cgroup is
> > > > a "max" cgroup, while just a single override means that the
> > > > cgroup has its own budget.
> > > > + This new interface disallows zero-runtime overrides, as a
> > > > zero override is equal to an empty override when there is at
> > > > least one non-zero override. As an example: [<CPU0 10ms 100ms>,
> > > > <CPU1 0ms 100ms>] is equal to just having [<CPU0 10ms 100ms>].
> > > > This seems a bit odd, how to make this better?
> > >
> > > It's odd because a missing entry means "max" when there are no
> > > entries and zero when there are some. Can a missing entry always
> > > mean "max", per CPU? On a CPU that a cgroup has no entry for, its
> > > tasks would be served by the nearest ancestor which has one. Zero
> > > then becomes a regular value which is shown on read, and the two
> > > configs in your example are different. The former can't run RT
> > > tasks on CPU1. The latter runs them on the ancestor's budget there.
> > >
> > > This means that a cgroup can have its own budget on some CPUs and be
> > > "max" on the others. That is a fundamental part of the interface,
> > > so I think it's best to get it right from the get-go rather than
> > > starting with a restriction.
> > >
> >
> > Agree on making zero a regular value and on a missing entry always
> > meaning "max".
>
> I think "0" as a regular value is agreed upon by everyone :)
> But I would avoid "missing entries meaning max"... See my previous
> email.
> Basically, I think a cgroup should either inherit from the parent for
> every core, or define a deadline entity (even one with 0 runtime) on
> every core. Mixing the two is, in my opinion, problematic (for example,
> it would change the serving reservation when a task migrates from one
> core to another)).
> I hope this "all-or-nothing" approach is acceptable...
>
>
> [...]
> > Making a missing entry mean zero at the top would bring back the
> > ambiguity you pointed out, so what about keeping your semantics and
> > adding a validity rule instead:
> >
> > A cgroup can be served by the root either on all of its CPUs or on
> > none of them.
>
> As mentioned above, I would be even more drastic:
> A cgroup can be served by an ancestor either on all of its CPUs or on
> none of them.
>
> What do you think? Can this approach be acceptable?
Yes, works for me. The example I gave is just the extreme case of the
problem you describe (the "other reservation" being no reservation at
all), and your argument covers the nested case I still allowed.
Tejun, I think this still addresses the original concern: zero is a
regular value with a single meaning ("no RT on this CPU"), and no
entry only exists for a cgroup that is "max" as a whole. It's not a
restriction to relax later, though: a task changing reservation on
migration is what we want to rule out.
> > In practice, a cgroup whose parent is entirely root served (e.g. a
> > top level group) has entries for all possible CPUs or for none. A
> > write that would leave some CPUs served by a server (or zero) and
> > others by the root fails with -EINVAL, like the existing admission
> > checks. Further down, a missing entry is fine and means the parent's
> > context on that CPU, which is always a real server or an explicit
> > zero.
> >
> > With that, per CPU:
> >
> > - no entry: served by the parent's context on that CPU
> > - runtime=0: no RT on that CPU, shown on read
> > - runtime=N period=P: own server on that CPU
>
> If what I wrote above is OK, I would just avoid the "no entry" case,
> and I would define the interface so that missing entries are not
> possible (for example, see my previous email: we first set a (runtime,
> period) for all core, then we can override some of them... But not with
> "max").
OK.
> > > The root has no ancestor to fall back to, so a missing entry can't
> > > mean anything there. It has to always carry an entry for every
> > > possible CPU, which defaults to zero, and reject "max":
> > >
> > > 0-127 runtime=0 period=1000000
> >
> > Also, related to the above: on the root these numbers mean something
> > else (how much DL bandwidth cgroups may reserve, not a server for the
> > root's tasks), and showing them in the same file and format is part of
> > what makes the example misleading. Would you be OK with dropping
> > cpu.rt.max on the root and exposing the parameters through a
> > separately named root-only file (or the existing sysctls)? That way,
> > having no entry all the way up consistently means unmanaged FIFO/RR,
> > and nothing on the root looks like a budget those tasks share.
>
> Regardless of the filenames we decide, the defaults for these settings
> indicate the fraction of CPU time that can be consumed by all real-time
> tasks that are not in the root cgroup, right?
Not quite, I think? They cap the bandwidth that can be reserved for
cgroup servers, so they only cover RT tasks that are served by one.
RT tasks in a non-root "max" cgroup (with no configured ancestor) are
not in the root cgroup, but run in the root runqueue and aren't limited
by those values at all (sched_rt_can_attach() lets them in
unconditionally). Which is the kind of confusion I'd like the root
interface to avoid.
Thanks,
Juri