Re: [PATCH v5 13/29] sched/deadline: Keep departing bandwidth across a root domain rebuild

From: Juri Lelli

Date: Mon Nov 16 2026 - 05:21:42 EST


On 11/11/26 14:19, Mathieu Desnoyers wrote:
> On 2026-11-11 11:19, Peter Zijlstra wrote:
> > On Wed, Nov 11, 2026 at 10:35:17AM -0500, Mathieu Desnoyers wrote:
> >
> > > Measured in a 64-vCPU guest with sleeping deadline tasks:
> > >
> > > tasks today: rebuild, longest irqs-off single list, all irqs-off
> > > 1000 0.75ms, under 0.1ms 0.12ms (0.19ms worst)
> > > 10000 7.4ms, under 0.1ms (0.3ms once) 0.9ms (1.2ms worst)
> > >
> > > The rebuild itself is 8 times shorter. That comes from updating the
> > > extra_bw of the CPUs once per root domain, not once per task. But the
> > > longest section with interrupts disabled grows with the number of
> > > deadline tasks. I do not think that is a step forward for a real-time
> > > class.
> >
> > This is domain rebuild? Do we really care about performance there?
>
> My worry is not performance, but rather the contribution of this
> "iteration on all tasks" on the irqoff latency of the system, which
> is relevant for RT.

My take, of course let's have Peter keep me honest here. :)

> What I don't know:
>
> 1) is 10k deadline tasks something that we should consider as realistic
> for sched deadline, or it's way beyond practical scale ?

It doesn't seem a very realistic scale to me. Yes, I've used this kind
of numbers in the (far!) past when running experiments with DEADLINE,
but I can't think of a real workload reaching these numbers.

> 2) is a irqoff latency of 1.2ms (rather than 0.3ms) something which is
> relevant in terms of irq off latency on a preempt rt kernel, or is it
> in the noise ?

Very relevant I'd say, but 0.3ms, unfortunately, very relevant already.

> 3) are those domain rebuild expected to happen with a live RT workload,
> or are those operations expected to never happen on a RT system under
> real-time constraints ?

In small systems, single workload with few housekeeping CPUs, I would
expect the setup to happen only once before the workload is started. So
no worries here. But we seem to have people interested in consolidating
multiple workloads on big machine by, say, using containers. So, in this
case there are several exclusive root domains each running its own RT
workload. It would be undesirable (to say the least :) that a
reconfiguration of one (or a subset) of such domains could potentially
introduce latencies on all the others.

All that said, I now wonder a couple of things (which can be addressed
separately as an RFC if also Peter agrees we don't need to tackle this
right away):

- The 8x from updating extra_bw once per root domain instead of once per
task does not look like it needs the big lock? Maybe it's a nice
improvement we could have on the current scheme as well?

- The deeper issue the scenario above exposes is scope, not lock
granularity. I wonder if a rebuild really needs to re-count root
domains whose span and membership did not change. Maybe some kind of
re-walk of only the domains that actually moved CPUs would shrink the
scope making reconfiguring one partition a non-event for the others?
Again, we can think about/tackle this separately if it makes sense.

Thanks!
Juri