Re: [PATCH 1/2] sched/deadline: add per rq tracking of admitted bandwidth

[Juri Lelli]

Hi Peter,

On 24/02/16 20:17, Peter Zijlstra wrote:
> On Fri, Feb 12, 2016 at 06:05:30PM +0100, Peter Zijlstra wrote:
> > Having two separate means of accounting this also feels more fragile
> > than one would want.
> > 
> > Let me think a bit about this.
> 
> I think there's a fundamental problem that makes the whole notion of
> per-rq accounting 'impossible'.
>
> On hot-unplug we only migrate runnable tasks, all blocked tasks remain
> on the dead cpu. This would very much include their bandwidth
> requirements.
> 
> This means that between a hot-unplug and the moment that _all_ those
> blocked tasks have ran at least once, the sum of online bandwidth
> doesn't match and we can get into admission trouble (same for GRUB,
> which can also use per-rq bw like this).
> 
> The main problem is that there is no real way to find blocked tasks;
> currently the only way is to iterate _all_ tasks and filter on
> task_cpu().
> 
> We could of course add a blocked tree/list for deadline tasks, to
> explicitly keep track of all these; this would allow migrating blocked
> tasks on hotplug and avoid the real ugly I think. But I've not tried
> yet.
> 

Argh, this makes lot of sense to me. I've actually pondered a tree/list
solution, but then decided to try the cumulative approach because it
looked nicer. But it contains holes, I'm afraid. As Luca already said,
GRUB shouldn't have these problems though.

I'll try and see what introducting a list of blocked/throttled deadline
tasks means, considering also the interaction with cpusets and such.
Maybe it's simpler than it seems.

I'm not sure this will come anytime soon, unfortunately. I'm almost 100%
on the sched-freq/schedutil discussion these days.

Anyway, do you also think that what we want to solve the root domain
issue is something based on rq_online/offline and per-rq information?
Everything else that I tried or thought of was broken/more horrible. :-/

Best,

- Juri