Re: [RFC PATCH 1/6] sched/dl: Improve deadline admission control for asymmetric CPU capacities

[Dietmar Eggemann]

On 5/7/19 4:43 PM, luca abeni wrote:
> On Tue, 7 May 2019 15:31:27 +0100
> Quentin Perret <quentin.perret@xxxxxxx> wrote:
> 
>> On Tuesday 07 May 2019 at 16:25:23 (+0200), luca abeni wrote:
>>> On Tue, 7 May 2019 14:48:52 +0100
>>> Quentin Perret <quentin.perret@xxxxxxx> wrote:
>>>   
>>>> Hi Luca,
>>>>
>>>> On Monday 06 May 2019 at 06:48:31 (+0200), Luca Abeni wrote:  

[...]

>> Right and things moved recently in this area, see bb1fbdd3c3fd
>> ("sched/topology, drivers/base/arch_topology: Rebuild the sched_domain
>> hierarchy when capacities change")
> 
> Ah, thanks! I missed this change when rebasing the patchset.
> I guess this part of the patch has to be updated (and probably became
> useless?), then.

[...]

>>> This achieved the effect of correctly setting up the "rd_capacity"
>>> field, but I do not know if there is a better/simpler way to achieve
>>> the same result :)  
>>
>> OK, that's really an implementation detail, so no need to worry too
>> much about it at the RFC stage I suppose :-)

What about we integrate the code to calculate the rd capacity into
build_sched_domains() (next to the code to establish the rd
max_cpu_capacity)?

root@juno:~# cat /sys/devices/system/cpu/cpu*/cpu_capacity
446
1024
1024
446
446
446

root@juno:~# dmesg | grep "rd capacity"
/*  before CPUfreq max CPU freq calibration */
[    0.749389] rd span: 0-5 rd capacity: 4360 max cpu_capacity: 1024
/* after CPUfreq max CPU freq calibration */
[    3.372759] rd span: 0-5 rd capacity: 3832 max cpu_capacity: 1024

/* 2*1024 + 4*446 = 3832 */

root@juno:~# echo 0 > /sys/devices/system/cpu/cpu5/online

root@juno:~# dmesg | grep "rd capacity"
...
[ 2715.068198] rd span: 0-4 rd capacity: 3386 max cpu_capacity: 1024

root@juno:~# echo 1 > /sys/devices/system/cpu/cpu5/online

root@juno:~# dmesg | grep "rd capacity"
...
[ 2807.200662] rd span: 0-5 rd capacity: 3832 max cpu_capacity: 1024


--->8---

@@ -768,6 +769,7 @@ struct root_domain {
        cpumask_var_t           rto_mask;
        struct cpupri           cpupri;
 
+       unsigned long           capacity;
        unsigned long           max_cpu_capacity;
 
        /*
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index f751ce0b783e..68acdca27eaf 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -2000,6 +2000,9 @@ build_sched_domains(const struct cpumask *cpu_map, struct sched_domain_attr *att
                if (rq->cpu_capacity_orig > READ_ONCE(d.rd->max_cpu_capacity))
                        WRITE_ONCE(d.rd->max_cpu_capacity, rq->cpu_capacity_orig);
 
+               WRITE_ONCE(d.rd->capacity,
+                          READ_ONCE(d.rd->capacity) + rq->cpu_capacity_orig);
+
                cpu_attach_domain(sd, d.rd, i);
        }
        rcu_read_unlock();
@@ -2008,8 +2011,9 @@ build_sched_domains(const struct cpumask *cpu_map, struct sched_domain_attr *att
                static_branch_enable_cpuslocked(&sched_asym_cpucapacity);
 
        if (rq && sched_debug_enabled) {
-               pr_info("root domain span: %*pbl (max cpu_capacity = %lu)\n",
-                       cpumask_pr_args(cpu_map), rq->rd->max_cpu_capacity);
+               pr_info("rd span: %*pbl rd capacity: %lu max cpu_capacity: %lu\n",
+                       cpumask_pr_args(cpu_map), rq->rd->capacity,
+                       rq->rd->max_cpu_capacity);
        }