Re: [Patch v9 1/8] sched/pelt: Add support to track thermal pressure

[Thara Gopinath]

On 02/13/2020 07:29 AM, Amit Kucheria wrote:
> On Wed, Jan 29, 2020 at 4:06 AM Thara Gopinath
> <thara.gopinath@xxxxxxxxxx> wrote:
>>
>> Extrapolating on the existing framework to track rt/dl utilization using
>> pelt signals, add a similar mechanism to track thermal pressure. The
>> difference here from rt/dl utilization tracking is that, instead of
>> tracking time spent by a cpu running a rt/dl task through util_avg, the
>> average thermal pressure is tracked through load_avg. This is because
>> thermal pressure signal is weighted "delta" capacity and is not
>> binary(util_avg is binary). "delta capacity" here means delta between the
>> actual capacity of a cpu and the decreased capacity a cpu due to a thermal
>> event.
>>
>> In order to track average thermal pressure, a new sched_avg variable
>> avg_thermal is introduced. Function update_thermal_load_avg can be called
>> to do the periodic bookkeeping (accumulate, decay and average) of the
>> thermal pressure.
>>
>> Signed-off-by: Thara Gopinath <thara.gopinath@xxxxxxxxxx>
>> Reviewed-by: Vincent Guittot <vincent.guittot@xxxxxxxxxx>
>> ---
>>
>> v6->v7:
>>         - Added CONFIG_HAVE_SCHED_THERMAL_PRESSURE to stub out
>>           update_thermal_load_avg in unsupported architectures as per
>>           review comments from Peter, Dietmar and Quentin.
>>         - Updated comment for update_thermal_load_avg as per review
>>           comments from Peter and Dietmar.
>> v7->v8:
>>         - Fixed typo in defining update_thermal_load_avg which was
>>           causing build errors (reported by kbuild test report)
>> v8->v9:
>>         - Defined thermal_load_avg to read rq->avg_thermal.load_avg and
>>           avoid cacheline miss in unsupported cases as per Peter's
>>           suggestion.
>>
>>  include/trace/events/sched.h |  4 ++++
>>  init/Kconfig                 |  4 ++++
>>  kernel/sched/pelt.c          | 31 +++++++++++++++++++++++++++++++
>>  kernel/sched/pelt.h          | 31 +++++++++++++++++++++++++++++++
>>  kernel/sched/sched.h         |  3 +++
>>  5 files changed, 73 insertions(+)
>>
>> diff --git a/include/trace/events/sched.h b/include/trace/events/sched.h
>> index 420e80e..a8fb667 100644
>> --- a/include/trace/events/sched.h
>> +++ b/include/trace/events/sched.h
>> @@ -613,6 +613,10 @@ DECLARE_TRACE(pelt_dl_tp,
>>         TP_PROTO(struct rq *rq),
>>         TP_ARGS(rq));
>>
>> +DECLARE_TRACE(pelt_thermal_tp,
>> +       TP_PROTO(struct rq *rq),
>> +       TP_ARGS(rq));
>> +
>>  DECLARE_TRACE(pelt_irq_tp,
>>         TP_PROTO(struct rq *rq),
>>         TP_ARGS(rq));
>> diff --git a/init/Kconfig b/init/Kconfig
>> index bd9f1fd..055c3bf 100644
>> --- a/init/Kconfig
>> +++ b/init/Kconfig
>> @@ -463,6 +463,10 @@ config HAVE_SCHED_AVG_IRQ
>>         depends on IRQ_TIME_ACCOUNTING || PARAVIRT_TIME_ACCOUNTING
>>         depends on SMP
>>
>> +config HAVE_SCHED_THERMAL_PRESSURE
>> +       bool "Enable periodic averaging of thermal pressure"
>> +       depends on SMP
>> +
>>  config BSD_PROCESS_ACCT
>>         bool "BSD Process Accounting"
>>         depends on MULTIUSER
>> diff --git a/kernel/sched/pelt.c b/kernel/sched/pelt.c
>> index bd006b7..5d1fbf0 100644
>> --- a/kernel/sched/pelt.c
>> +++ b/kernel/sched/pelt.c
>> @@ -367,6 +367,37 @@ int update_dl_rq_load_avg(u64 now, struct rq *rq, int running)
>>         return 0;
>>  }
>>
>> +#ifdef CONFIG_HAVE_SCHED_THERMAL_PRESSURE
>> +/*
>> + * thermal:
>> + *
>> + *   load_sum = \Sum se->avg.load_sum but se->avg.load_sum is not tracked
>> + *
>> + *   util_avg and runnable_load_avg are not supported and meaningless.
>> + *
>> + * Unlike rt/dl utilization tracking that track time spent by a cpu
>> + * running a rt/dl task through util_avg, the average thermal pressure is
>> + * tracked through load_avg. This is because thermal pressure signal is
>> + * weighted "delta" capacity and is not binary(util_avg is binary). "delta
> 
> May I suggest a slight rewording here and in the commit description,
> 
>    This is because the thermal pressure signal is weighted "delta"
> capacity unlike util_avg which is binary.

Sure! Will fix it.

> 
> It would also help, if you expanded on what you mean by binary in the
> commit description and how the delta capacity is weighted.

I don't understand this. Binary means  0 or 1. delta capacity is time
weighted, i will update this.

> 
>> + * capacity" here means delta between the actual capacity of a cpu and the
>> + * decreased capacity a cpu due to a thermal event.
> 
> Use consistent wording throughout the series - either capped or
> decreased capacity.
> 
>> + */
> 
> This could be shortened to:
> 
> Delta capacity of cpu = Actual capacity - Capped capacity due to thermal event

Will fix this.

> 
>> +int update_thermal_load_avg(u64 now, struct rq *rq, u64 capacity)
>> +{
>> +       if (___update_load_sum(now, &rq->avg_thermal,
>> +                              capacity,
>> +                              capacity,
>> +                              capacity)) {
>> +               ___update_load_avg(&rq->avg_thermal, 1, 1);
>> +               trace_pelt_thermal_tp(rq);
>> +               return 1;
>> +       }
>> +
>> +       return 0;
>> +}
>> +#endif
>> +
>>  #ifdef CONFIG_HAVE_SCHED_AVG_IRQ
>>  /*
>>   * irq:
>> diff --git a/kernel/sched/pelt.h b/kernel/sched/pelt.h
>> index afff644..916979a 100644
>> --- a/kernel/sched/pelt.h
>> +++ b/kernel/sched/pelt.h
>> @@ -7,6 +7,26 @@ int __update_load_avg_cfs_rq(u64 now, struct cfs_rq *cfs_rq);
>>  int update_rt_rq_load_avg(u64 now, struct rq *rq, int running);
>>  int update_dl_rq_load_avg(u64 now, struct rq *rq, int running);
>>
>> +#ifdef CONFIG_HAVE_SCHED_THERMAL_PRESSURE
>> +int update_thermal_load_avg(u64 now, struct rq *rq, u64 capacity);
>> +
>> +static inline u64 thermal_load_avg(struct rq *rq)
>> +{
>> +       return READ_ONCE(rq->avg_thermal.load_avg);
>> +}
>> +#else
>> +static inline int
>> +update_thermal_load_avg(u64 now, struct rq *rq, u64 capacity)
>> +{
>> +       return 0;
>> +}
>> +
>> +static inline u64 thermal_load_avg(struct rq *rq)
>> +{
>> +       return 0;
>> +}
>> +#endif
>> +
>>  #ifdef CONFIG_HAVE_SCHED_AVG_IRQ
>>  int update_irq_load_avg(struct rq *rq, u64 running);
>>  #else
>> @@ -159,6 +179,17 @@ update_dl_rq_load_avg(u64 now, struct rq *rq, int running)
>>  }
>>
>>  static inline int
>> +update_thermal_load_avg(u64 now, struct rq *rq, u64 capacity)
>> +{
>> +       return 0;
>> +}
>> +
>> +static inline u64 thermal_load_avg(struct rq *rq)
>> +{
>> +       return 0;
>> +}
>> +
>> +static inline int
>>  update_irq_load_avg(struct rq *rq, u64 running)
>>  {
>>         return 0;
>> diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
>> index 1a88dc8..1f256cb 100644
>> --- a/kernel/sched/sched.h
>> +++ b/kernel/sched/sched.h
>> @@ -944,6 +944,9 @@ struct rq {
>>  #ifdef CONFIG_HAVE_SCHED_AVG_IRQ
>>         struct sched_avg        avg_irq;
>>  #endif
>> +#ifdef CONFIG_HAVE_SCHED_THERMAL_PRESSURE
>> +       struct sched_avg        avg_thermal;
>> +#endif
>>         u64                     idle_stamp;
>>         u64                     avg_idle;
>>
>> --
>> 2.1.4
>>


-- 
Warm Regards
Thara