Re: [PATCH] perf/x86/rapl: Enable Core RAPL for AMD

[Wyes Karny]

Hi Stephane,

On 2/21/2023 2:20 PM, Stephane Eranian wrote:
> Hi,
> 
> Testing Wyes' patch for energy-cores on Zen3 server loaded with triad
> bench on socket0:
> 
> $ perf stat --per-core -a -C0-63 -I 1000 -e
> power/energy-cores/,power/energy-pkg/
> #           time core               cpus             counts   unit events
>      1.001019203 S0-D0-C0              1               1.28 Joules
> power/energy-cores/
>      1.001019203 S0-D0-C0              1             231.38 Joules
> power/energy-pkg/
>      1.001019203 S0-D0-C1              1   4,294,967,130.96 Joules
> power/energy-cores/
>      1.001019203 S0-D0-C1              1             231.38 Joules
> power/energy-pkg/
>      1.001019203 S0-D0-C2              1   4,294,967,126.23 Joules
> power/energy-cores/
>      1.001019203 S0-D0-C2              1             231.38 Joules
> power/energy-pkg/
>      1.001019203 S0-D0-C3              1   4,294,967,122.50 Joules
> power/energy-cores/
>      1.001019203 S0-D0-C3              1             231.38 Joules
> power/energy-pkg/
>      1.001019203 S0-D0-C4              1   4,294,967,129.92 Joules
> power/energy-cores/
>      1.001019203 S0-D0-C4              1             231.38 Joules
> power/energy-pkg/
>      1.001019203 S0-D0-C5              1   4,294,967,121.49 Joules
> power/energy-cores/
>      1.001019203 S0-D0-C5              1             231.39 Joules
> power/energy-pkg/
> 
> I think the result of energy-cores is not reliable and I think that is
> why I did not
> include it in the patch.
> 
> Could also be a problem with the kernel code, but I don't know why it would only
> impact energy-cores given energy-pkg looks reasonable here.

This is a kernel issue. I've addressed this in v2.

Thanks,
Wyes
> 
> 
> On Mon, Feb 20, 2023 at 8:53 PM Wyes Karny <wyes.karny@xxxxxxx> wrote:
>>
>> On 20 Feb 13:29, Stephane Eranian wrote:
>>> On Mon, Feb 20, 2023 at 3:45 AM Peter Zijlstra <peterz@xxxxxxxxxxxxx> wrote:
>>>>
>>>> On Fri, Feb 17, 2023 at 04:13:54PM +0000, Wyes Karny wrote:
>>>>> AMD processors support per-package and per-core energy monitoring
>>>>> through RAPL counters which can be accessed by users running in
>>>>> supervisor mode.
>>>>>
>>>>> Core RAPL counters gives power consumption information per core.  For
>>>>> AMD processors the package level RAPL counter are already exposed to
>>>>> perf. Expose the core level RAPL counters also.
>>>>>
>>>>> sudo perf stat -a --per-core -C 0-127 -e power/energy-cores/
>>>>>
>>>>> Output:
>>>>> S0-D0-C0           2               8.73 Joules power/energy-cores/
>>>>> S0-D0-C1           2               8.73 Joules power/energy-cores/
>>>>> S0-D0-C2           2               8.73 Joules power/energy-cores/
>>>>> S0-D0-C3           2               8.73 Joules power/energy-cores/
>>>>> S0-D0-C4           2               8.73 Joules power/energy-cores/
>>>>>
>>>>> Signed-off-by: Wyes Karny <wyes.karny@xxxxxxx>
>>>>> ---
>>>>>  arch/x86/events/rapl.c | 5 +++--
>>>>>  1 file changed, 3 insertions(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/arch/x86/events/rapl.c b/arch/x86/events/rapl.c
>>>>> index 52e6e7ed4f78..d301bbbc3b93 100644
>>>>> --- a/arch/x86/events/rapl.c
>>>>> +++ b/arch/x86/events/rapl.c
>>>>> @@ -537,7 +537,7 @@ static struct perf_msr intel_rapl_spr_msrs[] = {
>>>>>   * - want to use same event codes across both architectures
>>>>>   */
>>>>>  static struct perf_msr amd_rapl_msrs[] = {
>>>>> -     [PERF_RAPL_PP0]  = { 0, &rapl_events_cores_group, 0, false, 0 },
>>>>> +     [PERF_RAPL_PP0]  = { MSR_AMD_CORE_ENERGY_STATUS, &rapl_events_cores_group, test_msr, false, RAPL_MSR_MASK },
>>>>
>>>> Stephane, this was an oversight?
>>>>
>>> I think it may depend on the CPU model. I remember it returning either
>>> 0 or bogus values on my systems. They may have improved that.
>>> The commit msg does not show which CPU model this is run on.
>>
>> I've tested this on Zen 2, 3 and 4 server systems.
>>
>> Thanks,
>> Wyes
>>>
>>>>
>>>>>       [PERF_RAPL_PKG]  = { MSR_AMD_PKG_ENERGY_STATUS,  &rapl_events_pkg_group,   test_msr, false, RAPL_MSR_MASK },
>>>>>       [PERF_RAPL_RAM]  = { 0, &rapl_events_ram_group,   0, false, 0 },
>>>>>       [PERF_RAPL_PP1]  = { 0, &rapl_events_gpu_group,   0, false, 0 },
>>>>> @@ -764,7 +764,8 @@ static struct rapl_model model_spr = {
>>>>>  };
>>>>>
>>>>>  static struct rapl_model model_amd_hygon = {
>>>>> -     .events         = BIT(PERF_RAPL_PKG),
>>>>> +     .events         = BIT(PERF_RAPL_PP0) |
>>>>> +                       BIT(PERF_RAPL_PKG),
>>>>>       .msr_power_unit = MSR_AMD_RAPL_POWER_UNIT,
>>>>>       .rapl_msrs      = amd_rapl_msrs,
>>>>>  };
>>>>> --
>>>>> 2.34.1
>>>>>

-- 
Thanks & Regards,
Wyes