Re: [PATCH v3 1/2] x86/fpu: track AVX-512 usage of tasks

From: Li, Aubrey
Date: Fri Nov 16 2018 - 19:36:42 EST


On 2018/11/17 7:10, Dave Hansen wrote:
> On 11/15/18 4:21 PM, Li, Aubrey wrote:
>> "Core cycles where the core was running with power delivery for license
>> level 2 (introduced in Skylake Server microarchitecture). This includes
>> high current AVX 512-bit instructions."
>>
>> I translated license level 2 to frequency drop.
>
> BTW, the "high" in that text: "high-current AVX 512-bit instructions" is
> talking about high-current, not "high ... instructions" or high-numbered
> registers. I think that might be the source of some of the confusion
> about which XSAVE state needs to be examined.
>
> Just to be clear: there are 3 AVX-512 XSAVE states:
>
> XFEATURE_OPMASK,
> XFEATURE_ZMM_Hi256,
> XFEATURE_Hi16_ZMM,
>
> I honestly don't know what XFEATURE_OPMASK does. It does not appear to
> be affected by VZEROUPPER (although VZEROUPPER's SDM documentation isn't
> looking too great).
>
> But, XFEATURE_ZMM_Hi256 is used for the upper 256 bits of the
> registers ZMM0-ZMM15. Those are AVX-512-only registers. The only way
> to get data into XFEATURE_ZMM_Hi256 state is by using AVX512 instructions.
>
> XFEATURE_Hi16_ZMM is the same. The only way to get state in there is
> with AVX512 instructions.
>
> So, first of all, I think you *MUST* check XFEATURE_ZMM_Hi256 and
> XFEATURE_Hi16_ZMM. That's without question.

No, XFEATURE_ZMM_Hi256 does not request turbo license 2, so it's less
interested to us.

>
> It's probably *possible* to run AVX512 instructions by loading state
> into the YMM register and then executing AVX512 instructions that only
> write to memory and never to register state. That *might* allow
> XFEATURE_Hi16_ZMM and XFEATURE_ZMM_Hi256 to stay in the init state, but
> for the frequency to be affected since AVX512 instructions _are_
> executing. But, there's no way to detect this situation from XSAVE
> states themselves.
>

Andi should have more details on this. FWICT, not all AVX512 instructions
has high current, those only touching memory do not cause notable frequency
drop.

Thanks,
-Aubrey