Re: [PATCH 0/5] AVIC bugfixes and workarounds

[Joao Martins]

On 26/03/2024 15:59, mlevitsk@xxxxxxxxxx wrote:
> On Mon, 2024-03-25 at 20:15 -0700, Jim Mattson wrote:
>>> On Thu, Sep 28, 2023 at 8:05 AM Maxim Levitsky <mlevitsk@xxxxxxxxxx> wrote:
>>>>>
>>>>> Hi!
>>>>>
>>>>> This patch series includes several fixes to AVIC I found while working
>>>>> on a new version of nested AVIC code.
>>>>>
>>>>> Also while developing it I realized that a very simple workaround for
>>>>> AVIC's errata #1235 exists and included it in this patch series as well.
>>>>>
>>>>> Best regards,
>>>>>         Maxim Levitsky
>>>
>>> Can someone explain why we're still unwilling to enable AVIC by
>>> default? Have the performance issues that plagued the Rome
>>> implementation been fixed? What is AMD's guidance?
>>>
> Hi
> 
> This is what I know:
> 
> Zen1:
> 	I never tested it, so I don't know how well AVIC works there and if it has any erratas.
> 
> Zen2:
> 	Has CPU errata in regard to IPI virtualization that makes it unusable in production,
>  	but if AVIC's IPI virtualization (borrowing the Intel term here) is disabled,
> 	then it works just fine and 1:1 equivalent to APICv without IPI.
> 
> 	I posted patches for this several times, latest version is here, it still applies I think:
> 	https://lkml.iu.edu/hypermail/linux/kernel/2310.0/00790.html
> 
> Zen3:
> 	For some reason AVIC got disabled by AMD in CPUID. It is still there though and force_avic=1 kvm_amd option
> 	can make KVM use it and AFAIK it works just fine.
> 
> 	It is possible that it got disabled due to Zen2 errata that is fixed on Zen3,
> 	but maybe AMD wasn't sure back then that it will be fixed or it might be due to performance issues with broadcast
> 	IPIs which I think ended up being a software issue and was fixed a long time ago.
> 
> Zen4+
> 	I haven't tested it much, but AFAIK it should work out of the box. It also got x2avic mode which allows
> 	to use AVIC with VMs that have more that 254 vCPUs.
> 
> IMHO if we merge the workaround I have for IPI virtualization and make IPI virtualization off for Zen2
> (and maybe Zen1 as well), then I don't see why we can't make AVIC be the default on.

Additionally, I think right now with avic=1 it fails vcpu creation when creating
more a vcpu with an id bigger than what's supported i.e. the MAX_VCPUS we
currently advertised in UAPI isn't quite honored. So that's the only wrinkle at
least I am aware. Sean had send this one series to inhibit AVIC when such config:

https://lore.kernel.org/linux-iommu/20230815213533.548732-1-seanjc@xxxxxxxxxx/#r

But I am not sure if it was respinned.

	Joao