Re: [PATCH 0/4] x86/hyper-v: optimize PV IPIs

[Wanpeng Li]

On Wed, 27 Jun 2018 at 17:25, Vitaly Kuznetsov <vkuznets@xxxxxxxxxx> wrote:
>
> Wanpeng Li <kernellwp@xxxxxxxxx> writes:
>
> > Hi Vitaly, (fix my reply mess this time)
> > On Sat, 23 Jun 2018 at 01:09, Vitaly Kuznetsov <vkuznets@xxxxxxxxxx> wrote:
> >>
> >> When reviewing my "x86/hyper-v: use cheaper HVCALL_FLUSH_VIRTUAL_ADDRESS_
> >> {LIST,SPACE} hypercalls when possible" patch Michael suggested to apply the
> >> same idea to PV IPIs. Here we go!
> >>
> >> Despite what Hyper-V TLFS says about HVCALL_SEND_IPI hypercall, it can
> >> actually be 'fast' (passing parameters through registers). Use that too.
> >>
> >> This series can collide with my "KVM: x86: hyperv: PV IPI support for
> >> Windows guests" series as I rename ipi_arg_non_ex/ipi_arg_ex structures
> >> there. Depending on which one gets in first we may need to do tiny
> >> adjustments.
> >
> > As hyperv PV TLB flush has already been merged, is there any other
> > obvious multicast IPIs scenarios? qemu supports interrupt remapping
> > since two years ago, I think windows guest can switch to cluster mode
> > after entering x2APIC, so sending IPI per cluster.
>
>I got confused, which of my patch series are you actually looking at?
>:-)

Yeah, actually originally I want to reply the thread which you sent
out to kvm ml "KVM: x86: hyperv: PV IPI support for Windows guests"
and miss to reply this one since the subject is similar.

> When we manifest ourselves as Hyper-V Windows 'forgets' about x2apic
> mode: Hyper-V has a concept of 'Synthetic interrupt controller' - an
> xapic extension which we also support in KVM. I don't really know any
> obvious scenarios for mass IPIs in Windows besides TLB flush but I'm
> worried they may exist. Without PV IPIs any such attempt will likely
> lead to a crash.
>
> In general, I do care more about completeness and correctness of our
> Hyper-V emulation at this point: Windows is only being tested on 'real'
> Hyper-Vs so when we emulate a subset of enlightenments we're on our own
> when something is not working. It is also very helpfult for
> Linux-on-Hyper-V depelopment as we can see how Windows-on-Hyper-v
> behaves :-)
>
> > In addition, you
> > can also post the benchmark result for this PV IPI optimization,
> > although it also fixes the bug which you mentioned above.
>
> I'd love to get to know how to trigger mass IPIs in Windows so a
> benchmark can be performed...

I also not sure about windows. I use
https://lkml.org/lkml/2017/12/19/141 as a linux kernel module to
evaluate broadcast IPI performance in the linux guest laster year. :)

>
> > I can post one variant for Linux guest PV IPI if it also makes
> > sense. :)
>
> With x2apic support I'm actually not sure. Maybe configurations with
> a very large number of vCPUs and IPIs going to > 256 vCPUs can benefit
> from a 'single hypercall' solution.

Each cluster of x2apic cluster mode can just support 16 unique logical
IDs, so I think linux guest can also get benefit as long as VM has >
16 vCPUs. I will cook patches to evaluate it. :)

Regards,
Wanpeng Li