Re: [PATCH v5 06/13] KVM: x86/vmx: Save/Restore host MSR_ARCH_LBR_CTL state
From: Like Xu
Date: Wed Jul 14 2021 - 09:33:58 EST
On 14/7/2021 1:00 am, Jim Mattson wrote:
On Tue, Jul 13, 2021 at 2:49 AM Like Xu <like.xu.linux@xxxxxxxxx> wrote:
On 13/7/2021 1:45 am, Jim Mattson wrote:
On Mon, Jul 12, 2021 at 10:20 AM Jim Mattson <jmattson@xxxxxxxxxx> wrote:
On Mon, Jul 12, 2021 at 3:19 AM Like Xu <like.xu.linux@xxxxxxxxx> wrote:
On 12/7/2021 5:53 pm, Yang Weijiang wrote:
On Fri, Jul 09, 2021 at 04:41:30PM -0700, Jim Mattson wrote:
On Fri, Jul 9, 2021 at 3:54 PM Jim Mattson <jmattson@xxxxxxxxxx> wrote:
On Fri, Jul 9, 2021 at 2:51 AM Yang Weijiang <weijiang.yang@xxxxxxxxx> wrote:
If host is using MSR_ARCH_LBR_CTL then save it before vm-entry
and reload it after vm-exit.
I don't see anything being done here "before VM-entry" or "after
VM-exit." This code seems to be invoked on vcpu_load and vcpu_put.
In any case, I don't see why this one MSR is special. It seems that if
the host is using the architectural LBR MSRs, then *all* of the host
architectural LBR MSRs have to be saved on vcpu_load and restored on
vcpu_put. Shouldn't kvm_load_guest_fpu() and kvm_put_guest_fpu() do
that via the calls to kvm_save_current_fpu(vcpu->arch.user_fpu) and
restore_fpregs_from_fpstate(&vcpu->arch.user_fpu->state)?
It does seem like there is something special about IA32_LBR_DEPTH, though...
Section 7.3.1 of the Intel® Architecture Instruction Set Extensions
and Future Features Programming Reference
says, "IA32_LBR_DEPTH is saved by XSAVES, but it is not written by
XRSTORS in any circumstance." It seems like that would require some
special handling if the host depth and the guest depth do not match.
In our vPMU design, guest depth is alway kept the same as that of host,
so this won't be a problem. But I'll double check the code again, thanks!
KVM only exposes the host's depth value to the user space
so the guest can only use the same depth as the host.
The allowed depth supplied by KVM_GET_SUPPORTED_CPUID isn't enforced,
though, is it?
Like other hardware dependent features, the functionality will not
promise to work properly if the guest uses the unsupported CPUID.
It's fine if it doesn't work in the guest, but can't a guest with the
wrong depth prevent the host LBRs from being reloaded when switching
back to the host state? It's definitely not okay for an ill-configured
guest to break host functionality.
If the ownership of LBR changes, there must be two (or more) LBR events
for perf event scheduling switch for current task, and the perf
subsystem callback will save the previous LBR state and restore
the state of the next LBR event considering different depths.
Also, doesn't this end up being a major constraint on future
platforms? Every host that this vCPU will ever run on will have to use
the same LBR depth as the host on which it was started.
As a first step, we made the guest LBR feature only available for the
"migratable=off" user space, which is why we intentionally did not add
MSR_ARCH_LBR_* stuff to msrs_to_save_all[] in earlier versions.
We have no such concept in our user space. Features that are not
migratable should clearly be identified as such by an appropriate KVM
API. At present, I don't believe there is such an API.
I couldn't agree with you more on this point.
We do have a code gap to make Arch LBR migratable for any KVM user.
But hopefully, we may make it at least migratable for Arch LBR.
I'm personally curious about the cost of using XSAVES to swicth
guest lbr msrs during vmx transaction, and if the cost is unacceptable,
we may ask the perf host to adjust different depths for threads.