Re: [PATCH 0/4] KVM: nVMX: prepare_vmcs02 optimizations

[Wanpeng Li]

2018-01-02 7:01 GMT+08:00 Paolo Bonzini <pbonzini@xxxxxxxxxx>:
> On 01/01/2018 10:36, Paolo Bonzini wrote:
>> On 28/12/2017 09:39, Wanpeng Li wrote:
>>> 2017-12-27 22:28 GMT+08:00 Paolo Bonzini <pbonzini@xxxxxxxxxx>:
>>>> On 25/12/2017 11:08, Wanpeng Li wrote:
>>>>>> I observe L1(latest kvm/queue) panic and L0(latest kvm/queue)
>>>>>> calltrace, I'm not sure whether it is caused by this patchset.
>>>>> It can be reproduced steadily by running kvm-unit-tests in L1.
>>>>
>>>> It works here, can you show the L0 call trace and/or bisect it?
>>>
>>> L0 call trace has already been posted here.
>>> https://lkml.org/lkml/2017/12/25/53 In addition, the splatting is
>>> still there after I revert the last 9 nVMX optimization patches in
>>> kvm/queue. So it is not caused by this patchset. :)
>>
>> Hmm, maybe you're using "-cpu host,+umip"?  I'll check when I get back
>> to work tomorrow.
>
> Yeah, I think this could be it:
>
> diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
> index 30e6115d4f09..6404e96179b4 100644
> --- a/arch/x86/kvm/vmx.c
> +++ b/arch/x86/kvm/vmx.c
> @@ -10780,6 +10780,7 @@ static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12,
>                 exec_control &= ~(SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES |
>                                   SECONDARY_EXEC_ENABLE_INVPCID |
>                                   SECONDARY_EXEC_RDTSCP |
> +                                 SECONDARY_EXEC_DESC |
>                                   SECONDARY_EXEC_XSAVES |
>                                   SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY |
>                                   SECONDARY_EXEC_APIC_REGISTER_VIRT |

The issue is still there after applying this to both L0 and L1,
actually, I can observe a vmentry fail just before the splatting in L0
w/ and w/o the above code. In addition, I comment out the other
testcases in unittests.cfg except vmx_controls, then run
./run_tests.sh.

 [334079.689931] nested_vmx_exit_reflected failed vm entry 7
 [334079.689980] WARNING: CPU: 6 PID: 6911 at
/home/kernel/data/kvm/arch/x86/kvm//vmx.c:6376 handle_desc+0x2d/0x40
[kvm_intel]
 [334079.689982] Modules linked in: kvm_intel(OE) kvm(OE) binfmt_misc
nls_iso8859_1 snd_hda_codec_hdmi snd_hda_codec_realtek
snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core
snd_hwdep x86_pkg_temp_thermal snd_pcm intel_powerclamp coretemp
crc32_pclmul snd_seq_midi pcbc snd_seq_midi_event snd_rawmidi
aesni_intel snd_seq aes_x86_64 crypto_simd cryptd glue_helper joydev
input_leds wmi_bmof snd_seq_device snd_timer mei_me snd mei shpchp
lpc_ich soundcore mac_hid irqbypass parport_pc ppdev lp parport
autofs4 hid_generic usbhid hid i915 i2c_algo_bit drm_kms_helper
syscopyarea sysfillrect sysimgblt fb_sys_fops drm e1000e ahci ptp
libahci pps_core wmi video [last unloaded: kvm]
 [334079.690080] CPU: 6 PID: 6911 Comm: qemu-system-x86 Tainted: G
      OE    4.15.0-rc3+ #1
 [334079.690082] Hardware name: LENOVO ThinkCentre
M8500t-N000/SHARKBAY, BIOS FBKTC1AUS 02/16/2016
 [334079.690086] RIP: 0010:handle_desc+0x2d/0x40 [kvm_intel]
 [334079.690088] RSP: 0018:ffffaf010029bca0 EFLAGS: 00010246
 [334079.690091] RAX: ffffffffc0785160 RBX: 000000000000002e RCX:
0000000000000001
 [334079.690093] RDX: 0000000000000000 RSI: 00000000ffffffff RDI:
ffffa0512c740000
 [334079.690094] RBP: ffffaf010029bca0 R08: 00000000c61dd84f R09:
1e890e0400000000
 [334079.690096] R10: 0000000000000000 R11: 0000000000000001 R12:
ffffa051aa6c0000
 [334079.690098] R13: 0000000000000000 R14: 0000000000000001 R15:
ffffa0512c740000
 [334079.690100] FS:  00007f1cf5450700(0000) GS:ffffa051ce000000(0000)
knlGS:0000000000000000
 [334079.690102] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 [334079.690104] CR2: 0000000000000000 CR3: 00000003eb98d006 CR4:
00000000001626e0
 [334079.690106] Call Trace:
 [334079.690111]  vmx_handle_exit+0xbd/0xe20 [kvm_intel]
 [334079.690131]  ? kvm_arch_vcpu_ioctl_run+0xcea/0x1c20 [kvm]
 [334079.690144]  kvm_arch_vcpu_ioctl_run+0xd66/0x1c20 [kvm]
 [334079.690159]  kvm_vcpu_ioctl+0x3e9/0x720 [kvm]
 [334079.690167]  ? kvm_vcpu_ioctl+0x3e9/0x720 [kvm]
 [334079.690173]  ? __fget+0xfc/0x210
 [334079.690176]  ? __fget+0xfc/0x210
 [334079.690181]  do_vfs_ioctl+0xa4/0x6a0
 [334079.690184]  ? __fget+0x11d/0x210
 [334079.690190]  SyS_ioctl+0x79/0x90
 [334079.690195]  entry_SYSCALL_64_fastpath+0x1f/0x96
 [334079.690197] RIP: 0033:0x7f1d01faef07
 [334079.690199] RSP: 002b:00007f1cf544f8b8 EFLAGS: 00000246 ORIG_RAX:
0000000000000010
 [334079.690203] RAX: ffffffffffffffda RBX: 000000000000ae80 RCX:
00007f1d01faef07
 [334079.690204] RDX: 0000000000000000 RSI: 000000000000ae80 RDI:
0000000000000011
 [334079.690206] RBP: 0000561713c40f70 R08: 0000000000000000 R09:
0000000000000001
 [334079.690208] R10: 0000000000000058 R11: 0000000000000246 R12:
0000000000000000
 [334079.690209] R13: 00007f1d0468d000 R14: 0000000000000000 R15:
0000561713c40f70
 [334079.690218] Code: 44 00 00 f6 87 f1 03 00 00 08 55 48 89 e5 74 1b
45 31 c0 31 c9 31 f6 ba 10 00 00 00 e8 2d 4e dc ff 85 c0 0f 94 c0 0f
b6 c0 5d c3 <0f> ff eb e1 0f 1f 44 00 00 66 2e 0f 1f 84 00 00 00 00 00
0f 1f
 [334079.690314] ---[ end trace 16ed3250b9b651d2 ]---

Regards,
Wanpeng Li