Re: [PATCH v2 12/12] x86/kvm/emulate: Avoid RET for fastops
From: Sean Christopherson
Date: Mon Nov 11 2024 - 12:28:00 EST
KVM: x86:
On Mon, Nov 11, 2024, Peter Zijlstra wrote:
> Since there is only a single fastop() function, convert the FASTOP
> stuff from CALL_NOSPEC+RET to JMP_NOSPEC+JMP, avoiding the return
> thunks and all that jazz.
>
> Specifically FASTOPs rely on the return thunk to preserve EFLAGS,
> which not all of them can trivially do (call depth tracing suffers
> here).
Maybe add an example? Mostly as a reminder of how to reproduce the call depth
issues.
E.g. booting with "retbleed=force,stuff spectre_v2=retpoline,generic" causes
KVM-Unit-Test's "emulator" test to fail due to flags being clobbered.
> Objtool strenuously complains about this:
>
> - indirect call without a .rodata, fails to determine JUMP_TABLE,
> annotate
> - fastop functions fall through, exception
> - unreachable instruction after fastop_return, save/restore
>
> Signed-off-by: Peter Zijlstra (Intel) <peterz@xxxxxxxxxxxxx>
The original patch works, but with the fixup KVM fails emulation of an ADC and
generates:
------------[ cut here ]------------
Unpatched return thunk in use. This should not happen!
WARNING: CPU: 4 PID: 1452 at arch/x86/kernel/cpu/bugs.c:3063 __warn_thunk+0x26/0x30
Modules linked in: vhost_net vhost vhost_iotlb tap kvm_intel kvm
CPU: 4 UID: 1000 PID: 1452 Comm: qemu Not tainted 6.12.0-rc5-22582d7d68a6-rev/fastops-miti #11
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015
RIP: 0010:__warn_thunk+0x26/0x30
Code: 5e ff 7e 05 0f 1f 44 00 00 80 3d 5d 06 5c 01 00 74 05 e9 bd 7d a0 00 48 c7 c7 10 4a 13 82 c6 05 48 06 5c 01 01 e8 31 48 04 00 <0f> 0b e9 a3 7d a0 00 cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90
RSP: 0018:ffffc90001743c78 EFLAGS: 00010287
RAX: 0000000000000000 RBX: ffff88811877a040 RCX: 0000000000000027
RDX: ffff88846f91b208 RSI: 0000000000000001 RDI: ffff88846f91b200
RBP: ffffc90001743cc8 R08: ffffffff825195c8 R09: 0000000000000003
R10: ffffffff824395e0 R11: ffffffff824e95e0 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 00007f3027400700(0000) GS:ffff88846f900000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f3029ba3001 CR3: 000000010cdc0000 CR4: 0000000000352eb0
Call Trace:
<TASK>
? __warn+0x85/0x120
? __warn_thunk+0x26/0x30
? report_bug+0x17d/0x190
? handle_bug+0x53/0x90
? exc_invalid_op+0x14/0x70
? asm_exc_invalid_op+0x16/0x20
? __warn_thunk+0x26/0x30
? __warn_thunk+0x26/0x30
warn_thunk_thunk+0x16/0x30
? __kvm_mmu_topup_memory_cache+0x57/0x150 [kvm]
init_emulate_ctxt+0xae/0x110 [kvm]
x86_decode_emulated_instruction+0x25/0x80 [kvm]
x86_emulate_instruction+0x2cb/0x6f0 [kvm]
vmx_handle_exit+0x394/0x6e0 [kvm_intel]
kvm_arch_vcpu_ioctl_run+0xf40/0x1db0 [kvm]
kvm_vcpu_ioctl+0x2e9/0x870 [kvm]
? futex_wake+0x81/0x180
? call_depth_return_thunk+0x6c/0x90
? call_depth_return_thunk+0x66/0x90
? call_depth_return_thunk+0x60/0x90
? call_depth_return_thunk+0x5a/0x90
__x64_sys_ioctl+0x8a/0xc0
do_syscall_64+0x5b/0x170
entry_SYSCALL_64_after_hwframe+0x71/0x79
RIP: 0033:0x7f30290cedeb
Code: 0f 92 c0 84 c0 75 b0 49 8d 3c 1c e8 ff 47 04 00 85 c0 78 b1 48 83 c4 08 4c 89 e0 5b 41 5c c3 f3 0f 1e fa b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f30273ff748 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 000000000000ae80 RCX: 00007f30290cedeb
RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 000000000000000e
RBP: 0000555587e2f1e0 R08: 00007f302923fc10 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007f3029b90780 R14: 00007ffea5ab9640 R15: 00007f30273ffa00
</TASK>
---[ end trace 0000000000000000 ]---