Re: [PATCH v6 00/28] KVM: combined patchset for MBEC/GMET support
From: David Riley
Date: Mon May 11 2026 - 06:54:59 EST
Thanks again for the updated version of this patch series.
I have been testing v6 on Intel and AMD platforms again and observed
a regression on Intel when CET and MBEC are both exposed to a Windows
guest.
Environment:
- Kernel: mainline 7.1.0-rc2 (with v6 patches applied)
- QEMU: downstream 11.0.0-1
- Guest: Windows Server 2026 (24H2, Build 26100.1742)
- virtio-win: 0.1.271
Hosts:
Intel: Intel(R) Xeon(R) Gold 6426Y
AMD: Epyc 7302P
Both hosts are running Proxmox VE (based on Debian Trixie).
Windows Guest Setup:
After the initial installation and verification [0] I enabled
Virtualization-Based Security (VBS) and Hypervisor-Protected Code
Integrity (HVCI).
I set the following in the Group Policy Editor (DeviceGuard):
* Select Platform Security Level: Secure Boot
* Virtualization Based Protection of Code Integrity: Enabled without
lock
* Require UEFI Memory Attributes Table: Checked
Issue: Host Lockups and Guest Hangs
On the Intel platform, the guest fails to boot when using:
QEMU options: -cpu host,level=30,+vmx-mbec
I observed two behaviors:
* The guest hangs indefinitely during the early boot phase. (Most
frequent)
* The guest fails to boot and ends up in Windows Recovery Mode.
When the guest hangs during early boot, the host experiences hard/soft
lockups:
watchdog: CPU11: Watchdog detected hard LOCKUP on cpu 11
watchdog: BUG: soft lockup - CPU#11 stuck for 28s [CPU 0/KVM:16105]
I also recorded a trace of the virtual guest getting stuck using:
`trace-cmd record -e kvm`
and found the following:
Frequency of top RIP:
* 987816 rip 0xfffff801b031bf36
* 985670 rip 0xfffff801b031bf35
* 184002 rip 0x7ffd3e35
Sequence of Events:
--
CPU 0/KVM-16105 [001] ..... 4327.371276: kvm_cr: cr_write 4 = 0xb50ef8
...
CPU 0/KVM-16105 [001] ..... 4327.373469: kvm_pio: pio_read at 0x608 size 4 count 1 val 0x5bcbb1
CPU 0/KVM-16105 [001] d..1. 4327.373469: kvm_entry: vcpu 0 rip 0xfffff801b031bf36
CPU 0/KVM-16105 [001] d..1. 4327.373470: kvm_exit: reason IO_INSTRUCTION rip 0xfffff801b031bf35 info 608000b 0
--
the last three lines seem to be repeating in a infinite loop.
On the AMD platform, the guest had no issue booting when using:
QEMU options: -cpu host
This seem to be the case because CET is not present on AMD.
I confirmed this using:
`cpuid -1 -l 7 -s 0`
which shows that:
- CET_SS: CET shadow stack = false
- CET_IBT: CET indirect branch tracking = false
On Intel `cpuid -1 -l 7 -s 0` shows:
- CET_SS: CET shadow stack = true
- CET_IBT: CET indirect branch tracking = true
If I explicitly disable them on Intel using:
QEMU options: -cpu host,level=30,+vmx-mbec,-cet-ss,-cet-ibt
the guest boots without issues.
This regression previously did not occur because I was using QEMU
version 10.2.0-1, where these options did not yet get exposed for this
particular Intel CPU [1].
Please let me know if you need further information or if there is
something else I could try/test.
[0] https://learn.microsoft.com/en-us/windows/security/hardware-security/enable-virtualization-based-protection-of-code-integrity?tabs=security
[1] https://gitlab.com/qemu-project/qemu/-/commit/5cb89cad7f30be3175dd5abbb79ae5e634476cfa
On 5/5/26 9:50 PM, Paolo Bonzini wrote:
This version can also be found in the "queue" branch of kvm.git.
Since it should be final I'm including again for reference the full
description.
Both MBEC and GMET allow more granular control over execute permissions,
with different levels of separation between supervisor and user mode.
MBEC provides support for separate supervisor and user-mode bits in the
PTEs; GMET instead lacks supervisor-mode only execution (with NX=0,
"both" is represented by U=0 and user-mode only by U=1). GMET was
clearly inspired by SMEP though with some differences and annoyances.
The implementation starts from two changes to core MMU code, both
of which help making the actual feature almost trivial to implement:
- first, I'm cleaning up the implementation of nVMX exec-only, by
properly adding read permissions to the ACC_* constant and to the
permission bitmask machinery. Jon also had to add a fourth ACC_*
bit, but used it only in the special case of nested MBEC; here
instead ACC_READ_MASK is the normality, which simplifies testing
a lot and removes gratuitous complexity.
- second, I'm enforcing that KVM runs with MBEC/GMET enabled even in
non-nested mode, if it wants to provide the feature to nested
hypervisors. This makes the creation of SPTEs looks exactly the
same for L1 and L2 guests, despite only the latter using MBEC/GMET
fully; the difference lies only in the input access permissions.
This strategy adds a limited amount of complexity to the core is limited,
while providing for an almost entirely seamless support of nested
hypervisors.
Later patches have to use slightly different meanings for ACC_* in Intel
and AMD. On the Intel side, some work is needed in order to split
shadow_x_mask and ACC_EXEC_MASK in two; now that there is an actual
ACC_READ_MASK to be used for exec-only pages, ACC_USER_MASK is unused
and can be reused as ACC_USER_EXEC_MASK. However, unlike the older
ACC_USER_MASK hack these differences are backed by concrete concepts
of the page table format, and there is always a 1:1 mapping from ACC_*
bits to PT_*_MASK or shadow_*_mask:
Intel AMD
-------------------- ------------------- -------------------
ACC_READ_MASK PT_PRESENT_MASK PT_PRESENT_MASK
ACC_WRITE_MASK PT_WRITABLE_MASK PT_WRITABLE_MASK
ACC_EXEC_MASK shadow_xs_mask shadow_nx_mask
ACC_USER_MASK --- shadow_user_mask
ACC_USER_EXEC_MASK shadow_xu_mask ---
On Intel, ACC_EXEC_MASK is used for kernel-mode execution and is tied to
shadow_xs_mask (when MBEC is disabled, ACC_USER_EXEC_MASK and the XU bit
are computed but ineffective). update_permission_bitmask() precomputes
all the necessary conditions. On the AMD side, the U bit maps to
ACC_USER_MASK but nNPT adjusts the permission bitmask to ignore it for
reads and writes when GMET is active. Despite the smaller scale of the
changes compared to MBEC, there are some changes to make to use GMET
for L1 guests, because the page tables have to be created with U=0.
This means that the root page has role.access != ACC_ALL and its
permissions have to be propagated down.
Note that with MBEC the user/supervisor distinction depends on the U
bit of the page tables rather than the CPL. Processors provide this
information to the hypervisor through the "advanced EPT violation
vmexit info" feature, which is a requirement for KVM to use MBEC,
and kvm-intel.ko passes it to the MMU in PFERR_USER_MASK (unlike
kvm-amd.ko which computes it from the CPL). This needs a small change
to pass the effective XWU permissions of the page tables down to
translate_nested_gpa().
The former "smep_andnot_wp" bit of cpu_role.base, now named "cr4_smep",
is repurposed for nested TDP to indicate that MBEC/GMET is on. The minor
pessimization for shadow page tables (toggling CR4.SMEP now always forces
building a separate version of the shadow page tables, even though that's
technically unnecessary if CR4.WP=1) is not really worth fretting about;
in practice, guests are not going to flip CR4.SMEP in a way that would
prevent efficient reuse of shadow page tables.
Paolo
v5->v6:
- rename make_spte_executable to change_spte_executable
- rename byte index in update_permission_bitmask to index
- use (u8) casts before "KVM: x86/mmu: introduce ACC_READ_MASK"
- make commit message for "KVM: x86/mmu: split XS/XU bits for EPT" more accurate
- add XU to shadow_acc_track_mask already in "KVM: x86/mmu: split XS/XU bits for EPT"
- fix compilation error
- use alternative code for __vmx_handle_ept_violation suggested by Sean
Jon Kohler (5):
KVM: TDX/VMX: rework EPT_VIOLATION_EXEC_FOR_RING3_LIN into PROT_MASK
KVM: x86/mmu: remove SPTE_PERM_MASK
KVM: x86/mmu: free up bit 10 of PTEs in preparation for MBEC
KVM: nVMX: advertise MBEC to nested guests
KVM: nVMX: allow MBEC with EVMCS
Paolo Bonzini (23):
KVM: x86/mmu: shuffle high bits of SPTEs in preparation for MBEC
KVM: x86/mmu: remove SPTE_EPT_*
KVM: x86/mmu: merge make_spte_{non,}executable
KVM: x86/mmu: rename and clarify BYTE_MASK
KVM: x86/mmu: separate more EPT/non-EPT permission_fault()
KVM: x86/mmu: introduce ACC_READ_MASK
KVM: x86/mmu: pass PFERR_GUEST_PAGE/FINAL_MASK to kvm_translate_gpa
KVM: x86/mmu: pass pte_access for final nGPA->GPA walk
KVM: x86: make translate_nested_gpa vendor-specific
KVM: x86/mmu: split XS/XU bits for EPT
KVM: x86/mmu: move cr4_smep to base role
KVM: VMX: enable use of MBEC
KVM: nVMX: pass advanced EPT violation vmexit info to guest
KVM: nVMX: pass PFERR_USER_MASK to MMU on EPT violations
KVM: x86/mmu: add support for MBEC to EPT page table walks
KVM: x86/mmu: propagate access mask from root pages down
KVM: x86/mmu: introduce cpu_role bit for availability of PFEC.I/D
KVM: SVM: add GMET bit definitions
KVM: x86/mmu: hard code more bits in kvm_init_shadow_npt_mmu
KVM: x86/mmu: add support for GMET to NPT page table walks
KVM: SVM: enable GMET and set it in MMU role
KVM: SVM: work around errata 1218
KVM: nSVM: enable GMET for guests
Documentation/virt/kvm/x86/mmu.rst | 10 +-
arch/x86/include/asm/cpufeatures.h | 1 +
arch/x86/include/asm/kvm-x86-ops.h | 1 +
arch/x86/include/asm/kvm_host.h | 48 +++++---
arch/x86/include/asm/svm.h | 1 +
arch/x86/include/asm/vmx.h | 14 ++-
arch/x86/kvm/hyperv.c | 4 +-
arch/x86/kvm/mmu.h | 30 +++--
arch/x86/kvm/mmu/mmu.c | 182 ++++++++++++++++++++---------
arch/x86/kvm/mmu/mmutrace.h | 19 +--
arch/x86/kvm/mmu/paging_tmpl.h | 73 ++++++++----
arch/x86/kvm/mmu/spte.c | 92 +++++++++------
arch/x86/kvm/mmu/spte.h | 70 ++++++-----
arch/x86/kvm/mmu/tdp_mmu.c | 6 +-
arch/x86/kvm/svm/nested.c | 38 +++++-
arch/x86/kvm/svm/svm.c | 31 +++++
arch/x86/kvm/svm/svm.h | 1 +
arch/x86/kvm/vmx/capabilities.h | 12 +-
arch/x86/kvm/vmx/common.h | 26 +++--
arch/x86/kvm/vmx/hyperv_evmcs.h | 1 +
arch/x86/kvm/vmx/main.c | 9 ++
arch/x86/kvm/vmx/nested.c | 46 +++++++-
arch/x86/kvm/vmx/tdx.c | 2 +-
arch/x86/kvm/vmx/vmx.c | 27 ++++-
arch/x86/kvm/vmx/vmx.h | 1 +
arch/x86/kvm/vmx/x86_ops.h | 1 +
arch/x86/kvm/x86.c | 18 +--
27 files changed, 536 insertions(+), 228 deletions(-)