Re: [PATCH 1/4] KVM: X86: Fix tlb flush for tdp in kvm_invalidate_pcid()

[Lai Jiangshan]

On 2021/10/21 22:52, Sean Christopherson wrote:
> On Thu, Oct 21, 2021, Lai Jiangshan wrote:
> > On 2021/10/21 02:26, Sean Christopherson wrote:
> > > On Wed, Oct 20, 2021, Lai Jiangshan wrote:
> > > > On 2021/10/19 23:25, Sean Christopherson wrote:
> > > > I just read some interception policy in vmx.c, if EPT=1 but vmx_need_pf_intercept()
> > > > return true for some reasons/configs, #PF is intercepted.  But CR3 write is not
> > > > intercepted, which means there will be an EPT fault _after_ (IIUC) the CR3 write if
> > > > the GPA of the new CR3 exceeds the guest maxphyaddr limit.  And kvm queues a fault to
> > > > the guest which is also _after_ the CR3 write, but the guest expects the fault before
> > > > the write.
> > > >
> > > > IIUC, it can be fixed by intercepting CR3 write or reversing the CR3 write in EPT
> > > > violation handler.
> > >
> > > KVM implicitly does the latter by emulating the faulting instruction.
> > >
> > >     static int handle_ept_violation(struct kvm_vcpu *vcpu)
> > >     {
> > > 	...
> > >
> > > 	/*
> > > 	 * Check that the GPA doesn't exceed physical memory limits, as that is
> > > 	 * a guest page fault.  We have to emulate the instruction here, because
> > > 	 * if the illegal address is that of a paging structure, then
> > > 	 * EPT_VIOLATION_ACC_WRITE bit is set.  Alternatively, if supported we
> > > 	 * would also use advanced VM-exit information for EPT violations to
> > > 	 * reconstruct the page fault error code.
> > > 	 */
> > > 	if (unlikely(allow_smaller_maxphyaddr && kvm_vcpu_is_illegal_gpa(vcpu, gpa)))
> > > 		return kvm_emulate_instruction(vcpu, 0);
> > >
> > > 	return kvm_mmu_page_fault(vcpu, gpa, error_code, NULL, 0);
> > >     }
> > >
> > > and injecting a #GP when kvm_set_cr3() fails.
> >
> > I think the EPT violation happens *after* the cr3 write.  So the instruction to be
> > emulated is not "cr3 write".  The emulation will queue fault into guest though,
> > recursive EPT violation happens since the cr3 exceeds maxphyaddr limit.
>
> Doh, you're correct.  I think my mind wandered into thinking about what would
> happen with PDPTRs and forgot to get back to normal MOV CR3.
>
> So yeah, the only way to correctly handle this would be to intercept CR3 loads.
> I'm guessing that would have a noticeable impact on guest performance.

I think we can detect it in handle_ept_violation() via checking the cr3 value,
and make it triple-fault if it is the case, so that the VMM can exit.  I don't
think any OS would use the reserved bit in CR3 and the corresponding #GP.

> Paolo, I'll leave this one for you to decide, we have pretty much written off
> allow_smaller_maxphyaddr :-)