Re: [PATCH] kvm: x86: disable KVM_FAST_MMIO_BUS

[Michael S. Tsirkin]

On Wed, Aug 16, 2017 at 03:30:31PM +0200, Paolo Bonzini wrote:
> On 16/08/2017 15:16, Michael S. Tsirkin wrote:
> > On Wed, Aug 16, 2017 at 03:05:51PM +0200, Paolo Bonzini wrote:
> >> On 16/08/2017 14:58, Michael S. Tsirkin wrote:
> >>> On Wed, Aug 16, 2017 at 01:22:49PM +0200, Paolo Bonzini wrote:
> >>>> Microsoft pointed out privately to me that KVM's handling of
> >>>> KVM_FAST_MMIO_BUS is invalid.  Using skip_emulation_instruction is invalid
> >>>> in EPT misconfiguration vmexit handlers, because neither EPT violations
> >>>> nor misconfigurations are listed in the manual among the VM exits that
> >>>> set the VM-exit instruction length field.
> >>>>
> >>>> While physical processors seem to set the field, this is not architectural
> >>>> and is just a side effect of the implementation.  I couldn't convince
> >>>> myself of any condition on the exit qualification where VM-exit
> >>>> instruction length "has" to be defined; there are no trap-like VM-exits
> >>>> that can be repurposed; and fault-like VM-exits such as descriptor-table
> >>>> exits provide no decoding information.  So I don't really see any elegant
> >>>> way to fix it except by disabling KVM_FAST_MMIO_BUS, which means virtio
> >>>> 1 will go slower.
> >>>
> >>> How about I will try asking Intel about it? If they can commit to length
> >>> being there in the future, we are all set.
> >>
> >> Nope, "I couldn't convince myself of any condition on the exit
> >> qualification where VM-exit instruction length "has" to be defined".  So
> >> assuming Intel can do it, it would only apply to future processors (2
> >> years+ for server SKUs).
> > 
> > Well maybe there's a reason it's actually working.  Let's see what can
> > be done.
> 
> Sure there is.  It just happens that the actual condition for VM-exit
> instruction length being set correctly is "the fault was taken after the
> accessing instruction has been decoded".  But there's no way according
> to the spec to detect whether that has happened.
> 
> While you can filter out instruction fetches, that's not enough.  A data
> read could happen because someone pointed the IDT to MMIO area, and who
> knows what the VM-exit instruction length points to in that case.

Thinking more about it, I don't really see how anything
legal guest might be doing with virtio would trigger anything
but a fault after decoding the instruction. How does
skipping instruction even make sense in the example you give?


Please note that
1. we have verified the address and we know it's the one
   that belongs to MMIO of a virtio device
2. only write accesses are allowed by the eventfd or virtio spec
   guests doing anything else are on their own.

Are there any reasonable examples on real hardware? Could you share this pls?


> >> Plus of course it wouldn't be guaranteed to work on nested.
> > 
> > Not sure I got this one.
> 
> Not all nested hypervisors are setting the VM-exit instruction length
> field on EPT violations, since it's documented not to be set.

So that's probably the real issue - nested virt which has to do it
in software at extra cost. We already limit this to intel processors,
how about we blacklist nested virt for this optimization?

I agree it's skating it a bit close to the dangerous edge,
but so are other tricks we play with PTEs to speed up MMIO.

> >>>> Adding a hypercall or MSR write that does a fast MMIO write to a physical
> >>>> address would do it, but it adds hypervisor knowledge in virtio, including
> >>>> CPUID handling.
> >>>
> >>> Another issue is that it will break DPDK on virtio.
> >>
> >> Not break, just make it slower.
> > 
> > I thought hypercalls can only be triggered from ring 0, userspace can't call them.
> > Dod I get it wrong?
> 
> That's just a limitation that KVM makes on currently-defined hypercalls.
> 
> VMCALL causes a vmexit if executed from ring 3.
> 
> Paolo