RE: [v4 08/16] KVM: kvm-vfio: User API for IRQ forwarding

[Wu, Feng]

> -----Original Message-----
> From: Joerg Roedel [mailto:joro@xxxxxxxxxx]
> Sent: Wednesday, June 24, 2015 11:46 PM
> To: Alex Williamson
> Cc: Wu, Feng; Eric Auger; Avi Kivity; kvm@xxxxxxxxxxxxxxx;
> linux-kernel@xxxxxxxxxxxxxxx; pbonzini@xxxxxxxxxx; mtosatti@xxxxxxxxxx
> Subject: Re: [v4 08/16] KVM: kvm-vfio: User API for IRQ forwarding
> 
> On Thu, Jun 18, 2015 at 02:04:08PM -0600, Alex Williamson wrote:
> > There are plenty of details to be filled in,
> 
> I also need to fill plenty of details in my head first, so here are some
> suggestions based on my current understanding. Please don't hesitate to
> correct me if where I got something wrong.
> 
> So first I totally agree that the handling of PI/non-PI configurations
> should be transparent to user-space.
> 
> I read a bit through the VT-d spec, and my understanding of posted
> interrupts so far is that:
> 
> 	1) Each VCPU gets a PI-Descriptor with its pending Posted
> 	   Interrupts. This descriptor needs to be updated when a VCPU
> 	   is migrated to another PCPU and should thus be under control
> 	   of KVM.
> 
> 	   This is similar to the vAPIC backing page in the AMD version
> 	   of this, except that the PCPU routing information is stored
> 	   somewhere else on AMD.
> 
> 	2) As long as the VCPU runs the IRTEs are configured for
> 	   posting, when the VCPU goes to sleep the old remapped entry is
> 	   established again. So when the VCPU sleeps the interrupt
> 	   would get routed to VFIO and forwarded through the eventfd.

When the vCPU sleeps, says, blocked when guest is running HLT, the
interrupt is still in posted mode. The solution is when the vCPU is blocked,
we use another notification vector (named wakeup notification vector) to
wakeup the blocked vCPU when interrupts happens. And in the wakeup
event handler, we unblock the vCPU.

Thanks,
Feng

> 
> 	   This would be different to the AMD version, where we have a
> 	   running bit. When this is clear the IOMMU will trigger an event
> 	   in its event-log. This might need special handling in VFIO
> 	   ('might' because VFIO does not need to forward the interrupt,
> 	    it just needs to make sure the VCPU wakes up).
> 
> 	   Please correct me if my understanding of the Intel version is
> 	   wrong.
> 
> So most of the data structures the IOMMU reads for this need to be
> updated from KVM code (either x86-generic or AMD/Intel specific code),
> as KVM has the information about VCPU load/unload and the IRQ routing.

Yes, this part has nothing to do with VFIO, KVM itself can handle it well.

> 
> What KVM needs from VFIO are the informations about the physical
> interrupts, and it makes total sense to attach them as metadata to the
> eventfd.

When guest set the irq affinity, QEMU first gets the MSI/MSIx configuration,
then it passes these information to kernel space via VFIO infrastructure, we
need these MSI/MSIx configuration to update the associated posted-format
IRTE according. This is the key point for PI in term of VFIO.

Thanks,
Feng

> 
> But the problems start at how this metadata should look like. It would
> be good to have some generic description, but not sure if this is
> possible. Otherwise this metadata would need to be requested by VFIO
> from the IOMMU driver and passed on to KVM, which it then passes back to
> the IOMMU driver. Or something like that.
> 
> 
> 
> 	Joerg

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/