Re: [PATCHv8 0/5] Driver for new "VMD" device

[Keith Busch]

Hi Bryan

I want to make sure I get the right message for Bjorn, so held off on
sending anything before confirming.

As far as clearing the 64-bit resource flag when it's a 32-bit address,
that seems pretty straight forward as captured in the comment.

But do we actually rely on a memory window below 4GB? If the memory
window is above 4GB, we just won't be able to support devices with 32-bit
BARs there, but is that a concern? Should the comment just say that if
the window is assigned >4GB, we simply won't be able to support 32-bit
BAR devices on that domain, and that's 'ok'?

Thanks,
Keith

On Thu, Feb 25, 2016 at 08:42:19AM -0600, Bjorn Helgaas wrote:
> On Tue, Feb 23, 2016 at 06:24:00PM +0000, Keith Busch wrote:
> > On Mon, Feb 22, 2016 at 04:10:24PM -0600, Bjorn Helgaas wrote:
> > > I'm not sure how to deal with the question of a hot-added VMD.  Maybe
> > > all we can do now is add a comment to the effect that we assume BIOS
> > > has assigned the non-prefetchable BAR below 4GB, and if Linux assigns
> > > that BAR for hot-added VMDs, that assumption will likely break.
> > 
> > Yes, we can assume BIOS always assigns. There are other BIOS dependencies
> > in order for the host to see the h/w as a VMD endpoint.
> > 
> > > diff --git a/arch/x86/pci/vmd.c b/arch/x86/pci/vmd.c
> > > index d57e480..7554722 100644
> > > --- a/arch/x86/pci/vmd.c
> > > +++ b/arch/x86/pci/vmd.c
> > > @@ -532,6 +532,16 @@ static int vmd_enable_domain(struct vmd_dev *vmd)
> > >  		.flags = IORESOURCE_BUS | IORESOURCE_PCI_FIXED,
> > >  	};
> > >  
> > > +	/*
> > > +	 * If the window is below 4GB, clear IORESOURCE_MEM_64 so we can
> > > +	 * put 32-bit resources in the window.
> > > +	 *
> > > +	 * There's no hardware reason why a 64-bit window *couldn't*
> > > +	 * contain a 32-bit resource, but pbus_size_mem() computes the
> > > +	 * bridge window size assuming a 64-bit window will contain no
> > > +	 * 32-bit resources.  __pci_assign_resource() enforces that
> > > +	 * artificial restriction to make sure everything will fit.
> > > +	 */
> > 
> > This sounds good to me. Thanks!
> > 
> > >  	res = &vmd->dev->resource[VMD_MEMBAR1];
> > >  	upper_bits = upper_32_bits(res->end);
> > >  	flags = res->flags & ~IORESOURCE_SIZEALIGN;
> 
> Can you prepare a patch, that adds both comments, please?  (The one
> about how we assume BIOS assigns the BAR below 4GB, and the one I
> drafted above.)