Re: [PATCH v3 2/9] cxl/acpi: Extract component registers of restricted hosts from RCRB

[Dan Williams]

Robert Richter wrote:
> On 14.11.22 13:30:01, Dan Williams wrote:
> > Robert Richter wrote:
> 
> > > diff --git a/drivers/cxl/core/regs.c b/drivers/cxl/core/regs.c
> > > index ec178e69b18f..7a5bde81e949 100644
> > > --- a/drivers/cxl/core/regs.c
> > > +++ b/drivers/cxl/core/regs.c
> > > @@ -307,3 +307,49 @@ int cxl_find_regblock(struct pci_dev *pdev, enum cxl_regloc_type type,
> > >  	return -ENODEV;
> > >  }
> > >  EXPORT_SYMBOL_NS_GPL(cxl_find_regblock, CXL);
> > > +
> > > +resource_size_t cxl_rcrb_to_component(struct device *dev,
> > > +				      resource_size_t rcrb,
> > > +				      enum cxl_rcrb which)
> > > +{
> > > +	resource_size_t component_reg_phys;
> > > +	u32 bar0, bar1;
> > > +	void *addr;
> > > +
> > > +	if (which == CXL_RCRB_UPSTREAM)
> > > +		rcrb += SZ_4K;
> > > +
> > > +	/*
> > > +	 * RCRB's BAR[0..1] point to component block containing CXL
> > > +	 * subsystem component registers. MEMBAR extraction follows
> > > +	 * the PCI Base spec here, esp. 64 bit extraction and memory
> > > +	 * ranges alignment (6.0, 7.5.1.2.1).
> > > +	 */
> > 
> > A request_mem_region() is needed here to ensure ownership and expected
> > sequencing of accessing the RCRB to locate the component registers, and
> > accessing the RCRB to manipulate the component registers. It also helps
> > to sanity check that the BIOS mapped an exclusive range for the RCRB.
> 
> Right, that is missing.
> 
> > 
> > > +	addr = ioremap(rcrb, PCI_BASE_ADDRESS_0 + SZ_8);
> > 
> > That PCI_BASE_ADDRESS_0 does not belong there. It ends up being benign
> > and forcing ioremap to map 12K instead of 8K, but it is a
> > config-register offset, not part of the RCRB size.
> 
> Note this is BAR0 + 8 bytes, not 8k, and it does not map the whole
> RCRB region but instead the first part of the config space up to
> including the 64 bit BAR.

Oh, sorry, yes, my mistake. However, there is not much value in mapping
less than 4K since all ioremap requests are rounded up to PAGE_SIZE.
Since an RCRB is only 4K per port lets just map the whole thing.

> > > +	if (!addr) {
> > > +		dev_err(dev, "Failed to map region %pr\n", addr);
> > > +		return CXL_RESOURCE_NONE;
> > > +	}
> > > +
> > > +	bar0 = readl(addr + PCI_BASE_ADDRESS_0);
> > > +	bar1 = readl(addr + PCI_BASE_ADDRESS_1);
> > > +	iounmap(addr);
> > 
> > ...corresponding release_mem_region() would go here.
> > 
> > > +
> > > +	/* sanity check */
> > > +	if (bar0 & (PCI_BASE_ADDRESS_MEM_TYPE_1M | PCI_BASE_ADDRESS_SPACE_IO))
> > > +		return CXL_RESOURCE_NONE;
> > 
> > I would have also expected:
> > 
> > - a sanity check for "Memory Space Enable" being set in the command
> >   register.
> 
> Ok.
> 
> > 
> > - an explicit check for 0xffffffff for the case when the upstream-port
> >   implements "no RCRB" mode.
> 
> Yes, I left support for this to a later patch, but it's better to
> check it here already and possibly fall back to reg loc DVSEC then.

Yeah, I think simply failing on 0xffffffff is sufficient for now.

> > 
> > - some check that BIOS initialized the BAR values post reset given these
> >   BARs are invisible to the PCI core resource assignment 
> 
> What check do you have in mind here? There is already the NULL check
> which would be the out-of-reset value.

I was thinking more along the lines of sanity checking that the
programmed RCRB range falls within the assigned MMIO space of the
host-bridge, but perhaps that is overkill since it would just be
validating self consistency between 2 BIOS provided values. Robustness
principle would say try to continue if those disagree.