Re: [PATCH V11 4/8] cxl/pci: Create PCI DOE mailbox's for memory devices

[Ira Weiny]

On Fri, Jun 17, 2022 at 04:44:27PM -0700, Dan Williams wrote:
> ira.weiny@ wrote:

[snip]

> > diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h
> > index 60d10ee1e7fc..4d2764b865ab 100644
> > --- a/drivers/cxl/cxlmem.h
> > +++ b/drivers/cxl/cxlmem.h
> > @@ -191,6 +191,8 @@ struct cxl_endpoint_dvsec_info {
> >   * @component_reg_phys: register base of component registers
> >   * @info: Cached DVSEC information about the device.
> >   * @serial: PCIe Device Serial Number
> > + * @doe_mbs: PCI DOE mailbox array
> > + * @num_mbs: Number of DOE mailboxes
> >   * @mbox_send: @dev specific transport for transmitting mailbox commands
> >   *
> >   * See section 8.2.9.5.2 Capacity Configuration and Label Storage for
> > @@ -224,6 +226,10 @@ struct cxl_dev_state {
> >  	resource_size_t component_reg_phys;
> >  	u64 serial;
> >  
> > +	bool doe_use_irq;
> 
> Don't pass temporary state through a long lived data structure. Just
> pass flag by reference between the functions that want to coordinate
> this.

Done.

[snip]

> > +
> > +static void cxl_alloc_irq_vectors(struct cxl_dev_state *cxlds)
> > +{
> > +	struct device *dev = cxlds->dev;
> > +	struct pci_dev *pdev = to_pci_dev(dev);
> > +	int max_irqs = 0;
> > +	int off = 0;
> > +	int rc;
> > +
> > +	/* Account for all the DOE vectors needed */
> > +	pci_doe_for_each_off(pdev, off) {
> > +		int irq = pci_doe_get_irq_num(pdev, off);
> > +
> > +		if (irq < 0)
> > +			continue;
> > +		max_irqs = max(max_irqs, irq + 1);
> 
> This seems to assume that different DOEs will get independent vectors.
> The driver needs to be prepared for DOE instances, Event notifications,
> and mailbox commands to share a single MSI vector in the worst case.
> Lets focus on polled mode DOE, or explicitly only support interrupt
> based operation when no vector sharing is detected.
> 

Ok I see now.  I was under the impression they had to be unique.

Do you think it is sufficient to check in this loop for duplicates and bail if
any are shared?

Ira