Re: [PATCH] PCI: Enhance the ACS quirk for Cavium devices

[Bjorn Helgaas]

On Mon, Sep 30, 2019 at 11:20:50PM +0000, Jayachandran Chandrasekharan Nair wrote:
> On Mon, Sep 30, 2019 at 03:34:10PM -0500, Bjorn Helgaas wrote:
> > On Thu, Sep 19, 2019 at 02:43:34AM +0000, George Cherian wrote:
> > > Enhance the ACS quirk for Cavium Processors. Add the root port
> > > vendor ID's in an array and use the same in match function.
> > > For newer devices add the vendor ID's in the array so that the
> > > match function is simpler.
> > > 
> > > Signed-off-by: George Cherian <george.cherian@xxxxxxxxxxx>
> > > ---
> > >  drivers/pci/quirks.c | 28 +++++++++++++++++++---------
> > >  1 file changed, 19 insertions(+), 9 deletions(-)
> > > 
> > > diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
> > > index 44c4ae1abd00..64deeaddd51c 100644
> > > --- a/drivers/pci/quirks.c
> > > +++ b/drivers/pci/quirks.c
> > > @@ -4241,17 +4241,27 @@ static int pci_quirk_amd_sb_acs(struct pci_dev *dev, u16 acs_flags)
> > >  #endif
> > >  }
> > >  
> > > +static const u16 pci_quirk_cavium_acs_ids[] = {
> > > +	/* CN88xx family of devices */
> > > +	0xa180, 0xa170,
> > > +	/* CN99xx family of devices */
> > > +	0xaf84,
> > > +	/* CN11xxx family of devices */
> > > +	0xb884,
> > > +};
> > > +
> > >  static bool pci_quirk_cavium_acs_match(struct pci_dev *dev)
> > >  {
> > > -	/*
> > > -	 * Effectively selects all downstream ports for whole ThunderX 1
> > > -	 * family by 0xf800 mask (which represents 8 SoCs), while the lower
> > > -	 * bits of device ID are used to indicate which subdevice is used
> > > -	 * within the SoC.
> > > -	 */
> > > -	return (pci_is_pcie(dev) &&
> > > -		(pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) &&
> > > -		((dev->device & 0xf800) == 0xa000));
> > > +	int i;
> > > +
> > > +	if (!pci_is_pcie(dev) || pci_pcie_type(dev) != PCI_EXP_TYPE_ROOT_PORT)
> > > +		return false;
> > > +
> > > +	for (i = 0; i < ARRAY_SIZE(pci_quirk_cavium_acs_ids); i++)
> > > +		if (pci_quirk_cavium_acs_ids[i] == dev->device)
> > 
> > I'm a little skeptical of this because the previous test:
> > 
> >   (dev->device & 0xf800) == 0xa000
> > 
> > could match *many* devices, but of those, the new code only matches two
> > (0xa180, 0xa170).
> > 
> > And the comment says the new code matches the CN99xx and CN11xxx
> > *families*, but it only matches a single device ID for each, which
> > makes me think there may be more devices to come.
> > 
> > Maybe this is all what you want, but please confirm.
> 
> There are only a very few device IDs for root ports, so just listing
> them out like this maybe better. The earlier match covered a lot of
> ThunderX1 devices, but did not really match the ThunderX2 root ports.
> 
> This looks ok for ThunderX2. Sunil & Robert can comment on other
> processor families I hope.

I don't know which of these are ThunderX2 vs ThunderX1.

I currently have the change below on a branch waiting for confirmation
that this is what you intend.

> > The commit log should be explicit that this adds CN99xx and CN11xxx,
> > which previously were not matched.
> > 
> > This looks like stable material?
> > 
> > > +			return true;
> > > +
> > > +	return false;
> > >  }
> > >  
> > >  static int pci_quirk_cavium_acs(struct pci_dev *dev, u16 acs_flags)

commit 37b22fbfec2d
Author: George Cherian <george.cherian@xxxxxxxxxxx>
Date:   Thu Sep 19 02:43:34 2019 +0000

    PCI: Apply Cavium ACS quirk to CN99xx and CN11xxx Root Ports
    
    Add an array of Cavium Root Port device IDs and apply the quirk only to the
    listed devices.
    
    Instead of applying the quirk to all Root Ports where
    "(dev->device & 0xf800) == 0xa000", apply it only to CN88xx 0xa180 and
    0xa170 Root Ports.
    
    Also apply the quirk to CN99xx (0xaf84) and CN11xxx (0xb884) Root Ports.
    
    Link: https://lore.kernel.org/r/20190919024319.GA8792@xxxxxxxxxxxxxxxxxxxxxxxx
    Fixes: f2ddaf8dfd4a ("PCI: Apply Cavium ThunderX ACS quirk to more Root Ports")
    Fixes: b404bcfbf035 ("PCI: Add ACS quirk for all Cavium devices")
    Signed-off-by: George Cherian <george.cherian@xxxxxxxxxxx>
    Signed-off-by: Bjorn Helgaas <bhelgaas@xxxxxxxxxx>
    Cc: stable@xxxxxxxxxxxxxxx      # v4.12+

diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 320255e5e8f8..4e5048cb5ec6 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -4311,17 +4311,24 @@ static int pci_quirk_amd_sb_acs(struct pci_dev *dev, u16 acs_flags)
 #endif
 }
 
+static const u16 pci_quirk_cavium_acs_ids[] = {
+	0xa180, 0xa170,		/* CN88xx family of devices */
+	0xaf84,			/* CN99xx family of devices */
+	0xb884,			/* CN11xxx family of devices */
+};
+
 static bool pci_quirk_cavium_acs_match(struct pci_dev *dev)
 {
-	/*
-	 * Effectively selects all downstream ports for whole ThunderX 1
-	 * family by 0xf800 mask (which represents 8 SoCs), while the lower
-	 * bits of device ID are used to indicate which subdevice is used
-	 * within the SoC.
-	 */
-	return (pci_is_pcie(dev) &&
-		(pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) &&
-		((dev->device & 0xf800) == 0xa000));
+	int i;
+
+	if (!pci_is_pcie(dev) || pci_pcie_type(dev) != PCI_EXP_TYPE_ROOT_PORT)
+		return false;
+
+	for (i = 0; i < ARRAY_SIZE(pci_quirk_cavium_acs_ids); i++)
+		if (pci_quirk_cavium_acs_ids[i] == dev->device)
+			return true;
+
+	return false;
 }
 
 static int pci_quirk_cavium_acs(struct pci_dev *dev, u16 acs_flags)