Re: [PATCH v11 2/4] PCI: Factor out pci_bus_wait_crs()

[Bjorn Helgaas]

On Mon, Aug 21, 2017 at 04:32:26PM -0400, Sinan Kaya wrote:
> On 8/21/2017 4:23 PM, Bjorn Helgaas wrote:
> > On Mon, Aug 21, 2017 at 03:37:06PM -0400, Sinan Kaya wrote:
> >> On 8/21/2017 3:18 PM, Bjorn Helgaas wrote:
> >> ...
> >> if (pci_bus_crs_pending(id))
> >> 	return pci_bus_wait_crs(dev->bus, dev->devfn, &id, 60000);
> >>
> >>> I think that makes sense.  We'd want to check for CRS SV being
> >>> enabled, e.g., maybe read PCI_EXP_RTCTL_CRSSVE back in
> >>> pci_enable_crs() and cache it somewhere.  Maybe a crs_sv_enabled bit
> >>> in the root port's pci_dev, and check it with something like what
> >>> pcie_root_rcb_set() does?
> >>>
> >>
> >> You can observe CRS under the following conditions
> >>
> >> 1. root port <-> endpoint 
> >> 2. bridge <-> endpoint 
> >> 3. root port<->bridge
> >>
> >> I was relying on the fact that we are reading 0x001 as an indication that
> >> this device detected CRS. Maybe, this is too indirect.
> >>
> >> If we also want to capture the capability, I think the right thing is to
> >> check the parent capability.
> >>
> >> bool pci_bus_crs_vis_supported(struct pci_dev *bridge)
> >> {
> >> 	if (device type(bridge) == root port)
> >> 		return read(root_crs_register_reg);
> >>
> >> 	if (device type(bridge) == switch)
> >> 		return read(switch_crs_register);
> > 
> > I don't understand this part.  AFAIK, CRS SV is only a feature of root
> > ports.  The capability and enable bits are in the Root Capabilities
> > and Root Control registers.
> > 
> 
> No question about it.
> 
> > It's certainly true that a device below a switch can respond with a
> > CRS completion, but the switch is not the requester, and my
> > understanding is that it would not take any action on the completion
> > other than passing it upstream.
> > 
> 
> I saw some bridge references in the spec for CRS. I was going to do
> some research for it. You answered my question. I was curious how this
> would impact the behavior.
> 
> "Bridge Configuration Retry Enable â When Set, this bit enables PCI Express
> to PCI/PCI-X bridges to return Configuration Request Retry Status (CRS) in
> response to Configuration Requests that target devices below the bridge. 
> Refer to the PCI Express to PCI/PCI-X Bridge Specification, Revision 1.0 for
> further details."

(The above is from PCIe r3.1, sec 7.8.4, Device Control Register, and
also discussed in sec 4.3 of the PCIe-to-PCI/PCI-X bridge spec.)

In any event, the Bridge Configuration Retry Enable only determines
whether the bridge ever returns a CRS completion.  The bridge itself
never converts a CRS completion into the 0x0001 vendor ID.