Re: [PATCH v6 2/2] PCI: xilinx-cpm: Add support for PCIe RP PERST# signal

[Manivannan Sadhasivam]

On Sun, Apr 13, 2025 at 04:28:55AM +0000, Musham, Sai Krishna wrote:
> [AMD Official Use Only - AMD Internal Distribution Only]
> 
> Hi Manivannan,
> 
> Thanks for the review.
> 
> > -----Original Message-----
> > From: Manivannan Sadhasivam <manivannan.sadhasivam@xxxxxxxxxx>
> > Sent: Wednesday, April 9, 2025 12:25 PM
> > To: Musham, Sai Krishna <sai.krishna.musham@xxxxxxx>
> > Cc: bhelgaas@xxxxxxxxxx; lpieralisi@xxxxxxxxxx; kw@xxxxxxxxx; robh@xxxxxxxxxx;
> > krzk+dt@xxxxxxxxxx; conor+dt@xxxxxxxxxx; cassel@xxxxxxxxxx; linux-
> > pci@xxxxxxxxxxxxxxx; devicetree@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx;
> > Simek, Michal <michal.simek@xxxxxxx>; Gogada, Bharat Kumar
> > <bharat.kumar.gogada@xxxxxxx>; Havalige, Thippeswamy
> > <thippeswamy.havalige@xxxxxxx>
> > Subject: Re: [PATCH v6 2/2] PCI: xilinx-cpm: Add support for PCIe RP PERST#
> > signal
> >
> > Caution: This message originated from an External Source. Use proper caution
> > when opening attachments, clicking links, or responding.
> >
> >
> > On Fri, Apr 04, 2025 at 06:59:23AM +0000, Musham, Sai Krishna wrote:
> > > [AMD Official Use Only - AMD Internal Distribution Only]
> > >
> > > Hi Manivannan,
> > >
> > > Thanks for the review.
> > >
> > > > -----Original Message-----
> > > > From: Manivannan Sadhasivam <manivannan.sadhasivam@xxxxxxxxxx>
> > > > Sent: Thursday, March 27, 2025 10:56 PM
> > > > To: Musham, Sai Krishna <sai.krishna.musham@xxxxxxx>
> > > > Cc: bhelgaas@xxxxxxxxxx; lpieralisi@xxxxxxxxxx; kw@xxxxxxxxx;
> > > > robh@xxxxxxxxxx;
> > > > krzk+dt@xxxxxxxxxx; conor+dt@xxxxxxxxxx; cassel@xxxxxxxxxx; linux-
> > > > pci@xxxxxxxxxxxxxxx; devicetree@xxxxxxxxxxxxxxx;
> > > > linux-kernel@xxxxxxxxxxxxxxx; Simek, Michal <michal.simek@xxxxxxx>;
> > > > Gogada, Bharat Kumar <bharat.kumar.gogada@xxxxxxx>; Havalige,
> > > > Thippeswamy <thippeswamy.havalige@xxxxxxx>
> > > > Subject: Re: [PATCH v6 2/2] PCI: xilinx-cpm: Add support for PCIe RP
> > > > PERST# signal
> > > >
> > > > Caution: This message originated from an External Source. Use proper
> > > > caution when opening attachments, clicking links, or responding.
> > > >
> > > >
> > > > On Wed, Mar 26, 2025 at 07:58:11AM +0530, Sai Krishna Musham wrote:
> > > > > Add PCIe IP reset along with GPIO-based control for the PCIe Root
> > > > > Port PERST# signal. Synchronizing the PCIe IP reset with the
> > > > > PERST# signal's assertion and deassertion avoids Link Training failures.
> > > > >
> > > > > Adapt to use GPIO framework and make reset optional to maintain
> > > > > backward compatibility with existing DTBs.
> > > > >
> > > > > Add clear firewall after Link reset for CPM5NC.
> > > > >
> > > > > Signed-off-by: Sai Krishna Musham <sai.krishna.musham@xxxxxxx>
> > > > > ---
> > > > > Changes for v6:
> > > > > - Correct version check condition of CPM5NC_HOST.
> > > > >
> > > > > Changes for v5:
> > > > > - Handle probe defer for reset_gpio.
> > > > > - Resolve ABI break.
> > > > >
> > > > > Changes for v4:
> > > > > - Add PCIe PERST# support for CPM5NC.
> > > > > - Add PCIe IP reset along with PERST# to avoid Link Training Errors.
> > > > > - Remove PCIE_T_PVPERL_MS define and PCIE_T_RRS_READY_MS after
> > > > >   PERST# deassert.
> > > > > - Move PCIe PERST# assert and deassert logic to
> > > > >   xilinx_cpm_pcie_init_port() before cpm_pcie_link_up(), since
> > > > >   Interrupts enable and PCIe RP bridge enable should be done after
> > > > >   Link up.
> > > > > - Update commit message.
> > > > >
> > > > > Changes for v3:
> > > > > - Use PCIE_T_PVPERL_MS define.
> > > > >
> > > > > Changes for v2:
> > > > > - Make the request GPIO optional.
> > > > > - Correct the reset sequence as per PERST#
> > > > > - Update commit message
> > > > > ---
> > > > >  drivers/pci/controller/pcie-xilinx-cpm.c | 86
> > > > > ++++++++++++++++++++++--
> > > > >  1 file changed, 82 insertions(+), 4 deletions(-)
> > > > >
> > > > > diff --git a/drivers/pci/controller/pcie-xilinx-cpm.c
> > > > > b/drivers/pci/controller/pcie-xilinx-
> > > > cpm.c
> > > > > index d0ab187d917f..b10c0752a94f 100644
> > > > > --- a/drivers/pci/controller/pcie-xilinx-cpm.c
> > > > > +++ b/drivers/pci/controller/pcie-xilinx-cpm.c
> > > > > @@ -6,6 +6,8 @@
> > > > >   */
> > > > >
> > > > >  #include <linux/bitfield.h>
> > > > > +#include <linux/delay.h>
> > > > > +#include <linux/gpio/consumer.h>
> > > > >  #include <linux/interrupt.h>
> > > > >  #include <linux/irq.h>
> > > > >  #include <linux/irqchip.h>
> > > > > @@ -21,6 +23,13 @@
> > > > >  #include "pcie-xilinx-common.h"
> > > > >
> > > > >  /* Register definitions */
> > > > > +#define XILINX_CPM_PCIE0_RST         0x00000308
> > > > > +#define XILINX_CPM5_PCIE0_RST                0x00000318
> > > > > +#define XILINX_CPM5_PCIE1_RST                0x0000031C
> > > > > +#define XILINX_CPM5NC_PCIE0_RST              0x00000324
> > > > > +
> > > > > +#define XILINX_CPM5NC_PCIE0_FRWALL   0x00001140
> > > > > +
> > > > >  #define XILINX_CPM_PCIE_REG_IDR              0x00000E10
> > > > >  #define XILINX_CPM_PCIE_REG_IMR              0x00000E14
> > > > >  #define XILINX_CPM_PCIE_REG_PSCR     0x00000E1C
> > > > > @@ -99,6 +108,7 @@ struct xilinx_cpm_variant {
> > > > >       u32 ir_status;
> > > > >       u32 ir_enable;
> > > > >       u32 ir_misc_value;
> > > > > +     u32 cpm_pcie_rst;
> > > > >  };
> > > > >
> > > > >  /**
> > > > > @@ -106,6 +116,8 @@ struct xilinx_cpm_variant {
> > > > >   * @dev: Device pointer
> > > > >   * @reg_base: Bridge Register Base
> > > > >   * @cpm_base: CPM System Level Control and Status Register(SLCR)
> > > > > Base
> > > > > + * @crx_base: CPM Clock and Reset Control Registers Base
> > > > > + * @cpm5nc_attr_base: CPM5NC Control and Status Registers Base
> > > > >   * @intx_domain: Legacy IRQ domain pointer
> > > > >   * @cpm_domain: CPM IRQ domain pointer
> > > > >   * @cfg: Holds mappings of config space window @@ -118,6 +130,8
> > > > > @@ struct xilinx_cpm_pcie {
> > > > >       struct device                   *dev;
> > > > >       void __iomem                    *reg_base;
> > > > >       void __iomem                    *cpm_base;
> > > > > +     void __iomem                    *crx_base;
> > > > > +     void __iomem                    *cpm5nc_attr_base;
> > > > >       struct irq_domain               *intx_domain;
> > > > >       struct irq_domain               *cpm_domain;
> > > > >       struct pci_config_window        *cfg;
> > > > > @@ -475,12 +489,45 @@ static int xilinx_cpm_setup_irq(struct
> > > > > xilinx_cpm_pcie
> > > > *port)
> > > > >   * xilinx_cpm_pcie_init_port - Initialize hardware
> > > > >   * @port: PCIe port information
> > > > >   */
> > > > > -static void xilinx_cpm_pcie_init_port(struct xilinx_cpm_pcie
> > > > > *port)
> > > > > +static int xilinx_cpm_pcie_init_port(struct xilinx_cpm_pcie
> > > > > +*port)
> > > > >  {
> > > > >       const struct xilinx_cpm_variant *variant = port->variant;
> > > > > +     struct device *dev = port->dev;
> > > > > +     struct gpio_desc *reset_gpio;
> > > > > +
> > > > > +     /* Request the GPIO for PCIe reset signal */
> > > > > +     reset_gpio = devm_gpiod_get_optional(dev, "reset", GPIOD_OUT_HIGH);
> > > > > +     if (IS_ERR(reset_gpio)) {
> > > > > +             if (PTR_ERR(reset_gpio) != -EPROBE_DEFER)
> > > > > +                     dev_err(dev, "Failed to request reset GPIO\n");
> > > > > +             return PTR_ERR(reset_gpio);
> > > > > +     }
> > > > >
> > > > > -     if (variant->version == CPM5NC_HOST)
> > > > > -             return;
> > > > > +     if (reset_gpio && port->crx_base) {
> > > > > +             /* Assert the PCIe IP reset */
> > > > > +             writel_relaxed(0x1, port->crx_base +
> > > > > + variant->cpm_pcie_rst);
> > > > > +
> > > > > +             /* Controller specific delay */
> > > > > +             udelay(50);
> > > > > +
> > > >
> > > > There should be atleast 100ms delay before PERST# deassert as per
> > > > the spec. So use PCIE_T_PVPERL_MS. I know that you had it before,
> > > > but removed in v4. I don't see a valid reason for that.
> > >
> > > For CPM/CPM5/CPM5NC, the "Power Up" sequence mentioned in section
> > > 2.2.1 of PCIe Electromechanical Spec is handled in the design. The
> > > PERST# we are using here is applied after the Power Up sequence and
> > > will be used for warm reset, where power of the system is already stable.
> > >
> >
> > I don't quite understand what you mean by 'warm reset' here. Even if the power was
> > already stable, what is the guarantee that the 100ms time is elapsed before
> > deasserting the PERST#? Does the hardware logic ensure 100ms time is elapsed
> > before the driver is probed?
> >
> 
> The Initial Power Up sequence is handled in hardware logic, and 100ms
> (T_PVPERL) delay is provided after the power becomes stable. Yes, this part
> is handled before the driver is probed.
> 

Ok, in that case, please mention it in comments before deasserting PERST#.
Otherwise, no one except you will know.

> By "warm reset" here, I'm referring to a reset that does not involve power
> cycling the device, as per PCIe spec section 6.6.1. The power rails remain
> stable, and only PERST# is toggled through the driver.
> 
> As per the PCIe Spec replaced 50us with 100us (T_PERST) before PERST#
> deassert in driver, will send it in next patch.
> 

You mean T_PERST-CLK? I don't think you need to wait for this. IIUC, this delay
is already part of in T_PVPERL. This requirement is to make sure that the refclk
becomes active atleast T_PERST-CLK time before deasserting PERST#. I don't think
you can guarantee that in software by introducing a delay without controlling
refclk.

- Mani

-- 
மணிவண்ணன் சதாசிவம்