Re: [PATCH v2 15/17] PCI: dwc: Introduce dma-ranges property support for RC-host
From: Manivannan Sadhasivam
Date: Thu May 12 2022 - 09:57:42 EST
On Wed, May 04, 2022 at 12:46:36AM +0300, Serge Semin wrote:
> In accordance with the generic PCIe Root Port DT-bindings the "dma-ranges"
> property has the same format as the "ranges" property. The only difference
> is in their semantics. The "dma-ranges" property describes the PCIe-to-CPU
> memory mapping in opposite to the CPU-to-PCIe mapping of the "ranges"
> property. Even though the DW PCIe controllers are normally equipped with
> internal Address Translation Unit which inbound and outbound tables can be
> used to implement both properties semantics, it was surprise for me to
> discover that the host-related part of the DW PCIe driver currently
> supports the "ranges" property only while the "dma-ranges" windows are
> just ignored. Having the "dma-ranges" supported in the driver would be
> very handy for the platforms, that don't tolerate the 1:1 CPU-PCIe memory
> mapping and require customized the PCIe memory layout. So let's fix that
> by introducing the "dma-ranges" property support.
>
> First of all we suggest to rename the dw_pcie_prog_inbound_atu() method to
> dw_pcie_prog_ep_inbound_atu() and create a new version of the
> dw_pcie_prog_inbound_atu() function. Thus we'll have two methods for RC
> and EP controllers respectively in the same way as it has been developed
> for the outbound ATU setup methods.
>
> Secondly aside with the memory window index and type the new
> dw_pcie_prog_inbound_atu() function will accept CPU address, PCIe address
> and size as its arguments. These parameters define the PCIe and CPU memory
> ranges which will be used to setup the respective inbound ATU mapping. The
> passed parameters need to be verified against the ATU ranges constraints
> in the same way as it is done for the outbound ranges.
>
> Finally the DMA-ranges detected for the PCIe controller need to be
> converted into the inbound ATU entries during the host controller
> initialization procedure. It will be done in the framework of the
> dw_pcie_iatu_setup() method. Note before setting the inbound ranges up we
> need to disable all the inbound ATU entries in order to prevent unexpected
> PCIe TLPs translations defined by some third party software like
> bootloader.
>
> Signed-off-by: Serge Semin <Sergey.Semin@xxxxxxxxxxxxxxxxxxxx>
> ---
> .../pci/controller/dwc/pcie-designware-ep.c | 4 +-
> .../pci/controller/dwc/pcie-designware-host.c | 32 ++++++++++-
> drivers/pci/controller/dwc/pcie-designware.c | 57 ++++++++++++++++++-
> drivers/pci/controller/dwc/pcie-designware.h | 6 +-
> 4 files changed, 90 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c
> index c62640201246..9b0540cfa9e8 100644
> --- a/drivers/pci/controller/dwc/pcie-designware-ep.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c
> @@ -167,8 +167,8 @@ static int dw_pcie_ep_inbound_atu(struct dw_pcie_ep *ep, u8 func_no, int type,
> return -EINVAL;
> }
>
> - ret = dw_pcie_prog_inbound_atu(pci, func_no, free_win, type,
> - cpu_addr, bar);
> + ret = dw_pcie_prog_ep_inbound_atu(pci, func_no, free_win, type,
> + cpu_addr, bar);
> if (ret < 0) {
> dev_err(pci->dev, "Failed to program IB window\n");
> return ret;
> diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
> index 7caca6c575a5..9cb406f5c185 100644
> --- a/drivers/pci/controller/dwc/pcie-designware-host.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-host.c
> @@ -612,12 +612,15 @@ static int dw_pcie_iatu_setup(struct pcie_port *pp)
> }
>
> /*
> - * Ensure all outbound windows are disabled before proceeding with
> - * the MEM/IO ranges setups.
> + * Ensure all out/inbound windows are disabled before proceeding with
> + * the MEM/IO (dma-)ranges setups.
> */
> for (i = 0; i < pci->num_ob_windows; i++)
> dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_OB, i);
>
> + for (i = 0; i < pci->num_ib_windows; i++)
> + dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_IB, i);
> +
> i = 0;
> resource_list_for_each_entry(entry, &pp->bridge->windows) {
> if (resource_type(entry->res) != IORESOURCE_MEM)
> @@ -654,9 +657,32 @@ static int dw_pcie_iatu_setup(struct pcie_port *pp)
> }
>
> if (pci->num_ob_windows <= i)
> - dev_warn(pci->dev, "Resources exceed number of ATU entries (%d)\n",
> + dev_warn(pci->dev, "Ranges exceed outbound iATU size (%d)\n",
> pci->num_ob_windows);
>
> + i = 0;
> + resource_list_for_each_entry(entry, &pp->bridge->dma_ranges) {
> + if (resource_type(entry->res) != IORESOURCE_MEM)
> + continue;
> +
> + if (pci->num_ib_windows <= i)
> + break;
> +
> + ret = dw_pcie_prog_inbound_atu(pci, i++, PCIE_ATU_TYPE_MEM,
> + entry->res->start,
> + entry->res->start - entry->offset,
> + resource_size(entry->res));
> + if (ret) {
> + dev_err(pci->dev, "Failed to set DMA range %pr\n",
> + entry->res);
> + return ret;
> + }
> + }
> +
> + if (pci->num_ib_windows <= i)
> + dev_warn(pci->dev, "Dma-ranges exceed inbound iATU size (%u)\n",
> + pci->num_ib_windows);
> +
> return 0;
> }
>
> diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
> index 747e252c09e6..33718ed6c511 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.c
> +++ b/drivers/pci/controller/dwc/pcie-designware.c
> @@ -397,8 +397,61 @@ static inline void dw_pcie_writel_atu_ib(struct dw_pcie *pci, u32 index, u32 reg
> dw_pcie_writel_atu(pci, PCIE_ATU_REGION_DIR_IB, index, reg, val);
> }
>
> -int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
> - int type, u64 cpu_addr, u8 bar)
> +int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, int index, int type,
> + u64 cpu_addr, u64 pci_addr, u64 size)
> +{
> + u64 limit_addr = pci_addr + size - 1;
> + u32 retries, val;
> +
> + if ((limit_addr & ~pci->region_limit) != (pci_addr & ~pci->region_limit) ||
> + !IS_ALIGNED(cpu_addr, pci->region_align) ||
> + !IS_ALIGNED(pci_addr, pci->region_align) ||
> + !IS_ALIGNED(size, pci->region_align) ||
Why do you want the size to be aligned? What if I want to transfer a small size
buffer?
Same question applies to outbound programming as well.
Thanks,
Mani
--
மணிவண்ணன் சதாசிவம்