Re: [PATCH v7 3/3] IOMMU/PCI: Reserve IOVA for inbound memory for PCI masters

[Oza Oza]

Hi Robin,

My apology for noise.

I have taken care of your comments.
but these whole patch-set, (specially PCI patch-set) inbound memory
addition depends on Lorenzo's patch-set
.
So I will be posting version 8 patches for IOVA reservation soon after
Lorenzo's patches are made in.

Regards,
Oza.

On Mon, May 22, 2017 at 10:09 PM, Oza Pawandeep <oza.oza@xxxxxxxxxxxx> wrote:
> This patch reserves the inbound memory holes for PCI masters.
> ARM64 based SOCs may have scattered memory banks.
> For e.g as iproc based SOC has
>
> <0x00000000 0x80000000 0x0 0x80000000>, /* 2G @ 2G */
> <0x00000008 0x80000000 0x3 0x80000000>, /* 14G @ 34G */
> <0x00000090 0x00000000 0x4 0x00000000>, /* 16G @ 576G */
> <0x000000a0 0x00000000 0x4 0x00000000>; /* 16G @ 640G */
>
> But incoming PCI transaction addressing capability is limited
> by host bridge, for example if max incoming window capability
> is 512 GB, then 0x00000090 and 0x000000a0 will fall beyond it.
>
> To address this problem, iommu has to avoid allocating IOVA which
> are reserved.
>
> Which in turn does not allocate IOVA if it falls into hole.
> and the holes should be reserved before any of the IOVA allocations
> can happen.
>
> Signed-off-by: Oza Pawandeep <oza.oza@xxxxxxxxxxxx>
>
> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> index 8348f366..efe3d07 100644
> --- a/drivers/iommu/dma-iommu.c
> +++ b/drivers/iommu/dma-iommu.c
> @@ -171,16 +171,15 @@ void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list)
>  {
>         struct pci_host_bridge *bridge;
>         struct resource_entry *window;
> +       struct iommu_resv_region *region;
> +       phys_addr_t start, end;
> +       size_t length;
>
>         if (!dev_is_pci(dev))
>                 return;
>
>         bridge = pci_find_host_bridge(to_pci_dev(dev)->bus);
>         resource_list_for_each_entry(window, &bridge->windows) {
> -               struct iommu_resv_region *region;
> -               phys_addr_t start;
> -               size_t length;
> -
>                 if (resource_type(window->res) != IORESOURCE_MEM)
>                         continue;
>
> @@ -193,6 +192,43 @@ void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list)
>
>                 list_add_tail(&region->list, list);
>         }
> +
> +       /* PCI inbound memory reservation. */
> +       start = length = 0;
> +       resource_list_for_each_entry(window, &bridge->inbound_windows) {
> +               end = window->res->start - window->offset;
> +
> +               if (start > end) {
> +                       /* multiple ranges assumed sorted. */
> +                       pr_warn("PCI: failed to reserve iovas\n");
> +                       return;
> +               }
> +
> +               if (start != end) {
> +                       length = end - start - 1;
> +                       region = iommu_alloc_resv_region(start, length, 0,
> +                               IOMMU_RESV_RESERVED);
> +                       if (!region)
> +                               return;
> +
> +                       list_add_tail(&region->list, list);
> +               }
> +
> +               start += end + length + 1;
> +       }
> +       /*
> +        * the last dma-range should honour based on the
> +        * 32/64-bit dma addresses.
> +        */
> +       if ((start) && (start < DMA_BIT_MASK(sizeof(dma_addr_t) * 8))) {
> +               length = DMA_BIT_MASK((sizeof(dma_addr_t) * 8)) - 1;
> +               region = iommu_alloc_resv_region(start, length, 0,
> +                       IOMMU_RESV_RESERVED);
> +               if (!region)
> +                       return;
> +
> +               list_add_tail(&region->list, list);
> +       }
>  }
>  EXPORT_SYMBOL(iommu_dma_get_resv_regions);
>
> --
> 1.9.1
>