Re: [PATCH v3 0/7] iommu: Allow IOVA rcache range be configured

[John Garry]

On 02/06/2021 05:37, Lu Baolu wrote:
> On 6/1/21 10:29 PM, John Garry wrote:
> > For streaming DMA mappings involving an IOMMU and whose IOVA len 
> > regularly
> > exceeds the IOVA rcache upper limit (meaning that they are not cached),
> > performance can be reduced.
> >
> > This is much more pronounced from commit 4e89dce72521 ("iommu/iova: Retry
> > from last rb tree node if iova search fails"), as discussed at [0].
> >
> > IOVAs which cannot be cached are highly involved in the IOVA ageing 
> > issue,
> > as discussed at [1].
> >
> > This series allows the IOVA rcache range be configured, so that we may
> > cache all IOVAs per domain, thus improving performance.
> >
> > A new IOMMU group sysfs file is added - max_opt_dma_size - which is used
> > indirectly to configure the IOVA rcache range:
> > /sys/kernel/iommu_groups/X/max_opt_dma_size
> >
> > This file is updated same as how the IOMMU group default domain type is
> > updated, i.e. must unbind the only device in the group first.
>
> Could you explain why it requires singleton group and driver unbinding
> if the user only wants to increase the upper limit? I haven't dived into
> the details yet, sorry if this is a silly question.

Hi Baolu,

I did actually try increasing the range for a 'live' domain in the v1 
series, but it turned out too messy. First problem is reallocating the 
memory to hold the rcaches. Second problem is that we need to deal with 
the issue that all IOVAs in the rcache need to be a pow-of-2, which is 
difficult to enforce for IOVAs which weren't being cached before, but 
now would be.

So now I changed to work similar to how we change the default domain 
type, i.e. don't operate on a 'live' domain.

Thanks,
John