Re: [PATCH v2] iommu/arm-smmu-v3: Add a user-configurable tlb_invalidate_threshold

[Will Deacon]

On Thu, Aug 17, 2023 at 11:36:18AM -0700, Nicolin Chen wrote:
> On Wed, Aug 16, 2023 at 01:43:50PM -0700, Nicolin Chen wrote:
>  
> > When receiving an __arm_smmu_tlb_inv_range() call with a large size, there
> > could be a long latency at this function call: one part is coming from a
> > large software overhead in the routine of building commands, and the other
> > part is coming from CMDQ hardware consuming the large number of commands.
> > This latency could be significantly large on an SMMU that does not support
> > range invalidation commands, i.e. no ARM_SMMU_FEAT_RANGE_INV.
> > 
> > One way to optimize this is to replace a large number of VA invalidation
> > commands with one single per-asid invalidation command, when the requested
> > size reaches a threshold. This threshold can be configurable depending on
> > the SMMU implementaion.
> 
> I'm rethinking about this size-based threshold, since what really
> affects the latency is the number of the invalidation commands in
> the request. So having an npages-based threshold might be optimal,
> though the idea and implementation would be similar.

On the CPU side, we just have:

#define MAX_TLBI_OPS    PTRS_PER_PTE

in asm/tlbflush.h

Can we start off with something similar for the SMMU? I'm not massively
keen on exposing this as a knob to userspace, because I don't think most
people will have a clue about how to tune it.

Will