Re: [PATCHv3] iommu/arm-smmu: Optimize ->tlb_flush_walk() for qcom implementation

From: Sai Prakash Ranjan
Date: Wed Aug 11 2021 - 02:10:22 EST


On 2021-08-10 23:38, Will Deacon wrote:
On Tue, Aug 03, 2021 at 11:09:17AM +0530, Sai Prakash Ranjan wrote:
On 2021-08-02 21:13, Will Deacon wrote:
> On Wed, Jun 23, 2021 at 07:12:01PM +0530, Sai Prakash Ranjan wrote:
> > diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.c
> > b/drivers/iommu/arm/arm-smmu/arm-smmu.c
> > index d3c6f54110a5..f3845e822565 100644
> > --- a/drivers/iommu/arm/arm-smmu/arm-smmu.c
> > +++ b/drivers/iommu/arm/arm-smmu/arm-smmu.c
> > @@ -341,6 +341,12 @@ static void arm_smmu_tlb_add_page_s1(struct
> > iommu_iotlb_gather *gather,
> > ARM_SMMU_CB_S1_TLBIVAL);
> > }
> >
> > +static void arm_smmu_tlb_inv_walk_impl_s1(unsigned long iova,
> > size_t size,
> > + size_t granule, void *cookie)
> > +{
> > + arm_smmu_tlb_inv_context_s1(cookie);
> > +}
> > +
> > static void arm_smmu_tlb_inv_walk_s2(unsigned long iova, size_t size,
> > size_t granule, void *cookie)
> > {
> > @@ -388,6 +394,12 @@ static const struct iommu_flush_ops
> > arm_smmu_s1_tlb_ops = {
> > .tlb_add_page = arm_smmu_tlb_add_page_s1,
> > };
> >
> > +const struct iommu_flush_ops arm_smmu_s1_tlb_impl_ops = {
> > + .tlb_flush_all = arm_smmu_tlb_inv_context_s1,
> > + .tlb_flush_walk = arm_smmu_tlb_inv_walk_impl_s1,
> > + .tlb_add_page = arm_smmu_tlb_add_page_s1,
> > +};
>
> Hmm, dunno about this. Wouldn't it be a lot cleaner if the
> tlb_flush_walk
> callbacks just did the right thing based on the smmu_domain (maybe in
> the
> arm_smmu_cfg?) rather than having an entirely new set of ops just
> because
> they're const and you can't overide the bit you want?
>
> I don't think there's really an awful lot qcom-specific about the
> principle
> here -- there's a trade-off between over-invalidation and invalidation
> latency. That happens on the CPU as well.
>

Sorry didn't understand, based on smmu_domain what? How do we make
this implementation specific? Do you mean something like a quirk?
The reason we didn't make this common was because nvidia folks weren't
so happy with that, you can find the discussion in this thread [1].

[1] https://lore.kernel.org/lkml/20210609145315.25750-1-saiprakash.ranjan@xxxxxxxxxxxxxx/

The ->tlb_flush_walk() callbacks take a 'void *cookie' which, for this
driver, is a 'struct arm_smmu_domain *'. From that, you can get to the
'struct arm_smmu_cfg' which could have something as coarse as:

bool flush_walk_prefer_tlbiasid;

which you can set when you initialise the domain (maybe in the
->init_context callback?). It shouldn't affect anybody else.


Ah ok, you meant a new flag in arm_smmu_cfg, right getting it from cookie
is no big deal but nonetheless thanks for detailing it. I have made the
changes and sent a v4 after testing.

Thanks,
Sai

--
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation