Re: [PATCH v7 9/9] iommu/vt-d: Add iotlb flush for nested domain

[Yi Liu]

On 2023/12/22 14:57, Tian, Kevin wrote:
> > From: Liu, Yi L <yi.l.liu@xxxxxxxxx>
> > Sent: Thursday, December 21, 2023 11:40 PM
> >
> > +
> > +static void intel_nested_flush_cache(struct dmar_domain *domain, u64
> > addr,
> > +				     unsigned long npages, u32 *error)
> > +{
> > +	struct iommu_domain_info *info;
> > +	unsigned long i;
> > +	unsigned mask;
> > +	u32 fault = 0;
> > +
> > +	if (npages == U64_MAX)
> > +		mask = 64 - VTD_PAGE_SHIFT;
> > +	else
> > +		mask = ilog2(__roundup_pow_of_two(npages));
> > +
> > +	xa_for_each(&domain->iommu_array, i, info) {
> > +		nested_flush_pasid_iotlb(info->iommu, domain, addr,
> > npages, 0);
>
> so IOMMU_VTD_INV_FLAGS_LEAF is defined but ignored?

yeah... it is. It is named as ih in the driver code. But it appears only
the below code is set ih. When calling iommu_flush_iotlb_psi(), the 5th
parameter (ih) may be true.

static int intel_iommu_memory_notifier(struct notifier_block *nb,
				       unsigned long val, void *v)
{
	struct memory_notify *mhp = v;
	unsigned long start_vpfn = mm_to_dma_pfn(mhp->start_pfn);
	unsigned long last_vpfn = mm_to_dma_pfn(mhp->start_pfn +
			mhp->nr_pages - 1);

	switch (val) {
	case MEM_GOING_ONLINE:
		if (iommu_domain_identity_map(si_domain,
					      start_vpfn, last_vpfn)) {
			pr_warn("Failed to build identity map for [%lx-%lx]\n",
				start_vpfn, last_vpfn);
			return NOTIFY_BAD;
		}
		break;

	case MEM_OFFLINE:
	case MEM_CANCEL_ONLINE:
		{
			struct dmar_drhd_unit *drhd;
			struct intel_iommu *iommu;
			LIST_HEAD(freelist);

			domain_unmap(si_domain, start_vpfn, last_vpfn, &freelist);

			rcu_read_lock();
			for_each_active_iommu(iommu, drhd)
				iommu_flush_iotlb_psi(iommu, si_domain,
					start_vpfn, mhp->nr_pages,
					list_empty(&freelist), 0);
			rcu_read_unlock();
			put_pages_list(&freelist);
		}
		break;
	}

	return NOTIFY_OK;
}

> > +
> > +		if (domain->has_iotlb_device)
> > +			continue;
> > +
> > +		nested_flush_dev_iotlb(domain, addr, mask, &fault);
> > +		if (fault & (DMA_FSTS_ITE | DMA_FSTS_ICE))
> > +			break;
>
> here you may add a note that we don't plan to forward invalidation
> queue error (i.e. IQE) to the caller as it's caused only by driver
> internal bug.

yes.

> > +
> > +		if (!IS_ALIGNED(inv_entry.addr, VTD_PAGE_SIZE) ||
> > +		    ((inv_entry.npages == U64_MAX) && inv_entry.addr)) {
> > +			ret = -EINVAL;
> > +			break;
> > +		}
> > +
>
> why is [non-zero-addr, U64_MAX] an error? Is it explicitly stated to
> be not supported by underlying helpers?

no such limitation by underlying helpers. But in such case, the 
addr+npages*PAGE_SIZE would exceed U64_MAX, this seems a bit
strange. But I'm fine to relax the check since the underlying helper
only checks npages when determining paid-selective or not.

--
Regards,
Yi Liu