Re: [PATCH v7 07/11] iommu/vt-d: Add nested translation helper function

From: Jacob Pan
Date: Fri Nov 01 2019 - 17:06:05 EST

Next message: dann frazier: "tracefs splats in lockdown=confidentiality mode"
Previous message: Stefan Agner: "Re: [PATCH] arm: replace Sun/Solaris style flag on section directive"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On Fri, 25 Oct 2019 07:04:28 +0000
"Tian, Kevin" <kevin.tian@xxxxxxxxx> wrote:

> > From: Jacob Pan [mailto:jacob.jun.pan@xxxxxxxxxxxxxxx]
> > Sent: Friday, October 25, 2019 3:55 AM
> >
> > Nested translation mode is supported in VT-d 3.0 Spec.CH 3.8.
> > With PASID granular translation type set to 0x11b, translation
> > result from the first level(FL) also subject to a second level(SL)
> > page table translation. This mode is used for SVA virtualization,
> > where FL performs guest virtual to guest physical translation and
> > SL performs guest physical to host physical translation.
>
> I think we really differentiate what is the common logic for
> first-level usages (GVA, GIOVA, etc.) in scalable mode, and
> what is specific to SVA. I have the feeling that SVA is over-used
> to cause confusing interpretation.
>
Good point, it should be clearly stated that nest mode is not for gSVA
only, gIOVA shares this common code.
> >
> > Signed-off-by: Jacob Pan <jacob.jun.pan@xxxxxxxxxxxxxxx>
> > Signed-off-by: Liu, Yi L <yi.l.liu@xxxxxxxxxxxxxxx>
> > ---
> > drivers/iommu/intel-pasid.c | 207
> > ++++++++++++++++++++++++++++++++++++++++++++
> > drivers/iommu/intel-pasid.h | 12 +++
> > 2 files changed, 219 insertions(+)
> >
> > diff --git a/drivers/iommu/intel-pasid.c
> > b/drivers/iommu/intel-pasid.c index ffbd416ed3b8..f846a907cfcf
> > 100644 --- a/drivers/iommu/intel-pasid.c
> > +++ b/drivers/iommu/intel-pasid.c
> > @@ -415,6 +415,76 @@ pasid_set_flpm(struct pasid_entry *pe, u64
> > value) pasid_set_bits(&pe->val[2], GENMASK_ULL(3, 2), value << 2);
> > }
> >
> > +/*
> > + * Setup the Extended Memory Type(EMT) field (Bits 91-93)
> > + * of a scalable mode PASID entry.
> > + */
> > +static inline void
> > +pasid_set_emt(struct pasid_entry *pe, u64 value)
> > +{
> > + pasid_set_bits(&pe->val[1], GENMASK_ULL(29, 27), value <<
> > 27); +}
> > +
> > +/*
> > + * Setup the Page Attribute Table (PAT) field (Bits 96-127)
> > + * of a scalable mode PASID entry.
> > + */
> > +static inline void
> > +pasid_set_pat(struct pasid_entry *pe, u64 value)
> > +{
> > + pasid_set_bits(&pe->val[1], GENMASK_ULL(63, 32), value <<
> > 27); +}
> > +
> > +/*
> > + * Setup the Cache Disable (CD) field (Bit 89)
> > + * of a scalable mode PASID entry.
> > + */
> > +static inline void
> > +pasid_set_cd(struct pasid_entry *pe)
> > +{
> > + pasid_set_bits(&pe->val[1], 1 << 25, 1);
> > +}
> > +
> > +/*
> > + * Setup the Extended Memory Type Enable (EMTE) field (Bit 90)
> > + * of a scalable mode PASID entry.
> > + */
> > +static inline void
> > +pasid_set_emte(struct pasid_entry *pe)
> > +{
> > + pasid_set_bits(&pe->val[1], 1 << 26, 1);
> > +}
> > +
> > +/*
> > + * Setup the Extended Access Flag Enable (EAFE) field (Bit 135)
> > + * of a scalable mode PASID entry.
> > + */
> > +static inline void
> > +pasid_set_eafe(struct pasid_entry *pe)
> > +{
> > + pasid_set_bits(&pe->val[2], 1 << 7, 1);
> > +}
> > +
> > +/*
> > + * Setup the Page-level Cache Disable (PCD) field (Bit 95)
> > + * of a scalable mode PASID entry.
> > + */
> > +static inline void
> > +pasid_set_pcd(struct pasid_entry *pe)
> > +{
> > + pasid_set_bits(&pe->val[1], 1 << 31, 1);
> > +}
> > +
> > +/*
> > + * Setup the Page-level Write-Through (PWT)) field (Bit 94)
> > + * of a scalable mode PASID entry.
> > + */
> > +static inline void
> > +pasid_set_pwt(struct pasid_entry *pe)
> > +{
> > + pasid_set_bits(&pe->val[1], 1 << 30, 1);
> > +}
> > +
> > static void
> > pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu,
> > u16 did, int pasid)
> > @@ -647,3 +717,140 @@ int intel_pasid_setup_pass_through(struct
> > intel_iommu *iommu,
> >
> > return 0;
> > }
> > +
> > +static int intel_pasid_setup_bind_data(struct intel_iommu *iommu,
> > + struct pasid_entry *pte,
> > + struct iommu_gpasid_bind_data_vtd
> > *pasid_data)
> > +{
> > + /*
> > + * Not all guest PASID table entry fields are passed down
> > during bind,
> > + * here we only set up the ones that are dependent on guest
> > settings.
> > + * Execution related bits such as NXE, SMEP are not
> > meaningful to IOMMU,
> > + * therefore not set. Other fields, such as snoop related,
> > are set based
> > + * on host needs regardless of guest settings.
> > + */
> > + if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_SRE) {
> > + if (!ecap_srs(iommu->ecap)) {
> > + pr_err("No supervisor request support on
> > %s\n",
> > + iommu->name);
> > + return -EINVAL;
> > + }
> > + pasid_set_sre(pte);
> > + }
> > +
> > + if ((pasid_data->flags & IOMMU_SVA_VTD_GPASID_EAFE) &&
> > ecap_eafs(iommu->ecap))
> > + pasid_set_eafe(pte);
> > +
> > + if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_EMTE) {
> > + pasid_set_emte(pte);
> > + pasid_set_emt(pte, pasid_data->emt);
> > + }
>
> above conditional checks are not consistent. The 1st check may
> return error but latter two don't. Can you confirm whether it's
> desired way?
>
should be consistent and under the check of host MTS capability. Will
change.
> > +
> > + /*
> > + * Memory type is only applicable to devices inside
> > processor coherent
> > + * domain. PCIe devices are not included. We can skip the
> > rest of the
> > + * flags if IOMMU does not support MTS.
> > + */
> > + if (!ecap_mts(iommu->ecap)) {
> > + pr_info("%s does not support memory type bind guest
> > PASID\n",
> > + iommu->name);
> > + return 0;
>
> why not -EINVAL?
>
right, if host does not support MTS and guest wants to set MTS related
bits, -EINVAL should be returned.

> > + }
> > +
> > + if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_PCD)
> > + pasid_set_pcd(pte);
> > + if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_PWT)
> > + pasid_set_pwt(pte);
> > + if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_CD)
> > + pasid_set_cd(pte);
> > + pasid_set_pat(pte, pasid_data->pat);
> > +
> > + return 0;
> > +
> > +}
> > +
> > +/**
> > + * intel_pasid_setup_nested() - Set up PASID entry for nested
> > translation
> > + * which is used for vSVA. The first level page tables are used for
> > + * GVA-GPA translation in the guest, second level page tables are
> > used
> > + * for GPA to HPA translation.
>
> It's too restricting on how 1st level is used by guest.
>
will fix. stating FL can be used for gIOVA in nested mode.
> > + *
> > + * @iommu: Iommu which the device belong to
> > + * @dev: Device to be set up for translation
> > + * @gpgd: FLPTPTR: First Level Page translation pointer in
> > GPA
> > + * @pasid: PASID to be programmed in the device PASID table
> > + * @pasid_data: Additional PASID info from the guest bind request
> > + * @domain: Domain info for setting up second level page tables
> > + * @addr_width: Address width of the first level (guest)
> > + */
> > +int intel_pasid_setup_nested(struct intel_iommu *iommu,
> > + struct device *dev, pgd_t *gpgd,
> > + int pasid, struct
> > iommu_gpasid_bind_data_vtd *pasid_data,
> > + struct dmar_domain *domain,
> > + int addr_width)
> > +{
> > + struct pasid_entry *pte;
> > + struct dma_pte *pgd;
> > + u64 pgd_val;
> > + int agaw;
> > + u16 did;
> > +
> > + if (!ecap_nest(iommu->ecap)) {
> > + pr_err("IOMMU: %s: No nested translation
> > support\n",
> > + iommu->name);
> > + return -EINVAL;
> > + }
> > +
> > + pte = intel_pasid_get_entry(dev, pasid);
> > + if (WARN_ON(!pte))
> > + return -EINVAL;
> > +
> > + pasid_clear_entry(pte);
> > +
> > + /* Sanity checking performed by caller to make sure address
> > + * width matching in two dimensions:
> > + * 1. CPU vs. IOMMU
> > + * 2. Guest vs. Host.
> > + */
> > + switch (addr_width) {
> > + case 57:
>
> AW_5LEVEL
>
> > + pasid_set_flpm(pte, 1);
> > + break;
> > + case 48:
>
> AW_4LEVEL
>
will add macros

> > + pasid_set_flpm(pte, 0);
> > + break;
> > + default:
> > + dev_err(dev, "Invalid paging mode %d\n",
> > addr_width);
> > + return -EINVAL;
> > + }
> > +
> > + pasid_set_flptr(pte, (u64)gpgd);
> > +
> > + intel_pasid_setup_bind_data(iommu, pte, pasid_data);
> > +
> > + /* Setup the second level based on the given domain */
> > + pgd = domain->pgd;
> > +
> > + for (agaw = domain->agaw; agaw != iommu->agaw; agaw--) {
> > + pgd = phys_to_virt(dma_pte_addr(pgd));
> > + if (!dma_pte_present(pgd)) {
> > + dev_err(dev, "Invalid domain page
> > table\n");
>
> pasid_clear_entry?
>
right, even though present bit is not set still a good practice to
clear.

> > + return -EINVAL;
> > + }
> > + }
> > + pgd_val = virt_to_phys(pgd);
> > + pasid_set_slptr(pte, pgd_val);
> > + pasid_set_fault_enable(pte);
> > +
> > + did = domain->iommu_did[iommu->seq_id];
> > + pasid_set_domain_id(pte, did);
> > +
> > + pasid_set_address_width(pte, agaw);
> > + pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
> > +
> > + pasid_set_translation_type(pte, PASID_ENTRY_PGTT_NESTED);
> > + pasid_set_present(pte);
> > + pasid_flush_caches(iommu, pte, pasid, did);
> > +
> > + return 0;
> > +}
> > diff --git a/drivers/iommu/intel-pasid.h
> > b/drivers/iommu/intel-pasid.h index e413e884e685..09c85db73b77
> > 100644 --- a/drivers/iommu/intel-pasid.h
> > +++ b/drivers/iommu/intel-pasid.h
> > @@ -46,6 +46,7 @@
> > * to vmalloc or even module mappings.
> > */
> > #define PASID_FLAG_SUPERVISOR_MODE BIT(0)
> > +#define PASID_FLAG_NESTED BIT(1)
> >
> > struct pasid_dir_entry {
> > u64 val;
> > @@ -55,6 +56,11 @@ struct pasid_entry {
> > u64 val[8];
> > };
> >
> > +#define PASID_ENTRY_PGTT_FL_ONLY (1)
> > +#define PASID_ENTRY_PGTT_SL_ONLY (2)
> > +#define PASID_ENTRY_PGTT_NESTED (3)
> > +#define PASID_ENTRY_PGTT_PT (4)
> > +
> > /* The representative of a PASID table */
> > struct pasid_table {
> > void *table; /*
> > pasid table pointer */ @@ -103,6 +109,12 @@ int
> > intel_pasid_setup_second_level(struct intel_iommu *iommu,
> > int intel_pasid_setup_pass_through(struct intel_iommu *iommu,
> > struct dmar_domain *domain,
> > struct device *dev, int pasid);
> > +int intel_pasid_setup_nested(struct intel_iommu *iommu,
> > + struct device *dev, pgd_t *pgd,
> > + int pasid,
> > + struct iommu_gpasid_bind_data_vtd
> > *pasid_data,
> > + struct dmar_domain *domain,
> > + int addr_width);
> > void intel_pasid_tear_down_entry(struct intel_iommu *iommu,
> > struct device *dev, int pasid);
> > int vcmd_alloc_pasid(struct intel_iommu *iommu, unsigned int
> > *pasid); --
> > 2.7.4
>

[Jacob Pan]

Next message: dann frazier: "tracefs splats in lockdown=confidentiality mode"
Previous message: Stefan Agner: "Re: [PATCH] arm: replace Sun/Solaris style flag on section directive"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]