Re: [PATCH 1/1] iommu/vt-d: Fix dmar pte read access not set error

[Jerry Snitselaar]

On Fri Dec 13 19, Lu Baolu wrote:
> Hi,
>
> On 12/13/19 8:30 AM, Jerry Snitselaar wrote:
> > On Thu Dec 12 19, Lu Baolu wrote:
> > > Hi,
> > >
> > > On 12/12/19 9:49 AM, Jerry Snitselaar wrote:
> > > > On Wed Dec 11 19, Lu Baolu wrote:
> > > > > If the default DMA domain of a group doesn't fit a device, it
> > > > > will still sit in the group but use a private identity domain.
> > > > > When map/unmap/iova_to_phys come through iommu API, the driver
> > > > > should still serve them, otherwise, other devices in the same
> > > > > group will be impacted. Since identity domain has been mapped
> > > > > with the whole available memory space and RMRRs, we don't need
> > > > > to worry about the impact on it.
> > > > >
> > > > > Link: https://www.spinics.net/lists/iommu/msg40416.html
> > > > > Cc: Jerry Snitselaar <jsnitsel@xxxxxxxxxx>
> > > > > Reported-by: Jerry Snitselaar <jsnitsel@xxxxxxxxxx>
> > > > > Fixes: 942067f1b6b97 ("iommu/vt-d: Identify default domains 
> > > > > replaced with private")
> > > > > Cc: stable@xxxxxxxxxxxxxxx # v5.3+
> > > > > Signed-off-by: Lu Baolu <baolu.lu@xxxxxxxxxxxxxxx>
> > > >
> > > > Reviewed-by: Jerry Snitselaar <jsnitsel@xxxxxxxxxx>
> > >
> > > Can you please try this fix and check whether it can fix your problem?
> > > If it helps, do you mind adding a Tested-by?
> > >
> > > Best regards,
> > > baolu
> >
> > I'm testing with this patch, my patch that moves the direct mapping call,
> > and Alex's patch for the ISA bridge. It solved the 2 iommu mapping errors
> > I was seeing with default passthrough, I no longer see all the dmar pte
> > read access errors, and the system boots allowing me to login. I'm tracking
> > down 2 issues at the moment. With passthrough I see a problem with 01:00.4
> > that I mentioned in the earlier email:
> >
> > [ÂÂ 78.978573] uhci_hcd: USB Universal Host Controller Interface driver
> > [ÂÂ 78.980842] uhci_hcd 0000:01:00.4: UHCI Host Controller
> > [ÂÂ 78.982738] uhci_hcd 0000:01:00.4: new USB bus registered, 
> > assigned bus number 3
> > [ÂÂ 78.985222] uhci_hcd 0000:01:00.4: detected 8 ports
> > [ÂÂ 78.986907] uhci_hcd 0000:01:00.4: port count misdetected? 
> > forcing to 2 ports
> > [ÂÂ 78.989316] uhci_hcd 0000:01:00.4: irq 16, io base 0x00003c00
> > [ÂÂ 78.994634] uhci_hcd 0000:01:00.4: DMAR: 32bit DMA uses 
> > non-identity mapping
> > [ÂÂ 7 0000:01:00.4: unable to allocate consistent memory for frame list
> > [ÂÂ 79.499891] uhci_hcd 0000:01:00.4: startup error -16
> > [ÂÂ 79.501588] uhci_hcd 0000:01:00.4: USB bus 3 deregistered
> > [ÂÂ 79.503494] uhci_hcd 0000:01:00.4: init 0000:01:00.4 fail, -16
> > [ÂÂ 79.505497] uhci_hcd: probe of 0000:01:00.4 failed with error -16
> >
> > If I boot the system with iommu=nopt I see an iommu map failure due to
> > the prot check in __domain_mapping:
> >
> > [ÂÂ 40.940589] pci 0000:00:1f.0: iommu_group_add_device: calling 
> > iommu_group_create_direct_mappings
> > [ÂÂ 40.943558] pci 0000:00:1f.0: iommu_group_create_direct_mappings: 
> > iterating through mappings
> > [ÂÂ 40.946402] pci 0000:00:1f.0: iommu_group_create_direct_mappings: 
> > calling apply_resv_region
> > [ÂÂ 40.949184] pci 0000:00:1f.0: iommu_group_create_direct_mappings: 
> > entry type is direct
> > [ÂÂ 40.951819] DMAR: intel_iommu_map: enter
> > [ÂÂ 40.953128] DMAR: __domain_mapping: prot & 
> > (DMA_PTE_READ|DMA_PTE_WRITE) == 0
> > [ÂÂ 40.955486] DMAR: domain_mapping: __domain_mapping failed
> > [ÂÂ 40.957348] DMAR: intel_iommu_map: domain_pfn_mapping returned -22
> > [ÂÂ 40.959466] DMAR: intel_iommu_map: leave
> > [ÂÂ 40.959468] iommu: iommu_map: ops->map failed iova 0x0 pa 
> > 0x0000000000000000 pgsize 0x1000
> > [ÂÂ 40.963511] pci 0000:00:1f.0: iommu_group_create_direct_mappings: 
> > iommu_map failed
> > [ÂÂ 40.966026] pci 0000:00:1f.0: iommu_group_create_direct_mappings: 
> > leaving func
> > [ÂÂ 40.968487] pci 0000:00:1f.0: iommu_group_add_device: calling 
> > __iommu_attach_device
> > [ÂÂ 40.971016] pci 0000:00:1f.0: Adding to iommu group 19
> > [ÂÂ 40.972731] pci 0000:00:1f.0: DMAR: domain->type is dma
> >
> > /sys/kernel/iommu_groups/19
> > [root@hp-dl388g8-07 19]# cat reserved_regions 0x0000000000000000 
> > 0x0000000000ffffff direct
> > 0x00000000bdf6e000 0x00000000bdf84fff direct
> > 0x00000000fee00000 0x00000000feefffff msi
> >
> > 00:1f.0 ISA bridge: Intel Corporation C600/X79 series chipset LPC 
> > Controller
>
> This seems to be another issue?
>
> Best regards,
> baolu

In intel_iommu_get_resv_regions this iommu_alloc_resv_region is called
with prot set to 0:

                if ((pdev->class >> 8) == PCI_CLASS_BRIDGE_ISA) {
                        reg = iommu_alloc_resv_region(0, 1UL << 24, 0,
                                                      IOMMU_RESV_DIRECT_RELAXABLE);
                        if (reg)

I wonder if this is an issue with the region starting at 0x0 and this
bit in iommu_group_create_mappings:

			phys_addr = iommu_iova_to_phys(domain, addr);
			if (phys_addr)
				continue;

Off to stick in some more debugging statements.

Regards,
Jerry

> _______________________________________________
> iommu mailing list
> iommu@xxxxxxxxxxxxxxxxxxxxxxxxxx
> https://lists.linuxfoundation.org/mailman/listinfo/iommu