Re: IOMMU vs Ryzen embedded EMMC controller

[Jiri Kosina]

On Sat, 21 Sep 2019, Kurt Garloff wrote:

> Hi,
> 
>     
> enabling the IOMMU on my Ryzen v1605b (UDOO Bolt v8) does result in a
> non-working EMMC driver.
> Without enabling IOMMU, it works like a charm.
> From my POV this needs fixing, and I consider this a bug.

[ CCing Joerg ]

> I looked into sdhci to see whether the right dma_map_sg() calls are 
> missing, but they are there. The sdhci driver appears to do the right 
> thing. It seems that the EMMC controller is not considered and reported 
> as PCI device while it still goes through the PCI IOMMU :-O
> 
> Here is what happens on loading the sdhci-acpi driver on kernel 5.3:
> 
> [12916.740148] mmc0: ADMA error
> [12916.740154] mmc0: sdhci: ============ SDHCI REGISTER DUMP ===========
> [12916.740163] mmc0: sdhci: Sys addr:Â 0x00000000 | Version:Â 0x00001002
> [12916.740170] mmc0: sdhci: Blk size:Â 0x00007200 | Blk cnt:Â 0x00000001
> [12916.740179] mmc0: sdhci: Argument:Â 0x00000000 | Trn mode: 0x00000013
> [12916.740184] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:13.1
> domain=0x0000 address=0x6f2163200 flags=0x0050]
> [12916.740193] mmc0: sdhci: Present:ÂÂ 0xf1ff0000 | Host ctl: 0x00000019
> [12916.740202] mmc0: sdhci: Power:ÂÂÂÂ 0x0000000f | Blk gap:Â 0x00000000
> [12916.740211] mmc0: sdhci: Wake-up:ÂÂ 0x00000000 | Clock:ÂÂÂ 0x0000f447
> [12916.740219] mmc0: sdhci: Timeout:ÂÂ 0x0000000a | Int stat: 0x00000000
> [12916.740226] mmc0: sdhci: Int enab:Â 0x03ff000b | Sig enab: 0x03ff000b
> [12916.740232] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000000
> [12916.740239] mmc0: sdhci: Caps:ÂÂÂÂÂ 0x71fec8b2 | Caps_1:ÂÂ 0x00000577
> [12916.740246] mmc0: sdhci: Cmd:ÂÂÂÂÂÂ 0x0000083a | Max curr: 0x00c80064
> [12916.740253] mmc0: sdhci: Resp[0]:ÂÂ 0x00000700 | Resp[1]:Â 0xffffffff
> [12916.740259] mmc0: sdhci: Resp[2]:ÂÂ 0x328f5903 | Resp[3]:Â 0x00d00f00
> [12916.740264] mmc0: sdhci: Host ctl2: 0x00000000
> [12916.740273] mmc0: sdhci: ADMA Err:Â 0x00000001 | ADMA Ptr:
> 0x00000006f2163200
> [12916.740274] mmc0: sdhci: ============================================
> [12916.740337] mmc0: error -5 whilst initialising MMC card

Do you have BAR memory allocation failures in dmesg with IOMMU on? 
Actually, sharing both working and non-working dmesg, as well as 
/proc/iomem contents, would be helpful.

Keeping the rest of mail below untouched for reference.

> 
> As you can see, from an IOMMU perspective, this is PCI device 00:13.1.
> However, from a kernel perspective, it's not on the PCI bus and does not
> require IOMMU translation, but rather just direct DMA mapping.
> 
> 
> linux@udookurt(/):~/linux-53/drivers/mmc/host [0]$ sudo /sbin/lspci
> 00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Root
> Complex
> 00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 IOMMU
> 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models
> 00h-1fh) PCIe Dummy Host Bridge
> 00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP
> Bridge [6:0]
> 00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP
> Bridge [6:0]
> 00:01.6 PCI bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP
> Bridge [6:0]
> 00:01.7 PCI bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP
> Bridge [6:0]
> 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models
> 00h-1fh) PCIe Dummy Host Bridge
> 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Internal
> PCIe GPP Bridge 0 to Bus A
> 00:08.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Internal
> PCIe GPP Bridge 0 to Bus B
> 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev
> 61)
> 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51)
> 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device
> 24: Function 0
> 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device
> 24: Function 1
> 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device
> 24: Function 2
> 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device
> 24: Function 3
> 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device
> 24: Function 4
> 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device
> 24: Function 5
> 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device
> 24: Function 6
> 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device
> 24: Function 7
> 01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD
> Controller SM981/PM981/PM983
> 03:00.0 Network controller: Intel Corporation Dual Band Wireless-AC 3168NGW
> [Stone Peak] (rev 10)
> 04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411
> PCI Express Gigabit Ethernet Controller (rev 0c)
> 05:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI]
> Raven Ridge [Radeon Vega Series / Radeon Vega Mobile Series] (rev 83)
> 05:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI]
> Raven/Raven2/Fenghuang HDMI/DP Audio Controller
> 05:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Family 17h
> (Models 10h-1fh) Platform Security Processor
> 05:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Raven USB 3.1
> 05:00.4 USB controller: Advanced Micro Devices, Inc. [AMD] Raven USB 3.1
> 05:00.5 Multimedia controller: Advanced Micro Devices, Inc. [AMD]
> Raven/Raven2/FireFlight/Renoir Audio Processor
> 05:00.6 Audio device: Advanced Micro Devices, Inc. [AMD] Family 17h (Models
> 10h-1fh) HD Audio Controller
> 05:00.7 Non-VGA unclassified device: Advanced Micro Devices, Inc. [AMD]
> Raven/Raven2/Renoir Non-Sensor Fusion Hub KMDF driver
> 06:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA
> Controller [AHCI mode] (rev 61)
> 
> Looks like we'd need some quirks to actually create a pci_device handle for
> the embedded AMD eMMC controller?
> Thoughts?
> 
> PS: Please copy me on responses, I'm off LKML for half a decade now :-O
> 
> -- 
> Kurt Garloff<kurt@xxxxxxxxxx>
> Cologne, Germany
> 

-- 
Jiri Kosina
SUSE Labs