Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge systems

[Kai-Heng Feng]

> On Dec 5, 2019, at 00:08, Deucher, Alexander <alexander.deucher@xxxxxxx> wrote:
> 
>> -----Original Message-----
>> From: Deucher, Alexander
>> Sent: Monday, December 2, 2019 11:37 AM
>> To: Lucas Stach <dev@xxxxxxxxxx>; Kai-Heng Feng
>> <kai.heng.feng@xxxxxxxxxxxxx>; joro@xxxxxxxxxx; Koenig, Christian
>> (Christian.Koenig@xxxxxxx) <Christian.Koenig@xxxxxxx>
>> Cc: iommu@xxxxxxxxxxxxxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx
>> Subject: RE: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge
>> systems
>> 
>>> -----Original Message-----
>>> From: Lucas Stach <dev@xxxxxxxxxx>
>>> Sent: Sunday, December 1, 2019 7:43 AM
>>> To: Kai-Heng Feng <kai.heng.feng@xxxxxxxxxxxxx>; joro@xxxxxxxxxx
>>> Cc: Deucher, Alexander <Alexander.Deucher@xxxxxxx>;
>>> iommu@xxxxxxxxxxxxxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx
>>> Subject: Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge
>>> systems
>>> 
>>> Am Freitag, den 29.11.2019, 22:21 +0800 schrieb Kai-Heng Feng:
>>>> Serious screen flickering when Stoney Ridge outputs to a 4K monitor.
>>>> 
>>>> According to Alex Deucher, IOMMU isn't enabled on Windows, so let's
>>>> do the same here to avoid screen flickering on 4K monitor.
>>> 
>>> This doesn't seem like a good solution, especially if there isn't a
>>> method for the user to opt-out.  Some users might prefer having the
>>> IOMMU support to 4K display output.
>>> 
>>> But before using the big hammer of disabling or breaking one of those
>>> features, we should take a look at what's the issue here. Screen
>>> flickering caused by the IOMMU being active hints to the IOMMU not
>>> being able to sustain the translation bandwidth required by the high-
>>> bandwidth isochronous transfers caused by 4K scanout, most likely due
>>> to insufficient TLB space.
>>> 
>>> As far as I know the framebuffer memory for the display buffers is
>>> located in stolen RAM, and thus contigous in memory. I don't know the
>>> details of the GPU integration on those APUs, but maybe there even is
>>> a way to bypass the IOMMU for the stolen VRAM regions?
>>> 
>>> If there isn't and all GPU traffic passes through the IOMMU when
>>> active, we should check if the stolen RAM is mapped with hugepages on
>>> the IOMMU side. All the stolen RAM can most likely be mapped with a
>>> few hugepage mappings, which should reduce IOMMU TLB demand by a
>> large margin.
>> 
>> The is no issue when we scan out of the carve out region.  The issue occurs
>> when we scan out of regular system memory (scatter/gather).  Many newer
>> laptops have very small carve out regions (e.g., 32 MB), so we have to use
>> regular system pages to support multiple high resolution displays.  The
>> problem is, the latency gets too high at some point when the IOMMU is
>> involved.  Huge pages would probably help in this case, but I'm not sure if
>> there is any way to guarantee that we get huge pages for system memory.  I
>> guess we could use CMA or something like that.
> 
> Thomas recently sent out a patch set to add huge page support to ttm:
> https://patchwork.freedesktop.org/series/70090/
> We'd still need a way to guarantee huge pages for the display buffer.

Is there an amdgpu counterpart to let me test out?

Kai-Heng

> 
> Alex
> 
>> 
>> Alex
>> 
>>> 
>>> Regards,
>>> Lucas
>>> 
>>>> Cc: Alex Deucher <alexander.deucher@xxxxxxx>
>>>> Bug:
>>>> 
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgi
>>>> tl
>>>> 
>>> 
>> ab.freedesktop.org%2Fdrm%2Famd%2Fissues%2F961&amp;data=02%7C01%
>>> 7Calexa
>>>> 
>>> 
>> nder.deucher%40amd.com%7C30540b2bf2be417c4d9508d7765bf07f%7C3dd
>>> 8961fe4
>>>> 
>>> 
>> 884e608e11a82d994e183d%7C0%7C0%7C637108010075463266&amp;sdata=1
>>> ZIZUWos
>>>> cPiB4auOY10jlGzoFeWszYMDBQG0CtrrOO8%3D&amp;reserved=0
>>>> Signed-off-by: Kai-Heng Feng <kai.heng.feng@xxxxxxxxxxxxx>
>>>> ---
>>>> v2:
>>>> - Find Stoney graphics instead of host bridge.
>>>> 
>>>> drivers/iommu/amd_iommu_init.c | 13 ++++++++++++-
>>>> 1 file changed, 12 insertions(+), 1 deletion(-)
>>>> 
>>>> diff --git a/drivers/iommu/amd_iommu_init.c
>>>> b/drivers/iommu/amd_iommu_init.c index 568c52317757..139aa6fdadda
>>>> 100644
>>>> --- a/drivers/iommu/amd_iommu_init.c
>>>> +++ b/drivers/iommu/amd_iommu_init.c
>>>> @@ -2516,6 +2516,7 @@ static int __init early_amd_iommu_init(void)
>>>> 	struct acpi_table_header *ivrs_base;
>>>> 	acpi_status status;
>>>> 	int i, remap_cache_sz, ret = 0;
>>>> +	u32 pci_id;
>>>> 
>>>> 	if (!amd_iommu_detected)
>>>> 		return -ENODEV;
>>>> @@ -2603,6 +2604,16 @@ static int __init early_amd_iommu_init(void)
>>>> 	if (ret)
>>>> 		goto out;
>>>> 
>>>> +	/* Disable IOMMU if there's Stoney Ridge graphics */
>>>> +	for (i = 0; i < 32; i++) {
>>>> +		pci_id = read_pci_config(0, i, 0, 0);
>>>> +		if ((pci_id & 0xffff) == 0x1002 && (pci_id >> 16) == 0x98e4) {
>>>> +			pr_info("Disable IOMMU on Stoney Ridge\n");
>>>> +			amd_iommu_disabled = true;
>>>> +			break;
>>>> +		}
>>>> +	}
>>>> +
>>>> 	/* Disable any previously enabled IOMMUs */
>>>> 	if (!is_kdump_kernel() || amd_iommu_disabled)
>>>> 		disable_iommus();
>>>> @@ -2711,7 +2722,7 @@ static int __init state_next(void)
>>>> 		ret = early_amd_iommu_init();
>>>> 		init_state = ret ? IOMMU_INIT_ERROR :
>>> IOMMU_ACPI_FINISHED;
>>>> 		if (init_state == IOMMU_ACPI_FINISHED &&
>>> amd_iommu_disabled) {
>>>> -			pr_info("AMD IOMMU disabled on kernel command-
>>> line\n");
>>>> +			pr_info("AMD IOMMU disabled\n");
>>>> 			init_state = IOMMU_CMDLINE_DISABLED;
>>>> 			ret = -EINVAL;
>>>> 		}