Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge systems

[Kai-Heng Feng]

> On Dec 20, 2019, at 10:13, Kai-Heng Feng <kai.heng.feng@xxxxxxxxxxxxx> wrote:
> 
> 
> 
>> On Dec 20, 2019, at 03:15, Deucher, Alexander <Alexander.Deucher@xxxxxxx> wrote:
>> 
>>> -----Original Message-----
>>> From: Kai-Heng Feng <kai.heng.feng@xxxxxxxxxxxxx>
>>> Sent: Wednesday, December 18, 2019 12:45 PM
>>> To: Joerg Roedel <joro@xxxxxxxxxx>
>>> Cc: Christoph Hellwig <hch@xxxxxxxxxxxxx>; Deucher, Alexander
>>> <Alexander.Deucher@xxxxxxx>; iommu@xxxxxxxxxxxxxxxxxxxxxxxxxx; Kernel
>>> development list <linux-kernel@xxxxxxxxxxxxxxx>
>>> Subject: Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge
>>> systems
>>> 
>>> 
>>> 
>>>> On Dec 17, 2019, at 17:53, Joerg Roedel <joro@xxxxxxxxxx> wrote:
>>>> 
>>>> On Fri, Dec 06, 2019 at 01:57:41PM +0800, Kai-Heng Feng wrote:
>>>>> Hi Joerg,
>>>>> 
>>>>>> On Dec 3, 2019, at 01:00, Christoph Hellwig <hch@xxxxxxxxxxxxx> wrote:
>>>>>> 
>>>>>> On Fri, Nov 29, 2019 at 10:21:54PM +0800, Kai-Heng Feng wrote:
>>>>>>> Serious screen flickering when Stoney Ridge outputs to a 4K monitor.
>>>>>>> 
>>>>>>> According to Alex Deucher, IOMMU isn't enabled on Windows, so let's
>>>>>>> do the same here to avoid screen flickering on 4K monitor.
>>>>>> 
>>>>>> Disabling the IOMMU entirely seem pretty severe.  Isn't it enough to
>>>>>> identity map the GPU device?
>>>>> 
>>>>> Ok, there's set_device_exclusion_range() to exclude the device from
>>> IOMMU.
>>>>> However I don't know how to generate range_start and range_length,
>>> which are read from ACPI.
>>>> 
>>>> set_device_exclusion_range() is not the solution here. The best is if
>>>> the GPU device is put into a passthrough domain at boot, in which it
>>>> will be identity mapped. DMA still goes through the IOMMU in this
>>>> case, but it only needs to lookup the device-table, page-table walks
>>>> will not be done anymore.
>>>> 
>>>> The best way to implement this is to put it into the
>>>> amd_iommu_add_device() in drivers/iommu/amd_iommu.c. There is this
>>>> check:
>>>> 
>>>>      if (dev_data->iommu_v2)
>>>> 		iommu_request_dm_for_dev(dev);
>>>> 
>>>> The iommu_request_dm_for_dev() function causes the device to be
>>>> identity mapped. The check can be extended to also check for a device
>>>> white-list for devices that need identity mapping.
>>> 
>>> My patch looks like this but the original behavior (4K screen flickering) is still
>>> the same:
>> 
>> Does reverting the patch to disable ATS along with this patch help?
> 
> Unfortunately it doesn't help.

Any further suggestion to let me try?

Kai-Heng

> 
> Kai-Heng
> 
>> 
>> Alex
>> 
>>> 
>>> diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
>>> index bd25674ee4db..f913a25c9e92 100644
>>> --- a/drivers/iommu/amd_iommu.c
>>> +++ b/drivers/iommu/amd_iommu.c
>>> @@ -42,6 +42,7 @@
>>> #include <asm/iommu.h>
>>> #include <asm/gart.h>
>>> #include <asm/dma.h>
>>> +#include <asm/pci-direct.h>
>>> 
>>> #include "amd_iommu_proto.h"
>>> #include "amd_iommu_types.h"
>>> @@ -2159,6 +2160,8 @@ static int amd_iommu_add_device(struct device
>>> *dev)
>>>       struct iommu_domain *domain;
>>>       struct amd_iommu *iommu;
>>>       int ret, devid;
>>> +       bool need_identity_mapping = false;
>>> +       u32 header;
>>> 
>>>       if (!check_device(dev) || get_dev_data(dev))
>>>               return 0;
>>> @@ -2184,7 +2187,11 @@ static int amd_iommu_add_device(struct device
>>> *dev)
>>> 
>>>       BUG_ON(!dev_data);
>>> 
>>> -       if (dev_data->iommu_v2)
>>> +       header = read_pci_config(0, PCI_BUS_NUM(devid), PCI_SLOT(devid),
>>> PCI_FUNC(devid));
>>> +       if ((header & 0xffff) == 0x1002 && (header >> 16) == 0x98e4)
>>> +               need_identity_mapping = true;
>>> +
>>> +       if (dev_data->iommu_v2 || need_identity_mapping)
>>>               iommu_request_dm_for_dev(dev);
>>> 
>>>       /* Domains are initialized for this device - have a look what we ended up
>>> with */
>>> 
>>> 
>>> $ dmesg | grep -i direct
>>> [    0.011446] Using GB pages for direct mapping
>>> [    0.703369] pci 0000:00:01.0: Using iommu direct mapping
>>> [    0.703830] pci 0000:00:08.0: Using iommu direct mapping
>>> 
>>> So the graphics device (pci 0000:00:01.0:) is using direct mapping after the
>>> change.
>>> 
>>> Kai-Heng
>>> 
>>>> 
>>>> HTH,
>>>> 
>>>> 	Joerg