Re: [REGRESSION] GPU passes into VM improperly after c376a3456d8b or a98db518dde2

[70sp]

Hello,

this regression thread became abandoned unfortunately.

In case there's anybody who would like to help me (or generally) solve this regression, I am sending all information about what I've found out.

I had a private conversation with Lu Baolu regarding this issue and we found out several new facts about this. I am currently working on creating and testing my own patches to add backward compatibility for hardware affecting this issue with my limited knowledge of C and IOMMU.

This issue consists of improper GPU passthrough into VM on a specific platform (Haswell) or a specific GPU (NVIDIA Maxwell - GTX 970). I personally believe (could be incorrect), that it is caused by the GPU being too old and becoming unsupported by referenced commits. The GPU doesn't seem to be using all levels of PTE properly and some data need to be trimmed for proper passthrough but the patches seem to remove (or alter) the trimming functionality.

The issue is caused by one of these commits:
a98db518dde246e01ead53617dc0a30d6aaa3752
c376a3456d8bef43ec556a98c0a04c35086c2737

The IOMMU code changed a lot since these commits were merged.

With IOMMU DebugFS enabled these messages appear in dmesg on affected kernels:
[  107.365019] DMAR: DRHD: handling fault status reg 3
[  107.365021] DMAR: [DMA Read NO_PASID] Request device [01:00.0] fault addr 0x37fff1000 [fault reason 0x0c] non-zero reserved fields in PTE
[  107.365023] DMAR: Dump dmar1 table entries for IOVA 0x37fff1000
[  107.365024] DMAR: root entry: 0x00000004034fc001
[  107.365025] DMAR: context entry: hi 0x0000000000000202, low 0x000000008bda5001
[  107.365026] DMAR: pte level: 4, pte value: 0x00000000846ce403
[  107.365027] DMAR: pte level: 3, pte value: 0x00000003c0000883

I would be grateful for any useful information or help so I can finally forget about this and freely use newer kernels. I also know that there are several other people experiencing this issue, and I would be happy if we solved this for everybody affected.

Thanks to everybody,
70sp