Re: [EXTERNAL] [tip: x86/apic] x86/io_apic: Cleanup trigger/polarity helpers

[David Woodhouse]

On Tue, 2020-11-10 at 10:17 -0600, Tom Lendacky wrote:
> Yep. The warning started triggering with:
> 47bea873cf80 ("x86/msi: Only use high bits of MSI address for DMAR unit")
> 
> Here's the backtrace:
> 
> [   15.611109] ------------[ cut here ]------------
> [   15.616274] WARNING: CPU: 184 PID: 1 at arch/x86/kernel/apic/apic.c:2505 __irq_msi_compose_msg+0x79/0x80
> [   15.626855] Modules linked in:
> [   15.630263] CPU: 184 PID: 1 Comm: swapper/0 Not tainted 5.10.0-rc1-sos-custom #1
> [   15.638516] Hardware name: AMD Corporation ETHANOL_X/ETHANOL_X, BIOS REX1006G 01/25/2020
> [   15.647549] RIP: 0010:__irq_msi_compose_msg+0x79/0x80
> [   15.653188] Code: 0f f0 ff 09 d0 89 06 8b 47 04 c7 46 04 00 00 00 00 88 46 08 45 84 c0 74 08 8b 07 30 c0 89 46 04 c3 81 3f ff 00 00 00 77 01 c3 <0f> 0b c3 0f 1f 40 00 0f 1f 44 00 00 8b 05 81 f9 04 02 85 c0 75 16
> [   15.674140] RSP: 0018:ffffc900000c7c30 EFLAGS: 00010212
> [   15.679971] RAX: 0000000000000021 RBX: ffff888143789028 RCX: 0000000000000000
> [   15.687936] RDX: 0000000000000000 RSI: ffffc900000c7c40 RDI: ffff8881447dd1c0
> [   15.695899] RBP: ffff888143789028 R08: 0000000000000000 R09: 0000000000000005
> [   15.703861] R10: 0000000000000002 R11: 0000000000000004 R12: ffff88900e7b80c0
> [   15.711825] R13: 0000000000000000 R14: ffff88907646ba80 R15: 0000000000000000
> [   15.719789] FS:  0000000000000000(0000) GS:ffff88900f000000(0000) knlGS:0000000000000000
> [   15.728821] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [   15.735234] CR2: 0000000000000000 CR3: 000000114d40a000 CR4: 0000000000350ee0
> [   15.743197] Call Trace:
> [   15.745929]  irq_chip_compose_msi_msg+0x2e/0x40
> [   15.750984]  msi_domain_activate+0x4b/0x90
> [   15.755556]  __irq_domain_activate_irq+0x53/0x80
> [   15.760707]  ? irq_set_msi_desc_off+0x5a/0x90
> [   15.765568]  irq_domain_activate_irq+0x25/0x40
> [   15.770525]  __msi_domain_alloc_irqs+0x16a/0x310
> [   15.775680]  __pci_enable_msi_range+0x182/0x2b0
> [   15.780738]  ? e820__memblock_setup+0x7d/0x7d
> [   15.785597]  pci_enable_msi+0x16/0x30
> [   15.789685]  iommu_init_msi+0x30/0x190
> [   15.793860]  state_next+0x39d/0x665
> [   15.797754]  ? e820__memblock_setup+0x7d/0x7d
> [   15.802615]  iommu_go_to_state+0x24/0x28
> [   15.806993]  amd_iommu_init+0x11/0x46
> [   15.811076]  pci_iommu_init+0x16/0x3f

Oh joy.

It's asking the core code to generate a PCI MSI message for it and
actually program that to the PCI device (since the IOMMU itself is a
PCI device).

That isn't actually used for generating MSI, but is instead interpreted
to write the intcapxt registers which *do* generate the interrupts.

That wants fixing, preferably not to go via MSI format at all, or maybe
just to use the 'dmar' flag to __irq_msi_compose_msg(). Either way by
having an irqdomain of its own like the Intel IOMMU does.

If I could get post-5.5 kernels to boot at all with the AMD IOMMU
enabled, I'd have a go at throwing that together now...

Attachment: smime.p7s
Description: S/MIME cryptographic signature