Re: "oneshot" interrupt causes another interrupt to be fired erroneously in Haswell system

[Thomas Gleixner]

On Thu, 31 Oct 2019, Bjorn Helgaas wrote:
> On Thu, Oct 31, 2019 at 03:53:50AM +0000, Kar Hin Ong wrote:
> > I've an Intel Haswell system running Linux kernel v4.14 with
> > preempt_rt patch. The system contain 2 IOAPICs: IOAPIC 1 is on the
> > PCH where IOAPIC 2 is on the CPU.
> > 
> > I observed that whenever a PCI device is firing interrupt (INTx) to
> > Pin 20 of IOAPIC 2 (GSI 44); the kernel will receives 2 interrupts: 
> >    1. Interrupt from Pin 20 of IOAPIC 2  -> Expected
> >    2. Interrupt from Pin 19 of IOAPIC 1  -> UNEXPECTED, erroneously
> >       triggered
> >
> > The unexpected interrupt is unhandled eventually. When this scenario
> > happen more than 99,000 times, kernel disables the interrupt line
> > (Pin 19 of IOAPIC 1) and causing device that has requested it become
> > malfunction.
> > 
> > I managed to also reproduced this issue on RHEL 8 and Ubuntu 19-04
> > (without preempt_rt patch) after added "threadirqs" to the kernel
> > command line.
> > 
> > After digging further, I noticed that the said issue is happened
> > whenever an interrupt pin on IOAPIC 2 is masked:
> >  - Masking Pin 20 of IOAPIC 2 triggers Pin 19 of IOAPIC 1  
> >  - Masking Pin 22 of IOAPIC 2 triggers Pin 18 of IOAPIC 1

This is pretty much the same problem which we had analyzed and worked
around years ago.

> > From Intel Xeon Processor E5/E7 v3 Product Family External Design
> > Specification (EDS), Volume One: Architecture, section 13.1 (Legacy
> > PCI Interrupt Handling), it mention: "If the I/OxAPIC entry is
> > masked (via the 'mask' bit in the corresponding Redirection Table
> > Entry), then the corresponding PCI Express interrupt(s) is forwarded
> > to the legacy PCH"

Oh well. Really useful behaviour - NOT!

> > I would like to understand if my interpretation is make sense. If
> > yes, should the "oneshot" algorithm need to be updated to support
> > Haswell system?

No. You cannot change the oneshot algorithm.

The workarounds for this are enabled by PCI quirls and either
CONFIG_X86_REROUTE_FOR_BROKEN_BOOT_IRQS=y or 'ioapicreroute' on the command
line.

It might be wortha try to add the PCI ID of that box to the quirk list,
i.e. the PCI ID matches in drivers/pci/quirks.c which belong to the
function: quirk_reroute_to_boot_interrupts_intel().

Thanks,

	tglx