Re: Help with IRQ-MSI-IRQ bridges

From: Marc Zyngier
Date: Fri Apr 03 2020 - 06:18:51 EST


Hi Luis,

On 2020-04-03 02:35, LuÃs Matallui wrote:
Hi,

I've got this SoC which uses IRQ-MSI and MSI-IRQ bridges in order to
get interrupts from devices external to the ARM subsystem.
I already got some pointers from Maz and have been able to create the
drivers with the stacked domains and can now see the mappings working
fine across domains.

Maz pointed me to the Marvell mvebu-gicp (for my MSI controller, a.k.a
MSI-IRQ bridge) and to mvebu-icu for the MSI client (IRQ-MSI bridge).

I now have the interrupts working, but it seems like I'm missing a
bunch of them. And therefore my device doesn't work properly.
The main difference between my HW and Marvell's is that my IRQs are
not level-triggered and the MSIs don't support the two messages for
level-triggered interrupts.

Which is probably a very good thing, as long as all your devices generate
only edge-triggered interrupts.


To illustrate my system:

DEV --line--> IRQ-MSI Bridge (MSIC) --msi--> MSI-IRQ Bridge (GICP)
--line--> GICv2

For MSIC, all I can do is configure the address and data for the MSI,
and I believe on every rising edge of the Device IRQ, an MSI is sent.
For GICP, all I have is a doorbell and a way to enable/disable it, and
whenever the doorbell is enabled and has a value != 0, the IRQ line to
GICv2 gets asserted.

The first thing I noticed is that when I get an interrupt, the IRQ
flow goes like:

handle_irq();
irq_eoi();

So, I guess my first question here is, how can I guarantee that I
don't get another MSI whilst in handle_irq()?

At the GIC level, once the interrupt is Ack'd, anything that is signed
after this ack is a separate interrupt. It will be made pending and will
fire once the GIC driver EOIs the first one.

If I do, then I will clear the doorbell on irq_eoi() (because that's
my only choice) and will lose the queued IRQs.

Why do you need to do anything at the doorbell level? This is just a write,
so there should be nothing to clear. If you do need to clear anything,
then your MSI-IRQ bridge isn't stateless as it should, and you'll need to
give much more details about the HW. Do you have a pointer to the TRM
for your HW?

It also seems that I'm missing IRQs in the beginning after probing the
device, and before it was working for me when I was setting up all
these registers manually and simply using GICv2 as my only interrupt
controller.

Well, setting all of this in firmware is always the preferred option
if you don't expect things to change dynamically.

I do see the unmask() ops being called for all my stacked irqchips, so
I don't understand how I'm missing so many interrupts.

unmask is just a static configuration to enable the interrupt. There shouldn't
be that many calls to that later on unless an endpoint driver disables/enables
interrupts by hand.

Please give us a bit more details to understand the context, as there is only
so much I can do with so little HW information.

Thanks,

M.
--
Jazz is not dead. It just smells funny...