Re: [PATCH 16/39] irqdomain: Make normal and nomap irqdomains exclusive

[Cédric Le Goater]

Hello Marc,

> > This patch is breaking the POWER9/POWER10 XIVE driver (these are not
> > old PPC systems :) on machines sharing the same LSI HW IRQ. For instance,
> > a linux KVM guest with a virtio-rng and a virtio-balloon device. In that
> > case, Linux creates two distinct IRQ mappings which can lead to some
> > unexpected behavior.
>
> Either the irq domain translates, or it doesn't. If the driver creates
> a nomap domain, and yet expects some sort of translation to happen,
> then the driver is fundamentally broken. And even without that: how do
> you end-up with a single HW interrupt having two mappings?
>
> > A fix to go forward would be to change the XIVE IRQ domain to use a
> > 'Tree' domain for reverse mapping and not the 'No Map' domain mapping.
> > I will keep you updated for XIVE.
>
> I bet there is a bit more to it. From what you are saying above,
> something rather ungodly is happening in the XIVE code.

It's making progress.

This change in irq_find_mapping() is what 'breaks' XIVE :

  +       if (irq_domain_is_nomap(domain)) {
  +               if (hwirq < domain->revmap_size) {
  +                      data = irq_domain_get_irq_data(domain, hwirq);
  +                      if (data && data->hwirq == hwirq)
  +                               return hwirq;
  +               }
  +
  +               return 0;


With the introduction of IRQ_DOMAIN_FLAG_NO_MAP, the revmap_tree lookup
is skipped and the previously mapped IRQ is not found. XIVE was relying
on a side effect of irq_domain_set_mapping() which is not true anymore.

I guess the easiest fix for 5.14 and 5.15 (in which was introduced MSI
domains) is to change the XIVE IRQ domain to a domain tree. Since the HW
can handle 1MB interrupts, this looks like a better choice for the driver.

Thanks,

C.