MSIs not freed in GICv3 ITS driver

[Manivannan Sadhasivam]

Hi Marc, Thomas,

I'm seeing a weird behavior with GICv3 ITS driver while allocating MSIs from
PCIe devices. When the PCIe driver (I'm using virtio_pci_common.c) tries to
allocate non power of 2 MSIs (like 3), then the GICv3 MSI driver always rounds
the MSI count to power of 2 to find the order. In this case, the order becomes 2
in its_alloc_device_irq(). So 4 entries are allocated by
bitmap_find_free_region().

But since the PCIe driver has only requested 3 MSIs, its_irq_domain_alloc()
will only allocate 3 MSIs, leaving one bitmap entry unused.

And when the driver frees the MSIs using pci_free_irq_vectors(), only 3
allocated MSIs were freed and their bitmap entries were also released. But the
entry for the additional bitmap was never released. Due to this,
its_free_device() was also never called, resulting in the ITS device not getting
freed.

So when the PCIe driver tries to request the MSIs again (PCIe device being
removed and inserted back), because the ITS device was not freed previously,
MSIs were again requested for the same ITS device. And due to the stale bitmap
entry, the ITS driver refuses to allocate 4 MSIs as only 3 bitmap entries were
available. This forces the PCIe driver to reduce the MSI count, which is sub
optimal.

This behavior might be applicable to other irqchip drivers handling MSI as well.
I want to know if this behavior is already known with MSI and irqchip drivers?

For fixing this issue, the PCIe drivers could always request MSIs of power of 2,
and use a dummy MSI handler for the extra number of MSIs allocated. This could
also be done in the generic MSI driver itself to avoid changes in the PCIe
drivers. But I wouldn't say it is the best possible fix.

Is there any other way to address this issue? Or am I missing something
completely?

- Mani

-- 
மணிவண்ணன் சதாசிவம்