PCI/kernel msi code vs GIC ITS driver conflict?

From: John Garry
Date: Tue Sep 03 2019 - 10:09:54 EST

Hi Marc, Bjorn, Thomas,

We've come across a conflict with the kernel/pci msi code and GIC ITS driver on our arm64 system, whereby we can't unbind and re-bind a PCI device driver under special conditions. I'll explain...

Our PCI device support 32 MSIs. The driver attempts to allocate msi vectors with min msi=17, max msi = 32, and affd.pre vectors = 16. For our test we make nr_cpus = 1 (just anything less than 16).

We find that the pci/kernel msi code gives us 17 vectors, but the GIC ITS code reserves 32 lpi maps in its_irq_domain_alloc(). The problem then occurs when unbinding the driver in its_irq_domain_free() call, where we only clear bits for 17 vectors. So if we unbind the driver and then attempt to bind again, it fails.

Where the fault lies, I can't say. Maybe the kernel msi code should always give power of 2 vectors - as I understand, the PCI spec mandates this. Or maybe the GIC ITS driver has a problem in the free path, as above. Or maybe the PCI driver should not be allowed to request !power of 2 min/max vectors.


Thanks in advance,