Re: [PATCH] PCI: brcmstb: Reserve only the MSI vectors that are handed out

[Thomas Gleixner]

On Mon, Sep 21 2026 at 18:05, Han / 한상우Sangwoo wrote:
> I tested Thomas's patch on the same hardware setup with the 5-vector
> MSI endpoint. The patch was applied as posted on top of 6.12.93
> (rpi-6.12.y).
>
> With the patch applied:
>
> - The MSI base hwirq remained at 0x8 across 200 driver reload cycles.
> - The driver got all 5 requested vectors on every cycle, with no
> single-MSI fallback. Multiple Message Enable remained at 8.
> - The brcmstb inner-domain mapping returned to baseline after each
> unload/reload, with 12 mapped while the driver was loaded and 4 after
> unload.
> - A kprobe showed one allocation of 8 vectors followed by eight
> single-vector frees, with nothing left over.

8 single vector frees?

Seems I got something wrong there verus the bulk free. Updated patch
below.

> The MSI vector exhaustion issue I originally observed no longer
> reproduces with the patch.

Good. Can you please retest with the updated patch?

> I also observed a KASAN report with managed affinity. Using a small
> out-of-tree test module bound to the same endpoint and requesting 1..3
> vectors with PCI_IRQ_MSI | PCI_IRQ_AFFINITY, a request for 3 vectors
> resulted in:
>
> BUG: KASAN: slab-out-of-bounds in __irq_alloc_descs+0x158/0x460
>
> Requests for 5 and 7 vectors were capped to 4 on this 4-CPU system and
> did not trigger the report. I have not checked this against the
> unpatched kernel yet, so I cannot tell whether it is related to the
> patch.

Any updates on that? Also please provide the source for that test.

Thanks,

        tglx
---
--- a/kernel/irq/irqdomain.c
+++ b/kernel/irq/irqdomain.c
@@ -1610,6 +1610,17 @@ static void irq_domain_free_irqs_hierarc
 	if (!domain->ops->free)
 		return;
 
+	/*
+	 * MSI device domains are capable of bulk free.
+	 *
+	 * CHECKME: Are all MSI parent domains capable?
+	 */
+	if (domain->flags & (IRQ_DOMAIN_FLAG_MSI_DEVICE | IRQ_DOMAIN_FLAG_MSI_PARENT)) {
+		if (irq_domain_get_irq_data(domain, irq_base))
+			domain->ops->free(domain, irq_base, nr_irqs);
+		return;
+	}
+
 	for (i = 0; i < nr_irqs; i++) {
 		if (irq_domain_get_irq_data(domain, irq_base + i))
 			domain->ops->free(domain, irq_base + i, 1);
--- a/kernel/irq/msi.c
+++ b/kernel/irq/msi.c
@@ -1333,20 +1333,28 @@ static int __msi_domain_alloc_irqs(struc
 
 		ops->set_desc(&arg, desc);
 
-		virq = __irq_domain_alloc_irqs(domain, -1, desc->nvec_used,
+		/* Make sure a MULTI-MSI allocation is power of two */
+		unsigned int nvec_aligned = roundup_pow_of_two(desc->nvec_used);
+
+		virq = __irq_domain_alloc_irqs(domain, -1, nvec_aligned,
 					       dev_to_node(dev), &arg, false,
 					       desc->affinity);
 		if (virq < 0)
 			return msi_handle_pci_fail(domain, desc, allocated);
 
-		for (i = 0; i < desc->nvec_used; i++) {
+		for (i = 0; i < nvec_aligned; i++) {
 			irq_set_msi_desc_off(virq, i, desc);
 			irq_debugfs_copy_devname(virq + i, dev);
 			ret = msi_init_virq(domain, virq + i, vflags);
 			if (ret)
 				return ret;
 		}
+
 		if (info->flags & MSI_FLAG_DEV_SYSFS) {
+			/*
+			 * This only exposes desc->nvec_used and ignores the
+			 * overallocated MULTI-MSI ones.
+			 */
 			ret = msi_sysfs_populate_desc(dev, desc);
 			if (ret)
 				return ret;
@@ -1610,13 +1618,15 @@ static void __msi_domain_free_irqs(struc
 			continue;
 
 		/* Make sure all interrupts are deactivated */
-		for (i = 0; i < desc->nvec_used; i++) {
+		unsigned int nvec_aligned = roundup_pow_of_two(desc->nvec_used);
+
+		for (i = 0; i < nvec_aligned; i++) {
 			irqd = irq_domain_get_irq_data(domain, desc->irq + i);
 			if (irqd && irqd_is_activated(irqd))
 				irq_domain_deactivate_irq(irqd);
 		}
 
-		irq_domain_free_irqs(desc->irq, desc->nvec_used);
+		irq_domain_free_irqs(desc->irq, nvec_aligned);
 		if (info->flags & MSI_FLAG_DEV_SYSFS)
 			msi_sysfs_remove_desc(dev, desc);
 		desc->irq = 0;