Re: [PATCH v2 1/2] msi: The MSI framework only supports 2048 platform MSIs

[Julian Vetter]

On Tue, Jun 14, 2022 at 05:23:27PM +0200, Greg KH wrote:
> On Tue, Jun 14, 2022 at 05:14:10PM +0200, Julian Vetter wrote:
> > Some devices need more MSIs. To support this the number must be
> > increased.

Hello Greg,
thank you for the feedback. See below my comments on it.

> What devices specifically?

You're right I didn't specify this. We have a mailbox device in our
system which supports up to 8192 MSIs. Our architecture is not yet upstreamed,
but the code can be found here https://github.com/kalray/linux_coolidge/blob/coolidge/drivers/irqchip/irq-kvx-apic-mailbox.c
It's a mailbox that can be written to, to trigger an interrupt. It has
128 x 64bit registers and each bit can trigger an interrupt.

> How many more did you just increase it by?
> 
> How much more memory are you now using?
> 
> Why is there a limit in the first place?  Why not just make it dynamic
> so that it never runs out in another month or so?
> 
> We need lots more specifics here...

I increased the number from 2048 -> 8192. But you're right. I just
increased it to the amount that it serves our device. But in my opinion could
it be increased to 2^16. Because the value is OR'ed with the device ID
((devid << (32 - DEV_ID_SHIFT)) | desc->msi_index;) and I assume the device ID
will not exceed 65536. They are stored in a variable of type irq_hw_number_t
which is at least 32bit. So, we could just use the lower 16bit for the
MSI index and the upper 16bit for the device ID.

In terms of memory usage I'm not entirely sure. But the value in the
MAX_DEV_MSIS is only a "global" limit that ensures that we don't overflow the
variable. So, no additional memory is consumed. Because the size of the
IRQ bitmap is handled by other variable(s) (i.e. NR_IRQS).

> thanks,
> 
> greg k-h

Thanks you,
Julian