Re: PCI MSI issue for maxcpus=1

[John Garry]

Hi Marc,

> diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c
> index 2bdfce5edafd..97e9eb9aecc6 100644
> --- a/kernel/irq/msi.c
> +++ b/kernel/irq/msi.c
> @@ -823,6 +823,19 @@ static int msi_init_virq(struct irq_domain *domain, int virq, unsigned int vflag
>   	if (!(vflags & VIRQ_ACTIVATE))
>   		return 0;
>   
> +	if (!(vflags & VIRQ_CAN_RESERVE)) {
> +		/*
> +		 * If the interrupt is managed but no CPU is available
> +		 * to service it, shut it down until better times.
> +		 */
> +		if (irqd_affinity_is_managed(irqd) &&
> +		    !cpumask_intersects(irq_data_get_affinity_mask(irqd),
> +					cpu_online_mask)) {
> +			irqd_set_managed_shutdown(irqd);
> +			return 0;
> +		}
> +	}
> +
>   	ret = irq_domain_activate_irq(irqd, vflags & VIRQ_CAN_RESERVE);
>   	if (ret)
>   		return ret;

Yeah, that seems to solve the issue. I will test it a bit more.

We need to check the isolcpus cmdline issue as well - wang xiongfeng, 
please assist here. I assume that this feature just never worked for 
arm64 since it was added.

> With this in place, I get the following results (VM booted with 4
> vcpus and maxcpus=1, the virtio device is using managed interrupts):
>
> root@debian:~# cat /proc/interrupts
>             CPU0
>   10:       2298     GICv3  27 Level     arch_timer
>   12:         84     GICv3  33 Level     uart-pl011
>   49:          0     GICv3  41 Edge      ACPI:Ged
>   50:          0   ITS-MSI 16384 Edge      virtio0-config
>   51:       2088   ITS-MSI 16385 Edge      virtio0-req.0
>   52:          0   ITS-MSI 16386 Edge      virtio0-req.1
>   53:          0   ITS-MSI 16387 Edge      virtio0-req.2
>   54:          0   ITS-MSI 16388 Edge      virtio0-req.3
>   55:      11641   ITS-MSI 32768 Edge      xhci_hcd
>   56:          0   ITS-MSI 32769 Edge      xhci_hcd
> IPI0:         0       Rescheduling interrupts
> IPI1:         0       Function call interrupts
> IPI2:         0       CPU stop interrupts
> IPI3:         0       CPU stop (for crash dump) interrupts
> IPI4:         0       Timer broadcast interrupts
> IPI5:         0       IRQ work interrupts
> IPI6:         0       CPU wake-up interrupts
> Err:          0
> root@debian:~# echo 1 >/sys/devices/system/cpu/cpu2/online
> root@debian:~# cat /proc/interrupts
>             CPU0       CPU2
>   10:       2530         90     GICv3  27 Level     arch_timer
>   12:        103          0     GICv3  33 Level     uart-pl011
>   49:          0          0     GICv3  41 Edge      ACPI:Ged
>   50:          0          0   ITS-MSI 16384 Edge      virtio0-config
>   51:       2097          0   ITS-MSI 16385 Edge      virtio0-req.0
>   52:          0          0   ITS-MSI 16386 Edge      virtio0-req.1
>   53:          0         12   ITS-MSI 16387 Edge      virtio0-req.2
>   54:          0          0   ITS-MSI 16388 Edge      virtio0-req.3
>   55:      13487          0   ITS-MSI 32768 Edge      xhci_hcd
>   56:          0          0   ITS-MSI 32769 Edge      xhci_hcd
> IPI0:        38         45       Rescheduling interrupts
> IPI1:         3          3       Function call interrupts
> IPI2:         0          0       CPU stop interrupts
> IPI3:         0          0       CPU stop (for crash dump) interrupts
> IPI4:         0          0       Timer broadcast interrupts
> IPI5:         0          0       IRQ work interrupts
> IPI6:         0          0       CPU wake-up interrupts
> Err:          0

Out of interest, is the virtio managed interrupts support just in your 
sandbox? You did mention earlier in the thread that you were considering 
adding this feature.

Thanks,
John